Commit graph

52 commits

Author SHA1 Message Date
Peter Ammon
5b489ca30f Remove caret redirection
This removes the caret as a shorthand for redirecting stderr.

Note that stderr may be redirected to a file via 2>/some/path...
and may be redirected with a pipe via 2>|.

Fixes #4394
2018-04-01 13:48:21 -07:00
Mahmoud Al-Qudsi
1441cca9c5 Restore localization to tokenizer error strings
Work around #4810 by retrieving localizations at runtime to avoid issues
possibly caused by inserting into the static unordered_map during static
initialization.

Closes #810.
2018-03-13 13:45:15 -05:00
Mahmoud Al-Qudsi
054bc88b82 Add gettext operator back to strings in tokenizer
This was purposely disabled as it causes a floating point exception immediately on run:

    Program received signal SIGFPE, Arithmetic exception.
    0x000000000056eae8 in std::__detail::_Mod_range_hashing::operator()(unsigned long, unsigned long) const ()
    (gdb) bt
    #0  0x000000000056eae8 in std::__detail::_Mod_range_hashing::operator()(unsigned long, unsigned long) const ()
    #1  0x00000000005bb4f1 in std::__detail::_Hash_code_base<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::pair<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::__detail::_Select1st, std::hash<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, true>::_M_bucket_index(std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&, unsigned long, unsigned long) const ()
    #2  0x00000000005bb22a in std::_Hashtable<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::pair<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::allocator<std::pair<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::hash<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_bucket_index(std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&, unsigned long) const ()
    #3  0x00000000005bb115 in std::__detail::_Map_base<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::pair<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::allocator<std::pair<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > >, std::__detail::_Select1st, std::equal_to<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::hash<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true>, true>::operator[](std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&) ()
    #4  0x00000000005af388 in std::unordered_map<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >, std::hash<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::equal_to<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > >, std::allocator<std::pair<std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const, std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > > > >::operator[](std::__cxx11::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> > const&) ()
    #5  0x000000000067dc6b in wgettext[abi:cxx11](wchar_t const*) ()
    #6  0x0000000000551df7 in __cxx_global_var_init.1 ()
    #7  0x000000000055225e in _GLOBAL__sub_I_tokenizer.cpp ()
    #8  0x000000000069d45d in __libc_csu_init ()
    #9  0x00007ffffdc907bf in __libc_start_main (main=0x552500 <main>, argc=1, argv=0x7ffffffed298, init=0x69d410 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffffffed288) at ../csu/libc-start.c:247
    #10 0x0000000000552389 in _start ()
    (gdb)
2018-03-12 12:44:11 -05:00
Mahmoud Al-Qudsi
0f6c763d9e Revert "Restore sanity to line continuations"
This reverts commit 2cdacbdce4.
Our fish scripts need to be audited for compliance before this can be
safely merged. cc @faho
2018-03-12 08:35:09 -05:00
Mahmoud Al-Qudsi
2cdacbdce4 Restore sanity to line continuations
Line continuations (i.e. escaped new lines) now make sense again. With
the smart pipe support (pipes continue on to next line) recently added,
this hack to have continuations ignore comments makes no sense.

This is valid code:

```fish
echo hello |
# comment here
tr -d 'l'
```

this isn't:
```fish
echo hello | \
# comment here
tr -d 'l'
```

Reverts @snnw's 318daaffb2
Closes #2928. Closes #2929.
2018-03-12 08:12:23 -05:00
Mahmoud Al-Qudsi
0620cdf711 Fix tokenizer errors for nested, alternating {} and () 2018-03-11 20:18:03 -05:00
Mahmoud Al-Qudsi
00f95a978e Make { and } valid, first-class tokenizer elements 2018-03-11 19:36:10 -05:00
Mahmoud Al-Qudsi
7447432471 Fix tests for new ) token error 2018-03-11 17:16:53 -05:00
Mahmoud Al-Qudsi
df89d71237 Correct escaping behavior in new tokenizer code 2018-03-11 17:10:16 -05:00
Mahmoud Al-Qudsi
f508a1f274 Reset tokenizer state on start and improve slice error detection 2018-03-11 12:13:55 -05:00
Mahmoud Al-Qudsi
fb1c7a3d68 Simplify token parser 2018-03-10 18:42:56 -06:00
ridiculousfish
8ded041352 Add && and || support to tokenizer 2018-03-05 12:20:56 -08:00
Fabian Homborg
c793570f2c Fix punctuation movement with one char tokens
Previously, in

    ls ^a bcd

(with "^" as the cursor), kill-word would delete the "a" and then go
on, remove the space and the "bcd".

With this, it will only kill the "a".

Fixes #4747.
2018-02-27 22:56:15 +01:00
ridiculousfish
c4d903ff98 Rationalize how the parser reports tokenizer errors
Remove the unnecessary SQUASH_ERROR flag and correctly report errors
generated from the tokenizer.
2018-02-23 17:28:12 -08:00
ridiculousfish
99fb7bb6aa Refactor how redirections are represented by the tokenizer
Prior to this fix, each redirection type was a separate token_type.
Unify these under a single type TOK_REDIRECT and break the redirection
type out into a new sub-type redirection_type_t.
2018-02-23 15:19:58 -08:00
ridiculousfish
6673fe5457 Clean up tokenizer implementation
Rather than storing a bunch of "next_foo" fields, simply populate the
tok_t directly.
2018-02-23 14:31:13 -08:00
ridiculousfish
0f62161b2b Clean up tokenizer error handling. 2018-02-19 16:31:39 -08:00
ridiculousfish
f30bf40300 Clean up comment parsing in tokenizer
Unify the show_comments and non-show_comments path.
2018-02-19 16:20:47 -08:00
ridiculousfish
b13ee818d2 Some early cleanup of tokenizer
Prior to this the tokenizer ran "one ahead", where tokenizer_t::next()
would in fact return the last-parsed token. Switch to parsing on demand
instead of running one ahead; this is simpler and prepares for tokenizer
changes.
2018-02-19 15:10:59 -08:00
ridiculousfish
fd2a0dffa9 Reflect newlines after pipes in fish grammar
The previous attempt to support newlines after pipes changed the lexer to
swallow newlines after encountering a pipe. This has two problems that are
difficult to fix:

1. comments cannot be placed after the pipe
2. fish_indent won't know about the newlines, so it will erase them

Address these problems by removing the lexer behavior, and replacing it
with a new parser symbol "optional_newlines" allowing the newlines to be
reflected directly in the fish grammar.
2018-02-18 14:44:08 -08:00
slama
38418d6356 Enable multi-line edit when the line ends with a pipe (#1285) 2018-02-18 13:01:38 -08:00
Kurtis Rader
509ee64fc9 implement our own assert() function
I recently upgraded the software on my macOS server and was dismayed to
see that cppcheck reported a huge number of format string errors due to
mismatches between the format string and its arguments from calls to
`assert()`. It turns out they are due to the macOS header using `%lu`
for the line number which is obviously wrong since it is using the C
preprocessor `__LINE__` symbol which evaluates to a signed int.

I also noticed that the macOS implementation writes to stdout, rather
than stderr. It also uses `printf()` which can be a problem on some
platforms if the stream is already in wide mode which is the normal case
for fish.

So implement our own `assert()` implementation. This also eliminates
double-negative warnings that we get from some of our calls to
`assert()` on some platforms by oclint.

Also reimplement the `DIE()` macro in terms of our internal
implementation.

Rewrite `assert(0 && msg)` statements to `DIE(msg)` for clarity and to
eliminate oclint warnings about constant expressions.

Fixes #3276, albeit not in the fashion I originally envisioned.
2017-02-14 18:48:27 -08:00
Kurtis Rader
4ffb0adb78 lint cleanups 2017-02-11 21:30:38 -08:00
ridiculousfish
1efb81456b Use std::move instead of swap in a few places where it improves clarity 2017-01-26 16:14:50 -08:00
Kurtis Rader
2e9a349dd0 normalize use of stdio functions taking a stream
We should never use stdio functions that use stdout implicitly. Saving a
few characters isn't worth the inconsistency. Too, using the forms such
as `fwprintf()` which take an explicit stream makes it easier to find
the places we write to stdout versus stderr.

Fixes #3728
2017-01-13 20:48:55 -08:00
Fabian Homborg
177e06808c Replace all printf invocations with wprintf
Remember, GNU doesn't allow mixing narrow and wide IO.
2017-01-07 14:05:42 +01:00
Kurtis Rader
521546a986 fix some lint warnings
This fixes some of the IWYU and cppcheck lint warnings. And only on
macOS (formerly OS X). Fixing these types of warnings on a broader set
of platforms should be done but this is a baby step to making `make
lint-all` have few, if any, warnings. This reduces the number of lines
in the `make lint-all` output on macOS by over 500 lines.
2016-11-15 21:15:22 -08:00
Kurtis Rader
1155c4b413 silence false positive errors from some compilers
Fixes #3529
2016-11-06 17:48:26 -08:00
Kurtis Rader
f05fe4e292 lint: problems with default in switch statements 2016-11-03 16:19:41 -07:00
Kurtis Rader
4f4d34e664 lint: missing break in switch statement 2016-11-02 14:07:12 -07:00
Kurtis Rader
f6047f02d6 lint: constant conditional operator 2016-11-01 21:19:34 -07:00
Kurtis Rader
6e6b294a3f lint: deal with "double negative" warnings 2016-10-20 21:14:40 -07:00
Aaron Gyes
ee26eafc25 Ensure we continue to cover enums in switches
Where we already manage to cover an enum entirely in a switch
statement such that default: cannot be reached, help ensure
it stays that way by condemning that route.

Also adjust a 'const' I came across that is ignored.
2016-07-30 13:10:21 -07:00
Aaron Gyes
1499ade89c Restyle touched .cpp files 2016-06-12 11:34:35 -07:00
ridiculousfish
3a7719015d Don't allow specifying an fd with a caret redirection
For example, an argument 12345^ is a real argument, not a redirection
There's no reason to use ^ here instead of >, and it's annoying to git
users.

Fixes #1873
2016-06-12 02:16:46 -07:00
David Adam
9225b16d12 add (or restore) config.h to all files
The autoconf-generated config.h contains a number of directives which
may alter the behaviour of system headers on certain platforms. Always
include it in every C++ file as the first include.

Closes #2993.
2016-05-18 22:30:21 +00:00
Kurtis Rader
79f342b954 lint cleanup: eliminate "redundant" errors
This removes some pointless parentheses but the primary focus is removing
redundancies like unnecessary "else" clauses.
2016-05-04 15:32:04 -07:00
Kurtis Rader
fc44cffac5 restyle switch blocks to match project style
I missed restyling a few "switch" blocks to make them consistent with the rest
of the code base. This fixes that oversight. This should be the final step in
restyling the C++ code to have a consistent style. This also includes a few
trivial cleanups elsewhere.

I also missed restyling the "complete" module when working my way from a to z
so this final change includes restyling that module.

Total lint errors decreased 36%. Cppcheck errors went from 47 to 24. Oclint P2
errors went from 819 to 778. Oclint P3 errors went from 3252 to 1842.

Resolves #2902.
2016-05-03 17:14:56 -07:00
Kurtis Rader
c14bac4284 restyle tokenizer module to match project style
Reduces lint errors from 70 to 46 (-34%). Line count from 1158 to 936 (-19%).

Another step in resolving issue #2902.
2016-05-03 15:00:44 -07:00
Kurtis Rader
1f06e5f0b9 add better support for IWYU and fix things
Remove the "make iwyu" build target. Move the functionality into the
recently introduced lint.fish script. Fix a lot, but not all, of the
include-what-you-use errors. Specifically, it fixes all of the IWYU errors
on my OS X server but only removes some of them on my Ubuntu 14.04 server.

Fixes #2957
2016-04-26 15:02:22 -07:00
Kurtis Rader
f2246dfb34 reduce number of Unicode private-use characters
This narrows the range of Unicode codepoints fish reserves for its own
use from U+E000 thru U+F8FE (6399 codepoints) to U+F600 thru U+F73F (320
codepoints). This is still not ideal since fish shouldn't be using any
Unicode private-use codepoints but it's a step in the right direction.

This partially addresses issue #2684.
2016-02-28 18:36:34 -08:00
ridiculousfish
c1c35e0f21 Fix a subtle problem that caused lots of unnecessary memory allocation 2016-02-27 16:51:28 -08:00
ridiculousfish
e8605cb3ef Fix a dereference-past-the-end bug in read_redirection_or_fd_pipe
Fixes #2464. Credit to zanchey for reporting it and ASAN for finding it!
2015-10-07 11:38:13 -07:00
ridiculousfish
aa76f64a94 Correct the caret position for unbalanced square brackets 2015-08-10 19:30:21 -07:00
ridiculousfish
e34a8da5d7 Correct the positioning of the error caret
When an error occurs midway through a token, like abc(def,
make the caret point at the location of the error (i.e. the paren)
instead of at the beginning of the token.
2015-08-10 18:31:20 -07:00
David Adam
a6a16808e3 Merge branch 'iwyu'
http://include-what-you-use.org/
2015-07-29 09:30:19 +08:00
ridiculousfish
c9349f2ef6 Continue to refine tokenizer interface
Migrate some functions into tokenizer instance methods
2015-07-26 00:58:32 -07:00
ridiculousfish
4ebaa7b6bd Continue migration to the new tokenizer interface 2015-07-26 00:12:36 -07:00
ridiculousfish
618896c043 Early reworking of tokenizer interface 2015-07-25 23:05:47 -07:00
ridiculousfish
0dbd83ffaf Remove some dead code 2015-07-25 22:47:44 -07:00