We were reporting "no match" when sorting something like "0 ". This is
because we don't distinguish between 0 and invalid lines when sorting.
For debug output we have to get this information back.
Closes#2254. We should only inherit global settings for keys when there
are absolutely no options attached to the key.
The default key (matching the whole line) is implicitly added only if no
keys are supplied.
Improved some error messages by including more context.
* sort: disable support for thousand separators
In order to be compatible with GNU, we have to disable thousands
separators. GNU does not enable them for the C locale, either.
Once we add support for locales we can add this feature back.
* sort: delete unused fixtures
* sort: compare -0 and 0 equal
I must have misunderstood this when implementing, but GNU considers
-0, 0, and invalid numbers to be equal.
* sort: strip blanks before applying the char index
* sort: don't crash when key start is after key end
* sort: add "no match" for months at the first non-whitespace char
We should put the "^ no match for key" indicator at the first
non-whitespace character of a field.
* sort: improve support for e notation
* sort: use maches! macros
Fix a bug in which `head` failed to print headings for `stdin` inputs
when reading from multiple files, and fix another bug in which `head`
failed to print a blank line between the contents of a file and the
heading for the next file when reading multiple files. The output now
matches that of GNU `head`.
When merging files we need to prioritize files that occur earlier in the
command line arguments with -m.
This also makes the extsort merge step (and thus extsort itself) stable again.
Moved argument parsing to clap and added tests to cover using "-" as
stdin, passing in too many file arguments, and updated the "wrap" error
message in the tests.
This adds a --debug flag, which, when activated, will draw lines below
the characters that are actually used for comparisons.
This is not a complete implementation of --debug. It should, quoting the man page
for GNU sort: "annotate the part of the line used to sort, and warn
about questionable usage to stderr". Warning about "questionable usage"
is not part of this patch.
This change required some adjustments to be able to get the range that
is actually used for comparisons. Most notably, general numeric comparisons
were rewritten, fixing some bugs along the lines.
Testing is mostly done by adding fixtures for the expected debug output of
existing tests.
* sort: implement numeric string comparison
This implements -n and -h using a string comparison algorithm instead
of parsing each number to a f64 and comparing those.
This should result in a moderate performance increase and eliminate loss
of precision.
* cache parsed f64 numbers
For general numeric comparisons we have to parse numbers as f64,
as this behavior is explicitly documented by GNU coreutils.
We can however cache the parsed value to speed up comparisons.
* fix leading zeroes for negative numbers
* use more appropriate name for exponent
* improvements to the parse function
* move checks into main loop and fix thousands separator condition
* remove unneeded checks
* rustfmt
* Various fixes and performance improvements
* fix a typo
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
* Fix month parse for months with leading whitespace
* Implement test for months whitespace fix
* Confirm human numeric works as expected with whitespace with a test
* Correct arg help value name for --parallel
* Fix SemVer non version lines/empty line sorting with a test
Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
* cat: Unrevert splice patch
* cat: Add fifo test
* cat: Add tests for error cases
* cat: Add tests for character devices
* wc: Make sure we handle short splice writes
* cat: Fix tests for 1.40.0 compiler
* cat: Run rustfmt on test_cat.rs
* Run 'cargo +1.40.0 update'
* Various fixes and performance improvements
* fix a typo
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
Treat tab chars as advancing to the next tab stop rather than having a fixed
8-column width.
Also treat tab as a whitespace split target only when splitting on word
boundaries.
* feat: move unexpand to clap
* chore: allow muliple files
* test: add test fixture, test reading from a file
* test: fix typo on file name, add test for multiple inputs
* chore: use 'success()' instead of asserting
* chore: delete unused variables
* chore: use help instead of long_help, break long line
Current implementation of the skip fields logic does not handle
multibyte code points correctly. It assumes each code point (`char`) is
one byte. If the skipped part of the input line has any multibyte code
points then this can cause fields not being skipped correctly (field
start index is calculated to be before it actually starts).
When converting to SI or IEC, produce values that align with the conventions
used by GNU numfmt.
- values > 10 are represented without a decimal place, so 10000 becomes 10K
instead of 10.0K
- when truncating, take the ceiling of the value, so 100001 becomes 101K
- values < 10 are truncated to the highest tenth, so 1001 becomes 1.1K
closes#1726
The flag makes 'sort' command ignore non-dictionary symbols
(non-alphanumeric and non-spaces). The only difference with GNU sort is
that it takes ALL alphanumeric symbols, not only ASCII ones.
.# Discussion
This commit adds support for a '-f'/'--file' option which reads "KEY=VALUE" lines from
a config (or ini) style text file and sets the corresponding environment key. This is
modeled after the same option in the `dotenv` and `godotenv` commands. Notably, this
commit does *not* add automatic loading of ".env" configuration files.
The environment variables set by reading the configuration file are set prior to any
unset (eg, `-u BAR`) or set (eg, `FOO=bar`) actions. Files are loaded in order with
later files overwriting any overlapping environment variables, then, unset actions (in
command line order) are executed, then, finally, set actions (in command line order)
are executed.
[1] [`dotenv`](https://github.com/bkeepers/dotenv)
[2] [`godotenv`](https://github.com/joho/godotenv)
There was an issue with autoformat when the files had a different
number of columns in the first line. This commit fixes the issue and
extends the related test to cover this case.
Make at_line_start persist between printing each file. This fixes an
issue when numbering lines in the output and one of the input files
does not have a trailing newline.
FileMerger receives Lines Iterables of the pre-sorted input files
via push_file() It implements Iterator, which yields lines from the
input files in (merged) sorted order. If the input files are not sorted,
then the behavior is undefined.
Internally, FileMerger uses a
std::collections::BinaryHeap<MergeableFile>.
MergeableFile is an internal helper that implements Ord in a way that
BinaryHeap can use (note that we want smallest-first, but BinaryHeap
returns largest first, so MergeableFile::cmp() calls reverse() on
whatever compare_by() returns.
Made a new function sort_by(lines, compare_fns), which accepts a
list of compare_fns and calls lines.sort_by() with a closure that
calls each compare_fn in turn until one returns something other
than equal.
Default behavior ensures that String::cmp is the last element in the
compare_fns list (referred to as 'last resort' sorting by man sort).
Passing --stable (-s) turns this behaviour off.
Test cases provided for `sort --month` and `sort --month --stable`.
Instead of using numerals to denote individual cases, I used descriptive
case names. I also changed the extension for the expected output fixture
to match other tests.
I removed one redundant test and another unnecessary helper function.
The repetition of "foo" and "bar" made for difficult-to-read assertion failures
when hacking on `tail`. I think that having each line have unique contents makes
it a bit easier to parse.
