Commit graph

136 commits

Author SHA1 Message Date
Jan Verbeek
6f7d740592 wc: Do a chunked read with proper UTF-8 handling
This brings the results mostly in line with GNU wc and solves nasty
behavior with long lines.
2021-08-26 01:38:16 +02:00
jfinkels
bdc0f4b7c3
hashsum: support --check for algorithms with variable output length (#2583)
* hashsum: support --check for var. length outputs

Add the ability for `hashsum --check` to work with algorithms with
variable output length. Previously, the program would terminate with an
error due to constructing an invalid regular expression.

* fixup! hashsum: support --check for var. length outputs
2021-08-23 18:35:19 +02:00
jfinkels
4ef35d4a96
tac: correct behavior of -b option (#2523)
* tac: correct behavior of -b option

Correct the behavior of `tac -b` to match that of GNU coreutils
`tac`. Specifically, this changes `tac -b` to assume *leading* line
separators instead of the default *trailing* line separators.

Before this commit, the (incorrect) behavior was

    $ printf "/abc/def" | tac -b -s "/"
    def/abc/

After this commit, the behavior is

    $ printf "/abc/def" | tac -b -s "/"
    /def/abc

Fixes #2262.

* fixup! tac: correct behavior of -b option

* fixup! tac: correct behavior of -b option

Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
2021-08-22 21:01:17 +02:00
Justin Tracey
1bb0237281 join: add support for full outer joins 2021-08-12 23:52:35 -04:00
Tyler
601c9fc620 Merge branch 'master' of https://github.com/uutils/coreutils into uutils-master-2 2021-08-03 17:33:43 -07:00
Michael Debertol
418f5b7692 sort: handle empty merge inputs 2021-07-31 21:02:20 +02:00
Tyler
076ff32e85 Removes project-specific cspell files. 2021-07-23 14:53:24 -07:00
Tyler
885a875552 Addresses build errors
- Adds words to cspell exceptions
- Converts test macros to use Default trait.
- Converts parser to use Default trait.
- Adds Windows-friendly test files for block/unblock when nl is present
  in test/spec file.
2021-07-22 16:04:35 -07:00
Tyler
88363858d5 Minor changes. 2021-07-12 10:13:47 -07:00
Tyler
2e9e984b3a Adds test with unicode filename
Filenames should be the only spot where unicode can appear in dd's cli.
2021-07-06 18:17:30 -07:00
backwaterred
9c38583c6b
Merge pull request #2 from uutils/master
catchup with uutils main
2021-07-02 11:34:22 -07:00
Tyler
92281585a7 Merge branch 'master' of https://github.com/uutils/coreutils into uutils-master 2021-07-01 14:33:30 -07:00
Tyler
17cfba41cc Implements project testfing from root.
- conv=FLAG testing. (1) WIP conv=nocreat
- iflag & oflag testing.
- conv=CONV ascii,...,ucase,...,block,...sync tests at unit-test-level
  (project root is todo)
2021-06-30 14:47:48 -07:00
Michael Debertol
233a778963 sort/ls: implement version cmp matching GNU spec
This reimplements version_cmp, which is used in sort and ls to sort
according to versions.
However, it is not bug-for-bug identical with GNU's implementation.
I reported a bug with GNU here:
https://lists.gnu.org/archive/html/bug-coreutils/2021-06/msg00045.html
This implementation does not contain the bugs regarding the handling of
file extensions and null bytes.
2021-06-27 15:29:17 +02:00
Michael Debertol
548a895cd6 sort: compatibility of human-numeric sort
Closes #1985.
This makes human-numeric sort follow the same algorithm as GNU's/FreeBSD's sort.
As documented by GNU in https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html,
we first compare by sign, then by si unit and finally by the numeric value.
2021-06-25 18:19:00 +02:00
Syukron Rifail M
bc8415c9db du: add --dereference 2021-06-17 14:06:41 +07:00
Terts Diepraam
8afc923796
Merge pull request #2237 from wfscheper/wfscheper/issue2118
chgrp: replace getopts with clap (#2118)
2021-06-12 11:20:24 +02:00
Walter Scheper
cff75f242a chgrp: replace getopts with clap (#2118) 2021-06-10 16:38:44 -04:00
Michael Debertol
66359a0f56 sort: insert line separators after non-empty files
If files don't end witht a line separator we have to insert one,
otherwise the last line will be combined with the first line of the next
file.
2021-06-06 18:01:08 +02:00
Michael Debertol
7ffc7d073c cp: test that file descriptors are closed 2021-06-02 19:21:16 +02:00
Michael Debertol
06b3092f5f sort: fix debug output for zeros / invalid numbers
We were reporting "no match" when sorting something like "0 ". This is
because we don't distinguish between 0 and invalid lines when sorting.
For debug output we have to get this information back.
2021-06-01 18:18:51 +02:00
Sylvestre Ledru
badf7aacb7
Merge pull request #2300 from tertsdiepraam/pr
Implement `pr` (resurrection of the resurrected PR)
2021-05-31 21:14:57 +02:00
Roy Ivy III
4e20dedf58 tests ~ refactor/polish spelling (comments, names, and exceptions) 2021-05-31 08:23:57 -05:00
Terts Diepraam
7690dc018f Merge branch 'master' into pr 2021-05-31 15:23:06 +02:00
Michael Debertol
dc63133f14
sort: correctly inherit global flags for keys (#2302)
Closes #2254. We should only inherit global settings for keys when there
are absolutely no options attached to the key.

The default key (matching the whole line) is implicitly added only if no
keys are supplied.

Improved some error messages by including more context.
2021-05-29 23:25:56 +02:00
Terts Diepraam
bc1870c0a7 Merge branch 'master' into pr 2021-05-29 19:21:31 +02:00
Michael Debertol
e9656a6c32
sort: make GNU test sort-debug-keys pass (#2269)
* sort: disable support for thousand separators

In order to be compatible with GNU, we have to disable thousands
separators. GNU does not enable them for the C locale, either.

Once we add support for locales we can add this feature back.

* sort: delete unused fixtures

* sort: compare -0 and 0 equal

I must have misunderstood this when implementing, but GNU considers
-0, 0, and invalid numbers to be equal.

* sort: strip blanks before applying the char index

* sort: don't crash when key start is after key end

* sort: add "no match" for months at the first non-whitespace char

We should put the "^ no match for key" indicator at the first
non-whitespace character of a field.

* sort: improve support for e notation

* sort: use maches! macros
2021-05-28 22:38:29 +02:00
Jeffrey Finkelstein
659bf58a4c head: print headings when reading multiple files
Fix a bug in which `head` failed to print headings for `stdin` inputs
when reading from multiple files, and fix another bug in which `head`
failed to print a blank line between the contents of a file and the
heading for the next file when reading multiple files. The output now
matches that of GNU `head`.
2021-05-16 12:03:10 -04:00
Michael Debertol
e0ebf907a4 sort: make merging stable
When merging files we need to prioritize files that occur earlier in the
command line arguments with -m.

This also makes the extsort merge step (and thus extsort itself) stable again.
2021-05-09 11:43:38 +02:00
Jeffrey Finkelstein
0cafe2b70d wc: add tests for edge cases for wc on files 2021-05-03 21:07:32 -04:00
Daniel Rocco
c3912d53ac test: add tests for basic tests & edge cases
Some edge cases covered:

- no args
- operator by itself (!, -a, etc.)
- string, file tests of nothing
- compound negations
2021-05-01 22:40:47 -04:00
Ricardo Iglesias
05b20c32a9 base64: Moved argument parsing to clap.
Moved argument parsing to clap and added tests to cover using "-" as
stdin, passing in too many file arguments, and updated the "wrap" error
message in the tests.
2021-05-01 11:36:46 -07:00
Sylvestre Ledru
a37e3181a2
Merge pull request #2130 from electricboogie/master
sort: implement --buffer-size and --temporary-directory (external sort)
2021-04-28 09:21:14 +02:00
Ricardo Iglesias
d56462a4b3 base32: Fixed style violations. Added tests
Tests now cover using "-" as standard input and reading from a file.
2021-04-26 08:00:55 -07:00
electricboogie
4c395146dd Merge branch 'master' of https://github.com/uutils/coreutils 2021-04-25 10:11:27 -05:00
Michael Debertol
e6f6b109a5 sort: implement --debug
This adds a --debug flag, which, when activated, will draw lines below
the characters that are actually used for comparisons.

This is not a complete implementation of --debug. It should, quoting the man page
for GNU sort: "annotate the part of the line used to sort, and warn
about questionable usage to stderr". Warning about "questionable usage"
is not part of this patch.

This change required some adjustments to be able to get the range that
is actually used for comparisons. Most notably, general numeric comparisons
were rewritten, fixing some bugs along the lines.

Testing is mostly done by adding fixtures for the expected debug output of
existing tests.
2021-04-23 22:36:15 +02:00
electricboogie
25021f31eb Incorporate overhead of Line struct 2021-04-19 21:24:52 -05:00
Michael Debertol
4bbbe3a3f2
sort: implement numeric string comparison (#2070)
* sort: implement numeric string comparison

This implements -n and -h using a string comparison algorithm instead
of parsing each number to a f64 and comparing those.

This should result in a moderate performance increase and eliminate loss
of precision.

* cache parsed f64 numbers

For general numeric comparisons we have to parse numbers as f64,
as this behavior is explicitly documented by GNU coreutils.
We can however cache the parsed value to speed up comparisons.

* fix leading zeroes for negative numbers

* use more appropriate name for exponent

* improvements to the parse function

* move checks into main loop and fix thousands separator condition

* remove unneeded checks

* rustfmt
2021-04-17 13:49:35 +02:00
electricboogie
a76d452f75
Sort: More small fixes (#2065)
* Various fixes and performance improvements

* fix a typo

Co-authored-by: Michael Debertol <michael.debertol@gmail.com>

* Fix month parse for months with leading whitespace

* Implement test for months whitespace fix

* Confirm human numeric works as expected with whitespace with a test

* Correct arg help value name for --parallel

* Fix SemVer non version lines/empty line sorting with a test

Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
2021-04-17 10:06:19 +02:00
Árni Dagur
eb4971e6f4
cat: Unrevert splice patch (#2020)
* cat: Unrevert splice patch

* cat: Add fifo test

* cat: Add tests for error cases

* cat: Add tests for character devices

* wc: Make sure we handle short splice writes

* cat: Fix tests for 1.40.0 compiler

* cat: Run rustfmt on test_cat.rs

* Run 'cargo +1.40.0 update'
2021-04-10 22:19:53 +02:00
Sylvestre Ledru
bf1944271c remove .DS_Store 2021-04-10 21:57:03 +02:00
Michael Debertol
69f4410a8a
sort: dedup using compare_by (#2064)
compare_by is the function used for sorting, we should use it for dedup
as well.
2021-04-10 19:49:10 +02:00
electricboogie
e5113ad00e
Sort: Various fixes and performance improvements (#2057)
* Various fixes and performance improvements

* fix a typo

Co-authored-by: Michael Debertol <michael.debertol@gmail.com>

Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
2021-04-10 11:56:20 +02:00
Sivachandran
ee070028e4
install: implement stripping symbol table (#2047) 2021-04-10 11:53:29 +02:00
Sylvestre Ledru
844e318a67
Merge branch 'master' into pr 2021-04-09 22:02:25 +02:00
electricboogie
8474249e5f
Sort: Implement stable sort, ignore non-printing, month sort dedup, auto parallel sort through rayon, zero terminated sort, check silent (#2008) 2021-04-08 22:07:09 +02:00
Yagiz Degirmenci
c965effe07
fold: move to clap, add tests (#2015) 2021-04-06 22:51:27 +02:00
Sylvestre Ledru
f57eb0fdfa
Merge pull request #1993 from cbjadwani/master
uniq: Implement --group option
2021-04-05 22:33:04 +02:00
Yagiz Degirmenci
cbe07c93c6
cksum: add tests and fixtures (#1923) 2021-04-05 22:21:21 +02:00
Daniel Rocco
e5c61a28be fold: variable width tabs, guard treating tab as whitespace
Treat tab chars as advancing to the next tab stop rather than having a fixed
8-column width.

Also treat tab as a whitespace split target only when splitting on word
boundaries.
2021-04-05 08:55:07 -04:00