Commit graph

158 commits

Author SHA1 Message Date
Sylvestre Ledru
f9e04ae5ef
Merge pull request #2966 from allan-silva/wc-files0-from-opt
wc: implement files0-from option
2022-02-12 19:05:05 +01:00
Sylvestre Ledru
6b6d5ee7db
Merge pull request #2827 from jfinkels/split-std-io-copy
split: use std::io::copy() with new writer implementation to improve maintainability and speed
2022-02-12 11:33:12 +01:00
Shreyans Jain
3176ad5c1b
tests/hashsum: Fix missing space in checkfile 2022-02-10 13:55:53 +05:30
Shreyans Jain
30d7a4b167
hashsum: Add BLAKE3 to Hashing Algorithms
Signed-off-by: Shreyans Jain <shreyansthebest2007@gmail.com>
2022-02-10 12:46:44 +05:30
Jeffrey Finkelstein
1d7e1b8732 split: use ByteChunkWriter and LineChunkWriter
Replace `ByteSplitter` and `LineSplitter` with `ByteChunkWriter` and
`LineChunkWriter` respectively. This results in a more maintainable
design and an increase in the speed of splitting by lines.
2022-02-08 22:57:57 -05:00
Jeffrey Finkelstein
ca7af808d5 tests: correct a test case for split
Correct the `test_split::test_suffixes_exhausted` test case so that it
actually exercises the intended behavior of `split`. Previously, the
test fixture contained 26 bytes. After this commit, the test fixture
contains 27 bytes. When using a suffix width of one, only 26 filenames
should be available when naming chunk files---one for each lowercase
ASCII letter. This commit ensures that the filenames will be exhausted
as intended by the test.
2022-02-08 22:53:57 -05:00
Allan Silva
6a6875012e wc: implement files0-from option
When this option is present, the files argument is not processed. This option processes the file list from provided file, splitting them by the ascii NUL (\0) character. When files0-from is '-', the file list is processed from stdin.
2022-02-04 10:12:08 -03:00
Terts Diepraam
7fc82cd376
Merge pull request #2902 from jtracey/join-non-unicode-sep
join: add support for non-unicode field separators
2022-01-31 21:54:56 +01:00
Terts Diepraam
7477761428
Merge pull request #2882 from jtracey/join-bigfields-compat
join: "support" field numbers larger than usize::MAX
2022-01-31 21:52:13 +01:00
Justin Tracey
58d65fb953 join: add support for non-unicode field separators
This allows for `-t` to take invalid unicode (but still single-byte) values
on unix-like platforms. Other platforms, which as of the time of this commit
do not support `OsStr::as_bytes()`, could possibly be supported in the future,
but would require design decisions as to what that means.
2022-01-30 20:04:22 -05:00
Sylvestre Ledru
7c1abdb7d9
Merge pull request #2866 from jfinkels/split-number-2
split: implement -n option
2022-01-30 09:58:04 +01:00
Sylvestre Ledru
52ab6325a0
Merge pull request #2881 from jtracey/join-null-field-sep
join: add support for `-t '\0'`
2022-01-29 10:55:04 +01:00
Jeffrey Finkelstein
b636ff04a0 split: implement -n option
Implement the `-n` command-line option to `split`, which splits a file
into a specified number of chunks by byte.
2022-01-27 21:16:27 -05:00
Cecylia Bocovich
c8f9ea5b15
tests/join: test default check order behaviour 2022-01-22 17:51:29 -05:00
Justin Tracey
ce3df12eaa join: "support" field numbers larger than usize::MAX
They silently get folded to usize::MAX, which is the official GNU behavior.
2022-01-17 17:49:41 -05:00
Terts Diepraam
08efa1fe5a
Merge branch 'main' into join-null-field-sep 2022-01-17 12:59:52 +01:00
Justin Tracey
109277d405 join: add support for -t '\0' 2022-01-16 18:05:58 -05:00
Justin Tracey
346415e1d2 join: add support for -z option 2022-01-16 17:56:07 -05:00
Sylvestre Ledru
00c11b184f
Merge pull request #2851 from jtracey/join-strless
join: operate on bytes instead of Strings
2022-01-16 16:24:38 +01:00
Jeffrey Finkelstein
cfe5a0d82c split: correct filename creation algorithm
Fix two issues with the filename creation algorithm. First, this
corrects the behavior of the `-a` option. This commit ensures a
failure occurs when the number of chunks exceeds the number of
filenames representable with the specified fixed width:

    $ printf "%0.sa" {1..11} | split -d -b 1 -a 1
    split: output file suffixes exhausted

Second, this corrects the behavior of the default behavior when `-a`
is not specified on the command line. Previously, it was always
settings the filenames to have length 2 suffixes. This commit corrects
the behavior to follow the algorithm implied by GNU split, where the
filename lengths grow dynamically by two characters once the number of
chunks grows sufficiently large:

    $ printf "%0.sa" {1..91} | ./target/debug/coreutils split -d -b 1 \
    >   && ls x* | tail
    x81
    x82
    x83
    x84
    x85
    x86
    x87
    x88
    x89
    x9000
2022-01-10 20:43:22 -05:00
Justin Tracey
4df2f3c148 join: add test for non-Unicode files 2022-01-08 21:28:29 -05:00
Justin Tracey
cdfe64369d join: add test for non-linefeed newline characters 2022-01-08 19:51:16 -05:00
Jan Verbeek
6f7d740592 wc: Do a chunked read with proper UTF-8 handling
This brings the results mostly in line with GNU wc and solves nasty
behavior with long lines.
2021-08-26 01:38:16 +02:00
jfinkels
bdc0f4b7c3
hashsum: support --check for algorithms with variable output length (#2583)
* hashsum: support --check for var. length outputs

Add the ability for `hashsum --check` to work with algorithms with
variable output length. Previously, the program would terminate with an
error due to constructing an invalid regular expression.

* fixup! hashsum: support --check for var. length outputs
2021-08-23 18:35:19 +02:00
jfinkels
4ef35d4a96
tac: correct behavior of -b option (#2523)
* tac: correct behavior of -b option

Correct the behavior of `tac -b` to match that of GNU coreutils
`tac`. Specifically, this changes `tac -b` to assume *leading* line
separators instead of the default *trailing* line separators.

Before this commit, the (incorrect) behavior was

    $ printf "/abc/def" | tac -b -s "/"
    def/abc/

After this commit, the behavior is

    $ printf "/abc/def" | tac -b -s "/"
    /def/abc

Fixes #2262.

* fixup! tac: correct behavior of -b option

* fixup! tac: correct behavior of -b option

Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
2021-08-22 21:01:17 +02:00
Justin Tracey
1bb0237281 join: add support for full outer joins 2021-08-12 23:52:35 -04:00
Tyler
601c9fc620 Merge branch 'master' of https://github.com/uutils/coreutils into uutils-master-2 2021-08-03 17:33:43 -07:00
Michael Debertol
418f5b7692 sort: handle empty merge inputs 2021-07-31 21:02:20 +02:00
Tyler
076ff32e85 Removes project-specific cspell files. 2021-07-23 14:53:24 -07:00
Tyler
885a875552 Addresses build errors
- Adds words to cspell exceptions
- Converts test macros to use Default trait.
- Converts parser to use Default trait.
- Adds Windows-friendly test files for block/unblock when nl is present
  in test/spec file.
2021-07-22 16:04:35 -07:00
Tyler
88363858d5 Minor changes. 2021-07-12 10:13:47 -07:00
Tyler
2e9e984b3a Adds test with unicode filename
Filenames should be the only spot where unicode can appear in dd's cli.
2021-07-06 18:17:30 -07:00
backwaterred
9c38583c6b
Merge pull request #2 from uutils/master
catchup with uutils main
2021-07-02 11:34:22 -07:00
Tyler
92281585a7 Merge branch 'master' of https://github.com/uutils/coreutils into uutils-master 2021-07-01 14:33:30 -07:00
Tyler
17cfba41cc Implements project testfing from root.
- conv=FLAG testing. (1) WIP conv=nocreat
- iflag & oflag testing.
- conv=CONV ascii,...,ucase,...,block,...sync tests at unit-test-level
  (project root is todo)
2021-06-30 14:47:48 -07:00
Michael Debertol
233a778963 sort/ls: implement version cmp matching GNU spec
This reimplements version_cmp, which is used in sort and ls to sort
according to versions.
However, it is not bug-for-bug identical with GNU's implementation.
I reported a bug with GNU here:
https://lists.gnu.org/archive/html/bug-coreutils/2021-06/msg00045.html
This implementation does not contain the bugs regarding the handling of
file extensions and null bytes.
2021-06-27 15:29:17 +02:00
Michael Debertol
548a895cd6 sort: compatibility of human-numeric sort
Closes #1985.
This makes human-numeric sort follow the same algorithm as GNU's/FreeBSD's sort.
As documented by GNU in https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html,
we first compare by sign, then by si unit and finally by the numeric value.
2021-06-25 18:19:00 +02:00
Syukron Rifail M
bc8415c9db du: add --dereference 2021-06-17 14:06:41 +07:00
Terts Diepraam
8afc923796
Merge pull request #2237 from wfscheper/wfscheper/issue2118
chgrp: replace getopts with clap (#2118)
2021-06-12 11:20:24 +02:00
Walter Scheper
cff75f242a chgrp: replace getopts with clap (#2118) 2021-06-10 16:38:44 -04:00
Michael Debertol
66359a0f56 sort: insert line separators after non-empty files
If files don't end witht a line separator we have to insert one,
otherwise the last line will be combined with the first line of the next
file.
2021-06-06 18:01:08 +02:00
Michael Debertol
7ffc7d073c cp: test that file descriptors are closed 2021-06-02 19:21:16 +02:00
Michael Debertol
06b3092f5f sort: fix debug output for zeros / invalid numbers
We were reporting "no match" when sorting something like "0 ". This is
because we don't distinguish between 0 and invalid lines when sorting.
For debug output we have to get this information back.
2021-06-01 18:18:51 +02:00
Sylvestre Ledru
badf7aacb7
Merge pull request #2300 from tertsdiepraam/pr
Implement `pr` (resurrection of the resurrected PR)
2021-05-31 21:14:57 +02:00
Roy Ivy III
4e20dedf58 tests ~ refactor/polish spelling (comments, names, and exceptions) 2021-05-31 08:23:57 -05:00
Terts Diepraam
7690dc018f Merge branch 'master' into pr 2021-05-31 15:23:06 +02:00
Michael Debertol
dc63133f14
sort: correctly inherit global flags for keys (#2302)
Closes #2254. We should only inherit global settings for keys when there
are absolutely no options attached to the key.

The default key (matching the whole line) is implicitly added only if no
keys are supplied.

Improved some error messages by including more context.
2021-05-29 23:25:56 +02:00
Terts Diepraam
bc1870c0a7 Merge branch 'master' into pr 2021-05-29 19:21:31 +02:00
Michael Debertol
e9656a6c32
sort: make GNU test sort-debug-keys pass (#2269)
* sort: disable support for thousand separators

In order to be compatible with GNU, we have to disable thousands
separators. GNU does not enable them for the C locale, either.

Once we add support for locales we can add this feature back.

* sort: delete unused fixtures

* sort: compare -0 and 0 equal

I must have misunderstood this when implementing, but GNU considers
-0, 0, and invalid numbers to be equal.

* sort: strip blanks before applying the char index

* sort: don't crash when key start is after key end

* sort: add "no match" for months at the first non-whitespace char

We should put the "^ no match for key" indicator at the first
non-whitespace character of a field.

* sort: improve support for e notation

* sort: use maches! macros
2021-05-28 22:38:29 +02:00
Jeffrey Finkelstein
659bf58a4c head: print headings when reading multiple files
Fix a bug in which `head` failed to print headings for `stdin` inputs
when reading from multiple files, and fix another bug in which `head`
failed to print a blank line between the contents of a file and the
heading for the next file when reading multiple files. The output now
matches that of GNU `head`.
2021-05-16 12:03:10 -04:00