Commit graph

189 commits

Author SHA1 Message Date
Justin Tracey
cdfe64369d join: add test for non-linefeed newline characters 2022-01-08 19:51:16 -05:00
Jan Scheer
94cc966535
tail: change notify backend on macOS from FSEvents to kqueue
On macOS only `kqueue` is suitable for our use case because `FSEvents`
waits until file close to delivers modify events.
2021-10-01 21:33:30 +02:00
Jan Scheer
5615ba9fe1
test_tail: add tests for --follow=name 2021-09-27 23:18:00 +02:00
Jan Verbeek
6f7d740592 wc: Do a chunked read with proper UTF-8 handling
This brings the results mostly in line with GNU wc and solves nasty
behavior with long lines.
2021-08-26 01:38:16 +02:00
jfinkels
bdc0f4b7c3
hashsum: support --check for algorithms with variable output length (#2583)
* hashsum: support --check for var. length outputs

Add the ability for `hashsum --check` to work with algorithms with
variable output length. Previously, the program would terminate with an
error due to constructing an invalid regular expression.

* fixup! hashsum: support --check for var. length outputs
2021-08-23 18:35:19 +02:00
jfinkels
4ef35d4a96
tac: correct behavior of -b option (#2523)
* tac: correct behavior of -b option

Correct the behavior of `tac -b` to match that of GNU coreutils
`tac`. Specifically, this changes `tac -b` to assume *leading* line
separators instead of the default *trailing* line separators.

Before this commit, the (incorrect) behavior was

    $ printf "/abc/def" | tac -b -s "/"
    def/abc/

After this commit, the behavior is

    $ printf "/abc/def" | tac -b -s "/"
    /def/abc

Fixes #2262.

* fixup! tac: correct behavior of -b option

* fixup! tac: correct behavior of -b option

Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
2021-08-22 21:01:17 +02:00
Justin Tracey
1bb0237281 join: add support for full outer joins 2021-08-12 23:52:35 -04:00
Tyler
601c9fc620 Merge branch 'master' of https://github.com/uutils/coreutils into uutils-master-2 2021-08-03 17:33:43 -07:00
Michael Debertol
418f5b7692 sort: handle empty merge inputs 2021-07-31 21:02:20 +02:00
Tyler
076ff32e85 Removes project-specific cspell files. 2021-07-23 14:53:24 -07:00
Tyler
885a875552 Addresses build errors
- Adds words to cspell exceptions
- Converts test macros to use Default trait.
- Converts parser to use Default trait.
- Adds Windows-friendly test files for block/unblock when nl is present
  in test/spec file.
2021-07-22 16:04:35 -07:00
Tyler
88363858d5 Minor changes. 2021-07-12 10:13:47 -07:00
Tyler
2e9e984b3a Adds test with unicode filename
Filenames should be the only spot where unicode can appear in dd's cli.
2021-07-06 18:17:30 -07:00
backwaterred
9c38583c6b
Merge pull request #2 from uutils/master
catchup with uutils main
2021-07-02 11:34:22 -07:00
Tyler
92281585a7 Merge branch 'master' of https://github.com/uutils/coreutils into uutils-master 2021-07-01 14:33:30 -07:00
Tyler
17cfba41cc Implements project testfing from root.
- conv=FLAG testing. (1) WIP conv=nocreat
- iflag & oflag testing.
- conv=CONV ascii,...,ucase,...,block,...sync tests at unit-test-level
  (project root is todo)
2021-06-30 14:47:48 -07:00
Michael Debertol
233a778963 sort/ls: implement version cmp matching GNU spec
This reimplements version_cmp, which is used in sort and ls to sort
according to versions.
However, it is not bug-for-bug identical with GNU's implementation.
I reported a bug with GNU here:
https://lists.gnu.org/archive/html/bug-coreutils/2021-06/msg00045.html
This implementation does not contain the bugs regarding the handling of
file extensions and null bytes.
2021-06-27 15:29:17 +02:00
Michael Debertol
548a895cd6 sort: compatibility of human-numeric sort
Closes #1985.
This makes human-numeric sort follow the same algorithm as GNU's/FreeBSD's sort.
As documented by GNU in https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html,
we first compare by sign, then by si unit and finally by the numeric value.
2021-06-25 18:19:00 +02:00
Syukron Rifail M
bc8415c9db du: add --dereference 2021-06-17 14:06:41 +07:00
Terts Diepraam
8afc923796
Merge pull request #2237 from wfscheper/wfscheper/issue2118
chgrp: replace getopts with clap (#2118)
2021-06-12 11:20:24 +02:00
Walter Scheper
cff75f242a chgrp: replace getopts with clap (#2118) 2021-06-10 16:38:44 -04:00
Michael Debertol
66359a0f56 sort: insert line separators after non-empty files
If files don't end witht a line separator we have to insert one,
otherwise the last line will be combined with the first line of the next
file.
2021-06-06 18:01:08 +02:00
Michael Debertol
7ffc7d073c cp: test that file descriptors are closed 2021-06-02 19:21:16 +02:00
Michael Debertol
06b3092f5f sort: fix debug output for zeros / invalid numbers
We were reporting "no match" when sorting something like "0 ". This is
because we don't distinguish between 0 and invalid lines when sorting.
For debug output we have to get this information back.
2021-06-01 18:18:51 +02:00
Sylvestre Ledru
badf7aacb7
Merge pull request #2300 from tertsdiepraam/pr
Implement `pr` (resurrection of the resurrected PR)
2021-05-31 21:14:57 +02:00
Roy Ivy III
4e20dedf58 tests ~ refactor/polish spelling (comments, names, and exceptions) 2021-05-31 08:23:57 -05:00
Terts Diepraam
7690dc018f Merge branch 'master' into pr 2021-05-31 15:23:06 +02:00
Michael Debertol
dc63133f14
sort: correctly inherit global flags for keys (#2302)
Closes #2254. We should only inherit global settings for keys when there
are absolutely no options attached to the key.

The default key (matching the whole line) is implicitly added only if no
keys are supplied.

Improved some error messages by including more context.
2021-05-29 23:25:56 +02:00
Terts Diepraam
bc1870c0a7 Merge branch 'master' into pr 2021-05-29 19:21:31 +02:00
Michael Debertol
e9656a6c32
sort: make GNU test sort-debug-keys pass (#2269)
* sort: disable support for thousand separators

In order to be compatible with GNU, we have to disable thousands
separators. GNU does not enable them for the C locale, either.

Once we add support for locales we can add this feature back.

* sort: delete unused fixtures

* sort: compare -0 and 0 equal

I must have misunderstood this when implementing, but GNU considers
-0, 0, and invalid numbers to be equal.

* sort: strip blanks before applying the char index

* sort: don't crash when key start is after key end

* sort: add "no match" for months at the first non-whitespace char

We should put the "^ no match for key" indicator at the first
non-whitespace character of a field.

* sort: improve support for e notation

* sort: use maches! macros
2021-05-28 22:38:29 +02:00
Jeffrey Finkelstein
659bf58a4c head: print headings when reading multiple files
Fix a bug in which `head` failed to print headings for `stdin` inputs
when reading from multiple files, and fix another bug in which `head`
failed to print a blank line between the contents of a file and the
heading for the next file when reading multiple files. The output now
matches that of GNU `head`.
2021-05-16 12:03:10 -04:00
Michael Debertol
e0ebf907a4 sort: make merging stable
When merging files we need to prioritize files that occur earlier in the
command line arguments with -m.

This also makes the extsort merge step (and thus extsort itself) stable again.
2021-05-09 11:43:38 +02:00
Jeffrey Finkelstein
0cafe2b70d wc: add tests for edge cases for wc on files 2021-05-03 21:07:32 -04:00
Daniel Rocco
c3912d53ac test: add tests for basic tests & edge cases
Some edge cases covered:

- no args
- operator by itself (!, -a, etc.)
- string, file tests of nothing
- compound negations
2021-05-01 22:40:47 -04:00
Ricardo Iglesias
05b20c32a9 base64: Moved argument parsing to clap.
Moved argument parsing to clap and added tests to cover using "-" as
stdin, passing in too many file arguments, and updated the "wrap" error
message in the tests.
2021-05-01 11:36:46 -07:00
Sylvestre Ledru
a37e3181a2
Merge pull request #2130 from electricboogie/master
sort: implement --buffer-size and --temporary-directory (external sort)
2021-04-28 09:21:14 +02:00
Ricardo Iglesias
d56462a4b3 base32: Fixed style violations. Added tests
Tests now cover using "-" as standard input and reading from a file.
2021-04-26 08:00:55 -07:00
electricboogie
4c395146dd Merge branch 'master' of https://github.com/uutils/coreutils 2021-04-25 10:11:27 -05:00
Michael Debertol
e6f6b109a5 sort: implement --debug
This adds a --debug flag, which, when activated, will draw lines below
the characters that are actually used for comparisons.

This is not a complete implementation of --debug. It should, quoting the man page
for GNU sort: "annotate the part of the line used to sort, and warn
about questionable usage to stderr". Warning about "questionable usage"
is not part of this patch.

This change required some adjustments to be able to get the range that
is actually used for comparisons. Most notably, general numeric comparisons
were rewritten, fixing some bugs along the lines.

Testing is mostly done by adding fixtures for the expected debug output of
existing tests.
2021-04-23 22:36:15 +02:00
electricboogie
25021f31eb Incorporate overhead of Line struct 2021-04-19 21:24:52 -05:00
Michael Debertol
4bbbe3a3f2
sort: implement numeric string comparison (#2070)
* sort: implement numeric string comparison

This implements -n and -h using a string comparison algorithm instead
of parsing each number to a f64 and comparing those.

This should result in a moderate performance increase and eliminate loss
of precision.

* cache parsed f64 numbers

For general numeric comparisons we have to parse numbers as f64,
as this behavior is explicitly documented by GNU coreutils.
We can however cache the parsed value to speed up comparisons.

* fix leading zeroes for negative numbers

* use more appropriate name for exponent

* improvements to the parse function

* move checks into main loop and fix thousands separator condition

* remove unneeded checks

* rustfmt
2021-04-17 13:49:35 +02:00
electricboogie
a76d452f75
Sort: More small fixes (#2065)
* Various fixes and performance improvements

* fix a typo

Co-authored-by: Michael Debertol <michael.debertol@gmail.com>

* Fix month parse for months with leading whitespace

* Implement test for months whitespace fix

* Confirm human numeric works as expected with whitespace with a test

* Correct arg help value name for --parallel

* Fix SemVer non version lines/empty line sorting with a test

Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
2021-04-17 10:06:19 +02:00
Árni Dagur
eb4971e6f4
cat: Unrevert splice patch (#2020)
* cat: Unrevert splice patch

* cat: Add fifo test

* cat: Add tests for error cases

* cat: Add tests for character devices

* wc: Make sure we handle short splice writes

* cat: Fix tests for 1.40.0 compiler

* cat: Run rustfmt on test_cat.rs

* Run 'cargo +1.40.0 update'
2021-04-10 22:19:53 +02:00
Sylvestre Ledru
bf1944271c remove .DS_Store 2021-04-10 21:57:03 +02:00
Michael Debertol
69f4410a8a
sort: dedup using compare_by (#2064)
compare_by is the function used for sorting, we should use it for dedup
as well.
2021-04-10 19:49:10 +02:00
electricboogie
e5113ad00e
Sort: Various fixes and performance improvements (#2057)
* Various fixes and performance improvements

* fix a typo

Co-authored-by: Michael Debertol <michael.debertol@gmail.com>

Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
Co-authored-by: Michael Debertol <michael.debertol@gmail.com>
2021-04-10 11:56:20 +02:00
Sivachandran
ee070028e4
install: implement stripping symbol table (#2047) 2021-04-10 11:53:29 +02:00
Sylvestre Ledru
844e318a67
Merge branch 'master' into pr 2021-04-09 22:02:25 +02:00
electricboogie
8474249e5f
Sort: Implement stable sort, ignore non-printing, month sort dedup, auto parallel sort through rayon, zero terminated sort, check silent (#2008) 2021-04-08 22:07:09 +02:00
Yagiz Degirmenci
c965effe07
fold: move to clap, add tests (#2015) 2021-04-06 22:51:27 +02:00
Sylvestre Ledru
f57eb0fdfa
Merge pull request #1993 from cbjadwani/master
uniq: Implement --group option
2021-04-05 22:33:04 +02:00
Yagiz Degirmenci
cbe07c93c6
cksum: add tests and fixtures (#1923) 2021-04-05 22:21:21 +02:00
Daniel Rocco
e5c61a28be fold: variable width tabs, guard treating tab as whitespace
Treat tab chars as advancing to the next tab stop rather than having a fixed
8-column width.

Also treat tab as a whitespace split target only when splitting on word
boundaries.
2021-04-05 08:55:07 -04:00
Chirag Jadwani
19c6a42de5 uniq: implement group option 2021-04-04 15:22:17 +05:30
Paul Otten
7859bf885f Consistency with GNU version of du when doing du -h on an empty file 2021-04-01 19:42:43 -04:00
Mikadore
8320b1ec5f
Rewrote head (#1911)
See https://github.com/uutils/coreutils/pull/1911
for the details
2021-03-29 13:08:48 +02:00
jaggededgedjustice
88d0bb01c0
Add shuf tests (#1958)
* Add tests for shuf

* Fixup GNU tests for shuf
2021-03-28 17:52:01 +02:00
Max Semenik
62fe68850e pr: Fixes after rebasing
Only the minimum needed to:
* Make everything compile without warnings
* Move files according to the new project structure
* Make tests pass
2021-03-26 17:57:19 +03:00
tilakpatidar
75b35e6002 pr: remove not required tests 2021-03-26 14:11:15 +03:00
tilakpatidar
054c05d5d8 pr: refactor get_lines_for_printing, write_columns, recreate_arguments
pr: extract recreate_arguments

pr: refactor get_line_for_printing

pr: refactor get_lines_for_printing

pr: refactor fetch_indexes generate for write_columns

pr: refactor write_columns

pr: refactor write_columns
2021-03-26 14:11:15 +03:00
tilakpatidar
40e7f3d900 pr: add -J and -S option
pr: add -J option

pr: add -S option
2021-03-26 14:11:15 +03:00
tilakpatidar
a4b723233a pr: add more tests for form feed and include page_width option W 2021-03-26 14:11:15 +03:00
tilakpatidar
3be5dc6923 pr: fix form feed
pr: fix form feed

pr: Rustfmt

pr: add test for ff and -l option
2021-03-26 14:11:15 +03:00
Tilak Patidar
5956894d00 pr: add -m and -o option
pr: Add -o option
2021-03-26 14:11:14 +03:00
Tilak Patidar
dd07aed4d1 pr: add column separator option 2021-03-26 14:11:14 +03:00
Tilak Patidar
69371ce3ce pr: add tests for --column with across option 2021-03-26 14:11:14 +03:00
Tilak Patidar
f497fb9d88 pr: read from stdin 2021-03-26 14:11:14 +03:00
Tilak Patidar
d9084a7399 pr: implement across option and fix tests 2021-03-26 14:11:14 +03:00
Tilak Patidar
f3676573b5 pr: print padded string for each column and handle tab issues
pr: Print fixed padded string for each column

pr: Fix display length vs str length due to tabs
2021-03-26 14:11:14 +03:00
Tilak Patidar
b578bb6563 pr: add test for -t -l -r option
pr: Add test for -l option

pr: Add test for -r suppress error option
2021-03-26 14:11:14 +03:00
Tilak Patidar
b742230dbb pr: fix page ranges
pr: Fix page ranges
2021-03-26 14:11:14 +03:00
Tilak Patidar
88ec02a61c pr: add suport for -n [char][width] and -N
pr: Fix long name for -n

pr: Add -N first line number option

pr: Add -n[char][width] support
2021-03-26 14:11:14 +03:00
Tilak Patidar
afc58eb6ea pr: add tests for -n -h -d option
pr: Add test for -h option

pr: Add test for -d option
2021-03-26 14:11:14 +03:00
tilakpatidar
629236bd39 pr: add first test 2021-03-26 14:11:14 +03:00
Yagiz Degirmenci
545fe7d887
feat(unexpand): move from getopts to clap (#1883)
* feat: move unexpand to clap

* chore: allow muliple files

* test: add test fixture, test reading from a file

* test: fix typo on file name, add test for multiple inputs

* chore: use 'success()' instead of asserting

* chore: delete unused variables

* chore: use help instead of long_help, break long line
2021-03-23 09:42:05 +01:00
Neculai Balaban
c6927d97c8
cp: add support for -x/--one-file-system (#1840) 2021-03-19 21:15:35 +01:00
Sylvestre Ledru
fcccc2a973
Merge pull request #1826 from pfnsec/master
head: add support for -z/--zero-terminated
2021-03-18 21:37:12 +01:00
Peter Sherman
13e61c3234 head: add support for -z/--zero-terminated 2021-03-15 15:56:11 +00:00
Chirag Jadwani
116e253cc0 uniq: Fix skip fields
Current implementation of the skip fields logic does not handle
multibyte code points correctly. It assumes each code point (`char`) is
one byte. If the skipped part of the input line has any multibyte code
points then this can cause fields not being skipped correctly (field
start index is calculated to be before it actually starts).
2021-03-15 18:48:54 +05:30
Chirag Jadwani
ce4342d12e uniq: Fix panic on invalid utf-8 input 2021-03-15 18:48:54 +05:30
Michael Kefeder
e1626b8c64
ptx: ignore empty regex (#1808)
* ptx: ignore empty -W regex

* ptx: test to verify empty -W regex is ignored

* tests/ptx: verify word regexp from cmd-line is used
2021-03-12 21:52:37 +01:00
Daniel Rocco
0e02607dc7 numfmt: implement --field 2021-03-08 10:53:34 -05:00
Daniel Rocco
d06f91fbe2
numfmt: align format output values with GNU (#1745)
When converting to SI or IEC, produce values that align with the conventions
used by GNU numfmt.

- values > 10 are represented without a decimal place, so 10000 becomes 10K
  instead of 10.0K

- when truncating, take the ceiling of the value, so 100001 becomes 101K

- values < 10 are truncated to the highest tenth, so 1001 becomes 1.1K

closes #1726
2021-03-06 18:26:05 +01:00
Chad Brewbaker
05d8cc59c4
bug(wc): Add a test for unexpected behavior (#1723) 2021-02-16 13:36:49 +01:00
jaggededgedjustice
18c39daed7
fold: Handle input with newlines (#1680)
The read_line function appends to the given buffer which is never
cleared. This leads to lines being duplicated.
2021-01-05 09:10:01 +01:00
Stéphane Campinas
89bf7a726e csplit: refresh of the previous PR 2020-12-28 17:21:29 +01:00
Piyush Jaipuriayar
3ca8ba997c
hashsum: added blake2 as a hashing algorithm (#1651) 2020-12-08 23:32:02 +01:00
Sylvestre Ledru
601690b079 test(expand): Add some tests for expand (#1505) 2020-05-15 18:09:10 -05:00
Roy Ivy III
7b96432eff
Merge pull request #1504 from sylvestre/fmt
test(fmt): add some small initial tests
2020-05-10 12:25:18 -05:00
elgris
71ba8b3fd6 sort: add "dictionary-order" flag.
The flag makes 'sort' command ignore non-dictionary symbols
(non-alphanumeric and non-spaces). The only difference with GNU sort is
that it takes ALL alphanumeric symbols, not only ASCII ones.
2020-05-07 23:08:24 +02:00
Sylvestre Ledru
9b0129716f test(fmt): Add some tests for fmt
Even if it is a bit broken for now
2020-05-06 00:38:22 +02:00
Roy Ivy III
31655fc004 env: add support for new '--file' option (includes testing)
.# Discussion

This commit adds support for a '-f'/'--file' option which reads "KEY=VALUE" lines from
a config (or ini) style text file and sets the corresponding environment key. This is
modeled after the same option in the `dotenv` and `godotenv` commands. Notably, this
commit does *not* add automatic loading of ".env" configuration files.

The environment variables set by reading the configuration file are set prior to any
unset (eg, `-u BAR`) or set (eg, `FOO=bar`) actions. Files are loaded in order with
later files overwriting any overlapping environment variables, then, unset actions (in
command line order) are executed, then, finally, set actions (in command line order)
are executed.

[1] [`dotenv`](https://github.com/bkeepers/dotenv)
[2] [`godotenv`](https://github.com/joho/godotenv)
2019-04-29 09:50:22 -05:00
Julio Rincon
29c6ad5f6a tests: untrimmed stdout assertion (fix #1235) 2019-02-08 07:54:48 +11:00
xplorld
47f5f12759 sort: treat "NaN" as string in numeric sort 2018-09-03 22:28:18 -07:00
Konstantin Pospelov
e9fd9d623d join: implement option to suppress joined lines 2018-06-09 12:11:58 +03:00
Konstantin Pospelov
4b8d4bfc05 join: fix autoformat
There was an issue with autoformat when the files had a different
number of columns in the first line. This commit fixes the issue and
extends the related test to cover this case.
2018-04-15 17:42:52 +03:00
Konstantin Pospelov
7dc8ff62cc join: support headers 2018-04-11 22:55:44 +03:00
bootandy
ea504bf0ec du: add test for -d flag 2018-03-20 17:55:31 -04:00
bootandy
e253406026 du: Fix incorrect block size assumption.
du and other tools like stat assume a 512 byte block. ls is the only
tool to use 1024.

Add Simple set of tests
2018-03-20 16:59:29 -04:00
Konstantin Pospelov
128a38965e join: support custom empty filler 2018-01-17 23:26:00 +03:00