Commit graph

7623 commits

Author SHA1 Message Date
Eli Youngs
6e0fedc277 Fix panic when canonicalizing a nonexistent path 2022-02-08 20:19:13 -08:00
Jeffrey Finkelstein
b37718de10 split: add BENCHMARKING.md documentation file 2022-02-08 22:58:00 -05:00
Jeffrey Finkelstein
70ca1f45ea split: remove unused ByteSplitter and LineSplitter 2022-02-08 22:58:00 -05:00
Jeffrey Finkelstein
1d7e1b8732 split: use ByteChunkWriter and LineChunkWriter
Replace `ByteSplitter` and `LineSplitter` with `ByteChunkWriter` and
`LineChunkWriter` respectively. This results in a more maintainable
design and an increase in the speed of splitting by lines.
2022-02-08 22:57:57 -05:00
Jeffrey Finkelstein
ca7af808d5 tests: correct a test case for split
Correct the `test_split::test_suffixes_exhausted` test case so that it
actually exercises the intended behavior of `split`. Previously, the
test fixture contained 26 bytes. After this commit, the test fixture
contains 27 bytes. When using a suffix width of one, only 26 filenames
should be available when naming chunk files---one for each lowercase
ASCII letter. This commit ensures that the filenames will be exhausted
as intended by the test.
2022-02-08 22:53:57 -05:00
Jeffrey Finkelstein
b31d63eaa9 split: add ByteChunkWriter and LineChunkWriter
Add the `ByteChunkWriter` and `LineChunkWriter` structs and
implementations, but don't use them yet. This structs offer an
alternative approach to writing chunks of output (contrasted with
`ByteSplitter` and `LineSplitter`). The main difference is that
control of which underlying file is being written is inside the writer
instead of outside.
2022-02-08 22:53:56 -05:00
Terts Diepraam
dc24c9563e shuf: BENCHMARKING.md 2022-02-08 21:05:39 +01:00
Sylvestre Ledru
47b12b31a6
Merge pull request #3006 from jfinkels/split-errors
split: correct error message on invalid arg. to -a
2022-02-08 20:51:32 +01:00
Sylvestre Ledru
daaae90113
Merge branch 'main' into dd-skip-beyond-file 2022-02-08 20:46:07 +01:00
Sylvestre Ledru
936ac0db38
Merge pull request #3053 from snobee/stat-handle-negative-time
stat: allow formatting of negative numbers
2022-02-08 20:45:21 +01:00
Sylvestre Ledru
12d5139320
Merge pull request #3095 from jfinkels/dd-concise-main-loop
dd: make main loop more concise
2022-02-08 20:44:39 +01:00
Sylvestre Ledru
42d3c5f76e
Merge pull request #3100 from daissi/wip/build_dd
Add dd in GNUmakefile
2022-02-08 20:44:02 +01:00
Dylan Aïssi
2b59b011f6 dd - add dd in GNUmakefile 2022-02-08 15:50:09 +01:00
Terts Diepraam
9538814702 shuf: use split_once for parsing the range 2022-02-08 14:39:32 +01:00
Terts Diepraam
30ae952b83 shuf: remove custom randomization logic 2022-02-08 14:39:32 +01:00
Sylvestre Ledru
9c813be5f1
Merge pull request #3076 from jfinkels/dd-seek-truncate
dd: truncate to specified seek length
2022-02-08 11:16:15 +01:00
Justin Tracey
b873d46ca0 join: flush stdout before final error message 2022-02-07 22:32:02 -05:00
Justin Tracey
bf67c5d981 join: add tests for --check-order and stdout error 2022-02-07 22:26:27 -05:00
Narasimha Prasanna HN
0379fca260 Fix merge conflict in tests/by-util/test_cp.rs 2022-02-07 23:26:06 +05:30
Jeffrey Finkelstein
c002b16c67 dd: make main loop more concise
Add some helper functions and adjust some error-handling to make the
`Output::dd_out()` method, containing the main loop of the `dd`
program, more concise. This commit also adds documentation and
comments describing the main loop procedure in more detail.
2022-02-07 10:00:49 -05:00
Allan Silva
e6c94c1cd7 wc: Fix clippy error 2022-02-07 10:20:52 -03:00
Sylvestre Ledru
b14e396983
Merge pull request #3091 from jfinkels/df-flatten-filter-mount-list
df: refactor filter_mount_list() to be more flat
2022-02-07 10:59:23 +01:00
Sylvestre Ledru
4d07083bac
Merge pull request #3085 from jfinkels/dd-zero-multiplier
dd: show warning when using 0x size multiplier
2022-02-07 08:07:06 +01:00
Justin Tracey
41c90d79c4 join: add benchmarking documentation 2022-02-06 23:53:03 -05:00
Justin Tracey
ac9d006886 join: guess the number of fields in each line
This lets us use fewer reallocations when parsing each line.
The current guess is set to the maximum fields in a line so far. This is
a free performance win in the common case where each line has the same
number of fields, but comes with some memory overhead in the case where
there is a line with lots of fields at the beginning of the file, and
fewer later, but each of these lines are typically not kept for very
long anyway.
2022-02-06 23:37:17 -05:00
Justin Tracey
f33e058a5a join: faster field parsing and representation
Using indexes into the line instead of Vec<u8>s means we don't have to copy
the line to store the fields (indexes instead of slices because it avoids
self-referential structs). Using memchr also empirically saves a lot of
intermediate allocations.
2022-02-06 23:37:17 -05:00
Justin Tracey
e6f59b12f7 join: lock and buffer stdout
By abstracting the writer we write to, we can lock stdout once at the
beginning, then use buffered writes to it throughout.
2022-02-06 23:37:17 -05:00
Justin Tracey
c12f393150 join: improve error handling 2022-02-06 23:37:06 -05:00
Jeffrey Finkelstein
9528d514bf df: refactor data table into Row, Header structs
Refactor the code for representing the `df` data table into `Header`
and `Row` structs. These structs live in a new module `table.rs`. When
combined with the `Options` struct, these structs can be
`Display`ed. Organizing the code this way makes it possible to test
the display settings independently of the machinery for getting the
filesystem data. New unit tests have been added to `table.rs` to
demonstrate this benefit.
2022-02-06 21:54:58 -05:00
Jeffrey Finkelstein
44772a8dbb uucore: set meaningless FsUsage.ffree value to 0
Set the value of the `FsUsage.ffree` value to 0 on Windows, because
even though it is meaningless, it should not exceed the
`FsUsage.files` value so that client code can rely on the guarantee
that `FsUsage.ffree <= FsUsage.files`.
2022-02-06 21:54:58 -05:00
Jeffrey Finkelstein
9f8ec676c5 dd: show warning if skipping past end of input
Show a warning if the `skip=N` command-line argument would cause `dd`
to skip past the end of the input. For example:

    $ printf "abcd" | dd bs=1 skip=5 count=0 status=noxfer
    'standard input': cannot skip to specified offset
    0+0 records in
    0+0 records out
2022-02-06 20:10:42 -05:00
Jeffrey Finkelstein
84d4f24b8c dd: avoid infinite loop in Input::force_fill()
Avoid an infinite loop in `Input::force_fill()` when the input has
fewer bytes than are being requested to be read from the input.
2022-02-06 20:10:41 -05:00
Jeffrey Finkelstein
8fa6797255 split: add structure to errors that can be created
Add some structure to errors that can be created during parsing of
settings from command-line options. This commit creates
`StrategyError` and `SettingsError` enumerations to represent the
various parsing and other errors that can arise when transforming
`ArgMatches` into `Settings`.
2022-02-06 20:09:29 -05:00
Jeffrey Finkelstein
e5361a8c11 split: correct error message on invalid arg. to -a
Correct the error message displayed on an invalid parameter to the
`--suffix-length` or `-a` command-line option.
2022-02-06 20:09:29 -05:00
Jeffrey Finkelstein
572b2e032c df: refactor filter_mount_list() to be more flat
Use a `for` loop in the `filter_mount_list()` function to make the
filtering logic easier to read.
2022-02-06 20:04:03 -05:00
Jeffrey Finkelstein
e6a63a78f6 tests: fix no_stderr check in stderr_only_bytes()
Fix a bug in `stderr_only_bytes()` where it was unintentionally
checking `no_stderr()` when it should have been checking
`no_stdout()`.
2022-02-06 18:03:13 -05:00
ndd7xv
3842ecb1b4 dd: status=progress rewrites once/sec 2022-02-06 17:56:08 -05:00
Jeffrey Finkelstein
fec662a623 dd: show warning when using 0x size multiplier
Show a warning when a block size includes "0x" since this is
ambiguous: the user may have meant "multiply the next number by zero"
or they may have meant "the following characters should be interpreted
as a hexadecimal number".
2022-02-06 17:13:15 -05:00
Sylvestre Ledru
1f7c08d87b
Merge pull request #3028 from jfinkels/dd-multipliers
dd: add support for 'b' and 'x' multipliers in numeric arguments
2022-02-06 22:03:54 +01:00
Sylvestre Ledru
16b397a648
Merge pull request #3029 from hbina/hbina-test_ls_order_time-is-flaky
test_ls: Do not rely on the system time of metadata'access time
2022-02-06 22:03:31 +01:00
Sylvestre Ledru
1ac45c9961
Merge pull request #3044 from water-ghosts/cp-dir-vs-file
cp: Fail when copying a directory to a file
2022-02-06 22:01:37 +01:00
Andreas Molzer
a2e9329918 seq: Allow option to receive immediate arguments
WIP: this needs to be adjusted
2022-02-06 09:45:38 -06:00
Andreas Molzer
66733ca994 seq: Add difficult cases to test suite 2022-02-06 09:45:37 -06:00
Sylvestre Ledru
f27f827751
Merge pull request #3077 from rivy/fix.gnutests
maint/CICD ~ implement 'GnuTests' workflow fixes/refactor
2022-02-06 15:34:03 +01:00
Hanif Bin Ariffin
9ce9a44052 Silencing clippy about unused variables
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.43262@gmail.com>
2022-02-06 10:26:01 +08:00
Hanif Bin Ariffin
ff1d1c3b19 Merge branch 'main' of github.com:uutils/coreutils into hbina-ls-propagate-write-errors 2022-02-06 10:25:17 +08:00
Sylvestre Ledru
57cf3a533f
Merge pull request #3067 from ndd7xv/printf-clap-defaults
printf: use clap default help and version
2022-02-05 22:16:44 +01:00
Jeffrey Finkelstein
1af709f642 dd: truncate to specified seek length
When specifying `seek=N` and *not* specifying `conv=notrunc`, truncate
the output file to `N` blocks instead of truncating it to zero before
starting to write output. For example

    $ printf "abc" > outfile
    $ printf "123" | dd bs=1 skip=1 seek=1 count=1 status=noxfer of=outfile
    1+0 records in
    1+0 records out
    $ cat outfile
    a2

Fixes #3068.
2022-02-05 16:07:44 -05:00
Roy Ivy III
578e5c8aba maint/CICD ~ implement 'GnuTests' workflow fixes/refactor
- consolidate configuration
- DRY improvements
- improve flexibility/robustness in the face of missing reference test info
- add reference test info IDs and additional logging to help diagnose testing failures
- includes parallel refactor of 'util/run-gnu-test.sh'
2022-02-05 14:21:15 -06:00
Roy Ivy III
cc61ea807e docs/CICD ~ add spell-checker exceptions 2022-02-05 14:21:15 -06:00