Improve the display of the total time spent transferring bytes so that
the number of seconds is displayed using the `%g` format specifier as
in `printf`. This matches the behavior of GNU `dd`.
Before this commit, the precision was always set to one digit after
the decimal point. For example,
$ dd count=100000 if=/dev/zero of=/dev/null
100000+0 records in
100000+0 records out
51200000 bytes (51 MB, 49 MiB) copied, 0.2 s, 268.1 MB/s
After this commit, the precision increases dynamically as the duration
decreases. For example,
$ dd count=100000 if=/dev/zero of=/dev/null
100000+0 records in
100000+0 records out
51200000 bytes (51 MB, 49 MiB) copied, 0.1019 s, 507 MB/s
$ dd count=1000 if=/dev/zero of=/dev/null
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 0.002663 s, 256 MB/s
$ dd count=10 if=/dev/zero of=/dev/null
10+0 records in
10+0 records out
5120 bytes (5.1 kB, 5.0 KiB) copied, 0.000182 s, 5.1 MB/s
Open stdin using its file descriptor so that a `dd skip=N` command in
a subshell does not consume all bytes from stdin.
For example, before this commit, multiple instances of `dd` reading
from stdin and appearing in a single command line would incorrectly
result in an empty stdin for each instance of `dd` after the first:
$ printf "abcdef\n" | (dd bs=1 skip=3 count=0 && dd) 2> /dev/null
# incorrectly results in no output
After this commit, the `dd skip=3` process reads three bytes from the
file descriptor referring to stdin without draining the remaining
three bytes when it terminates:
$ printf "abcdef\n" | (dd bs=1 skip=3 count=0 && dd) 2> /dev/null
def
du --time formats a file's timestamp according to the local timezone,
but the test implicitly assumed UTC. This caused it to fail when
running locally in my UTC+1 machine.
$ cargo test --features "du touch" --no-default-features
Failure: https://gist.github.com/eggpi/651e01559b7c59e9457c1b22fe4c0c19
* Move the user and group resolution to the behavior decoding instead of
re-running on every file/directory creation. Simplifies code.
* Update error output to match GNU coreutils.
* Add tests to verify invalid owner and group.
Summary of changes in UCommand:
* Extend UCommand by builder methods and simplify methods in TestScenario
* Simplify code structures where possible. Add documentation.
* Store bin_path as PathBuf and util_name as String in all structs
* Remove UCommand::util and make bin_path, temp_dir private
* Rename UCommand::with_limit -> UCommand::limit
Summary of changes in TestScenario:
* Rename some parameters in TestScenario methods to be more descriptive
* Remove ucmd_keepenv, cmd_keepenv from TestScenario. Use UCommand::keep_env instead.
Summary of changes in tests/util:
* Introduce global constant TESTS_BINARY holding the path to `coreutils`
* Implement running an arbitrary command in a sh or cmd shell per default.
* Implement std::fmt::Display for UCommand.
* Change usages of UCommand::util_name from &Option<OsStr> to Option<OsStr>
* `UCommand::new_from_tmp`: Use OsStr directly instead of TempDir -> String -> OsStr
* Collect arguments in `UCommand::args` field
* Build environment variables in `UCommand` itself instead of `std::process::Command`
* Move building of std::process:Command from fields in UCommand to own method `build`
* Remove assertions of UCommand::has_run in arg, args and env.
Summary of changes in tests/by-util:
* Remove usages of UCommand::raw. Fix tests to use UCommand::to_string.
* test_test: Adjust test_invalid_utf8_integer_compare to use `UCommand::args`
* test_chmod: run_single_test
* test_pwd: symlinked_env
* test_cp:
Fix the usage of &Option<OsStr> in `test_src_base_dot`
Refactor test_src_base_dot to not use UCommand::new but ts.ucmd() instead
* tests/util: Fix documentation of UCommand::stderr_only and usage_error
* tests/util: Remove trimming from CmdResult::stdout_matches and stdout_does_not_match. Fix tests.
The tests are fixed to match the trailing newline instead of ignoring it.
* cp: require preserve only certain attributes
# Conflicts:
# src/uu/cp/src/copydir.rs
# src/uu/cp/src/cp.rs
* tests/cp: preserve all and preserve xattr tests with todos
* tests/cp: rename preserve tests
* tests/cp: add android fail test for preserve=xattr
On Android, this cp with explicit preserve of xattr must fail, because of the limitations of the filesystem setup used on Android.
* cp: verify some metadata in cp preserve tests
# Conflicts:
# tests/by-util/test_cp.rs
* cp: run test_cp_preserve_all in all OS's but only check metadata on linux
* test/cp: don't expect the mode to change in explicit cp preserve
* cp: attributes struct instead of enum for unified required tracking
* cp: refactor preserver and handle_preserve
# Conflicts:
# src/uu/cp/src/cp.rs
* cp: update preserve attr to max
* test/cp: fix the preserve xattr test
Access timestamps appear to be modified only in this test. Running the command directly does not alter the access timestamp.
* cp/test: preserve all and context case
* cp: fix preserve args value source
* test/cp: don't check mtime on freebsd
* test/cp: don't check mtime on macos
* test/cp: fix freebsd deps
* test/cp: support freebsd tests
* cp: simplify try_set_from_string
* cp: parse context attr in preserve in any case to show warning later
* cp: print warnings for attribute errors if not required
* cp: show SELinux warning only once
* cp: show SELinux warning without error
* Revert "cp: show SELinux warning without error"
This reverts commit d130cf0d8c8e28ac2c903413992613241decf879.
* cp: add documentation for preserve components
* cp: simplify try_set_from_string
* cp: EN_US "behavior" spelling for cspell
This introduces an argument preprocessing step for the nice tool in
order to support the legacy nice adjustment syntax (`nice -1 foo`,
`nice --1 foo`, `nice -+1 foo` and so forth).
This is a preprocessing step because the interaction between these
arguments and the existing `nice -n -1` syntax becomes context
dependent, in a way that is currently difficult to capture through
clap.
Signed-off-by: Ed Smith <ed.smith@collabora.com>
Other changes summmary:
* Use module level imports instead of fully qualified imports where appropriate
* Fix intermittent failing test_positive_zero_bytes because of broken pipe
* Cleanup old disabled test for warning of following indefinitely
* install: fix installing one file when using -Dt options
* install: fix installing multiple files with -Dt
Code was missing the logic to create the target dir when multiple files
should be copied and target dir is given by -t option.
This simplifies the copy logic also when only one file should be copied
to the target dir.
* install: fix verbose output when using -D
Also adds a unit test to verify the same behaviour as the gnu tools.
* install: add more testcases for create leading dir
Tests various combinations of "-D" with and w/o "-t" when installing
either a single file or multiple files into a non existing directory.
install -D file1 file2 (-t) not_existing_dir
install -D file1 (-t) not_existing_dir/
Also fixes file formatting, spelling and adds some more test asserts.
* install: fix error for nonex. dir path ending on /
The install command failed with a different error message than the
original GNU install tool. Checking for a trailing slash fixes this.
Only works on unix though.
* install: add windows support when checking for '/'
* install.rs: fix spelling
* install.rs: add more tests regarding omitting dir
This increases the CI test coverage and also checks for more corner
cases to ensure uu_install is compliant with GNU's original.
export C=coreutils/target/debug/
rm -rf dir1 no-dir2 dir3 file
mkdir -p dir1 dir3
touch file
${C}install dir1/file1 dir1/.. no-dir2
${C}install dir1/file1 dir1/.. dir3
${C}install dir1/.. dir3
* install: improve test_install_missing_arguments
Also check that install returns the correct error messages, when only a
target directory is given via -t and that is is not created (-D option).
* install: rework the checks for missing file args
This ensures correct (GNU install like) behavior. Tests from the last
commit will pass now.
This commit corrects the behavior of `cp -r --parents --verbose` when
the source path is a directory, so that it prints the copied ancestor
directories. For example,
$ mkdir -p a/b/c d
$ cp -r --verbose --parents a/b/c d
a -> d/a
a/b -> d/a/b
'a/b/c' -> 'd/a/b/c'
This commit corrects the behavior of `cp --parents --verbose` when the
source path is a file so that it prints the copied ancestor
directories. For example,
$ mkdir -p a/b d
$ touch a/b/c
$ cp --verbose --parents a/b/c d
a -> d/a
a/b -> d/a/b
'a/b/c' -> 'd/a/b/c'
Fixes#3332.
Adjust the rendering of the concise byte counts in both SI and IEC
units to better match the behavior of GNU dd.
Before this commit,
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1 KB, 1024 B) copied, 0.0 s, 1.0 MB/s
After this commit,
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0 s, 1.0 MB/s
For comparison, GNU dd produces the following:
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000332864 s, 3.1 MB/s
Prevent a panic in `cp -a` when the target of a hard link already
exists in the target directory structure.
For example,
$ mkdir -p src dest/src
$ touch src/f dest/src/f
$ ln src/f src/link
$ cp -a src dest
The `cp` command now succeeds without error.
Summary:
* Disable test_retry6 on android because of intermittent failures
* Use wait() instead of wait_with_output in test_cat, test_cp, test_sort
* tests/sort: Simplify usage of test_sigpipe_panic
* Fix tests in test_tee.
tests/tac:
There was a change in the `tests/common/util.rs` test api concerning piped input which may have
revealed a bug in the implementation of tac. Please see also
https://github.com/uutils/coreutils/pull/4136
A short summary of changes:
* Add some basic tests for UChild and the run methods.
* Try more often in a fixed interval to create the tempfile for CapturedOutput.
* Fix drop order of struct fields for better cleanup of temporary files/dirs.
* Mark UChild::wait_with_output and UChild::pipe_in_and_wait_with_output deprecated
* Make CapturedOutput private
* Panic in stdout_all, stdout_all_bytes etc. if output is not captured.
* Rename some methods, refactor, clean up, fix documentation, add try_... methods
Before this commit, if `sparsefile` were a regular file of non-zero
size whose contents are all null bytes, then
dd if=sparsefile of=outfile conv=notrunc
would have resulted in `outfile` having zero size as reported by
`stat`. After this commit, `outfile` will have the same size as
`sparsefile` (even if the contents are represented sparsely by the
filesystem).
Move some tests that simulate a slow reader from `dd.rs` to
`tests/by-util/test_dd.rs`, and employ a FIFO and `sleep()` to
simulate the slow reader instead of a custom struct that implements
`Read`. This change restricts the type of `Input`s the
`Output::dd_out()` function can accept, facilitating a future change
to make `Input` an enum.
Allow uppercase "B" on its own as a unit specifier for the `count`,
`seek`, and `skip` arguments to `dd`.
For example,
$ printf "abcdef" | dd count=3B status=none
abc
Correct the behavior of `cp --reflink=never --sparse=always` so that
it performs a sparse copy. Before this commit, it was incorrectly
performing a dense copy.
Implement the `--copy-contents` option when the source is a FIFO, so
that the contents of the FIFO are copied (when the bytes become
available for reading) instead of the FIFO object itself. For example,
$ mkfifo fifo
$ cp --copy-contents fifo outfile &
[1] 1614080
$ echo foo > fifo
$ cat outfile
foo
[1]+ Done cp --copy-contents fifo outfile
Fix the behavior of `cp` when both `--backup` and `--force` are
specified and the source and destination are the same file. Before
this commit, `cp` terminated without copying and without making a
backup. After this commit, the copy is made and the backup file is
made. For example,
$ touch f
$ cp --force --backup f f
results in a backup file `f~` being created.
If the arg starts with an id numeric value, the group isn't set but the separator is provided,
we should fail with an error
Should fix tests/chown/separator.sh
Change `cp` to terminate with an error when attempting to copy through
a dangling symbolic link with the `--force` option. Before this
commit,
touch src
ln -s no-such-file dest
cp -f src dest
would incorrectly replace `dest` with the contents of `src`. After
this commit, it correctly fails with the error message
cp: not writing through dangling symlink 'dest'
Fix a bug where `cp` failed to copy ancestor directories when using
the `--parents` option. For example, before this commit:
$ mkdir -p a/b/c d
$ cp --parents a/b/c d
$ find d
d
d/c
After this commit
$ mkdir -p a/b/c d
$ cp --parents a/b/c d
$ find d
d
d/a
d/a/b
d/a/b/c
This commit also adds the correct messages for `--verbose` mode:
$ cp -r --parents --verbose a/b/c d
a -> d/a
a/b -> d/a/b
'a/b/c' -> 'd/a/b/c'
Fixes#3332.
Implement distributing lines of a file in a round-robin manner to a
specified number of chunks. For example,
$ (seq 1 10 | split -n r/3) && head -v xa[abc]
==> xaa <==
1
4
7
10
==> xab <==
2
5
8
==> xac <==
3
6
9
* hashsum: test b3sum::test_nonames for real
Signed-off-by: Huijeong Kim <herehuijeong@gmail.com>
* apply cargo format
Signed-off-by: Huijeong Kim <herehuijeong@gmail.com>
Before, the sort could work faster and we could be late with
the signal.
Now we create a new big file, `sort` can't process it in a minute,
so we can safely wait for the temporary directory to be created
and send a signal afterwards
Before the change it slept for 0.1 seconds and right after that
asserted if `sort` has created the directory. Sometimes `sort`
didn't manage to create the directory in 0.1 seconds.
So the change is it tries to wait for `timeout` starting with
0.1 seconds, and if directory was not found, it tries 4 more times,
each time increasing timeout twice. Once the directory is found
it breaks.
