Before, the sort could work faster and we could be late with
the signal.
Now we create a new big file, `sort` can't process it in a minute,
so we can safely wait for the temporary directory to be created
and send a signal afterwards
Before the change it slept for 0.1 seconds and right after that
asserted if `sort` has created the directory. Sometimes `sort`
didn't manage to create the directory in 0.1 seconds.
So the change is it tries to wait for `timeout` starting with
0.1 seconds, and if directory was not found, it tries 4 more times,
each time increasing timeout twice. Once the directory is found
it breaks.
Make cp preserve the permissions of a directory when copying
it. Before this commit,
cp -pR src/ dest/
failed to copy the permissions of `src/` to `dest/`. After this
commit, the permissions are correctly copied.
While the rust coreutils semantics were arguably more correct,
they were different than the gnu split semantics when handling a
file without a trailing EOF. This patch addresses that difference
and allows passing one more GNU test suite.
Correct the error message produced when attempting to copy a directory
into itself with `cp`. Before this commit, the error message was
$ cp -R d d
cp: cannot copy a directory, 'd', into itself, 'd'
After this commit, the error message is
$ cp -R d d
cp: cannot copy a directory, 'd', into itself, 'd/d'
Allow `cp --remove-destination` to remove a symbolic link loop (or a
symbolic link that initiates a chain of too many symbolic
links). Before this commit, if `loop` were a symbolic link to itself,
then
cp --remove-destination file loop
would fail with an error message. After this commit, it succeeds. This
matches the behaviotr of GNU cp.
Generating the tests to run in build.rs created problems for tooling. For example, cargo fmt, was ignoring the test_*.rs files and needed to be passed these files manually to be formatted. Now we simply use the feature mechanism to decide which tests to run.
Add special handling in `mktemp` for when the directory that will
contain the temporary file is not found. This situation now produces
the message
mktemp: failed to create file via template 'XXX': No such file or directory
to match the behavior of GNU mktemp.
Update the usage message when too many template arguments are given on
the command line to match that of GNU mktemp:
mktemp: too many templates
Try 'mktemp --help' for more information.
This fixes the test case `too-many` in the GNU test suite file
`tests/misc/mktemp.pl`.
`cp` in interactive mode used to write to stdout asking for
overwrite. GNU version writes to stderr.
Changed: write to stderr to make compatible with GNU.
Change `mktemp` so that it respects the value of the `TMPDIR`
environment variable if no directory is otherwise specified in its
arguments. For example, before this commit
$ TMPDIR=. mktemp
/tmp/tmp.WDJ66MaS1T
After this commit,
$ TMPDIR=. mktemp
./tmp.h96VZBhv8P
This matches the behavior of GNU `mktemp`.
It seems that `chrono` is the reason of deadlock or UB in android
CI. Also `chrono` had some security issues and wasn't maintained for
two years until March 2022, so other unstabilities can happen. Plus
`chrono` uses old `time` dependency.
Previously, if stdin redirect pointed to a regular file,
tailing started at the beginning of the file. However,
tailing needs to start at the current position because this
is expected by tests/tail-2/start-middle.sh.
This fixes the issue by taking the current offset into account
while going backwards through the stdin redirected file.
This fixes a bug where calling `tail - < file.txt` would result
in invoking `unbounded_tail()`.
However, it is a stdin redirect to a seekable regular file and
therefore `bounded_tail` should be invoked as if `tail file.txt` had
been called.
The previous encoding handling was unnecessarily complex. This commit removes the enum that specifies the handling and instead has two separate methods to collect the strings either with lossy conversion or by ignoring invalidly encoded strings.
Outside of tests, only `accept_any` was used, meaning that this unnecessarily complicated the code. The behaviour of `accept_any` is now the default (and only) option.
* Fix a bug in split where chunking would be skipped when the chunk size
happened to be an exact divisor of the buffer size used to read the
input stream.
The issue here was that file was being split byte-wise in chunks of 1G.
The input stream was being read in chunks of 8KB, which evenly divides
the chunk size. Because the check to allocate the next output chunk was
done at the bottom of the loop previously, it would never occur because
the current input chunk was fully consumed at that point. By moving the
check to the top of the loop (but still late enough that we know we have
bytes to write) we resolve this issue.
This scenario is unfortunately hard to write a test for, since we don't
explicitly control the input chunk size.
Fixes https://github.com/uutils/coreutils/issues/3790
* test/cp: -p changes ctime and added sleep for better timestamp testing
* test/cp: add nanoseconds checking for copied timestamps
* test/cp: made compilable on other OSes
* test/cp: added error messages to assert_eq calls
* cp: Refactor `reflink`/`sparse` handling to enable `--sparse` flag
`--sparse` and `--reflink` options have a lot of similarities:
- They have similar options (`always`, `never`, `auto`)
- Both need OS specific handling
- They can be mutually exclusive
Prior to this change, `sparse` was defined as `CopyMode`, but `reflink`
wasn't. Given the similarities, it makes sense to handle them similarly.
The idea behind this change is to move all OS specific file copy
handling in the `copy_on_write_*` functions. Those function then
dispatch to the correct logic depending on the arguments (at the moment,
the tuple `(reflink, sparse)`).
Also, move the handling of `--reflink=never` from `copy_file` to the
`copy_on_write_*` functions, at the cost of a bit of code duplication,
to allow `copy_on_write_*` to handle all cases (and later handle
`--reflink=never` with `--sparse`).
* cp: Implement `--sparse` flag
This begins to address #3362
At the moment, only the `--sparse=always` logic matches the requirement
form GNU cp info page, i.e. always make holes in destination when
possible.
Sparse copy is done by copying the source to the destination block by
block (blocks being of the destination's fs block size). If the block
only holds NUL bytes, we don't write to the destination.
About `--sparse=auto`: according to GNU cp info page, the destination
file will be made sparse if the source file is sparse as well. The next
step are likely to use `lseek` with `SEEK_HOLE` detect if the source
file has holes. Currently, this has the same behaviour as
`--sparse=never`. This `SEEK_HOLE` logic can also be applied to
`--sparse=always` to improve performance when copying sparse files.
About `--sparse=never`: from my understanding, it is not guaranteed that
Rust's `fs::copy` will always produce a file with no holes, as
["platform-specific behavior may change in the
future"](https://doc.rust-lang.org/std/fs/fn.copy.html#platform-specific-behavior)
About other platforms:
- `macos`: The solution may be to use `fcntl` command `F_PUNCHHOLE`.
- `windows`: I only see `FSCTL_SET_SPARSE`.
This should pass the following GNU tests:
- `tests/cp/sparse.sh`
- `tests/cp/sparse-2.sh`
- `tests/cp/sparse-extents.sh`
- `tests/cp/sparse-extents-2.sh`
`sparse-perf.sh` needs `--sparse=auto`, and in particular a way to skip
holes in the source file.
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
* ls: Implement --zero flag. (#2929)
This flag can be used to provide a easy machine parseable output from
ls, as discussed in the GNU bug report
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49716.
There are some peculiarities with this flag:
- Current implementation of GNU ls of the `--zero` flag implies some
other flags. Those can be overridden by setting those flags after
`--zero` in the command line.
- This flag is not compatible with `--dired`. This patch is not 100%
compliant with GNU ls: GNU ls `--zero` will fail if `--dired` and
`-l` are set, while with this patch only `--dired` is needed for the
command to fail.
We also add `--dired` flag to the parser, with no additional behaviour
change.
Testing done:
```
$ bash util/build-gnu.sh
[...]
$ bash util/run-gnu-test.sh tests/ls/zero-option.sh
[...]
PASS: tests/ls/zero-option.sh
============================================================================
Testsuite summary for GNU coreutils 9.1.36-8ec11
============================================================================
# TOTAL: 1
# PASS: 1
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
============================================================================
```
* Use the US way to spell Behavior
* Fix formatting with cargo fmt -- tests/by-util/test_ls.rs
* Simplify --zero flag overriding logic by using index_of
Also, allow multiple --zero flags, as this is possible with GNU ls
command. Only the last one is taken into account.
Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
