* Changed dd parsing error message to be in line with GNU dd
* Correct logic to make dd incorrect number error message fully compatible with GNU. Add test
---------
Co-authored-by: just-an-engineer <Julian.Beltz@zetier.com>
If the first four decimal digits are zero, GNU dd elides them altogether.
Therefore, this test just contained an overly-strict regex.
See also ede944e1f8.
* dd: check file is a dir for iflag directory
* Improve english
Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>
* dd: stderr output checking for "iflag directory" testcase
* dd: replace #[cfg(unix)] with #[test]
---------
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
Co-authored-by: Daniel Hofstetter <daniel.hofstetter@42dh.com>
If the first four decimal digits are zero, GNU dd elides them altogether.
Here's an execution on my PC:
```console
$ for i in $(seq 20000); do LC_ALL=C gnu_dd if=/dev/null of=/dev/null \
2>&1; done | grep copied | grep -E ' [0-9]e'
0 bytes copied, 1e-05 s, 0 B/s
0 bytes copied, 9e-06 s, 0.0 kB/s
```
Our implementation conforms to this, resulting in the following CI flake:
```
---- test_dd::test_final_stats_unspec stdout ----
run: D:\a\coreutils\coreutils\target\x86_64-pc-windows-gnu\debug\coreutils.exe dd
thread 'test_dd::test_final_stats_unspec' panicked at 'Stderr does not match regex:
0+0 records in
0+0 records out
0 bytes copied, 8e-05 s, 0.0 B/s
', tests\by-util\test_dd.rs:280:10
stack backtrace:
0: rust_begin_unwind
at /rustc/90c541806f23a127002de5b4038be731ba1458ca/library\std\src/panicking.rs:578:5
```
Of course, this is just an overly strict regex in the test. This was a
one-in-tenthousand flaky test.
On *BSD and macOS, the system provided dd utility opens up the output
file for both reading and writing. This means that the open/write to the
FIFO does not block, and almost instantly completes. The system dd then
exits, leaving nothing left to be read by the time the coreutils-rs dd
tries to open/read the FIFO.
Avoid this problem by just writing to the FIFO from the test case
itself, rather than relying on the system provide dd.
These tests try to read or write past a block device, where the block device is either given as
stdin or stdout. It requires access to the block device, and therefore is executed as root. For now,
it is assumed that a block device "/dev/sda1" with a size smaller than 10000000000000000 exists.
Add support for the `iflag=nocache` and `oflag=nocache` to make `dd`
discard the filesystem cache for the processed portion of the input or
output file.
Improve the display of the total time spent transferring bytes so that
the number of seconds is displayed using the `%g` format specifier as
in `printf`. This matches the behavior of GNU `dd`.
Before this commit, the precision was always set to one digit after
the decimal point. For example,
$ dd count=100000 if=/dev/zero of=/dev/null
100000+0 records in
100000+0 records out
51200000 bytes (51 MB, 49 MiB) copied, 0.2 s, 268.1 MB/s
After this commit, the precision increases dynamically as the duration
decreases. For example,
$ dd count=100000 if=/dev/zero of=/dev/null
100000+0 records in
100000+0 records out
51200000 bytes (51 MB, 49 MiB) copied, 0.1019 s, 507 MB/s
$ dd count=1000 if=/dev/zero of=/dev/null
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 0.002663 s, 256 MB/s
$ dd count=10 if=/dev/zero of=/dev/null
10+0 records in
10+0 records out
5120 bytes (5.1 kB, 5.0 KiB) copied, 0.000182 s, 5.1 MB/s
Open stdin using its file descriptor so that a `dd skip=N` command in
a subshell does not consume all bytes from stdin.
For example, before this commit, multiple instances of `dd` reading
from stdin and appearing in a single command line would incorrectly
result in an empty stdin for each instance of `dd` after the first:
$ printf "abcdef\n" | (dd bs=1 skip=3 count=0 && dd) 2> /dev/null
# incorrectly results in no output
After this commit, the `dd skip=3` process reads three bytes from the
file descriptor referring to stdin without draining the remaining
three bytes when it terminates:
$ printf "abcdef\n" | (dd bs=1 skip=3 count=0 && dd) 2> /dev/null
def
Adjust the rendering of the concise byte counts in both SI and IEC
units to better match the behavior of GNU dd.
Before this commit,
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1 KB, 1024 B) copied, 0.0 s, 1.0 MB/s
After this commit,
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0 s, 1.0 MB/s
For comparison, GNU dd produces the following:
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000332864 s, 3.1 MB/s
Before this commit, if `sparsefile` were a regular file of non-zero
size whose contents are all null bytes, then
dd if=sparsefile of=outfile conv=notrunc
would have resulted in `outfile` having zero size as reported by
`stat`. After this commit, `outfile` will have the same size as
`sparsefile` (even if the contents are represented sparsely by the
filesystem).
Move some tests that simulate a slow reader from `dd.rs` to
`tests/by-util/test_dd.rs`, and employ a FIFO and `sleep()` to
simulate the slow reader instead of a custom struct that implements
`Read`. This change restricts the type of `Input`s the
`Output::dd_out()` function can accept, facilitating a future change
to make `Input` an enum.
Allow uppercase "B" on its own as a unit specifier for the `count`,
`seek`, and `skip` arguments to `dd`.
For example,
$ printf "abcdef" | dd count=3B status=none
abc
Update `dd` to only print a concise form of the number of bytes with
an SI prefix (like "1 MB" or "2 GB") if the number is at least
1000. Similarly, only print the concise form with an IEC prefix (like
"1 MiB" or "2 GiB") if the number is at least 1024. For example,
$ head -c 999 /dev/zero | dd > /dev/null
1+1 records in
1+1 records out
999 bytes copied, 0.0 s, 999.0 KB/s
$ head -c 1000 /dev/zero | dd > /dev/null
1+1 records in
1+1 records out
1000 bytes (1000 B) copied, 0.0 s, 1000.0 KB/s
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1 KB, 1024 B) copied, 0.0 s, 1.0 MB/s
These are the first half of changes needed to pass the dd/bytes.sh tests:
- Add iseek and oseek options (additive with skip and seek options)
- Implement tests for the new flags, matching those from dd/bytes.sh
https://github.com/uutils/coreutils/pull/3084 (2a333ab391) had some
missing coverage and was merged before I had a chance to fix it.
This PR adds some coverage / improved error messages that were missing
from that previous PR.
