These tests try to read or write past a block device, where the block device is either given as
stdin or stdout. It requires access to the block device, and therefore is executed as root. For now,
it is assumed that a block device "/dev/sda1" with a size smaller than 10000000000000000 exists.
Add support for the `iflag=nocache` and `oflag=nocache` to make `dd`
discard the filesystem cache for the processed portion of the input or
output file.
Improve the display of the total time spent transferring bytes so that
the number of seconds is displayed using the `%g` format specifier as
in `printf`. This matches the behavior of GNU `dd`.
Before this commit, the precision was always set to one digit after
the decimal point. For example,
$ dd count=100000 if=/dev/zero of=/dev/null
100000+0 records in
100000+0 records out
51200000 bytes (51 MB, 49 MiB) copied, 0.2 s, 268.1 MB/s
After this commit, the precision increases dynamically as the duration
decreases. For example,
$ dd count=100000 if=/dev/zero of=/dev/null
100000+0 records in
100000+0 records out
51200000 bytes (51 MB, 49 MiB) copied, 0.1019 s, 507 MB/s
$ dd count=1000 if=/dev/zero of=/dev/null
1000+0 records in
1000+0 records out
512000 bytes (512 kB, 500 KiB) copied, 0.002663 s, 256 MB/s
$ dd count=10 if=/dev/zero of=/dev/null
10+0 records in
10+0 records out
5120 bytes (5.1 kB, 5.0 KiB) copied, 0.000182 s, 5.1 MB/s
Open stdin using its file descriptor so that a `dd skip=N` command in
a subshell does not consume all bytes from stdin.
For example, before this commit, multiple instances of `dd` reading
from stdin and appearing in a single command line would incorrectly
result in an empty stdin for each instance of `dd` after the first:
$ printf "abcdef\n" | (dd bs=1 skip=3 count=0 && dd) 2> /dev/null
# incorrectly results in no output
After this commit, the `dd skip=3` process reads three bytes from the
file descriptor referring to stdin without draining the remaining
three bytes when it terminates:
$ printf "abcdef\n" | (dd bs=1 skip=3 count=0 && dd) 2> /dev/null
def
Adjust the rendering of the concise byte counts in both SI and IEC
units to better match the behavior of GNU dd.
Before this commit,
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1 KB, 1024 B) copied, 0.0 s, 1.0 MB/s
After this commit,
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.0 s, 1.0 MB/s
For comparison, GNU dd produces the following:
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1.0 kB, 1.0 KiB) copied, 0.000332864 s, 3.1 MB/s
Before this commit, if `sparsefile` were a regular file of non-zero
size whose contents are all null bytes, then
dd if=sparsefile of=outfile conv=notrunc
would have resulted in `outfile` having zero size as reported by
`stat`. After this commit, `outfile` will have the same size as
`sparsefile` (even if the contents are represented sparsely by the
filesystem).
Move some tests that simulate a slow reader from `dd.rs` to
`tests/by-util/test_dd.rs`, and employ a FIFO and `sleep()` to
simulate the slow reader instead of a custom struct that implements
`Read`. This change restricts the type of `Input`s the
`Output::dd_out()` function can accept, facilitating a future change
to make `Input` an enum.
Allow uppercase "B" on its own as a unit specifier for the `count`,
`seek`, and `skip` arguments to `dd`.
For example,
$ printf "abcdef" | dd count=3B status=none
abc
Update `dd` to only print a concise form of the number of bytes with
an SI prefix (like "1 MB" or "2 GB") if the number is at least
1000. Similarly, only print the concise form with an IEC prefix (like
"1 MiB" or "2 GiB") if the number is at least 1024. For example,
$ head -c 999 /dev/zero | dd > /dev/null
1+1 records in
1+1 records out
999 bytes copied, 0.0 s, 999.0 KB/s
$ head -c 1000 /dev/zero | dd > /dev/null
1+1 records in
1+1 records out
1000 bytes (1000 B) copied, 0.0 s, 1000.0 KB/s
$ head -c 1024 /dev/zero | dd > /dev/null
2+0 records in
2+0 records out
1024 bytes (1 KB, 1024 B) copied, 0.0 s, 1.0 MB/s
These are the first half of changes needed to pass the dd/bytes.sh tests:
- Add iseek and oseek options (additive with skip and seek options)
- Implement tests for the new flags, matching those from dd/bytes.sh
https://github.com/uutils/coreutils/pull/3084 (2a333ab391) had some
missing coverage and was merged before I had a chance to fix it.
This PR adds some coverage / improved error messages that were missing
from that previous PR.
If `conv=block,sync` command-line arguments are given and there is at
least one partial record read from the input (for example, if the
length of the input is not divisible by the value of the `ibs`
argument), then output an extra block of `cbs` spaces.
For example, no extra spaces are printed in this example because the
input is of length 10, a multiple of `ibs`:
$ printf "012\nabcde\n" \
> | dd ibs=5 cbs=5 conv=block,sync status=noxfer \
> && echo $
012 abcde$
2+0 records in
0+1 records out
But in this example, 5 extra spaces are printed because the length of
the input is not a multiple of `ibs`:
$ printf "012\nabcdefg\n" \
> | dd ibs=5 cbs=5 conv=block,sync status=noxfer \
> && echo $
012 abcde $
2+1 records in
0+1 records out
1 truncated record
The number of spaces printed is the size of the conversion block,
given by `cbs`.
Prevent `dd` from terminating with an error when given the
command-line argument `of=/dev/null`. This commit allows the call to
`File::set_len()` to result in an error without causing the process to
terminate prematurely.
Place the "truncated records" line below the "records out" line in the
status report produced by `dd` and properly handle the singularization
of the word "record" in the case of 1 truncated record. This matches
the behavior of GNU `dd`.
For example
$ printf "ab" | dd cbs=1 conv=block status=noxfer > /dev/null
0+1 records in
0+1 records out
1 truncated record
$ printf "ab\ncd\n" | dd cbs=1 conv=block status=noxfer > /dev/null
0+1 records in
0+1 records out
2 truncated records
Clean up unit tests in the `dd` crate to make them easier to
manage. This commit does a few things.
* move test cases that test the complete functionality of the `dd`
program from the `dd_unit_tests` module up to the
`tests/by-util/test_dd.rs` module so that they can take advantage of
the testing framework and common testing tools provided by uutils,
* move test cases that test internal functions of the `dd`
implementation into the `tests` module within `dd.rs` so that they
live closer to the code they are testing,
* replace test cases defined by macros with test cases defined by
plain old functions to make the test cases easier to read at a
glance.
Correct the accounting for partial records written by `dd` to the
output file. After this commit, if fewer than `obs` bytes are written,
then that is counted as a partial record. For example,
$ printf 'abc' | dd bs=2 status=noxfer > /dev/null
1+1 records in
1+1 records out
That is, one complete record and one partial record are read from the
input, one complete record and one partial record are written to the
output. Previously, `dd` reported two complete records and zero
partial records written to the output in this case.
Show a warning if the `skip=N` command-line argument would cause `dd`
to skip past the end of the input. For example:
$ printf "abcd" | dd bs=1 skip=5 count=0 status=noxfer
'standard input': cannot skip to specified offset
0+0 records in
0+0 records out
Show a warning when a block size includes "0x" since this is
ambiguous: the user may have meant "multiply the next number by zero"
or they may have meant "the following characters should be interpreted
as a hexadecimal number".
When specifying `seek=N` and *not* specifying `conv=notrunc`, truncate
the output file to `N` blocks instead of truncating it to zero before
starting to write output. For example
$ printf "abc" > outfile
$ printf "123" | dd bs=1 skip=1 seek=1 count=1 status=noxfer of=outfile
1+0 records in
1+0 records out
$ cat outfile
a2
Fixes#3068.
