coreutils/tests/by-util
Pierre Marsais e1991525af
cp: Implement --sparse flag (#3766)
* cp: Refactor `reflink`/`sparse` handling to enable `--sparse` flag

`--sparse` and `--reflink` options have a lot of similarities:
 - They have similar options (`always`, `never`, `auto`)
 - Both need OS specific handling
 - They can be mutually exclusive

Prior to this change, `sparse` was defined as `CopyMode`, but `reflink`
wasn't. Given the similarities, it makes sense to handle them similarly.

The idea behind this change is to move all OS specific file copy
handling in the `copy_on_write_*` functions. Those function then
dispatch to the correct logic depending on the arguments (at the moment,
the tuple `(reflink, sparse)`).

Also, move the handling of `--reflink=never` from `copy_file` to the
`copy_on_write_*` functions, at the cost of a bit of code duplication,
to allow `copy_on_write_*` to handle all cases (and later handle
`--reflink=never` with `--sparse`).

* cp: Implement `--sparse` flag

This begins to address #3362

At the moment, only the `--sparse=always` logic matches the requirement
form GNU cp info page, i.e. always make holes in destination when
possible.

Sparse copy is done by copying the source to the destination block by
block (blocks being of the destination's fs block size). If the block
only holds NUL bytes, we don't write to the destination.

About `--sparse=auto`: according to GNU cp info page, the destination
file will be made sparse if the source file is sparse as well. The next
step are likely to use `lseek` with `SEEK_HOLE` detect if the source
file has holes. Currently, this has the same behaviour as
`--sparse=never`. This `SEEK_HOLE` logic can also be applied to
`--sparse=always` to improve performance when copying sparse files.

About `--sparse=never`: from my understanding, it is not guaranteed that
Rust's `fs::copy` will always produce a file with no holes, as
["platform-specific behavior may change in the
future"](https://doc.rust-lang.org/std/fs/fn.copy.html#platform-specific-behavior)

About other platforms:
 - `macos`: The solution may be to use `fcntl` command `F_PUNCHHOLE`.
 - `windows`: I only see `FSCTL_SET_SPARSE`.

This should pass the following GNU tests:
 - `tests/cp/sparse.sh`
 - `tests/cp/sparse-2.sh`
 - `tests/cp/sparse-extents.sh`
 - `tests/cp/sparse-extents-2.sh`

`sparse-perf.sh` needs `--sparse=auto`, and in particular a way to skip
holes in the source file.

Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
2022-08-04 13:22:59 +02:00
..
test_arch.rs Fix some tests to not use CmdResult fields 2021-04-10 23:30:50 +03:00
test_base32.rs bump clap from 3.1.18 to 3.2.15 2022-07-29 14:05:02 +02:00
test_base64.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_basename.rs Basename arguments simple format (#3736) 2022-07-22 13:28:54 +02:00
test_basenc.rs basenc: add utility 2021-08-06 00:07:19 +02:00
test_cat.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_chcon.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_chgrp.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_chmod.rs Replace lazy_static with once_cell 2022-07-12 14:08:30 +01:00
test_chown.rs fixup! chown: allow setting arbitrary numeric user ID 2022-05-12 08:41:17 +02:00
test_chroot.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_cksum.rs cksum: accept directories as empty files 2021-12-26 21:50:13 +01:00
test_comm.rs comm: enable "output_delimiter" test 2022-07-07 15:22:43 +02:00
test_cp.rs cp: Implement --sparse flag (#3766) 2022-08-04 13:22:59 +02:00
test_csplit.rs tests ~ refactor/polish spelling (comments, names, and exceptions) 2021-05-31 08:23:57 -05:00
test_cut.rs cut: fix argument parsing for the delimiter 2022-06-09 22:14:43 +02:00
test_date.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_dd.rs dd: only print concise byte counts if large enough 2022-06-11 23:16:16 -04:00
test_df.rs Adds a test for sync. 2022-06-05 09:43:59 +02:00
test_dir.rs dir, vdir: fix incorrect regex in tests 2022-05-01 13:48:51 +02:00
test_dircolors.rs dircolors: escape "'" and ":" 2022-06-15 15:22:14 +02:00
test_dirname.rs tr/dirname: fix clap short_alias 2021-04-30 20:19:43 +02:00
test_du.rs uucore: add InvalidSuffix to ParseSizeError 2022-05-17 15:37:49 +02:00
test_echo.rs Allow echo with escapes to work with \0 2022-01-30 17:24:29 -05:00
test_env.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_expand.rs expand: add support for "--tabs" shortcuts 2022-06-10 13:55:27 +02:00
test_expr.rs expr: Use chars().count() as we can have some multibytes chars 2022-02-13 14:19:25 +01:00
test_factor.rs update to sha 0.10.0 (#3110) 2022-02-12 18:12:02 +01:00
test_false.rs false,true: Implement custom help, version 2022-02-01 14:29:26 +01:00
test_fmt.rs tests: delete 'error:' prefix from the tests 2021-05-26 02:45:53 +03:00
test_fold.rs refactor ~ cargo make format 2021-05-31 08:23:58 -05:00
test_groups.rs tests: fix tests for CICD environment 2021-07-08 11:17:18 +02:00
test_hashsum.rs hashsum: add --no-names option from official b3sum tool (#3361) 2022-04-06 09:09:37 +02:00
test_head.rs head, tail: include presume-input-pipe parameter 2022-04-01 11:55:33 +02:00
test_hostid.rs Refactored tests for more utilities 2021-04-18 02:32:43 +03:00
test_hostname.rs silent the failed tests for now on freebsd 2021-08-24 09:32:27 +02:00
test_id.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_install.rs Merge pull request #3663 from niyaznigmatullin/fix_install_C_test 2022-06-23 17:09:02 +02:00
test_join.rs join: flush stdout before final error message 2022-02-07 22:32:02 -05:00
test_kill.rs use 'Self' and derive 'Default' where possible 2022-01-30 15:08:26 +01:00
test_link.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_ln.rs tests/ln: fix tests according to new messages 2022-07-19 17:34:52 +03:00
test_logname.rs tests: delete 'error:' prefix from the tests 2021-05-26 02:45:53 +03:00
test_ls.rs ls: Implement --zero flag. (#2929) (#3746) 2022-07-31 19:02:50 +02:00
test_mkdir.rs Fix clippy 2022-04-02 09:54:11 +02:00
test_mkfifo.rs tests: delete 'error:' prefix from the tests 2021-05-26 02:45:53 +03:00
test_mknod.rs fix clippy errors related to clap upgrade from 3.0.10 to 3.1.6 2022-03-17 22:46:56 +01:00
test_mktemp.rs mktemp: respect POSIXLY_CORRECT env var when parsing args 2022-06-09 20:51:51 +08:00
test_more.rs tests: use CmdResult::usage_error 2021-11-09 17:37:05 -03:00
test_mv.rs mv: allow a single source with --target-directory 2022-05-14 03:00:29 +00:00
test_nice.rs nice: set exit code for clap errors to 125 2022-05-04 21:32:14 +02:00
test_nl.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_nohup.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_nproc.rs nproc: make tests/misc/nproc-override.sh pass by implementing OMP_NUM_THREADS=X,Y,Z (#3296) 2022-03-23 12:12:54 +01:00
test_numfmt.rs numfmt: preserve trailing zeros 2022-08-01 09:48:13 +02:00
test_od.rs uucore: add InvalidSuffix to ParseSizeError 2022-05-17 15:37:49 +02:00
test_paste.rs paste: implement "-z" flag 2022-06-22 09:30:00 +02:00
test_pathchk.rs tests ~ fix cargo clippy complaint (clippy::needless_borrow) 2021-06-06 19:28:25 -05:00
test_pinky.rs fix null pointer derefs 2022-04-20 08:44:49 +02:00
test_pr.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_printenv.rs use assert_eq instead of x == y. Better output in case of error 2022-03-23 10:41:31 +01:00
test_printf.rs printf: Default left-justify integer conversion to 1 width 2022-05-01 20:00:09 +02:00
test_ptx.rs tests/ptx: added breakfile option tests 2022-04-29 10:15:06 +02:00
test_pwd.rs tests/mkdir: reduced tested permission combinations 2022-03-31 15:50:52 +09:00
test_readlink.rs readlink: symlink loop handling (#3717) 2022-07-14 22:32:55 +02:00
test_realpath.rs Realpath relative options (#3710) 2022-07-12 08:29:20 +02:00
test_relpath.rs fix Rust 1.59 clippy lints 2022-02-25 09:39:48 +01:00
test_rm.rs tests/rm: add test for issue #3722 2022-07-17 23:34:04 +03:00
test_rmdir.rs rmdir: match GNU 2021-08-28 02:26:01 +02:00
test_runcon.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_seq.rs seq: Allow option to receive immediate arguments 2022-02-06 09:45:38 -06:00
test_shred.rs shred, stat, tail: Remove direct usage of CmdResult fields in test 2021-04-17 18:50:41 +03:00
test_shuf.rs fix lint errors attempt 2 2022-03-28 11:36:50 -04:00
test_sleep.rs uucore: error on negative interval in parse_time 2022-03-21 21:11:31 -04:00
test_sort.rs uucore: add InvalidSuffix to ParseSizeError 2022-05-17 15:37:49 +02:00
test_split.rs split: Don't overwrite files 2022-07-21 12:06:13 -04:00
test_stat.rs test_stat: expand scope for stdin tests 2022-05-12 22:49:34 +02:00
test_stdbuf.rs Improve coverage / error messages from parse_size PR 2022-02-22 22:09:45 +01:00
test_sum.rs Update message quoting and filename printing 2021-09-07 19:49:01 +02:00
test_sync.rs test: fix tests 2021-05-26 03:07:49 +03:00
test_tac.rs tac: add support for --regex option to tac 2021-09-11 22:54:55 -04:00
test_tail.rs tail: reduce CPU load for polling (#3618) 2022-06-21 22:21:19 +02:00
test_tee.rs Implement tee -p and --output-error 2022-07-07 15:23:50 +02:00
test_test.rs test: fix wsl executable permission 2022-01-31 00:22:21 +01:00
test_timeout.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_touch.rs also support for tests/touch/relative.sh 2022-05-08 21:52:12 +02:00
test_tr.rs tr: fix octal interpretation of repeat count string (#3178) 2022-02-25 12:11:53 +01:00
test_true.rs false,true: Implement custom help, version 2022-02-01 14:29:26 +01:00
test_truncate.rs Remove impractical test creating a file too large 2022-02-22 14:23:46 +01:00
test_tsort.rs refactor tests (#1982) 2021-04-23 02:28:46 +02:00
test_tty.rs tty: should not return 2 when --help is used 2022-04-04 22:55:36 +02:00
test_uname.rs get android builds to compile and pass tests 2022-04-20 08:44:49 +02:00
test_unexpand.rs unexpand: handle too large "tabs" arguments 2022-06-22 09:30:11 +02:00
test_uniq.rs uniq: set default missing value for args 2022-06-30 11:52:46 +02:00
test_unlink.rs unlink: Simplify, remove unsafe, move to core 2021-09-10 21:50:38 +02:00
test_uptime.rs tests ~ refactor/polish spelling (comments, names, and exceptions) 2021-05-31 08:23:57 -05:00
test_users.rs users and who: ignore failing tests for now 2022-03-07 12:32:07 -05:00
test_vdir.rs dir, vdir: fix incorrect regex in tests 2022-05-01 13:48:51 +02:00
test_wc.rs Add missing testcases for wc. 2022-07-24 21:56:07 -07:00
test_who.rs all: use array intoiterator 2022-04-05 10:39:31 +02:00
test_whoami.rs whoami: Run tests on Windows 2021-09-14 14:13:58 +02:00
test_yes.rs Do not trap pipe errors in yes 2022-07-07 15:23:50 +02:00