Fix a bug in which "\r\n" was not being replaced with "\n" in text mode
on Windows. This would happen only if one call to `write()` ended with a
"\r" character and the next call to `write()` started with a "\n"
character. This commit fixes the bug by buffering a "\r" character if it
appears at the end of one call to `write()` and only writing if the
first character in the next call to `write()` is *not* a "\n" character.
Fixes issue #2681.
Since tac must read its input files completely to start processing them
from the end, it is particularly suited to use memory maps to benefit
from the page cache maintained by the operating systems to bring the
necessary data into memory as required.
This does also include situations where the input is stdin, but not via
a pipe but for example a file descriptor set up by the user's shell
through an input redirection.
Replace two loops that print leading and trailing 0s when printing a
number in fixed-width mode with a single call to `write!()` with the
appropriate formatting parameters.
Ensure that the `print_seq_integers()` function is called when the first
number and the increment are integers, regardless of the type of the
last value specified.
Create a `DigestWriter` struct that implements `Write` by passing bytes
directly to `Digest::input()`, so that `hashsum` can use
`std::io::copy()`. Using `std::io::copy()` eliminates some boilerplate
code around reading and writing bytes. And defining `DigestWriter` makes
it easier to add a `#[cfg(windows)]` guard around the Windows-specific
replacement of "\r\n" with "\n".
Change the way `seq` computes the number of digits needed to print a
number so that it works for inputs given in scientific notation.
Specifically, this commit parses the input string to determine whether
it is an integer, a float in decimal notation, or a float in scientific
notation, and then computes the number of integral digits and the number
of fractional digits based on that. This also supports floating point
negative zero, expressed in both decimal and scientific notation.
- Attach context to I/O errors
- Make flags override each other
- Support invalid unicode as argument
- Call WsaCleanup() even on panic
- Do not use deprecated std::mem::uninitialized()
Add the `is_first_iteration` Boolean variable to the `print_seq()`
function in order to avoid unnecessary comparisons. Specifically, before
this change, the `done_printing()` function was called twice on each
iteration of the main loop. After this change, it is only called once
per iteration.
Furthermore, this change makes the `print_seq()` function similar in
structure to the `print_seq_integers()` function.
Co-authored-by: Jan Verbeek <jan.verbeek@posteo.nl>
This commit aim to correct #2645.
The origin of the problem was that `handle_obsolete` was not looking for
named signals but only for numbers preceded by '-'. I have made a few
changes:
- Change `handle_obsolete` to use `signal_by_name_or_value` to parse the
possible signal.
- Since now the signal is already parsed return an `Option<usize>`
instead of `Option<&str>` to parse later.
- Change the way to decide the pid to use from a `match` to a `if`
because the tested element are actually not the same for each branch.
- Parse the signal from the `-s` argument outside of `kill` function for
consistency with the "obsolete" signal case.
- Again for consistency, parse the pids outside the `kill` function.
This makes it no longer possible to pass multiple filenames, but every
other implementation I tried (GNU, busybox, FreeBSD, sbase) also
forbids that so I think it's for the best.
* chown: support '.' as 'user.group' separator (like ':')
It also manages the case:
user.name:group (valid too)
Should fix:
gnu/tests/chown/separator.sh
* Fixed some documentation of display_item_long that was missed in #2623
* Implemented coloring of `ls -l` symlink targets( after the arrow `->`).
* Documented display_file_name to some extent.
* Ran rustfmt as part of mitigating CI chain errors.
* Removed unused variables and code in test_ls_long_format as per #2623
specifically, as per
https://github.com/uutils/coreutils/pull/2623#pullrequestreview-742386707
* Added a thorough test for `ls -laR --color` symlink coloring implemented in this branch.
* renamed test files and dirs to shorter names and ran rustfmt.
* Changed the order with which files are expected to match the change in their name.
* Bettered some comments.
* Removed needless borrow. Fixed that one clippy warning.
* Moved the cfg not windows up to the function level
because this function is meant to only run in non-windows OS (with
groups and unames).
Fixes the unused variable warning in CI.
Part of the code was transliterated from GNU's implementation in C,
and used flags to store settings. Instead, we can use enums to avoid
magic values or binary operations to extract flags.
This way later code can assume `src` and `dest` to be the actual paths
of source and destination, and do not have to constantly check
`options.dereference`.
This requires moving the error context calculation to the beginning as
well, since later steps no longer operate with the same file paths as
supplied by the user.
Fix a mix-up between ownership and mode. The latter (mode / file permissions)
can also be set on windows (which however only affects the read-only flag),
while there doesn't seem to be a straight-forward way to change file ownership
on windows.
Copy the acl as well when copying the mode. This is a non-default feature and can be
enabled with --features feat_acl, because it doesn't seem to work on CI.
It is only available for unix so far.
Copy the SELinux context if possible.
Makes the -o auto option construct a format at initialization, rather
than try to handle it as a special case when printing lines. Fixes bugs
when combined with -e, especially when combined with -a.
* Used .as_path() and .as_str() when required:
when the argument required is a Path and not a PathBuf,
or an str and not a Path, respectively.
* Changed display_items to take Vec<PathData>, which is passed, instead of [PathData]
* Added a pad_right function.
* Implemented column-formating to mimic the behavior of GNU coreutils's ls
Added returns in display_dir_entry_size that keep track of uname and
group lengths.
Renamed variables to make more sense.
Changed display_item_long to take all the lengths it needs to render
correctly.
Implemented owner, group, and author padding right to mimic GNU ls.
* Added a todo for future quality-of-life cache addition.
* Documented display_item_long, as a first step in documenting all functions.
* Revert "Used .as_path() and .as_str() when required:"
This reverts commit b88db6a817.
* Revert "Changed display_items to take Vec<PathData>, which is passed, instead of [PathData]"
This reverts commit 0c690dda8d.
* Ran cargo fmt to get rid of Style/format `fmt` testing error.
* Added a test for `ls -l` and `ls -lan` line formats.
* Changed uname -> username for cspell. Removed extra blank line for rustfmt.
Fix a bug in `seq` where the number of characters needed to print the
number was computed incorrectly in some cases. This commit changes the
computation of the width to be after parsing the number instead of
before, in order to accommodate inputs like `1e3`, which requires four
digits when printing the number, not three.
Combine the `first`, `increment`, and `last` parameters of the
`print_seq()` and `print_seq_integers()` functions into a `RangeF64` or
`RangeInt` type, respectively.
- Implement all of GNU's fiddly little details
- Don't assume Linux for error codes
- Accept badly-encoded filenames
- Report errors after the fact instead of checking ahead of time
- General cleanup
rmdir now passes GNU's tests.
If reading fails midway through then a count should be reported for
what could be read.
If opening a file fails then no count should be reported.
The exit code shouldn't report the number of failures, that's fragile
in case of many failures.
Errors are now always shown with the corresponding filename.
Errors are no longer converted into warnings. Previously `wc < .`
would cause a loop.
Checking whether something is a directory is no longer done in
advance. This removes race conditions and the edge case where stdin is
a directory.
The custom error type is removed because io::Error is now enough.
- Reuse allocations for read lines
- Increase splice size
- Check if /dev/null was opened correctly
- Do not discard read bytes after I/O error
- Add fast line counting with bytecount
Unify the usage of macros `return_if_err` and `crash_if_err`. As
`return_if_err` is used only in `uumain` routines of utilities, it
achieves the same thing as `crash_if_err`, which calls the `crash!`
macro to terminate function execution immediately.
Unify the usage of macros `safe_unwrap` and `crash_if_err` that are
identical to each other except for the assumption of a default error
code. Use the more generic `crash_if_err` so that `safe_unwrap` is now
obsolete and can be removed.
Remove a copy operation of the input buffer being read for digest when
reading in text mode on Windows. Previously, the code was copying the
buffer to a completely new `Vec`, replacing "\r\n" with "\n". Instead,
the code now scans for the indices at which each "\r\n" occurs in the
input buffer and inputs into the digest only the characters before the
"\r" and after it.
Report errors properly instead of panicking.
Replace zero_copy by a simpler specialized private module.
Do not assume splices move all data at once.
Use the modern uutils machinery.
Remove the "latency" feature. The time it takes to prepare the buffer
is drowned out by the startup time anyway.
yes: Add tests
yes: Fix long input test on Windows
Fix a bug in `tac` where multi-character line separators would cause
incorrect behavior when there was overlap between candidate matches in
the input string. This commit adds a dependency on `memchr` in order to
use the `memchr::memmem::rfind_iter()` function to scan for
non-overlapping instances of the specified line separator characters,
scanning from right to left.
Fixes#2580.
* hashsum: support --check for var. length outputs
Add the ability for `hashsum --check` to work with algorithms with
variable output length. Previously, the program would terminate with an
error due to constructing an invalid regular expression.
* fixup! hashsum: support --check for var. length outputs
* tac: correct behavior of -b option
Correct the behavior of `tac -b` to match that of GNU coreutils
`tac`. Specifically, this changes `tac -b` to assume *leading* line
separators instead of the default *trailing* line separators.
Before this commit, the (incorrect) behavior was
$ printf "/abc/def" | tac -b -s "/"
def/abc/
After this commit, the behavior is
$ printf "/abc/def" | tac -b -s "/"
/def/abc
Fixes#2262.
* fixup! tac: correct behavior of -b option
* fixup! tac: correct behavior of -b option
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
Hopefully will be feature parity with GNU `tr`.
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Implemented a bit of new expansion module
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Implemented delete operation
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Partially implemented delete operation
Will go through translate next.
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Fix formatting...
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Implemented translation feature
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Move the creation of temporary files into next_file so that it happens
while the lock is taken.
Previously, only the computation of the new file path happened while
the lock was taken, while the creation happened later.
csplit fails when suffix has no flags:
$ csplit result.expected -f /tmp/EXPECT -b "%d" "/^## alternative ##$/" {*}
csplit: error: incorrect conversion specification in suffix
This is supported by original csplit
If options::WIDTH is not given, we should try to use the terminal width.
If that is unavailable, we should fall back to the 'COLUMNS' environment variable.
If that is unavailable (or invalid), we should fall back to a default of 80.
As custom errors are prefered over wrapping around common errors, the
distinction between UCommonError and UCustomError is removed. This
reduces the number of types and makes the error handling easier to
understand.
The following test case read stdin instead of file:
```
echo abcdefg > file
cargo run -- od --format x1 file
```
This is because the -t/--format argument was able to absorb multiple
arguments after it. This has now been fixed, and a test case is added
to ensure it will not happen again.
When hitting Ctrl+C sort now deletes any temporary files. To make this easier I
created a new struct `TmpDirWrapper` that handles the creation of new temporary
files and the registration of the signal handler.
The ToDo list was updated to mark `chcon` as done.
Building and testing `chcon` requires enabling the `feat_selinux` feature. If `make` is used for building, then please specify `SELINUX_ENABLED=1` if building and testing on a system where SELinux is not enabled.
Fix an issue where `basename` would print the empty path when provided
an input comprising only slashes (for example, "/" or "//" or
"///"). Now it correctly prints the path "/".
In such cases we have to create a temporary copy of the input file to prevent
overwriting the input with the output. This only affects merge sort, because it
is the only mode where we start writing to the output before having read all inputs.
Since lines that compare equal should be sorted together, we need to first
compare the lines (taking settings into account). Only if they do not compare
equal we should compare the hashes.
Use new macro to construct UIoError types from `std::io::Error` before
printing. This gives consistent error messages across all utilities as it
prepends custom errors to the error description for the respective application
and error type. This saves the developers from manually appending the
`std::io::Error`-specific error messages.
Drop the previous flags that would tell whether a noncritical error occured
during execution in favor of the `show!` macro from the error submodule.
This allows us to generate regular error types during execution to signify
failures inside the program, but without prematurely aborting program execution
if not needed or specified.
Also make verbose outputs use `print!` and friends instead of `show_error!` to
ensure verbose output is redirected to stdout, not stderr.
Add support for
* space-separated list of tab stops, like `--tabs="2 4 6"`,
* mixed comma- and space-separated lists, like `--tabs="2,4 6"`,
* the slash specifier in the last tab stop, like `--tabs=1,/5`,
* the plus specifier in the last tab stop, like `--tabs=1,+5`.
For whatever reason, the following is equivalent,
cargo run -- rm --presume-input-tty
cargo run -- rm ---presume-input-tty
cargo run -- rm -----presume-input-tty
cargo run -- rm ---------presume-input-tty
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
The custom usage string does not have to include the "sort\nUsage:" part,
because this part is already printed by clap.
It prevents the following duplication:
USAGE:
sort
Usage:
sort [OPTION]... [FILE]..
Now, only the following is printed:
USAGE:
sort [OPTION]... [FILE]...
- Adds more jargon to the spellchecker jargon file.
- Adds more ignores to the per-project ignores.
- Fixes clippy warnigns about uneeded clones on non-linux.
- Block/Unblock now account for system dependend newlines.
Change the behavior of `tac` when there are no line separators in the
input. Previously, it was appending an extra line separator; this commit
prevents that from happening. The input is now writted directly to
stdout.
Correct the error message produced by `tac` when trying to read from a
directory. Previously if the path 'a' referred to a directory, then
running `tac a` would produce the error message
dir: read error: Invalid argument
after this commit it produces
a: read error: Invalid argument
which matches GNU `tac`.
Handle additional edge cases arising from test(1)’s lack of reserved words.
For example, left parenthesis followed by an operator could indicate
either
* string comparison of a literal left parenthesis, e.g. `( = foo`
* parenthesized expression using an operator as a literal, e.g. `( = != foo )`
- Adds words to cspell exceptions
- Converts test macros to use Default trait.
- Converts parser to use Default trait.
- Adds Windows-friendly test files for block/unblock when nl is present
in test/spec file.
- Removes dd from feat_require_unix (keeps it in feat_common_core)
- Changes name of make_linux_iflags parameter from oflags to iflags.
- Removes roughed out SIGINFO impl.
- Renames plen -> target_len.
- Removes internal fn def for build_blocks and replaces with a call to
chunks from std.
- Renames apply_ct to the more descriptive apply_conversion
- Replaces manual swap steps in perform_swab with call to swap from std.
- Replaces manual solution with chunks where appropriate (in Read, and Write impl).
- Renames xfer -> transfer (in local variable names).
- Improves documentation for dd_filout/dd_stdout issue.
- Removes commented debug_assert statements.
- Modifies ProdUpdate to contain ReadStat and WriteStat rather than
copying their fields.
- Addresses verbose return in `Output<File>::new(...)`
- Resoves compiler warning when built as release when signal handler fails to
register.
- Derives _Default_ trait on ReadStat.
- Adds comments for truncated lines in block unblock tests
- Removes `as u8` in block unblock tests.
- Removes unecessary `#[inline]`
- Delegates multiplier string parsing to uucore::parse_size.
- Renames 'unfailed' -> 'succeeded' for clairity.
- Removes #dead_code warnings. No clippy warnings on my local machine.
- Reworks signal handler to better accomodate platform-specific signals.
- Removes explicit references to "if" and "of" in dd.rs.
- Removes explicit references to "bs", "ibs", "cbs" and "status" in
parseargs.rs.
- Removes `#[allow(deadcode)]` for OFlags, and IFlags.
- Removes spellchecker ignore from all dd files.
- Adds tests for 'traditional' and 'modern' CLI.
- Address Rust fmt issue
- ignores run-without-noatime test which fails on some build machines
- adds cspell:disable to all project files.
- adds .../dd/fixtures/cspell.json to ignore test fixtures.
- Adds cspell.json. Hopefully this will make you happy, spellchecker.
- Removes non-functional spellchecker-ignore tags
- Adds a sleep call to the no noatime test. Some systems were did not notice a changed
atime without the option present.
- Adds a test with a unicode filename.
- Addresses clippy lints and rustfmt issues.
- adds project header to multiple files
- updates spell check skip words
- removes linux only flags direct,noatime from mac_os build
- applies rustfmt to test_dd
- runs rustfmt on test_dd.rs
- eliminates compiler warnings
- adds many words to spellchecker ignore list
- adds sanity test for vexing conv=nocreat issue. Still WIP.
The `uname` `-o` switch reports the operating system used. If the GNU C
standard library (glibc) is not in use, for example if musl is being
used instead, report "Linux" instead of "GNU/Linux".
Fails with:
```
error: use of irregular braces for `write!` macro
--> src/uucore/src/lib/features/encoding.rs:19:17
|
19 | #[derive(Debug, Error)]
| ^^^^^
|
= note: `-D clippy::nonstandard-macro-braces` implied by `-D warnings`
help: consider writing `Error`
--> src/uucore/src/lib/features/encoding.rs:19:17
|
19 | #[derive(Debug, Error)]
| ^^^^^
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#nonstandard_macro_braces
= note: this error originates in the derive macro `Error` (in Nightly builds, run with -Z macro-backtrace for more info)
```
This makes clap wrap the help text according to the terminal width,
which improves readability for terminal widths < 120 chars,
because clap defaults to a width of 120 chars without this feature.
This reimplements version_cmp, which is used in sort and ls to sort
according to versions.
However, it is not bug-for-bug identical with GNU's implementation.
I reported a bug with GNU here:
https://lists.gnu.org/archive/html/bug-coreutils/2021-06/msg00045.html
This implementation does not contain the bugs regarding the handling of
file extensions and null bytes.
Adds the ability to perform file backups before installing newer files on top
of existing ones. Adds a status message about backups to stdout if running in
verbose mode.
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Keep one of the texts in-place
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Reduced the fix to just formatting changes
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
The '--backup' option would previously accept arguments separated from the
option either by a space or an equals sign. The GNU implementation strictly
requires an "equals" for argument separation.
As the argument to '--backup' is optional, the equals sign mustn't be ommited
as otherwise there is no way to tell a file argument apart from an argument
that's meant for the '--backup' option. This ensures that if '--backup' is
present it either has no further associated arguments (i.e. fallback to the
default), or the arguments are separated by an equals sign.
Gerenal numeric sort works by comparing pre-parsed floating point
numbers. That means that we don't have to store the &str the float was
parsed from.
As a result, memory usage was slightly reduced for general numeric sort.
`LANGUAGE=C` is not enough, `LC_ALL=C` is needed as the environment
variable that overrides all the other localization settings.
e.g.
```bash
$ LANGUAGE=C id foobar
id: ‘foobar’: no such user
$ LC_ALL=C id foobar
id: 'foobar': no such user
```
* replace `LANGUAGE` with `LC_ALL` as environment variable in the tests
* fix the the date string of affected uutils
* replace `‘` and `’` with `'`
Data that was previously boxed inside the `Line` struct was moved to
separate vectors. Inside of each `Line` remains only an index that
allows to access that data.
This helps with keeping the `Line` struct small and therefore reduces
memory usage in most cases.
Additionally, this improves performance because one big allocation (the
vectors) are faster than many small ones (many boxes inside of each
`Line`). Those vectors can be reused as well, reducing the amount of
(de-)allocations.
- Windows hidden file attribute determination would assume symbolic link
to be valid and would panic
- Check symbolic link's attributes if the link points to non-existing
file
- For dangling symlinks, errors should only be reported if
dereferencing options were passed and dereferencing was applicable to
the particular symlink
- With -i parameter, report '?' as the inode number for dangling
symlinks
When invoked via '[' name, last argument must be ']' or we bail out with
syntax error. Then the trailing ']' is simply disregarded and processing
happens like usual.
- The dd info page mentions a special fast-read framework if no conv=FLAG is
specified (see bs=N) which I left space for. As it turns out, this is performed already
so it does not need to be implemented.
- When we have finished reading from a temproary file, we can immediately
delete it.
- Use one single directory for all temporary files.
- Only create the temporary directory when needed.
- Also compress temporary files created by the merge step if requested.
