Add the `ByteChunkWriter` and `LineChunkWriter` structs and
implementations, but don't use them yet. This structs offer an
alternative approach to writing chunks of output (contrasted with
`ByteSplitter` and `LineSplitter`). The main difference is that
control of which underlying file is being written is inside the writer
instead of outside.
Using indexes into the line instead of Vec<u8>s means we don't have to copy
the line to store the fields (indexes instead of slices because it avoids
self-referential structs). Using memchr also empirically saves a lot of
intermediate allocations.
- Change the main! proc_macro to a bin! macro_rules macro.
- Reexport uucore_procs from uucore
- Make utils to not import uucore_procs directly
- Remove the `syn` dependency and don't parse proc_macro input (hopefully for faster compile times)
Move the `printf::memo` module to `uucore` so that it can be used by
other programs, not just `printf`. For example, the `-f` option to `seq`
requires parsing and formatting numbers according to the same logic as
`printf`.
* seq: use BigDecimal to represent floats
Use `BigDecimal` to represent arbitrary precision floats in order to
prevent numerical precision issues when iterating over a sequence of
numbers. This commit makes several changes at once to accomplish this
goal.
First, it creates a new struct, `PreciseNumber`, that is responsible for
storing not only the number itself but also the number of digits (both
integer and decimal) needed to display it. This information is collected
at the time of parsing the number, which lives in the new
`numberparse.rs` module.
Second, it uses the `BigDecimal` struct to store arbitrary precision
floating point numbers instead of the previous `f64` primitive
type. This protects against issues of numerical precision when
repeatedly accumulating a very small increment.
Third, since neither the `BigDecimal` nor `BigInt` types have a
representation of infinity, minus infinity, minus zero, or NaN, we add
the `ExtendedBigDecimal` and `ExtendedBigInt` enumerations which extend
the basic types with these concepts.
* fixup! seq: use BigDecimal to represent floats
* fixup! seq: use BigDecimal to represent floats
* fixup! seq: use BigDecimal to represent floats
* fixup! seq: use BigDecimal to represent floats
* fixup! seq: use BigDecimal to represent floats
Replace the custom `split::walk_lines()` function with a call to
`std::io::copy()`, using a new `TakeLines` reader as the source and
`stdout` as the destination. The `TakeLines` reader is an adaptor that
scans the bytes being read for line ending characters and stops the
reading after a given number of lines has been read (similar to the
`std::io::Take` adaptor).
This change
* makes the `read_n_lines()` function more concise,
* allows it to mirror the implementation of `read_n_bytes()`,
* increases the speed of `head -n NUM`.
Since tac must read its input files completely to start processing them
from the end, it is particularly suited to use memory maps to benefit
from the page cache maintained by the operating systems to bring the
necessary data into memory as required.
This does also include situations where the input is stdin, but not via
a pipe but for example a file descriptor set up by the user's shell
through an input redirection.
This makes it no longer possible to pass multiple filenames, but every
other implementation I tried (GNU, busybox, FreeBSD, sbase) also
forbids that so I think it's for the best.
Fix a mix-up between ownership and mode. The latter (mode / file permissions)
can also be set on windows (which however only affects the read-only flag),
while there doesn't seem to be a straight-forward way to change file ownership
on windows.
Copy the acl as well when copying the mode. This is a non-default feature and can be
enabled with --features feat_acl, because it doesn't seem to work on CI.
It is only available for unix so far.
Copy the SELinux context if possible.
- Implement all of GNU's fiddly little details
- Don't assume Linux for error codes
- Accept badly-encoded filenames
- Report errors after the fact instead of checking ahead of time
- General cleanup
rmdir now passes GNU's tests.
Errors are now always shown with the corresponding filename.
Errors are no longer converted into warnings. Previously `wc < .`
would cause a loop.
Checking whether something is a directory is no longer done in
advance. This removes race conditions and the edge case where stdin is
a directory.
The custom error type is removed because io::Error is now enough.
- Reuse allocations for read lines
- Increase splice size
- Check if /dev/null was opened correctly
- Do not discard read bytes after I/O error
- Add fast line counting with bytecount
Remove a copy operation of the input buffer being read for digest when
reading in text mode on Windows. Previously, the code was copying the
buffer to a completely new `Vec`, replacing "\r\n" with "\n". Instead,
the code now scans for the indices at which each "\r\n" occurs in the
input buffer and inputs into the digest only the characters before the
"\r" and after it.
Report errors properly instead of panicking.
Replace zero_copy by a simpler specialized private module.
Do not assume splices move all data at once.
Use the modern uutils machinery.
Remove the "latency" feature. The time it takes to prepare the buffer
is drowned out by the startup time anyway.
yes: Add tests
yes: Fix long input test on Windows
Fix a bug in `tac` where multi-character line separators would cause
incorrect behavior when there was overlap between candidate matches in
the input string. This commit adds a dependency on `memchr` in order to
use the `memchr::memmem::rfind_iter()` function to scan for
non-overlapping instances of the specified line separator characters,
scanning from right to left.
Fixes#2580.
Hopefully will be feature parity with GNU `tr`.
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Implemented a bit of new expansion module
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Implemented delete operation
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Partially implemented delete operation
Will go through translate next.
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Fix formatting...
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
Implemented translation feature
Signed-off-by: Hanif Bin Ariffin <hanif.ariffin.4326@gmail.com>
When hitting Ctrl+C sort now deletes any temporary files. To make this easier I
created a new struct `TmpDirWrapper` that handles the creation of new temporary
files and the registration of the signal handler.
The ToDo list was updated to mark `chcon` as done.
Building and testing `chcon` requires enabling the `feat_selinux` feature. If `make` is used for building, then please specify `SELINUX_ENABLED=1` if building and testing on a system where SELinux is not enabled.
- Adds words to cspell exceptions
- Converts test macros to use Default trait.
- Converts parser to use Default trait.
- Adds Windows-friendly test files for block/unblock when nl is present
in test/spec file.
Make all tests lock a mutex to ensure that they're run in series rather than
parallel. We must take this precaution due to the fact that all tests are run
in parallel as threads of one parent process. As all threads in a process share
e.g. environment variables, we use the Mutex to ensure they're run one after
another.
This way we can guarantee that tests that rely on environment variables to have
specific values will see these variables, too.
An alternative implementation could have used the [rusty fork][1] crate to run
all tests that need env variables in separate processes rather than threads.
However, rusty fork likely wouldn't run on all platforms that the utilities are
supposed to run on.
- runs rustfmt on test_dd.rs
- eliminates compiler warnings
- adds many words to spellchecker ignore list
- adds sanity test for vexing conv=nocreat issue. Still WIP.
This makes clap wrap the help text according to the terminal width,
which improves readability for terminal widths < 120 chars,
because clap defaults to a width of 120 chars without this feature.
This reimplements version_cmp, which is used in sort and ls to sort
according to versions.
However, it is not bug-for-bug identical with GNU's implementation.
I reported a bug with GNU here:
https://lists.gnu.org/archive/html/bug-coreutils/2021-06/msg00045.html
This implementation does not contain the bugs regarding the handling of
file extensions and null bytes.
This adds a hidden `completion` subcommand to coreutils. When invoked with
`coreutils completion <utility> <shell>` a completion file will be printed
to stdout. When running `make install` those files will be created for all
utilities and copied to the appropriate locations.
`make install` will install completions for zsh, fish and bash; however,
clap also supports generating completions for powershell and elvish.
With this patch all utilities are required to have a publich uu_app function
that returns a clap::App in addition to the uumain function.
- Adds print fn's
- Modifies internal fn's as needed to track read/write state
- Modifies status update thread to respect status level
- Adds signal handler for SIGUSR1 (print xfer stats)
- Use `unicode_segmentation` and `unicode_width` to determine proper `break_line` position.
- Keep track of total_width as suggested by @tertsdiepraam.
- Add unittest for ZWJ unicode case
Related to #2319.
Port argument parsing from getopts to clap.
The only difference I have observed is that clap auto-generates -h and
-V short options for help and version, and there is no way (in clap 2.x)
to disable them.
If we notice that we can represent all arguments as BigInts, take a
different code path. Just like GNU seq this means we can print an
infinite amount of numbers in this case.
* expr: support arbitrary precision integers
Instead of i64s we now use BigInts for integer operations. This means
that no result or input can be out of range.
The representation of integer flags was changed from i64 to u8 to make
their intention clearer.
* expr: allow big numbers as arguments as well
Also adds some tests
* expr: use num-traits to check bigints for 0 and 1
* expr: remove obsolete refs
match ergonomics made these avoidable.
* formatting
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
Instead of using a BufReader and reading each line separately,
allocating a String for each one, we read to a chunk. Lines are
references to this chunk. This makes the allocator's job much easier
and yields performance improvements.
Chunks are read on a separate thread to further improve performance.
