Refactor the counting code from the inner loop of the `wc` program
into the `WordCount::from_line()` associated function. This commit
also splits that function up into other helper functions that
encapsulate decoding characters and finding word boundaries from raw
bytes.
This commit also implements the `Sum` trait for the `WordCount`
struct, so that we can simply call `sum()` on an iterator that yields
`WordCount` instances.
This is a refactor to reduce duplicate code, it affects chmod/ls/stat.
* merge `stat/src/fsext::pretty_access` into `uucore/src/lib/feature/fs::display_permissions_unix`
* move tests for `fs::display_permissions` from `test_stat::test_access` to `uucore/src/lib/features/fs::test_display_permissions`
* adjust `uu_chmod`, `uu_ls` and `uu_stat` to use `uucore::fs::display_permissions`
FileMerger is much more efficient than the previous algorithm,
which looped over all elements every time to determine the next element.
FileMerger uses a BinaryHeap, which should bring the complexity for
the merge step down from O(n²) to O(n log n).
* ls: Implement total size feature
- Implement total size reporting that was missing
- Fix minor formatting / readability nits
* tests: Add tests for ls total sizes feature
* ls: Fix MSRV build errors due to unsupported attributes for if blocks
* ls: Add windows support for total sizes feature
- Add windows support (defaults to file size as block sizes related
infromation is not avialable on windows)
- Renamed some functions
Add the `WordCountable::lines()` method that returns an iterator over
lines of a file-like object. This mirrors the
`std::io::BufRead::lines()` method, with some minor differences due to
the particular use case of `wc`.
This commit also creates a new module, `countable.rs`, to contain the
`WordCountable` trait and the new `Lines` struct returned by `lines()`.
Use clap for argument parsing instead of getopts
Also, make the following changes
* Use `executable!()` macro to output the name of utility
* Add another usage to help message
- Replace the parser with a recursive descent implementation that handles
parentheses and produces a stack of operations in postfix order.
Parsing now operates directly on OsStrings passed by the uucore framework.
- Replace the dispatch mechanism with a stack machine operating on the
symbol stack produced by the parser.
- Add tests for parenthesized expressions.
- Begin testing character encoding handling.
Moved argument parsing to clap and added tests to cover using "-" as
stdin, passing in too many file arguments, and updated the "wrap" error
message in the tests.
It is much faster to just write the lines to disk, separated by \n
(or \0 if zero-terminated is enabled), instead of serializing to json.
external_sort now knows of the Line struct instead of interacting with
it using the ExternallySortable trait. Similarly, it now uses the
crash_if_err! macro to handle errors, instead of bubbling them up.
Some functions were changed from taking &[Line] as the input to taking
an Iterator<Item = Line>. This removes the need to collect to a Vec
when not necessary.
This removes the need to allocate a new string for each line when used
with -f, -d or -i. Instead, a custom string comparison algorithm takes
care of these cases.
The resulting performance improvement is about 20% per flag (i.e. there
is a 60% improvement when combining all three flags)
As a side-effect, the size of the Line struct was reduced from 96 to 80
bytes, reducing the overhead for each line.
* ls: added creation time
* ls: Added most time features
Missing support for posix-,Format+, translating via locales. Also required more tests
* ls: rustfmt
* ls: Additional changes and fixes
Fixed the argument order, fixed a wrong iso format.
* ls: additional tests for styles
* ls: perfected arg parsing on time styles
* fix birthime test
* ls: Use 'stdout_str' in new tests
* ls: Disabled birthtime test for windows
* ls: removed indoc as a dependency
* ls: birthime test, sync first created file
* ls: birthime test, add comment explaining sync
* Removed ruby testfile birth_test.rb
This accidentally got commited in a merge
Note, I needed to change the error messages in one of the tests because
getopt and clap have different error messages when not providing a
default value
* Change unchecked unwrapping to unwrap_or_default for argument parsing (resolving #1845)
* Added unit-testing for the collect_str function on invalid utf8 OsStrs
* Added a warning-message for identification purpose to the collect_str method.
* - Add removal of wrongly encoded empty strings to basename
- Add testing of broken encoding to basename
- Changed UCommand to use collect_str in args method to allow for integration testing of that method
- Change UCommand to use unwarp_or_default in arg method to match the behaviour of collect_str
* Trying out a new pattern for convert_str for getting a feeling of how the API feels with more control
* Adding convenience API for compact calls
* Add new API to everywhere, fix test for basename
* Added unit-testing for the conversion options
* Added unit-testing for the conversion options for windows
* fixed compilation and some merge hiccups
* Remove windows tests in order to make merge request build
* Fix formatting to match rustfmt for the merged file
* Improve documentation of the collect_str method and the unit-tests
* Fix compilation problems with test
Co-authored-by: Christopher Regali <chris.vdop@gmail.com>
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
* ls: ignore leading period when sorting by name
ls now behaves like GNU ls with respect to sorting files by ignoring
leading periods when sorting by main.
Added tests to ensure "touch a .a b .b ; ls" returns ".a a .b b"
* Replaced clone/collect calls.
* Use buffered stdout to reduce write sys calls.
This simple change yielded the biggest performace gain.
* Use `for_byte_record_with_terminator` from the `bstr` crate.
This is to minimize the per line copying needed by
`BufReader::read_until`. The `cut_fields` and `cut_fields_delimiter`
functions used `read_until` to iterate over lines. That required copying
each input line to the line buffer. With
`for_byte_record_with_terminator` copying is minimized as it calls our
closure with a reference to BufReader's buffer most of the time. It
needs to copy (internally) only to process any incomplete lines at the
end of the buffer.
* Re-write `Searcher` to use `memchr`.
Switch from the naive implementation to one that uses `memchr`.
* Rewrite `cut_bytes` almost entirely.
This was already well optimized. The performance gain in this case is
not from avoiding copying. In fact, it needed zero copying whereas new
implementation introduces some copying similar to `cut_fields` described
above. But the occassional copying cost is more than offset by the use
of the very fast `memchr` inside `for_byte_record_with_terminator`.
This change also simplifies the code significantly. Removed the `buffer`
module.
