Commit graph

845 commits

Author SHA1 Message Date
Sylvestre Ledru
7e07438b38
Merge pull request #2151 from jfinkels/2141-translate-and-squeeze
tr: implement translate and squeeze (-s) mode
2021-05-01 23:27:43 +02:00
Michael Debertol
484558e37d
Update src/uu/sort/BENCHMARKING.md
Co-authored-by: Sylvestre Ledru <sledru@mozilla.com>
2021-05-01 21:38:36 +02:00
Michael Debertol
b21a309c3f add a benchmarking example 2021-05-01 21:29:18 +02:00
Michael Debertol
83554f4475 add benchmarking instructions 2021-05-01 21:16:29 +02:00
Ricardo Iglesias
193ad56c2a Removed clippy warnings. 2021-05-01 11:36:46 -07:00
Ricardo Iglesias
f307de22d0 base64: Refactor argument parsing
Moved most of the argument parsing logic to `base32/base_common.rs` to
allow for significant code reuse.
2021-05-01 11:36:46 -07:00
Ricardo Iglesias
05b20c32a9 base64: Moved argument parsing to clap.
Moved argument parsing to clap and added tests to cover using "-" as
stdin, passing in too many file arguments, and updated the "wrap" error
message in the tests.
2021-05-01 11:36:46 -07:00
Jeffrey Finkelstein
5674d09327 fixup! tr: implement translate and squeeze (-s) mode 2021-05-01 13:01:55 -04:00
Jan Scheer
83eb704415 Merge branch 'master' into issue2147 2021-05-01 18:52:35 +02:00
Jan Scheer
117e84eed3 tr: implement complement separately from delete or squeeze (#2147) 2021-05-01 18:46:13 +02:00
Sylvestre Ledru
bffcb431b5
Merge pull request #2148 from jhscheer/pinky2clap
pinky: move from getopts to clap (#2123)
2021-05-01 17:49:10 +02:00
Sylvestre Ledru
34bf7cc5ea
Merge pull request #2150 from jhscheer/fix_clap_short
tr/dirname: fix clap short_alias
2021-05-01 17:39:15 +02:00
Michael Debertol
be0c924c95 Merge branch 'master' of https://github.com/uutils/coreutils into sort-no-json-extsort 2021-05-01 17:29:03 +02:00
Michael Debertol
01d178cf17 sort: don't rely on serde-json for extsort
It is much faster to just write the lines to disk, separated by \n
(or \0 if zero-terminated is enabled), instead of serializing to json.

external_sort now knows of the Line struct instead of interacting with
it using the ExternallySortable trait. Similarly, it now uses the
crash_if_err! macro to handle errors, instead of bubbling them up.

Some functions were changed from taking &[Line] as the input to taking
an Iterator<Item = Line>. This removes the need to collect to a Vec
when not necessary.
2021-05-01 17:20:56 +02:00
Nicolas Thery
70ab0d01d2 kill: change default signal
The default signal is SIGTERM, not SIGKILL.
2021-05-01 16:47:42 +02:00
Sylvestre Ledru
d2913f8080 rustfmt the recent change 2021-05-01 13:12:10 +02:00
Sylvestre Ledru
59ea28628b printf: remove useless declaration 2021-05-01 13:11:41 +02:00
Jeffrey Finkelstein
0f3bc23739 tr: implement translate and squeeze (-s) mode
Add translate and squeeze mode to the `tr` program. For example:

    $ printf xx | tr -s x y
    y

Fixes #2141.
2021-04-30 18:17:05 -04:00
Jan Scheer
798a033311 pinky: move from getopts to clap (#2123) 2021-04-30 20:57:38 +02:00
Jan Scheer
45dd9d4e96 tr/dirname: fix clap short_alias 2021-04-30 20:19:43 +02:00
Terts Diepraam
d300895d28 ls: add birth time for windows and attampt to fix test 2021-04-29 22:23:04 +02:00
Michael Debertol
fecbf3dc85 sort: remove an unneeded clone() 2021-04-29 18:05:55 +02:00
Michael Debertol
a4813c2646 sort: actually use the f64 cache
This was probably reverted accidentally.
2021-04-29 18:05:43 +02:00
Michael Debertol
9f45431bf0 sort: add some custom string comparisons
This removes the need to allocate a new string for each line when used
with -f, -d or -i. Instead, a custom string comparison algorithm takes
care of these cases.

The resulting performance improvement is about 20% per flag (i.e. there
is a 60% improvement when combining all three flags)

As a side-effect, the size of the Line struct was reduced from 96 to 80
bytes, reducing the overhead for each line.
2021-04-29 18:05:14 +02:00
nicoo
b89978a4c9
factor: Add annotations for coz, the causal profiler (#2142)
* factor: Add annotations for coz, the causal profiler

* Update Cargo.lock

Generated with `nix-shell -p rustup --run 'cargo +1.40.0 update'`
2021-04-29 15:56:56 +02:00
Jan Scheer
512d206f1e who: move from getopts to clap 2.33.3 (#2124) 2021-04-29 00:11:21 +02:00
Jan Scheer
6f16cafe88 who: move from getopts to clap (#2124) 2021-04-28 22:58:28 +02:00
Rein F
a60fd07bc3
ls: improvements on time handling (#1986)
* ls: added creation time

* ls: Added most time features

Missing support for posix-,Format+, translating via locales. Also required more tests

* ls: rustfmt

* ls: Additional changes and fixes

Fixed the argument order, fixed a wrong iso format.

* ls: additional tests for styles

* ls: perfected arg parsing on time styles

* fix birthime test

* ls: Use 'stdout_str' in new tests

* ls: Disabled birthtime test for windows

* ls: removed indoc as a dependency

* ls: birthime test, sync first created file

* ls: birthime test, add comment explaining sync

* Removed ruby testfile birth_test.rb

This accidentally got commited in a merge
2021-04-28 20:54:27 +02:00
Sylvestre Ledru
167520067c
Merge pull request #2111 from cbjadwani/cut_optimizations
cut: optimizations
2021-04-28 20:40:28 +02:00
Chirag Jadwani
25f99097cc cut: add BENCHMARKING.md
and minor refactoring
2021-04-28 23:28:26 +05:30
Sylvestre Ledru
a37e3181a2
Merge pull request #2130 from electricboogie/master
sort: implement --buffer-size and --temporary-directory (external sort)
2021-04-28 09:21:14 +02:00
Sylvestre Ledru
33139817a2
Merge pull request #2136 from jaggededgedjustice/allow-truncate-size-and-reference
Allow truncate to take --size and --reference
2021-04-27 22:43:25 +02:00
electricboogie
ec19bb72d5 Modified to remove 2 unnecessary consts now that we use std::env::temp_dir 2021-04-27 15:39:20 -05:00
Sylvestre Ledru
30cf6ec235
Merge pull request #2131 from ricardoaiglesias/base32-clap
Base32 clap
2021-04-27 09:20:45 +02:00
Ricardo Iglesias
ae0cabc60a Moved argument parsing to uumain. 2021-04-26 20:15:11 -07:00
Sylvestre Ledru
7a3b44d972
Merge pull request #2133 from tertsdiepraam/ls/fix_color_grid_alignment
`ls`: fix grid alignment with `--color`
2021-04-26 22:51:21 +02:00
Sylvestre Ledru
ece5e14b0d
fix a typo 2021-04-26 22:51:02 +02:00
James Robson
a7037b1ca9 Allow truncate to take --size and --reference 2021-04-26 18:39:32 +01:00
Terts Diepraam
35838dc8a9 ls: document hyperfine script 2021-04-26 18:36:15 +02:00
Terts Diepraam
4023e40174 ls: further reduce OsStr -> String conversions 2021-04-26 18:03:56 +02:00
Ricardo Iglesias
11d0565f0e base32: Moved clap argument parsing to base32.rs
Now, all base_common.rs has is the handle_input function.
2021-04-26 08:22:41 -07:00
Ricardo Iglesias
d56462a4b3 base32: Fixed style violations. Added tests
Tests now cover using "-" as standard input and reading from a file.
2021-04-26 08:00:55 -07:00
electricboogie
f3ed5a100f Possible fix to Windows issues, ext_sort bool setting 2021-04-26 08:54:40 -05:00
Terts Diepraam
c69b72c840 ls: forgot to commit Cargo.{toml, lock} 2021-04-26 15:04:55 +02:00
Terts Diepraam
58fd61b3e8 ls: fix grid alignment for unicode 2021-04-26 15:00:39 +02:00
Terts Diepraam
cfc11b47a5 ls: fix grid alignment with --color 2021-04-26 14:41:41 +02:00
Terts Diepraam
e4c0069493 ls: remove path strip 2021-04-26 09:53:13 +02:00
Terts Diepraam
322478d9a2 ls: document flamegraph 2021-04-26 09:37:47 +02:00
Sylvestre Ledru
7dcc8c2960
Merge pull request #1968 from alstolten/feat2
ls: Implements extension sorting
2021-04-26 09:03:06 +02:00
Ricardo Iglesias
99c13f202e Merge branch 'master' of github.com:uutils/coreutils into base32-clap 2021-04-25 22:36:26 -07:00
Ricardo Iglesias
5578ba6eed base32: move from getopts to clap
Note, I needed to change the error messages in one of the tests because
getopt and clap have different error messages when not providing a
default value
2021-04-25 22:24:55 -07:00
electricboogie
c01c6a7d78 Ran rustfmt 2021-04-25 22:41:11 -05:00
electricboogie
6654519c7d Specify a default tempdir for Windows 2021-04-25 22:39:17 -05:00
electricboogie
e5c19734c8 Change Default Buffer to usize::MAX 2021-04-25 21:38:22 -05:00
electricboogie
1a407c2328 Set a dynamic minimum buffer size 2021-04-25 21:17:56 -05:00
electricboogie
8e258075f6 Potential fix to tests on Windows 2021-04-25 19:21:19 -05:00
electricboogie
fc899ffe7a Implement a minimum readback buffer 2021-04-25 19:07:24 -05:00
electricboogie
32222c1ee7 Remove unneeded condition for use of NumCache 2021-04-25 17:52:20 -05:00
electricboogie
0f707cdb25 Adjust max buffer size for read back as well 2021-04-25 16:33:12 -05:00
Christopher Regali
368e984fac
Change unchecked unwrapping to unwrap_or_default for Args-trait (#1845) (#1852)
* Change unchecked unwrapping to unwrap_or_default for argument parsing (resolving #1845)

* Added unit-testing for the collect_str function on invalid utf8 OsStrs

* Added a warning-message for identification purpose to the collect_str method.

* - Add removal of wrongly encoded empty strings to basename
- Add testing of broken encoding to basename
- Changed UCommand to use collect_str in args method to allow for integration testing of that method
- Change UCommand to use unwarp_or_default in arg method to match the behaviour of collect_str

* Trying out a new pattern for convert_str for getting a feeling of how the API feels with more control

* Adding convenience API for compact calls

* Add new API to everywhere, fix test for basename

* Added unit-testing for the conversion options

* Added unit-testing for the conversion options for windows

* fixed compilation and some merge hiccups

* Remove windows tests in order to make merge request build

* Fix formatting to match rustfmt for the merged file

* Improve documentation of the collect_str method and the unit-tests

* Fix compilation problems with test

Co-authored-by: Christopher Regali <chris.vdop@gmail.com>
Co-authored-by: Sylvestre Ledru <sylvestre@debian.org>
2021-04-25 23:28:42 +02:00
electricboogie
6f82cd4f15 Fix errors for usize on 32bit platforms 2021-04-25 16:27:36 -05:00
electricboogie
dbdac22262 Add back unstable sort 2021-04-25 15:48:20 -05:00
electricboogie
5fb7014c2b Add a BufWriter for writes out to temp files 2021-04-25 15:42:36 -05:00
electricboogie
733949b2e7 Add dynamic buffer adjustment, fix test comment 2021-04-25 15:13:27 -05:00
Ricardo Iglesias
c3d7358df6
ls: ignore leading period when sorting by name (#2112)
* ls: ignore leading period when sorting by name

ls now behaves like GNU ls with respect to sorting files by ignoring
leading periods when sorting by main.

Added tests to ensure "touch a .a b .b ; ls" returns ".a  a  .b  b"

* Replaced clone/collect calls.
2021-04-25 21:08:05 +02:00
Alessandro Stoltenberg
43f3f7e01c feat2: Rebased on current master and incorporated changes done to the filetype handling. 2021-04-25 20:13:42 +02:00
electricboogie
2f37b85426 unwrap_or_else can be an unwrap_or 2021-04-25 12:58:04 -05:00
Alessandro Stoltenberg
9c221148a8 ls: Extension sorting, use file_stem() instead of to_string_lossy() 2021-04-25 19:45:59 +02:00
Alessandro Stoltenberg
bbcca3eefd ls: Implements https://github.com/uutils/coreutils/issues/1880 extension sorting. 2021-04-25 19:45:59 +02:00
electricboogie
f0a473f40e Fix tests 2021-04-25 12:38:43 -05:00
electricboogie
094d9a9e47 Fix bug in human_numeric convert 2021-04-25 12:27:11 -05:00
electricboogie
4c395146dd Merge branch 'master' of https://github.com/uutils/coreutils 2021-04-25 10:11:27 -05:00
electricboogie
26fc8e57c7 Fix NumCache and Serde JSON conflict by disabling NumCache during extsort general numeric compares 2021-04-25 10:03:29 -05:00
Sylvestre Ledru
e667cc2641
Merge pull request #2115 from tertsdiepraam/ls/reduce_write_calls
`ls`: reduce write syscalls & cleanup
2021-04-25 11:52:51 +02:00
Sylvestre Ledru
c19e191360
Merge pull request #2113 from siebenHeaven/ls-optimize-sort
ls: Use sort_by_cached_key
2021-04-25 11:13:23 +02:00
Terts Diepraam
fc6c7a279e ls: clean up imports 2021-04-25 10:46:51 +02:00
Anup Mahindre
7e06316ece ls: Use sort_by_cached_key 2021-04-25 13:37:07 +05:30
Sylvestre Ledru
441763b73d
Merge pull request #2059 from cbjadwani/master
uniq: avoid building list of duplicate lines
2021-04-25 09:48:48 +02:00
Sylvestre Ledru
d3775ea0e8
Merge pull request #2110 from nthery/cp_reflink_macos
cp: add  --reflink support to macos, fixes #1773
2021-04-25 09:28:14 +02:00
electricboogie
2b8a6e98ee Working ExtSort 2021-04-25 00:20:56 -05:00
Terts Diepraam
e995eea579 ls: general cleanup 2021-04-25 00:23:14 +02:00
Terts Diepraam
ce04f8a759 ls: use bufwriter to write stdout 2021-04-24 23:46:19 +02:00
Nicolas Thery
4bf33e98a8 cp: add --reflink support for macOS
Fixes #1773
2021-04-24 19:26:15 +02:00
Nicolas Thery
b8e23c20c2 cp: extract linux COW logic into function 2021-04-24 19:22:12 +02:00
Chirag Jadwani
2c1459cbfc cut: optimizations
* Use buffered stdout to reduce write sys calls.

This simple change yielded the biggest performace gain.

* Use `for_byte_record_with_terminator` from the `bstr` crate.

This is to minimize the per line copying needed by
`BufReader::read_until`. The `cut_fields` and `cut_fields_delimiter`
functions used `read_until` to iterate over lines. That required copying
each input line to the line buffer. With
`for_byte_record_with_terminator` copying is minimized as it calls our
closure with a reference to BufReader's buffer most of the time.  It
needs to copy (internally) only to process any incomplete lines at the
end of the buffer.

* Re-write `Searcher` to use `memchr`.

Switch from the naive implementation to one that uses `memchr`.

* Rewrite `cut_bytes` almost entirely.

This was already well optimized. The performance gain in this case is
not from avoiding copying. In fact, it needed zero copying whereas new
implementation introduces some copying similar to `cut_fields` described
above. But the occassional copying cost is more than offset by the use
of the very fast `memchr` inside `for_byte_record_with_terminator`.
This change also simplifies the code significantly. Removed the `buffer`
module.
2021-04-24 22:29:48 +05:30
Sylvestre Ledru
2f17bfc14c
Merge pull request #2106 from miDeb/sort-debug
sort: implement --debug
2021-04-24 18:46:58 +02:00
Sylvestre Ledru
c9b0378ca3
Merge pull request #2104 from tertsdiepraam/ls/skip_metadata
`ls`: skip reading metadata
2021-04-24 18:13:53 +02:00
Sylvestre Ledru
d7e8a03237
Merge pull request #2097 from miDeb/sort-disable-dictionary-mode
sort: disallow certain flags with -d and -i
2021-04-24 14:58:32 +02:00
Sylvestre Ledru
b41951614b
Merge branch 'master' into sort-disable-dictionary-mode 2021-04-24 13:56:39 +02:00
Terts Diepraam
1328d18878 ls: remove outdated comment 2021-04-24 13:19:50 +02:00
Michael Debertol
5dcfb51110 flip default for debug to the effective default 2021-04-24 10:52:40 +02:00
Terts Diepraam
728f0bd61d ls: remove redundant parentheses 2021-04-24 10:47:36 +02:00
Terts Diepraam
ce8c58b93e Merge branch 'master' into ls/skip_metadata 2021-04-24 10:45:43 +02:00
Sylvestre Ledru
8ccc6ade61
Merge branch 'master' into split-wsl-detection 2021-04-24 10:24:13 +02:00
Sylvestre Ledru
9517395839
Merge pull request #2088 from nthery/cp_reflink_never
cp: add support for --reflink=never
2021-04-24 10:07:41 +02:00
Sylvestre Ledru
fb6394554e
Merge pull request #2096 from tertsdiepraam/ls/fix_backslash_escape
ls: improve code cov
2021-04-24 10:05:32 +02:00
Sylvestre Ledru
513ff4e45f
Merge branch 'master' into sort-disable-dictionary-mode 2021-04-24 10:04:23 +02:00
Sylvestre Ledru
b96f7dbaea
Merge pull request #2087 from pedrohjordao/printf-clap-opts
Changes parameter parsing to clap
2021-04-24 10:02:05 +02:00
Sylvestre Ledru
372d08c341
Merge pull request #2098 from miDeb/sort-trailing-separator
sort: fix tokenization for trailing separators
2021-04-24 10:00:20 +02:00
Sylvestre Ledru
a9fa4adddf
Merge pull request #2102 from jaggededgedjustice/fix-tail-sleep-interval
tail --sleep-interval takes a value
2021-04-24 09:59:03 +02:00