Mirrors/coreutils
2
0
Fork 0
mirror of https://github.com/uutils/coreutils synced 2024-12-17 16:43:16 +00:00
Code Issues Projects Releases Packages Wiki Activity
coreutils/src/uu/tail
History
Jan Scheer 75edeea5e4
tail: reduce CPU load for polling (#3618) 
* tail: reduce CPU load for polling

This reduces the CPU load for polling drastically (from ~80% down to ~5%)
by removing/fixing several previous workarounds related to polling,
while still passing all related GNU test-suite checks.
* set Notify::PollWatcher delay to: sleep_sec/10 instead of
  sleep_sec/100
* set recv_timeout to sleep_sec instead of sleep_sec/100
* remove the manual polling of watched files

Bugs:
* fix an issue with headers to consistently pass
"test_follow_name_retry_headers" and "gnu/tests/tail-2/overlay-headers.sh"

Code clean-up and refactor
* make fields of struct FileHandling private (and add getters/setters)
to ensure that the paths are absolute and match the paths returned by
Notify::Events
* replace calls to "crash!" with "return USimpleError"
* clean-up formatting
2022-06-21 22:21:19 +02:00
..
src tail: reduce CPU load for polling (#3618) 2022-06-21 22:21:19 +02:00
Cargo.toml tail: reduce CPU load for polling (#3618) 2022-06-21 22:21:19 +02:00
LICENSE Include license text in all published crates 2022-03-05 21:21:46 +01:00
README.md tail: reduce CPU load for polling (#3618) 2022-06-21 22:21:19 +02:00

README.md

Notes / ToDO

Missing features

  • --max-unchanged-stats

Note: There's a stub for --max-unchanged-stats so GNU test-suite checks using it can run, however this flag has no functionality yet.

Platform support for --follow and --retry

The --follow=descriptor, --follow=name and --retry flags have very good support on Linux (inotify backend). They work good enough on macOS/BSD (kqueue backend) with some tests failing due to differences of how kqueue works compared to inotify. Windows support is there in theory due to ReadDirectoryChanges support by the notify-crate, however these flags are completely untested on Windows.

Note: The undocumented ---disable-inotify flag is used to disable the inotify backend to test polling. However inotify is a Linux only backend and polling is now supported also for the other backends. Because of this, disable-inotify is now an alias to the new and more versatile flag name: --use-polling.

Possible optimizations

  • Don't read the whole file if not using -f and input is regular file. Read in chunks from the end going backwards, reading each individual chunk forward.
  • Reduce number of system calls to e.g. fstat
  • Improve resource management by adding more system calls to inotify_rm_watch when appropriate.

GNU test-suite results (9.1.8-e08752)

The functionality for the test "gnu/tests/tail-2/follow-stdin.sh" is implemented. It fails because it is provoking closing a file descriptor with tail -f <&- and as part of a workaround, Rust's stdlib reopens closed FDs as /dev/null which means uu_tail cannot detect this. See also, e.g. the discussion at: https://github.com/uutils/coreutils/issues/2873

The functionality for the test "gnu/tests/tail-2/inotify-rotate-resources.sh" is implemented. It fails with an error because it is using strace to look for calls to inotify_add_watch and inotify_rm_watch, however in uu_tail these system calls are invoked from a separate thread. If the GNU test would follow threads, i.e. use strace -f, this issue could be resolved.

There are 5 tests which are fixed but do not (always) pass the test suite if it's run inside the CI. The reason for this is probably related to load/scheduling on the CI test VM. The tests in question are:

  • tail-2/F-vs-rename.sh
  • tail-2/follow-name.sh
  • tail-2/inotify-rotate.sh
  • tail-2/overlay-headers.sh
  • tail-2/retry.sh