u-boot/common
Tom Rini 2b28c3b871 Merge patch series "Modernize U-Boot shell"
Francis Laniel <francis.laniel@amarulasolutions.com> says:

During 2021 summer, Sean Anderson wrote a contribution to add a new shell, based
on LIL, to U-Boot [1, 2].
While one of the goals of this contribution was to address the fact actual
U-Boot shell, which is based on Busybox hush, is old there was a discussion
about adding a new shell versus updating the actual one [3, 4].

So, in this series, with Harald Seiler, we updated the actual U-Boot shell to
reflect what is currently in Busybox source code.
Basically, this contribution is about taking a snapshot of Busybox shell/hush.c
file (as it exists in commit 37460f5da) and adapt it to suit U-Boot needs.

This contribution was written to be as backward-compatible as possible to avoid
breaking the existing.
So, the modern hush flavor offers the same as the actual, that is to say:
1. Variable expansion.
2. Instruction lists (;, && and ||).
3. If, then and else.
4. Loops (for, while and until).
No new features offered by Busybox hush were implemented (e.g. functions).

It is possible to change the parser at runtime using the "cli" command:
=> cli print
old
=> cli set modern
=> cli print
modern
=> cli set old
The default parser is the old one.
Note that to use both parser, you would need to set both
CONFIG_HUSH_MODERN_PARSER and CONFIG_HUSH_OLD_PARSER.

In terms of testing, new unit tests were added to ut to ensure the new behavior
is the same as the old one and it does not add regression.
Nonetheless, if old behavior was buggy and fixed upstream, the fix is then added
to U-Boot [5].
In sandbox, all of these tests pass smoothly:
=> printenv board
board=sandbox
=> ut hush
Running 20 hush tests
...
Failures: 0
=> cli set modern
=> ut hush
Running 20 hush tests
...
Failures: 0

Thanks to the effort of Harald Seiler, I was successful booting a board:
=> printenv fdtfile
fdtfile=amlogic/meson-gxl-s905x-libretech-cc.dtb
=> cli get
old
=> boot
...
root@lepotato:~#
root@lepotato:~# reboot
...
=> cli set modern
=> cli get
modern
=> printenv fdtfile
fdtfile=amlogic/meson-gxl-s905x-libretech-cc.dtb
=> boot
...
root@lepotato:~#

This contribution indeed adds a lot of code and there were concern about its
size [6, 7].
With regard to the amount of code added, the cli_hush_upstream.c is 13030 lines
long but it seems a smaller subset is really used:
gcc -D__U_BOOT__ -E common/cli_hush_upstream.c | wc -l
2870
Despite this, it is better to still have the whole upstream code for the sake of
easing maintenance.
With regard to memory size, I conducted some experiments for version 8 of this
series and for a subset of arm64 boards and found the worst case to be 4K [8].
Tom Rini conducted more research on this and also found the increase to be
acceptable [9].

If you want to review it - your review will really be appreciated - here are
some information regarding the commits:
* commits marked as "test:" deal with unit tests.
* commit "cli: Add Busybox upstream hush.c file." copies Busybox shell/hush.c
into U-Boot tree, this explain why this commit contains around 12000 additions.
* commit "cli: Port Busybox 2021 hush to U-Boot." modifies previously added file
to permit us to use this as new shell.
The really good idea of #include'ing Busybox code into a wrapper file to define
some particular functions while minimizing modifications to upstream code comes
from Harald Seiler.
* commit "cmd: Add new parser command" adds a new command which permits
selecting parser at runtime.
I am not really satisfied with the fact it calls cli_init() and cli_loop() each
time the parser is set, so your reviews would be welcomed.
* Other commits focus on enabling features we need (e.g. if).
2023-12-28 14:38:25 -05:00
..
eeprom SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
init spl: Use SYS_MALLOC_F instead of SYS_MALLOC_F_LEN 2023-10-06 14:38:12 -04:00
spl Prepare v2024.01-rc5 2023-12-18 09:55:32 -05:00
autoboot.c autoboot: Rework CONFIG_AUTOBOOT_STOP_STR_* usage 2023-01-20 12:27:06 -05:00
avb_verify.c common: avb_verify: prevent opening incorrect session 2023-02-17 10:51:28 -05:00
bloblist.c bloblist: Add missing name 2023-10-06 14:38:13 -04:00
board_f.c bloblist: Support initing from multiple places 2023-10-06 14:38:13 -04:00
board_info.c sysinfo: Allow displaying more info on startup 2023-11-28 12:53:04 -05:00
board_r.c scsi: Forceably finish migration to DM_SCSI 2023-11-07 18:36:06 -05:00
bootstage.c bootstage: Correct exhasuted typo 2023-11-14 20:04:01 -07:00
bouncebuf.c common: bouncebuf: Add missing cast to dma_addr_t 2023-08-22 15:17:52 -04:00
cli.c cli: add modern hush as parser for run_command*() 2023-12-28 12:02:56 -05:00
cli_getch.c cli: Consume invalid escape sequences early 2023-10-24 16:34:45 -04:00
cli_hush.c common: hush: Remove unused NEEDS_MANUAL_RELOC code bits 2023-09-14 10:42:24 -04:00
cli_hush_modern.c Merge patch series "Modernize U-Boot shell" 2023-12-28 14:38:25 -05:00
cli_hush_upstream.c cli: modern_hush: Add upstream commits up to 2nd October 2023. 2023-12-28 12:02:56 -05:00
cli_readline.c command: Allocate history buffer using calloc() 2023-12-20 10:46:54 -05:00
cli_simple.c cli_simple: Rework this support slightly 2023-11-07 14:48:51 -05:00
command.c command: Move command completion temporary buffer to stack 2023-12-20 10:46:54 -05:00
console.c video: Add a function to clear the display 2023-12-13 18:39:05 -05:00
cros_ec.c common: Drop log.h from common header 2020-05-18 21:19:18 -04:00
cyclic.c cyclic: get rid of cyclic_init() 2022-11-02 08:42:03 +01:00
ddr_spd.c SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
dfu.c common: Drop linux/printk.h from common header 2023-09-24 09:54:57 -04:00
dlmalloc.c malloc: Enable assertions if UNIT_TEST is enabled 2023-10-11 13:22:29 -04:00
dlmalloc.src Consolidate bool type 2013-04-01 16:33:52 -04:00
edid.c doc: replace @return by Return: 2022-01-19 18:11:34 +01:00
event.c event: add new EVT_SETTINGS_R event 2023-10-23 17:07:06 +03:00
exports.c common: Drop asm/global_data.h from common header 2021-02-02 15:33:42 -05:00
flash.c Audit <flash.h> inclusion 2022-08-04 16:18:47 -04:00
hash.c common: hash: Remove unused NEEDS_MANUAL_RELOC code bits 2023-09-14 10:42:24 -04:00
hwconfig.c hwconfig: Allow to use restricted env 2022-08-20 18:12:51 -04:00
iomux.c IOMUX: Fix buffer overflow in iomux_replace_device() 2021-04-27 08:05:30 -04:00
iotrace.c common: Drop asm/global_data.h from common header 2021-02-02 15:33:42 -05:00
kallsyms.c global: Convert simple_strtoul() with hex to hextoul() 2021-08-02 13:32:14 -04:00
Kconfig Kconfig: Move CONFIG_SYS_[CP]BSIZE to common/Kconfig 2023-11-07 14:48:19 -05:00
kgdb.c common: board_r: drop initr_kgdb wrapper 2022-01-18 08:31:02 -05:00
kgdb_stubs.c common: Drop asm/ptrace.h from common header 2020-05-18 21:19:23 -04:00
log.c log: Tidy up an ambiguous comment 2023-07-17 17:12:07 +08:00
log_console.c log: Support outputing function names in SPL 2023-07-17 17:23:15 +08:00
log_syslog.c common: Drop asm/global_data.h from common header 2021-02-02 15:33:42 -05:00
main.c bootstd: Introduce programmatic boot 2023-12-13 11:51:24 -05:00
Makefile cli: Enables using modern hush parser as command line parser 2023-12-28 12:02:56 -05:00
malloc_simple.c malloc: Annotate allocator for valgrind 2022-04-11 10:00:30 -04:00
memsize.c common/memsize.c: Fix get_ram_size() when cache is enabled 2023-06-21 16:54:25 -04:00
menu.c menu: Re-enable the ANSI codes 2023-06-23 14:38:16 -04:00
miiphyutil.c Convert CONFIG_PHY_RESET_DELAY to Kconfig 2022-03-25 12:01:15 +00:00
qfw.c qemu: Move qfw kernel setup into a common file 2023-02-06 13:04:53 -05:00
s_record.c SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
scp03.c common: SCP03 control (enable and provision of keys) 2021-03-13 13:14:52 -05:00
splash.c common: Replace #ifdef and #if with if's 2023-04-24 21:37:45 +02:00
splash_source.c common: splash_source: Fix type casting errors 2023-07-21 13:13:43 -04:00
stackprot.c Fix stack-protector for powerpc 2023-11-29 09:32:15 -05:00
stdio.c stdio: fix stdio_deregister_dev() 2023-10-09 15:24:31 -04:00
system_map.c Add support for Linux-like kallsysms 2009-06-12 20:45:48 +02:00
update.c common: Drop linux/printk.h from common header 2023-09-24 09:54:57 -04:00
usb.c usb: Tidy up the usb_start flag 2023-05-13 09:52:32 -04:00
usb_hub.c usb: hub: Add missing reset recovery delay 2023-12-01 14:06:12 +01:00
usb_kbd.c cyclic: Use schedule() instead of WATCHDOG_RESET() 2022-09-18 10:26:33 +02:00
usb_onboard_hub.c usb: onboard-hub: add driver to manage onboard hub supplies 2023-01-12 16:19:53 +01:00
usb_storage.c usb: storage: Use the correct CBW lengths 2023-12-01 14:06:12 +01:00
version.c version: Separate our version string from the version command 2023-11-07 14:48:19 -05:00
xyzModem.c cyclic: Use schedule() instead of WATCHDOG_RESET() 2022-09-18 10:26:33 +02:00