Commit graph

372 commits

Author SHA1 Message Date
Jean-Jacques Hiblot
09da18deab test/py: add MMC/SD block write test
Add a standalone MMC block write test. This allows direct testing of MMC
access rather than relying on doing so as a side-effect of e.g. DFU or
UMS testing, which may not be enabled on all platforms.

Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2019-07-31 15:31:36 +08:00
Julius Werner
b1307f884a fit: Support compression for non-kernel components (e.g. FDT)
This patch adds support for compressing non-kernel image nodes in a FIT
image (kernel nodes could already be compressed previously). This can
reduce the size of FIT images and therefore improve boot times
(especially when an image bundles many different kernel FDTs). The
images will automatically be decompressed on load.

This patch does not support extracting compatible strings from
compressed FDTs, so it's not very helpful in conjunction with
CONFIG_FIT_BEST_MATCH yet, but it can already be used in environments
that select the configuration to load explicitly.

Signed-off-by: Julius Werner <jwerner@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Goldschmidt <simon.k.r.goldschmidt@gmail.com>
2019-07-29 09:32:07 -04:00
Ruslan Trofymenko
31a38799d5 test/py: Add base test case for A/B updates
Add sandbox test for 'ab_select' command.

Test: ./test/py/test.py --bd sandbox --build -k test_ab

Signed-off-by: Ruslan Trofymenko <ruslan.trofymenko@linaro.org>
Signed-off-by: Igor Opaniuk <igor.opaniuk@gmail.com>
Signed-off-by: Sam Protsenko <semen.protsenko@linaro.org>
Reviewed-by: Alistair Strachan <astrachan@google.com>
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-07-24 13:16:29 -04:00
Eugeniu Rosca
cef4de8835 treewide: Fix stale references of Android docs
Commit 9bdf0e8fef ("doc: relocate/rename Android README and add BCB
overview") left some obsolete references of Android documents/paths.

This has been pointed out by Sam (thanks!) in:
https://patchwork.ozlabs.org/patch/1104245/#2208134

Fixes: 9bdf0e8fef ("doc: relocate/rename Android README and add BCB overview")
Reported-by: Sam Protsenko <semen.protsenko@linaro.org>
Suggested-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Eugeniu Rosca <erosca@de.adit-jv.com>
Reviewed-by: Igor Opaniuk <igor.opaniuk@gmail.com>
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
2019-07-24 13:16:29 -04:00
Sam Protsenko
bdfc9e8ad7 test/py: gpt: Use long options for sgdisk
sgdisk 0.8.10.2 from AOSP doesn't support short options, failing with
errors like this:

    sgdisk: invalid option -- 'U'

Test fails due to that error. Let's use long options to make the test
work with any sgdisk version.

Signed-off-by: Sam Protsenko <semen.protsenko@linaro.org>
Acked-by: Stephen Warren <swarren@nvidia.com>
2019-07-18 11:31:26 -04:00
Patrick Delaunay
099ed45c6c test: check u-boot properties in SPL device tree
Add a test to check the management of the U-boot relocation properties
for device tree SPL generation (fdtgrep result) and platdata:
- 'dm-pre-proper' and 'dm-tpl' not included in SPL
- 'dm-pre-reloc' and 'dm-spl' included in SPL

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:58 -06:00
Heinrich Schuchardt
0b1da53e53 test/py: not all boards support UEFI runtime reset
As not all boards support resets at runtime do not test for it in the
Python tests.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-07-06 21:25:32 +02:00
Heinrich Schuchardt
983142dff8 test/py: error message test_efi_selftest_device_tree
Correct the error message in test_efi_selftest_device_tree().

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-07-06 21:25:32 +02:00
Stephen Warren
89a5317ae2 test/py: don't use mmc_rd config for other mmc tests
Fix test_mmc_dev(), test_mmc_rescan(), test_mmc_info() not to use the
same configuration data that test_mmc_rd() does. Doing so causes the
following issues:

* The new code uncondtionally expects certain keys to exist in the
configuration data. These keys do not exist in existing configuration
data since they were not previously required, and there was no
notification re: a requirement to add these new keys. This causes test
failures due to thrown exceptions when accessing the non-existent keys.

* The new tests logically operate on different objects. test_mmc_rd()
operates on ranges of sectors on an MMC device (which may be the entire
set of sectors of a device, or a part of a device), whereas all the new
tests operate solely on entire devices. These are separate things, and
it's entirely likely that the user will wish to runs the two types of
tests on different sets of data; see the example configuration data that
this commit adds. Ideally, the new tests would have been added to a
separate Python file, since they aren' closely related to the existing
tests.

FIXME: Marek, can you please replace the "???" in this patch with some
reasonable looking data? Thanks.

Cc: Marek Vasut <marek.vasut@gmail.com>
Fixes: 4ffec8cdf5 ("test/py: mmc: Add 'mmc info' test")
Fixes: ce4b2cafa7 ("test/py: mmc: Add 'mmc rescan' test")
Fixes: 86dfd152c9 ("test/py: mmc: Add 'mmc dev' test")
Signed-off-by: Stephen Warren <swarren@nvidia.com>
2019-05-09 19:52:55 -04:00
Igor Opaniuk
fc1fe01b08 avb: add support for named persistent values
AVB 2.0 spec. revision 1.1 introduces support for named persistent values
that must be tamper evident and allows AVB to store arbitrary key-value
pairs [1].

Introduce implementation of two additional AVB operations
read_persistent_value()/write_persistent_value() for retrieving/storing
named persistent values.

Correspondent pull request in the OP-TEE OS project repo [2].

[1]: https://android.googlesource.com/platform/external/avb/+/android-9.0.0_r22
[2]: https://github.com/OP-TEE/optee_os/pull/2699

Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Sam Protsenko <semen.protsenko@linaro.org>
Signed-off-by: Igor Opaniuk <igor.opaniuk@gmail.com>
2019-04-26 18:58:22 -04:00
Heinrich Schuchardt
68066d5bcd efi_selftest: do not run FDT test with ACPI table.
The EBBR specification prescribes that we should have either an ACPI table
or a device tree but not both.

So do not run the device tree unit test on boards with an ACPI table.
Hence there is no need any longer to make it 'on request' only.
Do not pass $fdtcontroladdr to `bootefi selftest`.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-04-23 00:37:28 +02:00
Heinrich Schuchardt
10feb30c70 test/py: pytest.mark.notbuildconfigspec()
We already can let a Python test depend on a build option being set via
@pytest.mark.buildconfigspec(). It may be necessary to let a test depend on
a build option *not* being set. So let's introduce

    @pytest.mark.notbuildconfigspec

for this purpose.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-04-23 00:37:28 +02:00
Michal Simek
963482120d test: py: Extend fpga test with fit image with external data
Images are created
mkimage -f fit.its -E  download-fit-external.ub

and test expects these entries.

env__fpga_under_test = {
    ...
    "mkimage_fit_external": download-fit-external.ub",
    "mkimage_fit_external_size": xxxxx,
    ...
}

Test download file and loads it to fpga.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-04-16 11:51:34 +02:00
Tom Rini
0a8406602a test.py: Disable fsck for FAT tests for now
Currently enabling fsck on FAT16/FAT32 exposes that we have problems
with:
TestFsBasic.test_fs13[fat16]
TestFsBasic.test_fs11[fat32]
TestFsBasic.test_fs12[fat32]
TestFsBasic.test_fs13[fat32]
TestFsExt.test_fs_ext1[fat32]
TestFsExt.test_fs_ext2[fat32]
TestFsExt.test_fs_ext3[fat32]
TestFsExt.test_fs_ext4[fat32]
TestFsExt.test_fs_ext5[fat32]
TestFsExt.test_fs_ext6[fat32]
TestFsExt.test_fs_ext7[fat32]
TestFsExt.test_fs_ext8[fat32]
TestFsExt.test_fs_ext9[fat32]
TestMkdir.test_mkdir6[fat16]
TestMkdir.test_mkdir1[fat32]
TestMkdir.test_mkdir2[fat32]
TestMkdir.test_mkdir3[fat32]
TestMkdir.test_mkdir4[fat32]
TestMkdir.test_mkdir5[fat32]
TestMkdir.test_mkdir6[fat32]
TestUnlink.test_unlink1[fat16]
TestUnlink.test_unlink2[fat16]
TestUnlink.test_unlink3[fat16]
TestUnlink.test_unlink4[fat16]
TestUnlink.test_unlink5[fat16]
TestUnlink.test_unlink6[fat16]
TestUnlink.test_unlink7[fat16]
TestUnlink.test_unlink1[fat32]
TestUnlink.test_unlink2[fat32]
TestUnlink.test_unlink3[fat32]
TestUnlink.test_unlink4[fat32]
TestUnlink.test_unlink5[fat32]
TestUnlink.test_unlink6[fat32]
TestUnlink.test_unlink7[fat32]

This is because we don't update the "information sector" on FAT32.
While in the future we should resolve this problem and include that
feature, we should enable fsck for ext4 to ensure that things remain in
good shape there.

Signed-off-by: Tom Rini <trini@konsulko.com>
2019-04-10 08:15:56 -04:00
Marek Vasut
e551979790 test/py: mmc: Add 'mmc read' performance check
Add option to the mmc rd test to check the duration of the
execution of the mmc read command. This allows intercepting
read performance regressions.

Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Tom Rini <trini@konsulko.com>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-04-09 20:04:05 -04:00
Marek Vasut
4ffec8cdf5 test/py: mmc: Add 'mmc info' test
Add test for 'mmc info' subcommand. This tests whether the card
information is obtained correctly and verifies the device, bus
speed, bus mode and bus width.

Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Tom Rini <trini@konsulko.com>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-04-09 20:04:05 -04:00
Marek Vasut
ce4b2cafa7 test/py: mmc: Add 'mmc rescan' test
Add test for 'mmc rescan' subcommand. This tests whether the
system can switch to a specific card and then rescan the card.

Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Tom Rini <trini@konsulko.com>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-04-09 20:04:05 -04:00
Marek Vasut
86dfd152c9 test/py: mmc: Add 'mmc dev' test
Add separate test for 'mmc dev' subcommand. This tests whether
the system can switch to a specific card.

Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Tom Rini <trini@konsulko.com>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-04-09 20:04:04 -04:00
Marek Vasut
d22f7ad86a test/py: mmc: Factor out device selection
Factor out the 'mmc dev' call so it can be recycled by other tests.

Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Tom Rini <trini@konsulko.com>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-04-09 20:04:04 -04:00
Jean-Jacques Hiblot
ef79284e7a test: fs: Added tests for symlinks
Test cases are:
1) basic link creation, verify it can be followed
2) chained links, verify it can be followed
3) replace exiting file a with a link, and a link with a link. verify it
   can be followed
4) create a broken link, verify it can't be followed

Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2019-04-09 20:04:03 -04:00
Jean-Jacques Hiblot
290100583d test: fs: Add filesystem integrity checks
We need to make sure that file writes,file creation, etc. are properly
performed and do not corrupt the filesystem.
To help with this, introduce the assert_fs_integrity() function that
executes the appropriate fsck tool. It should be called at the end of any
test that modify the content/organization of the filesystem.
Currently only supports FATs and EXT4.

Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2019-04-09 15:34:15 -04:00
Jean-Jacques Hiblot
5cfc73e6e2 test: fs: disable the metadata checksums on ext4 filesystems
If the metadata checksums are enabled, all write operations will fail.

Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2019-04-09 15:34:15 -04:00
Heinrich Schuchardt
bfc2dd5381 efi_loader: correct CTRL-A - CTRL-Z console input
In the extended text input protocol CTRL-A - CTRL-Z have to be signaled as
Unicode characters a-z or A-Z depending on the shift state and not as 0x01
to 0x1a.

Update Python unit test.

This patch is required for using the EFI shell `edit` command.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-04-07 15:11:33 +02:00
Igor Opaniuk
7374b15522 test: let use gdbserver for all sandbox targets
Enable usage of gdbserver for all sandbox targets (sandbox,
sandbox_flattree etc.).

Signed-off-by: Igor Opaniuk <igor.opaniuk@linaro.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-02-19 08:55:43 -05:00
Heinrich Schuchardt
b1b1bab7f9 test/py: use default load address for tftp
On x86_64 the size of the file u-boot loaded by the tftp test has grown in
size such that when loading the file to 0x200000 it overwrites a memory
area reserved for PCI.

If no load address is specified for tftp do not use the ram base address
(or if zero 0x200000) but the default address.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-02-12 07:19:24 -05:00
Tom Rini
aac0c29d4b Fix recent changes to serial API for driver model
Buildman clang support and a few fixes
 Small fixes to 'dm tree' and regmap test
 Improve sandbox build compatibility
 A few other minor fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEslwAIq+Gp8wWVbYnfxc6PpAIreYFAlw+AlIACgkQfxc6PpAI
 reYchAf+L/P5JsLoCKAY84v11eWQkCS44idAYStD8Q023dyuyNC/W6G3qEQVGkHu
 SXBzNrZ5wSJanN5I3h3ktJ7yfMOYIvSO2qLHphxDstPaZbCY4Zac6NywHrHxQpA9
 fbSilQRbcVWrPo5rsjjZhXmMdQRZVOmGD6CYZj1AihWAiHfYth7f6laNWNTWQA8C
 z+aoiF70t/PbvwFbgdzzYOjoGdXI9XML0xTqdLWRlsBKjf3z54pCV6LmN2xrsrDZ
 k7lr+x7ajJUFhivxzjowf4aOjhDB+/+I3sr+hJ5vkMM0Kqg14bbEU+xJRGpDc18K
 QPvVHT5JGe/nq1cZXuCqzrYxTrrA7A==
 =w9ln
 -----END PGP SIGNATURE-----

Merge tag 'dm-pull-15jan19' of git://git.denx.de/u-boot-dm

Fix recent changes to serial API for driver model
Buildman clang support and a few fixes
Small fixes to 'dm tree' and regmap test
Improve sandbox build compatibility
A few other minor fixes
2019-01-15 22:05:34 -05:00
Simon Glass
871bf7d9ba test: Use single quote consistently
Some tests have ended up using double quotes where single quotes could be
used. Adjust this for consistency with the rest of U-Boot's Python code.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2019-01-15 15:28:41 -05:00
Simon Glass
54d2cfe6ee dm: Tidy up 'dm tree' output when there are many devices
At present the 'Index' column assumes there is only one digit. But on some
devices (e.g. snow) there are a lot of regulators and GPIO banks. Adjust
the output to allow for two digits without messing up the display.

Also capatalise the heading to match.

Fixes: 5197dafc42 (dm: core: Widen the dump tree to show more of the
driver's name.)

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Liviu Dudau <liviu.dudau@foss.arm.com>
2019-01-14 17:47:13 -07:00
Heiko Schocher
8fb2391ea6 spl/tpl: change banner into upper case
commit d633006463 ("spl: Add a define for SPL_TPL_PROMPT")

changes the SPL/TPL banner from upper case into lower
case. As SPL and TPL are three-letter acronyms and they
are written in upper case, change it back to upper case.

Signed-off-by: Heiko Schocher <hs@denx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-12-07 08:13:51 -05:00
Tom Rini
f388e3bed7 Patch queue for efi - 2018-12-03
This release is fully packed with lots of glorious improvements in UEFI
 land again!
 
   - Make PE images more standards compliant
   - Improve sandbox support
   - Improve correctness
   - Fix RISC-V execution on virt model
   - Honor board defined top of ram (fixes a few boards)
   - Imply DM USB access when distro boot is available
   - Code cleanups
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJcBYKAAAoJECszeR4D/txgBgwQALmYioI67R5/Iizpv7bg+rIQ
 0TyPKZHmfHtVjGHd5X4X+9NBsoaSKqGnoI12bJ+V9hIMuiu5qFKyM3icTOOJ6LI6
 wggnvMWZl5nfZmdEgETHTmaZkQZzKwhzbftGlGf2j19FdDk1OOI7hRNLeaIZUTv0
 VHiUV68PP/1Of1y7iqB5jij1wTUHWlCufKjGXELP0bAXx86/tecgCuvjBihXizz/
 sMsCxEF8++pb5l/l4yFEkKd5rr5D/ZkKMLR4KniZVq2qP1S4calolP14ykHN+a/l
 uKP4e4GDuYyrgXXTNRFhVTlaAn18bmvxH4ialnpYVZKRtfsdHPHQXfqmgf8ZgJPE
 JK3mmG6eLCbBPeND2Yz3b7G/Ec04z+RJXx0hriyLdejLgp5jM9SBtygiz6FmQLpQ
 VfDJNEWV7ot6Ejou55O0d9u5ATF0jAd4tikmsrStWWZOVHvie6nG0wFYiRxnWCKP
 sid0p7lWSUKEl0sAvA0LglNMzd4tCAq7vtkfLj/BVrDc9Jpir9CVJ13ppXIGk1HC
 YIGWLo0uXAGC9wgRE7ZgGCKtQ6VFZRbSiJQOowi4MrHzHXH218oSNz2w25tAVTBw
 le2WbxlGNYhV1xnoWMks1GTdWGQDCXdfBAhfzRIvQq2kz4z9V8hzpnVDIk2ZL3L1
 o3nqUatR0ZVXPcinf+Ke
 =6o9t
 -----END PGP SIGNATURE-----

Merge tag 'signed-efi-next' of git://github.com/agraf/u-boot

Patch queue for efi - 2018-12-03

This release is fully packed with lots of glorious improvements in UEFI
land again!

  - Make PE images more standards compliant
  - Improve sandbox support
  - Improve correctness
  - Fix RISC-V execution on virt model
  - Honor board defined top of ram (fixes a few boards)
  - Imply DM USB access when distro boot is available
  - Code cleanups
2018-12-03 17:52:40 -05:00
Philippe Reynes
ed47097a04 test: vboot: clean its file
This update the its file used in vboot test to respect the new
node style name defined in doc/uImage.FIT (for example: replace
kernel@1 by kernel and fdt@1 by fdt-1)

Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-12-03 10:44:10 -05:00
Philippe Reynes
e246b728ee test: vboot: add padding pss for rsa signature
The padding pss is now supported for rsa signature.
This add test with padding pss on vboot test.

Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-12-03 10:44:10 -05:00
Heinrich Schuchardt
34c96659ed efi_selftest: check fdt is marked as runtime data
Check that the memory area containing the device tree is marked as runtime
data.

Update the Python test to pass ${fdtcontroladdr} to bootefi.

Update the description of the Python test.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-12-02 21:59:37 +01:00
Simon Glass
499fde5c23 test: Add a 'make qcheck' target for quicker testing
At present tests are quite slow to run, over a minute on my machine. This
presents a considerable barrier to bisecting for failures.

The slowest tests are the filesystem ones and the buildman --fetch-arch
test. Add a new 'qcheck' target that skips these tests. This reduces test
time down to about 40 second, still too long, but bearable.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-29 09:30:05 -07:00
Simon Glass
b0edea3c27 spl: Add support for passing handoff info to U-Boot proper
There is some basic informaton that SPL normally wants to pass through to
U-Boot, such as the SDRAM size and bank information.

Mkae use of the new bloblist structure for this. Add a new 'handoff' blob
which is set up in SPL and passed to U-Boot proper. Also adda  test for
sandbox_spl that checks that this works correctly and a new 'sb' command
to show the information passed from SPL.

Reviewed-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-26 08:25:37 -05:00
Simon Glass
6d07d63d2f sandbox: Drop the deprecated 'sb' command
The old 'sb' command was deprecated in 2015 and replaced with 'host'.
Remove the remaining users and the command, so that the name is available
for other purposes.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-26 08:25:35 -05:00
Simon Glass
1ca910be5d sandbox: Add an option to display of-platdata in SPL
At present we don't have a test that of-platdata can be accessed in SPL.
Add this in as a command-line option to SPL.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-26 08:25:35 -05:00
Simon Glass
20d4440189 test/py: Add a way to pass flags to sandbox
It is sometimes useful to restart sandbox with some particular flags to
test certain functionality. Add a new method to ConsoleSandbox to handle
this, without changing the existing APIs.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Stephen Warren <swarren@nvidia.com>
2018-11-26 08:25:35 -05:00
Simon Glass
d633006463 spl: Add a define for SPL_TPL_PROMPT
We should use a macro rather than hard-coding the SPL prompt to 'spl'
since the code can be used by TPL too. Add a macro that works for both
and use it in various places.

This allows TPL to use the same code without printing confusing messages.

Note that the string is lower case ('spl', 'tpl') which is a change from
previously.

Reviewed-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-26 08:25:34 -05:00
Liviu Dudau
5197dafc42 dm: core: Widen the dump tree to show more of the driver's name.
With drivers that have prefix names that are quite long (like
'versatile_') it is useful to have a wider column for the driver's
name when dumping the device driver tree.

Also update the tests to take into account the wider output format.

Signed-off-by: Liviu Dudau <liviu.dudau@foss.arm.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Patrice Chotard
f41a824b23 test/py: test pinmux command
Add pinmux test which test the following commands:
  - pinmux list
  - pinmux dev
  - pinmux status

Signed-off-by: Patrice Chotard <patrice.chotard@st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
[trini: Mark some tests as sandbox-centric]
Signed-off-by: Tom Rini <trini@konsulko.com>

Signed-off-by: Tom Rini <trini@konsulko.com>
2018-11-16 16:51:57 -05:00
Simon Glass
cdd140af5c log: Add helpers for common log levels
At present to output a log message you need something like:

   log(UCLASS_SPI, LOCL_INFO, "message1");
   log(UCLASS_SPI, LOCL_INFO, "message2");

but many files use the same category throughout. Also it is helpful to
shorten the length of log names, providing helpers for common logging
levels. Add some macros so that it is possible to do:

   (top of file, before #includes)
   #define LOG_CATEGORY UCLASS_SPI

   (later in the file)
   log_info("message1");
   log_debug("message2");
   log_err("message3");

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-09 04:40:26 -06:00
Simon Glass
11ae93eef4 binman: Run tests concurrently
At present the tests run one after the other using a single CPU. This is
not very efficient. Bring in the concurrencytest module and run the tests
concurrently, using one process for each CPU by default. A -P option
allows this to be overridden, which is necessary for code-coverage to
function correctly.

This requires fixing a few tests which are currently not fully
independent.

At some point we might consider doing this across all pytests in U-Boot.
There is a pytest version that supports specifying the number of processes
to use, but it did not work for me.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
87b05ee3a9 test/py: Fix unicode handling for log filtering
At present the unicode filtering seems to get confused at times with
this error:

  UnicodeDecodeError: 'ascii' codec can't decode byte 0x80 in position
     32: ordinal not in range(128)

It seems to be due to self._nonprint being interpreted as UTF-8. Fix it
by using ordinals instead of characters, changing the string to set.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Michal Simek <michal.simek@xilinx.com>
2018-10-08 07:34:34 -06:00
Jens Wiklander
07cbae7672 test_avb: Update pymark.buildconfigspec information for the AVB tests
Update the pymark.buildconfigspec to depend on 'cmd_mmc' in addition to
'cmd_avb' for those tests that needs more a more complete MMC
implementation or the "mmc" command.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
2018-10-07 11:07:26 -04:00
Akashi Takahiro
c906f372b2 test/py: test_fs: add docstring comments to helper functions
After Siomon's comment, add a descriptive comment (docstring) to each of
helper functions in conftest.py. No functionality changed.

Signed-off-by: Akashi Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-10-06 09:04:18 -04:00
Akashi Takahiro
e4040df555 test/py: test_fs: remove fs_type argument from umount_fs()
Since there is no use of fs_type in umount_fs(), just remove it.

Signed-off-by: Akashi Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-10-06 09:04:18 -04:00
Stephen Warren
dddea0d0d7 test/py: ignore console read exceptions after test failure
After a test has failed, test/py drains the U-Boot console log to ensure
that any relevant output is captured. At this point, we don't care about
detecting any additional errors, since the test is already known to have
failed, and U-Boot will be restarted. To ensure that the test cleanup code
is not interrupted, and can correctly terminate the log sections for the
failed test, ignore any exception that occurs while reading the U-Boot
console output during this limited period of time.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2018-10-06 09:04:18 -04:00
Heinrich Schuchardt
6dec87396e efi_selftest: test key notification functions
Use a key notification function to leave the
EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL test.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
Heinrich Schuchardt
4f17d8d391 test/py: test EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL
Add a unit test for the EFI_SIMPLE_TEXT_INPUT_EX_PROTOCOL.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
Heinrich Schuchardt
ca09f1df0c test/py: rework test_efi_selftest_text_input()
Use more precise regular expressions.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
Heinrich Schuchardt
7d044886c9 test/py: Unicode w/ EFI_SIMPLE_TEXT_INPUT_PROTOCOL
Test that the Euro sign is correctly retrieved from the console via the
EFI_SIMPLE_TEXT_INPUT_PROTOCOL.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
Akashi, Takahiro
cce289a928 test/py: fs: add fstest/unlink test
In this commit, test cases for unlink interfaces are added as part of
"test_fs" test suite.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
AKASHI Takahiro
50ca19cca9 test/py: fs: add fstest/mkdir test
In this commit, test cases for mkdir interfaces are added as part of
"test_fs" test suite.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
AKASHI Takahiro
71f27af58e test/py: fs: add extended write operation test
In this commit and the following, test scripts for new filesystem
functionalities introduced by my patch set, "fs: fat: extend FAT write
operations," are provided.

In particular, this patch adds test cases for sub-directory write
and write with non-zero offset.

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
AKASHI Takahiro
de8106c198 test/py: convert fs-test.sh to pytest
In this commit, the same set of test cases as in test/fs/fs-test.sh
is provided using pytest framework.
Actually, fs-test.sh provides three variants:"sb" (sb command), "nonfs"
(fatxx and etc.) and "fs" (hostfs), and this patch currently supports
only "nonfs" variant; So it is not a replacement of fs-test.sh for now.

Simple usage:
  $ py.test test/py/tests/test_fs [<other options>]

You may also specify filesystem types to be tested:
  $ py.test test/py/tests/test_fs --fs-type fat32 [<other options>]

Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:30 +02:00
Heinrich Schuchardt
da9ea5bb0d test/py: typo occured
%s/occured/occurred/

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:29 +02:00
Heinrich Schuchardt
cb03ec06f5 test/py: test EFI_SIMPLE_TEXT_INPUT_PROTOCOL
Execute the EFI selftest for the EFI_SIMPLE_TEXT_INPUT_PROTOCOL.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-09-23 21:55:29 +02:00
Michal Simek
65a6b3c2af test/py: Extend fpga command to test all fpga load types
Add support for info, load, loadp, loadb, loadbp, loadmk_legacy,
loadmk_legacy_gz, loadmk_fit, loadfs also with variable support.

There are probably missing failed tests.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-09-11 10:58:41 +02:00
Jean-Jacques Hiblot
49c752c93a cmd: Add bind/unbind commands to bind a device to a driver from the command line
In some cases it can be useful to be able to bind a device to a driver from
the command line.
The obvious example is for versatile devices such as USB gadget.
Another use case is when the devices are not yet ready at startup and
require some setup before the drivers are bound (ex: FPGA which bitsream is
fetched from a mass storage or ethernet)

usage example:

bind usb_dev_generic 0 usb_ether
unbind usb_dev_generic 0 usb_ether
or
unbind eth 1

bind /ocp/omap_dwc3@48380000/usb@48390000 usb_ether
unbind /ocp/omap_dwc3@48380000/usb@48390000

Signed-off-by: Jean-Jacques Hiblot <jjhiblot@ti.com>
2018-08-21 16:21:37 +02:00
Miquel Raynal
d87434a2ba test/py: tpm2: switch from 'tpm' to 'tpm2' command
While using the 'tpm' command should work on most cases, this test suite
only works with TPMv2 and since the work to make both versions build at
the same time, we might end up having both 'tpm' (TPMv1) and 'tpm2'
(TPMv2) commands available at the same time. Ensure this test suite
always use the right one.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-07-28 11:58:08 -04:00
Quentin Schulz
b7a7c4113c test/py: add test for whitelist of variables while importing environment
This tests that the importing of an environment with a specified
whitelist works as intended.

If there are variables passed as parameter to the env import command,
those only should be imported in the current environment.

For each variable passed as parameter, if
 - foo is bar in current env and bar2 in exported env, after importing
 exported env, foo shall be bar2,
 - foo does not exist in current env and foo is bar2 in exported env,
 after importing exported env, foo shall be bar2,
 - foo is bar in current env and does not exist in exported env (but is
 passed as parameter), after importing exported env, foo shall be empty
 ONLY if the -d option is passed to env import, otherwise foo shall be
 bar,

Any variable not passed as parameter should be left untouched.

Two other tests are made to test that size cannot be '-' if the checksum
protection is enabled.

Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
2018-07-20 15:55:09 -04:00
Quentin Schulz
f4eef40b01 test/py: remove hacks for non-zero RAM base address in tests
Some functions have different behaviour when the given address is 0
(assumed to be NULL by the function).

find_ram_base() does not return 0 anymore so it's safe to remove those
offsets.

Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
2018-07-20 15:55:08 -04:00
Quentin Schulz
abba76354a test/py: return a RAM address different from 0 as it can be interpreted as NULL
Some functions test that the given address is not NULL (0) and fail or
have a different behaviour if that's the case (e.g. hexport_r).

Let's make the RAM base address to be not zero by setting it to 2MiB if
that's the case.

2MiB is chosen because it represents the size of an ARM LPAE/v8 section.

Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Quentin Schulz <quentin.schulz@bootlin.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
2018-07-20 15:55:08 -04:00
Tom Rini
1703fbefd9 Merge git://git.denx.de/u-boot-dm 2018-07-11 09:40:38 -04:00
Teddy Reed
72239fc85f vboot: Add FIT_SIGNATURE_MAX_SIZE protection
This adds a new config value FIT_SIGNATURE_MAX_SIZE, which controls the
max size of a FIT header's totalsize field. The field is checked before
signature checks are applied to protect from reading past the intended
FIT regions.

This field is not part of the vboot signature so it should be sanity
checked. If the field is corrupted then the structure or string region
reads may have unintended behavior, such as reading from device memory.
A default value of 256MB is set and intended to support most max storage
sizes.

Suggested-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Teddy Reed <teddy.reed@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-07-10 16:55:58 -04:00
Paul Burton
8793631ec1 test/py: vboot: Remove stderr redirect from openssl command
The openssl command specified in test_with_algo() ultimately ends up
being run by RunAndLog::run(), which uses it to construct a Popen object
with the default shell=False. The stderr redirect in the command is
therefore simply passed to openssl as an argument. With at least openssl
1.1.0f this causes openssl, and therefore test_vboot, to fail with:

  genpkey: Use -help for summary.
  Exit code: 1

Any stderr output ought to be captured & stored in the RunAndLog
object's output field and returned from run() via run_and_log() to
test_with_algo() which then ignores it anyway, so we can drop the
shell-like redirection with no ill effects. With this fix test_vboot now
passes for me.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-07-10 14:50:50 -06:00
Paul Burton
9f9e8a4dda test/py: hush_if_test: Use open() in place of file()
In python 3.x the file() function has been removed. Use open() instead,
which works on both python 2.x & 3.x, and is described as the preferred
method of opening a file by python 2.x documentation anyway.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-07-10 14:50:50 -06:00
Paul Burton
57bf9bea34 test/py: fit: Open files as binary files
The read_file() function in test_fit is used with files that are not
text files, as well as some that are. It is never used in a way that
requires it to decode text files to characters, so open all files in
binary mode such that read() doesn't attempt to decode characters for
files which are not text files.

Without this test_fit fails on python 3.x when reading an FDT in
run_fit_test() with:

  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position
                       0: invalid continuation byte

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-07-10 14:50:50 -06:00
Paul Burton
052ca37daa test/py: Import 'configparser' lower case to be python 3.x safe
In python 3.x the configparser module is named with all lower case.
Import it as such in order to avoid errors when running on python 3.x,
and fall back to the CamelCase version in order to keep working with
python 2.x.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-07-10 14:50:50 -06:00
Paul Burton
b8c455500a test/py: Use range() rather than xrange()
In python 3.x the xrange() function has been removed, and range()
returns an iterator much like Python 2.x's xrange(). Simply use range()
in place of xrange() in order to work on both python 2.x & 3.x. This
will mean a small cost on python 2.x since range() will return a list
there rather than an iterator, but the cost should be negligible.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-07-10 14:50:50 -06:00
Paul Burton
dffd56d1d2 test/py: Make print statements python 3.x safe
In python 3.x print must be called as a function rather than used as a
statement. Update uses of print to the function call syntax in order to
be python 3.x safe.

Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-07-10 14:50:50 -06:00
Tom Rini
4e899d868d test_avb: Add pymark.buildconfigspec information for the AVB tests
Signed-off-by: Tom Rini <trini@konsulko.com>
2018-06-19 11:55:06 -04:00
Igor Opaniuk
f96c9482e6 test/py: avb2.0: add tests for avb commands
1. Run AVB 2.0 full verification chain, avb verify
2. Check if 'avb get_uuid' works, compare results with
'part list mmc 1' output
3. Test `avb read` commands, which reads N bytes from a partition
identified by a name

Signed-off-by: Igor Opaniuk <igor.opaniuk@linaro.org>
2018-06-18 14:02:02 -04:00
Tom Rini
680a52c350 tests: Make test_log.py tests depend on cmd_log not log
While the tests in this testcase are for the log subsystem they are only
able to be run if CONFIG_CMD_LOG is enabled as well as CONFIG_LOG, so
update the buildconfigspec requirement.

Signed-off-by: Tom Rini <trini@konsulko.com>
2018-05-31 08:53:11 -04:00
Miquel Raynal
2dffe1c6ad test/py: add TPMv2.x test suite
Add tests for the TPMv2.x commands.
These commands may run both on a physical TPM and with the sandbox
driver.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2018-05-25 20:13:00 -04:00
Tom Rini
83d290c56f SPDX: Convert all of our single license tags to Linux Kernel style
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from.  So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry.  Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents.  There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>
2018-05-07 09:34:12 -04:00
Heinrich Schuchardt
06c3d5b989 efi_selftest: check installation of the device tree
The unit test checks if a device tree is installed. It requires that the
'compatible' property of the root node exists. If available it prints the
'serial-number' property.

The serial-number property is derived from the environment variable
'serial#'. This can be used to check if the image_setup_libfdt() function
is executed.

A Python test is supplied. It sets a value for serial# and checks that the
selftest shows this as serial-number.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2018-04-04 11:00:06 +02:00
Liam Beguin
7f9b50a27c test/py: add spi_flash tests
Add basic tests for the spi_flash subsystem.

Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-03-22 16:32:20 -04:00
Liam Beguin
c3342cd58f test/py: add generic CRC32 function
Add a generic function which can be used to compute the CRC32 value of
a region of RAM.

Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-03-22 13:25:20 -04:00
Liam Beguin
64a2cebb6e test/py: do not import pytest multiple times
Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-03-22 13:25:20 -04:00
Liam Beguin
be91691d08 test/py: README: add HOSTNAME to PYTHONPATH
As opposed to PATH, HOSTNAME is not appended to PYTHONPATH
automatically. Lets add it to the examples to make it more
obvious to new users.

Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-03-22 13:25:20 -04:00
Liam Beguin
0e5dd786e9 test/py: README: fix typo
Fix a minor typo causing vim (and possibly other) to get confused with
coloring.

Signed-off-by: Liam Beguin <liambeguin@gmail.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2018-03-22 13:25:20 -04:00
Stephen Warren
32090e5070 test/py: highlight warnings in the log summary
Currently, if a test emits a warning message but otherwise passes, there's
no indication of this in the log summary, which can lead to warnings being
missed. Enhance the test logic to explicitly mention warnings in otherwise
passing tests, and not to collapse the log sections for tests with
warnings, so that they're more easily seen when scanning the log.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2018-03-13 21:59:26 -04:00
Stephen Warren
4bdc90f9c7 test/py: add MMC/SD block read test
Add a standalone MMC block read test. This allows direct testing of MMC
access rather than relying on doing so as a side-effect of e.g. DFU or
UMS testing, which may not be enabled on all platforms.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2018-03-13 21:59:26 -04:00
Simon Glass
aa4e0e005b log: Add tests for the new log features
Add a test of the 'log format' and 'log rec' commands. This also covers
things like log_get_cat_by_name(), since they are used by these commands.
Fix a style nit in the tests also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-02-03 10:09:27 -07:00
Simon Glass
deca50fbd5 log: Update log_console to honour the log format
At present this just outputs the message. Update it to output whatever the
format requests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-02-03 10:09:27 -07:00
Masahiro Yamada
94b13bbae9 host-tools: use python2 explicitly for shebang
All of these host tools are apparently written for Python2,
not Python3.

Use 'python2' in the shebang line according to PEP 394
(https://www.python.org/dev/peps/pep-0394/).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-01-28 12:27:30 -05:00
Stephen Warren
3e229a83bd test/py: Setup variables based on HUSH selection
After adding our small zynq uboot which has hush parser off same
variable tests start to failed. Use quotes only when hush is enabled.

Reported-by: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-12-18 09:32:07 +01:00
Michal Simek
040f5f1067 test: py: Add an option to skip sleep test
Some QEMUs have a problem with time setup that's why
sleep test is failing. Introduce env__sleep_accurate
boardenv variable to have an option to skip sleep test.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2017-12-12 21:33:38 -05:00
Patrick Delaunay
f03146480d test/py: gpt: update size of gpt partition
- avoid disturbing 0MiB partition size (in fact < 1MiB)
- test overlap limit between part1 and part2
- test gpt write with data with modifier 'M' for MiB

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2017-12-12 21:33:38 -05:00
Simon Glass
20faa27c2b log: test: Add a pytest for logging
Add a test which tries out various filters and options to make sure that
logging works as expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-12-07 15:17:00 -05:00
Tom Rini
9188c4315c Patch queue for efi - 2017-12-05
Highlights for this release:
 
   - Dynamic EFI object creation (lists instead of static arrays)
   - EFI selftest improvements
   - Minor fixes
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJaJxZhAAoJECszeR4D/txgtX4P/iTp602iM+2m4WP0P2K/L72b
 EsPAL4W/QXMBZBnq9MowQl7Ia3JU6bS1g2EDlbAvoQgbhmbTBfA6VxJmQj/AuOn0
 waxjyfBw/YL4010YAfmExdVy8Qj0k/SEXcxOiHYZJzmW1o3PXvLuLnetUNJ0Uip9
 GJQPcGjSaXY1VM7oAwpOVMWkbpaIIfgw9yrAc2Cw6KN8iHmaf6rPFBdWARGB7w9m
 znFtq0FpNQSLEEoyJVYAlqyRRsj+1wS3ExGnsL2no74njAgzKSWmdN7qFjkO7WC1
 w+CnZLiUv5MaboYK2uORW2qBQzRylyEXWCkqIaZL+EWElVqPRz7/VSDLUJxnZLbl
 vhDTRUB/kVwdrmB8lP21Rruk1EjCIyYDcGQ+nwhlDes/NJztfNlyOEm47N8LNyt6
 obRxOVC8nIjf4YptHB4CRgccGQAE7VqS/DqstV1plC1Ms+puG6SQvuPwlCkKX9XP
 qkZnirgzrUfGjmFMM3owoelA40jXWV4vv90HjQvwmaOftjPY3IJ+X0rgqUGbuOmn
 /R8OCW6PV0FIOx1k5/waZlqDnfFkChm56Rhxqf9LqEro37YTUjapXEgBn7a+132x
 /icaOySkFx0hy+cvPNQ5aXyNLQPDPS++Af9e/hcMewh4Zn5FrJqkJrmtLDZzGJIG
 LC5DWr8o9vwT41UftPb0
 =uyBA
 -----END PGP SIGNATURE-----

Merge tag 'signed-efi-next' of git://github.com/agraf/u-boot

Patch queue for efi - 2017-12-05

Highlights for this release:

  - Dynamic EFI object creation (lists instead of static arrays)
  - EFI selftest improvements
  - Minor fixes
2017-12-05 17:52:16 -05:00
Simon Glass
1f0fe88db1 test/py: Allow any unit test suite to be found
The u-boot.sym file is scanned to find unit test suites for execution. At
present it only finds those whose names start with 'dm' or 'env'. This
code is buried in the bowels of the test code so when adding a new suite
it is not easy to discover why it is ignored by the test framework.

There seems to be no need to make this restriction. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Stephen Warren <swarren@nvidia.com>
2017-12-02 18:32:58 -05:00
Heinrich Schuchardt
45055aac9d test/py: check return code of helloworld
Check that helloworld.efi returns EFI_SUCCESS.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2017-12-01 13:39:32 +01:00
Heinrich Schuchardt
83dee949cb test/py: fix typo in test_efi_loader.py
Make a comment line easier to read.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2017-12-01 13:22:55 +01:00
Heinrich Schuchardt
df172e117d test/py: test reboot by EFI watchdog
Clear environment variable efi_selftest before executing the
default tests.

Provide a test verifying that the EFI watchdog
reboots the system upon timeout.

The test depends on CONFIG_CMD_EFI_SELFTEST=y.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2017-12-01 13:22:55 +01:00
Stephen Warren
9679d339ad test/py: add timestamps to log
It can be useful to record how long tests take; this can help debug slow
running test systems or track changes in performance over time. Enhance
the test system to record timestamps while running test:
- Whenever a new log file section is started.
- After U-Boot is started and communication has been established.
- After each host or U-Boot command is executed.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-11-17 07:44:13 -05:00
Stephen Warren
ac122efdb6 test/py: regenerate persistent GPT image if code changes
test_gpt generates a persistent disk image which can be re-used across
multiple test runs. Currently, if the Python code that generates the disk
image change, the image is not regenerated, which could cause test
failures e.g. if a test was updated to expect some new partition name or
size, yet the persistent disk image contained the old name or size. This
change introduces functionality to regenerate the disk image if the
instructions to generate the image have changed.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-11-06 09:59:00 -05:00
Masahiro Yamada
3aca4a44f8 test/py: fix typos in README.md
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2017-10-23 17:28:17 -04:00
Patrick Delaunay
3a2605fa87 cmd: gpt: solve issue for swap and rename command
don't use prettyprint_part_size() in create_gpt_partitions_list()
that avoid to align offset and size to 1 MiB and increase precision for
start and size.
This patch avoid the risk to change partition size and lost data during
rename or swap.

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
2017-10-23 17:28:11 -04:00
Patrick Delaunay
0cf02ff612 test/py: gpt: test start LBA for sub-command rename and swap
Add test of first and last LBA in gpt for rename and swap.
Only the name is expected to change, so test 3 columns
for part command
1: first LBA (start)
2: last LBA (end)
3: partition name

After rename, the last LBA change and it is a error in current U-Boot code
+ "first" = 0x7ff : invalid value (<start)
+ "second" = 0x17ff => size increasing !

Acked-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
2017-10-23 17:28:08 -04:00
Patrick Delaunay
b61a3b5c8d test/py: gpt: add test for sub-command write
+ test write for one partition on all the device (size=0)
+ test write with disk uuid and 2 partitions

Acked-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
2017-10-23 17:27:58 -04:00
Patrick Delaunay
30ef7cbb81 test/py: gpt: add test for sub-command read and verify
add sandbox test for some gpt sub-command
- gpt read / part list : read the gpt partition created by sgdisk on host
  test start, size, LBA and name output
- gpt verify : verify the gpt partition create by sgdisk on host

PS: persistent data test_gpt_disk_image.bin are udpated

Acked-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
2017-10-23 17:27:42 -04:00
Patrick Delaunay
da4c4bbd61 test/py: gpt: copy persistent file
copy the persistent gpt binary file as it can be modified during the test
that avoid issue if the test fail: the test always restart with clean file

Acked-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
2017-10-23 17:27:05 -04:00
Heinrich Schuchardt
6c7c3dcc59 test/py/tests/test_sleep.py: test time approximately
On qemu errors like
assert 2.999650001525879 >= 3
occur.

According to the comment in the code the test is meant to be
approximate. So we should accept some milliseconds less.

Cc: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2017-10-16 13:45:37 -04:00
Tom Rini
958046fc78 Patch queue for efi - 2017-10-01
Lots of new things this time. High level highlights are:
 
   - Shim support (to boot Fedora)
   - Initial set of unit tests
   - Preparations to support UEFI Shell
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABAgAGBQJZ0KdpAAoJECszeR4D/txga+AQAL8ehBlkr3R6b/n00p3kuAVE
 cfM+o/16b+P/r3U2/wvlNtw7L+UPlGaQNk8o7n4xTu0DpVlIt7DhFrSBIcK6VbYB
 8BFJ/EL9NRj/sd/wb9B1Bc0SlXJcGxzRM+JxNseVmc7rz9sWdARvx/diQlP+YAX4
 qjp2YRzsErjUgix4eGDXvgFQdFmh4Pnp7UnHlSq4WkINo14navGLe/PwChxsL91M
 AZV7JulVcm7XtOPTON8udec3O15O8BExAM1cH2ITjtD99Je+4VcuI99nLm2aIE++
 U6vTuoogHAU+oCutD3qZU85uj2VJ2kYy2k5A/RK/i6Ug/QrTa+JA8PNWP78/TZ+8
 xmyDVzG5MQhMIddDjJlMXbxIwgKhr0SjUJ1iVaGI0bS/6Glc7h9m+hhIXKudqOvs
 lxqPhy7mahtBwETjoUlM1WJg+TG+vIGDJtT22njjOSTh0CVYw2H3xihyFKE3AjaW
 L26EOEsxArErtzF17GgF5KxFOGaYAmVmA3XnCcEZEIY5gzL65uok1GX/apf4sIsi
 j8HSbW3UH6QwjWKBhF+RVuK4CcTDUS9K2YmJlrdZ69elDZyg5PWbpMLNdYIP+VVG
 +j91c7e3O2q2AHAzTprpmkyeIeqZfFhgyu4H/jPQ8Yi8N/b7wbugcLm3fHuUNal4
 RB0KjbyJzJPQwbYGSLYi
 =9HkH
 -----END PGP SIGNATURE-----

Merge tag 'signed-efi-next' of git://github.com/agraf/u-boot

Patch queue for efi - 2017-10-01

Lots of new things this time. High level highlights are:

  - Shim support (to boot Fedora)
  - Initial set of unit tests
  - Preparations to support UEFI Shell
2017-10-01 08:48:38 -04:00
Stephen Warren
117eeb7f84 test/py: fix anchors in HTML status report
The current code wraps a pre tag inside an a tag. For some reason, this
causes at least Firefox to attempt to drag the pre section content when
using a mouse drag to select text. Re-order the tags so that the text can
be selected using the mouse, at least if you start the drag outside the
text (after the end of the line, for example).

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:53 -04:00
Stephen Warren
2d26bf6c26 test/py: add skip marker for reliance on tools
Some tests use external tools (executables) during their operation. Add
a test.py mark to indicate this. This allows those tests to be skipped if
the required tool is not present.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:53 -04:00
Stephen Warren
d5170448ae test/py: provide more information about test skip reason
When skipping tests, explicitly mention the board type or config option
that caused the skip. This will help people understand/fix any issues.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:52 -04:00
Stephen Warren
af3fae0b1b test/py: Document required tools/packages
Some tests rely on external tools. Mention these in the test/py README.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:52 -04:00
Heinrich Schuchardt
38b1b79021 test/py: add a test calling the EFI selftest
A Python test script is provided that runs the EFI selftest
if CONFIG_CMD_EFI_SELFTEST=y.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Alexander Graf <agraf@suse.de>
2017-09-18 23:53:57 +02:00
Stephen Warren
110ba62519 test/py: gpt: make use of infra-structure
Make various changes to the GPT test:

1) Reference the disk image using an absolute path in all cases. This
allows test/py to operate correctly if it's run from a directory other
than the root of the U-Boot source tree.

2) Store the disk image in the teswt/py persistent data directory. This
removes the need to re-generate it every time the tests are run.

3) Execute sgdisk using u_boot_utils.run_and_log() so that its output is
captured in the test log. This allows debugging any problems running it.

4) Make the disk image a test fixture. This removes the requirement to
always run all GPT tests, and run them in order. The current code doesn't
create the disk image if e.g. just test_gpt_uuid() is executed via the
test.py -k command-line option.

5) Use @pytest.mark.buildconfigspec for all feature dependencies, rather
than manually implementing some of them.

6) Make all tests depend on sandbox, since they use the sandbox-specific
host command.

Fixes: a2f422555f ("add pytests for 'gpt guid' command in sandbox")
Fixes: c5772188ed ("add pytests for 'gpt rename' and 'gpt swap'")
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Tom Rini <trini@konsulko.com>
2017-09-15 22:33:41 -04:00
Heinrich Schuchardt
cd3e8a72a6 test/py: u_boot_console_base.py: fix typo
run_command does not have a parameter wait_for_each.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2017-09-15 08:05:12 -04:00
Alison Chaiken
c5772188ed add pytests for 'gpt rename' and 'gpt swap'
Add unit tests for the 'gpt rename' and 'gpt swap' commands that
rely on the block device created by test/py/make_test_disk.py.
Add CONFIG_CMD_GPT_RENAME to the sandbox_defconfig.  Remove the
testdisk.raw test device at the end of the tests.

Signed-off-by: Alison Chaiken <alison@peloton-tech.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-09-14 21:32:59 -04:00
Alison Chaiken
a2f422555f add pytests for 'gpt guid' command in sandbox
Run unit tests for the 'gpt guid' command, making use of the block
device created by test/py/make_test_disk.py.  Remove this device at
the end of the tests.

Signed-off-by: Alison Chaiken <alison@peloton-tech.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-09-14 21:32:57 -04:00
Alison Chaiken
bf6d76b84a GPT: create block device for sandbox testing
Provide a Python function that creates a small block device for the
purpose of testing the cmd/gpt.c or cmd/part.c functions in the u-boot
sandbox.

Signed-off-by: Alison Chaiken <alison@peloton-tech.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-09-14 21:32:57 -04:00
Simon Glass
3a37aee30f test: Move the FIT test into the correct place
Move this test so that it will run when 'make tests' is used.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-08-13 15:17:36 -04:00
Tom Rini
845e10d165 tests: test_dfu.py: Add example udev rule for host_usb_dev_node
If one does not already have a rule to create a custom device node when
a given device enumerates it can be useful to have udev create a
bus path based node to the entry in /dev/bus/usb that was just
enumerated.  Given that DFU itself does not require a /dev entry it is a
good idea to provide a rule that will generate one.

Signed-off-by: Tom Rini <trini@konsulko.com>
2017-07-22 15:36:16 -04:00
Simon Glass
2028cc59f7 sandbox: Stop printing platdata at the start of SPL
Currently we have code which prints out platform data at the start of SPL.
Now that we have tests for dtoc this is probably not necessary. Drop it.
Update test_ofplatdata to check for empty output since it is useful to
check that sandbox_spl works as expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11 10:08:20 -06:00
Michal Simek
5bdb317065 test: py: hush: Add echo dependency
Some tests depends on echo command to be present.

Reported-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2017-06-05 14:13:02 -04:00
Michal Simek
a5b548138b test: py: Use global pytestmark for hush tests
All tests in test_hush_if_test depends on hush parser to be
present. This patch simplify test dependencies by using global
pytestmark.

Reported-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
2017-06-05 14:13:02 -04:00
Michal Simek
6b83c38d7a test: py: Add cmd_echo dependency
There is missing dependency on echo command. Mark tests which requires
echo.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2017-05-22 07:29:55 -04:00
Alexander Graf
0e4e38ae38 travis: Add efi_loader grub2 test
We have all the building blocks now to run arbitrary efi applications
in travis. The most important one out there is grub2, so let's add
a simple test to verify that grub2 still comes up.

Signed-off-by: Alexander Graf <agraf@suse.de>
2016-11-27 09:53:40 -05:00
Alexander Graf
95b62b2e28 efi_loader: Allow to compile helloworld.efi w/o bundling it
Today we can compile a self-contained hello world efi test binary that
allows us to quickly verify whether the EFI loader framwork works.

We can use that binary outside of the self-contained test case though,
by providing it to a to-be-tested system via tftp.

This patch separates compilation of the helloworld.efi file from
including it in the u-boot binary for "bootefi hello". It also modifies
the efi_loader test case to enable travis to pick up the compiled file.
Because we're now no longer bloating the resulting u-boot binary, we
can enable compilation always, giving us good travis test coverage.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-11-27 09:53:39 -05:00
Alexander Graf
4ca4b265ad tests: Add efi_loader hello world test
Now that we have working network tests and a hello world efi application
built inside our tree, we can automatically test that efi binary running
inside of U-Boot.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-11-26 15:50:53 -05:00
Alexander Graf
1bce3ad5f3 tests: net: Offset downloads to 4MB
The network test currently downloads files at 0MB offset of RAM start.
This works for most ARM systems, but x86 has weird memory layout constraints
on the first MB of RAM.

To not get caught into any of these, let's add a 4MB pad from start
of RAM to the default memory offset.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-11-26 15:50:52 -05:00
Stefan Brüns
d8c1e0331a test/py: expose config and log as session scoped fixture
If a test uses a fixture which is expensive to setup, the fixture can
possibly created with session or module scope. As u_boot_console has
function scope, it can not be used in this case.

Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Acked-by: Stephen Warren <swarren@wwwdotorg.org>
2016-11-21 14:07:27 -05:00
Stephen Warren
b0a928a15d test/py: ensure a log section exists for skipped tests
In pytest 3, runtestprotocol() may not call pytest_runtest_setup() if
the test is skipped. That call is required to create a section for the
test in the log file. If this is skipped, the call to log.end_section()
at the tail of pytest_runtest_protocol() will throw an exception. This
patch ensures that a log section always exists, both to avoid the
exception and to ensure that a consistently structured log file is
always created.

Cc: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reported-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Tom Rini <trini@konsulko.com>
2016-10-23 18:33:17 -04:00
Tom Rini
2b2c6e51e7 test/py/tests/test_sleep.py: Add check for CONFIG_CMD_MISC
We can only run this command if the sleep command is enabled and that
depends on CONFIG_CMD_MISC

Signed-off-by: Tom Rini <trini@konsulko.com>
2016-10-14 19:12:31 -04:00
Guillaume GARDET
6a2981a713 test: add NFS download test
Add a NFS download test, based on TFTP test.
Tested on i.MX6 SabreLite board.

Signed-off-by: Guillaume GARDET <guillaume.gardet@free.fr>

Cc: Tom Rini <trini@konsulko.com>
Cc: Joe Hershberger <joe.hershberger@ni.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Joe Hershberger <joe.hershberger@ni.com>
2016-10-13 12:25:36 -05:00
Tom Rini
de4be9ec17 test/py/tests/test_vboot.py: Add check that we boot the image
Make sure that when we're telling bootm to boot an image, and we expect
the image to boot we get the output from sandbox that we attempted to
run Linux and that U-Boot completed its job.

Cc: Simon Glass <sjg@chromium.org>
Cc: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Acked-by: Stephen Warren <swarren@nvidia.com>
2016-09-22 11:34:58 -04:00
Stephen Warren
4ba58bdabd test/py: match prompt only at line boundaries
This prevents capture of command output from terminating early on boards
that use a simple prompt (e.g. "=> ") that appears in the middle of
command output (e.g. crc32's "... ==> 2fa737e0").

Reported-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Tested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
2016-08-20 14:03:24 -04:00
Simon Glass
f6d34651d8 test: Adjust run_command_list() to return a list of strings
Return one string for each command that was executed. This seems cleaner.

Suggested-by: Teddy Reed <teddy.reed@gmail.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-08-05 20:55:23 -04:00
Simon Glass
27c087d58a test: Add a function to restart U-Boot
Add a proper function for this rather than using internal functions. Use it
in the single call site.

Also, do a restart at the end of the vboot test to reset to the normal
device tree.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Stephen Warren <swarren@nvidia.com>
2016-08-05 20:55:22 -04:00
Simon Glass
851271a71a test: vboot: Put each test variant in its own section
Use 'cons.log.section' feature to split up the test output. This makes it
easier to read.

Suggested-by: Stephen Warren <swarren@nvidia.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:22 -04:00
Simon Glass
ac9a23cffc test: Rename sha to sha_algo and pass it around
Rename this argument and pass it to each function that needs it, instead of
making it global.

Suggested-by: Stephen Warren <swarren@nvidia.com>
Suggested-by: Teddy Reed <teddy.reed@gmail.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:21 -04:00
Simon Glass
ec70f8a911 test: Drop the cmd() function
Instead of this, use the existing run_and_log() function, enhanced to
support a command string as well as a list of arguments.

Suggested-by: Stephen Warren <swarren@nvidia.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:20 -04:00
Simon Glass
72f5226894 test: Fix typos in comments
Fix some typos in various files introduced with the vboot test conversion.

Reported-by: Teddy Reed <teddy.reed@gmail.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:20 -04:00
Simon Glass
7f64b1874c test: Check exit status in run_and_log_expect_exception()
This check was missed. Add it and make the message more verbose.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
Fixes: 9e17b034 (test/py: Provide a way to check that a command fails)
2016-08-05 20:55:19 -04:00
Simon Glass
bcbd0c8fe1 test: Fix typos and tidy up
Fix review comments that were missed at the time. Also explain why we need
to regenerate the device tree for each test.

Reported-by: Teddy Reed <teddy.reed@gmail.com>
Suggested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: f6349c3c (test: Add a README)
2016-08-05 20:55:19 -04:00
Michal Simek
04a4786c7c test/py: vboot can be run only at Sandbox
Getting this error:
Zynq> sb load hostfs - 100
/home/monstr/data/disk/u-boot/build-zynq_zc706/test.fit
Unknown command 'sb' - try 'help'

because sb command is present only for Sandbox
obj-$(CONFIG_SANDBOX) += host.o

that's why mark this test to be run only at Sandbox

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-22 14:52:04 -04:00
Stephen Warren
c9ba60c438 test/py: use absolute dts path in vboot test
Without this, the test fails if the test is run with a cwd other than the
root of the U-Boot source tree.

Fixes: 8729d58259 ("test: Convert the vboot test to test/py")
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2016-07-22 14:46:24 -04:00
Simon Glass
f60d0603ed test: Adjust the of-platdata test run condition
This should be spl_of_platdata, since otherwise it will try to run on boards
that don't support of-platdata.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-16 21:01:44 -04:00
Tom Rini
ebe621d5fb Merge git://git.denx.de/u-boot-dm 2016-07-15 08:06:22 -04:00
Simon Glass
b979d3d4c5 dm: Add a test for of-platdata
Add a simple test which checks that the of-platdata system is working
correctly. The sequence is as follows:

- SPL starts up and probes all the UCLASS_MISC drivers
- There are 3 of these in sandbox.dts
- Therefore there should be 3 U_BOOT_DEVICE() declarations in dt-platdata.c
- These should produce 3 sandbox_spl_test devices
- Each device prints out its platform data when probed
- This test checks for this output and compares it against expectations

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
a811779b17 test/py: Start sandbox SPL when enabled
When sandbox SPL is enabled we want to start that rather than U-Boot proper,
since some tests may rely on running it first.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
ebec58fbcb test/py: Provide a way to get early console output
Some tests want to check the console output from SPL or U-Boot proper.
Provide a means to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
c7f636f59d test/py: Note which console produced unexpected output
At present the SPL and U-Boot consoles both present the same error message
when the expected console output does not appear. Add "SPL" to the SPL error
message to resolve this ambiguity.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
2fedbaa4ae test/py: Handle testing with the sandbox_spl board
This board can sometimes be used for tests. Handle it the same way as
sandbox.

Note: I plan to drop the sandbox_spl board at some point and merge its
features into sandbox. So this commit may not be necessary.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
8729d58259 test: Convert the vboot test to test/py
Now that we have a suitable test framework we should move all tests into it.
The vboot test is a suitable candidate. Rewrite it in Python and move the
data files into an appropriate directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 18:22:40 -04:00
Simon Glass
73a9054d0f test/py: Add a helper to run a list of U-Boot commands
Some tests want to execute a sequence of commands. Add a helper for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:36 -04:00
Simon Glass
9e17b0345a test/py: Provide a way to check that a command fails
Sometimes we want to run a command and check that it fails. Add a function
to handle this. It can check the return code and also make sure that the
output contains a given error message.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 18:22:35 -04:00
Simon Glass
8b304a37df test/py: Add an option to execute a string containing a command
It is sometimes inconvenient to convert a string into a list for execution
with run_and_log(). Provide a helper function to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:35 -04:00
Simon Glass
f3d3e95ce5 test/py: Return output from run_and_log()
It is useful to be able to obtain the output from a command. Return it from
this function.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:34 -04:00
Simon Glass
86845bf38d test/py: Provide output from exceptions with RunAndLog()
Tests may want to look at the output from running a command, even if it
fails (e.g. with a non-zero return code). Provide a means to obtain this.

Another approach would be to return a class object containing both the
output and the exception, but I'm not sure if that would result in a lot
of refactoring.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:34 -04:00
Simon Glass
3b8d9d977b test/py: Allow RunAndLog() to return the output
Tests may want to look at the output from running a command. Return it so
that this is possible.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:34 -04:00
Simon Glass
0671960bee test/py: Allow tests to control the sandbox device-tree file
Normally tests will run with the test.dtb file designed for this purpose.
However, the verified boot tests need to run with their own device-tree
file, containing a public key.

Make the device-tree file a config option so that it can be adjusted by
tests. The default is to keep the current behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:33 -04:00
Daniel Schwierzeck
d56dd0b1f8 test/py: support 'memstart =' in u_boot_utils.find_ram_base()
Some archs like MIPS or PPC have a different 'bdinfo' output
than ARM regarding the memory configuration. Also support
'memstart = 0x*' in u_boot_utils.find_ram_base() to make
all tests requiring the RAM base working on those archs.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
2016-07-08 17:16:45 -04:00
Stephen Warren
085e64dd42 test/py: strip VT100 codes from match buffer
Prior to this patch, any VT100 codes emitted by U-Boot are considered part
of a command's output, which often causes tests to fail. For example,
test_env_echo_exists executes printenv, and then considers any text on a
line before an = sign as a valid U-Boot environment variable name. This
includes any VT100 codes emitted. When the test later attempts to use that
variable, the name would be invalid since it includes the VT100 codes.
Solve this by stripping VT100 codes from the match buffer, so they are
never seen by higher level test code.

The codes are still logged unmodified, so that users can expect U-Boot's
exact output without interference. This does clutter the log file a bit.
However, it allows users to see exactly what U-Boot emitted rather than a
modified version, which hopefully is better for debugging. It's also much
simpler to implement, since logging happens as soon as text is received,
and so stripping the VT100 codes from the log would require handling
reception and stripping of partial VT100 codes.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-07-08 17:16:42 -04:00
Stephen Warren
a82642f398 test/py: fix CONFIG_ tests
Some CONFIG_ variables were recently renamed, but test/py wasn't updated
to match. This causes some tests to be skipped. Fix test/py so the tests
are run.

Fixes: 1163625898 ("Rename reset to sysreset")
Fixes: f1f9d4fac5 ("hush: complete renaming CONFIG_SYS_HUSH_PARSER to CONFIG_HUSH_PARSER")
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-07-08 12:47:58 -04:00
Stephen Warren
7a8f886558 test/py: fix printenv signon message disable code
CONFIG_VERSION_VARIABLE isn't always defined, so we can't simply look up
its value directly, or an exception will occur if it isn't defined.
Instead, we must use .get() to supply a default value if the variable
isn't defined.

Fixes: da37f006e7 ("tests: py: disable main_signon check for printenv cmd")
Acked-by: Heiko Schocher <hs@denx.de>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-06-19 09:50:56 -04:00
Heiko Schocher
da37f006e7 tests: py: disable main_signon check for printenv cmd
if CONFIG_VERSION_VARIABLE is set, the U-Boot environment
contains a "vers" variable with the current U-Boot version
string. If now "printenv" is called, test/py fails as it
detects the main_sign string, which is in this case correct.

So check only the main_sign as an error, if CONFIG_VERSION_VARIABLE
is not set.

Signed-off-by: Heiko Schocher <hs@denx.de>
2016-06-09 13:53:05 -04:00
Michal Simek
87861c1970 test/py: Support setting up specific timeout
Large file transfers, flash erasing and more complicated tests
requires more time to finish. Provide a way to setup specific
timeout directly in test.

For example description for 50s test:
timeout = 50000
with u_boot_console.temporary_timeout(timeout):
  u_boot_console.run_command(...)

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-05-27 15:39:57 -04:00
Heiko Schocher
7e6621a1ca test/py: fix NameError exception if bdi cmd is not supported
test/py raises an error, if a board has not enabled bdi command

>           pytest.skip('bdinfo command not supported')
E           NameError: global name 'pytest' is not defined

import pytest in test/py/u_boot_utils.py fixes this.

Signed-off-by: Heiko Schocher <hs@denx.de>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-05-27 10:01:09 -04:00
Heiko Schocher
b8218a9146 tests: py: fix NameError exception if bdi cmd is not supported
test/py raises an error, if a board has not enabled bdi command

>           pytest.skip('bdinfo command not supported')
E           NameError: global name 'pytest' is not defined

import pytest in test/py/u_boot_utils.py fixes this.

Signed-off-by: Heiko Schocher <hs@denx.de>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-05-13 09:17:33 -04:00
Stephen Warren
daa69f5f5d test/py: dfu: wait for USB device to go away at boot
It can take a while for a host machine to notice that a USB device has
disconnected, and process the change. At the end of the DFU test, we wait
up to 10 seconds for this to happen. This change makes the test wait the
same (up to) 10 seconds at the start of the test for any previously active
USB device-mode session to be cleaned up. Such as session might have been
used to download U-Boot into memory for example; this is certainly true
on my Tegra test systems. This changes should solve the DFU test
intermittency issues I've been seeing on some Tegra devices.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-05-06 10:10:53 -04:00
Lukasz Majewski
f3a87f5b79 tests: py: dfu: Provide functionality to set test and dummy files alt settings
After concatenation of "dfu_alt_info" variable from "dfu_alt_boot" and
"dfu_alt_system" it may happen that test and dummy files alt settings
are different than default 0 and 1.

This patch provides the ability to set different values for them.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
---
Changes for v3:
- replace variables declarations with ones read from configuration file
- remove not necessary str() conversion at DFU host command generation

Changes for v2:
- generate "alt_info" automatically
- use file names as alt settings instead of numerical values
- extend in-code documentation
2016-04-25 17:56:30 +02:00
Lukasz Majewski
8eb3752446 tests: py: dfu: Add functionality to set different u-boot's dfu env variable
By default (on almost all systems) the dfu env variable, which defines
available alt settings, is named as "dfu_alt_info".

However on some platforms (i.e. Odroid XU3), the 'dfu_alt_info' is concatenated
from other variables - namely 'dfu_alt_boot' and 'dfu_alt_system' at run time
(when one types 'dfu 0 mmc 0' for first time).

'dfu_alt_boot' describes alt settings which depend on boot medium - for example
boot loader's LBA sectors which are different on eMMC and SD card because of e.g.
MBR/GPT.

'dfu_alt_system' describes board agnostic alt settings - like rootfs, kernel.
On such system we can only append/modify this env variable.

Because of the above, we must have way to modify other than "dfu_ale_info"
variable to perform tests.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
---
Changes for v3:
- None

Changes for v2:
- Rewrite of "alt_info_env_name" variable description
- Use of get() method on python's dictionary to easily obtain default
  value
2016-04-25 17:56:30 +02:00
Lukasz Majewski
c6eb899c4d tests: py: dfu: Add variables to store dfu alt numbers for test and dummy files
This patch replaces hardcoded (i.e. 0 and 1) values passed to dfu_{read|write}
with variables.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
---
Changes for v3:
- Replace per module global variables with ones defined inside a function
Changes for v2:
- None
2016-04-25 17:56:29 +02:00
Stephen Warren
5b2beab5cd test/py: README: link to example hook scripts
When implementing test/py hook scripts, it's helpful to read some working
examples. Provide a link to some. The link was mentioned in the commit
message which first added test/py, but not in any documentation file.

Suggested-by: Lukasz Majewski <l.majewski@samsung.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-04-11 20:48:25 -04:00
Michal Simek
3ba1352b97 test/py: Add support for loading image via tftp to specified location
For example this setting:

env__net_tftp_readable_file = {
    "fn": "ep108/image.ub",
    "addr": 0x10000000,
    "size": 25846296,
    "crc32": "b726f9de",
}

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
2016-04-11 20:48:22 -04:00
Stephen Warren
eed095da30 test/py: pass -v option when executing sandbox
This shows more output, such as the internal output generated by the unit
test ("ut") command, which makes it easier to debug issues.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2016-04-11 20:48:21 -04:00
Stephen Warren
24862c640e test/py: skip tests that require large CONFIG_SYS_MAXARGS
test_hush_if_test.py executes commands that require large values of
CONFIG_SYS_MAXARGS. Detect cases where the configured value is too low
and skip those tests.

Ideally, this would be implemented inside console.run_command(). However,
the command passed to that function is already a completely formed string,
and determining its argument count usage would require splitting commands
at ;, handling quoting to deal with arguments containing spaces, etc. Even
passing the command as a list wouldn't solve all these issues, since we'd
still need to split commands on ; and deal with cases like "if test ..."
which consumes 0 of the argument count.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-26 08:42:12 -05:00
Stephen Warren
38831ca3be test/py: use space to interrupt autoboot
Sending CTRL-C to QEMU's stdin aborts the process, even if stdin is being
used as a serial port (at least in the raspi2 machine with "qemu -serial
stdin"). Avoid sending CTRL-C to U-Boot to prevent it exiting.

I'd originally used CTRL-C to make sure that if the character used to
abort autoboot ended up being treated as part of a command as well, it'd
abort command entry and return the prompt to a known state. However, this
is not needed, since aborting the autoboot eats the character used to do
that.

Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-26 08:42:11 -05:00
Michal Simek
299e5bb7c2 test/py: Add option to skip SPL signature checking
Provide user option to skip SPL signature verification for cases where
u-boot is build with SPL support but full U-Boot is also verified
without SPL.

If you want to support this feature please add env__spl_skipped = True
to your boardenv configuration file.

For example Xilinx Zynq is using this feature where the same U-Boot
binary is checked with SPL and without SPL(with FSBL).

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2016-02-25 10:21:19 -05:00
Heiko Schocher
b1309a23e0 test/py: only check for SPL signature if SPL uses serial output
check for U-Boot SPL signature only if SPL really has a serial output.
So check if CONFIG_SPL_SERIAL_SUPPORT is active in board config.

Signed-off-by: Heiko Schocher <hs@denx.de>
Tested-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-02-24 18:44:00 -05:00
Stephen Warren
9725543843 test/py: put "Starting U-Boot" into separate log section
The initial boot of U-Boot happens within the context of the first test
that needs to access the U-Boot console when there is no existing
connection. This keeps all activity nestled within test execution, which
fits well into the pytest model. However, this mingles the U-Boot startup
logs with the execution of some test(s), which hides find the boundary
between the two.

To solve this, wrap the "Starting U-Boot" logic into a separate log
section. If the user wishes, they can simply collapse this log section
when viewing the HTML log, to concentrate purely on the test's own
interaction.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-15 20:58:29 +00:00
Stephen Warren
93134e18e8 test/py: handle exceptions in console creation
u_boot_console.exec_attach.get_spawn() performs two steps:
1) Spawn a process to communicate with the serial console.
2) Reset the board so that U-Boot starts running from scratch.

Currently, if an exception happens in step (2), no cleanup is performed on
the process created in step (1). That process stays running and may e.g.
hold serial port locks, or simply continue to read data from the serial
port, thus preventing it from reaching any other process that attempts to
read from the same serial port later. While there is error cleanup code in
u_boot_console_base.ensure_spawned(), this is not triggered since the
exception prevents assignment to self.p there, and hence the exception
handler has no object to operate upon in cleanup_spawn().

Solve this by enhancing u_boot_console.exec_attach.get_spawn() to clean
up any objects it has created.

In theory, u_boot_spawn.Spawn's constructor has a similar issue, so fix
this too.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-15 20:58:29 +00:00
Stephen Warren
1326022c2e test/py: print summary in test order
Use lists rather than sets to record the status of tests. This causes
the test summary in the HTML file to be generated in the same order as
the tests are (or would have been) run. This makes it easier to locate
the first failed test. The log for this test might have interesting
first clues re: interaction with the environment (e.g. hardware flashing,
serial console, ...) and may help tracking down external issues.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-15 20:58:28 +00:00
Stephen Warren
1235c79182 test/py: fix CONFIG_SPL test
The Python ini file parser that's used to parse .config converts all keys
to lower-case. Hence, all queries against the results must use lower-case.
Fix u_boot_console.ensure_spawned() to test CONFIG_SPL correctly, or the
connection will fail for boards that have SPL.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-15 20:58:28 +00:00
Stephen Warren
06088b0473 test/py: don't import pexpect
The code replaced pexpect with custom code long ago. Don't import the
unused module so it doesn't need to be installed.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-15 20:58:27 +00:00
Stephen Warren
d70facf89c test/py: add docs for gdbserver and pytest options
Add documentation describing the new --gdbserver feature, and some common
pytest options.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-02-15 20:58:27 +00:00
Stephen Warren
1cd85f571d test/py: run all "ut" subtests
Invoke each "ut"-based unit test as a separate pytest.

Now that the DM unit test runs under test/py, remove the manual shell
script that invokes it.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org> # v2, on sandbox
2016-02-15 20:58:26 +00:00
Stephen Warren
c82ce04a3f test/py: capture the entire U-Boot version at boot
The existing regex simply ensures that the captured version string doesn't
go past the end of a line. We really want to grab as much as possible. Do
this by explicitly including a ) character at the end of the regex to
match the last character of the version test.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-02-09 15:41:19 -07:00
Stephen Warren
d8926811fd test/py: fix off-by-one error in spawn matching code
A regex match object's .end() value is already the index after the match,
not the index of the last character in the match, so there's no need to
add 1 to point past the match.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-02-09 15:41:19 -07:00
Stephen Warren
83357fd5c2 test/py: HTML awesome!
Implement three improvements to the HTML log file:
- Ability to expand/contract sections. All passing sections are contracted
  at file load time so the user can concentrate on issues requiring
  action.
- The overall status report is copied to the top of the log for easy
  access.
- Add links from the status report to the test logs, for easy navigation.

This all relies on Javascript and the jquery library. If the user doesn't
have Javascript enabled, or jquery can't be downloaded, the log should
look and behave identically to how it did before this patch.

A few notes on the diff:

- A few more 'with log.section("xxx")' were added, so that all stream
  blocks are kept within a section block for consistent HTML entity
  nesting structure. This changed indentation in a few places, making
  the diff look slightly larger.
- HTML entity IDs are cleaned up. We assign simple incrementing integer
  IDs now, rather than using mangled test names which were possibly
  invalid.
- Sections and streams now use common CSS class names (in addition to the
  current separate class names) to more easily share the new behaviour.
  This also reduces the CSS file size since rules don't need to be
  duplicated.
- An "OK" status is logged after some external command executions so that
  make and flash steps are auto-contracted at log file load time, assuming
  they passed.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-02-09 15:41:19 -07:00
Stephen Warren
ac99831b7d test/py: exit(1) if there are problems running py.test
The test/py/test.py wrapper script catches exceptions thrown when
exec()ing py.test in order to print a helpful error message. However,
the exception handling code squashes the exception and so the script
exits with a non-zero exit code, leading callers to believe that it
passed. Fix this.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-02-09 15:41:19 -07:00
Stephen Warren
89ab841088 test/py: support running sandbox under gdbserver
Implement command--line option --gdbserver COMM, which does two things:

a) Run the sandbox process under gdbserver, using COMM as gdbserver's
   communication channel.

b) Disables all timeouts, so that if U-Boot is halted under the debugger,
   tests don't fail. If the user gives up in the middle of a debugging
   session, they can simply CTRL-C the test script to abort it.

This allows easy debugging of test failures without having to manually
re-create the failure conditions. Usage is:

Window 1:
./test/py/test.py --bd sandbox --gdbserver localhost:1234

Window 2:
gdb ./build-sandbox/u-boot -ex 'target remote localhost:1234'

When using this option, it likely makes sense to use pytest's -k option
to limit the set of tests that are executed.

Simply running U-Boot directly under gdb (rather than gdbserver) was
also considered. However, this was rejected because:

a) gdb's output would then be processed by the test script, and likely
   confuse it causing false failures.

b) pytest by default hides stdout from tests, which would prevent the
   user from interacting with gdb.

   While gdb can be told to redirect the debugee's stdio to a separate
   PTY, this would appear to leave gdb's stdio directed at the test
   scripts and the debugee's stdio directed elsewhere, which is the
   opposite of the desired effect. Perhaps some complicated PTY muxing
   and process hierarchy could invert this. However, the current scheme
   is simple to implement and use, so it doesn't seem worth complicating
   matters.

c) Using gdbserver allows arbitrary debuggers to be used, even those with
   a GUI. If the test scripts invoked the debugger themselves, they'd have
   to know how to execute arbitary applications. While the user could hide
   this all in a wrapper script, this feels like extra complication.

An interesting future idea might be a --gdb-screen option, which could
spawn both U-Boot and gdb separately, and spawn the screen into a newly
created window under screen. Similar options could be envisaged for
creating a new xterm/... too.

--gdbserver  currently only supports sandbox, and not real hardware.
That's primarily because the test hooks are responsible for all aspects of
hardware control, so there's nothing for the test scripts themselves can
do to enable gdbserver on real hardware. We might consider introducing a
separate --disable-timeouts option to support use of debuggers on real
hardware, and having --gdbserver imply that option.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2016-02-08 10:22:39 -05:00
Stephen Warren
26db3a617b test/py: dfu: allow boardenv to specify test sizes
Allow the env__dfu_configs boardenv data to specify the set of DFU
transfer sizes to test. Manually specifying test sizes is useful if you
wish to test multiple DFU configurations (e.g. SD card ext4 filesystem, SD
card whole raw partition, RAM, etc.), but don't want to test every
single transfer size on each, to avoid bloating the overall time taken by
testing. If the boardenv doesn't specify a set of sizes, the built-in list
is used as a default, preserving backwards-compatibility.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
db261f0076 test/py: fix a couple typos in comments
s/updata/update/.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
d27f2fc1e1 test/py: run sandbox in source directory
Some unit tests expect the cwd of the sandbox process to be the root
of the source tree. Ensure that requirement is met.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
77bcb22d77 test/py: pass test DTB to sandbox
This is required for at least "ut dm" to operate correctly.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
78b39cc3e1 test/py: correctly log xfail/xpass tests
Tests can complete in passed, skipped, xpass, xfailed, or failed, states.
Currently the U-Boot log generation code doesn't handle the xfailed or
xpass states since they aren't used. Add support for the remaining states.
Without this, tests that xfail end up being reported as skipped.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
9129d9f5fd test/py: detect another "bad pattern" in console output
Many error situations in U-Boot print the message:
    ### ERROR ### Please RESET the board ###

Add this to the list of bad patterns the test system detects. One
practical advantage of this change is to detect the case where sandbox
is told to use a particular DTB file, and the file cannot be opened.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
0c6189b5d6 test/py: check for bad patterns everywhere we wait
Currently, bad patterns are only honored when executing a shell command.
Other cases, such as the initial boot-up of U-Boot or when interacting
with command output rather than gathering all output prior to the shell
prompt, do not currently look for bad patterns in console output. This
patch makes sure that bad patterns are honored everywhere.

One benefit of this change is that if U-Boot sandbox fails to start up,
the error message it emits can be caught immediately, rather than relying
on a (long) timeout when waiting for the expected signon message and/or
command prompt.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
e4119ebb6d test.py: calculate bad patterns on change only
A future patch will use the bad_patterns array in multiple places. Rather
than duplicating the code to calculate it, or even sharing it in a
function and simply calling it redundantly when nothing has changed, only
re-calculate the list when some change is made to it. This reduces work.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
44ac762b14 test/py: fix spawn.expect multiple match handling
Multiple patterns may be passed to spawn.expect(). The pattern which
matches at the earliest position should be designated as the match. This
aspect works correctly. When multiple patterns match at the same position,
priority should be given the the earliest entry in the list of patterns.
This aspect does not work correctly. This patch fixes it.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
d20e5e976f test/py: Provide custom IDs when parametrizing tests
When pytest generates the name for parametrized tests, simple parameter
values (ints, strings) get used directly, but more complex values such
as dicts are not handled. This yields test names such as:

    dfu[env__usb_dev_port0-env__dfu_config0]
    dfu[env__usb_dev_port0-env__dfu_config1]

Add some code to extract a custom fixture ID from the fixture values, so
that we end up with meaningful names such as:

    dfu[micro_b-emmc]
    dfu[devport2-ram]

If the boardenv file doesn't define custom names, the code falls back to
the old algorithm.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00
Stephen Warren
a2ec560647 test/py: Quote consistency
When converting test/py from " to ', I missed a few places (or added a
few inconsistencies later). Fix these.

Note that only quotes in code are converted; double-quotes in comments
and HTML are left as-is, since English and HTML use " not '.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2016-01-28 21:01:24 -07:00