Commit graph

114 commits

Author SHA1 Message Date
Stephen Warren
117eeb7f84 test/py: fix anchors in HTML status report
The current code wraps a pre tag inside an a tag. For some reason, this
causes at least Firefox to attempt to drag the pre section content when
using a mouse drag to select text. Re-order the tags so that the text can
be selected using the mouse, at least if you start the drag outside the
text (after the end of the line, for example).

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:53 -04:00
Stephen Warren
2d26bf6c26 test/py: add skip marker for reliance on tools
Some tests use external tools (executables) during their operation. Add
a test.py mark to indicate this. This allows those tests to be skipped if
the required tool is not present.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:53 -04:00
Stephen Warren
d5170448ae test/py: provide more information about test skip reason
When skipping tests, explicitly mention the board type or config option
that caused the skip. This will help people understand/fix any issues.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:52 -04:00
Stephen Warren
af3fae0b1b test/py: Document required tools/packages
Some tests rely on external tools. Mention these in the test/py README.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
2017-09-29 14:07:52 -04:00
Stephen Warren
110ba62519 test/py: gpt: make use of infra-structure
Make various changes to the GPT test:

1) Reference the disk image using an absolute path in all cases. This
allows test/py to operate correctly if it's run from a directory other
than the root of the U-Boot source tree.

2) Store the disk image in the teswt/py persistent data directory. This
removes the need to re-generate it every time the tests are run.

3) Execute sgdisk using u_boot_utils.run_and_log() so that its output is
captured in the test log. This allows debugging any problems running it.

4) Make the disk image a test fixture. This removes the requirement to
always run all GPT tests, and run them in order. The current code doesn't
create the disk image if e.g. just test_gpt_uuid() is executed via the
test.py -k command-line option.

5) Use @pytest.mark.buildconfigspec for all feature dependencies, rather
than manually implementing some of them.

6) Make all tests depend on sandbox, since they use the sandbox-specific
host command.

Fixes: a2f422555f ("add pytests for 'gpt guid' command in sandbox")
Fixes: c5772188ed ("add pytests for 'gpt rename' and 'gpt swap'")
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Tom Rini <trini@konsulko.com>
2017-09-15 22:33:41 -04:00
Heinrich Schuchardt
cd3e8a72a6 test/py: u_boot_console_base.py: fix typo
run_command does not have a parameter wait_for_each.

Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
2017-09-15 08:05:12 -04:00
Alison Chaiken
c5772188ed add pytests for 'gpt rename' and 'gpt swap'
Add unit tests for the 'gpt rename' and 'gpt swap' commands that
rely on the block device created by test/py/make_test_disk.py.
Add CONFIG_CMD_GPT_RENAME to the sandbox_defconfig.  Remove the
testdisk.raw test device at the end of the tests.

Signed-off-by: Alison Chaiken <alison@peloton-tech.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-09-14 21:32:59 -04:00
Alison Chaiken
a2f422555f add pytests for 'gpt guid' command in sandbox
Run unit tests for the 'gpt guid' command, making use of the block
device created by test/py/make_test_disk.py.  Remove this device at
the end of the tests.

Signed-off-by: Alison Chaiken <alison@peloton-tech.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-09-14 21:32:57 -04:00
Alison Chaiken
bf6d76b84a GPT: create block device for sandbox testing
Provide a Python function that creates a small block device for the
purpose of testing the cmd/gpt.c or cmd/part.c functions in the u-boot
sandbox.

Signed-off-by: Alison Chaiken <alison@peloton-tech.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-09-14 21:32:57 -04:00
Simon Glass
3a37aee30f test: Move the FIT test into the correct place
Move this test so that it will run when 'make tests' is used.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-08-13 15:17:36 -04:00
Tom Rini
845e10d165 tests: test_dfu.py: Add example udev rule for host_usb_dev_node
If one does not already have a rule to create a custom device node when
a given device enumerates it can be useful to have udev create a
bus path based node to the entry in /dev/bus/usb that was just
enumerated.  Given that DFU itself does not require a /dev entry it is a
good idea to provide a rule that will generate one.

Signed-off-by: Tom Rini <trini@konsulko.com>
2017-07-22 15:36:16 -04:00
Simon Glass
2028cc59f7 sandbox: Stop printing platdata at the start of SPL
Currently we have code which prints out platform data at the start of SPL.
Now that we have tests for dtoc this is probably not necessary. Drop it.
Update test_ofplatdata to check for empty output since it is useful to
check that sandbox_spl works as expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-07-11 10:08:20 -06:00
Michal Simek
5bdb317065 test: py: hush: Add echo dependency
Some tests depends on echo command to be present.

Reported-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2017-06-05 14:13:02 -04:00
Michal Simek
a5b548138b test: py: Use global pytestmark for hush tests
All tests in test_hush_if_test depends on hush parser to be
present. This patch simplify test dependencies by using global
pytestmark.

Reported-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Stephen Warren <swarren@nvidia.com>
2017-06-05 14:13:02 -04:00
Michal Simek
6b83c38d7a test: py: Add cmd_echo dependency
There is missing dependency on echo command. Mark tests which requires
echo.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2017-05-22 07:29:55 -04:00
Alexander Graf
0e4e38ae38 travis: Add efi_loader grub2 test
We have all the building blocks now to run arbitrary efi applications
in travis. The most important one out there is grub2, so let's add
a simple test to verify that grub2 still comes up.

Signed-off-by: Alexander Graf <agraf@suse.de>
2016-11-27 09:53:40 -05:00
Alexander Graf
95b62b2e28 efi_loader: Allow to compile helloworld.efi w/o bundling it
Today we can compile a self-contained hello world efi test binary that
allows us to quickly verify whether the EFI loader framwork works.

We can use that binary outside of the self-contained test case though,
by providing it to a to-be-tested system via tftp.

This patch separates compilation of the helloworld.efi file from
including it in the u-boot binary for "bootefi hello". It also modifies
the efi_loader test case to enable travis to pick up the compiled file.
Because we're now no longer bloating the resulting u-boot binary, we
can enable compilation always, giving us good travis test coverage.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-11-27 09:53:39 -05:00
Alexander Graf
4ca4b265ad tests: Add efi_loader hello world test
Now that we have working network tests and a hello world efi application
built inside our tree, we can automatically test that efi binary running
inside of U-Boot.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-11-26 15:50:53 -05:00
Alexander Graf
1bce3ad5f3 tests: net: Offset downloads to 4MB
The network test currently downloads files at 0MB offset of RAM start.
This works for most ARM systems, but x86 has weird memory layout constraints
on the first MB of RAM.

To not get caught into any of these, let's add a 4MB pad from start
of RAM to the default memory offset.

Signed-off-by: Alexander Graf <agraf@suse.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-11-26 15:50:52 -05:00
Stefan Brüns
d8c1e0331a test/py: expose config and log as session scoped fixture
If a test uses a fixture which is expensive to setup, the fixture can
possibly created with session or module scope. As u_boot_console has
function scope, it can not be used in this case.

Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Acked-by: Stephen Warren <swarren@wwwdotorg.org>
2016-11-21 14:07:27 -05:00
Stephen Warren
b0a928a15d test/py: ensure a log section exists for skipped tests
In pytest 3, runtestprotocol() may not call pytest_runtest_setup() if
the test is skipped. That call is required to create a section for the
test in the log file. If this is skipped, the call to log.end_section()
at the tail of pytest_runtest_protocol() will throw an exception. This
patch ensures that a log section always exists, both to avoid the
exception and to ensure that a consistently structured log file is
always created.

Cc: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reported-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Tom Rini <trini@konsulko.com>
2016-10-23 18:33:17 -04:00
Tom Rini
2b2c6e51e7 test/py/tests/test_sleep.py: Add check for CONFIG_CMD_MISC
We can only run this command if the sleep command is enabled and that
depends on CONFIG_CMD_MISC

Signed-off-by: Tom Rini <trini@konsulko.com>
2016-10-14 19:12:31 -04:00
Guillaume GARDET
6a2981a713 test: add NFS download test
Add a NFS download test, based on TFTP test.
Tested on i.MX6 SabreLite board.

Signed-off-by: Guillaume GARDET <guillaume.gardet@free.fr>

Cc: Tom Rini <trini@konsulko.com>
Cc: Joe Hershberger <joe.hershberger@ni.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Joe Hershberger <joe.hershberger@ni.com>
2016-10-13 12:25:36 -05:00
Tom Rini
de4be9ec17 test/py/tests/test_vboot.py: Add check that we boot the image
Make sure that when we're telling bootm to boot an image, and we expect
the image to boot we get the output from sandbox that we attempted to
run Linux and that U-Boot completed its job.

Cc: Simon Glass <sjg@chromium.org>
Cc: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Acked-by: Stephen Warren <swarren@nvidia.com>
2016-09-22 11:34:58 -04:00
Stephen Warren
4ba58bdabd test/py: match prompt only at line boundaries
This prevents capture of command output from terminating early on boards
that use a simple prompt (e.g. "=> ") that appears in the middle of
command output (e.g. crc32's "... ==> 2fa737e0").

Reported-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Stephen Warren <swarren@wwwdotorg.org>
Tested-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
2016-08-20 14:03:24 -04:00
Simon Glass
f6d34651d8 test: Adjust run_command_list() to return a list of strings
Return one string for each command that was executed. This seems cleaner.

Suggested-by: Teddy Reed <teddy.reed@gmail.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-08-05 20:55:23 -04:00
Simon Glass
27c087d58a test: Add a function to restart U-Boot
Add a proper function for this rather than using internal functions. Use it
in the single call site.

Also, do a restart at the end of the vboot test to reset to the normal
device tree.

Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Stephen Warren <swarren@nvidia.com>
2016-08-05 20:55:22 -04:00
Simon Glass
851271a71a test: vboot: Put each test variant in its own section
Use 'cons.log.section' feature to split up the test output. This makes it
easier to read.

Suggested-by: Stephen Warren <swarren@nvidia.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:22 -04:00
Simon Glass
ac9a23cffc test: Rename sha to sha_algo and pass it around
Rename this argument and pass it to each function that needs it, instead of
making it global.

Suggested-by: Stephen Warren <swarren@nvidia.com>
Suggested-by: Teddy Reed <teddy.reed@gmail.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:21 -04:00
Simon Glass
ec70f8a911 test: Drop the cmd() function
Instead of this, use the existing run_and_log() function, enhanced to
support a command string as well as a list of arguments.

Suggested-by: Stephen Warren <swarren@nvidia.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:20 -04:00
Simon Glass
72f5226894 test: Fix typos in comments
Fix some typos in various files introduced with the vboot test conversion.

Reported-by: Teddy Reed <teddy.reed@gmail.com>

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-08-05 20:55:20 -04:00
Simon Glass
7f64b1874c test: Check exit status in run_and_log_expect_exception()
This check was missed. Add it and make the message more verbose.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
Fixes: 9e17b034 (test/py: Provide a way to check that a command fails)
2016-08-05 20:55:19 -04:00
Simon Glass
bcbd0c8fe1 test: Fix typos and tidy up
Fix review comments that were missed at the time. Also explain why we need
to regenerate the device tree for each test.

Reported-by: Teddy Reed <teddy.reed@gmail.com>
Suggested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: f6349c3c (test: Add a README)
2016-08-05 20:55:19 -04:00
Michal Simek
04a4786c7c test/py: vboot can be run only at Sandbox
Getting this error:
Zynq> sb load hostfs - 100
/home/monstr/data/disk/u-boot/build-zynq_zc706/test.fit
Unknown command 'sb' - try 'help'

because sb command is present only for Sandbox
obj-$(CONFIG_SANDBOX) += host.o

that's why mark this test to be run only at Sandbox

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-07-22 14:52:04 -04:00
Stephen Warren
c9ba60c438 test/py: use absolute dts path in vboot test
Without this, the test fails if the test is run with a cwd other than the
root of the U-Boot source tree.

Fixes: 8729d58259 ("test: Convert the vboot test to test/py")
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2016-07-22 14:46:24 -04:00
Simon Glass
f60d0603ed test: Adjust the of-platdata test run condition
This should be spl_of_platdata, since otherwise it will try to run on boards
that don't support of-platdata.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-16 21:01:44 -04:00
Tom Rini
ebe621d5fb Merge git://git.denx.de/u-boot-dm 2016-07-15 08:06:22 -04:00
Simon Glass
b979d3d4c5 dm: Add a test for of-platdata
Add a simple test which checks that the of-platdata system is working
correctly. The sequence is as follows:

- SPL starts up and probes all the UCLASS_MISC drivers
- There are 3 of these in sandbox.dts
- Therefore there should be 3 U_BOOT_DEVICE() declarations in dt-platdata.c
- These should produce 3 sandbox_spl_test devices
- Each device prints out its platform data when probed
- This test checks for this output and compares it against expectations

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
a811779b17 test/py: Start sandbox SPL when enabled
When sandbox SPL is enabled we want to start that rather than U-Boot proper,
since some tests may rely on running it first.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
ebec58fbcb test/py: Provide a way to get early console output
Some tests want to check the console output from SPL or U-Boot proper.
Provide a means to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
c7f636f59d test/py: Note which console produced unexpected output
At present the SPL and U-Boot consoles both present the same error message
when the expected console output does not appear. Add "SPL" to the SPL error
message to resolve this ambiguity.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
2fedbaa4ae test/py: Handle testing with the sandbox_spl board
This board can sometimes be used for tests. Handle it the same way as
sandbox.

Note: I plan to drop the sandbox_spl board at some point and merge its
features into sandbox. So this commit may not be necessary.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 20:40:24 -06:00
Simon Glass
8729d58259 test: Convert the vboot test to test/py
Now that we have a suitable test framework we should move all tests into it.
The vboot test is a suitable candidate. Rewrite it in Python and move the
data files into an appropriate directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 18:22:40 -04:00
Simon Glass
73a9054d0f test/py: Add a helper to run a list of U-Boot commands
Some tests want to execute a sequence of commands. Add a helper for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:36 -04:00
Simon Glass
9e17b0345a test/py: Provide a way to check that a command fails
Sometimes we want to run a command and check that it fails. Add a function
to handle this. It can check the return code and also make sure that the
output contains a given error message.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-14 18:22:35 -04:00
Simon Glass
8b304a37df test/py: Add an option to execute a string containing a command
It is sometimes inconvenient to convert a string into a list for execution
with run_and_log(). Provide a helper function to do this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:35 -04:00
Simon Glass
f3d3e95ce5 test/py: Return output from run_and_log()
It is useful to be able to obtain the output from a command. Return it from
this function.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:34 -04:00
Simon Glass
86845bf38d test/py: Provide output from exceptions with RunAndLog()
Tests may want to look at the output from running a command, even if it
fails (e.g. with a non-zero return code). Provide a means to obtain this.

Another approach would be to return a class object containing both the
output and the exception, but I'm not sure if that would result in a lot
of refactoring.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:34 -04:00
Simon Glass
3b8d9d977b test/py: Allow RunAndLog() to return the output
Tests may want to look at the output from running a command. Return it so
that this is possible.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:34 -04:00
Simon Glass
0671960bee test/py: Allow tests to control the sandbox device-tree file
Normally tests will run with the test.dtb file designed for this purpose.
However, the verified boot tests need to run with their own device-tree
file, containing a public key.

Make the device-tree file a config option so that it can be adjusted by
tests. The default is to keep the current behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Teddy Reed <teddy.reed@gmail.com>
2016-07-14 18:22:33 -04:00