We have two further uses of raw string usage in the test/py codebase
that are used under CI. The first of which is under the bind test and
is a direct update. The second of which is to strip VT100 codes from
the match buffer. While switching this to a raw string is also a direct
update, the comment it notes that problems were encountered on Ubuntu
14.04 (and whatever Python 2 version that was) that required slight
tweaks to the regex. Replace that now that we're saying Python 3.5 is
the minimum.
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
As things stand today, we have tools that CI requires where "python"
must be "python2". We need to use a virtualenv and pip in order to
ensure that our pytest tests can be run. Rework things slightly so
that:
- On Travis-CI, we install python-pyelftools for the platforms that
require pyelftools to be installed.
- On GitLab-CI, we move to a newer base image that includes python3-pip
and continue to use a virtualenv per job that needs it, for the
correct set of packages.
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
To be more closely aligned with Python community best practices, we need
to better document our usage of pip and make use of a requirements.txt
file that shows the versions of the tools that we are using. This will
aide in ensuring reproducibility of our tests as well.
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
Now that we have moved to being based on pytest for python3 we need to
make our test.py wrapper more robust in terms of only calling python3
rather than possibly finding and using python2. To do this, change from
execvp()'ing pytest to invoking the package itself via python. In the
event that pytest is unavailable we still get a user-friendly error:
pkg_resources.DistributionNotFound: The 'pytest' distribution was not found and is required by the application
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
The check_output function from the subprocess Python module by default
returns data as encoded bytes and leaves decoding to the application.
Given our uses of the call, it makes the most sense to immediately
decode the results.
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
- In python 3 you must use raw strings for regex as other forms are
deprecated and would require further changes to the pattern here.
In one case this lets us have a simpler match pattern.
- As strings are now Unicode our complex tests (Euro symbol,
SHIFT+ALT+FN 5) we need to declare that as a bytes string and then
decode it for use.
Cc: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
In the case of some unit tests we are working with providing a fake
flash device that we have written some text strings in to. In this case
we want to tell Python to encode things to bytes for us.
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
- Modern pytest is more visible in telling us about parameters that we
had not described, so describe a few more.
- ConfigParser.readfp(...) is now configparser.read_file(...)
- As part of the "strings vs bytes" conversions in Python 3, we use the
default encoding/decoding of utf-8 but in some places tell Python to
replace problematic conversions rather than throw a fatal error.
- Fix a typo noticed while doing the above ("tot he" -> "to the").
- As suggested by Stephen, re-alphabetize the import list
- Per Heinrich, replace how we write contents in test_fit.py
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
Use the 2to3 tool to perform numerous automatic conversions from Python
2 syntax to Python 3. Also fix whitespace problems that Python 3
catches that Python 2 did not.
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
Fix the following spit from pytest:
u-boot/test/py/conftest.py:438: RemovedInPytest4Warning: MarkInfo objects are deprecated as they contain merged marks which are hard to deal with correctly.
Please use node.get_closest_marker(name) or node.iter_markers(name).
Docs: https://docs.pytest.org/en/latest/mark.html#updating-code
for board in mark.args:
In both cases, the later suggestion is applicable.
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Marek Vasut <marek.vasut+renesas@gmail.com>
Cc: Igor Opaniuk <igor.opaniuk@gmail.com>
[trini: Update for current file with a few more cases, un-pin pytest in CI]
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
We inconsistently note multiple dependencies today in our tests,
sometimes with a single line that declares multiple and sometimes
multiple single lines. Current pytest seems to fail on the single line
format so change to multiple declarations.
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Tested-by: Simon Glass <sjg@chromium.org> [on sandbox]
Signed-off-by: Tom Rini <trini@konsulko.com>
Currently we set the entire PATH rather than prepend the new paths that
we need to have searched. This however breaks parts of the "virtualenv"
that was have set up and need to use as that also will be modifying
PATH. To fix this, prepend our new locations instead.
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
In a number of our stanzas we had multi-line commands that were one
space short of alignment, correct this.
Reviewed-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
- Add I2C clocks for i.MX6Q CCF driver
- Fix check in clk_set_default_parents()
- Managed API to get clock from device tree
- Fixes for core clock code (including sandbox regression tests)
'make tests' on an 32bit ARM system leads to
In file included from ../lib/efi_loader/efi_variable.c:9:
../include/malloc.h:364:7: error: conflicting types for ‘memset’
void* memset(void*, int, size_t);
^~~~~~
In file included from ../include/compiler.h:126,
from ../include/env.h:12,
from ../lib/efi_loader/efi_variable.c:8:
../include/linux/string.h:103:15:
note: previous declaration of ‘memset’ was here
extern void * memset(void *,int,__kernel_size_t);
^~~~~~
In file included from ../lib/efi_loader/efi_variable.c:9:
../include/malloc.h:365:7: error: conflicting types for ‘memcpy’
void* memcpy(void*, const void*, size_t);
^~~~~~
In file included from ../include/compiler.h:126,
from ../include/env.h:12,
from ../lib/efi_loader/efi_variable.c:8:
../include/linux/string.h:106:15:
note: previous declaration of ‘memcpy’ was here
extern void * memcpy(void *,const void *,__kernel_size_t);
^~~~~~
Use common.h as first include as recommended by the U-Boot coding style
guide.
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
With this patch, when setting UEFI variable with "env set -e" command,
we will be able to
- specify vendor guid with "-guid guid",
- specify variable attributes, BOOTSERVICE_ACCESS, RUNTIME_ACCESS,
respectively with "-bs" and "-rt",
- append a value instead of overwriting with "-a",
- use memory as variable's value instead of explicit values given
at the command line with "-i address,size"
If guid is not explicitly given, default value will be used.
Meanwhile, "env print -e," will be modified so that it will NOT dump
a variable's value if '-n' is specified.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
The ext4 file system requires log2blksz to be set. So when setting the
block size on the block descriptor we should fill this field too.
This fixes a problem with EFI block devices providing ext4 partitions, cf.
https://lists.denx.de/pipermail/u-boot/2019-October/387702.html.
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
This expands current Azure Pipelines Windows host tools build
testing to cover all the CI testing in gitlab and travis CI.
Note for some unknown reason, the 'container' cannot be used for
any jobs that have buildman, for buildman does not exit properly
and hangs the job forever. As a workaround, we manually call
docker to run the image to perform the CI tasks.
A complete run on Azure Pipelines takes about 2 hours and 10
minutes.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
At present some boards generate kwbimage.cfg in the source tree
during the build. This breaks buildman testing on some systems
where the source tree is read-only. Update makefile rules to
generate it in the build tree instead.
Note some other boards have the kwbimage.cfg file written in
advance, hence we need check if the file exists in the build
tree first, otherwise we fall back to one in the source tree.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Stefan Roese <sr@denx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
In the 'Make' function, the codes tries to create a directory
if current stage is 'build'. But the directory isn't used at
all anywhere.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
buildman always generates boards.cfg in the U-Boot source tree.
When '-o' is given, we should generate boards.cfg to the given
output directory.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
On i.MX8, when booting from eMMC boot partition, the whole flash.bin
is stored in boot partition, however SPL switches to user partition
during the init of mmc driver:
spl_mmc_load() -> mmc_init()
Then it tries to load the container image in
spl_mmc_get_uboot_raw_sector(), but here it reads the data from user
partition and the header is not recognized as a valid header.
So we move spl_mmc_get_uboot_raw_sector after eMMC partition switch
to address this issue.
Anyway put spl_mmc_get_uboot_raw_sector before eMMC partition switch
is not correct, so let's move it after eMMC partition switch.
Reported-by: Anatolij Gustschin <agust@denx.de>
Signed-off-by: Peng Fan <peng.fan@nxp.com>
A previous patch below adding DDR mode support was actually for i.MX
platforms. Now i.MX eSDHC driver is fsl_esdhc_imx.c. For QorIQ eSDHC,
it uses different process for DDR mode, and hasn't been supported.
Let's drop DDR support code for i.MX in fsl_esdhc driver.
0e1bf61 mmc: fsl_esdhc: Add support for DDR mode
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
U-boot prefers DM_MMC + BLK for MMC. Now eSDHC driver has already
support it, so let's force to use it.
- Drop non-BLK support for DM_MMC introduced by below patch.
66fa035 mmc: fsl_esdhc: fix probe issue without CONFIG_BLK enabled
- Support only DM_MMC + BLK (assuming BLK is always enabled for DM_MMC).
- Use DM_MMC instead of BLK for conditional compile.
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
Microsoft Azure Pipelines [1] provides unlimited CI/CD minutes and
10 parallel jobs to every open source project for free.
This adds a configuration file for Azure Pipelines to utilize the
free Windows VM hosted by Microsoft to ensure no build broken in
building U-Boot host tools for Windows.
[1] https://azure.microsoft.com/en-us/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
When building U-Boot host tools for Windows from Microsoft Azure
Pipelines, the following errors were seen:
HOSTCC tools/mkenvimage.o
In file included from tools/mkenvimage.c:25:
./tools/version.h:1:1: error: expected identifier or ‘(’ before ‘.’ token
1 | ../include/version.h
| ^
tools/mkenvimage.c: In function ‘main’:
tools/mkenvimage.c:117:4: warning: implicit declaration of function ‘usage’ [-Wimplicit-function-declaration]
117 | usage(prg);
| ^~~~~
tools/mkenvimage.c:120:35: error: ‘PLAIN_VERSION’ undeclared (first use in this function)
120 | printf("%s version %s\n", prg, PLAIN_VERSION);
| ^~~~~~~~~~~~~
tools/mkenvimage.c:120:35: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [scripts/Makefile.host:114: tools/mkenvimage.o] Error 1
It turns out tools/version.h is a symbolic link and with Windows
default settings it is unsupported hence the actual content of
tools/version.h is not what file include/version.h has, but the
the linked file path, which breaks the build.
To fix this, remove the symbolic links for tools/version.h. Instead
we perform a copy from include/version.h during the build.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
When building U-Boot host tools for Windows from Microsoft Azure
Pipelines, we see tons of weird warnings and errors emitted from
every Kconfig files:
Kconfig:6:warning: ignoring unsupported character ''
Kconfig:6:warning: ignoring unsupported character ''
Kconfig:8:warning: ignoring unsupported character ''
Kconfig:9:warning: ignoring unsupported character ''
Kconfig:10:warning: ignoring unsupported character ''
Kconfig:10:warning: ignoring unsupported character ''
Kconfig:13:warning: ignoring unsupported character ''
arch/Kconfig:1:warning: ignoring unsupported character ''
arch/Kconfig:2:warning: ignoring unsupported character ''
arch/Kconfig:2:warning: ignoring unsupported character ''
arch/Kconfig:4:warning: ignoring unsupported character ''
...
After several rounds of experiments, it turns out this is caused
by line endings. Historically, Linux and macOS used linefeed (LF)
characters while Windows used a carriage return plus a linefeed
(CRLF). When Azure Pipelines checks out the U-Boot repo, Git tries
to compensate for the difference by automatically making lines end
in CRLF in the working directory on Windows, which confuses the
Kconfig file parsing logic.
Fortunately Git provides a way for repos to tell Git not to do such
automatical line endings conversion via .gitattributes file below:
* text eol=lf
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
This adds a reST document for how to build U-Boot host tools,
including information for both Linux and Windows.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
'struct ustat' uses linux-specific typedefs to declare its memebers:
__kernel_daddr_t and __kernel_ino_t. It is currently not used by any
U-Boot codes, but when we build U-Boot tools for other platform like
Windows, this becomes a problem.
Let's surround it with __linux__.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
__swab32() is a Linux specific macro defined in linux/swab.h. Let's
use the compiler equivalent builtin function __builtin_bswap32() for
better portability.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
__leXX has Linux kernel specific __attribute__((bitwise)) which is
not portable. Use corresponding uintXX_t instead.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Use log() insted of debug() for uclass_find_device_by_seq function,
since this print is very much and we can filter it out with log()
interface.
Signed-off-by: Kever Yang <kever.yang@rock-chips.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Move #define to top of file as per docs:
Signed-off-by: Simon Glass <sjg@chromium.org>
There is a contributor in Linux kernel with a comma in their name, which
confuses patman and results in invalid to- or cc- addresses on some
patches. To avoid this, let's use \0 as a separator when generating cc
file.
Signed-off-by: Dmitry Torokhov <dmitry.torokhov@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
It is possible to enable bootstage in TPL. TPL can stash the info for SPL.
But at present this information is then lost because SPL does not read
from the stash.
Add support for SPL not being the first phase to enable bootstage.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present bootstage in TPL and SPL use the same ID so it is not possible
to see the timing of each. Separate out the IDs and use the correct one
depending on which phase we are at.
Example output:
Timer summary in microseconds (14 records):
Mark Elapsed Stage
0 0 reset
224,787 224,787 TPL
282,248 57,461 end TPL
341,067 58,819 SPL
925,436 584,369 end SPL
931,710 6,274 board_init_f
1,035,482 103,772 board_init_r
1,387,852 352,370 main_loop
1,387,911 59 id=175
Accumulated time:
196 dm_r
8,300 dm_spl
14,139 dm_f
229,121 fsp-m
262,992 fsp-s
Signed-off-by: Simon Glass <sjg@chromium.org>
At present bootstage relocation assumes that it is possible to point back
to memory available before relocation, so it does not relocate the
strings. However this is not the case on some platforms, such as x86 which
uses the cache as RAM and loses access to this when the cache is enabled.
Move the relocation step to before U-Boot relocates, expand the allocated
region to include space for the strings and relocate the strings at the
same time as the bootstage records.
This ensures that bootstage data can remain accessible from TPL through
SPL to U-Boot before/after relocation.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present there is a single shared address for bootstage data in both
TPL and SPL. If SPL unstashs TPL bootstage info and then stashes it again
to pass it to U-Boot, the new stash overwrites the strings of the old
stash.
Fix this by duplicating the strings into the malloc() region. This should
be a small code. Fix the header-file order at the same time.
This problem doesn't happen at the next stage (SPL->U-Boot) since U-Boot
relocates the boostage data.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current code searches for empty records but these not existing with
bootstage now. This used to be needed when bootstage records were stored
in a spare array.
Drop the unnecessary code and fix a code-style nit at the same time.
Signed-off-by: Simon Glass <sjg@chromium.org>
When stashing bootstage info, store the next ID so that it can be used
when the stash is restored. This avoids the ID starting at zero and
potentially overwriting existing entries.
Signed-off-by: Simon Glass <sjg@chromium.org>
Make sure that the bloblist starts on an aligned boundary. This protects
against one of the early allocating causing the alignment to be lost.
Signed-off-by: Simon Glass <sjg@chromium.org>
With a bit of code reordering we can support %p using the existing code
for ulong.
Move the %p code up and adjust the logic accordingly.
Signed-off-by: Simon Glass <sjg@chromium.org>
