This script can take ages on some series. Try to limit the time by
using threads. If a few stubborn patches remain, show progress so the
user has some idea what is going on.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
This function is quite long. Moving the handling of a commit into a
separate function. This will make it easier to do the work in parallel.
Update function comments while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Patman does not do this anymore, as of this commit:
7428dc14b0 ("patman: Remove the -a option")
Drop the comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Create the necessary files to build this new package.
It is not actually clear whether this is useful, since buildman has no
purpose outside U-Boot.
Move the main program into a function so that it can easily be called by
the PyPi-created script.
Signed-off-by: Simon Glass <sjg@chromium.org>
It doesn't make much sense to expose tests when dtoc is running
outside of the U-Boot git checkout. Hide the option in this case.
Fix a long line while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Create the necessary files to build this new package.
This is needed for binman.
Move the main program into a function so that it can easily be called by
the PyPi-created script.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update this so that the directory being used is declared at the top of
the file. Use pathlib as it seems to be more modern.
Signed-off-by: Simon Glass <sjg@chromium.org>
It doesn't make much sense to expose tests when dtoc is running
outside of the U-Boot git checkout. Hide the option in this case.
Signed-off-by: Simon Glass <sjg@chromium.org>
Create the necessary files to build this new package.
It is not actually clear whether this is useful, since buildman has no
purpose outside U-Boot. It is included for completeness, since adding
this later would be more trouble.
Move the main program into a function so that it can easily be called by
the PyPi-created script.
Signed-off-by: Simon Glass <sjg@chromium.org>
Using 'str' as a variable makes it impossible to use it as a type in the
same function. Fix this by using a different name.
Signed-off-by: Simon Glass <sjg@chromium.org>
It doesn't make much sense to expose tests when buildman is running
outside of the U-Boot git checkout. Hide the option in this case
Signed-off-by: Simon Glass <sjg@chromium.org>
Tests are not packaged with patman so this file will not be accessible
when installing with pip. Move the import later in the file, when we
know the file is present.
Signed-off-by: Simon Glass <sjg@chromium.org>
The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
While our version is better, it is tricky to use it when we are trying
to package things with pip. Drop it.
Somewhat reduced functionality is provided by the upstream version[1],
along with a rather annoying message each time it is used[2] [3].
[1] pip install concurrencytest
[2] https://github.com/cgoldberg/concurrencytest/issues/12
[3] https://github.com/cgoldberg/concurrencytest/pull/14
Signed-off-by: Simon Glass <sjg@chromium.org>
Unfortunately this adds a new feature to concurrencytest and it has not
made it upstream to the project[1].
Drop it for now so we can use the upstream module. Once it is applied we
can bring this functionality back.
[1] https://github.com/cgoldberg/concurrencytest
This reverts commit ebcaafcded.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a command-line argument for setting the tooldir, so that the default
can be overridden. Add this directory to the toolpath automatically.
Create the directory if it does not already exist.
Put the default in the argument parser instead of the class, so that it
is more obvious.
Update a few tests that expect the utility name to be provided without
any path (e.g. 'futility'), so they can accept a path, e.g.
/path/to/futility
Update the documentation and add a few tests.
Improve the help for --toolpath while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman writes tools into the ~/bin directory. This is
convenient but some may be concerned about downloading unverified
binaries and running them. Place then in a special ~/.binman-tools
directory instead.
Mention this in the documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
We want to be able to change this directory. Use a class member to hold
the value, since changing a constant is not good.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is quite a useful thing to use when building since it avoids small
size changes between commits. Add a -r flag for it.
Also undefine CONFIG_LOCALVERSION_AUTO since this appends the git hash
to the version string, causing every build to be slightly different.
Signed-off-by: Simon Glass <sjg@chromium.org>
This cuts down build performance considerably and is not always needed,
when checking for build errors, etc.
Add a flag to disable it.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is sometimes useful to see the exact 'make' command used by buildman
for a commit. Add an output file for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Entry_blob_ext contains an implementation of SetAllowFakeBlob that is
identical to the one in the base Entry class, remove it.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add an entry flag called 'preserve' to indicate that an entry should be
preserved by firmware updates. Propagate this to FMAP too.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is needed to download tools, but we may not need to do this. At
present binman fails to start if HOME is not set.
Use the current directory as a default to avoid this.
Signed-off-by: Simon Glass <sjg@chromium.org>
This change add support for mkimage's -s option to kwbimage format. It will
create an kwbimage with empty data part of image (data part would contain
only required 32-bit checksum). mkimage's -s option is indicated by skipcpy
flag and it is basically in conflict with mkimage's -d (datafile) option.
"Empty" kwbimage with no data can still contain headers. For example it can
contain binary executable header which is copied by BootROM into L2SRAM.
This is useful for example for small images which can do not require DDR
RAM and can be run in L2SRAM (which do not require any initialization).
Signed-off-by: Pali Rohár <pali@kernel.org>
When mkimage was instructed to skip datafile via option -s then do not try
to validate or open datafile as it does not have to exist or to be
specified via -d option.
This change allows to use -s option for skipping datafile when -d option
for datafile was not specified.
Signed-off-by: Pali Rohár <pali@kernel.org>
When asking mkimage to create a new image file and option -d is not
specified then mkimage show human unfriendly error message:
mkimage: Can't open (null): Bad address
Without debugger it is hard to debug what is the issue. Function open() is
being called with file name set to NULL. So add a check for this and if it
happens then show human readable message that option -d was not specified.
Signed-off-by: Pali Rohár <pali@kernel.org>
Marvell BootROM can execute SPI images directly from NOR (either SPI/serial
or parallel) without copying them to DDR RAM. This is know at XIP - execute
in place. To achieve that, destination address in kwbimage must be set to
0xFFFFFFFF and execute address to the offset in bytes from the beginning of
NOR memory.
Kirkwood and Dove which use kwbimage v0 format and have SPI address space
mapped to physical memory at 0xE8000000-0xEFFFFFFF by BootROM.
Armada SoCs use kwbimage v1 format and have SPI address space mapped to
physical memory at 0xD4000000-0xD7FFFFFF and Device bus address space (used
for parallel NOR) at 0xD8000000-0xDFFFFFFF.
Add support for generating XIP kwbimages by mkimage -x flag and mark xflag
as valid option in kwbimage.c.
Signed-off-by: Pali Rohár <pali@kernel.org>
Skip filling legacy_img_hdr structure for XIP images which do not use
legacy_img_hdr structure header. Adding unwanted header to other image
formats, like kwbimage cause generation of broken image.
Signed-off-by: Pali Rohár <pali@kernel.org>
Secure boot header signature is calculated from the image header with
zeroed header checksum. Calculation is done in add_secure_header_v1()
function. So after calling this function no header member except
main_hdr->checksum can be modified. Commit 2b0980c240 ("tools: kwbimage:
Fill the real header size into the main header") broke this requirement as
final header size started to be filled into main_hdr->headersz_* members
after the add_secure_header_v1() call.
Fix this issue by following steps:
- Split header size and image data offset into two variables (headersz and
*dataoff).
- Change image_headersz_v0() and add_binary_header_v1() functions to return
real (unaligned) header size instead of image data offset.
- On every place use correct variable (headersz or *dataoff)
After these steps variable headersz is correctly filled into the
main_hdr->headersz_* members and so overwriting them in the end of the
image_create_v1() function is not needed anymore. Remove those overwriting
which effectively reverts changes in problematic commit without affecting
value in main_hdr->headersz_* members and makes secure boot header
signature valid again.
Fixes: 2b0980c240 ("tools: kwbimage: Fill the real header size into the main header")
Signed-off-by: Pali Rohár <pali@kernel.org>
Secure boot data image signature is calculated from the data image without
trailing 4-bit checksum. Commit 37cb9c15d7 ("tools: kwbimage: Simplify
aligning and calculating checksum") unintentionally broke this calculation
when it increased payloadsz variable by 4 bytes which was propagated also
into the add_secure_header_v1() function. Fix this issue by decreasing size
of buffer by 4 bytes from which is calculated secure boot data image
signature.
Fixes: 37cb9c15d7 ("tools: kwbimage: Simplify aligning and calculating checksum")
Signed-off-by: Pali Rohár <pali@kernel.org>
Variable imagesz in functions image_create_v0(), image_create_v1() and
kwbimage_set_header() stores offset to data from the beginning of the main
header. So it is not image size.
Signed-off-by: Pali Rohár <pali@kernel.org>
To make add_secure_header_v1() function more readable, call it directly
with arguments: header pointer with header size and data image pointer with
data image size. No functional change.
Signed-off-by: Pali Rohár <pali@kernel.org>
For all images except SATA is data offset in bytes. For SATA it is in LBA
format (number of sectors). This is how Marvell BootROM interprets it.
Signed-off-by: Pali Rohár <pali@kernel.org>
Use for it pretty print function: genimg_print_size(). This makes it more
human readable, like other offset and sizes printed by this tool.
Signed-off-by: Pali Rohár <pali@kernel.org>
In Dove functional specification, which use kwbimage v0, is also defined
nand block size field. So dump NAND_BLKSZ also for v0 images.
In Kirkwood functional specification, which also use kwbimage v0, this
field is not defined. So when it is zero and Kirkwood is detected, do not
dump it.
Fixes: f76ae2571f ("tools: kwbimage: Add support for dumping extended and binary v0 headers")
Signed-off-by: Pali Rohár <pali@kernel.org>
mkimage -F option (re-sign existing FIT image) signaled by fflag is not
supported by kwbimage. So mark its usage as invalid parameter.
Signed-off-by: Pali Rohár <pali@kernel.org>
Checksum for v0 image must be generated after filling all fields in the
main header. Otherwise it would be invalid.
Exactly same problem for v1 images was already fixed in the past in commit
9203c73895 ("tools: kwbimage: Fix checksum calculation for v1 images").
Fixes: 5c61710c98 ("tools: kwbimage: Properly set srcaddr in kwbimage v0")
Signed-off-by: Pali Rohár <pali@kernel.org>
kwbimage nandblocksize field is in 64 kB unit, but NAND_BLKSZ command
expects it in bytes. So do required unit conversion.
Also zero value in nandblocksize field has special meaning. When this field
is set to zero, the default block size is used. This default size is
defined by the NAND flash page size (16 KB for a 512B page or small page
NAND and 64 KB for a large page NAND flash).
Fixes: 1a8e6b63e2 ("tools: kwbimage: Dump kwbimage config file on '-p -1' option")
Signed-off-by: Pali Rohár <pali@kernel.org>
Value 0x0 for NAND_BADBLK_LOCATION/nandbadblklocation means that BBI is on
the first or second page and value 0x1 means that BBI is on the last page.
This indicates also NAND Flash Technology, value 0x0 is SLC NAND and value
0x1 is MLC NAND.
Therefore we need to dump NAND_BADBLK_LOCATION also when it is zero.
Note that in v0 images, nandbadblklocation field overlaps with ddrinitdelay
field in one union. ddrinitdelay is used in Kirkwood and nandbadblklocation
is used in Dove. For Dove images is_v0_ext should be set, so use it to
distinguish if nandbadblklocation is available or not. In v1 images there
is always nandbadblklocation field.
Fixes: 1a8e6b63e2 ("tools: kwbimage: Dump kwbimage config file on '-p -1' option")
Signed-off-by: Pali Rohár <pali@kernel.org>
Upper-bound for iterating for-loop over register set entries is incorrect.
Fix it byt calculating correct number of entries.
And fix also dumping the last entry DATA_DELAY, which is the last and not
first (zero).
Fixes: 1a8e6b63e2 ("tools: kwbimage: Dump kwbimage config file on '-p -1' option")
Signed-off-by: Pali Rohár <pali@kernel.org>
Show image type and version during parsing of kwbimage.
And show reasons in error messages when parsing failed.
This can help to debug issues with invalid images.
Signed-off-by: Pali Rohár <pali@kernel.org>
Marvell BootROM interprets execaddr of SPI/NOR XIP images as relative byte
offset from the from the beginning of the flash device. So if data image
offset and execute offset are not same then it is needed to adjust them
also in DDR RAM.
Fixes: f2c644e0b8 ("tools: kwboot: Patch destination address to DDR area for SPI image")
Signed-off-by: Pali Rohár <pali@kernel.org>
Some NOR images may be execute-in-place and do not contain DDR init code in
its kwbimage header. Such images cannot be booted over UART as BootROM
loads them to RAM. Add check that kwbimage contains DDR init code in its
header (either as binary code header or as the simple register-value set).
In some cases it is possible to load very small image into L2SRAM and when
DDR init code is not required. So check for L2SRAM load address and skip
DDR init code check in this case.
Signed-off-by: Pali Rohár <pali@kernel.org>
Before starting parsing of kwbimage, first validate that all optional v1
headers and correct. This prevents kwboot crashes on invalid input.
Signed-off-by: Pali Rohár <pali@kernel.org>
The 32-bit data checksum in UART image is not checked by the BootROM and
also Marvell tools do not generate it.
So if data checksum stored in UART image does not match calculated checksum
from the image then treat those checksum bytes as part of the executable
image code (and not as the checksum) and for compatibility with the rest of
the code manually insert data checksum into the in-memory image after the
executable code, without overwriting it.
This should allow to boot UART images generated by Marvell tools.
Signed-off-by: Pali Rohár <pali@kernel.org>
Add information of all available image types and where they should be
stored. Storage location offsets where documented from the disassembly of
the A385 BootROM image dump.
Signed-off-by: Pali Rohár <pali@kernel.org>
Add reference to Avanta Boot Flow documentation, BobCat2, AlleyCat3 and
PONCat3 BootROM Firmware documentation and links to public Marvell tools:
hdrparser.c and doimage.c
Signed-off-by: Pali Rohár <pali@kernel.org>
Despite the official specification, Marvell BootROM does not interpret
srcaddr from SATA image as number of sectors the beginning of the hard
drive, but as number of sectors relative to the main header.
To parse SATA kwbimage in the same way as Marvell BootROM, it is needed to
interpret srcaddr as relative offset to the main header. This change fixes
loading of SATA images via kwboot over UART.
Fixes: 792e423550 ("tools: kwboot: Patch source address in image header")
Signed-off-by: Pali Rohár <pali@kernel.org>
Despite the official specification, Marvell BootROM does not interpret
srcaddr from SATA image as number of sectors the beginning of the hard
drive, but as number of sectors relative to the main header.
The main header is stored at absolute sector number 1. So do not add or
subtract it when calculating with relative offsets to the main header.
Fixes: 501a54a29c ("tools: kwbimage: Fix generation of SATA, SDIO and PCIe images")
Fixes: 5c61710c98 ("tools: kwbimage: Properly set srcaddr in kwbimage v0")
Fixes: e0c243c398 ("tools: kwbimage: Validate data checksum of v1 images")
Fixes: aa6943ca31 ("kwbimage: Add support for extracting images via dumpimage tool")
Signed-off-by: Pali Rohár <pali@kernel.org>
Despite the official specification, Marvell BootROM does not interpret
srcaddr from SDIO image as offset in number of sectors (like for SATA
image), but as offset in bytes (like for all other images except SATA).
To parse SDIO kwbimage in the same way as Marvell BootROM, it is needed to
interpret srcaddr in bytes. This change fixes loading of SDIO images via
kwboot over UART.
Fixes: 792e423550 ("tools: kwboot: Patch source address in image header")
Signed-off-by: Pali Rohár <pali@kernel.org>
Despite the official specification, Marvell BootROM does not interpret
srcaddr from SDIO image as offset in number of sectors (like for SATA
image), but as offset in bytes (like for all other images except SATA).
To generate SDIO kwbimage compatible with Marvell BootROM, it is needed to
have srcaddr in bytes. This change fixes SDIO images for Armada 38x SoCs.
Fixes: 501a54a29c ("tools: kwbimage: Fix generation of SATA, SDIO and PCIe images")
Fixes: 5c61710c98 ("tools: kwbimage: Properly set srcaddr in kwbimage v0")
Fixes: e0c243c398 ("tools: kwbimage: Validate data checksum of v1 images")
Fixes: aa6943ca31 ("kwbimage: Add support for extracting images via dumpimage tool")
Signed-off-by: Pali Rohár <pali@kernel.org>
Add support for rk3588 package header in mkimage tool.
Signed-off-by: Jagan Teki <jagan@edgeble.ai>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
Using the mkimage entry with the multiple-data-files prop and having a
missing external blob result in an unexpected ValueError exception using
the --allow-missing flag.
ValueError: Filename 'missing.bin' not found in input path (...)
Fix this by using _pathname that is resolved by ObtainContents for blob
entries, ObtainContents also handles allow missing for external blobs.
Mark mkimage entry as missing and return without running mkimage when
missing entries is reported by CheckMissing.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
The current init size limit of 76KiB is too big to fit in the 64KiB SRAM
on RK3568, sync init size limit from vendor u-boot to fix this.
Set init size limit to 60KiB (-16KiB) for RK3568.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
Latest vendor TPL for RK3328 has grown past the current init size limit
of 28KiB, sync the init size limit from vendor u-boot to fix this.
Set init size limit to 30KiB (+2KiB) for RK3328.
This makes it possible to use latest vendor TPL on RK3328 without
getting a size limit error running the mkimage command.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
The rockchip-tpl entry can be used when an external TPL binary should be
used instead of the normal U-Boot TPL.
Signed-off-by: Jonas Karlman <jonas@kwiboo.se>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Kever Yang <kever.yang@rock-chips.com>
Tested-by: Eugen Hristev <eugen.hristev@collabora.com>
I had added this line locally, rebuild the image, but didn't ensure that
I had committed the correct version of the patch as well.
Fixes: 75b031ee4a ("Dockerfile: download binaries for Nokia RX-51")
Signed-off-by: Tom Rini <trini@konsulko.com>
Downloading files for a test may fail if the server is offline.
It is preferable to provide the files in our Docker image.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add the 'missing-msg' for more detailed output
on missing system firmware.
Signed-off-by: Rick Chen <rick@andestech.com>
Reviewed-by: Leo Yu-Chi Liang <ycliang@andestech.com>
Reviewed-by: Yu Chien Peter Lin <peterlin@andestech.com>
At present we show 'main section' as the top-level section name. It may
be more helpful to show the actual image name. This is tricky because
Image is a parent class of Entry_section, so there is no distinction
between an image and a section.
Update it to show the image name.
Signed-off-by: Simon Glass <sjg@chromium.org>
Build trace-cmd as part of the docker image, so that trace tests can be
used. Unfortunately the version provided by distributions is a little old
and has bugs. It also does not support specifying the time base for the
trace, which is required to convert microseconds to nanaseconds.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a second variant of the flame graph that shows records in terms of the
number of microseconds used by each call stack. This is a useful way of
seeing where time is going within the execution of U-Boot.
This requires a call stack that records the start time of each function,
as well as a way of subtracting all time consumed by child functions, so
that this time is not counted twice by the flamegraph. The time values in
the output are just for the function itself, not for its children.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to see how many times each function is called, particularly
in the context of its callers. A flamegraph is a way of showing this.
Support output in this format which can be used by the flamegraph.pl
script, to generate an SVG image for browsing.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for writing ftrace records in the 'funcgraph' format, which
shows function entry and exit points as well as the time taken by each
function.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the information in the trace header instead of reading it from the
trace records. Add debugging to check that System.map and the trace header
agree on this value.
Signed-off-by: Simon Glass <sjg@chromium.org>
The old text format is not much used anymore. Instead a new trace-cmd tool
has introduced a binary format for trace records.
Add support for generating this format. This involves removing the old
text format, adding various helpers for the new format and adjusting the
code to use an output file instead of stdout.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current use of 'profile' in some places is confusing. Update the code
to use the word 'trace' consistently. Change the flags to better match
their meaning and add some more comments.
Signed-off-by: Simon Glass <sjg@chromium.org>
The readme file for buildman is called buildman.rst.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
This is very old and does not appear to be used. The CONFIG option
enabling it is no-longer present. Drop it.
Signed-off-by: Simon Glass <sjg@chromium.org>
This seems to have been used by ppc4xx which was removed a while back.
The Kconfig does not exist so it is never built. Drop it.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is not a Kconfig option so we should not be setting it in the
Makefile. Rename it to use a CFS_ prefix, since this is still used in
mxsimage.c
In general tools should support all the features without reference to
CONFIG options, but this is left to the maintainer to look at.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes the Makefile rules or source code refers to Kconfig options
which don't exist. Update the moveconfig tool to check this and produce
a series of reports about inconsistencies.
This can then be used to generate patches to correct the problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
If image file is stored on flash partition then it contains padding, which
is not part of the image itself. Image data size is stored in the image
header. So use image size from the header instead of expecting that total
image file size is size of the header plus size of the image data. This
allows dumpimage to parse image files with padding (e.g. dumped from flash
partition).
Signed-off-by: Pali Rohár <pali@kernel.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Before reading image header, verify that image size is at least size of
the image header.
Signed-off-by: Pali Rohár <pali@kernel.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
