Binman's FIT entry type can have image subentries with "hash" subnodes
intended to be processed by mkimage, but not binman. However, the Entry
class and any subclass that reuses its implementation tries to process
these unconditionally. This can lead to an error when boards specify
hash algorithms that binman doesn't support, but mkimage supports.
Let entries skip processing these "hash" subnodes based on an instance
variable, and set this instance variable for FIT subsections. Also
re-enable processing of calculated and missing properties of FIT entries
which was disabled to mitigate this issue.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The binman FIT entry type shares some code with the Section entry type.
This shared code is bound to grow, since FIT entries are conceptually a
variation of Section entries.
Make FIT entry type a subclass of Section entry type, simplifying it a
bit and providing us the features that Section implements. Also fix the
subentry alignment test which now attempts to write symbols to a
nonexistent SPL ELF test file by creating it first.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Avoid AddMissingProperties() and SetCalculatedProperties() with FIT:
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman can check for missing binary tools and prints warnings if
anything required for an image is missing. The implementation of this
for the Section entry only checks the subentries, presumably because
Section does not use any binary tools itself. However, this means the
check is also skipped for subclasses of Section which might need binary
tools.
Make sure missing binary tools are checked for subclasses of the Section
entry type as well, by calling the parent class' implementation in
the relevant Section method.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman keeps track of binary tools each entry wants to use. The
implementation of this for the FIT entry only adds "mkimage", but not
the tools that would be used by its subentries.
Register the binary tools that FIT subentries will use in addition to
the one FIT itself uses, and check their existence by copying the
appropriate method from Section entry type. Also add tests that check if
these subentries can use and warn about binary tools.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman tries to expand some entries into parts that make it up, e.g.
'u-boot' into a 'u-boot-expanded' section that contains 'u-boot-nodtb'
and 'u-boot-dtb'. Entries with child entries must call ExpandEntries()
on them to build a correct image, as it's possible that unexpanded child
entries have no data of their own. The FIT entry type doesn't currently
do this, which means putting a "u-boot" entry inside it doesn't work as
expected.
Implement ExpandEntries() for FIT and add a copy of a simple FIT image
test that checks subentry expansion in FIT entries.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
This method has the same name as its class which is confusing. It is also
annoying when searching the code.
It builds a string with a colour, so rename it to build().
Signed-off-by: Simon Glass <sjg@chromium.org>
We can and should run the node generator only when creating a new image.
When we read it back, there is no need to generate nodes - they already
exits, and binman does not dive that deep into the image - and there is
no way to provide the required fdt-list. So store the mode in the image
object so that Entry_fit can simply skip generator nodes when reading
them from an fdtmap.
This unbreaks all read-backs of images that contain generator nodes in
their fdtmap. To confirm this, add a corresponding test case.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Add SPDX to dts file:
Signed-off-by: Simon Glass <sjg@chromium.org>
Each bintool has some documentation which can be useful for the user.
Add a new command that collects this and writes it into a .rst file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Drop the unused gzip code, update comments and add a test for an
invalid algorithm. The temporary file is not needed now, so drop that
also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Bintools can be missing, in which case binman continues operation but
reports an invalid image. Plumb in support for this and add tests for
entry types which use bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the code to use this bintool, instead of running lzma_alone
directly. This simplifies the code and provides more consistency.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to compress and decompress data.
It supports the features needed by binman as well as installing via the
lzma-alone package.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the code to use this bintool, instead of running lz4 directly. This
simplifies the code and provides more consistency.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to compress and decompress data.
It supports the features needed by binman as well as installing via the
lz4 package.
Signed-off-by: Simon Glass <sjg@chromium.org>
The compression functions are not actually used by patman, so we don't
need then in the tools module. Also we want to change them to use
bintools, which patman will not support.
Move these into a new comp_util module, within binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the fit and mkimage entry types to use this bintool, instead of
running mkimage directly. This simplifies the code and provides more
consistency as well as supporting missing bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the ifwi entry type to use this bintool, instead of running
ifwitool directly. This simplifies the code and provides more
consistency as well as supporting missing bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the GBB and vblock entry types to use this bintool, instead of
running futility directly. This simplifies the code and provides more
consistency as well as supporting missing bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the FIP tests to use this bintool, instead of running fiptool
directly. This simplifies the code and provides more consistency as well
as supporting missing bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the CBFS tests to use this bintool, instead of running cbfstool
directly. This simplifies the overall code and provides more consistency,
as well as supporting missing bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
The tests rely on having at least 5 bintool implementions. Now that we
have this, enable them. Add tests for the binman 'tool' subcommand.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to build images for use by U-Boot.
It supports the features needed by binman as well as installing via the
u-boot-tools packages. Although this is built in the U-Boot tree, it is
still useful to install a binary on the system.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to build Intel IFWI images. It
supports the features needed by the tests as well as downloading a binary
from Google Drive. Although this is built in the U-Boot tree, it is not
currently included with u-boot-tools, so it may be useful to install a
binary on the system.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to sign Chrome OS images and
build the Google Binary Block (GBB). It supports the features needed by
binman as well as fetching a binary from Google Drive. Building it from
source is possible but is left for another time, as it requires at least
one other library.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to run FIP tests. It supports
the features needed by the tests as well as building a binary from
the git tree.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a Bintool for this, which is used to run CBFS tests. It supports
the features needed by the tests as well as fetching a binary from
Google Drive. Building it from source is very slow since it is not
separately supported by the coreboot build system and it builds an
entire gcc toolchain before starting.
Signed-off-by: Simon Glass <sjg@chromium.org>
Support collecting the available bintools needed by an image, by
scanning the entries in the image.
Also add a command-line interface to access the basic bintool features,
such as listing the bintools and fetching them if needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman requires various tools to actually work, such as 'lz4' to compress
data and 'futility' to sign Chrome OS firmware. At present these are
handled in an ad-hoc manner and there is no easy way to find out what
tools are needd to build an image, nor where to obtain them.
Add an implementation of 'bintool', a base class which implements this
functionality. When a bintool is required, it can be requested from this
module, then executed. When the tool is missing, it can provide a way to
obtain it.
Note that this uses Command directly, not the tools.Run() function. This
allows proper handling of missing tools and avoids needing to catch and
re-raise exceptions.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since this is a list of blobs, each blob should have the ability to be
faked, as with blob-ext. Update the Entry base class to set allow_fake
and use the base class in the section code also, so that this propagagtes
to blob-ext-list, which is not a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this does not check that the external data is in the expected
place. Use a non-zero offset for the external data and check it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some newer toolchains do not create a symbol for the .ucode section that
this test relies on. Update the test to use the symbol that is explicitly
created, instead.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add some empty __init__ files for binman, buildman and dtoc so that
pylint is able to recognise these as Python modules and produce more
useful pylint output.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.
To avoid these problems, write them to the output directory instead.
Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use a unique number instead of the current 203, which is used by 203_fip
as well. Reformat the code to avoid a long line.
Signed-off-by: Simon Glass <sjg@chromium.org>
While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present one must hack the Makefile to see what is going on with these
files. Also it doesn't quite work correctly.
Fix this by using an environment variable for debugging. Update the docs
also.
Signed-off-by: Simon Glass <sjg@chromium.org>
This was added as a hack to work around not having an in-tree devicetree.
Now that this is fixed it is not needed.
Drop it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add the 'missing-msg' for blobs for more detailed output on missing system
firmware and SEBoot blobs.
Signed-off-by: Ivan Mikhaylov <ivan.mikhaylov@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Fix minor typos:
Signed-off-by: Simon Glass <sjg@chromium.org>
Instead of joining hard coded '..' to the run-time path of the executable,
take just a dirname out of it. Besides that, use $(srctree) where it makes
sense.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Importing libraries in Python caches the bytecode by default.
Since we run scripts in source tree it ignores the current directory
settings, which is $(srctree), and creates cache just in the middle
of the source tree. Move cache to the current directory.
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
This format is used in firmware binaries so we may as well supported it.
With this patch binman supports creating, listing and updating FIPs, as
well as extracting files from one, provided that an FDTMAP is also present
somewhere in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for this format which is used by ARM Trusted Firmware to find
firmware binaries to load.
FIP is like a simpler version of FMAP but uses a UUID instead of a name,
for each entry.
It supports reading a FIP, writing a FIP and parsing the ATF source code
to get a list of supported UUIDs.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases entries encapsulate other data and it is useful to access
the data within. An example is the fdtmap which consists of a 16-byte
header, followed by a devicetree.
Provide an option to specify an alternative format when extracting files.
In the case of fdtmap, this is 'fdt', which produces an FDT file which can
be viewed with fdtdump.
Signed-off-by: Simon Glass <sjg@chromium.org>
If an older version of binman is used to list images created by a newer
one, it is possible that it will contain entry types that are not
supported. At present this produces an error.
Adjust binman to use a plain 'blob' entry type to cope with this, so the
image can at least be listed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present it is necessary to symlink files containing external blobs into
the U-Boot tree in order for binman to find them. This is not very
convenient.
Add two new environment/Makefile variables to help with this. Add
documentation as well, fixing a related nit.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the constructor to work in the recommended way, where the node
properties are read in a separate function. This makes it more similar to
entry_Section.
Signed-off-by: Simon Glass <sjg@chromium.org>
This currently uses _cbfs_entries[] to store entries. Since the entries
are in fact valid etypes, we may as well use the same name as
entry_Section uses, which is _entries. This allows reusing more of the
code there (in a future patch).
Signed-off-by: Simon Glass <sjg@chromium.org>
It is easier to understand this file if reading the entries comes before
obtaining the contents, since that is the order in which Binman proceeds.
Move the function down a bit.
Signed-off-by: Simon Glass <sjg@chromium.org>
Expand this to explain subclassing better and also to tidy up formatting
for rST.
Fix a few pylint warnings to avoid dropping the score.
Signed-off-by: Simon Glass <sjg@chromium.org>
The ObtainContents() and GetEntryContents() methods in this file read
every single entry in the section. This is the common case.
However when one of the entries has had its data updated (e.g. with
'binman replace') we don't want to read it again from the file. Allow
the entry to be skipped, for this purpose. This is currently done in the
CBFS implementation, so adding it here will allow that to use more of
the entry_Section code.
Signed-off-by: Simon Glass <sjg@chromium.org>
This method is currently marked private. However it is useful to be able
to subclass it, since much of the entry_Section code can be reused. Rename
it.
Also document one confusing part of this code, so people can understand
how to add a test for this case.
Fix up a few pylint warnings to avoid regressing the score.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a -V option which shows the version number of binman. For now this
just uses a local 'version' file. Once the tool is packaged in some way
we can figure out an approach that suits.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update this file to improve the pylint score a little. The remaining item
is:
Function name "ParseArgs" doesn't conform to snake_case naming style
which needs some binman-wide renaming.
Signed-off-by: Simon Glass <sjg@chromium.org>
Otherwise the updated image will end up in the temporary folder that is
purged after completion.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
WIth EFI we must embed the devicetree in an ELF image so that it is loaded
as part of the executable file. We want it to include the binman
definition in there also, which in some cases cannot be created until the
ELF (u-boot) is built. Add an option to binman to support writing the
updated dtb to the ELF file u-boot.out
This is useful with the EFI app, which is always packaged as an ELF file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman needs to be able to update the contents of an ELF file after it has
been build. To support this, add a function to locate the position of a
symbol's contents within the file.
Fix the comments on bss_data.c and Symbol while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present any error from the 'make' command is silently swallowed by the
test system. Fix this by showing it when detected.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present testThreadTimeout() assumes that the expected timeout happens
first when building the section, but it can just as easily happen at the
top-level image. Update the test to cope with both.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Marek Behún <marek.behun@nic.cz>
The previous patches removed OF_PRIOR_STAGE from the last consumers of the
Kconfig option. Cleanup any references to it in documentation, code and
configuration options.
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
This allows to use the watchdog in custom scripts but does not enforce
that the OS has to support it as well.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Collect the code for printing the full help message of patman, buildman
and binman into a single function in patman.tools.
Signed-off-by: Paul Barker <paul.barker@sancloud.com>
One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.
This is for debugging purposes only.
Signed-off-by: Simon Glass <sjg@chromium.org>
The constructor should not read the node information. Move it to the
ReadNode() method instead. This allows this etype to be subclassed.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some images may take a while to build, e.g. if they are large and use slow
compression. Support compiling sections in parallel to speed things up.
Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed to use a separate test file to fix flakiness)
For scenarios like OF_BOARD or OF_PRIOR_STAGE, no device tree blob is
provided in the U-Boot build phase hence the binman node information
is not available. In order to support such use case, a new Kconfig
option BINMAN_STANDALONE_FDT is introduced, to tell the build system
that a device tree blob containing binman node is explicitly required
when using binman to package U-Boot.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add an entry for RISC-V OpenSBI's 'fw_dynamic' firmware payload.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Rick Chen <rick@andestech.com>
Reviewed-by: Rick Chen <rick@andestech.com>
Currently there are 2 binman test cases using the same 172 number.
It seems that 172_fit_fdt.dts was originally named as 170_, but
commit c0f1ebe9c1 ("binman: Allow selecting default FIT configuration")
changed its name to 172_ for no reason. Let's change it back.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
It needs a space around '-a'.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Rick Chen <rick@andestech.com>
When used with hierarchical images, use the Chromium OS convention of
adding a section before all the subentries it contains.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use an interator in two of the fmap tests so it is easier to add new
items. Also check the name first since that is the first indication
that something is wrong. Use a variable for the expected size of the
fmap to avoid repeating the code.
Signed-off-by: Simon Glass <sjg@chromium.org>
These two tests require an ELF image so that symbol information can be
written into the SPL/TPL binary. At present they rely on other tests
having set it up first, but every test must run independently. This can
cause occasional errors in CI.
Fix this by setting up the required files, as other tests do.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Sometimes it is useful to specify the default alignment for all entries
in a section, such as when word-alignment is necessary, for example. It
is tedious and error-prone to specify this individually for each section.
Add a property to control this for a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
Generally the content of sections is not built until the final assembly
of the image. This is partly to avoid wasting time, since the entries
within sections may change multiple times as binman works through its
various stages. This works quite well since sections exist in a strict
hierarchy, so they can be processed in a depth-first manner.
However the 'collection' entry type does not have this luxury. If it
contains a section within its 'content' list, then it must produce the
section contents, if available. That section is typically a sibling
node, i.e. not part oc the collection's hierarchy.
Add a new 'required' argument to section.GetData() to support this. When
required is True, any referenced sections are immediately built. If this
is not possible (because one of the subentries does not have its data yet)
then an error is produced.
The test for this uses a 'collection' entry type, referencing a section as
its first member. This forces a call to _BuildSectionData() with required
set to False, at first, then True later, when the image is assembled.
Signed-off-by: Simon Glass <sjg@chromium.org>
The vblock entry type includes code to collect the data from a number of
other entries (not necessarily subentries) and concatenating it. This is
a useful feature for other entry types.
Make it a base class, so that vblock can use it, along with other entry
types.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present there is a command-line flag to disable substitution of expanded
entries. Add an option to the entry node as well, so it can be controlled
at the node level.
Add a test to cover this. Fix up the comment to the checkSymbols() function
it uses, while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
