'make tests' fails on Ubuntu 22.04 with:
binman: ./tools/binman/binman:12: DeprecationWarning:
The distutils package is deprecated and slated for removal in Python 3.12.
Use setuptools or check PEP 632 for potential alternatives
from distutils.sysconfig import get_python_lib
./tools/binman/binman:12: DeprecationWarning:
The distutils.sysconfig module is deprecated, use sysconfig instead
from distutils.sysconfig import get_python_lib
<unittest.result.TestResult run=428 errors=0 failures=4>
AssertionError: 0 != 468
As we don't use Ubuntu 16.04 for our CI anymore drop the import.
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Binman interfaces allow attempts to replace any entry in the image with
arbitrary data. When trying to replace sections, the changes in the
section entry's data are not propagated to its child entries. This,
combined with how sections rebuild their contents from its children,
eventually causes the replaced contents to be silently overwritten by
rebuilt contents equivalent to the original data.
Add a simple test for replacing a section that is currently failing due
to this behaviour, and mark it as an expected failure. Also, raise an
error when replacing a section instead of silently pretending it was
replaced.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
A previous patch fixes binman to correctly extract FIT subentries. This
makes it easier to test replacing these entries as we can write tests
using an existing helper function that relies on extracting the replaced
entry.
Add tests that replace leaf entries in FIT subsections with data of
various sizes. Replacing the subsections or the whole FIT section does
not work yet due to the section contents being re-built from unreplaced
subentries' data.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
When reading images from a file, each entry's data is read from its
parent section as specified in the Entry.Create() call that created it.
The FIT entry type has been creating its subentries under its parent
(their grandparent), as creating them under the FIT entry resulted in an
error until FIT was converted into a proper section.
FIT subentries have their offsets relative to the FIT section, and
reading those offsets in the parent section results in wrong data. The
subentries rightfully belong under the FIT entries, so create them
there. Add tests checking that we can extract the correct data for a FIT
entry and its subentries.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman FIT entry nodes describe their subentries in an 'images' subnode,
same as how they would be written for the mkimage executable. The entry
type initially manually managed its subentries keyed by their node paths
relative to its base node. It was later converted to a proper section
while still keeping the same keys for subentries.
These subentry keys of sections are used as path fragments, so they must
not contain the path separator character '/'. Otherwise, they won't be
addressable by binman extract/replace commands. Change these keys from
the '/images/foo' forms to the subentry node names. Extend the simple
FIT tests to check for this.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
When an image has the 'allow-repack' property, binman includes the
original offset and size properties from the image description in the
fdtmap. These are later used as the packing constraints when replacing
entries in an image, so other unconstrained entries can be freely
positioned.
Replacing an entry in an image without 'allow-repack' (and therefore the
original offsets) follows the same logic and results in entries being
merely concatenated. Instead, skip resetting the calculated offsets and
sizes to the missing originals for these images so that every entry is
constrained to its existing offset/size.
Add tests that replace an entry with smaller or equal-sized data, in an
image that doesn't allow repacking. Attempting to do so with bigger-size
data is already an error that is already being tested.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman entries can use other executables to compute their data, usually
in their ObtainContents() methods. Subclasses of Entry_section would use
bintools in their BuildSectionData() method instead, which is called
from several places including their Pack().
These binary tools are resolved correctly while building an image from a
device-tree description so that they can be used from these methods.
However, this is not being done when replacing entries in an image,
which can result in an error as the Pack() methods attempt to use them.
Collect and resolve entries' bintools also when replacing entries to fix
Pack() errors. Add a way to mock bintool usage in the testing entry type
and tests that check bintools are being resolved for such an entry.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman can embed a copy of the image description into the images it
builds as a fdtmap entry, but it omits the /binman/<image-name> prefix
from the node paths while doing so. When reading an already-built image
file, entries are reconstructed using this fdtmap and their associated
nodes still lack that prefix.
Some entries like fit and vblock create intermediate files whose names
are based on an entry unique name. This name is constructed from their
node's path by concatenating the parents with dots up to the binman
node, e.g. /binman/image/foo/bar becomes 'image.foo.bar'.
However, we don't have this /binman/image prefix when replacing entries
in such an image. The /foo/bar entry we read when doing so erroneously
has the unique name of '/.foo.bar', causing permission errors when the
entry attempts to create files based on that.
Fix the unique-name generation by stopping at the '/' node like how it
stops at the binman node. As the unique names are used as filenames, add
tests that check if they're safe to use as filenames.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The conversion to bintools broke the invocation of the utility, since
the arguments are not correct. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Adds the support of the pre-load header with the image signature
to binman.
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com>
Some boards need to load an ELF file using the 'loadables' property, but
the file has segments at different memory addresses. This means that it
cannot be supplied as a flat binary.
Allow generating a separate node in the FIT for each segment in the ELF,
with a different load address for each.
Also add checks that the fit,xxx directives are valid.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
The current implementation sets up the FIT entries but then deletes the
'generator' ones so they don't appear in the final image.
This is a bit clumsy. We cannot build the image more than once, since the
generator entries are lost during the first build. Binman requires that
calling BuildSectionData() multiple times returns a valid result each
time.
Keep a separate, private list which includes the generator nodes and use
that where needed, to correct this problem. Ensure that the missing list
includes removed generator entries too.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
It doesn't make sense to use 'subnode' as a function parameter since it
is just a 'node' so far as the function is concerned. Update two functions
to use 'node' instead.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Add a new function to handling reporting errors within a particular
subnode of the FIT description. This can be used to make the format of
these errors consistent.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Some warnings have crept in, so fix those that are easy to fix.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
This should not be done in the constructor. Move it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
At present the entries are read twice, once by the entry_Section class
and once by the FIT implementation. This is harmless but can be confusing
when debugging. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Unfortunately mkimage gets upset with zero-sized files. Update the
ObtainContents() method to support specifying the size, if a fake blob is
created.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
On x86 devices having even a small amount of data can cause an overlap
between regions. For example, bayleybay complains when the intel-vga
region overlaps with u-boot-ucode:
ImagePos Offset Size Name
<none> 00000000 00800000 main-section
<none> ff800000 00000080 intel-descriptor
<none> ff800400 00000080 intel-me
<none> fff00000 00098f24 u-boot-with-ucode-ptr
<none> fff98f24 00001aa0 u-boot-dtb-with-ucode
<none> fff9a9d0 0002a000 u-boot-ucode
<none> fffb0000 00000080 intel-vga
...
It is safer to use an empty file in most cases. Add an option to set the
size for those uses that need it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
At present fake blobs are created but internally an empty blob is used.
Change it to use the contents of the faked file. Also return whether the
blob was faked, in case the caller needs to know that.
Add a TODO to put fake blobs in their own directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
This shadows the patman.tools library so rename it to avoid a pylint
warning.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
At present the fit implementation creates the output tree while
scanning the FIT description. Then it updates the tree later when the
data is known.
This works, but is a bit confusing, since it requires mixing the scanning
code with the generation code, with a fix-up step at the end.
It is actually possible to do this in two phases, one to scan everything
and the other to generate the FIT. Thus the FIT is generated in one pass,
when everything is known.
Update the code accordingly. The only functional change is that the 'data'
property for each node are now last instead of first, which is really a
more natural position. Update the affected test to deal with this.
One wrinkle is that the calculated properties (image-pos, size and offset)
are now added before the FIT is generated. so we must filter these out
when copying properties from the binman description to the FIT.
Most of the change here is splitting out some of the code from the
ReadEntries() implementation into _BuildInput(). So despite the large
diff, most of the code is the same. It is not feasible to split this patch
up, so far as I can tell.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Leave the 'expand' term for use by entry types which have an expanded
version of themselves. Rename this method to indicate that it generates
subentries.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
The word 'expand' is used for entries which generate subentries. It is
also used for entries that can have an '_expanded' version which is used
to break out its contents.
Rather than talking about expanding an entry's size, use the term
'extending'. It is slightly more precise and avoids the above conflicts.
This change renders the old 'expand-size' property invalid, so add an
error check for that.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Update the return value of this function, fix the 'create' typo and
update the documentation for clarity.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Suggested-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Rename this function to make it clear that it only reads loadable
segments. Also update the error for missing module to better match the
message emitted by Python.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Suggested-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
This allows to prefill fdt and config nodes with hash and signature
subnodes. It's just important to place the child nodes last so that
hashes do not come before the data - would be disliked by mkimage.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Adds build-sandbox in sys.path to look for libfdt,
otherwise py_test can't use binman.
Signed-off-by: Philippe Reynes <philippe.reynes@softathome.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Allow finding a symbol by its address. Also export the function to get
the file offset of a particular address, so it can be used by a script to
be added.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fix pylint errors that can be fixed and mask those that seem to be
incorrect.
A complication with binman is that it tries to avoid importing libfdt
(or anything that imports it) unless needed, so that things like help
still work if it is missing.
Note that two tests are duplicated in binman and two others have
duplicate names, so both of these issues are fixed also.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we only support expanding out FDT nodes. Make the operation
into an @operation property, so that others can be supported.
Re-arrange and tidy up the documentation so that it has separate
headings for each topic.
Signed-off-by: Simon Glass <sjg@chromium.org>
Split subnode and property processing into separate functions to make
the _AddNode() function a little smaller. Tweak a few comments.
This does not change any functionality.
Signed-off-by: Simon Glass <sjg@chromium.org>
Collecting the data from a list of entries and putting it in a file is
a useful operation that will be needed by other entry types. Put this into
a method in the Entry class.
Add some documentation about how to collect data for an entry type.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a file that has two text sections at different addresses, so we can
test this behaviour in binman, once added.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the 'args' property of the mkimage entry type is a string. This
makes it difficult to include CONFIG options in that property. In
particular, this does not work:
args = "-n CONFIG_SYS_SOC -E"
since the preprocessor does not operate within strings, nor does this:
args = "-n" CONFIG_SYS_SOC" "-E"
since the device tree compiler does not understand string concatenation.
With this new feature, we can do:
args = "-n", CONFIG_SYS_SOC, "-E";
Signed-off-by: Simon Glass <sjg@chromium.org>
Add an entry for OP-TEE Trusted OS 'BL32' payload.
This is required by platforms using Cortex-A cores with TrustZone
technology.
Signed-off-by: Roger Quadros <rogerq@kernel.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add missing-blob-help, renumber the test file, update entry-docs:
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a function which reads the segments and the entry address.
Also fix a comment nit in the tests while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
This shows an internal type at present, rather than the algorithm name.
Fix it and update the test to catch this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Binman keeps track of positions of each entry in the final image, but
currently this data is wrong for things included in FIT entries,
especially since a previous patch makes FIT a subclass of Section and
inherit its implementation.
There are three ways to put data into a FIT image. It can be directly
included as a "data" property, or it can be external to the FIT image
represented by an offset-size pair of properties. This external offset
is either "data-position" from the start of the FIT or "data-offset"
from the end of the FIT, and the size is "data-size" for both. However,
binman doesn't use the "data-offset" method while building FIT entries.
According to the Section docstring, its subclasses should calculate and
set the correct offsets and sizes in SetImagePos() method. Do this for
FIT subentries for the three ways mentioned above, and add tests for the
two ways binman can pack them in.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman's FIT entry type can have image subentries with "hash" subnodes
intended to be processed by mkimage, but not binman. However, the Entry
class and any subclass that reuses its implementation tries to process
these unconditionally. This can lead to an error when boards specify
hash algorithms that binman doesn't support, but mkimage supports.
Let entries skip processing these "hash" subnodes based on an instance
variable, and set this instance variable for FIT subsections. Also
re-enable processing of calculated and missing properties of FIT entries
which was disabled to mitigate this issue.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The binman FIT entry type shares some code with the Section entry type.
This shared code is bound to grow, since FIT entries are conceptually a
variation of Section entries.
Make FIT entry type a subclass of Section entry type, simplifying it a
bit and providing us the features that Section implements. Also fix the
subentry alignment test which now attempts to write symbols to a
nonexistent SPL ELF test file by creating it first.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Avoid AddMissingProperties() and SetCalculatedProperties() with FIT:
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman can check for missing binary tools and prints warnings if
anything required for an image is missing. The implementation of this
for the Section entry only checks the subentries, presumably because
Section does not use any binary tools itself. However, this means the
check is also skipped for subclasses of Section which might need binary
tools.
Make sure missing binary tools are checked for subclasses of the Section
entry type as well, by calling the parent class' implementation in
the relevant Section method.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman keeps track of binary tools each entry wants to use. The
implementation of this for the FIT entry only adds "mkimage", but not
the tools that would be used by its subentries.
Register the binary tools that FIT subentries will use in addition to
the one FIT itself uses, and check their existence by copying the
appropriate method from Section entry type. Also add tests that check if
these subentries can use and warn about binary tools.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman tries to expand some entries into parts that make it up, e.g.
'u-boot' into a 'u-boot-expanded' section that contains 'u-boot-nodtb'
and 'u-boot-dtb'. Entries with child entries must call ExpandEntries()
on them to build a correct image, as it's possible that unexpanded child
entries have no data of their own. The FIT entry type doesn't currently
do this, which means putting a "u-boot" entry inside it doesn't work as
expected.
Implement ExpandEntries() for FIT and add a copy of a simple FIT image
test that checks subentry expansion in FIT entries.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
This method has the same name as its class which is confusing. It is also
annoying when searching the code.
It builds a string with a colour, so rename it to build().
Signed-off-by: Simon Glass <sjg@chromium.org>
