Add lzop bintool to binman to support on-the-fly compression.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add gzip bintool to binman to support on-the-fly compression of Linux
kernel images and FPGA bitstreams. The SPL basic fitImage implementation
supports only gzip decompression.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Rename the module and support this, since gzip.py is a system module:
Signed-off-by: Simon Glass <sjg@chromium.org>
Add bzip2 bintool to binman to support on-the-fly compression.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Handle missing compression tools by returning empty data and record
missing bintool.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Move management of the bintool to compress and decompress data into the
entry class and add the bintool to the list of required bintools.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Remove the obsolete compressed data header handling from the utilities
to compress and decompress data. The header is uncommon, not supported
by U-Boot and incompatible with external compressed artifacts.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Disable the compressed data header of the utilities to compress and
decompress data. The header is uncommon, not supported by U-Boot and
incompatible with external compressed artifacts.
The header was introduced as part of commit eb0f4a4cb4 ("binman:
Support replacing data in a cbfs") to allow device tree entries to be
larger than the compressed contents.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add a DecompressData function to entry class to allow override in child
classes and to centralize the compress and decompress in a single class.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Avoid duplicate entries in the list of bintools used by the image and
the list of missing bintools.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present the collections etype only works with entries in the same
section. This can be limiting, since in some cases the data may be inside
a subsection, e.g. if there are alignment constraints.
Add a function to find the entries in an etype and have it search
recursively. Make use of this for mkimage also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some new entries are likely to have required properties. Support this in a
standard way, with a list of required properties which can be set up by
base classes. Check for missing properties when the entry is read.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present fake files from a previous build appear to be real files for
a subsequent build, since they sit in the output directory.
This can cause problems, since binman may need to parse the file, e.g.
with the Intel description.bin files.
Fix this by putting them in a 'binman-fake' subdirectory. Keep a track
of the fake filename so we only create it once. Subsequent builds will
still see that the file is missing and mark it as fake.
Update a few tests to check the behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add references in the documentation for each entry type, so we can refer
to them from other documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman can embed a copy of the image description into the images it
builds as a fdtmap entry, but it omits the /binman/<image-name> prefix
from the node paths while doing so. When reading an already-built image
file, entries are reconstructed using this fdtmap and their associated
nodes still lack that prefix.
Some entries like fit and vblock create intermediate files whose names
are based on an entry unique name. This name is constructed from their
node's path by concatenating the parents with dots up to the binman
node, e.g. /binman/image/foo/bar becomes 'image.foo.bar'.
However, we don't have this /binman/image prefix when replacing entries
in such an image. The /foo/bar entry we read when doing so erroneously
has the unique name of '/.foo.bar', causing permission errors when the
entry attempts to create files based on that.
Fix the unique-name generation by stopping at the '/' node like how it
stops at the binman node. As the unique names are used as filenames, add
tests that check if they're safe to use as filenames.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Unfortunately mkimage gets upset with zero-sized files. Update the
ObtainContents() method to support specifying the size, if a fake blob is
created.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
On x86 devices having even a small amount of data can cause an overlap
between regions. For example, bayleybay complains when the intel-vga
region overlaps with u-boot-ucode:
ImagePos Offset Size Name
<none> 00000000 00800000 main-section
<none> ff800000 00000080 intel-descriptor
<none> ff800400 00000080 intel-me
<none> fff00000 00098f24 u-boot-with-ucode-ptr
<none> fff98f24 00001aa0 u-boot-dtb-with-ucode
<none> fff9a9d0 0002a000 u-boot-ucode
<none> fffb0000 00000080 intel-vga
...
It is safer to use an empty file in most cases. Add an option to set the
size for those uses that need it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
At present fake blobs are created but internally an empty blob is used.
Change it to use the contents of the faked file. Also return whether the
blob was faked, in case the caller needs to know that.
Add a TODO to put fake blobs in their own directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
This shadows the patman.tools library so rename it to avoid a pylint
warning.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Leave the 'expand' term for use by entry types which have an expanded
version of themselves. Rename this method to indicate that it generates
subentries.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
The word 'expand' is used for entries which generate subentries. It is
also used for entries that can have an '_expanded' version which is used
to break out its contents.
Rather than talking about expanding an entry's size, use the term
'extending'. It is slightly more precise and avoids the above conflicts.
This change renders the old 'expand-size' property invalid, so add an
error check for that.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Update the return value of this function, fix the 'create' typo and
update the documentation for clarity.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Suggested-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Fix pylint errors that can be fixed and mask those that seem to be
incorrect.
A complication with binman is that it tries to avoid importing libfdt
(or anything that imports it) unless needed, so that things like help
still work if it is missing.
Note that two tests are duplicated in binman and two others have
duplicate names, so both of these issues are fixed also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Collecting the data from a list of entries and putting it in a file is
a useful operation that will be needed by other entry types. Put this into
a method in the Entry class.
Add some documentation about how to collect data for an entry type.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman's FIT entry type can have image subentries with "hash" subnodes
intended to be processed by mkimage, but not binman. However, the Entry
class and any subclass that reuses its implementation tries to process
these unconditionally. This can lead to an error when boards specify
hash algorithms that binman doesn't support, but mkimage supports.
Let entries skip processing these "hash" subnodes based on an instance
variable, and set this instance variable for FIT subsections. Also
re-enable processing of calculated and missing properties of FIT entries
which was disabled to mitigate this issue.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Bintools can be missing, in which case binman continues operation but
reports an invalid image. Plumb in support for this and add tests for
entry types which use bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
The compression functions are not actually used by patman, so we don't
need then in the tools module. Also we want to change them to use
bintools, which patman will not support.
Move these into a new comp_util module, within binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Support collecting the available bintools needed by an image, by
scanning the entries in the image.
Also add a command-line interface to access the basic bintool features,
such as listing the bintools and fetching them if needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since this is a list of blobs, each blob should have the ability to be
faked, as with blob-ext. Update the Entry base class to set allow_fake
and use the base class in the section code also, so that this propagagtes
to blob-ext-list, which is not a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.
To avoid these problems, write them to the output directory instead.
Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.
Signed-off-by: Simon Glass <sjg@chromium.org>
While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
In some cases entries encapsulate other data and it is useful to access
the data within. An example is the fdtmap which consists of a 16-byte
header, followed by a devicetree.
Provide an option to specify an alternative format when extracting files.
In the case of fdtmap, this is 'fdt', which produces an FDT file which can
be viewed with fdtdump.
Signed-off-by: Simon Glass <sjg@chromium.org>
If an older version of binman is used to list images created by a newer
one, it is possible that it will contain entry types that are not
supported. At present this produces an error.
Adjust binman to use a plain 'blob' entry type to cope with this, so the
image can at least be listed.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful to specify the default alignment for all entries
in a section, such as when word-alignment is necessary, for example. It
is tedious and error-prone to specify this individually for each section.
Add a property to control this for a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
Generally the content of sections is not built until the final assembly
of the image. This is partly to avoid wasting time, since the entries
within sections may change multiple times as binman works through its
various stages. This works quite well since sections exist in a strict
hierarchy, so they can be processed in a depth-first manner.
However the 'collection' entry type does not have this luxury. If it
contains a section within its 'content' list, then it must produce the
section contents, if available. That section is typically a sibling
node, i.e. not part oc the collection's hierarchy.
Add a new 'required' argument to section.GetData() to support this. When
required is True, any referenced sections are immediately built. If this
is not possible (because one of the subentries does not have its data yet)
then an error is produced.
The test for this uses a 'collection' entry type, referencing a section as
its first member. This forces a call to _BuildSectionData() with required
set to False, at first, then True later, when the image is assembled.
Signed-off-by: Simon Glass <sjg@chromium.org>
The vblock entry type includes code to collect the data from a number of
other entries (not necessarily subentries) and concatenating it. This is
a useful feature for other entry types.
Make it a base class, so that vblock can use it, along with other entry
types.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present there is a command-line flag to disable substitution of expanded
entries. Add an option to the entry node as well, so it can be controlled
at the node level.
Add a test to cover this. Fix up the comment to the checkSymbols() function
it uses, while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
As the first step in supporting expanded entries, add a way for binman to
automatically select an 'expanded' version of an entry type, if requested.
This is controlled by a class method.
Signed-off-by: Simon Glass <sjg@chromium.org>
Move the documentation to the base method as it is with other methods.
Also update it a little while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally when an entry is created, any entry arguments it has are required
to be provided, so it can actually generate its contents correctly.
However when an existing image is read, Entry objects are created for each
of the entries in the image. This happens as part of the process of
reading the image into binman.
In this case we don't need the entry arguments, since we do not intend to
regenerate the entries, or at least not unless requested. So there is no
sense in reporting an error for missing entry arguments.
Add a new property for the Image to handle this case. Update the error
reporting to be conditional on this property.
Signed-off-by: Simon Glass <sjg@chromium.org>
A recent change removed the base offset from the calculation. This is
used on coral to find the FSP-S binary. Fix it.
Fixes: a9fad07d4b ("binman: Avoid reporting image-pos with compression")
Signed-off-by: Simon Glass <sjg@chromium.org>
With a recent change this entry stores only part of the section data,
leaving out the padding at the end. Fix this by using GetPaddedData() to
get the data. Add this function to the base Entry class also.
Fixes: d1d3ad7d1f ("binman: Move section padding to the parent")
Signed-off-by: Simon Glass <sjg@chromium.org>
This function just calls CheckEntries() in the only non-trivial
implementation. Drop it and use CheckEntries() directly.
Signed-off-by: Simon Glass <sjg@chromium.org>
When a section is compressed, all entries within it are grouped together
into a compressed block of data. This obscures the start of each
individual child entry.
Avoid reporting bogus 'image-pos' properties in this case, since it is
not possible to access the entry at the location provided. The entire
section must be decompressed first.
CBFS does not support compressing whole sections, only individual files,
so needs no special handling here.
Signed-off-by: Simon Glass <sjg@chromium.org>
