The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a command-line argument for setting the tooldir, so that the default
can be overridden. Add this directory to the toolpath automatically.
Create the directory if it does not already exist.
Put the default in the argument parser instead of the class, so that it
is more obvious.
Update a few tests that expect the utility name to be provided without
any path (e.g. 'futility'), so they can accept a path, e.g.
/path/to/futility
Update the documentation and add a few tests.
Improve the help for --toolpath while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some blobs are actually not necessary for the board to work correctly. Add
a property to allow this to be indicated. Missing optional blobs do not
cause a build failure.
Signed-off-by: Simon Glass <sjg@chromium.org>
Support entries which can be optional depending on their contents. This
allows special entry types which appear in the image only when needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman returns success when told to handle missing/faked blobs
or missing bintools. This is confusing since in fact the resulting image
cannot work.
Use exit code 103 to signal this problem, with a -W option to convert
it to a warning.
Rename the flag to --ignore-missing since it controls bintools also.
Add documentation about exit codes while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Collect and thereby initialize bintools before any usage but after
generation of entries. This is needed to handle bintools for compress
and decompress like other bintools.
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present fake files from a previous build appear to be real files for
a subsequent build, since they sit in the output directory.
This can cause problems, since binman may need to parse the file, e.g.
with the Intel description.bin files.
Fix this by putting them in a 'binman-fake' subdirectory. Keep a track
of the fake filename so we only create it once. Subsequent builds will
still see that the file is missing and mark it as fake.
Update a few tests to check the behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
When an image has the 'allow-repack' property, binman includes the
original offset and size properties from the image description in the
fdtmap. These are later used as the packing constraints when replacing
entries in an image, so other unconstrained entries can be freely
positioned.
Replacing an entry in an image without 'allow-repack' (and therefore the
original offsets) follows the same logic and results in entries being
merely concatenated. Instead, skip resetting the calculated offsets and
sizes to the missing originals for these images so that every entry is
constrained to its existing offset/size.
Add tests that replace an entry with smaller or equal-sized data, in an
image that doesn't allow repacking. Attempting to do so with bigger-size
data is already an error that is already being tested.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman entries can use other executables to compute their data, usually
in their ObtainContents() methods. Subclasses of Entry_section would use
bintools in their BuildSectionData() method instead, which is called
from several places including their Pack().
These binary tools are resolved correctly while building an image from a
device-tree description so that they can be used from these methods.
However, this is not being done when replacing entries in an image,
which can result in an error as the Pack() methods attempt to use them.
Collect and resolve entries' bintools also when replacing entries to fix
Pack() errors. Add a way to mock bintool usage in the testing entry type
and tests that check bintools are being resolved for such an entry.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Leave the 'expand' term for use by entry types which have an expanded
version of themselves. Rename this method to indicate that it generates
subentries.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Fix pylint errors that can be fixed and mask those that seem to be
incorrect.
A complication with binman is that it tries to avoid importing libfdt
(or anything that imports it) unless needed, so that things like help
still work if it is missing.
Note that two tests are duplicated in binman and two others have
duplicate names, so both of these issues are fixed also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Each bintool has some documentation which can be useful for the user.
Add a new command that collects this and writes it into a .rst file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Bintools can be missing, in which case binman continues operation but
reports an invalid image. Plumb in support for this and add tests for
entry types which use bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Support collecting the available bintools needed by an image, by
scanning the entries in the image.
Also add a command-line interface to access the basic bintool features,
such as listing the bintools and fetching them if needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.
To avoid these problems, write them to the output directory instead.
Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.
Signed-off-by: Simon Glass <sjg@chromium.org>
While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
In some cases entries encapsulate other data and it is useful to access
the data within. An example is the fdtmap which consists of a 16-byte
header, followed by a devicetree.
Provide an option to specify an alternative format when extracting files.
In the case of fdtmap, this is 'fdt', which produces an FDT file which can
be viewed with fdtdump.
Signed-off-by: Simon Glass <sjg@chromium.org>
Otherwise the updated image will end up in the temporary folder that is
purged after completion.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
WIth EFI we must embed the devicetree in an ELF image so that it is loaded
as part of the executable file. We want it to include the binman
definition in there also, which in some cases cannot be created until the
ELF (u-boot) is built. Add an option to binman to support writing the
updated dtb to the ELF file u-boot.out
This is useful with the EFI app, which is always packaged as an ELF file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Collect the code for printing the full help message of patman, buildman
and binman into a single function in patman.tools.
Signed-off-by: Paul Barker <paul.barker@sancloud.com>
One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.
This is for debugging purposes only.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some images may take a while to build, e.g. if they are large and use slow
compression. Support compiling sections in parallel to speed things up.
Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed to use a separate test file to fix flakiness)
Add a link to binman's documentation and adjust the files so that it is
accessible. Use the name README.rst so it is easy to discover when binman
is installed without U-Boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for this feature in the control, image and section modules, so
that expanded entries will be selected by default. So far there are no
expanded entry types, so this is a nop.
Signed-off-by: Simon Glass <sjg@chromium.org>
Extracting files to the current directory is not normally a very friendly
thing to do, but it can be warranted, e.g. in a new temporary dir. At
present binman reports an error when such an attempt is made. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
This method introduces a separation between packing and checking that is
different for sections. In order to handle compression properly, we need
to be able to deal with a section's size being smaller than the
uncompressed size of its contents. It is easier to make this work if
everything happens in the Pack() method.
The only real user of CheckEntries() is entry_Section and it can call it
directly. Drop the call from 'control' and handle it locally.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is only used by entry_Section and that class already calls it. Avoid
calling it twice. Also drop it from the documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
When a section is compressed, all entries within it are grouped together
into a compressed block of data. This obscures the start of each
individual child entry.
Avoid reporting bogus 'image-pos' properties in this case, since it is
not possible to access the entry at the location provided. The entire
section must be decompressed first.
CBFS does not support compressing whole sections, only individual files,
so needs no special handling here.
Signed-off-by: Simon Glass <sjg@chromium.org>
When an external blob is missing it can be quite confusing for the user.
Add a way to provide a help message that is shown.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Tidy up a few test functions which lack argument comments. Rename one that
has the same name as a different test.
Also fix up the comment for PrepareImagesAndDtbs().
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we look for resources based on the path of the Python module
that wants them. Instead we should use Python's pkg_resources feature
which is designed for this purpose.
Update binman to use this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some of these were not converted when binman moved to use absolute paths.
Fix them.
Also drop the import of 'test' which is a directory, not a module.
Signed-off-by: Simon Glass <sjg@chromium.org>
When binman is installed its main program is in a different directory
to its modules. This means that __file__ is different and we cannot use
it to obtain the path to etype/ from main.py
To fix this, move the function to the 'control' module, since it is
installed with all the other modules, including the etype/ directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.
U-Boot does with with ATF (ARM Trusted Firmware) today.
Add a new flag to binman to request this behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.
U-Boot does with with ATF (ARM Trusted Firmware) today.
Add a new flag to binman to request this behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the symbol information is written to binaries just before
binman exits. This is fine for entries within sections since the section
contents is calculated when it is needed, so the updated symbol values are
included in the image that is written.
However some binaries are inside entries which have already generated
their contents and do not notice that the entries have changed (e.g. Intel
IFWI).
Move the symbol writing earlier to cope with this.
Signed-off-by: Simon Glass <sjg@chromium.org>
