Binman keeps track of positions of each entry in the final image, but
currently this data is wrong for things included in FIT entries,
especially since a previous patch makes FIT a subclass of Section and
inherit its implementation.
There are three ways to put data into a FIT image. It can be directly
included as a "data" property, or it can be external to the FIT image
represented by an offset-size pair of properties. This external offset
is either "data-position" from the start of the FIT or "data-offset"
from the end of the FIT, and the size is "data-size" for both. However,
binman doesn't use the "data-offset" method while building FIT entries.
According to the Section docstring, its subclasses should calculate and
set the correct offsets and sizes in SetImagePos() method. Do this for
FIT subentries for the three ways mentioned above, and add tests for the
two ways binman can pack them in.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman's FIT entry type can have image subentries with "hash" subnodes
intended to be processed by mkimage, but not binman. However, the Entry
class and any subclass that reuses its implementation tries to process
these unconditionally. This can lead to an error when boards specify
hash algorithms that binman doesn't support, but mkimage supports.
Let entries skip processing these "hash" subnodes based on an instance
variable, and set this instance variable for FIT subsections. Also
re-enable processing of calculated and missing properties of FIT entries
which was disabled to mitigate this issue.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The binman FIT entry type shares some code with the Section entry type.
This shared code is bound to grow, since FIT entries are conceptually a
variation of Section entries.
Make FIT entry type a subclass of Section entry type, simplifying it a
bit and providing us the features that Section implements. Also fix the
subentry alignment test which now attempts to write symbols to a
nonexistent SPL ELF test file by creating it first.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Avoid AddMissingProperties() and SetCalculatedProperties() with FIT:
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman keeps track of binary tools each entry wants to use. The
implementation of this for the FIT entry only adds "mkimage", but not
the tools that would be used by its subentries.
Register the binary tools that FIT subentries will use in addition to
the one FIT itself uses, and check their existence by copying the
appropriate method from Section entry type. Also add tests that check if
these subentries can use and warn about binary tools.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Binman tries to expand some entries into parts that make it up, e.g.
'u-boot' into a 'u-boot-expanded' section that contains 'u-boot-nodtb'
and 'u-boot-dtb'. Entries with child entries must call ExpandEntries()
on them to build a correct image, as it's possible that unexpanded child
entries have no data of their own. The FIT entry type doesn't currently
do this, which means putting a "u-boot" entry inside it doesn't work as
expected.
Implement ExpandEntries() for FIT and add a copy of a simple FIT image
test that checks subentry expansion in FIT entries.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
We can and should run the node generator only when creating a new image.
When we read it back, there is no need to generate nodes - they already
exits, and binman does not dive that deep into the image - and there is
no way to provide the required fdt-list. So store the mode in the image
object so that Entry_fit can simply skip generator nodes when reading
them from an fdtmap.
This unbreaks all read-backs of images that contain generator nodes in
their fdtmap. To confirm this, add a corresponding test case.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Add SPDX to dts file:
Signed-off-by: Simon Glass <sjg@chromium.org>
Each bintool has some documentation which can be useful for the user.
Add a new command that collects this and writes it into a .rst file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Drop the unused gzip code, update comments and add a test for an
invalid algorithm. The temporary file is not needed now, so drop that
also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Bintools can be missing, in which case binman continues operation but
reports an invalid image. Plumb in support for this and add tests for
entry types which use bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the code to use this bintool, instead of running lz4 directly. This
simplifies the code and provides more consistency.
Signed-off-by: Simon Glass <sjg@chromium.org>
The compression functions are not actually used by patman, so we don't
need then in the tools module. Also we want to change them to use
bintools, which patman will not support.
Move these into a new comp_util module, within binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Update the ifwi entry type to use this bintool, instead of running
ifwitool directly. This simplifies the code and provides more
consistency as well as supporting missing bintools.
Signed-off-by: Simon Glass <sjg@chromium.org>
The tests rely on having at least 5 bintool implementions. Now that we
have this, enable them. Add tests for the binman 'tool' subcommand.
Signed-off-by: Simon Glass <sjg@chromium.org>
Support collecting the available bintools needed by an image, by
scanning the entries in the image.
Also add a command-line interface to access the basic bintool features,
such as listing the bintools and fetching them if needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since this is a list of blobs, each blob should have the ability to be
faked, as with blob-ext. Update the Entry base class to set allow_fake
and use the base class in the section code also, so that this propagagtes
to blob-ext-list, which is not a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this does not check that the external data is in the expected
place. Use a non-zero offset for the external data and check it.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman writes fake blobs to the current directory. This is not
very helpful, since the files serve no useful purpose once binman has
finished. They clutter up the source directory and affect future runs,
since the files in the current directory are often used in preference to
those in the board directory.
To avoid these problems, write them to the output directory instead.
Move the file-creation code to the Entry base class, so it can be used by
any entry type that needs it. This is required since some entry types,
such as Entry_blob_ext_list, are not subclasses of Entry_blob.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use a unique number instead of the current 203, which is used by 203_fip
as well. Reformat the code to avoid a long line.
Signed-off-by: Simon Glass <sjg@chromium.org>
While converting to binman for an imx8mq board, it has been found that
building in the u-boot CI fails. This is because an imx8mq requires an
external binary (signed_hdmi_imx8m.bin). If this file cannot be found
mkimage fails.
To be able to build this board in the u-boot CI a binman option
(--fake-ext-blobs) is introduced that can be switched on via the u-boot
makefile option BINMAN_FAKE_EXT_BLOBS. With that the needed dummy files are
created.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
This format is used in firmware binaries so we may as well supported it.
With this patch binman supports creating, listing and updating FIPs, as
well as extracting files from one, provided that an FDTMAP is also present
somewhere in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful to have a list of related external blobs in a
single entry. An example is the DDR binaries used by meson. There are
9 files in total. Add support for this, so we don't have to have a
separate entry for each.
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases entries encapsulate other data and it is useful to access
the data within. An example is the fdtmap which consists of a 16-byte
header, followed by a devicetree.
Provide an option to specify an alternative format when extracting files.
In the case of fdtmap, this is 'fdt', which produces an FDT file which can
be viewed with fdtdump.
Signed-off-by: Simon Glass <sjg@chromium.org>
This method is currently marked private. However it is useful to be able
to subclass it, since much of the entry_Section code can be reused. Rename
it.
Also document one confusing part of this code, so people can understand
how to add a test for this case.
Fix up a few pylint warnings to avoid regressing the score.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a -V option which shows the version number of binman. For now this
just uses a local 'version' file. Once the tool is packaged in some way
we can figure out an approach that suits.
Signed-off-by: Simon Glass <sjg@chromium.org>
WIth EFI we must embed the devicetree in an ELF image so that it is loaded
as part of the executable file. We want it to include the binman
definition in there also, which in some cases cannot be created until the
ELF (u-boot) is built. Add an option to binman to support writing the
updated dtb to the ELF file u-boot.out
This is useful with the EFI app, which is always packaged as an ELF file.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present testThreadTimeout() assumes that the expected timeout happens
first when building the section, but it can just as easily happen at the
top-level image. Update the test to cope with both.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Marek Behún <marek.behun@nic.cz>
One of binman's attributes is that it is extremely fast, at least for a
Python program. Add some simple timing around operations that might take
a while, such as reading an image and compressing it. This should help
to maintain the performance as new features are added.
This is for debugging purposes only.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some images may take a while to build, e.g. if they are large and use slow
compression. Support compiling sections in parallel to speed things up.
Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed to use a separate test file to fix flakiness)
Add an entry for RISC-V OpenSBI's 'fw_dynamic' firmware payload.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Rick Chen <rick@andestech.com>
Reviewed-by: Rick Chen <rick@andestech.com>
Currently there are 2 binman test cases using the same 172 number.
It seems that 172_fit_fdt.dts was originally named as 170_, but
commit c0f1ebe9c1 ("binman: Allow selecting default FIT configuration")
changed its name to 172_ for no reason. Let's change it back.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
When used with hierarchical images, use the Chromium OS convention of
adding a section before all the subentries it contains.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use an interator in two of the fmap tests so it is easier to add new
items. Also check the name first since that is the first indication
that something is wrong. Use a variable for the expected size of the
fmap to avoid repeating the code.
Signed-off-by: Simon Glass <sjg@chromium.org>
These two tests require an ELF image so that symbol information can be
written into the SPL/TPL binary. At present they rely on other tests
having set it up first, but every test must run independently. This can
cause occasional errors in CI.
Fix this by setting up the required files, as other tests do.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Sometimes it is useful to specify the default alignment for all entries
in a section, such as when word-alignment is necessary, for example. It
is tedious and error-prone to specify this individually for each section.
Add a property to control this for a section.
Signed-off-by: Simon Glass <sjg@chromium.org>
Generally the content of sections is not built until the final assembly
of the image. This is partly to avoid wasting time, since the entries
within sections may change multiple times as binman works through its
various stages. This works quite well since sections exist in a strict
hierarchy, so they can be processed in a depth-first manner.
However the 'collection' entry type does not have this luxury. If it
contains a section within its 'content' list, then it must produce the
section contents, if available. That section is typically a sibling
node, i.e. not part oc the collection's hierarchy.
Add a new 'required' argument to section.GetData() to support this. When
required is True, any referenced sections are immediately built. If this
is not possible (because one of the subentries does not have its data yet)
then an error is produced.
The test for this uses a 'collection' entry type, referencing a section as
its first member. This forces a call to _BuildSectionData() with required
set to False, at first, then True later, when the image is assembled.
Signed-off-by: Simon Glass <sjg@chromium.org>
The vblock entry type includes code to collect the data from a number of
other entries (not necessarily subentries) and concatenating it. This is
a useful feature for other entry types.
Make it a base class, so that vblock can use it, along with other entry
types.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present there is a command-line flag to disable substitution of expanded
entries. Add an option to the entry node as well, so it can be controlled
at the node level.
Add a test to cover this. Fix up the comment to the checkSymbols() function
it uses, while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a link to binman's documentation and adjust the files so that it is
accessible. Use the name README.rst so it is easy to discover when binman
is installed without U-Boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
When creating an entry, check for an expanded version of that entry, then
use it instead. This allows, for example use of:
u-boot {
};
instead of having to write out in full:
u-boot {
type = "section";
u-boot-nodtb {
};
u-boot-dtb {
};
};
Add an implementaion of this and associated documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a new command-line option to disable expanded entries. This is needed
for most tests, since it is much easier to 'factor out' this function into
a separate test and keep the existing packing tests simple.
Add the option and select it by default from tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
This entry holds the padding between the end of of TPL binary and the
end of BSS. This region must be left empty so that the devicetree can be
appended correctly and remain accessible without interfering with BSS.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since this is an execuable we should be able insert symbol values into it.
Add support for this.
Use common code for this test and the original testSymbols. Use hex
consistently for the values and add some more comments.
Signed-off-by: Simon Glass <sjg@chromium.org>
The offset of an entry needs to be adjusted by its skip-at-start value.
This is currently missing when reading entry data. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
When packing files it is sometimes useful to align the start of each file,
e.g. if the flash driver can only access 32-bit-aligned data. Provides a
new property to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.
Fix this by adjusting how the vblock is created.
Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally when an entry is created, any entry arguments it has are required
to be provided, so it can actually generate its contents correctly.
However when an existing image is read, Entry objects are created for each
of the entries in the image. This happens as part of the process of
reading the image into binman.
In this case we don't need the entry arguments, since we do not intend to
regenerate the entries, or at least not unless requested. So there is no
sense in reporting an error for missing entry arguments.
Add a new property for the Image to handle this case. Update the error
reporting to be conditional on this property.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman only supports resolving symbols in the same section as
the binary that uses it. This is quite limited because we often need to
group entries into different sections.
Enhance the algorithm to search the entire image for symbols.
Signed-off-by: Simon Glass <sjg@chromium.org>
With the previous changes, it is now possible to compress entire
sections. Add some tests to check that compression works correctly,
including updating the metadata.
Also update the documentation.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this function adds up the total size of entries to work out the
size of a section's contents. With compression this is no-longer enough.
We may as well bite the bullet and build the section contents instead.
Call _BuildSectionData() to get the (possibly compressed) contents and
GetPaddedData() to get the same but with padding added.
Note that this is inefficient since the section contents is calculated
twice. Future work will improve this.
This affects testPackOverlapMap() since the error is reported with a
different section size now (enough to hold the contents). Update that at
the same time.
Signed-off-by: Simon Glass <sjg@chromium.org>
Section contents is not set up when ObtainContents() is called, since
packing often changes the layout of the contents. Ensure that the contents
are correctly recorded by making this function regenerate the section. It
is normally only called by the parent section (when packing) or by the
top-level image code, when writing out the image. So the performance
impact is fairly small.
Now that sections have their contents in their 'data' property, update
testSkipAtStartSectionPad() to check it.
Signed-off-by: Simon Glass <sjg@chromium.org>
When compressing an entry, the original uncompressed data is overwritten.
Store it so it is available if needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present padding of sections is inconsistent with other entry types, in
that different pad bytes are used.
When a normal entry is padded by its parent, the parent's pad byte is
used. But for sections, the section's pad byte is used.
Adjust logic to always do this the same way.
Note there is still a special case in entry_Section.GetPaddedData() where
an image is padded with the pad byte of the top-level section. This is
necessary since otherwise there would be no way to set the pad byte of
the image, without adding a top-level section to every image.
Signed-off-by: Simon Glass <sjg@chromium.org>
Each section is padded up to its size, if the contents are not large
enough. Move this logic from _BuildSectionData() to
GetPaddedDataForEntry() so that all the padding is in one place.
With this, the testDual test is working again, so enable it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Create a new _BuildSectionData() to hold the code that is now in
GetData(), so that it is clearly separated from entry.GetData() base
function.
Separate out the 'pad-before' processing to make this easier to
understand.
Unfortunately this breaks the testDual test. Rather than squash several
patches into an un-reviewable glob, disable the test for now.
This also affects testSkipAtStartSectionPad(), although it still not
quite what it should be. Update that temporarily for now.
Signed-off-by: Simon Glass <sjg@chromium.org>
Alignment does form part of the entry once the image is written out, but
within binman the entry contents does not include the padding. Add
documentation to make this clear, as well as a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Padding becomes part of the entry once the image is written out, but
within binman the entry contents does not include the padding. Add
documentation to make this clear, as well as a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Check the contents of each section to make sure it is actually in the
right place.
Also fix a whitespace error in the .dts file.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this feature is tested view the end-at-4gb feature. Add some
tests of its own, including the operation of padding.
The third test here shows binman's current, inconsistent approach to
padding in the top-level section. Future patches in this series will
address this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add an entry type for a firmware blob for a System Control Processor,
given by an entry arg. This firmware is a raw binary blob.
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Jagan Teki <jagan@amarulasolutions.com>
In some cases it is useful to include a U-Boot environment region in an
image. This allows the board to start up with an environment ready to go.
Add a new entry type for this. The input is a text file containing the
environment entries, one per line, in the format:
var=value
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
The recent support for missing external binaries does not show an error
message when a file is genuinely missing (i.e. it is missing but not
marked as 'external'). This means that when -m is passed to binman, it
will never report a missing file.
Fix this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
When an external blob is missing it can be quite confusing for the user.
Add a way to provide a help message that is shown.
Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Add a new entry argument to the fit entry which allows selection of the
default configuration to use. This is the 'default' property in the
'configurations' node.
Update the Makefile to pass in the value of DEVICE_TREE or
CONFIG_DEFAULT_DEVICE_TREE to provide this information.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Michal Simek <michal.simek@xilinx.com>
In some cases it is useful to generate a FIT which has a number of DTB
images, selectable by configuration. Add support for this in binman,
using a simple iterator and string substitution.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add an entry for ARM Trusted Firmware's 'BL31' payload, which is the
device's main firmware. Typically this is U-Boot.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tidy up a few test functions which lack argument comments. Rename one that
has the same name as a different test.
Also fix up the comment for PrepareImagesAndDtbs().
Signed-off-by: Simon Glass <sjg@chromium.org>
If an entry argument is needed by an entry but the entry argument is not
present, then a strange error can occur when trying to read the file.
Fix this by allowing arguments to be required. Select this option for the
cros-ec-rw entry. If a filename is provided in the node, allow that to be
used.
Also tidy up a few related tests to make the error string easier to find,
and fully ignore unused return values.
Signed-off-by: Simon Glass <sjg@chromium.org>
When reading subentries of each image, the FIT entry type directly
concatenates their contents without padding them according to their
offset, size, align, align-size, align-end, pad-before, pad-after
properties.
This patch makes sure these properties are respected by offloading this
image-data building to the section etype, where each subnode of the
"images" node is processed as a section. Alignments and offsets are
respective to the beginning of each image. For example, the following
fragment can end up having "u-boot-spl" start at 0x88 within the final
FIT binary, while "u-boot" would then end up starting at e.g. 0x20088.
fit {
description = "example";
images {
kernel-1 {
description = "U-Boot with SPL";
type = "kernel";
arch = "arm64";
os = "linux";
compression = "none";
u-boot-spl {
};
u-boot {
align = <0x10000>;
};
};
};
}
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reinstate check in testPadInSections(), squash in
"binman: Allow FIT binaries to have missing external blobs"
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Other relevant properties (pad-after, offset, size, align, align-size,
align-end) already work since Pack() sets correct ranges for subentries'
data (.offset, .size variables), but some padding here is necessary to
align the data within this range to match the pad-before property.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Switch to str.startswith for matching like the FIT etype does since the
current version doesn't ignore 'hash-1', 'hash-2', etc.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Some of these were not converted when binman moved to use absolute paths.
Fix them.
Also drop the import of 'test' which is a directory, not a module.
Signed-off-by: Simon Glass <sjg@chromium.org>
When binman is installed its main program is in a different directory
to its modules. This means that __file__ is different and we cannot use
it to obtain the path to etype/ from main.py
To fix this, move the function to the 'control' module, since it is
installed with all the other modules, including the etype/ directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present testPackX86RomMeNoDesc removes the contents of the
descriptor.bin file and testPackX86RomMeMissingDesc removes the file
completely.
If a test that relies on this file happens to run after it is removed, it
will not work. Since we have no control over the selecting of tests that
run in parallel and series, we must avoid changing the files.
Update this tests to use separate files instead.
Signed-off-by: Simon Glass <sjg@chromium.org>
FIT (Flat Image Tree) is the main image format used by U-Boot. In some
cases scripts are used to create FITs within the U-Boot build system. This
is not ideal for various reasons:
- Each architecture has its own slightly different script
- There are no tests
- Some are written in shell, some in Python
To help address this, add support for FIT generation to binman. This works
by putting the FIT source directly in the binman definition, with the
ability to adjust parameters, etc. The contents of each FIT image come
from sub-entries of the image, as is normal with binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some binary blobs unfortunately obtain their position in the image from
other binary blobs, such as Intel's 'descriptor'. In this case we cannot
rely on packing to work. It is not possible to produce a valid image in
any case, due to the missing blobs.
Allow zero-length overlaps so that this does not cause any problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
When warnings and errors are produced by tools they should be written to
stderr. Update the tout implementation to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.
U-Boot does with with ATF (ARM Trusted Firmware) today.
Add a new flag to binman to request this behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
It is useful to be able to distinguish between ordinary blobs such as
u-boot.bin and external blobs that cannot be build by the U-Boot build
system. If the external blobs are not available for some reason, then we
know that a value image cannot be built.
Introduce a new 'blob-ext' entry type for that.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
As a first step to integrating mkimage into binman, add a new entry type
that feeds data into mkimage for processing and incorporates that output
into the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
FIT (Flat Image Tree) is the main image format used by U-Boot. In some
cases scripts are used to create FITs within the U-Boot build system. This
is not ideal for various reasons:
- Each architecture has its own slightly different script
- There are no tests
- Some are written in shell, some in Python
To help address this, add support for FIT generation to binman. This works
by putting the FIT source directly in the binman definition, with the
ability to adjust parameters, etc. The contents of each FIT image come
from sub-entries of the image, as is normal with binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some binary blobs unfortunately obtain their position in the image from
other binary blobs, such as Intel's 'descriptor'. In this case we cannot
rely on packing to work. It is not possible to produce a valid image in
any case, due to the missing blobs.
Allow zero-length overlaps so that this does not cause any problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
When warnings and errors are produced by tools they should be written to
stderr. Update the tout implementation to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.
U-Boot does with with ATF (ARM Trusted Firmware) today.
Add a new flag to binman to request this behaviour.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
It is useful to be able to distinguish between ordinary blobs such as
u-boot.bin and external blobs that cannot be build by the U-Boot build
system. If the external blobs are not available for some reason, then we
know that a value image cannot be built.
Introduce a new 'blob-ext' entry type for that.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
As a first step to integrating mkimage into binman, add a new entry type
that feeds data into mkimage for processing and incorporates that output
into the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Python does not like the module name being the same as the module
directory. To allow buildman modules to be used from other tools, rename
it.
Signed-off-by: Simon Glass <sjg@chromium.org>
A recent change adjusted the symbol calculation to work on x86 but broke
it for Tegra. In fact this is because they have different needs.
On x86 devices the code is linked to a ROM address and the end-at-4gb
property is used for the image. In this case there is no need to add the
base address of the image, since the base address is already built into
the offset and image-pos properties.
On other devices we must add the base address since the offsets start at
zero.
In addition the base address is currently added to the 'offset' and 'size'
values. It should in fact only be added to 'image-pos', since 'offset' is
relative to its parent and 'size' is not actually an address. This code
should have been adjusted when support for 'image-pos' and 'size' was
added, but it was not.
To correct these problems:
- move the code that handles adding the base address to section.py, which
can check the end-at-4gb property and which property
(offset/size/image-pos) is being read
- add the base address only when needed (only for image-pos and not if the
image uses end-at-4gb)
- add a note to the documentation
- add a separate test to cover x86 behaviour
Fixes: 15c981cc (binman: Correct symbol calculation with non-zero image base)
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
At present all the 'command' methods return bytes. Most of the time we
actually want strings, so change this. We still need to keep the internal
representation as bytes since otherwise unicode strings might break over
a read() boundary (e.g. 4KB), causing errors. But we can convert the end
result to strings.
Add a 'binary' parameter to cover the few cases where bytes are needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
This entry is used to hold an Intel FSP-T (Firmware Support Package
Temp-RAM init) binary. Add support for this in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
This entry is used to hold an Intel FSP-S (Firmware Support Package
Silicon init) binary. Add support for this in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
The Intel FSP supports initialising memory early during boot using a binary
blob called 'fspm'. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to access the size of an image in SPL, with
something like:
binman_sym_declare(unsigned long, u_boot_any, size);
...
ulong u_boot_size = binman_sym(ulong, u_boot_any, size);
Add support for this and update the tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present these are large enough to hold 20 bytes of symbol data. Add
four more bytes so we can add another test.
Unfortunately at present this involves changing a few test files to make
room. We could adjust the test files to not specify sizes for entries.
Then we could make the tests check the actual sizes. But for now, leave it
as it is, since the effort is minor.
Signed-off-by: Simon Glass <sjg@chromium.org>
Entries which include a section and need to obtain its contents call
GetData(), as with any other entry. But the current implementation of this
method in entry_Section requires the size of the section to be known. If
it is unknown, an error is produced, since size is None:
TypeError: can't multiply sequence by non-int of type 'NoneType'
There is no need to know the size in advance since the code can be
adjusted to build up the section piece by piece, instead of patching each
entry into an existing bytearray.
Update the code to handle this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
Two of the test files somehow were not converted to three digits. Fix
them, using the next available numbers.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we only support symbols inside binaries which are at the top
level of an image. This restrictions seems unreasonable since more complex
images may want to group binaries within different sections.
Relax the restriction, adding a new _SetupTplElf() helper function.
Also fix a typo in the comment for testTpl().
Signed-off-by: Simon Glass <sjg@chromium.org>
We use the Makefile for all ELF test files now, so drop all the code that
checks whether to get the test file from the Makefile or from the git
repo.
Also add a comment to the Makefile indicating that it is run from binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Remove this file from git and instead build it using the Makefile.
With this change a few things need to be adjusted:
1. The 'notes' section no-longer appears at the start of the ELF file
(before the code), so update testSymbols to adjust the offsets.
2. The dynamic linker is disabled to avoid errors like:
"Not enough room for program headers, try linking with -N"
3. The interpreter note is moved to the end of the image, so that the
binman symbols appear first.
Signed-off-by: Simon Glass <sjg@chromium.org>
Remove this file from git and instead build it using the Makefile.
Update tools.GetInputFilename() to support reading files from an absolute
path, so that we can read the Elf test files easily. Also make sure that
the temp directory is report in ELF tests as this was commented out.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the ELF test files are checked into the U-Boot tree. This is
covenient since the files never change and can be used on non-x86
platforms. However it is not good practice to check in binaries and in
this case it does not seem essential.
Update the binman test-file Makefile to support having source in a
different directory. Adjust binman to run it to build bss_data, as a
start. We can add other files as needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this entry does not work correctly when a FIT image is used as
the input. It updates the FIT instead of the output image. The test passed
because the FIT image happened to have the right data already.
Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
A Firmware Image Table (FIT) is a data structure defined by Intel which
contains information about various things needed by the SoC, such as
microcode.
Add support for this entry as well as the pointer to it. The contents of
FIT are fixed at present. Future work is needed to support adding
microcode, etc.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present these two sections of code are linked together into a single
2KB chunk in a single file. Some Intel SoCs like to have a FIT (Firmware
Interface Table) in the ROM and the pointer for this needs to go at
0xffffffc0 which is in the middle of these two sections.
Make use of the new 'reset' entry and change the existing 16-bit entry to
include just the 16-bit data.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman has a single entry type for the 16-bit code code needed
to start up an x86 processor. This entry is intended to include both the
reset vector itself as well as the code to move to 32-bit mode.
However this is not very flexible since in some cases other data needs to
be included at the top of the SPI flash, in between these two pieces. For
example Intel requires that a FIT (Firmware Image Table) pointer be placed
0x40 bytes before the end of the ROM.
To deal with this, add a new reset entry for just the reset vector. A
subsequent change will adjust the existing 'start16' entry.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is more common to use the name 'cls' for the class object of a class
method, to distinguish it from normal methods, which use 'self' Update the
binman tests accordingly.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a 'replace' command to binman to permit entries to be replaced, either
individually or all at once (using a filter).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present some tests leave behind output directories. This happens
because some tests call binman, which sets up an output directory, then
call it again, which sets up another output directory and leaves the
original one behind.
Fix this by using a separate temporary directory when binman is called
twice, or by manually removing the output directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman cannot replace data within a CBFS since it does not
allow rewriting of the files in that CBFS. Implement this by using the
new WriteData() method to handle the case.
Add a header to compressed data so that the amount of compressed data can
be determined without reference to the size of the containing entry. This
allows the entry to be larger that the contents, without causing errors in
decompression. This is necessary to cope with a compressed device tree
being updated in such a way that it shrinks after the entry size is
already set (an obscure case). It is not used with CBFS since it has its
own metadata for this. Increase the number of passes allowed to resolve
the position of entries, to handle this case.
Add a test for this new logic.
Signed-off-by: Simon Glass <sjg@chromium.org>
The Intel descriptor must always appear at the start of an (x86) image,
so it is supposed to position itself there always. However there is no
explicit test for this. Add one and fix a bug introduced by the recent
change to adjust Entry to read the node in a separate call.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes an entry may shrink after it has already been packed. In that
case we must repack the items. Of course it is always possible to just
leave the entry at its original size and waste space at the end. This is
what binman does by default, since there is the possibility of the entry
changing size every time binman calculates its contents, thus causing a
loop.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes entries shrink after packing. As a start towards supporting
this, update the _testing entry to handle the test case.
Signed-off-by: Simon Glass <sjg@chromium.org>
So far we don't allow entries to change size when repacking. But this is
not very useful since it is common for entries to change size after an
updated binary is built, etc.
Add support for this, respecting the original offset/size/alignment
constraints of the image layout. For this to work the original image
must have been created with the 'allow-repack' property.
This does not support entry types with sub-entries such as files and
CBFS, but it does support sections.
Signed-off-by: Simon Glass <sjg@chromium.org>
The positioning does not currently work correctly if at the end of an
image with no fixed size. Also if the header is in the middle of an image
it can cause a gap in the image since the header position is normally at
the image end, so entries after it are placed after the end of the image.
Fix these problems and add more tests to cover these cases.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present it is not possible to discover the contraints to repacking an
image (e.g. maximum section size) since this information is not preserved
from the original image description.
Add new 'orig-offset' and 'orig-size' properties to hold this. Add them to
the main device tree in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
While it is useful and efficient to build images in a single pass from a
unified description, it is sometimes desirable to update the image later.
Add support for replace an existing file with one of the same size. This
avoids needing to repack the file. Support for more advanced updates will
come in future patches.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present EnsureCompiled() uses an file from the 'output' directory (in
the tools module) when compiling the device tree. This is fine in most
cases, allowing useful inspection of the output files from binman.
However in functional tests, _SetupDtb() creates an output directory and
immediately removes it afterwards. This serves no benefit and just
confuses things, since the 'official' output directory is supposed to be
created and destroyed in control.Binman().
Add a new parameter for the optional temporary directory to use, and use a
separate temporary directory in _SetupDtb().
Signed-off-by: Simon Glass <sjg@chromium.org>
We use this same combination of properties several times in tests. Add a
constant for it to avoid typos, etc.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since binman supports multiple images it is useful to know which one
created the image that has been read. Then it is possible to look up that
name in the 'master' device tree (containing the description of all
images).
Add a property for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Current test coverage is likely sufficient for the logic used to place
sections in the image. However it seems useful to add a test specifically
for nested sections, since these could have some unusual interactions.
Add a new test for this and aligned sections. This test failed before the
refactor to drop the bsection.py file (Section class), but passes now.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to extract all binaries from the image, or a
subset of them. Add a new 'extract' command to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.
Signed-off-by: Simon Glass <sjg@chromium.org>
When support for sections (and thus hierarchical images) was added to
binman, the decision was made to create a new Section class which could
be used by both Image and an Entry_section class. The decision between
using inheritance and composition was tricky to make, but in the end it
was decided that Image was different enough from Entry that it made sense
to put the implementation of sections in an entirely separate class. It
also has the advantage that core Image code does have to rely on an entry
class in the etype directory.
This work was mostly completed in commit:
8f1da50ccc "binman: Refactor much of the image code into 'section'
As a result of this, the Section class has its own version of things like
offset and size and these must be kept in sync with the parent
Entry_section class in some cases.
In the last year it has become apparent that the cost of keeping things in
sync is larger than expected, since more and more code wants to access
these properties.
An alternative approach, previously considered and rejected, now seems
better.
Adjust Image to be a subclass of Entry_section. Move the code from Section
(in bsection.py) to Entry_section and delete Section. Update all tests
accordingly.
This requires substantial changes to Image. Overall the changes reduce
code size by about 240 lines. While much of that is just boilerplate from
Section, there are quite a few functions in Entry_section which now do not
need to be overiden from Entry. This suggests the change is beneficial
even without further functionality being added.
A side benefit is that the properties of sections are now consistent with
other entries. This fixes a problem in testListCmd() where some properties
are missing for sections.
Unfortunately this is a very large commit since it is not feasible to do
the migration piecemeal. Given the substantial tests available and the
100% code coverage of binman, we should be able to do this safely.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is possible to read an Image, locate its FDT map and then read it into
the binman data structures. This allows full access to the entries that
were written to the image. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
