We can and should run the node generator only when creating a new image.
When we read it back, there is no need to generate nodes - they already
exits, and binman does not dive that deep into the image - and there is
no way to provide the required fdt-list. So store the mode in the image
object so that Entry_fit can simply skip generator nodes when reading
them from an fdtmap.
This unbreaks all read-backs of images that contain generator nodes in
their fdtmap. To confirm this, add a corresponding test case.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Add SPDX to dts file:
Signed-off-by: Simon Glass <sjg@chromium.org>
Support collecting the available bintools needed by an image, by
scanning the entries in the image.
Also add a command-line interface to access the basic bintool features,
such as listing the bintools and fetching them if needed.
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases entries encapsulate other data and it is useful to access
the data within. An example is the fdtmap which consists of a 16-byte
header, followed by a devicetree.
Provide an option to specify an alternative format when extracting files.
In the case of fdtmap, this is 'fdt', which produces an FDT file which can
be viewed with fdtdump.
Signed-off-by: Simon Glass <sjg@chromium.org>
If an older version of binman is used to list images created by a newer
one, it is possible that it will contain entry types that are not
supported. At present this produces an error.
Adjust binman to use a plain 'blob' entry type to cope with this, so the
image can at least be listed.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some images may take a while to build, e.g. if they are large and use slow
compression. Support compiling sections in parallel to speed things up.
Signed-off-by: Simon Glass <sjg@chromium.org>
(fixed to use a separate test file to fix flakiness)
Add support for this feature in the control, image and section modules, so
that expanded entries will be selected by default. So far there are no
expanded entry types, so this is a nop.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present if a devicetree blob is included in a vblock it does not deal
with updates. This is because the vblock is created once at the start and
does not have a method to update itself later, after all the entry
contents are finalised.
Fix this by adjusting how the vblock is created.
Also simplify Image.ProcessEntryContents() since it effectively duplicates
the code in Section.ProcessContents().
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally when an entry is created, any entry arguments it has are required
to be provided, so it can actually generate its contents correctly.
However when an existing image is read, Entry objects are created for each
of the entries in the image. This happens as part of the process of
reading the image into binman.
In this case we don't need the entry arguments, since we do not intend to
regenerate the entries, or at least not unless requested. So there is no
sense in reporting an error for missing entry arguments.
Add a new property for the Image to handle this case. Update the error
reporting to be conditional on this property.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman only supports resolving symbols in the same section as
the binary that uses it. This is quite limited because we often need to
group entries into different sections.
Enhance the algorithm to search the entire image for symbols.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this function does the padding needed around an entry. It is
easier to understand what is going on if we have a function that returns
the contents of an entry, with padding included.
Refactor the code accordingly, adding a new GetPaddedData() method.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
It is easier and less error-prone to use super() when the parent type is
needed. Update binman to remove the type names.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move binman to use absolute imports. This enables removable of the path
adjusting in Entry also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some x86 sections have special offsets which currently result in empty
data being returned from the 'extract' command. Fix this by taking account
of the skip-at-start property.
Add a little more debugging while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Bin Meng <bmeng.cn@gmail.com>
So far we don't allow entries to change size when repacking. But this is
not very useful since it is common for entries to change size after an
updated binary is built, etc.
Add support for this, respecting the original offset/size/alignment
constraints of the image layout. For this to work the original image
must have been created with the 'allow-repack' property.
This does not support entry types with sub-entries such as files and
CBFS, but it does support sections.
Signed-off-by: Simon Glass <sjg@chromium.org>
Now that an Image is an Entry_section, there is no need for the separate
BuildSection() function. Drop it and add a bit of logging.
Signed-off-by: Simon Glass <sjg@chromium.org>
When reading an image in, write its fdtmap to a file in the output
directory. This is useful for debugging. Update the 'ls' command to set up
the output directory; otherwise it will fail.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present it is not possible to discover the contraints to repacking an
image (e.g. maximum section size) since this information is not preserved
from the original image description.
Add new 'orig-offset' and 'orig-size' properties to hold this. Add them to
the main device tree in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
While it is useful and efficient to build images in a single pass from a
unified description, it is sometimes desirable to update the image later.
Add support for replace an existing file with one of the same size. This
avoids needing to repack the file. Support for more advanced updates will
come in future patches.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we have an 'image' property in the entry for this purpose, but
this is not necessary and seems error-prone in the presence of
inheritance. Add a function instead. The Entry_section class overrides
this with a special version, since top-level sections are in fact images,
since Image inherits Entry_section.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the Entry constructor sets up the object and then immediately
reads its device-tree node to obtain its properties.
This breaks a convention that constructors should not do any processing.
A consequence is that we must pass all arguments to the constructor and
cannot have the node-reading proceed in a different way unless we pass
flags to that constructor. We already have a 'test' flag in a few cases,
and now need to control whether the 'orig_offset' and 'orig_size'
properties are set or not.
Adjust the code to require a separate call to ReadNode() after
construction. The Image class remains as it was.
Signed-off-by: Simon Glass <sjg@chromium.org>
This data provides all the information about the position and size of each
entry. Store it for later use when loading an image. It can be reused as
is if the image is modified without changing offsets/sizes.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to extract entry contents from an image to see
what is inside. Add a simple function to read the contents of an entry,
decompressing it by default.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman generally operates silently but in some cases it is useful to see
what Binman is actually doing at each step. Enable some logging output
with different logging levels selectable via the -v flag.
Signed-off-by: Simon Glass <sjg@chromium.org>
When support for sections (and thus hierarchical images) was added to
binman, the decision was made to create a new Section class which could
be used by both Image and an Entry_section class. The decision between
using inheritance and composition was tricky to make, but in the end it
was decided that Image was different enough from Entry that it made sense
to put the implementation of sections in an entirely separate class. It
also has the advantage that core Image code does have to rely on an entry
class in the etype directory.
This work was mostly completed in commit:
8f1da50ccc "binman: Refactor much of the image code into 'section'
As a result of this, the Section class has its own version of things like
offset and size and these must be kept in sync with the parent
Entry_section class in some cases.
In the last year it has become apparent that the cost of keeping things in
sync is larger than expected, since more and more code wants to access
these properties.
An alternative approach, previously considered and rejected, now seems
better.
Adjust Image to be a subclass of Entry_section. Move the code from Section
(in bsection.py) to Entry_section and delete Section. Update all tests
accordingly.
This requires substantial changes to Image. Overall the changes reduce
code size by about 240 lines. While much of that is just boilerplate from
Section, there are quite a few functions in Entry_section which now do not
need to be overiden from Entry. This suggests the change is beneficial
even without further functionality being added.
A side benefit is that the properties of sections are now consistent with
other entries. This fixes a problem in testListCmd() where some properties
are missing for sections.
Unfortunately this is a very large commit since it is not feasible to do
the migration piecemeal. Given the substantial tests available and the
100% code coverage of binman, we should be able to do this safely.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is possible to read an Image, locate its FDT map and then read it into
the binman data structures. This allows full access to the entries that
were written to the image. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to summarise all the entries in an image, e.g. to
display this to this user. Add a new ListEntries() method to Entry, and
set up a way to call it through the Image class.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add support for detecting entries that change size after they have already
been packed, and re-running packing when it happens.
This removes the limitation that entry size cannot change after
PackEntries() is called.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present if this function tries to update the contents such that the
size changes, it raises an error. We plan to add the ability to change
the size of entries after packing is completed, since in some cases it is
not possible to determine the size in advance.
An example of this is with a compressed device tree, where the values
of the device tree change in SetCalculatedProperties() or
ProcessEntryContents(). While the device tree itself does not change size,
since placeholders for any new properties have already bee added by
AddMissingProperties(), we cannot predict the size of the device tree
after compression. If a value changes from 0 to 0x1234 (say), then the
compressed device tree may expand.
As a first step towards supporting this, make ProcessContentsUpdate()
return a value indicating whether the content size is OK. For now this is
always True (since otherwise binman raises an error), but later patches
will adjust this.
Signed-off-by: Simon Glass <sjg@chromium.org>
When we get a problem like overlapping regions it is sometimes hard to
figure what what is going on. At present we don't write the map file in
this case. However the file does provide useful information.
Catch any packing errors and write a map file (if enabled with -m) to aid
debugging.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present sections have no record of their parent so it is not possible
to traverse up the tree to the root and figure out the position of a
section within the image.
Change the constructor to record this information.
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases it is useful to add a group of files to the image and be
able to access them at run-time. Of course it is possible to generate
the binman config file with a set of blobs each with a filename. But for
convenience, add an entry type which can do this.
Add required support (for adding nodes and string properties) into the
state module.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman currently supports updating the main device tree with things like
the position of each entry. Extend this support to SPL and TPL as well,
since they may need (a subset of) this information.
Also adjust DTB output files to have a .out extension since this seems
clearer than having a .dtb extension with 'out' in the name somwhere.
Also add a few missing comments and update the DT setup code to use
ReadFile and WriteFile().
Signed-off-by: Simon Glass <sjg@chromium.org>
We always have a device tree for U-Boot proper. But we may also have one
for SPL and TPL. Add a new Entry method to find out what DTs an entry
has, and use that list when updating DTs.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the map only shows the offset and size for each region. The
image position provides the actual position of each entry in the image,
regardless of the section hierarchy.
Add the image position to the map.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present each entry has an offset within its parent section. This is
useful for figuring out how entries relate to one another. However it
is sometimes necessary to locate an entry within an image, regardless
of which sections it is nested inside.
Add a new 'image-pos' property to provide this information. Also add
some documentation for the -u option binman provides, which updates the
device tree with final entry information.
Since the image position is a better symbol to use for the position of
U-Boot as obtained by SPL, update the SPL symbols to use this instead of
offset, which might be incorrect if hierarchical sections are used.
Signed-off-by: Simon Glass <sjg@chromium.org>
After some thought, I believe there is an unfortunate naming flaw in
binman. Entries have a position and size, but now that we support
hierarchical sections it is unclear whether a position should be an
absolute position within the image, or a relative position within its
parent section.
At present 'position' actually means the relative position. This indicates
a need for an 'image position' for code that wants to find the location of
an entry without having to do calculations back through parents to
discover this image position.
A better name for the current 'position' or 'pos' is 'offset'. It is not
always an absolute position, but it is always an offset from its parent
offset.
It is unfortunate to rename this concept now, 18 months after binman was
introduced. However I believe it is the right thing to do. The impact is
mostly limited to binman itself and a few changes to in-tree users to
binman:
tegra
sunxi
x86
The change makes old binman definitions (e.g. downstream or out-of-tree)
incompatible if they use the 'pos = <...>' property. Later work will
adjust binman to generate an error when it is used.
Signed-off-by: Simon Glass <sjg@chromium.org>
Once binman has packed the image, the position and size of each entry is
known. It is then possible for binman to update the device tree with these
positions. Since placeholder values have been added, this does not affect
the size of the device tree and therefore the packing does not need to be
performed again.
Add a new SetCalculatedProperties method to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some entry types modify the device tree, e.g. to remove microcode or add a
property. So far this just modifies their local copy and does not affect
a 'shared' device tree.
Rather than doing this modification in the ObtainContents() method, and a
new ProcessFdt() method which is specifically designed to modify this
shared device tree.
Move the existing device-tree code over to use this method, reducing
ObtainContents() to the goal of just obtaining the contents without any
processing, even for device tree.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to be able to see a list of regions in each image produced by
binman. Add a -m option to output this information in a '.map' file
alongside the image file.
Signed-off-by: Simon Glass <sjg@chromium.org>
We want to support multiple sections within a single image. To do this,
move most of the Image class implementation into a new Section class. An
Image contains only a single Section, but at some point we will support
a new 'section' entry, thus allowing Sections within Sections.
Use the name 'bsection' for the module so we can use 'section' for the
etype module.
Signed-off-by: Simon Glass <sjg@chromium.org>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
Binman construct images consisting of multiple binary files. These files
sometimes need to know (at run timme) where their peers are located. For
example, SPL may want to know where U-Boot is located in the image, so
that it can jump to U-Boot correctly on boot.
In general the positions where the binaries end up after binman has
finished packing them cannot be known at compile time. One reason for
this is that binman does not know the size of the binaries until
everything is compiled, linked and converted to binaries with objcopy.
To make this work, we add a feature to binman which checks each binary
for symbol names starting with '_binman'. These are then decoded to figure
out which entry and property they refer to. Then binman writes the value
of this symbol into the appropriate binary. With this, the symbol will
have the correct value at run time.
Macros are used to make this easier to use. As an example, this declares
a symbol that will access the 'u-boot-spl' entry to find the 'pos' value
(i.e. the position of SPL in the image):
binman_sym_declare(unsigned long, u_boot_spl, pos);
This converts to a symbol called '_binman_u_boot_spl_prop_pos' in any
binary that includes it. Binman then updates the value in that binary,
ensuring that it can be accessed at runtime with:
ulong u_boot_pos = binman_sym(ulong, u_boot_spl, pos);
This assigns the variable u_boot_pos to the position of SPL in the image.
Signed-off-by: Simon Glass <sjg@chromium.org>
