Commit graph

146 commits

Author SHA1 Message Date
Simon Glass
1d0ebf7d63 tools: binman: Open all binary files in binary mode
At present some files are opened in text mode despite containing binary
data. This works on Python 2 but not always on Python 3, due to unicode
problems. BC&D are not my favourite people. Adjust the affected open()
statements to use binary mode.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:45 -06:00
Simon Glass
5097915428 binman: Use items() instead of iteritems()
Python 3 requires this, and Python 2 allows it. Convert the code over to
ensure compatibility with Python 3.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:45 -06:00
Simon Glass
d141f6c1ca binman: Remove use of Set()
A new built-in set() is used in both Python 2 and 3 now. Move it to use
that instead of Set().

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:45 -06:00
Simon Glass
35343dc414 binman: Don't show errors for failed tests
An unfortunate new feature in Python 3.5 causes binman to print errors for
non-existent tests, when the test filter is used. Work around this by
detecting the unwanted tests and removing them from the result.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:45 -06:00
Simon Glass
2ca8468026 binman: Convert print statements to Python 3
Update all print statements to be functions, as required by Python 3.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-07-10 16:52:45 -06:00
Simon Glass
9481c80f55 binman: Allow sections to have an offset
At present sections are always placed automatically. Even if an 'offset'
property is provided it is ignored. Update the logic to support an offset
for sections.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2019-05-08 12:44:07 +08:00
Simon Glass
0b96f6ecbe binman: Add a missing comment in Entry_vblock
An important property is missing. Update the entry comment to include
this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2019-05-08 12:44:07 +08:00
Simon Glass
eb833d871d binman: Don't show image-skip message by default
This message is not very important since it is simply indicating that the
user's instructions are being followed. Only show it when the verbosity
level is above the default.

Also drop the unnecessary extra newline on this message, which causes two
line breaks.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2019-05-08 12:44:07 +08:00
Simon Glass
5203258c6e binman: Don't generate an error in 'text' entry constructor
It is not good practice to raise an exception in a constructor. In this
case the 'text' entry may not actually be used, if -i is used to filter
out the images that get built.

Move the exception to where the data is actually used.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2019-05-08 12:44:07 +08:00
Simon Glass
4b6dbaa307 binman: Set the pathname correctly for ELF files
At present, stripped files don't have the right pathname which means that
blob compression cannot be used. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
26cc8fccc6 binman: Drop an unnecessary comma in blob handling
This comma is not needed. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
a3c005506a binman: Add a way to enable debugging from the build
When the build fails due to something wrong in binman it is sometimes
useful to get a full backtrace showing the location of the failure. Add
a BINMAN_DEBUG environment variable to support this along with some
documentation.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
3ae192c2a4 binman: Add a test for Intel reference code
Unfortunately the test was not included in the original implementation.
Add one.

Signed-off-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-22 13:57:12 -04:00
Simon Glass
27fb0cf1f0 binman: Add support for Intel reference code
Some platforms use this instead of FSP to set up the platform, including
memory. Add support for this in binman. This is needed for
chromebook_samus, for example.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2018-10-22 17:51:45 +08:00
Simon Glass
741f2d620c binman: Move to three-digit test-file numbers
We now have 99 tests. Before adding any more, rename everything to three
digits. This helps to preserve the ordering of tests and makes it easier
to find things.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-09 04:40:27 -06:00
Simon Glass
11ae93eef4 binman: Run tests concurrently
At present the tests run one after the other using a single CPU. This is
not very efficient. Bring in the concurrencytest module and run the tests
concurrently, using one process for each CPU by default. A -P option
allows this to be overridden, which is necessary for code-coverage to
function correctly.

This requires fixing a few tests which are currently not fully
independent.

At some point we might consider doing this across all pytests in U-Boot.
There is a pytest version that supports specifying the number of processes
to use, but it did not work for me.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
86af511d89 binman: Separate out testSplBssPad()
At present this test runs binman twice, which means that the temporary
files from the first run do not get cleaned up. Split this into two tests
to fix this problem.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
e0e6275f4c binman: Fix up removal of temporary directories
At present 'make check' leaves some temporary directories around. Part of
this is because we call tools.PrepareOutputDir() twice in some cases,
without calling tools.FinaliseOutputDir() in between.

Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
ed59e005e6 binman: Add a default path to libfdt.py
This module is often available in the sandbox_spl build created by
'make check'. Use this as a default path so that just typing 'binman -t'
(without setting PYTHONPATH) will generally run the tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Tom Rini
d24c1d0f4d Merge git://git.denx.de/u-boot-dm 2018-09-30 18:16:51 -04:00
Simon Glass
163ed6c342 binman: Allow writing a map file when something goes wrong
When we get a problem like overlapping regions it is sometimes hard to
figure what what is going on. At present we don't write the map file in
this case. However the file does provide useful information.

Catch any packing errors and write a map file (if enabled with -m) to aid
debugging.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
fe1ae3ecc3 binman: Support ELF files for U-Boot and SPL
For sandbox we want to put ELF files in the image since that is what we
need to execute. Add support for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
f8f8df6eb8 binman: Correct fmap output on x86
Normally x86 platforms use the end-at-4gb option. This currently produces
an FMAP with positions which have a large offset. The use of end-at-4gb is
a useful convenience within binman, but we don't really want to export
a map with these offsets.

Fix this by subtracting the 'skip at start' parameter.

Also put the code which convers names to fmap format, for clarity.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
08723a7abb binman: Record the parent section of each section
At present sections have no record of their parent so it is not possible
to traverse up the tree to the root and figure out the position of a
section within the image.

Change the constructor to record this information.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
f025363543 binman: Support x86 microcode in TPL
When TPL is used on x86 we may want to program the microcode (at least for
the first CPU) early in boot. Add support for this by refactoring the
existing code to be more generic.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
e0e5df9310 binman: Support hashing entries
Sometimesi it us useful to be able to verify the content of entries with
a hash. Add an easy way to do this in binman. The hash information can be
retrieved from the device tree at run time.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
9c888cca5e binman: Mention section attributes in docs
Images and sections have the same attributes, since an image is mostly
just a top-level section. Update the docs to explain this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
ba64a0bbb7 binman: Support expanding entries
It is useful to have entries which can grow automatically to fill
available space. Add support for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
0a98b28b06 binman: Support adding files
In some cases it is useful to add a group of files to the image and be
able to access them at run-time. Of course it is possible to generate
the binman config file with a set of blobs each with a filename. But for
convenience, add an entry type which can do this.

Add required support (for adding nodes and string properties) into the
state module.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
b4e1a38c29 binman: Allow zero-size sections
At present if there is only a zero-size entry in a section this is
reported as an error, e.g.:

   Offset 0x0 (0) is outside the section starting at 0x0 (0)

Adjust the logic in CheckEntries() to avoid this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Simon Glass
83d73c2f7c binman: Support compressed entries
Add support for compressing blob entries. This can help reduce image sizes
for many types of data. It requires that the firmware be able to
decompress the data at run-time.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:32 -06:00
Simon Glass
6ed45ba0a8 binman: Support updating all device tree files
Binman currently supports updating the main device tree with things like
the position of each entry. Extend this support to SPL and TPL as well,
since they may need (a subset of) this information.

Also adjust DTB output files to have a .out extension since this seems
clearer than having a .dtb extension with 'out' in the name somwhere.

Also add a few missing comments and update the DT setup code to use
ReadFile and WriteFile().

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
93d174135a binman: Allow control of whether a fake DT is used
We use a fake device tree in tests most of the time since tests don't
normally care about the actual data. For example, for U-Boot proper we use
U_BOOT_DTB_DATA which is just a four-character string. This makes testing
the image output against an expected value very easy.

However in some cases, such as when the test wants to check that the DT
output containing particular nodes, we do actually need the real DT. Add
support for this, along with a command-line option to select 'test mode'.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
539aece516 binman: Obtain the list of device trees from the config
We always have a device tree for U-Boot proper. But we may also have one
for SPL and TPL. Add a new Entry method to find out what DTs an entry
has, and use that list when updating DTs.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
f46621d255 binman: Centralise device-tree updates within binman
At present we have a few calls to device-tree functions in binman and plan
to add more as we add new entry types which need to report their results.

It makes sense to put this code in a central place so that we can make
sure all device trees are updated. At present we only have U-Boot proper,
but plan to add SPL and TPL too.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
2a72cc72ca binman: Move state logic into the state module
Rather than reaching into this module from control, move the code that
needs this info into state.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
c55a50f558 binman: Move state information into a new module
At present the control module has state information in it, since it is the
primary user of this. But it is a bit odd to have entries and other
modules importing control to obtain this information.

It seems better to have a dedicated state module, which control can use as
well. Create a new module using code from control and update other modules
to use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
6c234bfbf7 binman: Add an entry method for getting the default filename
Various entry implementations provide a way to obtain the default filename
for an entry. But at present there is no base-class implementation for
this function. Add one so that the API is defined.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
fa80c25c09 dtoc: Allow syncing of the device tree back to a file
At present we require the caller to manually update the device tree using
individual calls to libfdt functions. This is not ideal. It would be
better if we could make changes using the Python structure and then call a
Sync() function to write them back.

Add this feature to the Fdt class. Update binman and the tests to match.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
0bfa7b09ba binman: Support building a selection of images
Sometimes it is useful to build only a subset of the images provided by
the binman configuration. Add a -i option for this. It can be given
multiple times to build several images. If the option is not given, all
images are built.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
a326b495cd binman: Tidy up the vblock entry
At present if there are two vblock entries an image their contents are
written to the same file in the output directory. This prevents checking
the contents of each separately.

Fix this by adding part of the entry path to the filename, and add some
missing comments.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
35b384cbe5 binman: Add x86 support for starting TPL
Sometimes we want to include TPL for x86 platforms, such as when we want
to select between different SPL images (e.g. for Chrome OS verified boot).
Add support for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
0b489364f9 binman: Generate an error when text is not provided
When the value of a text entry is not provided an execption is generated
talking about a None type. This is confusing. Add a more explanatory error
and a test for this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
d178eab8f9 binman: Allow 'fill' entry to have a size of 0
The check for this should be for None, not 0. Fix it and add a test.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Simon Glass
f069303852 binman: Move 'special properties' docs to README.entries
This information should be in the entry it relates to, not in the main
README. Move it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-28 11:09:01 -06:00
Jagdish Gediya
9d368f32bc binman: Add support for PowerPC mpc85xx 'bootpg + resetvec' entry
This entry contains the PowerPC mpc85xx boot page and resetvec
sections.

Signed-off-by: Jagdish Gediya <jagdish.gediya@nxp.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: York Sun <york.sun@nxp.com>
2018-09-27 10:13:51 -07:00
Jagdish Gediya
94b57db069 binman: Add a new "skip-at-start" property in Section class
Currently binman calculates '_skip_at_start' based on 'end-at-4gb'
property and it is used for x86 images.

For PowerPC mpc85xx based CPU, CONFIG_SYS_TEXT_BASE is the entry
offset of the first entry. It can be 0xeff40000 or 0xfff40000 for
nor flash boot, 0x201000 for sd boot etc, so "_skip_at_start"
should be set to CONFIG_SYS_TEXT_BASE.

'end-at-4gb' property is not applicable where CONFIG_SYS_TEXT_BASE +
Image size != 4gb.

Add new property 'skip-at-start' in Section class so that
'_skip_at_start' can be calculated either based on 'end-at-4gb'
or based on "skip-at-start".

Add a test case to check that 'skip-at-start' and 'end-at-4gb'
property can't be used together.

Signed-off-by: Jagdish Gediya <jagdish.gediya@nxp.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: York Sun <york.sun@nxp.com>
2018-09-27 10:13:43 -07:00
Michael Heimpold
383d2568a7 binman: fix a few typos in documentation
This fixes four small typos in the README file.

Signed-off-by: Michael Heimpold <mhei@heimpold.de>
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-09-18 08:12:21 -06:00
Simon Glass
cee02e6ff4 binman: Adjust _GetPropTree() parameters
At present this function takes a filename, but it is better to use an Fdt
object so that the caller can control this, perhaps obtainint the device
tree from a bytearray. Update the method accordingly and also fix a
confusing parameter name.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01 16:30:48 -06:00
Simon Glass
15a587c9ce binman: Add a test to catch use of the old 'pos' property
This property has been changed to 'offset'. To help downstream users who
might still be using 'pos', add a check that this is not used by mistake.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-08-01 16:30:48 -06:00