Commit graph

198 commits

Author SHA1 Message Date
Matthias Brugger
2f7c53cbd3 buildman: Enable buildman on aarch64 hosts
At kernel.org aarch64 toolchains are published in folder
arm64. Fix the URL for that case, so that we can fetch
toolchains on aarch64 machines.

Signed-off-by: Matthias Brugger <mbrugger@suse.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-02-05 19:33:46 -07:00
Flavio Suligoi
9de5c397b2 tools: buildman: fix typo
Signed-off-by: Flavio Suligoi <f.suligoi@asem.it>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-01-30 13:30:35 -05:00
Tom Rini
d08c38c32f buildman: Ignore blank lines during size checks
Today when parsing the .sizes files we get a warning about an invalid
line in the file as it's blank.  Solve this by checking that we have a
non-blank line prior to processing.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-12-10 21:11:31 -07:00
Simon Glass
57cb9d5239 buildman: Add options to get the arch and toolchain info
Sometimes it is useful for external tools to use buildman to provide the
toolchain information. Add an -a option which shows the value to use for
the ARCH environment variable, and -A which does the same for
CROSS_COMPILE

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-12-10 21:11:31 -07:00
Simon Glass
7c66ead452 buildman: Figure out boards before commits
At present buildman looks at toolchains, then commits and then boards.
Move the board processing up above the commit processing, since it relates
to the toolchain code. This will make it easier to check the toolchains
needed for a board without processing commits first.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-12-10 21:11:31 -07:00
Simon Glass
6a3fc91ea7 buildman: Ask genboardscfg to be quiet
Now that this tool has a 'quiet' flag, use it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-12-10 21:11:31 -07:00
Cristian Ciocaltea
4251fbc6fb buildman: Improve [make-flags] section parser to allow quoted strings
The parser responsible for the '[make-flags]' section in
the '.buildman' settings file is currently not able to
handle quoted strings, as given in the sample bellow:

[make-flags]
qemu_arm=HOSTCC="cc -isystem /add/include" HOSTLDFLAGS="-L/add/lib"

This patch replaces the simple string splitter based on the <space>
delimiter with a regex tokenizer that preserves spaces inside double
quoted strings.

Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@gmail.com>
2019-12-10 05:54:55 -07:00
Tom Rini
70f6813985 buildman: Fix problem with non-existent output directories
Now that we have buildman telling genboards.cfg to use an output
directory we need to ensure that it exists.

Cc: Bin Meng <bmeng.cn@gmail.com>
Cc: Simon Glass <sjg@chromium.org>
Fixes: bc750bca12 ("tools: buildman: Honor output directory when generating boards.cfg")
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2019-11-23 14:53:48 -05:00
Simon Glass
c05aa03642 buildman: Convert to Python 3
Convert buildman to Python 3 and make it use that, to meet the 2020
deadline.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-11-04 18:15:32 -07:00
Bin Meng
578b637203 tools: buildman: Remove useless mkdir() in Make() in test.py
In the 'Make' function, the codes tries to create a directory
if current stage is 'build'. But the directory isn't used at
all anywhere.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-10-29 16:17:39 -04:00
Bin Meng
bc750bca12 tools: buildman: Honor output directory when generating boards.cfg
buildman always generates boards.cfg in the U-Boot source tree.
When '-o' is given, we should generate boards.cfg to the given
output directory.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-10-29 16:17:36 -04:00
Tom Rini
aae62584a6 buildman: Perform tests in a temporary directory
We may not always be able to write to the default output directory so
have a temporary directory for our output be created.

Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@wwwdotorg.org>
Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2019-10-08 12:10:38 -04:00
Tom Rini
65e05ddc1a kconfiglib: Update to the 12.14.0 release
A large number of changes have happened upstream since our last sync
which was to 375506d.  The reason to do the upgrade at this point is for
improved Python 3 support.

As part of this upgrade we need to update moveconfig.py and
genboardscfg.py the current API.  This is:
- Change "kconfiglib.Config" calls to "kconfiglib.Kconfig"
- Change get_symbol() calls to syms.get().
- Change get_value() to str_value.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
2019-10-08 08:37:29 -04:00
Baruch Siach
2429613601 buildman: fix invocation examples typos
Also, make command line example indentation consistent.

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
2019-07-24 12:54:08 -07:00
Simon Glass
347ea0b63e buildman: Deal more nicely with invalid build-status file
The 'done' files created by buildman may end up being empty if buildman
runs out of disk space while writing them. At present buildman dies with
an exception when using -s to check the build status. Fix this.

Seriesl-cc: trini

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-05-21 17:33:23 -06:00
Chris Packham
58804b8cf2 buildman: fix typo
Fix a typo in the error message from CheckOutputDir().

Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2019-02-09 12:50:22 -07:00
Simon Glass
e5fc79ea71 buildman: Write the environment out to an 'env' file
Sometimes it is useful to see the environment that was used to build
U-Boot. Write this out to a file in the build directory.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-02-09 12:50:22 -07:00
Simon Glass
ccd2979a8d buildman: Fix tabs in GetWrapper()
This function has tabs instead of spaces. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-01-14 17:47:13 -07:00
Simon Glass
00beb2485f buildman: Add support for building with clang
Add a -O option which allows building with clang.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-01-14 17:47:13 -07:00
Simon Glass
e74429bb17 buildman: Deal nicely with invalid build-status file
The 'done' files created by buildman may end up being empty if buildman
runs out of disk space while writing them. This error is then persistent,
since even if disk space is reclaimed and the build retries, the empty
file causes an exception in the builder thread.

Deal with this silently by doing a rebuild.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-01-14 17:47:13 -07:00
Simon Glass
90a29fcc1f buildman: Drop comment about Ctrl-C problem
This bug is now fixed, so drop this comment.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-01-14 17:47:13 -07:00
Trevor Woerner
b11f12642f buildman/toolchain.py: handle inconsistent tarball names
Unfortunately, for some releases the kernel.org toolchain tarball names adhere
to the following pattern:

	<hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xz

e.g.:
	x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz

while others use the following pattern:

	<hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xz

e.g.:

	x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xz

Notice that the first pattern has dashes throughout, while the second has
dashes throughout except just before the target architecture which has an
underscore.

The "dash throughout" versions from kernel.org are:

	8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1

while the "dash and underscore" versions from kernel.org are:

	7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4

This tweak allows the code to handle both versions. Note that this tweak also
causes the architecture parsing to get confused and find the following two
bogus architectures, "2.0" and "64", which are explicitly checked for, and
removed.

Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Change single quotes to double quotes:
Signed-off-by: Simon Glass <sjg@chromium.org>
2018-12-05 06:06:30 -07:00
Trevor Woerner
d82f539ab9 buildman/toolchain.py: fix toolchain directory
The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in
a way that is different from most toolchains. The first entry when unpacking
most toolchain tarballs is:

	gcc-<version>-nolib/<targetarch>-<system>

e.g.:

	gcc-8.1.0-nolibc/aarch64-linux/

The first entry of the hexagon toolchain, however, is:

	gcc-4.6.1-nolibc/

This causes the buildman logic in toolchain.py::ScanPath() to not be able to
find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin}
instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman
tries to download a set of toolchains that includes hexagon, the script fails.

This update takes the second line of the tarball unpacking (which works for
all the toolchains I've tested from kernel.org) and parses it to take the
first two elements, separated by '/'. It makes this logic a bit more robust.

Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-12-05 06:01:35 -07:00
Simon Glass
6af7101b75 buildman: Show boards with warning with w+
At present we should boards with warnings in the same way as those with
errors. This is not ideal. Add a new 'warn' state and show these listed
in yellow to match the actual warning lines printing with -e.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
4cf2b221c6 buildman: Rename the good, better, worse variables
At present we don't distinguish between errors and warnings when printing
the architecture summary. Rename the variables to better describe their
purpose.

'Worse' at present means we got an error, so use that as the name.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
2d48333e44 buildman: Detect dtc warnings
At present messages from the device-tree compiler like this:

  arch/arm/dts/socfpga_arria10_socdk_sdmmc.dtb: Warning
     (avoid_unnecessary_addr_size): /clocks: unnecessary
     #address-cells/#size-cells without "ranges" or child "reg" property

are detected as errors since they don't match the gcc warning regex. Add a
new one for dtc to fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
40232c91d7 buildman: Only print toolchain probing with -v
At present --list-tool-chains prints a lot of information about the
toolchain-probing process. This is generally not very interesting.
Update buildman to print this only if --list-tool-chains is given
with -v.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-20 19:14:22 -07:00
Simon Glass
0689036a35 buildman: Add a --boards option to specify particular boards to build
At present 'buildman sandbox' will build all 5 boards for the sandbox
architecture rather than the single board 'sandbox'. The only current way
to exclude sandbox_spl, sandbox_noblk, etc. is to use -x which is a bit
clumbsy.

Add a --boards option to allow individual build targets to be specified.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-11-14 09:16:27 -08:00
Simon Glass
4b4bc06ef4 buildman: dtoc: Suppress unwanted output from test
There are a few test cases which print output. Suppress this so that tests
can run silently in the normal case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
da753e35cb buildman: Make the toolchain test more forgiving
The filenames of the toolchains on kernel.org changes every now and then.
Fix it for the current change, and make the test use a regex so that it
has a better chance of passing with future changes too.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-10-08 07:34:34 -06:00
Simon Glass
e62a24ce27 buildman: Avoid hanging when the config changes
Something has changed in the last several month such that when buildman
builds U-Boot incrementally and a new CONFIG option has been added to the
Kconfig, the build hanges waiting for input:

    Test new config (NEW_CONFIG) [N/y/?] (NEW)

Since binamn does not connect the build's stdin to anything this waits on
stdin to the build thread, which never comes. Eventually I suspect all the
threads end up in this state and the build does not progress.

Fix this by passing /dev/null as input to the build. That way, if there is
a new CONFIG, the build will stop (and fail):

    Test new config (NEW_CONFIG) [N/y/?] (NEW)

    Error in reading or end of file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2018-09-29 11:49:35 -06:00
Masahiro Yamada
9865543ae6 Remove CONFIG_USE_STDINT
You do not need to use the typedefs provided by compiler.

Our compilers are either IPL32 or LP64.  Hence, U-Boot can/should
always use int-ll64.h typedefs like Linux kernel, whatever the
typedefs the compiler internally uses.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2018-09-10 20:48:16 -04:00
Alex Kiernan
48ae412424 buildman: Add support for environment delta in summary
When summarising the builds, add the -U option to emit delta lines for
the default environment built into U-Boot at each commit.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-06-07 11:25:07 -08:00
Alex Kiernan
0ddc510ea3 buildman: Extract environment as part of each build
As we're building the boards, extract the default U-Boot environment to
uboot.env so we can interrogate it later.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-06-07 11:25:07 -08:00
Daniel Schwierzeck
4c58d273e0 buildman: support newer gcc versions from kernel.org
Add support for gcc versions 7.3.0, 6.4.0 and 4.9.4.

Also use a regex for matching the tarball names. Some gcc versions
use '-ARCH-' instead of '_ARCH-'.

As part of this, we switch TravisCI to also using these toolchains for
all platforms.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
2018-05-22 22:08:57 -04:00
Lothar Waßmann
409fc029c4 tools: buildman: Don't use the working dir as build dir
When the U-Boot base directory happens to have the same name as the branch
that buildman is directed to use via the '-b' option and no output
directory is specified with '-o', buildman happily starts removing the
whole U-Boot sources eventually only stopped with the error message:

OSError: [Errno 20] Not a directory: '../<branch-name>/boards.cfg

Add a check to avoid this and also deal with the case where '-o' points
to the source directory, or any subdirectory of it.

Finally, tidy up the confusing logic for removing the old tree when using
-b. This is only done when building a branch.

Signed-off-by: Lothar Waßmann <LW@KARO-electronics.de>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Lothar Waßmann <LW@KARO-electronics.de>
2018-05-16 00:25:19 -06:00
Tom Rini
83d290c56f SPDX: Convert all of our single license tags to Linux Kernel style
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from.  So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry.  Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents.  There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>
2018-05-07 09:34:12 -04:00
Daniel Schwierzeck
2371d1bcbf buildman: add option -E for treating compiler warnings as errors
Add a new option '-E' for treating all compiler warnings as errors.
Eventually this will pass 'KCFLAGS=-Werror' to Kbuild.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2018-02-04 22:55:34 -05:00
Masahiro Yamada
94b13bbae9 host-tools: use python2 explicitly for shebang
All of these host tools are apparently written for Python2,
not Python3.

Use 'python2' in the shebang line according to PEP 394
(https://www.python.org/dev/peps/pep-0394/).

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-01-28 12:27:30 -05:00
Tom Rini
3759df0c08 tools: Update python "help" tests to cope with "more" oddities
In some cases when "more" is told to page a given file it will prepend
the output with:
::::::::::::::
/PATH/TO/THE/FILE
::::::::::::::

And when this happens the output will not match the expected length.
Further, if we use a different pager we will instead fail the coverage
tests as we will not have 100% coverage.  Update the help test to remove
the string in question.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2018-01-16 16:15:25 -05:00
Simon Glass
251f5867c9 buildman: Fix up tests
The tests were broken by two separate commits which adjusted the output
when boards are listed. Fix this by adding back a PowerPC board and
putting the name of each board in the test.

Fixes: b9f7d881 (powerpc, 5xx: remove some "5xx" remains)
Fixes: 8d7523c5 (buildman: Allow showing the list of boards with -n)

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-11-22 18:05:38 -07:00
Simon Glass
cb39a10979 buildman: Allow skipping of tests which use the network
Accessing the network slows down the test and limits the environment in
which it can be run. Add an option to disable network tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-11-22 18:05:38 -07:00
Ulf Magnusson
4e1102f6de kconfiglib: update with 'imply' support
Corresponds to 375506d (File writing nit) from upstream
(https://github.com/ulfalizer/Kconfiglib).

Adds proper 'imply' support and fixes a few minor issues, one of which
previously triggered the following weird warning:

  configs/taurus_defconfig: /tmp/tmpisI45S:6: warning: assignment to SPL_LDSCRIPT changes mode of containing choice from "arch/$(ARCH)/cpu/u-boot-spl.lds" to "y"

The change in 8639f69 (genconfig.py: Print defconfig next to warnings)
was reapplied.

tools/moveconfig.py previously depended on a hack that merged 'select's
with 'imply's. It was modified to look at the union of
Symbol.get_selected_symbols() and Symbol.get_implied_symbols(), which
should give the same behavior.

tools/genboardscfg.py was verified to produce identical board.cfg's
before and after the change.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2017-09-29 14:07:54 -04:00
Simon Glass
8639f69a61 genconfig.py: Print defconfig next to warnings
At present we sometimes see warnings of the form:

/tmp/tmpMA89kB:36: warning: overriding the value of CMD_SPL.
	Old value: "y", new value: "y".

This is not very useful as it does not show whch defconfig file it relates
to. Update the tool to show this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-09-11 21:43:58 -06:00
Andy Shevchenko
daab59ac05 avr32: Retire AVR32 for good
AVR32 is gone. It's already more than two years for no support in Buildroot,
even longer there is no support in GCC (last version is heavily patched 4.2.4).

Linux kernel v4.12 got rid of it (and v4.11 didn't build successfully).

There is no good point to keep this support in U-Boot either.

Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
2017-07-06 16:17:19 -04:00
Heiko Schocher
b9f7d88174 powerpc, 5xx: remove some "5xx" remains
we removed 5xx support. So delete some forgotten remains.

Signed-off-by: Heiko Schocher <hs@denx.de>
2017-06-16 10:14:56 -04:00
Daniel Schwierzeck
aafbe82fb6 buildman: properly translate strings for log and err files to ASCII
The build output can still produce unicode encoded output. But in
the buildman's log and err files we only want plain ASCII characters.

To handle all situations with unicode and non-unicode output, encode
the stdout and stderr strings to UTF-8 and afterwards to ASCII with
replacing all special characters.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2017-06-09 13:45:34 -06:00
Daniel Schwierzeck
b0e994c29e buildman: disable localized and unicode output of all build tools
Build tools like Make, gcc or binutils support localized output
or unicode encoded output dependent on the default system locale.
This is not useful for buildman, where we want reproducible
warning or error messages or where the output of binutils is
further processed.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
2017-06-09 13:45:34 -06:00
Tom Rini
d5686a61d6 buildman: Fix bloat option when 'new' only drops functions
In the case where a new build only decreases sizes and does not increase
any size we still want to report what functions have been dropped when
doing a bloat comparison.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-06-08 20:21:59 -06:00
Tom Rini
546a6f3a9b buildman: Allow 'gnueabihf' toolchains for ARM
Many toolchains for ARM use the 'gnueabihf' suffix rather than just
'gnueabi', so allow these to be used, but with a lower priority than
'gnueabi' ones.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-04-18 10:29:19 -04:00