It is a bad idea to use the default output directory ('..') with -w since
it does a build in that directory and writes various files these.
Require that -o is given to avoid this problem.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current exit codes of 128 and 129 are useful in that they do not
conflict with those returned by tools, but they are not actually valid.
It seems better to pick some codes which work with 'bit bisect run'.
Update them to 100 (for errors) and 101 (for warnings).
Signed-off-by: Simon Glass <sjg@chromium.org>
These are becoming more common now. They cause boards to show warnings
which can be mistaking for compiler warnings.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
Unfortunately the plague of device-tree warnings has not lifted. These
warnings infiltrate almost every build, adding noise and confusion.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman defaults to running 'mrproper' on every thread before
it starts building commits for each board. This can add a delay of about 5
seconds to the start of the process, since the tools and other invariants
must be rebuilt.
In particular, a build without '-b', to build current source, runs much
slower without -I, since any existing build is removed, thus losing the
possibility of an incremental build.
Partly this behaviour was to avoid strange build-system problems caused by
running 'make defconfig' for one board and then one with a different
architecture. But these problems were fixed quite a while ago.
The -I option (which disabled mrproper) was introduced four years ago and
does not seem to cause any problems with builds.
So make -I the default and deprecate the option. To allow use of
'mrproper', add a new -m flag.
Signed-off-by: Simon Glass <sjg@chromium.org>
When buildman finishes it leaves the last summary line visible, which
shows the number of successful builds, builds with warnings and builds
with errors.
It is useful also to see how many builds were done in total along with
the time taken. Show these on a separate line before buildman finishes.
Signed-off-by: Simon Glass <sjg@chromium.org>
If a progress message is longer than the terminal line it will scroll the
terminal. Limit the messages to the terminal width.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is nice to see the actual number of builds remaining to complete. Add
this in the progress message, using a different colour.
Drop the unnecessary 'name' variable while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
The commit counter is a hangover from when buildman processed each board
for a commit. Now buildman processes each commit for a board, so this
output is never triggered.
Delete it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fetching updated versions of a repo can take time. At present buildman
gives no indication that it is doing this.
Add a message to explain the delay.
Tidy up a few other messages while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the board names shown with -l are separated by commas. This
makes it hard to double-click to select a particular board. Also it is not
possible to select all boards and paste them as arguments to a subsequent
buildman run, since buildman requires spaces to separate the list on the
command line, not commas.
Change the output format to use spaces instead of commas.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is quite hard to see the list of board for each error line since the
colour is the same as the actual error line. Show the board list in
magenta so that it is easier to distinguish them.
There is no point in checking the colour of the overall line, since there
are now multiple colours. So drop those tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the string for each error line is created in _CalcErrorDelta()
and used to create the summary output. This is inflexible since all the
information (error/warning character, error line, list of boards with that
error line) is munged together in a string.
Create an object to hold this information and only convert it to a string
when printing the actual output.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present warnings are shown in yellow in the summary (-s) but magenta in
the detail listing (-e). Use yellow in both.
Signed-off-by: Simon Glass <sjg@chromium.org>
We want to add a few more tests similar to testOutput(). Split its logic
into a function which runs buildman to get the output and another which
checks the output. This will make it easier to reuse the code.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than having a few tests handle this themselves, create the
temporary directory in the setUp() method and remove it in tearDown().
This will make it easier to add more tests.
Only testOutput and testGit() actually need it, but it doesn't add to the
test time noticeably to do this for all tests in this file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than using the absolute array index, use an interator to work
through the expected output lines. This is easier to follow.
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman should output the right colours for each error/warning line. Some
of these checks are missing. Add them.
Signed-off-by: Simon Glass <sjg@chromium.org>
There is no point in setting the ARCH environment variable since the
U-Boot build system no-longer uses it.
It seems safe to drop this feature since it was only recently added.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
This is useful in some situations, in particular with -w and when building
in-tree. Now that we are more careful about what we remove in
_PrepareOutputSpace(), it should be safe to relax this restriction.
Update the progress information also so it is clear what buildman is
doing. Remove files can take a long time.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman removes any directory it doesn't intend to write
output into. This is overly expansive since if the output directory
happens to be somewhere with existing files, they may be removed. Using
an existing directory for buildman is not a good practice, but since the
result might be catastrophic, it is best to guard against it.
A previous commit[1] fixed this by refusing to write to a subdirectory
of the current directory, assumed to have U-Boot source code. But we can
do better by only removing directories that look like the ones buildman
creates.
Update the code to do this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
[1] 409fc029c4 tools: buildman: Don't use the working dir as build dir
Sometimes we don't want buildman to return failure if it seems warnings.
Add a -W option to support this. If buildman detects warnings (and no
errors) it will return an exit code of 0 (success).
Note that the definition of 'warnings' includes the migration warnings
produced by U-Boot, such as:
===================== WARNING ======================
This board does not use CONFIG_DM_MMC. Please update
...
====================================================
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
This help is a bit ambiguous. It only does anything if asked to show size
changes with -S. Update the help and the function comments.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to run a simple build and put all the output in a single
directory. Add a -w option to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
This class has a few more members now. Add documentation for them and fix
a nit in the 'commits' comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
At kernel.org aarch64 toolchains are published in folder
arm64. Fix the URL for that case, so that we can fetch
toolchains on aarch64 machines.
Signed-off-by: Matthias Brugger <mbrugger@suse.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Today when parsing the .sizes files we get a warning about an invalid
line in the file as it's blank. Solve this by checking that we have a
non-blank line prior to processing.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful for external tools to use buildman to provide the
toolchain information. Add an -a option which shows the value to use for
the ARCH environment variable, and -A which does the same for
CROSS_COMPILE
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman looks at toolchains, then commits and then boards.
Move the board processing up above the commit processing, since it relates
to the toolchain code. This will make it easier to check the toolchains
needed for a board without processing commits first.
Signed-off-by: Simon Glass <sjg@chromium.org>
The parser responsible for the '[make-flags]' section in
the '.buildman' settings file is currently not able to
handle quoted strings, as given in the sample bellow:
[make-flags]
qemu_arm=HOSTCC="cc -isystem /add/include" HOSTLDFLAGS="-L/add/lib"
This patch replaces the simple string splitter based on the <space>
delimiter with a regex tokenizer that preserves spaces inside double
quoted strings.
Signed-off-by: Cristian Ciocaltea <cristian.ciocaltea@gmail.com>
Now that we have buildman telling genboards.cfg to use an output
directory we need to ensure that it exists.
Cc: Bin Meng <bmeng.cn@gmail.com>
Cc: Simon Glass <sjg@chromium.org>
Fixes: bc750bca12 ("tools: buildman: Honor output directory when generating boards.cfg")
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
In the 'Make' function, the codes tries to create a directory
if current stage is 'build'. But the directory isn't used at
all anywhere.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
buildman always generates boards.cfg in the U-Boot source tree.
When '-o' is given, we should generate boards.cfg to the given
output directory.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
We may not always be able to write to the default output directory so
have a temporary directory for our output be created.
Cc: Simon Glass <sjg@chromium.org>
Reviewed-by: Stephen Warren <swarren@wwwdotorg.org>
Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
A large number of changes have happened upstream since our last sync
which was to 375506d. The reason to do the upgrade at this point is for
improved Python 3 support.
As part of this upgrade we need to update moveconfig.py and
genboardscfg.py the current API. This is:
- Change "kconfiglib.Config" calls to "kconfiglib.Kconfig"
- Change get_symbol() calls to syms.get().
- Change get_value() to str_value.
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
The 'done' files created by buildman may end up being empty if buildman
runs out of disk space while writing them. At present buildman dies with
an exception when using -s to check the build status. Fix this.
Seriesl-cc: trini
Signed-off-by: Simon Glass <sjg@chromium.org>
Fix a typo in the error message from CheckOutputDir().
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Sometimes it is useful to see the environment that was used to build
U-Boot. Write this out to a file in the build directory.
Signed-off-by: Simon Glass <sjg@chromium.org>
The 'done' files created by buildman may end up being empty if buildman
runs out of disk space while writing them. This error is then persistent,
since even if disk space is reclaimed and the build retries, the empty
file causes an exception in the builder thread.
Deal with this silently by doing a rebuild.
Signed-off-by: Simon Glass <sjg@chromium.org>
Unfortunately, for some releases the kernel.org toolchain tarball names adhere
to the following pattern:
<hostarch>-gcc-<ver>-nolib-<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-8.1.0-nolibc-aarch64-linux.tar.xz
while others use the following pattern:
<hostarch>-gcc-<ver>-nolib_<targetarch>-<type>.tar.xz
e.g.:
x86_64-gcc-7.3.0-nolibc_aarch64-linux.tar.xz
Notice that the first pattern has dashes throughout, while the second has
dashes throughout except just before the target architecture which has an
underscore.
The "dash throughout" versions from kernel.org are:
8.1.0, 6.4.0, 5.5.0, 4.9.4, 4.8.5, 4.6.1
while the "dash and underscore" versions from kernel.org are:
7.3.0, 4.9.0, 4.8.0, 4.7.3, 4.6.3, 4.6.2, 4.5.1, 4.2.4
This tweak allows the code to handle both versions. Note that this tweak also
causes the architecture parsing to get confused and find the following two
bogus architectures, "2.0" and "64", which are explicitly checked for, and
removed.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Change single quotes to double quotes:
Signed-off-by: Simon Glass <sjg@chromium.org>
The hexagon toolchain (4.6.1) from kernel.org, for example, was packaged in
a way that is different from most toolchains. The first entry when unpacking
most toolchain tarballs is:
gcc-<version>-nolib/<targetarch>-<system>
e.g.:
gcc-8.1.0-nolibc/aarch64-linux/
The first entry of the hexagon toolchain, however, is:
gcc-4.6.1-nolibc/
This causes the buildman logic in toolchain.py::ScanPath() to not be able to
find the "*gcc" executable since it looks in gcc-4.6.1-nolib/{.|bin|usr/bin}
instead of gcc-4.6.1/hexagon-linux/{.|bin|usr/bin}. Therefore when buildman
tries to download a set of toolchains that includes hexagon, the script fails.
This update takes the second line of the tarball unpacking (which works for
all the toolchains I've tested from kernel.org) and parses it to take the
first two elements, separated by '/'. It makes this logic a bit more robust.
Signed-off-by: Trevor Woerner <trevor@toganlabs.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present we should boards with warnings in the same way as those with
errors. This is not ideal. Add a new 'warn' state and show these listed
in yellow to match the actual warning lines printing with -e.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we don't distinguish between errors and warnings when printing
the architecture summary. Rename the variables to better describe their
purpose.
'Worse' at present means we got an error, so use that as the name.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present messages from the device-tree compiler like this:
arch/arm/dts/socfpga_arria10_socdk_sdmmc.dtb: Warning
(avoid_unnecessary_addr_size): /clocks: unnecessary
#address-cells/#size-cells without "ranges" or child "reg" property
are detected as errors since they don't match the gcc warning regex. Add a
new one for dtc to fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present --list-tool-chains prints a lot of information about the
toolchain-probing process. This is generally not very interesting.
Update buildman to print this only if --list-tool-chains is given
with -v.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present 'buildman sandbox' will build all 5 boards for the sandbox
architecture rather than the single board 'sandbox'. The only current way
to exclude sandbox_spl, sandbox_noblk, etc. is to use -x which is a bit
clumbsy.
Add a --boards option to allow individual build targets to be specified.
Signed-off-by: Simon Glass <sjg@chromium.org>
There are a few test cases which print output. Suppress this so that tests
can run silently in the normal case.
Signed-off-by: Simon Glass <sjg@chromium.org>
The filenames of the toolchains on kernel.org changes every now and then.
Fix it for the current change, and make the test use a regex so that it
has a better chance of passing with future changes too.
Signed-off-by: Simon Glass <sjg@chromium.org>
Something has changed in the last several month such that when buildman
builds U-Boot incrementally and a new CONFIG option has been added to the
Kconfig, the build hanges waiting for input:
Test new config (NEW_CONFIG) [N/y/?] (NEW)
Since binamn does not connect the build's stdin to anything this waits on
stdin to the build thread, which never comes. Eventually I suspect all the
threads end up in this state and the build does not progress.
Fix this by passing /dev/null as input to the build. That way, if there is
a new CONFIG, the build will stop (and fail):
Test new config (NEW_CONFIG) [N/y/?] (NEW)
Error in reading or end of file.
Signed-off-by: Simon Glass <sjg@chromium.org>
You do not need to use the typedefs provided by compiler.
Our compilers are either IPL32 or LP64. Hence, U-Boot can/should
always use int-ll64.h typedefs like Linux kernel, whatever the
typedefs the compiler internally uses.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
When summarising the builds, add the -U option to emit delta lines for
the default environment built into U-Boot at each commit.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
As we're building the boards, extract the default U-Boot environment to
uboot.env so we can interrogate it later.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add support for gcc versions 7.3.0, 6.4.0 and 4.9.4.
Also use a regex for matching the tarball names. Some gcc versions
use '-ARCH-' instead of '_ARCH-'.
As part of this, we switch TravisCI to also using these toolchains for
all platforms.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
When the U-Boot base directory happens to have the same name as the branch
that buildman is directed to use via the '-b' option and no output
directory is specified with '-o', buildman happily starts removing the
whole U-Boot sources eventually only stopped with the error message:
OSError: [Errno 20] Not a directory: '../<branch-name>/boards.cfg
Add a check to avoid this and also deal with the case where '-o' points
to the source directory, or any subdirectory of it.
Finally, tidy up the confusing logic for removing the old tree when using
-b. This is only done when building a branch.
Signed-off-by: Lothar Waßmann <LW@KARO-electronics.de>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Lothar Waßmann <LW@KARO-electronics.de>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
Add a new option '-E' for treating all compiler warnings as errors.
Eventually this will pass 'KCFLAGS=-Werror' to Kbuild.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
All of these host tools are apparently written for Python2,
not Python3.
Use 'python2' in the shebang line according to PEP 394
(https://www.python.org/dev/peps/pep-0394/).
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
In some cases when "more" is told to page a given file it will prepend
the output with:
::::::::::::::
/PATH/TO/THE/FILE
::::::::::::::
And when this happens the output will not match the expected length.
Further, if we use a different pager we will instead fail the coverage
tests as we will not have 100% coverage. Update the help test to remove
the string in question.
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
The tests were broken by two separate commits which adjusted the output
when boards are listed. Fix this by adding back a PowerPC board and
putting the name of each board in the test.
Fixes: b9f7d881 (powerpc, 5xx: remove some "5xx" remains)
Fixes: 8d7523c5 (buildman: Allow showing the list of boards with -n)
Signed-off-by: Simon Glass <sjg@chromium.org>
Accessing the network slows down the test and limits the environment in
which it can be run. Add an option to disable network tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
Corresponds to 375506d (File writing nit) from upstream
(https://github.com/ulfalizer/Kconfiglib).
Adds proper 'imply' support and fixes a few minor issues, one of which
previously triggered the following weird warning:
configs/taurus_defconfig: /tmp/tmpisI45S:6: warning: assignment to SPL_LDSCRIPT changes mode of containing choice from "arch/$(ARCH)/cpu/u-boot-spl.lds" to "y"
The change in 8639f69 (genconfig.py: Print defconfig next to warnings)
was reapplied.
tools/moveconfig.py previously depended on a hack that merged 'select's
with 'imply's. It was modified to look at the union of
Symbol.get_selected_symbols() and Symbol.get_implied_symbols(), which
should give the same behavior.
tools/genboardscfg.py was verified to produce identical board.cfg's
before and after the change.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
At present we sometimes see warnings of the form:
/tmp/tmpMA89kB:36: warning: overriding the value of CMD_SPL.
Old value: "y", new value: "y".
This is not very useful as it does not show whch defconfig file it relates
to. Update the tool to show this.
Signed-off-by: Simon Glass <sjg@chromium.org>
AVR32 is gone. It's already more than two years for no support in Buildroot,
even longer there is no support in GCC (last version is heavily patched 4.2.4).
Linux kernel v4.12 got rid of it (and v4.11 didn't build successfully).
There is no good point to keep this support in U-Boot either.
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
The build output can still produce unicode encoded output. But in
the buildman's log and err files we only want plain ASCII characters.
To handle all situations with unicode and non-unicode output, encode
the stdout and stderr strings to UTF-8 and afterwards to ASCII with
replacing all special characters.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Build tools like Make, gcc or binutils support localized output
or unicode encoded output dependent on the default system locale.
This is not useful for buildman, where we want reproducible
warning or error messages or where the output of binutils is
further processed.
Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
In the case where a new build only decreases sizes and does not increase
any size we still want to report what functions have been dropped when
doing a bloat comparison.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Many toolchains for ARM use the 'gnueabihf' suffix rather than just
'gnueabi', so allow these to be used, but with a lower priority than
'gnueabi' ones.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
When writing out some of our results we may now have UTF-8 characters
in there as well. Translate these to latin-1 and ignore any errors (as
this is for diagnostic and given the githash anything else can be
reconstructed by the user.
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
One of these has crept in in this commit:
40a808f1 ARCv2: SLC: Make sure busy bit is set properly on SLC flushing
Adjust buildman to handle it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Currently upstream does not yet understand the imply keyword. For what
we use kconfiglib.py for today, this is OK. We only need to be able to
evaluate in order to make boards.cfg and none of those choices will
depend on how imply evaluates out.
Signed-off-by: Tom Rini <trini@konsulko.com>
As well as showing the number of boards, allow showing the actual list of
boards that would be built, if -v is provided.
Signed-off-by: Simon Glass <sjg@chromium.org>
To troubleshoot unexpected bhavior during building and what's more
important during execution it is strongly recommended to use recent
ARC toolchain, and so we're now referring to arc-2016.09 which is the
latest as of today.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
When using #define CONFIG_SOME_OPTION, the value it set to '1'. When using
defconfig (i.e. CONFIG_SOME_OPTION=y) the value is set to 'y'. This results
in differences showing up with -K. These differences are seldom useful.
Adjust buildman to suppress these differences by default.
Signed-off-by: Simon Glass <sjg@chromium.org>
Normally buildman does a full build of a board. This includes creating the
u-boot.cfg file which contains all the configuration options. Buildman uses
this file with the -K option, to show differences in effective configuration
for each commit.
Doing a full build of U-Boot just to create the u-boot.cfg file is wasteful.
Add a -D option which causes buildman to only create the configuration. This
is enough to support use of -K and can be done much more quickly (typically
5-10 times faster).
Signed-off-by: Simon Glass <sjg@chromium.org>
The README for buildman says that we can use any field in boards.cfg to
decide what to build. However, we were not saving the options field
correctly.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
Now we can use compiler wrapper such as ccache or distcc for buildman.
Signed-off-by: York Sun <york.sun@nxp.com>
CC: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Signed-off-by: York Sun <york.sun@nxp.com>
CC: Simon Glass <sjg@chromium.org>
Fixed commit subject:
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
When Ctrl-C is pressed, just exited quietly. There is no sense in displaying
a stack trace since buildman will always be in the same place: waiting for
threads to complete building all the jobs on the queue.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is annoying that buildman does not respond cleanly to Ctrl-C or SIGINT,
particularly on machines with lots of CPUS. Unfortunately queue.join()
blocks the main thread and does not allow it to see the signal. Use a
separate thread instead,
Signed-off-by: Simon Glass <sjg@chromium.org>
If patman is installed on the machine (e.g. in the standard dist-packages
directory), it will find libraries from there in preference to our local
libraries. Adjust the order of the path to ensure that local libraries are
found first.
Signed-off-by: Simon Glass <sjg@chromium.org>
Make it clear when buildman actually starts building. This happens when it
has prepared the threads, working directory and output directories.
Signed-off-by: Simon Glass <sjg@chromium.org>
When buildman starts, it prepares its output directory by removing any old
build directories which will not be used this time. This can happen if a
previous build left directories around for commit hashes which are no-longer
part of the branch.
This can take quite a while, so print a message to indicate what is going
on.
Signed-off-by: Simon Glass <sjg@chromium.org>
On a machine with a lot of CPUs this prints a lot of useless lines of the
form:
Cloning repo for thread <n>
Adjust the output so that these all appear on one line, and disappear when
the cloning is complete.
Note: This cloning is actually unnecessary and very wasteful on disk space
(about 3.5GB each time). It would be better to create symlinks.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is more useful to have this method raise an error when something goes
wrong. Make this the default and adjust the few callers that don't want to
use it this way.
Signed-off-by: Simon Glass <sjg@chromium.org>
For those who just want to build a board, it is useful to see a quick hint
right at the start of the documentation. Add a few commands showing how to
download toolchains and build a board.
Signed-off-by: Simon Glass <sjg@chromium.org>
The current code for setting up the toolchain config always writes the new
paths to an item called 'toolchain'. This means that it will overwrite any
existing toolchain item with the same name. In practice, this means that:
buildman --fetch-arch all
will fetch all toolchains, but only the path of the final one will be added
to the config. This normally works out OK, since most toolchains are the
same version (e.g. gcc 4.9) and will be found on the same path. But it is
not correct and toolchains for archs which don't use the same version will
not function as expected.
Adjust the code to use a complete glob of the toolchain path.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
It doesn't make sense to complain about missing toolchains when the
--fetch-arch option is being used. The user is presumably aware that there
is a toolchain problem and is actively correcting it by running with this
option.
Refactor the code to avoid printing this confusing message.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use colour to make it easier to see what is going on. Also print a message
before downloading a new toolchain. Mention --fetch-arch in the message that
is shown when there are no available toolchains, since this is the quickest
way to resolve the problem.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
When there are no toolchains a warning is printed. But in some cases this is
confusing, such as when the user is fetching new toolchains.
Adjust the function to supress the warning in this case.
Signed-off-by: Simon Glass <sjg@chromium.org>
If there is no ~/.buildman file, buildman currently complains and exists. To
make things a little more friendly, create an empty one automatically. This
will not allow things to be built, but --fetch-arch can be used to handle
that.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
One use-case for buildman is to continually run it interactively after
each small step in a large refactoring operation. This gives more
immediate feedback than making a number of commits and then going back and
testing them. For this to work well, buildman needs to be extremely fast.
At present, a couple issues prevent it being as fast as it could be:
1) Each time buildman runs "make %_defconfig", it runs "make mrproper"
first. This throws away all previous build results, requiring a
from-scratch build. Optionally avoiding this would speed up the build, at
the cost of potentially causing or missing some build issues.
2) A build tree is created per thread rather than per board. When a thread
switches between building different boards, this often causes many files
to be rebuilt due to changing config options. Using a separate build tree
for each board would avoid this. This does put more strain on the system's
disk cache, but it is worth it on my system at least.
This commit adds two command-line options to implement the changes
described above; -I ("--incremental") turns of "make mrproper" and -P
("--per-board-out-dir") creats a build directory per board rather than per
thread.
Tested:
./tools/buildman/buildman.py tegra
./tools/buildman/buildman.py -I -P tegra
./tools/buildman/buildman.py -b tegra_dev tegra
./tools/buildman/buildman.py -b tegra_dev -I -P tegra
... each once after deleting the buildman result/work directory, and once
"incrementally" after a previous identical invocation.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org> # v1
Tested-by: Simon Glass <sjg@chromium.org> # v1
Acked-by: Simon Glass <sjg@chromium.org>
This option outputs to the log file, not to the terminal. Clarify that in
the help, and add a mention of it in the README.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
At present buildman allows you to specify the directory containing the
toolchain, but not the actual toolchain prefix. If there are multiple
toolchains in a single directory, this can be inconvenient.
Add a new 'toolchain-prefix' setting to the settings file, which allows
the full prefix (or path to the C compiler) to be specified.
Update the documentation to match.
Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
At present if you try to use buildman with the branch 'test' it will
complain that it is unsure whether you mean the branch or the directory.
This is a feature of the 'git log' command that buildman uses. Fix it
by resolving the ambiguity.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
At present the architecture is deduced from the toolchain filename. Allow it
to be specified by the caller.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com
At present the priority of a toolchain is calculated from its filename based
on hard-coded rules. Allow it to be specified by the caller. We will use
this in a later patch. Also display the priority and provide a message when
it is overriden by another toolchain of higher priority.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Normally we use a single quote for strings unless there is a reason not to
(such as an embedded single quote). Fix a few counter-examples in this file.
Also add a missing function-argument comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
It is convenient to install symlinks to buildman and patman in the search
patch, such as /usr/local/bin. But when this is done, the -H option fails to
work because it looks in the directory containing the symlink instead of its
target. Fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Since commit 87da2690ab
"openrisc: updating build tools naming convention", openrisc
kernel.org toolchain is out of date and cannot build U-Boot.
Update buildman and moveconfig tools to refer to the new one.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Add links for toolchains not available on kernel.org.
The sh4 toolchains from kernel.org dose not work for some boards,
so use the sh from Sourcery.
Signed-off-by: Thomas Chou <thomas@wytron.com.tw>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
At present buildman can compare configurations between commits but the
feature is less useful than it could be. There is no summary by architecture
and changes are not reported on a per-board basis.
Correct these deficiencies so that it is possible to see exactly what is
changing for any number of boards.
Note that 'buildman -b <branch> -C' is recommended for any build where you
will be comparing configuration. Without -C the correct configuration will
not be reported since changes will often not be picked up.
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Corresponds to ba71a0e (Fix _parse_block() 'parent' documentation re.
ifs.) from upstream, just adding the SPDX tag.
Has performance improvements, code cleanup, Python 3 support, and various
small fixes, including the following:
- Unset user values when loading a zero-byte .config. (5e54e2c)
- Ignore indented .config assignments. (f8a7510)
- Do not require $srctree to be set for non-kernel projects. (d56e9c1)
- Report correct locations in the presence of continuation lines.
(0cebc87)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
The doc wrongly put sandbox in the '--fetch-arch' command. Remove it.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
On i.MX platforms the SPL binary is called "SPL" so make sure we keep
that.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
This commit imports some updates of kconfiglib.py from
https://github.com/ulfalizer/Kconfiglib
- Warn about and ignore the "allnoconfig_y" Kconfig option
- Statements in choices inherit menu/if deps
- Add Symbol.is_allnoconfig_y()
- Hint that modules are still supported despite warnings.
- Add warning related to get_defconfig_filename().
- Fix typo in docs.
- Allow digits in $-references to symbols.
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Philip Craig <philipjcraig@gmail.com>
Signed-off-by: Jakub Sitnicki <jsitnicki@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Also read gcc 4.9.0 at kernel.org which also have Microblaze toolchain.
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Fixed unit test failure by updating the test:
Signed-off-by: Simon Glass <sjg@chromium.org>
The help text for -V says we will pass V=1 but all it really did was not
pass in -s. Change the logic to pass make V=1 with given to buildman -V or
-s to make otherwise.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
When told to keep outputs, be much more liberal in what files we keep.
In addition to adding 'MLO', keep anything that matches u-boot-spl.* (so
that we keep the map file as well) and anything we generate about
'u-boot itself. A large number of bootable formats now match this and
thus it's easier to build many targets and then boot them afterwards
using buildman.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
It is useful to be able to see CONFIG changes made by commits. Add this
feature to buildman using the -K flag so that all CONFIG changes are
reported.
The CONFIG options exist in a number of files. Each is reported
individually as well as a summary that covers all files. The output
shows three parts: green for additions, red for removals and yellow for
changes.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman tries to detect an aborted build and doesn't record a
result in that case. This is to make sure that an abort (e.g. with Ctrl-C)
does not mark the build as done. Without this option, buildman would never
retry the build unless -f/-F are provided. The effect is that aborting the
build creates 'fake errors' on whatever builds buildman happens to be
working on at the time.
Unfortunately the current test is not reliable and this detection can
trigger if a required toolchain tool is missing. In this case the toolchain
problem is never reported.
Adjust the logic to continue processing the build result, mark the build as
done (and failed), but with a return code which indicates that it should be
retried.
The correct fix is to fully and correctly detect an aborted build, quit
buildman immediately and not write any partial build results in this case.
Unfortunately this is currently beyond my powers and is left as an exercise
for the reader (and patches are welcome).
Signed-off-by: Simon Glass <sjg@chromium.org>
In accordance with our other modules supported by U-Boot and as agreed
upon for Apalis/Colibri T30 get rid of the carrier board in the board/
configuration/device-tree naming.
While at it also bring the prompt more in line with our other products.
Signed-off-by: Marcel Ziswiler <marcel@ziswiler.com>
Signed-off-by: Tom Warren <twarren@nvidia.com>
Tweak the output slightly so we don't get things like:
- board1 board2+ board3 board4
There should be a space before the '+'.
Signed-off-by: Simon Glass <sjg@chromium.org>
Commit d908898 updated the ScanPath() function but not its documentation
and not all its callers.
This breaks the toolchain check after it is downloaded. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Heiko Schocher <hs@denx.de>
When buildman scans a toolchain path, it stops at the
first toolchain found. However, a single path can contains
several toolchains, each with its own prefix.
This patch lets buildman scan all toolchains in the path.
Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Acked-by: Simon Glass <sjg@chromium.org>
Normally buildman runs with 'make -s' meaning that only errors and warnings
appear in the log file. Add a -V option to run make in verbose mode, and
with V=1, causing a full build log to be created.
Signed-off-by: Simon Glass <sjg@chromium.org>
The site at https://www.kernel.org/pub/tools/crosstool/ is a convenient
repository of toolchains which can be used for U-Boot. Add a feature to
download and install a toolchain for a selected architecture automatically.
It isn't clear how long this site will stay in the current place and
format, but we should be able to rely on bug reports if it changes.
Suggested-by: Marek Vašut <marex@denx.de>
Suggested-by: Fabio Estevam <festevam@gmail.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
We should create a test setting file when running testes, not use whatever
happens to be on the local machine.
Signed-off-by: Simon Glass <sjg@chromium.org>
Since we need a few modules which might not be available in a bare-bones
distribution, add a note about that to the README.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
