Commit graph

149 commits

Author SHA1 Message Date
Tom Rini
546a6f3a9b buildman: Allow 'gnueabihf' toolchains for ARM
Many toolchains for ARM use the 'gnueabihf' suffix rather than just
'gnueabi', so allow these to be used, but with a lower priority than
'gnueabi' ones.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-04-18 10:29:19 -04:00
Tom Rini
fbeb337529 buildman: Translate more strings to latin-1
When writing out some of our results we may now have UTF-8 characters
in there as well.  Translate these to latin-1 and ignore any errors (as
this is for diagnostic and given the githash anything else can be
reconstructed by the user.

Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
2017-04-14 12:21:48 -04:00
Simon Glass
9a6d2e2a6b buildman: Handle commit subjects containing unicode
One of these has crept in in this commit:

40a808f1 ARCv2: SLC: Make sure busy bit is set properly on SLC flushing

Adjust buildman to handle it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-04-13 14:44:49 -06:00
Tom Rini
d036107a1f kconfiglib.py: Kludge in 'imply' support
Currently upstream does not yet understand the imply keyword.  For what
we use kconfiglib.py for today, this is OK.  We only need to be able to
evaluate in order to make boards.cfg and none of those choices will
depend on how imply evaluates out.

Signed-off-by: Tom Rini <trini@konsulko.com>
2017-03-09 19:52:14 -05:00
Simon Glass
8d7523c55c buildman: Allow showing the list of boards with -n
As well as showing the number of boards, allow showing the actual list of
boards that would be built, if -v is provided.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-02-08 06:12:16 -07:00
Alexey Brodkin
a55bed1208 buildman: Update link to the most recent prebuilt ARC toolachin
To troubleshoot unexpected bhavior during building and what's more
important during execution it is strongly recommended to use recent
ARC toolchain, and so we're now referring to arc-2016.09 which is the
latest as of today.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-01-28 14:04:50 -05:00
Simon Glass
a9401b2bc9 buildman: Rename do_build to config_only
This variable name is needlessly confusion. Adjust it to use a 'positive'
name instead.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02 20:53:18 -07:00
Simon Glass
960421ecb3 buildman: Clean up odd characters on the terminal
At present buildman leaves behind a few characters during its progress
updates, which looks odd. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02 10:37:47 -07:00
Simon Glass
b464f8e7de buildman: Squash useless output from -K
When using #define CONFIG_SOME_OPTION, the value it set to '1'. When using
defconfig (i.e. CONFIG_SOME_OPTION=y) the value is set to 'y'. This results
in differences showing up with -K. These differences are seldom useful.

Adjust buildman to suppress these differences by default.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02 10:37:47 -07:00
Simon Glass
94d2ebe5bc buildman: Add documentation for CONFIG checking
The -K option is not mentioned in the README at present. Add some notes
to describe how this is used.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02 10:37:47 -07:00
Simon Glass
b50113f373 buildman: Add an option to just create the config
Normally buildman does a full build of a board. This includes creating the
u-boot.cfg file which contains all the configuration options. Buildman uses
this file with the -K option, to show differences in effective configuration
for each commit.

Doing a full build of U-Boot just to create the u-boot.cfg file is wasteful.
Add a -D option which causes buildman to only create the configuration. This
is enough to support use of -K and can be done much more quickly (typically
5-10 times faster).

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-12-02 10:37:47 -07:00
Tom Rini
e0f2406e73 buildman: Fix building based on 'options' field
The README for buildman says that we can use any field in boards.cfg to
decide what to build.  However, we were not saving the options field
correctly.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2016-11-06 07:33:41 -05:00
York Sun
d5fe013cee tools: buildman: Add compiler wrapper
Now we can use compiler wrapper such as ccache or distcc for buildman.

Signed-off-by: York Sun <york.sun@nxp.com>
CC: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
York Sun
f40fa9b36f tools: buildman: Remove duplicated code
Signed-off-by: York Sun <york.sun@nxp.com>
CC: Simon Glass <sjg@chromium.org>
Fixed commit subject:
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
2880e6b5e2 buildman: Drop the 'alive' flag in BuilderThread
This is not used, so drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
2f2566482f buildman: Don't show a stacktrace on Ctrl-C
When Ctrl-C is pressed, just exited quietly. There is no sense in displaying
a stack trace since buildman will always be in the same place: waiting for
threads to complete building all the jobs on the queue.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
63781bd65e buildman: Drop the 'active' flag in the builder
This serves no real purpose, since when we are not active, we exit. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
d436e38189 buildman: Allow builds to terminate cleanly
It is annoying that buildman does not respond cleanly to Ctrl-C or SIGINT,
particularly on machines with lots of CPUS. Unfortunately queue.join()
blocks the main thread and does not allow it to see the signal. Use a
separate thread instead,

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
a556eeebaa buildman: Put our local libraries first in the path
If patman is installed on the machine (e.g. in the standard dist-packages
directory), it will find libraries from there in preference to our local
libraries. Adjust the order of the path to ensure that local libraries are
found first.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
745b395aef buildman: Print a message indicating the build is starting
Make it clear when buildman actually starts building. This happens when it
has prepared the threads, working directory and output directories.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
b222abe736 buildman: Print a message when removing old directories
When buildman starts, it prepares its output directory by removing any old
build directories which will not be used this time. This can happen if a
previous build left directories around for commit hashes which are no-longer
part of the branch.

This can take quite a while, so print a message to indicate what is going
on.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
21f0eb332f buildman: Tidy up the 'cloning' message
On a machine with a lot of CPUs this prints a lot of useless lines of the
form:

   Cloning repo for thread <n>

Adjust the output so that these all appear on one line, and disappear when
the cloning is complete.

Note: This cloning is actually unnecessary and very wasteful on disk space
(about 3.5GB each time). It would be better to create symlinks.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-10-09 09:30:32 -06:00
Simon Glass
785f1548a9 patman: Adjust command.Output() to raise an error by default
It is more useful to have this method raise an error when something goes
wrong. Make this the default and adjust the few callers that don't want to
use it this way.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-09-18 21:04:38 -06:00
Simon Glass
c8d7393b73 buildman: Add a quick-start note
For those who just want to build a board, it is useful to see a quick hint
right at the start of the documentation. Add a few commands showing how to
download toolchains and build a board.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
c8785c5b49 buildman: Avoid overwriting existing toolchain entries
The current code for setting up the toolchain config always writes the new
paths to an item called 'toolchain'. This means that it will overwrite any
existing toolchain item with the same name. In practice, this means that:

   buildman --fetch-arch all

will fetch all toolchains, but only the path of the final one will be added
to the config. This normally works out OK, since most toolchains are the
same version (e.g. gcc 4.9) and will be found on the same path. But it is
not correct and toolchains for archs which don't use the same version will
not function as expected.

Adjust the code to use a complete glob of the toolchain path.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
7e92e46e63 buildman: Drop the toolchain error when downloading toolchains
It doesn't make sense to complain about missing toolchains when the
--fetch-arch option is being used. The user is presumably aware that there
is a toolchain problem and is actively correcting it by running with this
option.

Refactor the code to avoid printing this confusing message.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
2289b2763c buildman: Fix a typo in TestSettingsHasPath()
The function comment should say 'buildman'. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
713bea38dd buildman: Improve the toolchain progress/error output
Use colour to make it easier to see what is going on. Also print a message
before downloading a new toolchain. Mention --fetch-arch in the message that
is shown when there are no available toolchains, since this is the quickest
way to resolve the problem.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
80e6a48750 buildman: Allow the toolchain error to be suppressed
When there are no toolchains a warning is printed. But in some cases this is
confusing, such as when the user is fetching new toolchains.

Adjust the function to supress the warning in this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
bd6f5d98de buildman: Fix the 'help' test to use the correct path
When buildman is run via a symlink, this test fails. Fix it to work the same
way as buildman itself.

Signed-off-by: Simon Glass <sjg@chromium.org>
2016-07-31 19:37:08 -06:00
Simon Glass
8e605a5e3e buildman: Automatically create a config file if needed
If there is no ~/.buildman file, buildman currently complains and exists. To
make things a little more friendly, create an empty one automatically. This
will not allow things to be built, but --fetch-arch can be used to handle
that.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Simon Glass
8ea42101d2 buildman: Tidy up the README a little
Tidy up some problems found by a recent review.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
2016-07-31 19:37:08 -06:00
Stephen Warren
f79f1e0c0e buildman: allow more incremental building
One use-case for buildman is to continually run it interactively after
each small step in a large refactoring operation. This gives more
immediate feedback than making a number of commits and then going back and
testing them. For this to work well, buildman needs to be extremely fast.
At present, a couple issues prevent it being as fast as it could be:

1) Each time buildman runs "make %_defconfig", it runs "make mrproper"
first. This throws away all previous build results, requiring a
from-scratch build. Optionally avoiding this would speed up the build, at
the cost of potentially causing or missing some build issues.

2) A build tree is created per thread rather than per board. When a thread
switches between building different boards, this often causes many files
to be rebuilt due to changing config options. Using a separate build tree
for each board would avoid this. This does put more strain on the system's
disk cache, but it is worth it on my system at least.

This commit adds two command-line options to implement the changes
described above; -I ("--incremental") turns of "make mrproper" and -P
("--per-board-out-dir") creats a build directory per board rather than per
thread.

Tested:

    ./tools/buildman/buildman.py tegra
    ./tools/buildman/buildman.py -I -P tegra
    ./tools/buildman/buildman.py -b tegra_dev tegra
    ./tools/buildman/buildman.py -b tegra_dev -I -P tegra

... each once after deleting the buildman result/work directory, and once
"incrementally" after a previous identical invocation.

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org> # v1
Tested-by: Simon Glass <sjg@chromium.org> # v1
Acked-by: Simon Glass <sjg@chromium.org>
2016-05-17 09:54:43 -06:00
Simon Glass
c81d0d215e buildman: Clarify the use of -V
This option outputs to the log file, not to the terminal. Clarify that in
the help, and add a mention of it in the README.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Stephen Warren <swarren@nvidia.com>
2016-03-17 21:27:39 -06:00
Simon Glass
17bce66cbf buildman: Add a way to specific a full toolchain prefix
At present buildman allows you to specify the directory containing the
toolchain, but not the actual toolchain prefix. If there are multiple
toolchains in a single directory, this can be inconvenient.

Add a new 'toolchain-prefix' setting to the settings file, which allows
the full prefix (or path to the C compiler) to be specified.

Update the documentation to match.

Suggested-by: Stephen Warren <swarren@wwwdotorg.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stephen Warren <swarren@nvidia.com>
2016-03-17 21:27:39 -06:00
Simon Glass
d4c8572b71 buildman: Allow branch names which conflict with directories
At present if you try to use buildman with the branch 'test' it will
complain that it is unsure whether you mean the branch or the directory.
This is a feature of the 'git log' command that buildman uses. Fix it
by resolving the ambiguity.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-17 21:27:39 -06:00
Simon Glass
608e399fde buildman: Allow the toolchain architecture to be specified
At present the architecture is deduced from the toolchain filename. Allow it
to be specified by the caller.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com
2016-03-14 15:34:50 -06:00
Simon Glass
ff690df9fc buildman: Allow the toolchain priority to be specified
At present the priority of a toolchain is calculated from its filename based
on hard-coded rules. Allow it to be specified by the caller. We will use
this in a later patch. Also display the priority and provide a message when
it is overriden by another toolchain of higher priority.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Simon Glass
ad24ebace1 buildman: Fix up a few code inconsistencies in toolchain.py
Normally we use a single quote for strings unless there is a reason not to
(such as an embedded single quote). Fix a few counter-examples in this file.
Also add a missing function-argument comment.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Simon Glass
2bdeade0ea buildman: patman: Fix -H when installed as a symlink
It is convenient to install symlinks to buildman and patman in the search
patch, such as /usr/local/bin. But when this is done, the -H option fails to
work because it looks in the directory containing the symlink instead of its
target. Fix this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
2016-03-14 15:34:50 -06:00
Bin Meng
e8aebc4744 tools: Update openrisc toolchain information
Since commit 87da2690ab
"openrisc: updating build tools naming convention", openrisc
kernel.org toolchain is out of date and cannot build U-Boot.
Update buildman and moveconfig tools to refer to the new one.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
2016-02-24 18:44:11 -05:00
Thomas Chou
8951523c6a buildman: README: add links for toolchains not available on kernel.org
Add links for toolchains not available on kernel.org.

The sh4 toolchains from kernel.org dose not work for some boards,
so use the sh from Sourcery.

Signed-off-by: Thomas Chou <thomas@wytron.com.tw>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2015-12-10 20:34:23 -07:00
Simon Glass
8270e3c12e buildman: Improve the config comparison feature
At present buildman can compare configurations between commits but the
feature is less useful than it could be. There is no summary by architecture
and changes are not reported on a per-board basis.

Correct these deficiencies so that it is possible to see exactly what is
changing for any number of boards.

Note that 'buildman -b <branch> -C' is recommended for any build where you
will be comparing configuration. Without -C the correct configuration will
not be reported since changes will often not be picked up.

Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-09-09 07:48:03 -06:00
Ulf Magnusson
90c36d8ab9 kconfiglib: update to the latest version
Corresponds to ba71a0e (Fix _parse_block() 'parent' documentation re.
ifs.) from upstream, just adding the SPDX tag.

Has performance improvements, code cleanup, Python 3 support, and various
small fixes, including the following:

  - Unset user values when loading a zero-byte .config. (5e54e2c)
  - Ignore indented .config assignments. (f8a7510)
  - Do not require $srctree to be set for non-kernel projects. (d56e9c1)
  - Report correct locations in the presence of continuation lines.
    (0cebc87)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
2015-08-28 12:33:13 -04:00
Bin Meng
9f244b27cf buildman: Correct '--fetch-arch' command documentation
The doc wrongly put sandbox in the '--fetch-arch' command. Remove it.

Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-07-28 10:36:22 -06:00
Tom Rini
dd59211046 builderthread.py: Keep 'SPL'
On i.MX platforms the SPL binary is called "SPL" so make sure we keep
that.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-07-14 18:03:15 -06:00
Masahiro Yamada
9d01b7872e kconfiglib: sync with the latest in Kconfiglib project
This commit imports some updates of kconfiglib.py from
https://github.com/ulfalizer/Kconfiglib

 - Warn about and ignore the "allnoconfig_y" Kconfig option
 - Statements in choices inherit menu/if deps
 - Add Symbol.is_allnoconfig_y()
 - Hint that modules are still supported despite warnings.
 - Add warning related to get_defconfig_filename().
 - Fix typo in docs.
 - Allow digits in $-references to symbols.

Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Philip Craig <philipjcraig@gmail.com>
Signed-off-by: Jakub Sitnicki <jsitnicki@gmail.com>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2015-06-12 12:43:06 -04:00
Michal Simek
1246231c48 buildman: Add gcc 4.9.0 with Microblaze toolchain
Also read gcc 4.9.0 at kernel.org which also have Microblaze toolchain.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>

Fixed unit test failure by updating the test:
Signed-off-by: Simon Glass <sjg@chromium.org>
2015-04-23 12:31:54 -06:00
Tom Rini
f5e5ece0b7 buildman: Make -V (verbose_build) really be verbose
The help text for -V says we will pass V=1 but all it really did was not
pass in -s.  Change the logic to pass make V=1 with given to buildman -V or
-s to make otherwise.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:25 -06:00
Tom Rini
0eb4c04503 buildman: Keep more outputs with the --keep-outputs flag
When told to keep outputs, be much more liberal in what files we keep.
In addition to adding 'MLO', keep anything that matches u-boot-spl.* (so
that we keep the map file as well) and anything we generate about
'u-boot itself.  A large number of bootable formats now match this and
thus it's easier to build many targets and then boot them afterwards
using buildman.

Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Acked-by: Simon Glass <sjg@chromium.org>
2015-04-18 16:24:25 -06:00