Commit graph

104 commits

Author SHA1 Message Date
Simon Glass
823e60b62a buildman: Allow tests to have their own boards
Rather than reading boards.cfg, which may take time to generate and is not
necessarily suitable for running tests, create our own list of boards.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:29 -06:00
Simon Glass
8b985eebd0 buildman: Avoid looking at config file or toolchains in tests
These files may not exist in the environment, or may not be suitable for
testing. Provide our own config file and our own toolchains when running
tests.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:28 -06:00
Simon Glass
fd03d63f34 buildman: Set up bsettings outside the control module
Move the bsettings code back to the main buildman.py file, so we can do
something different when testing.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:28 -06:00
Simon Glass
d4144e45b4 buildman: Add a functional test
Buildman currently lacks testing in many areas, including its use of git,
make and many command-line flags.

Add a functional test which covers some of these areas. So far it does
a fake 'build' of all boards for the current source tree.

This version reads the real ~/.buildman and boards.cfg files. Future work
will improve this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:28 -06:00
Simon Glass
48ba5856eb buildman: Move full help code into the control module
There is no good reason to keep this code separate. Move it into control.py
so it is easier to test.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:27 -06:00
Simon Glass
d3d5c12331 buildman: Move the command line code into its own file
We want to be able to issue parser commands from within buildman for test
purposes. Move the parser code into its own file so we don't end up needing
the buildman and test modules to reference each other.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:27 -06:00
Simon Glass
6208fcef94 buildman: Enhance basic test to check summary output
Adjust the basic test so that it checks all console output. This will help
to ensure that the builder is behaving correctly with printing summary
information.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 13:54:23 -06:00
Simon Glass
4653a8826f buildman: Send builder output through a function for testing
To allow us to verify the builder's console output, send it through a
function which can collect it when running in test mode.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 13:54:23 -06:00
Simon Glass
e30965db0c buildman: Separate out display of warnings and errors
Some boards unfortunately build with warnings and it is useful to be able
to easily distinguish the warnings from the errors.

Use a simple pattern match to categorise gcc output into warnings and
errors, and display each separately. New warnings are shown in magenta (with
a w+ prefix) and fixed warnings are shown in yellow with a w- prefix.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
ed9666573e buildman: Add an option to show which boards caused which errors
Add a -l option to display a list of offending boards against each
error/warning line. The information will be shown in brackets as below:

02: wip
   sandbox: +   sandbox
       arm: +   seaboard
+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
+(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]
+(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc':
+(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable]
+(seaboard)      int fred;
+(seaboard)          ^

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
48c1b6a8ff buildman: Remove the directory prefix from each error line
The full path is long and also includes buildman private directories.
Clean this up, so that only a relative U-Boot path is shown.

This will change warnings like these:

/home/sjg/c/src/third_party/u-boot/buildman5/.bm-work/00/arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
/home/sjg/c/src/third_party/u-boot/buildman5/.bm-work/00/arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]

/home/sjg/c/src/third_party/u-boot/files/arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
/home/sjg/c/src/third_party/u-boot/files/arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]

to:

arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
3cf4ae6f86 buildman: Implement an option to exclude boards from the build
Some boards are known to be broken and it is convenient to be able to
exclude them from the build.

Add an --exclude option to specific boards to exclude. This uses the
same matching rules as the normal 'include' arguments, and is a comma-
separated list of regular expressions.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
f60c9d4f39 buildman: Allow make-flags variables to include '-' and '_'
These characters are commonly used in variables, so permit them. Also
document the permitted characters.

Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
2c3deb9758 buildman: Set the return code to indicate build result
When buildman finds errors/warnings when building, set the return code to
indicate this.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Masahiro Yamada
390f7035d3 buildman: run genboardscfg.py all the time
This commit makes sure boards.cfg is up to date before starting
the build tests.  tools/genboardscfg.py exits immediately printing
"boards.cfg is up to date. Nothing to do." when boards.cfg is
already new.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
7f6b8315d1 buildman: refactor help message
"buildman [options]" is displayed by default.

Append the rest of help messages to parser.usage
instead of replacing it.

Besides, "-b <branch>" is not mandatory since commit fea5858e.
Drop it from the usage.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-22 10:42:36 -04:00
Roger Meier
fd18a89e7f Makefile: remove generated boards.cfg within make distclean
Signed-off-by: Roger Meier <roger@bufferoverflow.ch>
Reviewed-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Tested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Tom Rini <trini@ti.com>
Cc: Wolfgang Denk <wd@denx.de>
Cc: Simon Glass <sjg@chromium.org>
2014-08-22 10:41:35 -04:00
Masahiro Yamada
31e2141d5a tools, scripts: refactor error-out statements of Python scripts
In Python, sys.exit() function can also take an object other
than an integer.

If an integer is given to the argument, Python exits with the return
code of it.  If a non-integer argument is given, Python outputs it
to stderr and exits with the return code of 1.

That means,

    print >> sys.stderr, "Blah Blah"
    sys.exit(1)

is equivalent to

    sys.exit("Blah Blah")

The latter is a useful shorthand.

Note:
Some error messages in Buildman and Patman were output to stdout.
But they should go to stderr.  They are also fixed by this commit.
This is a nice side effect.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-21 12:01:11 -04:00
Simon Glass
3b74ba5f1a buildman: Allow selection of the number of commits to build
It is useful to be able to build only some of the commits in a branch. Add
support for the -c option to allow this. It was previously parsed by
buildman but not implemented.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: York Sun <yorksun@freescale.com>
2014-08-13 08:34:16 -06:00
Simon Glass
6131beab69 buildman: Introduce an 'and' operator for board selection
Currently buildman allows a list of boards to build to be specified on the
command line. The list can include specific board names, architecture, SOC
and so on.

At present the list of boards is dealt with in an 'OR' fashion, and there
is no way to specify something like 'arm & freescale', meaning boards with
ARM architecture but only those made by Freescale. This would exclude the
PowerPC boards made by Freescale.

Support an '&' operator on the command line to permit this. Ensure that
arguments can be specified in a single string to permit easy shell quoting.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: York Sun <yorksun@freescale.com>
2014-08-13 08:34:16 -06:00
Simon Glass
e956947858 buildman: Add a few more toolchain examples to the README
The current README is a bit sparse in this area, so add a few more
examples.

Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:16 -06:00
Simon Glass
28370c1b99 buildman: Add a message indicating there are no errors
If buildman finds no problems it prints nothing. This can be a bit confusing,
so add a message that all is well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
0f7c9ddaed buildman: Add an option to specify the buildman config file
Add a new --config-file option (-G) to specify a different configuration
file from the default ~/.buildman.

Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
d3269ed380 buildman: Remove unused non-incremental build method code
The non-incremental build method is no longer used, so remove it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
e5a0e5d842 buildman: Add verbose option to display errors as they happen
Normally buildman operates in two passes - one to do the build and another
to summarise the errors. Add a verbose option (-v) to display build problems
as they happen. With -e also given, this will display errors too.

When building the current source tree (rather than a list of commits in a
branch), both -v and -e are enabled automatically.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
b2ea7ab252 buildman: Refactor output options
We need the output options to be available in several places. It's a pain
to pass them into each function. Make them properties of the builder and
add a single function to set them up. At the same time, add a function which
produces summary output using these options.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
c1de501492 buildman: Sort command line options
These options have got slightly out of order. Fix them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
190064b4da buildman: Move BuilderThread code to its own file
The builder.py file is getting too long, so split out some code.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:14 -06:00
Simon Glass
fea5858eb9 buildman: Allow building of current source tree
Originally buildman had some support for building the current source tree.
However this was dropped before it was submitted, as part of the effort to
make it faster when building entire branches.

Reinstate this support. If no -b option is given, buildman will build the
current source tree.

Reported-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:14 -06:00
Simon Glass
6eede34ce6 buildman: Add some notes about moving from MAKEALL
For those used to MAKEALL, buildman seems strange. Add some notes to ease
the transition.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:14 -06:00
Simon Glass
cec83c3e63 buildman: Fix a few typos
There are several typos in the README - fix them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:14 -06:00
Masahiro Yamada
73f30b9b80 buildman: adjust for Kconfig
Use "make <board>_defconfig" instead of "make <board>_config".

Invoke tools/genboardscfg.py to generate boards.cfg when it is missing.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-07-30 14:42:02 -04:00
Masahiro Yamada
9979692383 buildman: make sure to invoke GNU Make
Since the command name 'make' may not be GNU Make on some platforms
such as FreeBSD, buildman should call scripts/show-gnu-make to get
the command name for GNU MAKE (and error out if it is not found).

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Jeroen Hofstee <jeroen@myspectrum.nl>
2014-07-29 11:48:40 -04:00
Simon Glass
189a496825 buildman: Support in-tree builds
At present buildman always builds out-of-tree, that is it uses a separate
output directory from the source directory. Normally this is what you want,
but it is important that in-tree builds work also. Some Makefile changes may
break this.

Add a -i option to tell buildman to use in-tree builds, so that it is easy
to test this feature.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-07-28 04:52:55 +01:00
Simon Glass
97e915262e buildman: Add -C option to force a reconfigure for each commit
Normally buildman wil try to configure U-Boot for a particular board on the
first commit that it builds in a series. Subsequent commits are built
without reconfiguring which normally works. Where it doesn't, buildman
automatically reconfigures and retries.

To fully emulate the way MAKEALL works, we should have an option to disable
this optimisation.

Add a -C option to cause buildman to always reconfigure on each commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-07-28 04:52:48 +01:00
Simon Glass
21fe8ec355 buildman: Avoid retrying a build if it definitely failed
After a build fails buildman will reconfigure and try again, if it did not
reconfigure before the build. However it doesn't actually keep track of
whether it did reconfigure on the previous attempt.

Fix that logic to avoid a pointless rebuild. This speeds things up quite a
bit for failing builds. Previously they would always be built twice.

Change-Id: Ib37f21320baa7c60bed98f4042c0b7ed7c0dc85e
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-07-23 09:52:36 -06:00
Simon Glass
4266dc2882 buildman: Add -F flag to retry failed builds
Generally a build failure with a particular commit cannot be fixed except
by changing that commit. Changing the commit will automatically cause
buildman to retry when you run it again: buildman sees that the commit
hash is different and that it has no previous build result for the new
commit hash.

However sometimes the build failure is due to a toolchain issue or some
other environment problem. In that case, retrying failed builds may yield
a different result.

Add a flag to retry failed builds. This differs from the force rebuild
flag (-f) in that it will not rebuild commits which are already marked as
succeeded.

Series-to: u-boot

Change-Id: Iac4306df499d65ff0888b1c60f06fc162a6faad8
2014-07-23 09:52:33 -06:00
Masahiro Yamada
8708267f09 buildman: fix toolchain priority_list
'-elf' appears twice in the toolchain priority_list.
The second one is rudundant.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2014-07-07 17:22:54 -06:00
Masahiro Yamada
1826a18d8d buildman: fix to display warning message for missing [toolchain] section
Toolchains.__init__ is expected to display a warning message
when the [toolchain] section is missing from ~/.buildman file.
But it never works.
In that case, instead, buildmain fails with an error message
which is difficult to understand:

  Traceback (most recent call last):
    File "tools/buildman/buildman", line 126, in <module>
      control.DoBuildman(options, args)
    File "/home/foo/u-boot/tools/buildman/control.py", line 78, in DoBuildman
      toolchains = toolchain.Toolchains()
    File "/home/foo/u-boot/tools/buildman/toolchain.py", line 106, in __init__
    config_fname)
  NameError: global name 'config_fname' is not defined

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2014-07-07 17:22:48 -06:00
Daniel Schwierzeck
e0ba929982 buildman: make output dir configurable
Add an option to specify the output directory to override the
default path '../'. This is useful for building in a ramdisk.

Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-04-18 16:14:13 -04:00
Stephen Warren
8426d8b089 buildman: make board selector argument a regex
A common use-case is to build all boards for a particular SoC. This can
be achieved by:

./tools/buildman/buildman -b mainline_dev tegra20

However, when the SoC is a member of a family of SoCs, and each SoC has
a different name, it would be even more useful to build all boards for
every SoC in that family. This currently isn't possible since buildman's
board selection command-line arguments are compared to board definitions
using pure string equality.

To enable this, compare using a regex match instead. This matches
MAKEALL's handling of command-line arguments. This enables:

(all Tegra)
./tools/buildman/buildman -b mainline_dev tegra

(all Tegra)
./tools/buildman/buildman -b mainline_dev '^tegra.*$'

(all Tegra20, Tegra30 boards, but not Tegra114)
./tools/buildman/buildman -b mainline_dev 'tegra[23]'

Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
2013-11-21 13:35:58 -07:00
Andreas Bießmann
61242ac5f9 buildman: fix README
This is a trivial fix for c'n'p error.

Signed-off-by: Andreas Bießmann <andreas.devel@googlemail.com>
Acked-by: Simon Glass <sjg@chromium.org>
2013-11-21 13:35:58 -07:00
Jagannadha Sutradharudu Teki
a707b3db56 buildman: Use env to pick the python from $PATH
python used in buildman doesn't need to be placed in
/usr/bin/python, So use env to ensure that the interpreter
will pick the python from environment.

Usefull with several versions of python's installed on system.

Signed-off-by: Jagannadha Sutradharudu Teki <jaganna@xilinx.com>
Acked-by: Simon Glass <sjg@chromium.org>
2013-10-15 08:44:28 -04:00
Tom Rini
9cc18042e6 Merge branch 'buildman' of git://git.denx.de/u-boot-x86 2013-10-14 20:19:56 -04:00
Wolfgang Denk
16641d52fc Coding Style cleanup: drop some excessive empty lines
Signed-off-by: Wolfgang Denk <wd@denx.de>
2013-10-14 16:06:54 -04:00
Stephen Warren
8bb2bddc2c buildman: don't fail --list-toolchains when toolchains fail
When a toolchain invocation fails, an exception is thrown but not caught
which then aborts the entire toolchain detection process. To solve this,
request that exceptions not be thrown, since the toolchain init code
already error-checks the command result. This solves e.g.:

         - found '/usr/bin/winegcc'
Traceback (most recent call last):
...
Exception: Error running '/usr/bin/winegcc --version'

Change-Id: I579c72ab3b021e38b14132893c3375ea257c74f0
Signed-off-by: Stephen Warren <swarren@nvidia.com>
Acked-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
(formatted to 80cols)
2013-10-10 10:40:42 -06:00
Simon Glass
4281ad8e7f buildman: Allow make flags to be specified for each board
There are a few make options such as BUILD_TAG which can be provided when
building U-Boot. Provide a way for buildman to pass these flags to make
also.

The flags should be in a [make-flags] section and arranged by target name
(the 'target' column in boards.cfg. See the README for more details.

Signed-off-by: Simon Glass <sjg@chromium.org>
2013-10-01 14:39:14 -06:00
Simon Glass
e19d5781ec buildman: Adjust tests for new boards.cfg format
Commit 27af930e9a changed the boards.cfg format
but missed to change the parsing in buildman. A follow-on commit
03c1bb2425 fixed this but missed fixing the
tests.

This patch updates the tests to fit the new Board constructor.

./tools/buildman/buildman -t
<unittest.result.TestResult run=1 errors=0 failures=0>

Signed-off-by: Simon Glass <sjg@chromium.org>
2013-10-01 14:39:06 -06:00
Andreas Bießmann
03c1bb2425 buildman: fix boards.cfg parsing
Commit 27af930e9a changed the boards.cfg format
but missed to change the parsing in buildman.

This patch changes c'tor of Board class to the new sequence, but omits
maintainer field.

Signed-off-by: Andreas Bießmann <andreas.devel@googlemail.com>
2013-09-20 10:30:54 -04:00
Albert ARIBAUD
27af930e9a Merge and reformat boards.cfg and MAINTAINERS
Put all informations about targets, including state (active or
orphan) and maintainers, in boards.cfg; remove MAINTAINERS;
adjust the build system accordingly.

Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
2013-09-12 09:14:37 -04:00
Wolfgang Denk
1a4596601f Add GPL-2.0+ SPDX-License-Identifier to source files
Signed-off-by: Wolfgang Denk <wd@denx.de>
[trini: Fixup common/cmd_io.c]
Signed-off-by: Tom Rini <trini@ti.com>
2013-07-24 09:44:38 -04:00
Simon Glass
cce717a96c buildman: Produce a sensible error message when branch is missing
Rather than a backtrace, produce a nice error message when an invalid
branch is provided to buildman.

Signed-off-by: Simon Glass <sjg@chromium.org>
2013-05-14 15:37:58 -04:00
Simon Glass
f0b739f15f buildman: Allow conflicting tags to avoid spurious errors
Conflicting tags can prevent buildman from building two series which exist
one after the other in a branch. There is no reason not to allow this sort
of workflow with buildman, so ignore conflicting tags in buildman.

Change-Id: I2231d04d8684fe0f8fe77f8ea107e5899a3da5e8
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@ti.com>
2013-05-09 14:27:38 -07:00
Simon Glass
fc3fe1c287 buildman - U-Boot multi-threaded builder and summary tool
This tool handles building U-Boot to check that you have not broken it
with your patch series. It can build each individual commit and report
which boards fail on which commits, and which errors come up. It also
shows differences in image sizes due to particular commits.

Buildman aims to make full use of multi-processor machines.

Documentation and caveats are in tools/buildman/README.

Signed-off-by: Simon Glass <sjg@chromium.org>
2013-04-04 14:04:35 -07:00