Commit graph

651 commits

Author SHA1 Message Date
Simon Glass
82012dd284 patman: Provide a way to intercept commands for testing
Add a test point for the command module. This allows tests to emulate
the execution of commands. This provides more control (since we can make
the fake 'commands' do whatever we like), makes it faster to write tests
since we don't need to set up as much environment, and speeds up test
execution.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:27 -06:00
Simon Glass
48ba5856eb buildman: Move full help code into the control module
There is no good reason to keep this code separate. Move it into control.py
so it is easier to test.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:27 -06:00
Simon Glass
d3d5c12331 buildman: Move the command line code into its own file
We want to be able to issue parser commands from within buildman for test
purposes. Move the parser code into its own file so we don't end up needing
the buildman and test modules to reference each other.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:27 -06:00
Simon Glass
ddaf5c8f30 patman: RunPipe() should not pipe stdout/stderr unless asked
RunPipe() currently pipes the output of stdout and stderr to a pty, but
this is not the intended behaviour. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 16:38:24 -06:00
Simon Glass
6208fcef94 buildman: Enhance basic test to check summary output
Adjust the basic test so that it checks all console output. This will help
to ensure that the builder is behaving correctly with printing summary
information.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 13:54:23 -06:00
Simon Glass
4653a8826f buildman: Send builder output through a function for testing
To allow us to verify the builder's console output, send it through a
function which can collect it when running in test mode.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 13:54:23 -06:00
Simon Glass
3c6c0f81bf patman: Add a way of recording terminal output for testing
When running unit tests we don't want output to go to the terminal.
Provide a way of collecting it so that it can be examined by test code
later.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-09 13:54:22 -06:00
Simon Glass
0b703dbcee patman: Fix detection of git version
A missing 'global' declaration means that this feature does not currently
work. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
e30965db0c buildman: Separate out display of warnings and errors
Some boards unfortunately build with warnings and it is useful to be able
to easily distinguish the warnings from the errors.

Use a simple pattern match to categorise gcc output into warnings and
errors, and display each separately. New warnings are shown in magenta (with
a w+ prefix) and fixed warnings are shown in yellow with a w- prefix.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
ed9666573e buildman: Add an option to show which boards caused which errors
Add a -l option to display a list of offending boards against each
error/warning line. The information will be shown in brackets as below:

02: wip
   sandbox: +   sandbox
       arm: +   seaboard
+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
+(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]
+(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc':
+(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable]
+(seaboard)      int fred;
+(seaboard)          ^

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
48c1b6a8ff buildman: Remove the directory prefix from each error line
The full path is long and also includes buildman private directories.
Clean this up, so that only a relative U-Boot path is shown.

This will change warnings like these:

/home/sjg/c/src/third_party/u-boot/buildman5/.bm-work/00/arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
/home/sjg/c/src/third_party/u-boot/buildman5/.bm-work/00/arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]

/home/sjg/c/src/third_party/u-boot/files/arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
/home/sjg/c/src/third_party/u-boot/files/arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]

to:

arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
3cf4ae6f86 buildman: Implement an option to exclude boards from the build
Some boards are known to be broken and it is convenient to be able to
exclude them from the build.

Add an --exclude option to specific boards to exclude. This uses the
same matching rules as the normal 'include' arguments, and is a comma-
separated list of regular expressions.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:43 -06:00
Simon Glass
f60c9d4f39 buildman: Allow make-flags variables to include '-' and '_'
These characters are commonly used in variables, so permit them. Also
document the permitted characters.

Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
2c3deb9758 buildman: Set the return code to indicate build result
When buildman finds errors/warnings when building, set the return code to
indicate this.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
d0c5719d92 patman: Avoid changing the order of tags
patman collects tags that it sees in the commit and places them nicely
sorted at the end of the patch. However, this is not really necessary and
in fact is apparently not desirable.

Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-09-05 13:40:42 -06:00
Simon Glass
9447a6b2f6 patman: Use --no-pager' to stop git from forking a pager
In a headless environment the pager can apparently hang. We don't want a
pager anyway so let's request that none be used.

Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
7428dc14b0 patman: Remove the -a option
It seems that this is no longer needed, since checkpatch.pl will catch
whitespace problems in patches. Also the option is not widely used, so
it seems safe to just remove it.

Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:42 -06:00
Simon Glass
e752edcb6b patman: Correct unit tests to run correctly
It seems that doctest behaves differently now, and some of the unit tests
do not run. Adjust the tests to work correctly.

 ./tools/patman/patman --test
<unittest.result.TestResult run=10 errors=0 failures=0>

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:41 -06:00
Simon Glass
6ba5737ff0 patman: Fix indentation in terminal.py
This code came from a different project with 2-character indentation. Fix
it for U-Boot.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-09-05 13:40:41 -06:00
Masahiro Yamada
2134342e57 tools/genboardscfg.py: change shebang into /usr/bin/env python2
This tool only works on python 2 (python 2.6 or lator).

Change the shebang to make sure the script is run by python 2
and clearly say the supported version in the comment block.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
9a65cb7ffe tools/genboardscfg.py: improve performance
I guess some developers are already getting sick of this tool
because it generally takes a few minites to generate the boards.cfg
on a reasonable computer.

The idea popped up on my mind was to skip Makefiles and
to run script/kconfig/conf directly.
This tool should become about 4 times faster.
You might still not be satisfied, but better than doing nothing.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
d1bf4afda6 tools/genboardscfg.py: check if the boards.cfg is up to date
It looks silly to regenerate the boards.cfg even when it is
already up to date.

The tool should exit with doing nothing if the boards.cfg is newer
than any of defconfig, Kconfig and MAINTAINERS files.

Specify -f (--force) option to get the boards.cfg regenerated
regardless its time stamp.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
79d45d32bc tools/genboardscfg.py: fix minor problems on termination
This tool deletes the incomplete boards.cfg
if it encounters an error or is is terminated by the user.

I notice some problems even though they rarely happen.

[1] The boards.cfg is removed if the program is terminated
during __gen_boards_cfg() function but before boards.cfg
is actually touched.  In this case, the previous boards.cfg
should be kept as it is.

[2] If an error occurs while deleting the incomplete boards.cfg,
the program throws another exception.  This hides the privious
exception and we will not be able to know the real cause.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
d6538d22de tools/genboardscfg.py: wait for unfinished subprocesses before error-out
When an error occurs or the program is terminated by the user
on the way, the destructer __del__ of class Slot is invoked and
the work directories are removed.

We have to make sure there are no subprocesses (in this case,
"make O=<work_dir> ...") using the work directories before
removing them.  Otherwise the subprocess spits a bunch of error
messages possibly causing more problems.  Perhaps some users
may get upset to see too many error messages.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
13246f4861 tools/genboardscfg.py: be tolerant of insane Kconfig
The tools/genboardscfg.py expects all the Kconfig and defconfig are
written correctly.  Imagine someone accidentally has broken a board.
Error-out just for one broken board is annoying for the other
developers.  Let the tool skip insane boards and continue processing.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
b8828e8ff3 tools/genboardscfg.py: be tolerant of missing MAINTAINERS
tools/genboardscfg.py expects all the boards have MAINTAINERS.
If someone adds a new board but misses to add its MAINTAINERS file,
tools/genboardscfg.py fails to generate the boards.cfg file.
It is annoying for the other developers.

This commit allows tools/genboardscfg.py to display warning messages
and continue processing even if some MAINTAINERS files are missing
or have broken formats.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
04b43f3273 tools/genboardscfg.py: ignore defconfigs starting with a dot
Kconfig in U-Boot creates a temporary file configs/.tmp_defconfig
during processing "make <board>_defconfig".  The temporary file
might be left over for some reasons.

Just in case, tools/genboardscfg.py should make sure to
not read such garbage files.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
390f7035d3 buildman: run genboardscfg.py all the time
This commit makes sure boards.cfg is up to date before starting
the build tests.  tools/genboardscfg.py exits immediately printing
"boards.cfg is up to date. Nothing to do." when boards.cfg is
already new.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
9c2d60c378 tools/genboardscfg.py: fix a bug of MAINTAINERS handling
This patch fixes a minor problem:
If a block without "F:   configs/*_defconfig" is followed by another
block with "F:   configs/*_defconfig", the maintainers from the
former block are squashed into the latter.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-28 17:18:48 -04:00
Simon Glass
042a732cf5 patman: Support the 'reverse' option for 'git log'
This option is currently not supported, but needs to be, for buildman to
operate as expected.

Reported-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-28 07:56:19 -07:00
Tom Rini
7bee1c91a9 Merge branch 'agust@denx.de' of git://git.denx.de/u-boot-staging 2014-08-25 08:34:39 -04:00
Masahiro Yamada
7f6b8315d1 buildman: refactor help message
"buildman [options]" is displayed by default.

Append the rest of help messages to parser.usage
instead of replacing it.

Besides, "-b <branch>" is not mandatory since commit fea5858e.
Drop it from the usage.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-22 10:42:36 -04:00
Masahiro Yamada
e0a4d06af2 patman: refactor help message
"patman [options]" is displayed by default.

Append the rest of help messages to parser.usage
instead of replacing it.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-22 10:41:50 -04:00
Roger Meier
fd18a89e7f Makefile: remove generated boards.cfg within make distclean
Signed-off-by: Roger Meier <roger@bufferoverflow.ch>
Reviewed-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Tested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Tom Rini <trini@ti.com>
Cc: Wolfgang Denk <wd@denx.de>
Cc: Simon Glass <sjg@chromium.org>
2014-08-22 10:41:35 -04:00
Roger Meier
5b12b7a193 tools/genboardscfg.py: no exception if columns undetectable
The existing terminalsize detection raised an exception on build
server. Just removes the exception. This also deactivates the
progress indicator.

Remove a trainling whitespace.

Signed-off-by: Roger Meier <roger@bufferoverflow.ch>
CC: Masahiro Yamada <yamada.m@jp.panasonic.com>
CC: Tom Rini <trini@ti.com>
2014-08-21 12:01:26 -04:00
Masahiro Yamada
ca418dd74b tools/genboardscfg.py: Do not output SPLCPU field
Prior to Kconfig, the CPU field of boards.cfg could optionally have
":SPLCPU", like "armv7:arm720t".
(Actually this syntax was only used for Tegra platform.)

Now it is not necessary at all because CPU is defined by
CONFIG_SYS_CPU in Kconfig.

For Tegra platform, the Kconfig option is described as follows:

  config SYS_CPU
          string
          default "arm720t" if SPL_BUILD
          default "armv7" if !SPL_BUILD

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-21 12:01:13 -04:00
Masahiro Yamada
31e2141d5a tools, scripts: refactor error-out statements of Python scripts
In Python, sys.exit() function can also take an object other
than an integer.

If an integer is given to the argument, Python exits with the return
code of it.  If a non-integer argument is given, Python outputs it
to stderr and exits with the return code of 1.

That means,

    print >> sys.stderr, "Blah Blah"
    sys.exit(1)

is equivalent to

    sys.exit("Blah Blah")

The latter is a useful shorthand.

Note:
Some error messages in Buildman and Patman were output to stdout.
But they should go to stderr.  They are also fixed by this commit.
This is a nice side effect.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-21 12:01:11 -04:00
Luka Perkov
31cbe80c33 mkimage: fix compilation issues on OpenBSD
Signed-off-by: Luka Perkov <luka@openwrt.org>
2014-08-14 11:48:11 +02:00
Heiko Schocher
686dca0fc4 tools, fit_info: increase buffer for command name
currently the buffer for command name is 50 bytes only. If using
fit_info with long absolute paths, this is not enough, so raise
it to 256 (as it is in fit_check_sign)

Signed-off-by: Heiko Schocher <hs@denx.de>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-14 11:20:24 +02:00
Heiko Schocher
04a710a593 tools: fix typo in tools/image-host.c
fix a typo in error printf. If FIT_CONFS_PATH is not found
print FIT_CONFS_PATH not FIT_IMAGES_PATH.

Signed-off-by: Heiko Schocher <hs@denx.de>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-14 11:20:01 +02:00
Simon Glass
e49f14af13 patman: Only use git's --no-decorate when available
Older versions of git (e.g. Ubuntu 10.04) do not support this flag. By
default they do not decorate. So only enable this flag when supported.

Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:16 -06:00
Simon Glass
cda2a61152 patman: Move the 'git log' command into a function
Move the code that builds a 'git log' command into a function so we can more
easily adjust it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:16 -06:00
Simon Glass
3b74ba5f1a buildman: Allow selection of the number of commits to build
It is useful to be able to build only some of the commits in a branch. Add
support for the -c option to allow this. It was previously parsed by
buildman but not implemented.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: York Sun <yorksun@freescale.com>
2014-08-13 08:34:16 -06:00
Simon Glass
6131beab69 buildman: Introduce an 'and' operator for board selection
Currently buildman allows a list of boards to build to be specified on the
command line. The list can include specific board names, architecture, SOC
and so on.

At present the list of boards is dealt with in an 'OR' fashion, and there
is no way to specify something like 'arm & freescale', meaning boards with
ARM architecture but only those made by Freescale. This would exclude the
PowerPC boards made by Freescale.

Support an '&' operator on the command line to permit this. Ensure that
arguments can be specified in a single string to permit easy shell quoting.

Suggested-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Acked-by: York Sun <yorksun@freescale.com>
2014-08-13 08:34:16 -06:00
Simon Glass
e956947858 buildman: Add a few more toolchain examples to the README
The current README is a bit sparse in this area, so add a few more
examples.

Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:16 -06:00
Simon Glass
28370c1b99 buildman: Add a message indicating there are no errors
If buildman finds no problems it prints nothing. This can be a bit confusing,
so add a message that all is well.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
0f7c9ddaed buildman: Add an option to specify the buildman config file
Add a new --config-file option (-G) to specify a different configuration
file from the default ~/.buildman.

Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
d3269ed380 buildman: Remove unused non-incremental build method code
The non-incremental build method is no longer used, so remove it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
e5a0e5d842 buildman: Add verbose option to display errors as they happen
Normally buildman operates in two passes - one to do the build and another
to summarise the errors. Add a verbose option (-v) to display build problems
as they happen. With -e also given, this will display errors too.

When building the current source tree (rather than a list of commits in a
branch), both -v and -e are enabled automatically.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00
Simon Glass
b2ea7ab252 buildman: Refactor output options
We need the output options to be available in several places. It's a pain
to pass them into each function. Make them properties of the builder and
add a single function to set them up. At the same time, add a function which
produces summary output using these options.

Signed-off-by: Simon Glass <sjg@chromium.org>
2014-08-13 08:34:15 -06:00