The patman directory has a number of modules which are used by other tools
in U-Boot. This makes it hard to package the tools using pypi since the
common files must be copied along with the tool that uses them.
To address this, move these files into a new u_boot_pylib library. This
can be packaged separately and listed as a dependency of each tool.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is quite a useful thing to use when building since it avoids small
size changes between commits. Add a -r flag for it.
Also undefine CONFIG_LOCALVERSION_AUTO since this appends the git hash
to the version string, causing every build to be slightly different.
Signed-off-by: Simon Glass <sjg@chromium.org>
This cuts down build performance considerably and is not always needed,
when checking for build errors, etc.
Add a flag to disable it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a new flag to buildman so that we will in turn pass
BINMAN_ALLOW_MISSING=1 to 'make'. Make use of this flag in CI.
Allow the settings file to control this.
Cc: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
The 'nm' tool can produce lines without a symbol, for example:
00000004 t
Silently skip these and anything else without three fields. Drop the
warning since there is nothing the user can do about it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
We have a module called 'board'. Sometimes buildman uses 'brd' as an
instance variable but sometimes it uses 'board', which is confusing and
can mess with the module handling. Update the code to use 'brd'
consistently, making it easier for tools to determine when the module
is being referenced.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a flag to allow buildman to behave properly for use from an IDE. This
shows error/warning output on stderr and drops all summary and progress
information.
This should normally only be used when building a single board.
Fix up a confusing comment for GetResultSummary() while we are here, since
we want to use the Outcome object to access the unprocessed error lines
from the build.
Signed-off-by: Simon Glass <sjg@chromium.org>
As removal of nds32 has been ack'd for the Linux kernel, remove support
here as well.
Cc: Rick Chen <rick@andestech.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Rick Chen <rick@andestech.com>
This method has the same name as its class which is confusing. It is also
annoying when searching the code.
It builds a string with a colour, so rename it to build().
Signed-off-by: Simon Glass <sjg@chromium.org>
Rename this function so that when we convert it to snake case it will not
conflict with the built-in print() function.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a -a option to specify changes to the config before the build
commences. For example
buildman -a ~CONFIG_CMDLINE
disables CONFIG_CMDLINE before doing the build.
This makes it easier to try things out as well as to write tests without
creating a new board or manually manging the .config file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Hex and int Kconfig options are supposed to have defaults. This is so we
can configure U-Boot without having to enter particular values for the
items that don't have specific values in the board's defconfig file.
If this rule is not followed, then introducing a new Kconfig can produce
a loop like this:
Break things (BREAK_ME) [] (NEW)
Error in reading or end of file.
Break things (BREAK_ME) [] (NEW)
Error in reading or end of file.
The continues forever since buildman passes /dev/null to 'conf', and
the build system just tries again. Eventually there is so much output that
buildman runs out of memory.
We can detect this situation by looking for a symbol (like 'BREAK_ME')
which has no default (the '[]' above) and is marked as new. If this
appears multiple times in the output, we know something is wrong.
Add a filter function for the output which detects this situation. Allow
it to return True to terminate the process. Implement this termination in
cros_subprocess.
With this we get a nice message:
buildman --board sandbox -T0
Building current source for 1 boards (0 threads, 32 jobs per thread)
sandbox: w+ sandbox
+.config:66:warning: symbol value '' invalid for BREAK_ME
+
+Error in reading or end of file.
+make[3]: *** [scripts/kconfig/Makefile:75: syncconfig] Terminated
+make[2]: *** [Makefile:569: syncconfig] Terminated
+make: *** [Makefile:177: sub-make] Terminated
+(** did you define an int/hex Kconfig with no default? **)
Signed-off-by: Simon Glass <sjg@chromium.org>
There have been at least a few cases where an exception has occurred in a
thread and resulted in buildman hanging: running out of disk space and
getting a unicode error.
Handle these by collecting a list of exceptions, printing them out and
reporting failure if any are found. Add a test for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present even if only a single thread is in use, buildman still uses
threading.
For some debugging it is helpful to do everything in the main process.
Allow -T0 to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
The isAlive() method was deprecated in Python 3.8 and has been removed in
Python 3.9. See https://bugs.python.org/issue37804. Use is_alive() instead.
Since Python 2.6 is_alive() has been a synonym for isAlive(). So there
should be no problems for users using elder Python 3 versions.
Signed-off-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
This patch makes buildman create linked working trees instead of clones
of the source repository, but keeps updating the older clones of the
repository that might already exist. These worktrees share "everything
except working directory specific files such as HEAD, index, etc." with
the source repository. See the git-worktree(1) manual page for more
information.
If git-worktree isn't available, silently falls back to cloning the
repository.
Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
It is interesting to note the number of builds completed per second to
track machine performance and build speed. Add a 'rate' value at the end
of the build to show this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Currently, the following scenario will rebuild the first commit even
though it is not really necessary - the commit sha or the position in the
patchset did not change:
$ git am <local-patch-0001>
$ tools/buildman/buildman -P -E -W -b master mx6
<do some more development work>
$ git am <local-patch-0002>
$ tools/buildman/buildman -P -E -W -b master mx6 <- will rebuild the first
commit as well, even
though nothing has
changed about it.
This is due to the fact that previous results directories get removed
when the number of commits change. By removing the _of_#_ part of the
directory path, the commits will be rebuilt only if the commit sha or the
position in the patchset changes. Also, update the testcase to reflect this
change.
Signed-off-by: Ovidiu Panait <ovidiu.panait@windriver.com>
At present patman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move patman to use absolute imports. This requires changes in tools which
use the patman libraries (which is most of them).
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman sets the python path on startup so that it can access
the libraries it needs. If we convert to use absolute imports this is not
necessary.
Move buildman to use absolute imports. Also adjust moveconfig.py too since
it uses some buildman modules and cannot work without this.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman does not write its own output files (err, done, the
environment) when using -w. However this is useful for when the build is
run with -s to check it.
In fact ProduceResultSummary() reads the result from those files rather
than using the 'result' info directly. So ProcessResult() does not work
with -w at present. It does not print any output.
Fix this by writing output files even when -w is used.
Signed-off-by: Simon Glass <sjg@chromium.org>
These are becoming more common now. They cause boards to show warnings
which can be mistaking for compiler warnings.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
Unfortunately the plague of device-tree warnings has not lifted. These
warnings infiltrate almost every build, adding noise and confusion.
Add a buildman option to ignore them. This option works only with the
summary option (-s). It does not affect the build process.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman defaults to running 'mrproper' on every thread before
it starts building commits for each board. This can add a delay of about 5
seconds to the start of the process, since the tools and other invariants
must be rebuilt.
In particular, a build without '-b', to build current source, runs much
slower without -I, since any existing build is removed, thus losing the
possibility of an incremental build.
Partly this behaviour was to avoid strange build-system problems caused by
running 'make defconfig' for one board and then one with a different
architecture. But these problems were fixed quite a while ago.
The -I option (which disabled mrproper) was introduced four years ago and
does not seem to cause any problems with builds.
So make -I the default and deprecate the option. To allow use of
'mrproper', add a new -m flag.
Signed-off-by: Simon Glass <sjg@chromium.org>
When buildman finishes it leaves the last summary line visible, which
shows the number of successful builds, builds with warnings and builds
with errors.
It is useful also to see how many builds were done in total along with
the time taken. Show these on a separate line before buildman finishes.
Signed-off-by: Simon Glass <sjg@chromium.org>
If a progress message is longer than the terminal line it will scroll the
terminal. Limit the messages to the terminal width.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is nice to see the actual number of builds remaining to complete. Add
this in the progress message, using a different colour.
Drop the unnecessary 'name' variable while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
The commit counter is a hangover from when buildman processed each board
for a commit. Now buildman processes each commit for a board, so this
output is never triggered.
Delete it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fetching updated versions of a repo can take time. At present buildman
gives no indication that it is doing this.
Add a message to explain the delay.
Tidy up a few other messages while we are here.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the board names shown with -l are separated by commas. This
makes it hard to double-click to select a particular board. Also it is not
possible to select all boards and paste them as arguments to a subsequent
buildman run, since buildman requires spaces to separate the list on the
command line, not commas.
Change the output format to use spaces instead of commas.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is quite hard to see the list of board for each error line since the
colour is the same as the actual error line. Show the board list in
magenta so that it is easier to distinguish them.
There is no point in checking the colour of the overall line, since there
are now multiple colours. So drop those tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the string for each error line is created in _CalcErrorDelta()
and used to create the summary output. This is inflexible since all the
information (error/warning character, error line, list of boards with that
error line) is munged together in a string.
Create an object to hold this information and only convert it to a string
when printing the actual output.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present warnings are shown in yellow in the summary (-s) but magenta in
the detail listing (-e). Use yellow in both.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is useful in some situations, in particular with -w and when building
in-tree. Now that we are more careful about what we remove in
_PrepareOutputSpace(), it should be safe to relax this restriction.
Update the progress information also so it is clear what buildman is
doing. Remove files can take a long time.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present buildman removes any directory it doesn't intend to write
output into. This is overly expansive since if the output directory
happens to be somewhere with existing files, they may be removed. Using
an existing directory for buildman is not a good practice, but since the
result might be catastrophic, it is best to guard against it.
A previous commit[1] fixed this by refusing to write to a subdirectory
of the current directory, assumed to have U-Boot source code. But we can
do better by only removing directories that look like the ones buildman
creates.
Update the code to do this and add a test.
Signed-off-by: Simon Glass <sjg@chromium.org>
[1] 409fc029c4 tools: buildman: Don't use the working dir as build dir
This help is a bit ambiguous. It only does anything if asked to show size
changes with -S. Update the help and the function comments.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is useful to run a simple build and put all the output in a single
directory. Add a -w option to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Today when parsing the .sizes files we get a warning about an invalid
line in the file as it's blank. Solve this by checking that we have a
non-blank line prior to processing.
Cc: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The 'done' files created by buildman may end up being empty if buildman
runs out of disk space while writing them. At present buildman dies with
an exception when using -s to check the build status. Fix this.
Seriesl-cc: trini
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we should boards with warnings in the same way as those with
errors. This is not ideal. Add a new 'warn' state and show these listed
in yellow to match the actual warning lines printing with -e.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present we don't distinguish between errors and warnings when printing
the architecture summary. Rename the variables to better describe their
purpose.
'Worse' at present means we got an error, so use that as the name.
Signed-off-by: Simon Glass <sjg@chromium.org>
