Commit graph

27 commits

Author SHA1 Message Date
Andre Przywara
5ecdd529ae genboardscfg: limit to 240 jobs
When genboardscfg.py is run on machines with 255 or more cores, the
process will consume more than 1024 file descriptors, which is a common
standard ulimit for user processes. As a consequence it will fail with a
lenghty Python trace, with the almost hidden message:
OSError: [Errno 24] Too many open files

It's somewhat questionable whether that level of parallelity is actually
useful for genboardscfg, so we limit the *default* number of jobs to the
safe number of 240, to avoid the problem.
If a user persists, she can still force a higher number via the -j
parameter - hopefully having raised the ulimit accordingly beforehand.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
2022-01-13 09:13:41 -07:00
Simon Glass
b4fa94959d patman: Tidy up sys.path changes
Now that we are using absolute paths we can remove some of the sys.path
mangling that appears in the tools.

We only need to add the path to 'tools/' so that everything can find
modules relative to that directory.

The special paths for finding pylibfdt remain.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-04-26 14:25:21 -06:00
Baruch Siach
3166014d57 genboardscfg.py: drop python version comment
genboardscfg.py requires python 3.x since commit 3bc14098d8
("genboardscfg.py: Convert to Python 3").

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2020-01-25 12:04:36 -05:00
Simon Glass
69bbdd112b genboardcfg: Support a quiet mode
We don't really need buildman to print this every time it runs. Add a flag
to run quietly, that buildman can use.

Signed-off-by: Simon Glass <sjg@chromium.org>
2019-12-10 21:11:30 -07:00
Tom Rini
65e05ddc1a kconfiglib: Update to the 12.14.0 release
A large number of changes have happened upstream since our last sync
which was to 375506d.  The reason to do the upgrade at this point is for
improved Python 3 support.

As part of this upgrade we need to update moveconfig.py and
genboardscfg.py the current API.  This is:
- Change "kconfiglib.Config" calls to "kconfiglib.Kconfig"
- Change get_symbol() calls to syms.get().
- Change get_value() to str_value.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
2019-10-08 08:37:29 -04:00
Tom Rini
5e7c8a39e6 genboardscfg.py: Remove "warnings" print section
We tell kconfiglib to not print any warnings to us so drop this code as
it will be unused.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
2019-10-08 08:37:28 -04:00
Tom Rini
3bc14098d8 genboardscfg.py: Convert to Python 3
Convert this tool to requiring Python 3.  The bulk of this is done with
the 2to3 tool In addition, we need to use the '//' operator to have our
division result return an int rather than a float and ensure that we use
UTF-8 when reading/writing files.

Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Tom Rini <trini@konsulko.com>
2019-10-08 08:37:28 -04:00
Peng Fan
6080170265 tools: genboardscfg: move buildman path to first
To system which has kconfiglib installed, genboardscfg will
use system kconfiglib, we need it use U-Boot owned version,
so move the buildman path to first.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
Cc: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Cc: Tom Rini <trini@konsulko.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2018-07-30 07:18:49 -04:00
Tom Rini
83d290c56f SPDX: Convert all of our single license tags to Linux Kernel style
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from.  So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry.  Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.

In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.

This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents.  There's also a few places where I found we did not have a tag
and have introduced one.

Signed-off-by: Tom Rini <trini@konsulko.com>
2018-05-07 09:34:12 -04:00
Simon Glass
8639f69a61 genconfig.py: Print defconfig next to warnings
At present we sometimes see warnings of the form:

/tmp/tmpMA89kB:36: warning: overriding the value of CMD_SPL.
	Old value: "y", new value: "y".

This is not very useful as it does not show whch defconfig file it relates
to. Update the tool to show this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2017-09-11 21:43:58 -06:00
Lokesh Vutla
dee7c68f54 tools/genboardscfg.py: Make 'Supported' as known status
As per MAINTAINERS[1] file description, 'Supported' is
a valid status for a board. But buildman thinks 'Maintained'
is the only valid state and complains about boards with 'Supported'
status. Update buildman to accept 'Supported' as valid state.

[1] http://git.denx.de/?p=u-boot.git;a=blob;f=MAINTAINERS;h=0962b47bf9057b22e93624e070c0204b893790dc;hb=HEAD#l10

Reported-by: Sekhar Nori <nsekhar@ti.com>
Signed-off-by: Lokesh Vutla <lokeshvutla@ti.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2017-05-12 08:37:36 -04:00
Masahiro Yamada
d339df522b tools/genboardscfg.py: remove bogus import subprocess
Since f6c8f38ec6 ("tools/genboardscfg.py: improve performance more
with Kconfiglib"), this tool does not use the subprocess module.

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
2016-05-27 15:47:56 -04:00
Masahiro Yamada
5dff844d7f tools/genboardscfg.py: pick up also commented maitainers
We are still keeping invalid email addressed in MAINTAINERS
because they carry information.

The problem is that scripts/get_maintainer.pl adds emails in the
"M:" field including invalid ones.

We want to comment out invalid email addresses in MAINTAINERS
to prevent scripts/get_maintainer.pl from picking them up.
On the other hand, we want to collect them for boards.cfg
to know the last known maintainer of each board.

This commit adjusts tools/genboardscfg.py to parse also
the commented "M:" fields, which is useful for the next commit.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-09-24 18:30:28 -04:00
Masahiro Yamada
f6c8f38ec6 tools/genboardscfg.py: improve performance more with Kconfiglib
The idea of using Kconfiglib was given by Tom Rini.
It allows us to scan lots of defconfigs very quickly.
This commit also uses multiprocessing for further acceleration.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Suggested-by: Tom Rini <trini@ti.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-09-16 12:23:56 -04:00
Masahiro Yamada
2134342e57 tools/genboardscfg.py: change shebang into /usr/bin/env python2
This tool only works on python 2 (python 2.6 or lator).

Change the shebang to make sure the script is run by python 2
and clearly say the supported version in the comment block.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
9a65cb7ffe tools/genboardscfg.py: improve performance
I guess some developers are already getting sick of this tool
because it generally takes a few minites to generate the boards.cfg
on a reasonable computer.

The idea popped up on my mind was to skip Makefiles and
to run script/kconfig/conf directly.
This tool should become about 4 times faster.
You might still not be satisfied, but better than doing nothing.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
d1bf4afda6 tools/genboardscfg.py: check if the boards.cfg is up to date
It looks silly to regenerate the boards.cfg even when it is
already up to date.

The tool should exit with doing nothing if the boards.cfg is newer
than any of defconfig, Kconfig and MAINTAINERS files.

Specify -f (--force) option to get the boards.cfg regenerated
regardless its time stamp.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
79d45d32bc tools/genboardscfg.py: fix minor problems on termination
This tool deletes the incomplete boards.cfg
if it encounters an error or is is terminated by the user.

I notice some problems even though they rarely happen.

[1] The boards.cfg is removed if the program is terminated
during __gen_boards_cfg() function but before boards.cfg
is actually touched.  In this case, the previous boards.cfg
should be kept as it is.

[2] If an error occurs while deleting the incomplete boards.cfg,
the program throws another exception.  This hides the privious
exception and we will not be able to know the real cause.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:49 -04:00
Masahiro Yamada
d6538d22de tools/genboardscfg.py: wait for unfinished subprocesses before error-out
When an error occurs or the program is terminated by the user
on the way, the destructer __del__ of class Slot is invoked and
the work directories are removed.

We have to make sure there are no subprocesses (in this case,
"make O=<work_dir> ...") using the work directories before
removing them.  Otherwise the subprocess spits a bunch of error
messages possibly causing more problems.  Perhaps some users
may get upset to see too many error messages.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
13246f4861 tools/genboardscfg.py: be tolerant of insane Kconfig
The tools/genboardscfg.py expects all the Kconfig and defconfig are
written correctly.  Imagine someone accidentally has broken a board.
Error-out just for one broken board is annoying for the other
developers.  Let the tool skip insane boards and continue processing.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
b8828e8ff3 tools/genboardscfg.py: be tolerant of missing MAINTAINERS
tools/genboardscfg.py expects all the boards have MAINTAINERS.
If someone adds a new board but misses to add its MAINTAINERS file,
tools/genboardscfg.py fails to generate the boards.cfg file.
It is annoying for the other developers.

This commit allows tools/genboardscfg.py to display warning messages
and continue processing even if some MAINTAINERS files are missing
or have broken formats.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
04b43f3273 tools/genboardscfg.py: ignore defconfigs starting with a dot
Kconfig in U-Boot creates a temporary file configs/.tmp_defconfig
during processing "make <board>_defconfig".  The temporary file
might be left over for some reasons.

Just in case, tools/genboardscfg.py should make sure to
not read such garbage files.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-28 17:18:48 -04:00
Masahiro Yamada
9c2d60c378 tools/genboardscfg.py: fix a bug of MAINTAINERS handling
This patch fixes a minor problem:
If a block without "F:   configs/*_defconfig" is followed by another
block with "F:   configs/*_defconfig", the maintainers from the
former block are squashed into the latter.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-28 17:18:48 -04:00
Roger Meier
5b12b7a193 tools/genboardscfg.py: no exception if columns undetectable
The existing terminalsize detection raised an exception on build
server. Just removes the exception. This also deactivates the
progress indicator.

Remove a trainling whitespace.

Signed-off-by: Roger Meier <roger@bufferoverflow.ch>
CC: Masahiro Yamada <yamada.m@jp.panasonic.com>
CC: Tom Rini <trini@ti.com>
2014-08-21 12:01:26 -04:00
Masahiro Yamada
ca418dd74b tools/genboardscfg.py: Do not output SPLCPU field
Prior to Kconfig, the CPU field of boards.cfg could optionally have
":SPLCPU", like "armv7:arm720t".
(Actually this syntax was only used for Tegra platform.)

Now it is not necessary at all because CPU is defined by
CONFIG_SYS_CPU in Kconfig.

For Tegra platform, the Kconfig option is described as follows:

  config SYS_CPU
          string
          default "arm720t" if SPL_BUILD
          default "armv7" if !SPL_BUILD

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
2014-08-21 12:01:13 -04:00
Masahiro Yamada
31e2141d5a tools, scripts: refactor error-out statements of Python scripts
In Python, sys.exit() function can also take an object other
than an integer.

If an integer is given to the argument, Python exits with the return
code of it.  If a non-integer argument is given, Python outputs it
to stderr and exits with the return code of 1.

That means,

    print >> sys.stderr, "Blah Blah"
    sys.exit(1)

is equivalent to

    sys.exit("Blah Blah")

The latter is a useful shorthand.

Note:
Some error messages in Buildman and Patman were output to stdout.
But they should go to stderr.  They are also fixed by this commit.
This is a nice side effect.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-08-21 12:01:11 -04:00
Masahiro Yamada
3c08e8b856 tools: add genboardscfg.py
Now the primary data for each board is in Kconfig, defconfig and
MAINTAINERS.

It is true boards.cfg is needed for MAKEALL and buildman and might be
useful to brouse all the supported boards in a single database.
But it would be painful to maintain the boards.cfg in sync.

So, this is the solution.
Add a tool to generate the equivalent boards.cfg file based on
the latest Kconfig, defconfig and MAINTAINERS.

We can keep all the functions of MAKEALL and buildman with it.

The best thing would be to change MAKEALL and buildman for not
depending on boards.cfg in the future, but it would take some time.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
2014-07-30 14:42:02 -04:00