The tools/moveconfig.py has a feature to cleanup #define/#undef's
of moved config options, but I want this tool to do a better job.
For example, when we are moving CONFIG_FOO and its define is
surrounded by #ifdef ... #endif, like follows:
#ifdef CONFIG_BAR
# define CONFIG_FOO
#endif
The header cleanup will leave empty #ifdef ... #endif:
#ifdef CONFIG_BAR
#endif
Likewise, if a define line between two blank lines
<blank line>
#define CONFIG_FOO
<blank lines.
... is deleted, the result of the clean-up will be successive empty
lines, which is a coding-style violation.
It is tedious to remove left-over garbage lines manually, so I want
the tool to take care of this. The tool's job is still not perfect,
so we should check the output of the tool, but I hope our life will
be much easier with this patch.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The clean tree (make mrproper) and compilers are required when moving
config options, but not needed when we only cleanup headers.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
The files in include/generated are generated during build and removed
by "make mrproper", so it has no point to touch them by this tool.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
There are some cases where config options are moved, but they are
ripped off at the final savedefconfig stage:
- The moved option is not user-configurable, for example, due to
a missing prompt in the Kconfig entry
- The config was not defined in the original config header despite
the Kconfig specifies it as non-bool type
- The config define in the header contains reference to another
macro, for example:
#define CONFIG_CONS_INDEX (CONFIG_SYS_LPC32XX_UART - 2)
The current moveconfig does not support recursive macro expansion.
In these cases, the conversion is very likely to be an unexpected
result. That is why I decided to display the log in yellow color
in commit 5da4f857be ("tools: moveconfig: report when CONFIGs are
removed by savedefconfig").
It would be nice to display the list of suspicious boards when the
tool finishes processing. It is highly recommended to check the
defconfigs once again when this message is displayed.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Since commit 1d085568b3 ("tools: moveconfig: display log atomically
in more readable format"), the function color_text() is clever enough
to exclude LF from escape sequences. Exploit it for removing the
"for" loops from Slots.show_failed_boards().
Also, display "(the list has been saved in moveconfig.failed)" if
there are failed boards.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
The subprocess.Popen() does not change the child process's working
directory if cwd=None is given. Let's exploit this fact to refactor
the source directory handling.
We no longer have to pass "-C <reference_src_dir>" to the sub-process
because self.current_src_dir tracks the source tree against which we
want to run defconfig/autoconf.
The flag self.use_git_ref is not necessary either because we can know
the current state by checking whether the self.current_src_dir is a
valid string or None.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The class WorkDir can be used in a very generic way, but currently
it is only used for containing a reference source directory.
This commit changes it for a more dedicated use. The move_config
function can be more readable by enclosing the git-clone and git-
checkout in the class constructor.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
When moving an integer type option with default value 1, the tool
moves configs with the same value as the default (, and then removed
by the later savedefconfig). This is a needless operation.
The KconfigParser.parse_one_config() should compare the config after
the "=y -> =1" fixup.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
This option allows the 'make autoconf.mk' step to run against a former
repo state, while the savedefconfig step runs against the current repo
state. This is convenient for the case where something in the Kconfig
has changed such that the defconfig is no longer complete with the new
Kconfigs. This feature allows the .config to be built assuming those old
Kconfigs, but then savedefconfig based on the new state of the Kconfigs.
If in doubt, always specify this switch. It will always do the right
thing even if not required, but if it was required and you don't use it,
the moved configs will be incorrect. When not using this switch, you
must very carefully evaluate that all moved configs are correct.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The old color blends in with similar messages and makes them not stand
out.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
The Slot.poll() method is already complicated and a new feature
we are going to add will make it more difficult to understand
the execution flow.
Refactor it with helper methods, .handle_error(), .do_defconfig(),
.do_autoconf(), .do_savedefconfig, and .update_defconfig().
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
I found "tools/moveconfig -s" might be useful for defconfig re-sync.
I could optimize it for re-sync if I wanted, but I do not want to
make the code complex for this feature.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Now, this tools invokes "make savedefconfig" only when it needs to
do so, but there might be cases where a user wants the tool to do
savedefconfig forcibly, for example, some defconfigs were already
out of sync and the user wants to fix it as well.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
There are various factors that determine if the given defconfig is
updated, and it is probably what users are more interested in.
Show the log when the defconfig is updated. Also, copy the file
only when the file content was really updated to avoid changing
the time stamp needlessly.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
This is a rare case, but there is still possibility that some CONFIG
is moved to the .config, but it is removed by "make savedefconfig".
(For example, it happens when the specified CONFIG has no prompt in
the Kconfig entry, i.e. it is not user-configurable.)
It might be an unexpected case. So, display the log in this case
(in yellow color to gain user's attention if --color option is given).
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Now, "make savedefconfig" does not always happen. Display the log
when it happens.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
If no CONFIG option is moved to the .config, no need to sync the
defconfig file. This accelerates the processing by skipping
needless "make savedefconfig".
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Move similar code to finish() function.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Before this commit, the log was displayed in the format:
<defconfig_name> : <action1>
<defconfig_name> : <action2>
<defconfig_name> : <action3>
When we move multiple CONFIGs at the same time, we see as many
<defconfig_name> strings as actions for every defconfig, which is
redundant information.
Moreover, since normal log and error log are displayed separately,
Messages from different threads could be mixed, like this:
<foo> : <action1>
<foo> : <action2>
<bar> : <action1>
<bar> : <action2>
<foo> : <error_log>
This commit makes sure to call "print" once a defconfig, which
enables atomic logging for each defconfig. It also makes it
possible to refactor the log format as follows:
<foo_defconfig>
<action1>
<action2>
<error_log>
<bar_defconfig>
<action1>
<action2>
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
This will help further improvement/clean-up.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
The paths to .config, include/autoconf.mk, include/config/auto.conf
are not changed during the defconfig walk. Compute them only once
when a new class instance is created.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
We still pass the input file with CONFIG name, type, default value
in each line, but the last two fields are just ignored by the tool.
So, let's deprecate the input file and allow users to give CONFIG
names directly from the command line. The types and default values
are automatically detected and handled nicely by the tool.
Going forward, we can use this tool more easily like:
tools/moveconfig.py CONFIG_FOO CONFIG_BAR
Update the documentation and fix some typos I noticed while I was
working on.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Now types and defalut values given by the input file are just
ignored. Delete unnecessary code.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Commit 96464badc7 ("moveconfig: Always run savedefconfig on the
moved config") changed the work flow of this tool a lot from the
original intention when this tool was designed first.
Since then, before running this tool, users must edit the Kconfig to
add the menu entries for the configs they are moving. It means users
had already specified the type and the default value for each CONFIG
via its Kconfig entry. Nevertheless, users are still required to
dictate the same type and the default value in the input file. This
is tedious to use. So, my idea here is to deprecate the latter.
Before moving forward with it, there is one issue worth mentioning;
since the savedefconfig re-sync was introduced, this tool has not
been able to move bool options with "default y". Joe sent a patch
to solve this problem about a year ago, but it was not applied for
some reasons. Now, he came back with an updated patch, so this
problem will be fixed soon.
For other use cases, I see no reason to require redundant dictation
in the input file. Instead, the tool can know the types and default
values by parsing the .config file.
This commit changes the tool to use the CONFIG names, but ignore the
types and default values given by the input file.
This commit also fixes one bug. Prior to this commit, it could not
move an integer-typed CONFIG with value 1.
For example, assume we are moving CONFIG_CONS_INDEX. Please note
this is an integer type option.
Many board headers define this CONFIG as 1.
#define CONFIG_CONS_INDEX 1
It will be converted to
CONFIG_CONS_INDEX=y
and moved to include/autoconf.mk, by the tools/scripts/define2mk.sed.
It will cause "make savedefconfig" to fail due to the type conflict.
This commit takes care of it by detecting the type and converting the
CONFIG value correctly.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Currently, the progress " * defconfigs out of 1133" does not increase
monotonically.
Moreover, the number of processed defconfigs does not match the total
number of defconfigs when this tool finishes, like:
1132 defconfigs out of 1133
Clean up headers? [y/n]:
It looks like the task was not completed, and some users might feel
upset about it.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
When the source tree is not clean, this tool raises an exception
with a message like follows:
Traceback (most recent call last):
File "tools/moveconfig.py", line 939, in <module>
main()
File "tools/moveconfig.py", line 934, in main
move_config(config_attrs, options)
File "tools/moveconfig.py", line 808, in move_config
while not slots.available():
File "tools/moveconfig.py", line 733, in available
if slot.poll():
File "tools/moveconfig.py", line 645, in poll
self.parser.update_dotconfig(self.defconfig)
File "tools/moveconfig.py", line 503, in update_dotconfig
with open(autoconf_path) as f:
IOError: [Errno 2] No such file or directory: '/tmp/tmpDtzCgl/include/autoconf.mk'
This does not explain what is wrong. Show an appropriate error
message "source tree is not clean, please run 'make mrproper'"
in such a situation.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Since commit 25400090b1 ("moveconfig: Print a message for
missing compiler"), this tool parses an error message every time an
error occurs during the process in order to detect missing compiler.
Instead of that, we can look for compilers in the PATH environment
only once before starting the defconfig walk. If a desired compiler
is missing, "make include/config/auto.conf" will apparently fail for
that architecture. So, the tool can just skip those board, showing
"Compiler is missing. Do nothing.".
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
We must ensure this tool is run from the top of source directory
before calling update_cross_compile(). Otherwise, the following
exception is thrown:
Traceback (most recent call last):
File "./moveconfig.py", line 918, in <module>
main()
File "./moveconfig.py", line 908, in main
update_cross_compile()
File "./moveconfig.py", line 292, in update_cross_compile
for arch in os.listdir('arch'):
OSError: [Errno 2] No such file or directory: 'arch'
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Nesting by "else:" is not generally useful after such statements
as return, break, sys.exit(), etc.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Commit 96464badc7 ("moveconfig: Always run savedefconfig on the
moved config") changed how defconfig files were updated.
Since then, the function update_defconfig() does not modify defconfig
files at all (instead, they are updated by "make savedefconfig"), so
update_dotconfig() is a better fit for this function. Also, update
the comment block to match the actual behavior.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Since commit 96464badc7 ("moveconfig: Always run savedefconfig on
the moved config"), --dry-run option is broken.
The --dry-run option prevents the .config from being modified,
but defconfig files might be updated by "make savedefconfig"
regardless of the --dry-run option.
Move the "if not self.options.dry_run" conditional to the correct
place.
Fixes 96464badc7 ("moveconfig: Always run savedefconfig on the moved config")
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
Since commit 87da2690ab
"openrisc: updating build tools naming convention", openrisc
kernel.org toolchain is out of date and cannot build U-Boot.
Update buildman and moveconfig tools to refer to the new one.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Give a full URL for a working nds32 toolchain for U-Boot.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
This gives a basic idea about progress.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
In some cases the build for the autoconf breaks. This outputs the errors
following the status so that action can be taken without building again
manually.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
A common case for failed builds is a missing compiler. Print a message
for that case to tell the user concisely which compiler was expected
that was not found.
This patch also has the effect of not printing build errors any longer.
The next patch will add a switch to optionally bring that back.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
If boards fail, output that list to a file so that it can easily be
passed back into moveconfig.py using the -d option.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Some config.h files live in arch and board directories. They will need
to be cleaned up as well, so run the same filters there.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
In some case you may want to only cleanup the headers. Make it possible
without waiting for all boards to compile.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
This is helpful to re-attempt to move failed boards from a previous run
without starting over.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
When moving configs, it is important to know what was defined in the
config header even if it duplicates the configs coming from Kconfig.
This is specifically needed for the case where a config is set to
default 'y' in the Kconfig. This would previously cause the actual value
from the include config to be filtered out, and moveconfig.py would
think that it was 'n'... This means that the value that should be 'y'
is now (in every defconfig) set to 'not set'.
tools/moveconfig.py now defines KCONFIG_IGNORE_DUPLICATES to prevent the
filtering from happening and selecting wrong values for the defconfig.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
This will ensure that the order of the defconfig entries will always
match that of the Kconfig files. After one slightly painful (but
still early in the process) pass over all boards, this should keep
the defconfigs clean from here on.
Users must edit the Kconfig first to add the menu entries and then run
moveconfig.py to update the defconfig files and the include configs.
As such, moveconfig.py cannot compare against the '.config' contents.
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
This tool was originally written for my local use to ease the task
of tons of CONFIG moves, but there have been some requests for
mainlining it.
So, I have tidied up the code with nicer comments, and here it is.
See the comment block of the script for usage.
The first draft was
http://patchwork.ozlabs.org/patch/430422/
Main updates are:
- Adapted to the single .config configuration
- Support colored log
- Support moving multiple options at once
(and take configs via input file only)
- Continue even if some boards fail
(Idea provided by Joe Hershberger)
- Add more options
- More comments and code cleanups
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Joe Hershberger <joe.hershberger@ni.com>
