At present tools are not expected to fail. If they do an exception is
raised but there is no detail about what went wrong. This makes it hard
to debug if something does actually go wrong.
Fix this by outputting both stderr and stdout on failure.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present outdir remains set ever after the output directory has been
removed. Fix this to avoid trying to access it when it is not present.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present binman cannot replace data within a CBFS since it does not
allow rewriting of the files in that CBFS. Implement this by using the
new WriteData() method to handle the case.
Add a header to compressed data so that the amount of compressed data can
be determined without reference to the size of the containing entry. This
allows the entry to be larger that the contents, without causing errors in
decompression. This is necessary to cope with a compressed device tree
being updated in such a way that it shrinks after the entry size is
already set (an obscure case). It is not used with CBFS since it has its
own metadata for this. Increase the number of passes allowed to resolve
the position of entries, to handle this case.
Add a test for this new logic.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use the new logging feature to log information about progress with
packing. This is useful to see how binman is figuring things out.
Also update elf.py to use the same feature.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present tout writes directly to stdout. This is not necessary and it
prevents tests from redirecting output. Change it to use print() for the
non-progress output.
Signed-off-by: Simon Glass <sjg@chromium.org>
Binman generally operates silently but in some cases it is useful to see
what Binman is actually doing at each step. Enable some logging output
with different logging levels selectable via the -v flag.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present entry modules can only be accessed using Entry.Lookup() or
Entry.Create(). Most of the time this is fine, but sometimes a module
needs to provide constants or helper functions useful to other modules.
It is easier in this case to use 'import'.
Add an __init__ file to permit this.
Signed-off-by: Simon Glass <sjg@chromium.org>
This class is the new way to handle arguments in Python. Convert binman
over to use it. At the same time, introduce commands so that we can
separate out the different parts of binman functionality.
Signed-off-by: Simon Glass <sjg@chromium.org>
This tool has quite a few arguments and options, so put the functionality
in a function so that we call it from one place and hopefully get it
right.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add utility functions to compress and decompress using lz4 and lzma
algorithms. In the latter case these use the legacy lzma support favoured
by coreboot's CBFS.
No tests are provided as these functions will be tested by the CBFS
tests in a separate patch.
Signed-off-by: Simon Glass <sjg@chromium.org>
If kwargs contains raise_on_error then this function generates an error
due to a duplicate argument. Fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
We need slightly different commands to run code coverage with Python 3.
Update the RunTestCoverage() function to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
While reading files in binary mode is the norm, sometimes we want to use
text mode. Add an optional parameter to handle this.
Signed-off-by: Simon Glass <sjg@chromium.org>
In Python 3 bytes and str are separate types. Use bytes to ensure that
the code functions correctly with Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
The difference between the bytes and str types in Python 3 requires a
number of minor changes to this function. Update it to handle the input
data using the 'bytes' type. Create two useful helper functions which can
be used by other modules too.
Signed-off-by: Simon Glass <sjg@chromium.org>
Change the code so that it works on both Python 2 and Python 3. This works
by using unicode instead of latin1 for the test input, and ensuring that
the output is converted to a string rather than a unicode object on
Python 2.
Signed-off-by: Simon Glass <sjg@chromium.org>
The unicode type does not exist in Python 3 and when displaying strings
they do not have the 'u' prefix. Adjusts the settings unit tests to deal
with this difference, by converting the comparison value to a string, thus
dropping the 'u'.
Signed-off-by: Simon Glass <sjg@chromium.org>
We use sets to produce the list of To and Cc lines for a series. This does
not result in stable ordering of the recipients. Sort each list to ensure
that the output is repeatable. This is necessary for tests.
Signed-off-by: Simon Glass <sjg@chromium.org>
Create helper functions in the tools module to deal with the differences
between unicode in Python 2 (where we use the 'unicode' type) and Python 3
(where we use the 'str' type).
Signed-off-by: Simon Glass <sjg@chromium.org>
The method of multiplying a character by a number works well for creating
a repeated string in Python 2. But in Python 3 we need to use bytes()
instead, to avoid unicode problems, since 'bytes' is no-longer just an
alias of 'str'.
Create a function to handle this detail and call it from the relevant
places in binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present cros_subprocess and the tools library use a string to obtain
stdout from a program. This works fine on Python 2. With Python 3 we end
up with unicode errors in some cases. Fix this by providing a binary mode,
which returns the data as bytes() instead of a string.
Signed-off-by: Simon Glass <sjg@chromium.org>
Python 3 requires this, and Python 2 allows it. Convert the code over to
ensure compatibility with Python 3.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present this function uses lists and strings. This does not work so
well with Python 3, and testing against '' does not work for a bytearray.
Update the code to fix these issues.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present the tests run one after the other using a single CPU. This is
not very efficient. Bring in the concurrencytest module and run the tests
concurrently, using one process for each CPU by default. A -P option
allows this to be overridden, which is necessary for code-coverage to
function correctly.
This requires fixing a few tests which are currently not fully
independent.
At some point we might consider doing this across all pytests in U-Boot.
There is a pytest version that supports specifying the number of processes
to use, but it did not work for me.
Signed-off-by: Simon Glass <sjg@chromium.org>
At present calling Uninit() always called ClearProgress() which outputs
a \r character as well as spaces to remove any progress information on the
line. This can mess up the normal output of binman and other tools. Fix
this by outputing this only when progress information has actually been
previous written.
Signed-off-by: Simon Glass <sjg@chromium.org>
With Python 2.7.15rc1, ConfigParser.SafeConfigParser has unfortunately
started returning unicode, for unknown reasons. Adjust the code to handle
this by converting everything to unicode. We cannot convert things to
ASCII since email addresses may be encoded with UTF-8.
Signed-off-by: Simon Glass <sjg@chromium.org>
In some cases it is useful to add a group of files to the image and be
able to access them at run-time. Of course it is possible to generate
the binman config file with a set of blobs each with a filename. But for
convenience, add an entry type which can do this.
Add required support (for adding nodes and string properties) into the
state module.
Signed-off-by: Simon Glass <sjg@chromium.org>
When tools are needed but not present, at present we just get an error
which can be confusing for the user. Try to be helpful by reporting the
tool as missing and suggesting a possible remedy.
Also update the Run() method to support this.
Signed-off-by: Simon Glass <sjg@chromium.org>
A recent rename of the function did not rename the test file. Fix this.
Fixes: 12308b128f (lib: fdtdec: Rename routine fdtdec_setup_memory_size())
Signed-off-by: Simon Glass <sjg@chromium.org>
When this fails it is useful to see the current directory, since U-Boot's
build system will typically change into the output directory during the
build. Add this information to the error.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tests use the 'test_result' feature to return a predetermined command
result for particular commands. The avoids needing to have the real
command available just to run a test. It works by calling the function
provided by the test, to get the value.
However sometimes the test does need to run the real command. Allow it to
fall back to do this when the function does not return a result.
Signed-off-by: Simon Glass <sjg@chromium.org>
Previously the first three words in a git-mailrc alias entry could only
be separated by spaces. git-send-email and Mutt both allow arbitrary
whitespace here.
Signed-off-by: Adam Sampson <ats@offog.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
This patch renames the routine fdtdec_setup_memory_size()
to fdtdec_setup_mem_size_base() as it now fills the
mem base as well along with size.
Signed-off-by: Siva Durga Prasad Paladugu <siva.durga.paladugu@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
At present only binman has the logic for determining Python test coverage
but this is useful for other tools also. Move it out into a separate file
so it can be used by other tools.
Signed-off-by: Simon Glass <sjg@chromium.org>
Some environments require providing the '--smtp-server' argument to
'git send-email'. Add support for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a test to exercise the check for a valid SPDX license.
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Many mailing-lists consider a long Cc list a sign of spam and will
either drop the message or mark it for moderation. Because patman
automatically invokes get_maintainer.pl the Cc list can expand
unexpectedly. Allow the user to specify a limit for the Cc list.
This limit is applied after removing any known bouncing addresses. By
default no limit is applied.
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
The format of this line has changed. Update the patman test to suit.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Alex Kiernan <alex.kiernan@gmail.com>
When U-Boot started using SPDX tags we were among the early adopters and
there weren't a lot of other examples to borrow from. So we picked the
area of the file that usually had a full license text and replaced it
with an appropriate SPDX-License-Identifier: entry. Since then, the
Linux Kernel has adopted SPDX tags and they place it as the very first
line in a file (except where shebangs are used, then it's second line)
and with slightly different comment styles than us.
In part due to community overlap, in part due to better tag visibility
and in part for other minor reasons, switch over to that style.
This commit changes all instances where we have a single declared
license in the tag as both the before and after are identical in tag
contents. There's also a few places where I found we did not have a tag
and have introduced one.
Signed-off-by: Tom Rini <trini@konsulko.com>
All of these host tools are apparently written for Python2,
not Python3.
Use 'python2' in the shebang line according to PEP 394
(https://www.python.org/dev/peps/pep-0394/).
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
get_maintainer.pl quotes names which it considers unsafe, i.e. anything
containing [^a-zA-Z0-9_ \-]. This confuses patman, it will duplicate
addresses which are also in Series-to/cc. Strip the quotes.
Signed-off-by: Stefan Brüns <stefan.bruens@rwth-aachen.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
This section of the settings file may be missing. Handle that gracefully
rather than emitting an error.
Also update patman to write this section when a new settings file is
created.
Fixes: e11aa602 (patman: add support for omitting bouncing addresses)
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Chris Packham <judge.pckham@gmail.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Add support for reading a list of bouncing addresses from a in-tree file
(doc/bounces) and from the ~/.patman config file. These addresses are
stripped from the Cc list.
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com <mailto:philipp.tomsich@theobroma-systems.com>>
The existing test (patman --test) only covers basic checkpatch output.
We have had some problems with unicode processing and could use test
coverage for the various tags patman supports.
Add a new functional test which runs most of the patman flow on a few
test commits and checks that the results are correct.
See the documentation in the test for a description of what it does.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
This is not a good variable name in Python because 'list' is a type. It
shows up highlighted in some editors. Rename it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
Allow the add_maintainers parameter to be a list of maintainers, thus
allowing us to simulate calling the script in tests without actually
needing it to work.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
Add some unicode to the test patches to make sure that patman does the
right thing.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
There is no need for this function to return the same object that was
passed in. Drop the return value.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
Unicode characters may appear in input patches so we should not warn about
them. Drop this warning.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
This is not a good variable name in Python because 'str' is a type. It
shows up highlighted in some editors. Rename it.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
The communication filter reads data in blocks and converts each block to
unicode (if necessary) one at a time. In the unlikely event that a unicode
character in the input spans a block this will not work. We get an error
like:
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 1022-1023:
unexpected end of data
There is no need to change the input to unicode, so the easiest fix is to
drop this feature.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
Don't mess with the email address when outputting them. Just make sure
they are encoded with utf-8.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
This change encodes the CC list to UTF-8 to avoid failures on
maintainer-addresses that include non-ASCII characters (observed on
Debian 7.11 with Python 2.7.3).
Without this, I get the following failure:
Traceback (most recent call last):
File "tools/patman/patman", line 159, in <module>
options.add_maintainers)
File "[snip]/u-boot/tools/patman/series.py", line 234, in MakeCcFile
print(commit.patch, ', '.join(set(list)), file=fd)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 81: ordinal not in range(128)
from Heiko's email address:
[..., u'"Heiko St\xfcbner" <heiko@sntech.de>', ...]
While with this change added this encodes to:
"=?UTF-8?q?Heiko=20St=C3=BCbner?= <heiko@sntech.de>"
Signed-off-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Tested-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
os.read() returns a byte array in Python 3.5.2 and needs to be converted
into a string. Check if the returned value is an instance of bytes and
if it is decode it as a utf-8 string. If it is not a utf-8 encoded string
the decoding may fail with an exception.
Prior to this fix the comparisions check data == "" would fail when data
was b'' and would cause an infinite memory leaking loop. joins would
also fail with an exception below but due to the infinite loop it never
made it that far.
Signed-off-by: George McCollister <george.mccollister@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
When gathering addresses for the Cc list patman would encounter a
UnicodeDecodeError due to non-ascii characters in the author name.
Address this by explicitly using utf-8 when building the Cc list.
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
It is convenient to be able to deal with checkpatch warnings in the same
way as build warnings. Tools such as emacs and kate can quickly locate
the source file and line automatically.
To achieve this, adjust the format to match the C compiler, and output to
stderr.
Signed-off-by: Simon Glass <sjg@chromium.org>
In python 3.x StringIO is no longer a module, and the class can instead
be found in the io module. Adjust the code in the doctest input to
account for both.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
In python 3.x the iteritems() method has been removed from dictionaries,
and the items() method does effectively the same thing. On python 2.x
using items() is a little less efficient since it involves copying data,
but as speed isn't a concern in this code switch to using items() anyway
for simplicity.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
In python 3.x module names used in import statements are case sensitive,
and the configparser module is named in all lower-case. Import it as such
in order to avoid errors when running with python 3.x.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
Syntax for exception handling is a little more strict in python 3.x.
Convert all uses to a form accepted by both python 2.x & python 3.x.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
In python 3.x, print must be used as a function call. Convert all print
statements to the function call style, importing from __future__ where
we print with no trailing newline or print to a file object.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
In preparation for running on python 3.x, which will refuse to run
scripts which mix tabs & spaces for indentation, replace 2 tab
characters present in series.py with spaces.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Acked-by: Simon Glass <sjg@chromium.org>
Output which does not include a newline will not be displayed unless
flushed. Add a flush to ensure that it becomes visible.
Signed-off-by: Simon Glass <sjg@chromium.org>
When tools want to display information of varying levels of importance, it
helps to provide the user with control over the verbosity of these messages.
Progress messages work best if they are displayed and then removed from the
display when no-longer relevant.
Add a new tout library (terminal out) to handle these tasks.
Signed-off-by: Simon Glass <sjg@chromium.org>
For tools which want to use input files and temporary output, it is useful
to have the handling of these dealt with in one place. Add a new library
which allows input files to be read, and output files to be written, all
based on a common directory structure.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is more useful to have this method raise an error when something goes
wrong. Make this the default and adjust the few callers that don't want to
use it this way.
Signed-off-by: Simon Glass <sjg@chromium.org>
The following python error:
Traceback (most recent call last):
File "./tools/patman/patman", line 144, in <module>
series = patchstream.FixPatches(series, args)
File "./tools/patman/patchstream.py", line 477, in FixPatches
commit = series.commits[count]
IndexError: list index out of range
is seen when:
- 'END' is missing in those tags
- those tags are put in the last part in a commit message
- the commit is not the last commit of the series
Add testing logic to see if a new commit starts.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
'Series-changes' uses blank line to indicate its end. If that is
missing, series internal state variable 'in_change' may be wrong.
Correct its state.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
If 'END' is missing in a 'Cover-letter' section, and that section
happens to show up at the very end of the commit message, and the
commit is the last commit of the series, patman fails to generate
cover letter for us. Handle this in CloseCommit of patchstream.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
'Cover-letter', 'Series-notes' and 'Commit-notes' tags require an
'END' to be put at the end of its section. If we forget to put an
'END' in those sections, and these sections are followed by another
patman tag, patman generates incorrect patches. This adds codes to
handle such scenario.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
Like other patman tags, use a new variable cover_match to indicate
a match for 'Cover-letter'.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
At present if you try to use buildman with the branch 'test' it will
complain that it is unsure whether you mean the branch or the directory.
This is a feature of the 'git log' command that buildman uses. Fix it
by resolving the ambiguity.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
It is convenient to install symlinks to buildman and patman in the search
patch, such as /usr/local/bin. But when this is done, the -H option fails to
work because it looks in the directory containing the symlink instead of its
target. Fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Joe Hershberger <joe.hershberger@ni.com>
A patman series with a 'Series-notes' section causes
buildman to crash with:
self.series.notes += self.section
TypeError: cannot concatenate 'str' and 'list' objects
Fix by initializing series.notes as a one-element array
rather than a scalar.
Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Correct spelling of "U-Boot" shall be used in all written text
(documentation, comments in source files etc.).
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Heiko Schocher <hs@denx.de>
Reviewed-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Minkyu Kang <mk7.kang@samsung.com>
Add option to create threaded series of patches.
With it, it will be possible to create patch threads like this:
[PATCH 0/10] Add support for time travel
[PATCH 1/10] Add Flux Capacitor driver
[PATCH 2/10] Add Mr. Fusion driver
(...)
Internally it will call git send-email with --thread option
Signed-off-by: Mateusz Kulikowski <mateusz.kulikowski@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
This reverts commit 19b4a33698.
Since that commit, patman generates useless patches for file removal;
"git format -D" prints only the header but not the diff when deleting
files, and "git am" always refuses such patches.
The following is the quotation from "man git-format-patch":
-D, --irreversible-delete
Omit the preimage for deletes, i.e. print only the header but
not the diff between the preimage and /dev/null. The resulting
patch is not meant to be applied with patch nor git apply; this
is solely for people who want to just concentrate on reviewing
the text after the change. In addition, the output obviously
lack enough information to apply such a patch in reverse, even
manually, hence the name of the option.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Simon Glass <sjg@chromium.org>
Commit 488d19c (patman: add distutils based installer) has the side effect
of making patman run twice with each invocation. Fix this by checking for
'main program' invocation in patman.py. This is good practice in any case.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Chris Packham <judge.packham@gmail.com>
To make it easier to use patman on other projects add a distutils style
installer. Now patman can be installed with
cd u-boot/tools/patman && python setup.py install
There are also the usual distutils options for creating source/binary
distributions of patman.
Tested-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Chris Packham <judge.packham@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
For the local project, we may specified format.subjectprefix setting.
Then the patch will be formated as [Project_prefix][PATCH].
But patman will not check this setting. It will remove the
format.subjectprefix.
So This patch will let patman check this setting and add it as a
project prefix.
Signed-off-by: Josh Wu <josh.wu@atmel.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
We should read this file to obtain a set of aliases. This reduces the need
to create them in the ~/.patman file.
This feature did exist in some version of patman, and is mentioned in the
help but it did not find its way upstream.
Reported-by: Graeme Russ <gruss@tss-engineering.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
This causes an error when trying to build a local branch which has a local
branch as its upstream.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Add an explanation for how to set up git so that patman can find the alias
file. Fix up the get_maintainers message too.
Reported-by: Scott Wood <scottwood@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
True commit lines start at column zero. Anything that is indented
is part of the commit message instead. I noticed this by trying to
run buildman with commit e3a4facdfc
as master, which contained a reference to a Linux commit inside
the commit message. ProcessLine saw that as a genuite commit
line, and thus buildman tried to build it, and died with an
exception because that SHA is not present in the U-Boot tree.
Signed-off-by: Scott Wood <scottwood@freescale.com>
Acked-by: Simon Glass <sjg@chromium.org>
When run with the --dry-run argument patman prints out information
showing what it would do. This information currently doesn't line up
with what patman/git send-email really do. Some basic examples:
- If an email address is addressed via "Series-cc" and "Patch-cc" patman
shows that email address would be CC-ed two times.
- If an email address is addressed via "Series-to" and "Patch-cc" patman
shows that email address would be sent TO and CC-ed.
- If an email address is addressed from a combination of tag aliases,
get_maintainer.pl output, "Series-cc", "Patch-cc", etc patman shows
that the email address would be CC-ed multiple times.
Patman currently does try to send duplicate emails like the --dry-run
output shows, but "git send-email" intelligently removes duplicate
addresses so this patch shouldn't change the non-dry-run functionality.
Change patman's output and email addressing to line up with the
"git send-email" logic. This trims down patman's dry-run output and
prevents confusion about what patman will do when emails are actually
sent.
Signed-off-by: Peter Tyser <ptyser@xes-inc.com>
Acked-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
Adjust the -b flag to permit a range expression as well as a branch.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Tested-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Buildman normally obtains the upstream commit by asking git. Provided that
the branch was created with 'git checkout -b <branch> <some_upstream>' then
this normally works.
When there is no upstream, we can try to guess one, by looking up through
the commits until we find a branch. Add a function to try this and print
a warning if buildman ends up relying on it.
Also update the documentation to match.
Signed-off-by: Simon Glass <sjg@chromium.org>
Suggested-by: Wolfgang Denk <wd@denx.de>
The get_maintainers script is a useful default, but sometimes is copies
too many people, or takes a long time to run.
Add an option to disable it and update the README.
Signed-off-by: Simon Glass <sjg@chromium.org>
For an occasional user of patman some failures are not obvious: for
instance when checkpatch reports warnings, the dry run still reports
that the email would be sent. If it is not dry run, the warnings are
shown on the screen, but it is not clear that the email was not sent.
Add some code to report failure to send email explicitly.
Tested by running the script on a patch with style violations,
observed error messages in the script output.
Signed-off-by: Vadim Bendebury <vbendeb@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Tags like Series-version are normally expected to appear once, and with a
unique value. But buildman doesn't actually look at these tags. So ignore
conflicts.
This allows bulidman to build a branch containing multiple patman series.
Reported-by: Steve Rae <srae@broadcom.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
For reasons that are not well-understood, GetMetaDataForList() can end up
adding to an existing series even when it appears that it should be
starting a new one.
Change from using a default constructor parameter to an explicit one, to
work around this problem.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a test point for the command module. This allows tests to emulate
the execution of commands. This provides more control (since we can make
the fake 'commands' do whatever we like), makes it faster to write tests
since we don't need to set up as much environment, and speeds up test
execution.
Signed-off-by: Simon Glass <sjg@chromium.org>
RunPipe() currently pipes the output of stdout and stderr to a pty, but
this is not the intended behaviour. Fix it.
Signed-off-by: Simon Glass <sjg@chromium.org>
When running unit tests we don't want output to go to the terminal.
Provide a way of collecting it so that it can be examined by test code
later.
Signed-off-by: Simon Glass <sjg@chromium.org>
patman collects tags that it sees in the commit and places them nicely
sorted at the end of the patch. However, this is not really necessary and
in fact is apparently not desirable.
Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
In a headless environment the pager can apparently hang. We don't want a
pager anyway so let's request that none be used.
Reported-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
It seems that this is no longer needed, since checkpatch.pl will catch
whitespace problems in patches. Also the option is not widely used, so
it seems safe to just remove it.
Suggested-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
It seems that doctest behaves differently now, and some of the unit tests
do not run. Adjust the tests to work correctly.
./tools/patman/patman --test
<unittest.result.TestResult run=10 errors=0 failures=0>
Signed-off-by: Simon Glass <sjg@chromium.org>
This option is currently not supported, but needs to be, for buildman to
operate as expected.
Reported-by: York Sun <yorksun@freescale.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
"patman [options]" is displayed by default.
Append the rest of help messages to parser.usage
instead of replacing it.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
In Python, sys.exit() function can also take an object other
than an integer.
If an integer is given to the argument, Python exits with the return
code of it. If a non-integer argument is given, Python outputs it
to stderr and exits with the return code of 1.
That means,
print >> sys.stderr, "Blah Blah"
sys.exit(1)
is equivalent to
sys.exit("Blah Blah")
The latter is a useful shorthand.
Note:
Some error messages in Buildman and Patman were output to stdout.
But they should go to stderr. They are also fixed by this commit.
This is a nice side effect.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Acked-by: Simon Glass <sjg@chromium.org>
Older versions of git (e.g. Ubuntu 10.04) do not support this flag. By
default they do not decorate. So only enable this flag when supported.
Suggested-by: Tom Rini <trini@ti.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
If Series-to tag is missing, Patman exits with a message
"No recipient".
This is just annoying for those who had already added
sendemail.to configuration.
I guess many developers have
[sendemail]
to = u-boot@lists.denx.de
in their .git/config because the 'To: u-boot@lists.denx.de' field
should always be added when sending patches.
That seems more reasonable rather than adding
'Series-to: u-boot@lists.denx.de' to every patch series.
Patman should exit only when both Series-to tag and sendemail.to
configuration are mising.
Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Simon Glass <sjg@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
When patman applies the patches it checks out a new branch, uses 'git am'
to apply the patches one by one, and then tries to go back to the old
branch. If you try this when the branch is 'undefined', this doesn't work
as patman cannot restore the correct branch after applying the patches.
It seems that 'undefined' is created by git and is persistent after it is
created, so that you can end up on quite an old branch.
Add a check for the 'undefined' branch to avoid this.
Reported-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Signed-off-by: Simon Glass <sjg@chromium.org>
There is an unfortunate bug in the signoff suppression logic. The first
pass is performed with 'git log', and all signoffs are added to the
supression set, such that the second time (when processing the real
patches) we always suppress the signoffs.
Correct this by only suppressing signoffs in the second pass.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Michal Simek <monstr@monstr.eu>
Tested-by: Andreas Bießmann <andreas.devel@googlemail.com>
Add a new Patch-cc: tag which performs the service now provided by
the Cc: tag. The Cc: tag is interpreted by git send-email but
ignored by patman.
So now:
Cc: patman does nothing. (git send-email can cc patches)
Patch-cc: patman Cc's patch and removes this tag from the patch
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes a commit should have notes enclosed with it rather
than withing the cover letter -- possibly even because there
is no cover letter. Add a 'Commit-notes' tag, similar to the
'Series-notes' one; lines between this tag and the next END
line are inserted in the patch right after the '---' commit
delimiter.
Change-Id: I01e99ae125607dc6dec08f3be8a5a0b37f0a483d
Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Signed-off-by: Simon Glass <sjg@chromium.org>
(Updated README)
adjust instructions for the invocation of Patman's self test: the -t
flag appears to have a different meaning now, refer to the --test option
for the builtin unit test; adjust a directory location and make sure to
run the file which resides in the source directory
Signed-off-by: Gerhard Sittig <gsi@denx.de>
Acked-by: Simon Glass <sjg@chromium.org>
Patman requires python 2.7.4 to run but it doesn't
need to be placed in /usr/bin/python.
Use env to ensure that the interpreter used is
the first one on environment's $PATH on system
with several versions of Python installed.
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Tom Rini <trini@ti.com>
Acked-by: Simon Glass <sjg@chromium.org>
Conflicting tags can prevent buildman from building two series which exist
one after the other in a branch. There is no reason not to allow this sort
of workflow with buildman, so ignore conflicting tags in buildman.
Change-Id: I2231d04d8684fe0f8fe77f8ea107e5899a3da5e8
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@ti.com>
The git config parameter log.decorate is quite useful when working with git.
Patman, however can not handle the decorated output when parsing the commit.
To prevent this use the '--no-decorate' switch for git-log.
Signed-off-by: Andreas Bießmann <andreas.devel@googlemail.com>
Acked-by: Simon Glass <sjg@chromium.org>
For some series with lots of changes it is annoying that duplicate change
log items are not caught. It is also helpful sometimes to sort the change
logs.
Add a Series-process-log tag to enable this, which can be placed in a
commit to control this.
The change to the Cc: line is to fix a checkpatch warning.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
Especially with the Linux kernel, it takes a long time (a minute or more)
to test-apply the patches, so patman becomes significantly less useful.
The only real problem that is found with this apply step is trailing spaces.
Provide a -a option to skip this step, for those working with clean patches.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
Patman's regular expression for detecting the start of a
commit in a git log was a little simplistic and could be
confused if the git log itself had the word "commit" as
the start of a line (as this commit does). Make patman
a little more robust.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Often it happens that patches include tags which don't have aliases. It
is annoying that patman fails in this case, and provides no option to
continue other than adding empty tags to the .patman file.
Correct this by adding a '-t' option to ignore tags that don't exist.
Print a warning instead.
Since running the tests is not a common operation, move this to --test
instead, to reserve -t for this new option.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
These tags are used by Gerrit, so let's ignore all of them.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
A few of the help messages are not quite right, and there is a typo
in the README. Fix these.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
This comment is less than helpful. Since multiple tags are supported, add
an example of how multiple tags work.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
At present something like:
Revert "arm: Add cache operations"
will try to use
Revert "arm
as a tag. Clearly this is wrong, so fix it.
If the revert is intended to be tagged, then the tag can come before
the revert, perhaps. Alternatively the 'Cc' tag can be used in the commit
messages.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
checkpatch has a new type of warning, a 'CHECK'. At present patman fails
with these, which makes it less than useful.
Add support for checks, making it backwards compatible with the old
checkpatch.
At the same time, clean up formatting of the CheckPatches() output,
fix erroneous "internal error" if multiple patches have warnings and
be more robust to new types of problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
The cover letter is sent to everyone who is on the Cc list for any of
the patches in the series. Sometimes it is useful to send just the cover
letter to additional people, so that they are aware of the series, but
don't need to wade through all the individual patches.
Add a new Cover-letter-cc tag for this purpose.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Doug Anderson <dianders@chromium.org>
Some versions of git don't seem to prompt you for the message ID that
your series is in reply to. Allow specifying this from the command
line.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Although "Reviewed-by:" is a tag that gerrit adds, it's also a tag
used by upstream. Stripping it is undesirable. In fact, we should
treat it as important.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Reviewed-by: Otavio Salvador <otavio@ossystems.com.br>
Acked-by: Simon Glass <sjg@chromium.org>
We normally read from the current branch, but buildman will need to look
at commits from another branch. Allow the metadata to be read from any
list of commits, to provide this flexibility.
Signed-off-by: Simon Glass <sjg@chromium.org>
Make raise_on_error a parameter so that we can control which commands
raise and which do not. If we get an error reading the alias file, just
continue.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than returning a list of things, return an object. That makes it
easier to access the returned items, and easier to extend the return
value later.
Signed-off-by: Simon Glass <sjg@chromium.org>
This adds a new library on top of subprocess which permits access to
the subprocess output as it is being generated. We can therefore
give the illusion that a process is running independently, but still
monitor its output so that we know what is going on.
It is possible to display output on a terminal as it is generated
(a little like tee). The supplied output function is called with all
stdout/stderr data as it arrives.
Signed-off-by: Simon Glass <sjg@chromium.org>
Rather than the rather dull colours, use bright versions which normally
look better and are easier to read.
Signed-off-by: Simon Glass <sjg@chromium.org>
It is easy to detect whether or not the process is connected to a terminal,
or piped to a file. Disable ANSI colours automatically when output is
not to a terminal.
Signed-off-by: Simon Glass <sjg@chromium.org>
To make it usable in git trees not providing a patch checker
implementation, add a command line option, allowing to suppress patch
check. While we are at it, sort debug options alphabetically.
Also, do not raise an exception if checkpatch.pl is not found - just
print an error message suggesting to use the new option, and return
nonzero status.
. unit test passes:
$ ./patman -t
<unittest.result.TestResult run=7 errors=0 failures=0>
. successfully used patman in the autotest tree to generate a patch
email (with --no-check option)
. successfully used patman in the u-boot tree to generate a patch
email
. `patman --help' now shows command line options ordered
alphabetically
Signed-off-by: Vadim Bendebury <vbendeb@chromium.org>
Acked-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
There are cases that we want to support different settings (or maybe
even different aliases) for different projects. Add support for this
by:
* Adding detection for two big projects: U-Boot and Linux.
* Adding default settings for Linux (U-Boot is already good with the
standard patman defaults).
* Extend the new "settings" feature in .patman to specify per-project
settings.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
This patch adds support for a [settings] section in the .patman file.
In this section you can add settings that will affect the default
values for command-line options.
Support is added in a generic way such that any setting can be updated
by just referring to the "dest" of the option that is passed to the
option parser. At the moment options that would make sense to put in
settings are "ignore_errors", "process_tags", and "verbose". You
could override them like:
[settings]
ignore_errors: True
process_tags: False
verbose: True
The settings functionality is also used in a future change which adds
support for per-project settings.
Signed-off-by: Doug Anderson <dianders@chromium.org>
For Linux the best way to figure out where to send a patch is with the
"get_maintainer.pl" script. Add support for calling it from patman.
Support is added unconditionally for "scripts/get_maintainer.pl" in
case it is helpful for any other projects.
Signed-off-by: Doug Anderson <dianders@chromium.org>
If we're sending a cover letter make sure to CC everyone that we're
CCing on each of the individual patches.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Currently we go through and generate the CC list for patches twice.
This gets slow when (in a future CL) we add a call to
get_maintainer.pl on Linux. Instead of doing things twice, just cache
the CC list when it is first generated.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
The Linux kernel stores checkpatch.pl in the scripts directory. Add
that to the search path to make things more automatic for kernel
development.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Several of the patman doctests assume that patman was run with:
./patman
Fix them so that they work even if patman is run with just "patman"
(because patman is in the path).
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
The patman test code was failing because some extra spaces got
stripped when it was applied. These spaces are critical to the test
code working.
Signed-off-by: Doug Anderson <dianders@chromium.org>
Acked-by: Simon Glass <sjg@chromium.org>
Often a particular patch may change only for some versions of a series.
For versions where there is no change, issue a change log indicating
that (for example 'Changes in v4: None').
For such lines, don't add a blank line afterwards, to conserve space.
Use list.insert() instead of list = [item] + list.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Stefan Roese <sr@denx.de>
The BRANCH= tag can be used to indicate the destination branch for a
commit. Ignore this tag.
Also ignore the gerrit 'Commit-Ready:' tag.
Signed-off-by: Simon Glass <sjg@chromium.org>
Colored logs confuse patman when analyzing logs.
Add --no-color option in git log commands in case
the default config has color.
Signed-off-by: Albert ARIBAUD <albert.u.boot@aribaud.net>
Acked-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Tom Rini <trini@ti.com>
Sometimes we don't get a valid filename or line number from checkpatch.pl,
for example if the patch is in a bad format. Deal with this by using a
default value, rather than a stack trace.
Signed-off-by: Simon Glass <sjg@chromium.org>
Sometimes it is possible to forget the name of the branch you used to
generate an upstream series. To assist with this, add an optional
patman does not use this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Specially when many revisions are need for a patchset, the most
interesting information is about the last set of changes so we output
the changelog in reverse order to easy identification of most recent
change set.
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Acked-by: Simon Glass <sjg@chromium.org>
In case an address is listed in the To list, those will be skipped on
Cc list or user might end with a duplicated message.
This fixes the case when a tag points to same address used as series
destination thus avoiding duplicated sending.
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
When a patchset had a RFC series, a v1 might have a changelog of
changes done since the RFC. The patch changes the range checked for
changelog and allow it to start for version 1.
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Acked-by: Simon Glass <sjg@chromium.org>
Currently patman assumes that there should be only one Signoff line
and this is obviously incorrect: we often have to work with patches
containing other people signoffs. Moreover, it's really desirable
to preserve the comments between signoffs.
So until some sophisticated signoff processing will be developed I
suggest just don't mess with signoffs at all and treat them like
plain text lines. The only drawback I've found so far is the case
where you have a patch with someones else signoff but not yours and
also have to patman tags under signoff line. In this case you will
get extra empty line between signoffs.
Signed-off-by: Ilya Yanok <ilya.yanok@cogentembedded.com>
Don't try to sort and uniq changelog entries as this breaks
multiline entries. It will be better to add some real multi-line
support but for now just preserve the entries as is.
Signed-off-by: Ilya Yanok <ilya.yanok@cogentembedded.com>
We already got all changes from git log output and the comment
to the ProcessLine function clearly states that 'patch' mode
is not for scanning tags.
Signed-off-by: Ilya Yanok <ilya.yanok@cogentembedded.com>
Changes may end in '---' line or Signoff line (generated by
git format-patch) in case of Series-changes: lines being
the last ones in commit message. So detect it properly.
Signed-off-by: Ilya Yanok <ilya.yanok@cogentembedded.com>
patman shouts when it couldn't find a $(HOME)/.patman file.
Handle it in a sane way by creating a new one for the user.
It looks for a user.name and user.email in the global .gitconfig
file, waits for the user input if it can't find there. Update the
same in the README
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>
Acked-by: Simon Glass <sjg@chromium.org>
Cc: Simon Glass <sjg@chromium.org>
Cc: Wolfgang Denk <wd@denx.de>
Move the config file from ~/.config/patman to ~/.patman as it is
more appropriate to have it there. Update the same in the README.
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>
Cc: Simon Glass <sjg@chromium.org>
Cc: Wolfgang Denk <wd@denx.de>
Acked-by: Simon Glass <sjg@chromium.org>
What is this?
=============
This tool is a Python script which:
- Creates patch directly from your branch
- Cleans them up by removing unwanted tags
- Inserts a cover letter with change lists
- Runs the patches through checkpatch.pl and its own checks
- Optionally emails them out to selected people
It is intended to automate patch creation and make it a less
error-prone process. It is useful for U-Boot and Linux work so far,
since it uses the checkpatch.pl script.
It is configured almost entirely by tags it finds in your commits.
This means that you can work on a number of different branches at
once, and keep the settings with each branch rather than having to
git format-patch, git send-email, etc. with the correct parameters
each time. So for example if you put:
in one of your commits, the series will be sent there.
See the README file for full details.
END
Signed-off-by: Simon Glass <sjg@chromium.org>
