Commit graph

307 commits

Author SHA1 Message Date
Simon Glass
f31e83d6cf binman: Handle tool paths containing '~' correctly
At present if CROSS_COMPILE contains a tilde, such as
~/.buildman-toolchains/gcc-7.3.0-nolibc/i386-linux/bin/i386-linux-gcc
then binman gives a confusing error:

   binman: Error 255 running '~/..buildman-toolchains/gcc-7.3.0- ...

Fix this by expanding it out before running the tool.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-30 13:14:43 -07:00
Simon Glass
fcbec650e6 patman: Add a Series-patchwork-url option
Add a commit tag to allow the Patchwork URL to be specified in a commit.
This can be handy for when you submit code to multiple projects but don't
want to use the -p option.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass
a55be354c0 patman: Add a setting for the Patchwork URL
Add an argument to allow specifying the the patchwork URL. This also adds
this feature to the settings file, either globally, or on a per-project
basis.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass
7cbf02e94d patman: Allow specifying the patchwork URL
Add a new argument to allow the URL of the patchwork server to be
speciified. For now this is hard-coded in the main file, but future
patches will move it to the settings file.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass
3145b63513 patman: Update defaults in subparsers
At present values from the settings file are only applied to the main
parser. With the new parser structure this means that some settings are
ignored.

Update the implementation to set defaults across the main parser and all
subparsers. Also fix up the comments, since ArgumentParser is being used
now.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass
53336e6ca8 patman: Correct Change-Ids error message args
The arguments of this error are incorrectly formatted. Fix it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass
4600767d29 patman: Refactor how the default subcommand works
At present patman tries to assume a default subcommand of 'send', to
maintain backwards compatibility. However it does not cope with
arguments added to the default command, so for example 'patman -t'
does not work.

Update the logic to handle this. Also update the CC command to use 'send'
explicitly, since otherwise patman gets confused with the patch-filename
argument.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-14 15:23:41 -07:00
Simon Glass
dc4b2a9770 patman: Support listing comments from patchwork
While reviewing feedback it is helpful to see the review comments on the
command line to check that each has been addressed. Add an option to
support that.

Update the workflow documentation to describe the new features.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
6b3252e230 patman: Support parsing of review snippets
Add support for parsing the contents of a patchwork 'patch' web page
containing comments received from reviewers. This allows patman to show
these comments in a simple 'snippets' format.

A snippet is some quoted code plus some unquoted comments below it. Each
review is from a unique person/email and can produce multiple snippets,
one for each part of the code that attracts a comment.

Show the file and line-number info at the top of each snippet if
available.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
8f9ba3ab56 patman: Support updating a branch with review tags
It is tedious to add review tags into the local branch and errors can
sometimes be made. Add an option to create a new branch with the review
tags obtained from patchwork.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
dc6df972c9 patman: Support checking for review tags in patchwork
Before sending out a new version of a series for review, it is important
to add any review tags (e.g. Reviewed-by, Acked-by) collected by
patchwork. Otherwise people waste time reviewing the same patch
repeatedly, become frustrated and stop reviewing your patches.

To help with this, add a new 'status' subcommand that checks patchwork
for review tags, showing those which are not present in the local branch.

This allows users to see what new review tags have been received and then
add them.

Sample output:
   $ patman status
     1 Subject 1
       Reviewed-by: Joe Bloggs <joe@napierwallies.co.nz>
     2 Subject 2
       Tested-by: Lord Edmund Blackaddër <weasel@blackadder.org>
       Reviewed-by: Fred Bloggs <f.bloggs@napier.net>
     + Reviewed-by: Mary Bloggs <mary@napierwallies.co.nz>
   1 new response available in patchwork

The '+' indicates a new tag. Colours are used to make it easier to read.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
be051c0c77 patman: Detect missing upstream in CountCommitsToBranch
At present if we fail to find the upstream then the error output is piped
to wc, resulting in bogus results. Avoid the pipe and check the output
directly.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
b3348522b7 patman: Improve handling of files
Sometimes warnings are associated with a file and sometimes with the
patch as a whole. Update the regular expression to handle both cases,
even in emacs mode. Also add support for detecting new files.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
a12ad7c940 patman: Allow showing a Commit as a string
Use the subject of the Commit object when printing it out.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
b5e188131f patman: Don't ignore lines starting with hash
These lines can indicate a continuation of an error and should not be
ignored. Fix this.

Fixes: 666eb15e92 ("patman: Handle checkpatch output with notes and code")

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
9994baadc0 patman: Fix spelling of plural for warning
Tidy up the extra 's' when there is only a single warning. Fix the empty
print statement also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
8c17f8c5d2 patman: Convert testBasic() to use an interator
On balance it is easier to use an iterator here, particularly if we need
to insert lines due to new functionality. The only niggle is the need to
keep the previous iterator value around in one case.

Convert this test to use iter().

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
4af9987456 patman: Add some tests for warnings
Add tests that check that warnings are generated when expected.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
7457051e41 patman: Add a test for PatchStream tags
The current functional tests run most of patman. Add a smaller test that
just checks tag handling with the PatchStream class.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
47f62952cc patman: Drop unused signoff member
This is not used. Drop it.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
dffa42c3ef patman: Convert 'Series-xxx' tag errors into warnings
If the Series-xxx tag is not recognised patman currently reports a fatal
error. This is inconvenient if a new feature is later added to patman that
an earlier version does not support.

Report a warning instead, to allow the user to take action if needed, but
still allow operation to proceed.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
313ef5f897 patman: Attach warnings to individual patches
At present warnings are produced across the whole set of patches when
parsing them. It is more useful to associate each warning with the patch
(or commit) that generated it.

Attach warnings to the Commit object and move them out of PatchStream.
Also avoid generating duplicate warnings for the same commit.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
b5cc399053 patman: Move warning collection to a function
Add a new function in PatchStream to collect the warnings generated while
parsing the stream. This will allow us to adjust the logic, such as
dealing with per-commit warnings.

Two of the warnings are in fact internal errors, so change them to raise
and exception.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
1cb1c0fc8d patman: Fix up argument/return docs in patchstream
Add missing documentation and type information. Fix up some missing docs
on exceptions also.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
e3a816b9f4 patman: Drop unused args in patchstream
Drop a few arguments that are not used in functions.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
dd147eda10 patman: Rename variables in patchstream
Some variables are too short or shadow other variables or types. Fix these
to keep pylint3 happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
d93720e138 patman: Rename functions in patchstream
Rename these functions to lower case as per PEP8.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
5769904082 patman: Fix constant style in patchstream
These constants should use upper case. Update them to keep pylint3 happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
d06e55a7c6 patman: Fix indenting in patchstream
Update the indenting to keep pylint3 happy.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
f9e428489b patman: Allow linking a series with patchwork
Add a new Series-links tag to tell patman how to find the series in
patchwork. Each item is the series ID optionally preceded by the series
version that the link refers to. An empty version indicates this is the
latest series.

For example:

   Series-links: 209816 1:203302

Documentation is added in a later patch.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
fca99117a5 patman: Fix remaining pylint3 warnings in func_test
This fixes all but the ones about too many variables/statements.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
366954fdd7 patman: Use capture_sys_output() consistently
One test still uses its own function for capturing output. Modify it to
use the standard one in test_util

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
427b028aeb patman: Fix whitespace errors in func_test
Fix up various indentation and other minor things to make pylint3 happier.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
e21c515817 patman: Update how tests are run
The current instructions are out-of-date. Fix them.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Simon Glass
4a9e578138 patman: Correct operation of -n
This operation was unfortunately broken by a recent change. It is now
necessary to use -i in addition to -n, if there are errors or warnings in
the patches.

Correct this by always showing the summary information.

Fixes: f365375975 ("patman: Move main code out to a control module")
Signed-off-by: Simon Glass <sjg@chromium.org>
2020-11-05 09:11:31 -07:00
Alper Nebi Yasak
b9cca2c57a checkpatch.pl: Make CONFIG_IS_ENABLED(CONFIG_*) an error
CONFIG_IS_ENABLED() takes the kconfig name without the CONFIG_ prefix,
e.g. CONFIG_IS_ENABLED(CLK) for CONFIG_CLK. Make including the prefix
an error in checkpatch.pl so calls in the wrong format aren't
accidentally reintroduced.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-10-14 11:16:34 -04:00
Alper Nebi Yasak
29cc091842 binman: Allow resolving host-specific tools from env vars
This patch lets tools.Run() use host-specific versions with the
for_host keyword argument, based on the host-specific environment
variables (HOSTCC, HOSTOBJCOPY, HOSTSTRIP, etc.).

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:54:13 -06:00
Alper Nebi Yasak
1e4687aa47 binman: Use target-specific tools when cross-compiling
Currently, binman always runs the compile tools like cc, objcopy, strip,
etc. using their literal name. Instead, this patch makes it use the
target-specific versions by default, derived from the tool-specific
environment variables (CC, OBJCOPY, STRIP, etc.) or from the
CROSS_COMPILE environment variable.

For example, the u-boot-elf etype directly uses 'strip'. Trying to run
the tests with 'CROSS_COMPILE=i686-linux-gnu- binman test' on an arm64
host results in the '097_elf_strip.dts' test to fail as the arm64
version of 'strip' can't understand the format of the x86 ELF file.

This also adjusts some command.Output() calls that caused test errors or
failures to use the target versions of the tools they call. After this,
patch, an arm64 host can run all tests with no errors or failures using
a correct CROSS_COMPILE value.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:54:13 -06:00
Alper Nebi Yasak
76de29fc4f buildman: Use git worktrees instead of git clones when possible
This patch makes buildman create linked working trees instead of clones
of the source repository, but keeps updating the older clones of the
repository that might already exist. These worktrees share "everything
except working directory specific files such as HEAD, index, etc." with
the source repository. See the git-worktree(1) manual page for more
information.

If git-worktree isn't available, silently falls back to cloning the
repository.

Signed-off-by: Alper Nebi Yasak <alpernebiyasak@gmail.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested-by: Simon Glass <sjg@chromium.org>
2020-09-22 12:50:43 -06:00
Simon Glass
23552ba142 checkpatch: Don't allow common.h and dm.h in headers
These headers should not be included in other header files. Add a
checkpatch rule and test for this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-08-03 22:19:54 -04:00
Simon Glass
4620d46bf0 patman: Fix up the test comments
Many of the tests have the same comment and two have the same name. Fix
this.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-08-03 22:19:54 -04:00
Simon Glass
347e0f00e8 binman: Re-enable concurrent tests
With the change to absolute imports the concurrent tests feature
unfortunately broke. Fix it.

We cannot easy add a warning, since the output messes up tests which check
the output.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-26 19:59:57 -06:00
Nicolas Boichat
e1db5c9b08 patman: When no tracking branch is provided, tell the user
The user can either count the number of patches, or provide a
tracking branch.

Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Nicolas Boichat
949775689e patman: Make sure sendemail.suppresscc is (un)set correctly
Setting sendemail.suppresscc to all or cccmd leads to --cc-cmd
parameter being ignored, and emails going either nowhere, or
just to the To: line maintainer.

Signed-off-by: Nicolas Boichat <drinkcat@chromium.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Patrick Delaunay
e5ff9ab70a Add information for skipped commit options
The unsupported Commit-xxx option are silently skipped
and removed as 're_remove=Commit-\w*', this patch adds
warning message in this case to detect misspelled issue
for the 2 supported options:
  Commit-notes:
  Commit-changes:

For example: the final 's' is missing (Commit-note:)

NB: no issue for Series-xxx option as only the supported
    options are accepted (see valid_series in series.py)

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Patrick Delaunay
7058dd071a patman: Detect unexpected END
Detect unexpected 'END' line when a section is not detected.

This patch detect issue when tag name for section start is misspelled,
for example 'Commit-note:' for 'Commit-notes:'

  Commit-note:
  ....
  END

Then 'Commit-note:' is removed silently by re_remove = "Commit-\w*:"
but 'END' is kept in commit message.

Signed-off-by: Patrick Delaunay <patrick.delaunay@st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Simon Glass
b1cca9552c binman: Detect when valid images are not produced
When external blobs are missing, show a message indicating that the images
are not functional.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-25 14:46:57 -06:00
Simon Glass
38fdb4cb35 patman: Update errors and warnings to use stderr
When warnings and errors are produced by tools they should be written to
stderr. Update the tout implementation to handle this.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass
4f9f1056ec binman: Allow external binaries to be missing
Sometimes it is useful to build an image even though external binaries are
not present. This allows the build system to continue to function without
these files, albeit not producing valid images.

U-Boot does with with ATF (ARM Trusted Firmware) today.

Add a new flag to binman to request this behaviour.

Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
2020-07-25 14:46:57 -06:00
Simon Glass
32eb66d2d4 binman: Specify the toolpath when running test coverage
At present binman's test coverage runs without a toolpath set. This means
that the system tools will be used. That may not be correct if they are
out of date or missing and this can result in a reduction in test coverage
below 100%.

Provide the toolpath to binman in this case.

Signed-off-by: Simon Glass <sjg@chromium.org>
2020-07-24 19:25:15 -06:00