Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-26 06:00:22 +00:00
Code Issues Projects Releases Packages Wiki Activity

Quick Fix for Wrong Argument Splitting on wget

Browse source 
When setting the user agent, the resulting CLI arguments for wget where `['wget', '--timestamping', '--adjust-extension', '--no-parent', '--page-requisites', '--convert-links', '-', '-', 'u', 's', 'e', 'r', '-', 'a', 'g', 'e', 'n', 't', '=', '"', 'L', 'y', 'n', 'x', '"', 'https://example.org']`, with this fix it turns into `['wget', '--timestamping', '--adjust-extension', '--no-parent', '--page-requisites', '--convert-links', '--user-agent="Lynx"', '', 'https://example.org']`
This commit is contained in:
Tim Schuster 2017-07-08 15:19:43 +02:00 committed by GitHub
parent 4bc3698ee3
commit ba8307c27e
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
fetch.py
View file

@ -33,7 +33,7 @@ def fetch_wget(out_dir, link, overwrite=False, requisites=True, timeout=TIMEOUT)
CMD = [
*'wget --timestamping --adjust-extension --no-parent'.split(' '), # Docs: https://www.gnu.org/software/wget/manual/wget.html
*(('--page-requisites', '--convert-links') if requisites else ()),
*(('--user-agent="{}"'.format(WGET_USER_AGENT)) if WGET_USER_AGENT else ()),
*(('--user-agent="{}"'.format(WGET_USER_AGENT), '') if WGET_USER_AGENT else ()),
link['url'],
]
end = progress(timeout, prefix=' ')