Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-30 08:00:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
687 commits 6 branches 78 tags 19 MiB
Commit graph

97 commits

Author SHA1 Message Date
Nick Sweeting
14e66a6909 use derived_link_info more consistently in index generation 2019-03-03 14:10:18 -05:00
Nick Sweeting
552734241b put git clones in a git folder to avoid retries 2019-02-27 15:55:39 -05:00
Nick Sweeting
3eaa76267e fix keyerror domain bug 2019-02-27 15:42:53 -05:00
Nick Sweeting
b4dc80b0a7 when checking link invariants, check for regex match as well 2019-02-27 04:50:29 -05:00
Nick Sweeting
fa6f53f2af used derived info for all derivable info 2019-02-27 04:50:29 -05:00
Nick Sweeting
ef4c446c8b new compiled URL regex with better markdown support 2019-02-27 04:50:29 -05:00
Nick Sweeting
b2c22a73e6 move parsers to global instead of func 2019-02-27 04:50:29 -05:00
Nick Sweeting
af7e8df0eb rename download url func 2019-02-27 04:50:29 -05:00
Nick Sweeting
09d79e55a0 remove derivable link info from links 2019-02-27 04:50:29 -05:00
Nick Sweeting
eb003f6a26 better function naming 2019-02-27 04:50:29 -05:00
Nick Sweeting
328a59749b better remote file downloading cli output messages 2019-02-27 04:50:29 -05:00
Nick Sweeting
b03e9fade8 better link corruption guards, remove title prefetching, save index after run 2019-02-21 17:45:28 -05:00
Nick Sweeting
c95632883e fix without_hash to without_fragment 2019-02-21 16:03:19 -05:00
Nick Sweeting
d689264365 add new config and dependency options 2019-02-21 15:47:15 -05:00
Nick Sweeting
34cfde0c4e dont fetch titles when FETCH_TITLE=False 2019-02-21 12:58:51 -05:00
Nick Sweeting
d52c9c5304 allow passing COOKIES_FILE to wget 2019-02-21 12:58:51 -05:00
Nick Sweeting
935dcac0c7 fix title showing up as None in some UI spots 2019-02-19 02:40:02 -05:00
Nick Sweeting
eff0100971 fix RSS parser bailing out when lines have whitespace before tags 2019-02-19 02:31:53 -05:00
Nick Sweeting
3571ef24e4 fix logic for ONLY_NEW accidentally replacing all links 2019-02-19 02:21:28 -05:00
Nick Sweeting
1b36d5b29c fix pocket timestamps defaulting to now 2019-02-19 01:54:09 -05:00
Nick Sweeting
2c9aad559d use urllib for url parsing instead of hand written string commands 2019-02-19 01:45:19 -05:00
Nick Sweeting
8576a2f061 better parser explanation comment 2019-02-19 01:45:03 -05:00
Nick Sweeting
5a7d00a639 fetch page title during archiving process 2019-02-19 01:44:54 -05:00
Nick Sweeting
bb5879a4f7 fix some parser errors not being caught by bail out process 2019-02-18 23:45:49 -05:00
Nick Sweeting
74b99fe9eb fix import_path None error 2019-02-12 20:04:03 -05:00
Nick Sweeting
e6d5cd4432 ignore robots.txt when using wget 2019-02-06 22:06:36 -08:00
Nick Sweeting
56d382235f better progress output 2019-02-06 22:06:36 -08:00
Nick Sweeting
33ba29ea90 add optional parser error message 2019-02-06 22:06:36 -08:00
Nick Sweeting
ce257949b4 fix double parsing by accident 2019-02-04 22:08:08 -08:00
Nick Sweeting
1ca27a60b2 show dots as progress bar for titles fetching 2019-02-04 22:07:59 -08:00
Nick Sweeting
ee93807a0a tweak wording of parser cli output 2019-02-04 22:07:48 -08:00
Nick Sweeting
ca8f57ef5c add dedicated shaarli rss parser 2019-02-04 21:18:42 -08:00
Nick Sweeting
522b6288c1 hide frequent youtubedl incompatibility 2019-02-04 21:18:17 -08:00
root
fb00f1f336 fix flickering progress bars 2019-02-05 04:45:36 +00:00
Nick Sweeting
bc6de29a5a recreate chunks on each new progress bar 2019-02-04 20:41:35 -08:00
Nick Sweeting
78ffa57c06 ignore hsts as well 2019-02-04 20:37:19 -08:00
Nick Sweeting
637f63dc80 chmod media after downloading 2019-02-04 20:15:34 -08:00
Nick Sweeting
756f31a50b hide common youtubedl failures 2019-02-04 20:08:54 -08:00
Nick Sweeting
2b45792ad8 new test files 2019-02-04 20:04:03 -08:00
Nick Sweeting
265e1bd830 fix parsing links from stdin with whitespace 2019-02-04 20:03:32 -08:00
Nick Sweeting
794c043fa4 fix chrome data dir in wrong folder 2019-02-04 20:03:16 -08:00
Nick Sweeting
9f5441c4f3 quieter media downloads 2019-02-04 20:03:06 -08:00
Nick Sweeting
4e5b466bb2 fix broken wget user agents 2019-02-04 20:02:54 -08:00
Nick Sweeting
d8c7c98c01 fix exception format string 2019-02-04 20:02:40 -08:00
Nick Sweeting
705abda1a5 send ArchiveBox user agent to archive.org 2019-02-04 20:02:33 -08:00
Nick Sweeting
8a9b4d6975 dont git clone urls with hashes 2019-02-04 19:21:08 -08:00
Nick Sweeting
b8e5e1d5bd skip media when folder exists, even if empty 2019-02-04 19:21:00 -08:00
Nick Sweeting
901666bae6 dont show wget errors unless all files failed to download 2019-02-04 19:20:36 -08:00
Nick Sweeting
3ac0efb619 dont parse quotes as part of urls 2019-02-04 18:54:02 -08:00
Nick Sweeting
af8b9b5fdf use parsed timestamps instead of now 2019-02-04 18:53:54 -08:00