Commit graph

1505 commits

Author SHA1 Message Date
Nick Sweeting
64f92af60b working setuptools package distribution 2019-03-27 15:39:30 -04:00
Nick Sweeting
93216a3c3e new version handling and absolute imports 2019-03-27 15:35:13 -04:00
Nick Sweeting
bc1bc9fe02 htmldecode all urls and titles during parsing 2019-03-27 15:33:59 -04:00
Nick Sweeting
b1b0c8d1c5 show prettier failure output during link archiving 2019-03-27 15:33:12 -04:00
Nick Sweeting
ea695b8bef remove dataclass for ArchiveIndex in favor of plain dict to simplify schema file 2019-03-27 15:32:39 -04:00
Nick Sweeting
88721512d4 more detailed parsing and indexing cli output 2019-03-27 15:16:53 -04:00
Nick Sweeting
b9839500b2 make archivebox use current directory as OUTPUT_DIR by default 2019-03-27 15:15:51 -04:00
Nick Sweeting
0ad8d51f0e remove base32 crockford in favor of dependency 2019-03-27 15:14:47 -04:00
Nick Sweeting
2e8008009e make main compatible with setuptools entrypoint api 2019-03-27 15:14:22 -04:00
Nick Sweeting
79b319a48c make archive.py runnable as package and as module 2019-03-27 15:13:47 -04:00
mlazana
8502fa5cc3 config.py: update function exclude_blacklisted(links) 2019-03-27 20:10:05 +02:00
mlazana
a3705e31c6 Merge remote-tracking branch 'upstream/master' 2019-03-27 20:03:07 +02:00
Nick Sweeting
dff830196f add setup.py 2019-03-27 11:40:00 -04:00
Nick Sweeting
002206d4fe remove distutils in favor of shutil 2019-03-27 11:39:51 -04:00
Nick Sweeting
e1a1ea25dd add readability script 2019-03-27 11:39:28 -04:00
Nick Sweeting
7ae680a461 Merge branch 'master' into dev 2019-03-27 10:36:29 -04:00
Nick Sweeting
0835baf435 show link titles in black 2019-03-27 10:31:44 -04:00
Nick Sweeting
407484b91e show progress during main index writing 2019-03-27 10:31:22 -04:00
Nick Sweeting
19e1c35f1a support loading link indexes with extra keys 2019-03-27 10:31:07 -04:00
Nick Sweeting
e797a7a5f3
Merge pull request #193 from sbrl/patch-1
wget: Remove unsupported compression option
2019-03-27 04:49:10 -04:00
Nick Sweeting
6bec1709f6
Update config.py 2019-03-27 04:48:38 -04:00
Nick Sweeting
98870ba428 fit bigger titles with tighter spacing 2019-03-27 03:56:38 -04:00
Nick Sweeting
f7aa082dbf css tweaks 2019-03-27 03:52:50 -04:00
Nick Sweeting
d2cf260e7c css tweaks 2019-03-27 03:52:38 -04:00
Nick Sweeting
2ed91fe429 new details page design 2019-03-27 03:49:39 -04:00
Nick Sweeting
c9c5b04df0 full type-hinting coverage 2019-03-26 23:25:07 -04:00
Nick Sweeting
ab09560f14 working runtime type casting and enforcement for a wide range of types 2019-03-26 22:26:21 -04:00
Starbeamrainbowlabs
ab72a2dad1
Try to add --compression back again 2019-03-26 23:38:50 +00:00
Starbeamrainbowlabs
51859731d3
config.py: add WGET_AUTO_COMPRESSION 2019-03-26 23:34:45 +00:00
Nick Sweeting
0d8a076c1f add base32 crockford dependency 2019-03-26 19:21:48 -04:00
Nick Sweeting
25a107df43 switch to dataclasses, working Link type hints everywhere 2019-03-26 19:21:34 -04:00
Starbeamrainbowlabs
ed7ad04fd9
wget: Remove unsupported compression option
--compression=auto is not supported on Raspberry Pis. This PR removes it from ArchiveBox.
2019-03-26 15:46:57 +00:00
Nick Sweeting
346811fb78 ignore mypy cache 2019-03-26 05:35:20 -04:00
Nick Sweeting
76abc58135 switch to strict type hints with NamedTuples instead of dicts 2019-03-26 05:33:34 -04:00
Nick Sweeting
0a44779b21 save command versions in archive results 2019-03-26 05:32:16 -04:00
Nick Sweeting
52871f42e7 cleaner config with type hints 2019-03-26 05:31:27 -04:00
Nick Sweeting
4f8c99011a fix terminal resizing making progress bar go crazy 2019-03-26 05:30:23 -04:00
Nick Sweeting
63abc8980c add mypy type hints 2019-03-26 03:20:41 -04:00
Nick Sweeting
58c9b47d43 fix rss parsing when items have newlines between them 2019-03-25 16:27:55 -04:00
mlazana
81d846427e fix comments in links.py 2019-03-24 19:04:22 +02:00
mlazana
4d10568477 exclude links that are in blacklist 2019-03-24 14:40:26 +02:00
mlazana
417ee9e302 add env variable URL_BLACKLIST 2019-03-23 21:27:41 +02:00
Nick Sweeting
c48b1bbb3c
Merge pull request #192 from saritchie/master
Spelling error in ReadMe
2019-03-22 23:10:45 -04:00
Nick Sweeting
fb8d4e9369 fix title None error 2019-03-22 23:10:11 -04:00
Nick Sweeting
c28c0d0e71 enable gzip auto decompression 2019-03-22 23:01:01 -04:00
Nick Sweeting
f7a0568a6c better chrome options loading 2019-03-22 23:00:53 -04:00
Nick Sweeting
ef865dd76a better stats collection and printing 2019-03-22 23:00:43 -04:00
Nick Sweeting
4c499d77b6 move dependency checking into config file 2019-03-22 22:05:45 -04:00
Nick Sweeting
096832210c better logging totals counts 2019-03-22 21:38:24 -04:00
Nick Sweeting
d06775923b move latest to derived data using history 2019-03-22 21:38:08 -04:00