Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-26 22:20:21 +00:00
Code Issues Projects Releases Packages Wiki Activity
3077 commits 6 branches 78 tags 19 MiB
Commit graph

168 commits

Author SHA1 Message Date
Nick Sweeting
2db5e51b54 fix windows shutil not able to handle pathlib 2020-12-11 19:33:18 +02:00
Nick Sweeting
6623497f18 fix MERCURY_PATH in version output when missing 2020-12-11 16:21:33 +02:00
Nick Sweeting
c084e70ea8 fix TEMPLATES_DIR location 2020-12-11 16:21:09 +02:00
Cristian
9745a5ac56 fix: Migrations should be silent when running in setup_django 2020-12-08 18:42:25 -05:00
Cristian
9aac09a5e1 feat: Patch setup_django so we can use an inmemory db in specific commands 2020-12-08 18:42:25 -05:00
JDC
95382b3812 Add ripgrep rg search backend and set as default 2020-12-06 01:13:39 +02:00
JDC
47daa038eb Implement flush for search backend after remove command 2020-12-06 01:12:47 +02:00
JDC
c2c01af3ad Add config for search backend 2020-12-06 01:12:47 +02:00
mAAdhaTTah
ac7ad9e942
Add parser for Pocket API 
Pass a url like `pocket://Username` to import that username's archived Pocket
library. Tokens need to be stored in ArchveBox.conf with the following keys:

```
POCKET_CONSUMER_KEY = key-from-custom-pocket-app
POCKET_ACCESS_TOKENS = {"YourUsername": "pocket-token-for-app"}
```

`POCKET_ACCESS_TOKENS` MUST be on a single line, or the JSON will be
misinterpreted by the parser as a new key/value pair.
 2020-12-04 22:54:39 -05:00
Nick Sweeting
7f2c834ea3 fix check_data_folder mypy types 2020-11-28 01:05:35 -05:00
Nick Sweeting
5e7c2d0ab8 show archivebox and node versions in version cmd output 2020-11-23 20:24:44 -05:00
Nick Sweeting
0e2ccbc10d update urls to new repo path 2020-11-23 02:06:46 -05:00
Nick Sweeting
3f160eab8e correctly handle WGET_AUTO_COMPRESSION failing when wget is missing 2020-11-12 14:28:43 -05:00
Nick Sweeting
22fb9c2ad7 tweak icons 2020-10-31 19:32:43 -04:00
Nick Sweeting
b8bbb75f9c logarithmic progress bars woohoo 2020-10-31 07:57:11 -04:00
Nick Sweeting
ac9e0e356d config fixes 2020-10-31 07:57:11 -04:00
Nick Sweeting
18355dc2c6 clean up config loading in settings and config file layout 2020-10-31 03:08:03 -04:00
Nick Sweeting
27708152d2 wip initial django setup 2019-04-02 16:36:41 -04:00
Nick Sweeting
f4e018ba0c fix a bunch of mypy errors 2019-03-30 20:49:45 -04:00
Nick Sweeting
8283b353f4 fix wget_auto_compression check prining stderr to console 2019-03-30 17:43:44 -04:00
Nick Sweeting
5d0185b6dd Merge branch 'master' into dev 2019-03-30 15:36:54 -04:00
Nick Sweeting
35c05c321f minor bin version checking changes 2019-03-30 15:03:31 -04:00
Nick Sweeting
066b36b6a9
make URL_BLACKLIST empty by default 2019-03-30 14:56:19 -04:00
Nick Sweeting
a26c2fe467 show full version info using flag 2019-03-27 16:44:00 -04:00
Nick Sweeting
93216a3c3e new version handling and absolute imports 2019-03-27 15:35:13 -04:00
Nick Sweeting
b9839500b2 make archivebox use current directory as OUTPUT_DIR by default 2019-03-27 15:15:51 -04:00
mlazana
a3705e31c6 Merge remote-tracking branch 'upstream/master' 2019-03-27 20:03:07 +02:00
Nick Sweeting
7ae680a461 Merge branch 'master' into dev 2019-03-27 10:36:29 -04:00
Nick Sweeting
6bec1709f6
Update config.py 2019-03-27 04:48:38 -04:00
Nick Sweeting
c9c5b04df0 full type-hinting coverage 2019-03-26 23:25:07 -04:00
Starbeamrainbowlabs
51859731d3
config.py: add WGET_AUTO_COMPRESSION 2019-03-26 23:34:45 +00:00
Nick Sweeting
25a107df43 switch to dataclasses, working Link type hints everywhere 2019-03-26 19:21:34 -04:00
Nick Sweeting
52871f42e7 cleaner config with type hints 2019-03-26 05:31:27 -04:00
Nick Sweeting
4f8c99011a fix terminal resizing making progress bar go crazy 2019-03-26 05:30:23 -04:00
mlazana
4d10568477 exclude links that are in blacklist 2019-03-24 14:40:26 +02:00
mlazana
417ee9e302 add env variable URL_BLACKLIST 2019-03-23 21:27:41 +02:00
Nick Sweeting
f7a0568a6c better chrome options loading 2019-03-22 23:00:53 -04:00
Nick Sweeting
4c499d77b6 move dependency checking into config file 2019-03-22 22:05:45 -04:00
Nick Sweeting
69f837bbf6 simplify chrome_user_data_dir default 2019-03-22 21:37:02 -04:00
Nick Sweeting
8f73fdbe09 fix chrome profile precedence order to be equal 2019-03-22 21:31:55 -04:00
noncetonic
28758cf16c
Adds CHROME_USER_AGENT 2019-03-19 10:15:52 -07:00
Nick Sweeting
1c1bc76ac1 add chrome headless option and improve default data dir finding 2019-03-12 17:50:10 -04:00
Nick Sweeting
8319ccf064
add docs link to config.py 2019-03-12 12:48:46 -04:00
Nick Sweeting
32c39d0fd0 cleaner output dir spec in config 2019-03-08 17:51:49 -05:00
Nick Sweeting
2e10f57f6e fix relative links from index files 2019-03-08 17:46:14 -05:00
Nick Sweeting
a74d8410f4 also check for macOS binary defaults 2019-03-08 16:25:42 -05:00
Nick Sweeting
d689264365 add new config and dependency options 2019-02-21 15:47:15 -05:00
Nick Sweeting
d52c9c5304 allow passing COOKIES_FILE to wget 2019-02-21 12:58:51 -05:00
Nick Sweeting
5a7d00a639 fetch page title during archiving process 2019-02-19 01:44:54 -05:00
Nick Sweeting
9eb79258bb check chrome version on startup if using chrome 2019-01-29 17:08:15 -08:00
Nick Sweeting
e60070dbb2 add youtubedl to help str 2019-01-25 17:38:47 -08:00
Nick Sweeting
20de451515 add linux config example 2019-01-23 01:48:04 -05:00
Nick Sweeting
95301c9306 better default config 2019-01-23 01:42:55 -05:00
Nick Sweeting
e1be96e597 working docker-compose with google chrome 2019-01-23 01:08:23 -05:00
Nick Sweeting
f25be8bc24 add chrome user agent example in config 2019-01-20 14:08:33 -05:00
Nick Sweeting
cc8611de83
Update config.py 2019-01-14 22:46:35 -05:00
Nick Sweeting
c42fcd42d7 fetch warc file inline with wget instead of as separate step 2019-01-14 22:43:14 -05:00
Nick Sweeting
300b5c6182 put ArchiveBox and wget version in user agent 2019-01-14 18:17:30 -05:00
Nick Sweeting
cb60bad1d7 disable WARC by default 2019-01-12 20:19:17 -05:00
Nick Sweeting
b650c663a0 re-enable checking SSL by default 2019-01-11 22:48:09 -05:00
Nick Sweeting
f83750c545 cleanup options and make cli flags better for chrome headless timeouts 2019-01-11 22:38:50 -05:00
Nick Sweeting
e8808b0a1f add WARC downloading 2019-01-11 07:02:49 -05:00
Nick Sweeting
a15a331798 fix media download with longer timeout 2019-01-11 06:33:35 -05:00
Nick Sweeting
c33f7ba91c add ability to fetch media 2019-01-11 05:52:29 -05:00
Nick Sweeting
0df098717a make git domains configurable 2019-01-11 05:27:25 -05:00
Nick Sweeting
827e15b31a add git downloading 2019-01-11 05:18:49 -05:00
Nick Sweeting
5b6c768a47 autodetect common chrome binary locations 2019-01-11 04:34:16 -05:00
Nick Sweeting
57d42339a4 rename pip dir archive to archivebox 2018-12-31 20:53:01 -05:00
Renamed from archiver/config.py (Browse further)