2278 commits

This branch

This branch

All branches

Author	SHA1	Message	Date
Nick Sweeting	8aee5c408a	improve workflow names for clarity in PR status	2020-12-06 16:26:44 +02:00
Nick Sweeting	6ac48d7c35	tweak warning msg	2020-12-06 02:11:36 +02:00
Nick Sweeting	a0a79cead8	move utils and vendored libs into subfolders	2020-12-06 02:01:18 +02:00
Nick Sweeting	8440858751	bump pip_dist build	2020-12-06 01:38:51 +02:00
Nick Sweeting	8d103687d0	Merge pull request #570 from ArchiveBox/sonic-search	2020-12-05 18:22:17 -05:00
jdcaballerov	172197ae01	refactor: Remove if LENGTH and use text chunker for every input	2020-12-06 01:14:39 +02:00
jdcaballerov	5a6b814c79	Add exception handling for indexable content reader	2020-12-06 01:14:38 +02:00
JDC	b1d70185ed	Increase word_objects for Sonic default config	2020-12-06 01:14:38 +02:00
JDC	15fbd81480	Change MAX_SONIC_TEXT_LENGTH	2020-12-06 01:14:38 +02:00
JDC	db9c2edccc	Add log print for url indexing	2020-12-06 01:14:38 +02:00
JDC	0acf479b70	Partition long strings in chunks for sonic	2020-12-06 01:14:38 +02:00
JDC	caf4660ac8	Add indexing to update command and utilities	2020-12-06 01:14:37 +02:00
JDC	273c9d91c6	Add tag filter to update command	2020-12-06 01:13:39 +02:00
JDC	7903db6dfb	Add ArchiveResult Manager and sorted indexable filter	2020-12-06 01:13:39 +02:00
JDC	23a9beb4e0	Add ignored extensions in ripgrep search	2020-12-06 01:13:39 +02:00
JDC	95382b3812	Add ripgrep rg search backend and set as default	2020-12-06 01:13:39 +02:00
JDC	8484bdb973	Fix add search filter to update	2020-12-06 01:13:39 +02:00
JDC	c5b1b91708	fix: flush_search_index must be called before removing snapshots	2020-12-06 01:13:39 +02:00
JDC	70cc0c1950	Add search filter-type	2020-12-06 01:13:39 +02:00
JDC	4eeedae815	Exception handling for indexing and searching	2020-12-06 01:13:39 +02:00
JDC	0ed53cc117	Add search filter type for update	2020-12-06 01:13:39 +02:00
JDC	9bd40ed7f6	Max out number of queries	2020-12-06 01:13:39 +02:00
JDC	a38e3e0c90	Get searc backend password from env var SEARCH_BACKEND_PASSWORD	2020-12-06 01:13:39 +02:00
JDC	0773f12034	Add sonic to docker-compose	2020-12-06 01:13:38 +02:00
JDC	0f7dba07df	feat: add search filter-type to list command	2020-12-06 01:13:37 +02:00
JDC	fb67d6684c	fix: Return empty QuerySet instead of list	2020-12-06 01:12:47 +02:00
JDC	823df34080	Use QuerySets for search backend API instead of pks	2020-12-06 01:12:47 +02:00
JDC	f383648ffc	Use a generator for snapshot flush from index	2020-12-06 01:12:47 +02:00
JDC	47daa038eb	Implement flush for search backend after remove command	2020-12-06 01:12:47 +02:00
JDC	c2c01af3ad	Add config for search backend	2020-12-06 01:12:47 +02:00
JDC	5f6673c72c	Implement backend architecture for search engines	2020-12-06 01:12:46 +02:00
JDC	b1f70b2197	Initial implementation	2020-12-06 01:12:45 +02:00
Nick Sweeting	7bc13204e6	Merge branch 'master' into v0.5.0	2020-12-05 17:45:16 -05:00
Nick Sweeting	3b280e6b02	Merge pull request #569 from cdvv7788/extract-command-update ... feat: Add --extract flag to update command	2020-12-05 17:43:28 -05:00
Nick Sweeting	a05671f2be	Merge pull request #568 from cdvv7788/#544-update-index-only ... #544 update: --index-only flag	2020-12-05 17:40:22 -05:00
Cristian	35389608d1	feat: Add --extract flag to update command	2020-12-05 12:20:47 -05:00
Cristian	34cad4fe8d	fix: Update function with --index-only flag was not behaving as expected	2020-12-05 12:10:17 -05:00
Nick Sweeting	f467435797	Merge pull request #528 from mAAdhaTTah/pull-from-pocket-api ... Add parser for Pocket API	2020-12-04 22:58:41 -05:00
mAAdhaTTah	ac7ad9e942	Add parser for Pocket API ... Pass a url like `pocket://Username` to import that username's archived Pocket library. Tokens need to be stored in ArchveBox.conf with the following keys: ``` POCKET_CONSUMER_KEY = key-from-custom-pocket-app POCKET_ACCESS_TOKENS = {"YourUsername": "pocket-token-for-app"} ``` `POCKET_ACCESS_TOKENS` MUST be on a single line, or the JSON will be misinterpreted by the parser as a new key/value pair.	2020-12-04 22:54:39 -05:00
Nick Sweeting	04291c4d47	Merge pull request #555 from cdvv7788/cleanup	2020-12-04 20:40:38 -05:00
Nick Sweeting	00dfe2d449	Merge branch 'v0.5.0' into cleanup	2020-12-04 20:40:24 -05:00
Nick Sweeting	69f551736b	Merge pull request #567 from jdcaballerov/remove-legacy-templates	2020-12-04 20:38:08 -05:00
jdcaballerov	54b25d9a81	Linting	2020-12-03 15:59:45 -05:00
jdcaballerov	d4bca80b50	Use uppercase for constants	2020-12-03 15:44:59 -05:00
jdcaballerov	c8d8346e4d	Remove duplicate context after rebase	2020-12-03 15:44:21 -05:00
Nick Sweeting	b186e98cd2	only docker push on release or master commit	2020-12-03 11:33:10 -05:00
jdcaballerov	107c062141	Update README adding submodules setup	2020-12-03 10:01:58 -05:00
jdcaballerov	a1fba9887d	Remove write_html_main_index	2020-12-03 09:25:38 -05:00
jdcaballerov	7f39702bd2	Delete legacy/ folder	2020-12-03 09:17:34 -05:00
jdcaballerov	367b12ba40	Replace legacy templates for django templates	2020-12-03 09:16:18 -05:00