821 commits

Author	SHA1	Message	Date
Nick Sweeting	6ac48d7c35	tweak warning msg	2020-12-06 02:11:36 +02:00
Nick Sweeting	a0a79cead8	move utils and vendored libs into subfolders	2020-12-06 02:01:18 +02:00
jdcaballerov	172197ae01	refactor: Remove if LENGTH and use text chunker for every input	2020-12-06 01:14:39 +02:00
jdcaballerov	5a6b814c79	Add exception handling for indexable content reader	2020-12-06 01:14:38 +02:00
JDC	15fbd81480	Change MAX_SONIC_TEXT_LENGTH	2020-12-06 01:14:38 +02:00
JDC	db9c2edccc	Add log print for url indexing	2020-12-06 01:14:38 +02:00
JDC	0acf479b70	Partition long strings in chunks for sonic	2020-12-06 01:14:38 +02:00
JDC	caf4660ac8	Add indexing to update command and utilities	2020-12-06 01:14:37 +02:00
JDC	273c9d91c6	Add tag filter to update command	2020-12-06 01:13:39 +02:00
JDC	7903db6dfb	Add ArchiveResult Manager and sorted indexable filter	2020-12-06 01:13:39 +02:00
JDC	23a9beb4e0	Add ignored extensions in ripgrep search	2020-12-06 01:13:39 +02:00
JDC	95382b3812	Add ripgrep rg search backend and set as default	2020-12-06 01:13:39 +02:00
JDC	8484bdb973	Fix add search filter to update	2020-12-06 01:13:39 +02:00
JDC	c5b1b91708	fix: flush_search_index must be called before removing snapshots	2020-12-06 01:13:39 +02:00
JDC	70cc0c1950	Add search filter-type	2020-12-06 01:13:39 +02:00
JDC	4eeedae815	Exception handling for indexing and searching	2020-12-06 01:13:39 +02:00
JDC	0ed53cc117	Add search filter type for update	2020-12-06 01:13:39 +02:00
JDC	0f7dba07df	feat: add search filter-type to list command	2020-12-06 01:13:37 +02:00
JDC	fb67d6684c	fix: Return empty QuerySet instead of list	2020-12-06 01:12:47 +02:00
JDC	823df34080	Use QuerySets for search backend API instead of pks	2020-12-06 01:12:47 +02:00
JDC	f383648ffc	Use a generator for snapshot flush from index	2020-12-06 01:12:47 +02:00
JDC	47daa038eb	Implement flush for search backend after remove command	2020-12-06 01:12:47 +02:00
JDC	c2c01af3ad	Add config for search backend	2020-12-06 01:12:47 +02:00
JDC	5f6673c72c	Implement backend architecture for search engines	2020-12-06 01:12:46 +02:00
JDC	b1f70b2197	Initial implementation	2020-12-06 01:12:45 +02:00
Nick Sweeting	7bc13204e6	Merge branch 'master' into v0.5.0	2020-12-05 17:45:16 -05:00
Nick Sweeting	3b280e6b02	Merge pull request #569 from cdvv7788/extract-command-update ... feat: Add --extract flag to update command	2020-12-05 17:43:28 -05:00
Cristian	35389608d1	feat: Add --extract flag to update command	2020-12-05 12:20:47 -05:00
Cristian	34cad4fe8d	fix: Update function with --index-only flag was not behaving as expected	2020-12-05 12:10:17 -05:00
mAAdhaTTah	ac7ad9e942	Add parser for Pocket API ... Pass a url like `pocket://Username` to import that username's archived Pocket library. Tokens need to be stored in ArchveBox.conf with the following keys: ``` POCKET_CONSUMER_KEY = key-from-custom-pocket-app POCKET_ACCESS_TOKENS = {"YourUsername": "pocket-token-for-app"} ``` `POCKET_ACCESS_TOKENS` MUST be on a single line, or the JSON will be misinterpreted by the parser as a new key/value pair.	2020-12-04 22:54:39 -05:00
Nick Sweeting	00dfe2d449	Merge branch 'v0.5.0' into cleanup	2020-12-04 20:40:24 -05:00
jdcaballerov	54b25d9a81	Linting	2020-12-03 15:59:45 -05:00
jdcaballerov	d4bca80b50	Use uppercase for constants	2020-12-03 15:44:59 -05:00
jdcaballerov	c8d8346e4d	Remove duplicate context after rebase	2020-12-03 15:44:21 -05:00
jdcaballerov	a1fba9887d	Remove write_html_main_index	2020-12-03 09:25:38 -05:00
jdcaballerov	7f39702bd2	Delete legacy/ folder	2020-12-03 09:17:34 -05:00
jdcaballerov	367b12ba40	Replace legacy templates for django templates	2020-12-03 09:16:18 -05:00
jdcaballerov	8ac7a760c9	Fix num_links missing in public index	2020-12-03 08:32:49 -05:00
jdcaballerov	69897f6121	Hotfix public page search ... No ordering causes warning and fallback to default unfiltered QuerySet	2020-12-03 08:32:49 -05:00
Hawken Rives	7299b1f5ae	fix "inconsisntencies" typo in error message	2020-12-02 16:28:26 -06:00
Nick Sweeting	193dde03f0	Merge pull request #559 from jdcaballerov/hotfix-public-search ... Hotfix public page search	2020-12-01 10:56:32 -05:00
jdcaballerov	4d972571d0	Hotfix public page search ... No ordering causes warning and fallback to default unfiltered QuerySet	2020-12-01 10:46:11 -05:00
Cristian	7008f9b735	feat: move import	2020-11-28 13:11:15 -05:00
Cristian	648b4c8aab	feat: Remove unused function write_json_main_index	2020-11-28 13:02:39 -05:00
Nick Sweeting	a846916b82	remove unused argument	2020-11-28 12:38:15 -05:00
Cristian	10ee6db02f	lint: Remove unused variable	2020-11-28 12:35:13 -05:00
Cristian	fa5de72f9f	refactor: Move indexing logic out of logging module	2020-11-28 12:34:40 -05:00
Nick Sweeting	bee1f3e263	fix lint errors	2020-11-28 04:09:59 -05:00
Nick Sweeting	104553489f	remove redundant utils file	2020-11-28 02:12:27 -05:00
Nick Sweeting	84507b68b5	add legacy code warning to schema.py	2020-11-28 02:03:40 -05:00