98 commits

Author	SHA1	Message	Date
JDC	823df34080	Use QuerySets for search backend API instead of pks	2020-12-06 01:12:47 +02:00
JDC	f383648ffc	Use a generator for snapshot flush from index	2020-12-06 01:12:47 +02:00
JDC	47daa038eb	Implement flush for search backend after remove command	2020-12-06 01:12:47 +02:00
Nick Sweeting	3b280e6b02	Merge pull request #569 from cdvv7788/extract-command-update ... feat: Add --extract flag to update command	2020-12-05 17:43:28 -05:00
Cristian	35389608d1	feat: Add --extract flag to update command	2020-12-05 12:20:47 -05:00
Cristian	34cad4fe8d	fix: Update function with --index-only flag was not behaving as expected	2020-12-05 12:10:17 -05:00
Cristian	7008f9b735	feat: move import	2020-11-28 13:11:15 -05:00
Cristian	fa5de72f9f	refactor: Move indexing logic out of logging module	2020-11-28 12:34:40 -05:00
Nick Sweeting	c9162a6d09	remove finished/not finished spinners	2020-11-28 01:07:02 -05:00
Nick Sweeting	1b22f8eeef	Merge pull request #515 from cdvv7788/POC-setup-django-on-init	2020-11-27 23:56:37 -05:00
Nick Sweeting	e4d2ac432d	improve OS kernel output in archivebox version	2020-11-27 23:08:23 -05:00
Nick Sweeting	07a56f9d46	also print platform and CPU info in version output	2020-11-27 22:59:18 -05:00
Nick Sweeting	d9ef3d0bf8	ignore lost+found dir in data folder	2020-11-27 19:39:19 -05:00
mAAdhaTTah	be7a7f8548	Fix string checks in schedule ... `s` comes through as a `PosixPath`, so both the `' ' in s` & return value, later used by `join`, complain.	2020-11-23 18:34:07 -05:00
Nick Sweeting	0e2ccbc10d	update urls to new repo path	2020-11-23 02:06:46 -05:00
Nick Sweeting	b11d562445	fix splitting on multiple equals in val	2020-11-22 12:33:15 -05:00
Cristian	54df0a035b	fix: Move csv split to the add function to avoid optional nullable argument	2020-11-13 13:10:17 -05:00
Cristian	db523c9d82	fix: Avoid mutable default input argument	2020-11-13 11:41:50 -05:00
Cristian	44eede96e5	feat: Add extract flag to add command	2020-11-13 09:24:34 -05:00
Nick Sweeting	79051ca15b	new package build	2020-10-31 03:08:41 -04:00
Nick Sweeting	18355dc2c6	clean up config loading in settings and config file layout	2020-10-31 03:08:03 -04:00
Cristian	e1d0b8bce7	feat: Initialize django at the beginning	2020-10-26 07:45:21 -05:00
Cristian	f397634dd2	feat: Rename old indexes at the end of init process	2020-10-23 06:45:56 -05:00
Cristian	ae1484b8bf	feat: Remove index.json and index.html generation from the regular process	2020-10-23 06:45:56 -05:00
Cristian	b18bbf8874	test: Fix tests post-rebase	2020-09-17 09:09:52 -05:00
apkallum	594d9e49ce	first attempt to migrate to Pathlib	2020-09-17 09:09:52 -05:00
Cristian	5348f4735a	fix: Change check to avoid issues with empty querysets	2020-09-15 08:05:46 -05:00
Cristian	cf18130f85	feat: Add deprecation warning for index.json	2020-09-15 08:05:46 -05:00
Cristian	fa622d3e14	refactor: Replace --index with --with-headers in the list command to make it more explicit. Change it so it affects the csv output too.	2020-09-15 08:05:46 -05:00
Cristian	2aa8d69b72	fix: Save history in main index (to mimic previous behaviour)	2020-09-15 08:05:46 -05:00
Cristian	7e9d195d13	feat: Update list command to sort using sqlite	2020-09-15 08:05:46 -05:00
Cristian	f55153eab3	feat: Update update command to work with querysets	2020-09-15 08:05:46 -05:00
Cristian	a8ed72501d	feat: Refactor remove command to use querysets	2020-09-15 08:05:46 -05:00
Cristian	be520d137a	feat: Refactor add method to use querysets	2020-09-15 08:05:46 -05:00
Cristian	6a2e6aad2f	fix: status command was failing on empty archives	2020-09-15 08:05:46 -05:00
Cristian	be0dff8126	feat: Add tests to refactored init command	2020-09-15 08:05:46 -05:00
Cristian	404f333e17	feat: Refactor get_invalid_folders to work with a queryset instead of a list of links	2020-09-15 08:05:46 -05:00
Cristian	dae606de6e	feat: Update init to take advantage of querysets to reduce memory consumption	2020-09-15 08:05:46 -05:00
Cristian	6b4b7127b4	feat: Remove unused imports	2020-09-15 08:05:46 -05:00
Cristian	b8585dd92e	feat: load_main_index returns a queryset now	2020-09-15 08:05:46 -05:00
Cristian	885ff50449	feat: Add html export to list command	2020-09-15 08:05:46 -05:00
Cristian	aab8f96520	feat: Add flag to list command to support index like output	2020-09-15 08:05:46 -05:00
Cristian	be57db1369	feat: Save static indexes at the end of init	2020-09-15 08:05:46 -05:00
Cristian	c16fdf1b47	feat: Update data folder check	2020-09-15 08:05:46 -05:00
Cristian	874403e667	feat: Remove patch_main_index	2020-09-15 08:05:46 -05:00
Cristian	31343c1367	feat: Update extractors and add command to use sql index as source of truth	2020-09-15 08:05:46 -05:00
Cristian	e9caee6b10	feat: Update status command to consider sql as the main index	2020-09-15 08:05:46 -05:00
Cristian	02f36b2096	feat: Replace index.json with index.sql as the main index in init	2020-09-15 08:05:46 -05:00
Nick Sweeting	5cb13ff1dd	ignore node cruft in output folder during init	2020-08-18 18:20:26 -04:00
Nick Sweeting	15efb2d5ed	new generic_html parser for extracting hrefs	2020-08-18 08:29:05 -04:00