182 commits

Author	SHA1	Message	Date
Angel Rey	9264ad88e0	Fixed string casting	2020-10-02 15:46:39 -05:00
Angel Rey	7d513b9b19	Replaced os.path in schema.py	2020-10-02 15:46:39 -05:00
Angel Rey	4581ea956f	Fixed empty tags	2020-09-24 15:34:23 -05:00
Angel Rey	533ae7413c	Removed comments	2020-09-24 15:34:23 -05:00
Angel Rey	e06d3f9128	Fixed Link schema	2020-09-24 15:34:23 -05:00
Angel Rey	45775c607c	Fixed empty tags	2020-09-24 15:34:23 -05:00
Angel Rey	f26c0c6cd8	Fix serialization	2020-09-24 15:34:23 -05:00
Angel Rey	62c9028212	Improved tags	2020-09-24 15:34:23 -05:00
ttimasdf	2bf496e7e9	feat: Add mercury-parsed content to summary page	2020-09-22 18:44:12 -05:00
Cristian	b18bbf8874	test: Fix tests post-rebase	2020-09-17 09:09:52 -05:00
apkallum	b99784b919	pathlib with / syntax for config, index	2020-09-17 09:09:52 -05:00
apkallum	594d9e49ce	first attempt to migrate to Pathlib	2020-09-17 09:09:52 -05:00
Cristian Vargas	5e9b3099c6	Update fix_duplicate_links_in_index docstring ... Co-authored-by: Nick Sweeting <git@sweeting.me>	2020-09-15 08:05:46 -05:00
Cristian	f55153eab3	feat: Update update command to work with querysets	2020-09-15 08:05:46 -05:00
Cristian	fe9604a772	feat: Add tests for remove command	2020-09-15 08:05:46 -05:00
Cristian	a8ed72501d	feat: Refactor remove command to use querysets	2020-09-15 08:05:46 -05:00
Cristian	be520d137a	feat: Refactor add method to use querysets	2020-09-15 08:05:46 -05:00
Cristian	be0dff8126	feat: Add tests to refactored init command	2020-09-15 08:05:46 -05:00
Cristian	404f333e17	feat: Refactor get_invalid_folders to work with a queryset instead of a list of links	2020-09-15 08:05:46 -05:00
Cristian	6b4b7127b4	feat: Remove unused imports	2020-09-15 08:05:46 -05:00
Cristian	b8585dd92e	feat: load_main_index returns a queryset now	2020-09-15 08:05:46 -05:00
Cristian	885ff50449	feat: Add html export to list command	2020-09-15 08:05:46 -05:00
Cristian	c16fdf1b47	feat: Update data folder check	2020-09-15 08:05:46 -05:00
Cristian	874403e667	feat: Remove patch_main_index	2020-09-15 08:05:46 -05:00
Cristian	31343c1367	feat: Update extractors and add command to use sql index as source of truth	2020-09-15 08:05:46 -05:00
Cristian	02f36b2096	feat: Replace index.json with index.sql as the main index in init	2020-09-15 08:05:46 -05:00
Nick Sweeting	d803481bd8	better folder list column output	2020-08-18 18:46:21 -04:00
Nick Sweeting	e87f1d57a3	fix linters	2020-08-18 09:22:12 -04:00
Nick Sweeting	430d51756b	show archive dir size in link details page	2020-08-18 09:17:37 -04:00
Nick Sweeting	f18d92570e	wip attempt to fix timestamp unique constraint errors	2020-08-18 08:30:09 -04:00
Nick Sweeting	15efb2d5ed	new generic_html parser for extracting hrefs	2020-08-18 08:29:05 -04:00
Nick Sweeting	5f84a7bc6e	better handle the case where json index lags behind sql index	2020-08-18 08:13:13 -04:00
Nick Sweeting	77d2f08a5c	show more info in merge conflict error message	2020-08-18 08:12:35 -04:00
Nick Sweeting	6e142e3531	match existing snapshots on url only, ignoring timestamp	2020-08-18 04:39:04 -04:00
Nick Sweeting	f371032b71	show warning when killing archivebox during index writing	2020-08-18 04:38:29 -04:00
Nick Sweeting	225b63b732	skip invalid urls at all stages	2020-08-17 03:12:17 -04:00
Cristian	eb3528fa9f	feat: Add readability output to legacy index.html	2020-08-11 12:14:13 -05:00
Nick Sweeting	5b6eb5e4ad	make filenames consistent with program name	2020-08-03 13:23:05 -05:00
Cristian	91f63635e8	feat: Add singlefile in a couple more places	2020-08-03 13:22:06 -05:00
Cristian	b325c0dd9f	feat: Add singlefile to latest outputs	2020-08-03 13:22:06 -05:00
Cristian	42b0c80465	feat: Add singlefile to link_details	2020-08-03 13:22:06 -05:00
Cristian	c073ea141d	feat: Initial oneshot command proposal	2020-07-29 11:19:06 -05:00
Nick Sweeting	3aeca0e450	fix pending titles and favicons, improve add page, custom admin	2020-07-27 23:26:45 -04:00
Cristian	d04c9b3281	fix: if cmd in Link parsing is found to be a string, put it inside a list	2020-07-24 14:36:08 -05:00
Cristian	6006b4f93b	refactor: Organize code to remove flake8 issues	2020-07-24 12:25:25 -05:00
Cristian	82f8f8b661	fix: Use config information for path instead of hardcoded values	2020-07-24 10:34:47 -05:00
Cristian	100fa5d1f5	fix: Guess timestamps and add placeholders to support older indices	2020-07-24 09:24:52 -05:00
Cristian	fe0884f1ec	fix: Remove link from sql index on remove command	2020-07-23 15:07:00 -05:00
Nick Sweeting	02a2fefbba	Merge pull request #385 from apkallum/origin/output-permissions	2020-07-23 11:52:31 -04:00
Cristian Vargas	51716bbf74	Update warning message on detail index error ... Co-authored-by: Nick Sweeting <git@sweeting.me>	2020-07-23 10:23:41 -05:00
Cristian	5ca7121fd8	refactor: Change path calculation to use pathlib in a better way	2020-07-23 10:22:36 -05:00
apkallum	0ed2a23670	ensure correct permissions for output folder	2020-07-23 10:28:10 -04:00
Cristian	71f5f03a20	fix: Add notice for issues with index detail	2020-07-22 17:08:32 -05:00
Cristian Vargas	e58c3deb05	feat: Update path generation in detail index fallback ... Co-authored-by: Nick Sweeting <git@sweeting.me>	2020-07-22 14:46:03 -05:00
Cristian	263eb4e372	fix: Change path to use ARCHIVE_DIR_NAME	2020-07-22 14:37:10 -05:00
Cristian	9815241b78	feat: Fallback to link detail when there is an issue loading a link from main index	2020-07-22 14:22:00 -05:00
Cristian	a5550b2105	fix: Rename logging folder to avoid naming conflicts (and circular import issues)	2020-07-22 11:02:13 -05:00
Nick Sweeting	848977e7be	Merge pull request #371 from cdvv7788/circular_import ... refactor: Move logging.py to main module to avoid circular import issues	2020-07-17 19:27:21 -04:00
Cristian	f4d1b5121e	refactor: Move logging.py to main module to avoid circular import issues	2020-07-17 18:00:04 -05:00
Apkallum	1f91f5b102	remove commented lines	2020-07-16 19:42:20 -04:00
apkallum	b7785c4138	use dateparser for parsing, let it handle error	2020-07-16 19:38:38 -04:00
Nick Sweeting	58ac44c867	Merge pull request #365 from cdvv7788/hotfix/#330 ... fix: htmlencode titles before rendering the static html index and detail	2020-07-16 14:45:00 -04:00
Cristian	f845224d6f	fix: htmlencode titles before rendering the static html index and detail	2020-07-16 09:20:33 -05:00
apkallum	98dda68897	fix: timestamp comparison in to_json function	2020-07-14 10:26:33 -04:00
Cristian	5e2bf73f04	fix: Bugs related to add() refactor	2020-07-13 14:48:25 -05:00
Nick Sweeting	4c4b1e6a4b	fix link creation	2020-07-13 11:33:38 -04:00
Nick Sweeting	d3bfa98a91	fix depth flag and tweak logging	2020-07-13 11:26:34 -04:00
Nick Sweeting	354a63ccd4	dont dedupe snapshots in sqlite on every run	2020-07-13 11:25:43 -04:00
Nick Sweeting	7cbd068c95	add flake8	2020-07-13 11:22:23 -04:00
Nick Sweeting	bc1f925542	fix snapshot getter in sql saving	2020-06-30 15:24:23 -04:00
Nick Sweeting	dda3542d60	bump sql updated time after every link details save	2020-06-30 13:45:47 -04:00
Nick Sweeting	b9e17fa0d1	rename archivebox-info to archivebox-status	2020-06-25 23:32:01 -04:00
Nick Sweeting	cb67b09f9d	Merge branch 'master' into django	2020-06-25 21:30:29 -04:00
Nick Sweeting	fae2fdaf2b	fix if to elif bug in index snapshot sql writing	2020-06-25 17:48:27 -04:00
Nick Sweeting	9fc431102b	better timestamp handling	2020-04-22 21:15:15 -04:00
Mashiat Sarker Shakkhar	2ba17136e0	index/schema.py: Make cmd_version actually optional. ... During migration, we may have to import json index file which was produced with an older version. If the index file is missing cmd_version, migration will fail.	2019-09-10 23:51:49 -04:00
Nick Sweeting	ecfca13b6d	fix present folders docstring	2019-05-02 15:20:21 -04:00
Nick Sweeting	204de37eb9	fix parsing errors for older archive index formats	2019-05-01 02:28:48 -04:00
Nick Sweeting	1ac99621ab	show progress during validate_links	2019-05-01 02:28:26 -04:00
Nick Sweeting	cb2dd1ee28	rename model Page to Snapshot	2019-04-30 23:44:51 -04:00
Nick Sweeting	95007d9137	split up utils into separate files	2019-04-30 23:13:04 -04:00
Nick Sweeting	1b8abc0961	move everything out of legacy folder	2019-04-27 17:26:24 -04:00