22 commits

Author	SHA1	Message	Date
Nick Sweeting	94123ca68c	fix archive_dot_org repsonse parsing bytes vs str bug	2024-10-01 00:18:38 -07:00
Nick Sweeting	d21bc86075	finish migrating almost all config to new system	2024-09-30 23:21:34 -07:00
Nick Sweeting	363a499289	move util.py into misc folder	2024-09-30 17:25:15 -07:00
Nick Sweeting	dfca4b13b2	move system.py into misc folder	2024-09-30 17:13:55 -07:00
Nick Sweeting	3e5b6ddeae	move config into dedicated global app	2024-09-30 15:59:05 -07:00
Nick Sweeting	457c42bf84	load EXTRACTORS dynamically using importlib.import_module	2024-05-11 22:28:59 -07:00
Ben Muthalaly	d8cf09c21e	Remove unnecessary variable length args for dedupe	2024-03-05 21:13:45 -06:00
Ben Muthalaly	d74ddd42ae	Flip dedupe precedence order	2024-03-01 14:50:32 -06:00
Ben Muthalaly	4e69d2c9e1	Add EXTRA_*_ARGS for wget, curl, and singlefile	2024-02-22 23:04:11 -06:00
Nick Sweeting	bd6d9c165b	enforce utf8 on literally all file operations because windows sucks	2021-03-27 01:16:29 -04:00
Dan Arnfield	5420903102	Refactor should_save_extractor methods to accept overwrite parameter	2021-01-21 15:56:32 -06:00
Nick Sweeting	ac9e0e356d	config fixes	2020-10-31 07:57:11 -04:00
Cristian	563d0f94ec	feat: Use CURL_ARGS in favicon extractor	2020-10-22 08:46:16 -05:00
Cristian	972d57bd08	feat: Add CURL_ARGS to control curl arguments	2020-10-22 08:46:16 -05:00
Cristian	b18bbf8874	test: Fix tests post-rebase	2020-09-17 09:09:52 -05:00
Nick Sweeting	273059f054	accept gzipped responses when using curl	2020-07-28 05:55:54 -04:00
Cristian	a5550b2105	fix: Rename logging folder to avoid naming conflicts (and circular import issues)	2020-07-22 11:02:13 -05:00
Nick Sweeting	0965031d8f	fix archive_org header rename	2020-07-22 01:46:38 -04:00
Cristian	f4d1b5121e	refactor: Move logging.py to main module to avoid circular import issues	2020-07-17 18:00:04 -05:00
Nick Sweeting	5c2bbe7efe	bufixes	2020-06-25 22:14:40 -04:00
Nick Sweeting	95007d9137	split up utils into separate files	2019-04-30 23:13:04 -04:00
Nick Sweeting	1b8abc0961	move everything out of legacy folder	2019-04-27 17:26:24 -04:00