1840 commits

Author	SHA1	Message	Date
Nick Sweeting	6c39d27ccb	update singlefile plugin to use new npm binprovider and support installing	2024-09-21 04:12:33 -07:00
Nick Sweeting	dd6d7e4975	fix npm and pip binprovider setup and paths search	2024-09-21 04:12:33 -07:00
Nick Sweeting	30def925e7	move all ansible files into plugantic folder for now	2024-09-21 04:12:33 -07:00
Nick Sweeting	11f369ee2d	bump subdependency versions	2024-09-21 04:12:33 -07:00
Nick Sweeting	61df9ea059	fix duplicate when	2024-09-17 02:04:41 -07:00
Nick Sweeting	2c8779736a	change default node version to 21	2024-09-17 02:03:28 -07:00
Nick Sweeting	f29f72f383	fix os checking for npm install	2024-09-17 01:57:06 -07:00
Nick Sweeting	a5cefb5464	install nodesource first	2024-09-17 01:46:02 -07:00
Nick Sweeting	19c7b9c24e	install nodejs and npm packages properly in npm ansible	2024-09-17 01:42:06 -07:00
Nick Sweeting	7ab1a0b873	fix singlefile and puppeteer ansible install	2024-09-17 01:33:32 -07:00
Nick Sweeting	5c0aa6fe59	more ansible fixes	2024-09-17 01:12:49 -07:00
Nick Sweeting	c55cd46ecb	consolidate ansible setup into roles dir	2024-09-17 00:48:47 -07:00
Nick Sweeting	25db6826ec	ignore lib dirs	2024-09-17 00:47:55 -07:00
Nick Sweeting	8d69469887	silence ansible errors about implicit localhost	2024-09-15 20:31:11 -07:00
Nick Sweeting	fab80632b7	add ansible runner code to get facts after execution and benedict	2024-09-15 20:29:02 -07:00
Nick Sweeting	e9ddac0219	fix ansible installed_packages and cacheable facts	2024-09-15 20:28:35 -07:00
Nick Sweeting	56b851ea1b	more ansible playbooks improvements	2024-09-13 04:55:40 -07:00
Nick Sweeting	8557e77a70	add ansible playbooks	2024-09-13 03:27:38 -07:00
Nick Sweeting	3bbf8f69ab	cleanup settings.py sqlite settings more	2024-09-13 03:27:38 -07:00
Nick Sweeting	c887af0278	minor ruff fixes	2024-09-12 02:00:07 -07:00
Nick Sweeting	c00afce71f	upgrade dependency versions to django 5.1 minimum	2024-09-11 17:08:10 -07:00
Nick Sweeting	eae11cba19	add recommended SQLite db connection settings to avoid single-writer lock contention	2024-09-11 16:50:44 -07:00
Nick Sweeting	0187c8b6cb	bump version to 0.8.4	2024-09-10 03:10:30 -07:00
Nick Sweeting	a13f71a86c	allow supervisord to start if pid file is stale	2024-09-10 03:10:10 -07:00
Nick Sweeting	cecca8d169	allow deleting results from list page	2024-09-10 03:09:43 -07:00
Nick Sweeting	f5c878b267	point select2 js resources to local statifiles	2024-09-10 01:51:08 -07:00
Nick Sweeting	0640018426	bump packages	2024-09-10 01:50:49 -07:00
Nick Sweeting	0bd678c30f	fix init	2024-09-10 00:37:01 -07:00
Nick Sweeting	d680c48942	avoid auto-starting all supervisord workers on startup	2024-09-10 00:19:32 -07:00
Nick Sweeting	8d1d5df43a	add proxy model for SinglefileResult wrapping ArchiveResult	2024-09-10 00:05:46 -07:00
Nick Sweeting	04cb6bdfd6	ignore healtcheck requests in daphne logs	2024-09-10 00:05:46 -07:00
Nick Sweeting	1ce09b88d7	show logs and workers in Django Admin data views	2024-09-10 00:05:45 -07:00
Nick Sweeting	60154fba5f	add django_huey, huey_monitor, and replace Threads with huey tasks	2024-09-10 00:05:45 -07:00
Nick Sweeting	4df90fbb40	change plugins to have both a .register that runs at import and .ready that runs later	2024-09-10 00:05:45 -07:00
Nick Sweeting	f1cca5bbba	ignore tmp dirs	2024-09-10 00:05:45 -07:00
Nick Sweeting	db5428a486	increase maximum form field POST param size	2024-09-10 00:05:45 -07:00
Nick Sweeting	31576e21a2	fix plugins and modules dataviews	2024-09-06 06:14:18 -07:00
Nick Sweeting	3f76e0a87f	fix migrations import errors	2024-09-06 03:48:52 -07:00
Nick Sweeting	ab517d9848	fix infinite recursion on pydantic instantiation	2024-09-06 03:19:03 -07:00
Nick Sweeting	cfe5798ade	bump version to 0.8.3	2024-09-06 02:59:14 -07:00
Nick Sweeting	6e2ddc14df	make archivebox server spawn daphne process instead of runserver	2024-09-06 02:55:06 -07:00
Nick Sweeting	f6ca48835a	hide more 404 and 304 errors from daphne logs	2024-09-06 02:54:53 -07:00
Nick Sweeting	52386d9c16	run all blocking commands in background threads and show nice UI messages as confirmation	2024-09-06 02:54:22 -07:00
Nick Sweeting	b56b1cac35	cleanup plugantic and pkg apps, make BaseHook actually create its own settings	2024-09-06 01:48:18 -07:00
Nick Sweeting	0e79a8b683	minor type hint fixes	2024-09-05 23:24:44 -07:00
Nick Sweeting	2e1e1945f2	add django-object-actions to provide Regenerate ABID button	2024-09-05 23:19:21 -07:00
Nick Sweeting	00aa7dc19f	setup daphne and django channels to replace runserver	2024-09-05 21:45:43 -07:00
Nick Sweeting	a190745f36	add rich support for logging and ruff for formatting	2024-09-05 21:44:18 -07:00
Nick Sweeting	2c48dabfab	add preliminary support for ASGI/daphne serving	2024-09-05 21:43:42 -07:00
Nick Sweeting	26291f81ef	prevent ABID drift entirely when its set to False on a model	2024-09-05 21:43:14 -07:00
Nick Sweeting	c76c50e71f	add HTTP byte range request support to media file serving	2024-09-05 21:41:49 -07:00
Nick Sweeting	ba6c1fd69b	minor formatting and fixes	2024-09-05 04:39:46 -07:00
Nick Sweeting	d50aed9185	improve new Hook system docstring	2024-09-05 03:51:54 -07:00
Nick Sweeting	44669fab73	add BaseHook concept to underlie all Plugin hooks	2024-09-05 03:36:18 -07:00
Nick Sweeting	ed5357cec9	add migrations for datetime field renames	2024-09-04 23:44:13 -07:00
Nick Sweeting	0fef2357c7	change default datetime template rendering to include seconds	2024-09-04 23:43:25 -07:00
Nick Sweeting	0c236b034b	allow accessing admin change pages by .abid as well as .id	2024-09-04 23:43:07 -07:00
Nick Sweeting	cbf2a8fdc3	rename datetime fields to _at, massively improve ABID generation safety and determinism	2024-09-04 23:42:36 -07:00
Nick Sweeting	68a39b7392	remove .old_id entirely and make ABID generation only happen once on initial save	2024-09-04 16:40:15 -07:00
Nick Sweeting	22a6666bab	fix pkg management subcommand	2024-09-04 02:37:19 -07:00
Nick Sweeting	9072f4a999	fix ABID added to default to bookmarked value	2024-09-04 02:34:27 -07:00
Nick Sweeting	4427869ae8	fix ABID generation by chopping ts_src precision to consistent length	2024-09-04 02:02:29 -07:00
Nick Sweeting	d060eaa499	abid gradual improvements, some regrets	2024-09-04 00:08:14 -07:00
Nick Sweeting	ae13f1811f	better ABID display in admin UI	2024-09-03 17:11:10 -07:00
Nick Sweeting	3d7dd3c9cf	bump pydantic-pkgr	2024-09-03 14:17:50 -07:00
Nick Sweeting	f915ef6c5e	try to improve django console log 404 and 200 error filtering	2024-09-03 14:17:02 -07:00
Nick Sweeting	01094ecb03	fix REST API CSRF and auth handling	2024-09-03 14:16:44 -07:00
Nick Sweeting	41a318a8bd	fix config loading precedence order	2024-09-03 03:22:37 -07:00
Nick Sweeting	8d4ace017d	fix Binary overrides getting wiped after init	2024-09-03 02:50:30 -07:00
Nick Sweeting	a18857dbb9	add ytldp extractor stub	2024-09-03 01:40:38 -07:00
Nick Sweeting	b927f5f2a9	add FFMPEG_BINARY to ytdlp plugin	2024-09-03 01:39:02 -07:00
Nick Sweeting	c1c55d6da7	fix API token_auth and CSRF setup	2024-09-03 01:21:13 -07:00
Nick Sweeting	9af260df16	BasePlugin system expanded and registration system improved	2024-09-03 00:58:50 -07:00
Nick Sweeting	f1579bfdcd	add NODE_BIN_PATH to dynamic CONFIG and make ANSI an AttrDict	2024-09-03 00:19:18 -07:00
Nick Sweeting	23c7d2f860	bump pydantic-pkgr	2024-09-02 15:06:10 -07:00
Nick Sweeting	3ea116f0b6	tweak Pkgs app name and admin button CSS	2024-09-02 01:14:41 -07:00
Nick Sweeting	9be7e1a0bc	change Snapshot admin action UI button text	2024-09-02 01:14:06 -07:00
Nick Sweeting	5f7bca298c	bump USER_AGENT config default to chrome v128	2024-09-02 01:13:50 -07:00
Nick Sweeting	dd05ad04fa	disable cookie auth in API because csrf=False	2024-09-02 01:13:19 -07:00
Nick Sweeting	da76a84c45	rename pkgs app to pkg	2024-08-30 03:32:36 -07:00
Nick Sweeting	6e13cd4820	fix loading of plugin dependencies and bump pydantic_pkgr version	2024-08-30 03:30:28 -07:00
Nick Sweeting	a8f00caff8	bump singlefile version	2024-08-29 22:33:47 -07:00
Nick Sweeting	57ed2ebcb2	switch youtube_dl install for yt-dlp	2024-08-28 03:46:45 -07:00
Nick Sweeting	038e9f32af	bump deps versions	2024-08-28 03:03:09 -07:00
Nick Sweeting	4ac980a832	hide binary loading debug output for now	2024-08-28 03:02:54 -07:00
Nick Sweeting	1e73a06ba0	change ABIDModel.created to use AutoTimeField seeded on .save instead of auto_now_add so that ts_src for ABID is available on creation before DB row is created	2024-08-28 03:02:37 -07:00
Nick Sweeting	7a734ce410	fix archivebox setup User model import	2024-08-28 03:01:35 -07:00
Nick Sweeting	e558d71b10	fix uuid checks on save	2024-08-28 03:01:19 -07:00
Nick Sweeting	6456cb1727	fix NOT NULL constraint failed: core_snapshot.created_by_id	2024-08-28 00:51:16 -07:00
Nick Sweeting	b789be098d	fix type hint	2024-08-28 00:26:37 -07:00
Nick Sweeting	d0fefc0279	add chunk_size=500 to more iterator calls	2024-08-27 19:28:00 -07:00
Nick Sweeting	44849e1ba2	fix python vendor dependendcies	2024-08-26 20:24:25 -07:00
Nick Sweeting	d7e333b5cf	fix settings.py for prod	2024-08-26 20:20:33 -07:00
Nick Sweeting	24fe958ff3	massively improve Snapshot admin list view query performance	2024-08-26 20:16:43 -07:00
Nick Sweeting	6c4f3fc83a	fix chrome headless=new arg	2024-08-26 20:15:36 -07:00
Nick Sweeting	6ffa710bb3	fix headers Elapsed timedelta is not a JSON-serializable	2024-08-26 20:15:22 -07:00
Nick Sweeting	917af4b1d7	add pydantic logfire and improve settings for debugging	2024-08-26 20:14:47 -07:00
Nick Sweeting	3a57c28ca8	bump pydantic_pkgr version to 0.1.4	2024-08-23 02:03:26 -07:00
Nick Sweeting	5fe3edd79a	new plugin loading system	2024-08-23 02:02:34 -07:00
Nick Sweeting	34389e5e7c	improve CSRF_TRUSTED_ORIGINS loading logic	2024-08-23 02:01:40 -07:00
Nick Sweeting	c6a80ab256	use pydantic_pkgr to parse semver versions	2024-08-23 02:01:24 -07:00
Nick Sweeting	2c2d034d6d	move to new vendoring fallback logic	2024-08-23 02:01:02 -07:00
Nick Sweeting	9c35f3ddb7	add new CSRF_TRUSTED_ORIGINS config option	2024-08-22 18:40:47 -07:00
Nick Sweeting	1a03db2b1d	fix api auth endpoint checking wrong user field	2024-08-22 18:28:23 -07:00
Nick Sweeting	73a3e6aad0	handle tag with no slug or name	2024-08-22 18:25:15 -07:00
Nick Sweeting	cec8015b5b	fix missing STDOUT	2024-08-22 17:58:58 -07:00
Nick Sweeting	38ca5c3228	add extra info to headers.json	2024-08-22 17:57:40 -07:00
Nick Sweeting	80d7a30f13	improve version_str detection	2024-08-22 17:57:33 -07:00
Nick Sweeting	1d31b88fa3	fix migration failing when Tag name is empty	2024-08-22 16:30:25 -07:00
Nick Sweeting	9ecc2ff91c	bump more deps	2024-08-22 15:36:11 -07:00
Nick Sweeting	09553d8340	hardcode EXTRACTOR_CHOICES to prevent nondeterministic migrations	2024-08-22 15:36:02 -07:00
Nick Sweeting	0a5b22700c	bump deps	2024-08-22 15:20:52 -07:00
Nick Sweeting	afe1307617	fix created_by field migration to create User properly if none exists	2024-08-22 15:20:36 -07:00
Nick Sweeting	9b1659c72f	make created_by_id autoapply to any ArchiveResults created under Snapshot	2024-08-20 19:43:07 -07:00
Nick Sweeting	c30ae1d2cb	add created_by_id to all Snapshot creation functions	2024-08-20 19:28:28 -07:00
Nick Sweeting	aa282daadf	show CONSTANTS in separate section of django admin config	2024-08-20 19:27:21 -07:00
Nick Sweeting	ea81f2fc14	use ABIDAdmin as base ModelAdmin	2024-08-20 19:26:40 -07:00
Nick Sweeting	57e48b432c	show redacted API tokens in django admin	2024-08-20 19:25:16 -07:00
Nick Sweeting	ff619fd31f	add api migrations	2024-08-20 18:31:40 -07:00
Nick Sweeting	0285aa52a0	config and attr access improvements	2024-08-20 18:31:21 -07:00
Nick Sweeting	4ae186dfca	fix ABID generation consistency when self._state.adding is True	2024-08-20 05:56:19 -07:00
Nick Sweeting	9d2116ad9a	migration to make fields non editable	2024-08-20 03:46:04 -07:00
Nick Sweeting	9c8ece4d7c	minor css and APIToken Admin UI fixes	2024-08-20 03:45:49 -07:00
Nick Sweeting	75018ed10b	Merge branch 'dev' into feat/search-fields	2024-08-20 03:37:56 -07:00
Nick Sweeting	05880dba8b	bump version to 0.8.2	2024-08-20 03:36:07 -07:00
Nick Sweeting	267964881c	fix tags editor in Snapshot list view	2024-08-20 03:29:57 -07:00
Nick Sweeting	849b4963a1	add migrations	2024-08-20 01:58:44 -07:00
Nick Sweeting	52a813aa80	fix title display in admin UI and abid filter matching in urls	2024-08-20 01:58:36 -07:00
Nick Sweeting	9273db528e	fix abid generation migrations to be historically consistent	2024-08-20 01:58:19 -07:00
Nick Sweeting	506b3d28d4	fix admin UI TagInline and ArchiveResultInline form POST handling	2024-08-20 01:57:07 -07:00
Nick Sweeting	54acfd9f86	improve REST API filter parameters and pagination	2024-08-20 01:56:37 -07:00
Nick Sweeting	850448b42c	add salt args in more places	2024-08-20 01:56:07 -07:00
Nick Sweeting	c4ef2993b2	update REST API and Admin UI to use new id and old_id exclusively	2024-08-19 20:00:04 -07:00
Nick Sweeting	344e902fc6	migrate SnapshotTag to use new snapshot id	2024-08-19 19:42:25 -07:00
Nick Sweeting	74c11d41be	dont allow modifying Snapshot.timestamp as it would break folder dirs	2024-08-19 19:42:01 -07:00
Nick Sweeting	c68a66f74e	add salt kwarg to abid generation funcs	2024-08-19 19:41:43 -07:00
Nick Sweeting	a49739b41c	add hacky workaround for TagInline not showing in admin Snapshot change view	2024-08-19 18:36:35 -07:00
Nick Sweeting	cf2faecf61	add migrations for SnapshotTag through model	2024-08-19 18:36:20 -07:00
Nick Sweeting	7164fb961c	hotlink to snapshots using generic search query link instead of id__startswith	2024-08-19 18:35:35 -07:00
Nick Sweeting	3148d2a3ef	add squashmigrations to allowed mgmgt command list	2024-08-19 18:35:07 -07:00
Nick Sweeting	2c157f0b5b	increase max POST field limit for admin	2024-08-19 18:34:52 -07:00
Nick Sweeting	18ebaed7ed	fix admin UI display of tags	2024-08-19 18:34:01 -07:00
Nick Sweeting	57d31b2b14	fix snapshot uuid	2024-08-18 01:07:21 -07:00
Nick Sweeting	8c50257fe9	move snapshot id to old_id	2024-08-18 00:24:38 -07:00
Nick Sweeting	951025228f	add tag endpoint	2024-08-18 00:24:14 -07:00
Nick Sweeting	a271bcb4ce	use new ids for ArchiveResult API	2024-08-17 23:03:43 -07:00
Nick Sweeting	f72debfdb2	migrate ArchiveResult.id to old_id, and make uuid main id	2024-08-17 22:58:35 -07:00
Nick Sweeting	033ec08d0c	save snapshot ids during migration	2024-08-17 21:56:45 -07:00
Nick Sweeting	4d0bbfccfc	add uri salt and fix api url namespaces	2024-08-17 21:56:23 -07:00
Nick Sweeting	fba3995d86	reorder version output	2024-08-17 19:38:51 -07:00
Nick Sweeting	6a33de6715	remove archive.today link and add JSON	2024-08-17 19:31:43 -07:00
Nick Sweeting	3a87a7fb8c	re-arrange snapshot detail page preference and add opus support	2024-08-17 19:31:34 -07:00
Nick Sweeting	5e0cc926f1	show ulid in archivebox admin	2024-08-17 19:31:18 -07:00
Nick Sweeting	b7745ae37a	add uri salt when creating ABID	2024-08-17 19:30:58 -07:00
Lucas Schwiderski	10f0c4a077	Implement searching individual fields ... So far only for the public view, since we already have a custom search form there, where this is easy to add. This initial implementation supports the common set of metadata fields that the searchbar placeholder also mentions, but adding more fields is trivial.	2024-06-19 16:50:16 +02:00
Nick Sweeting	99e6f0c93f	bump versions	2024-06-04 04:17:33 -07:00
Nick Sweeting	de489d3c60	minor snapshot details ui fixes and migrations log msg improvements	2024-06-04 04:17:32 -07:00
Nick Sweeting	f574d34357	wrap migrations maker in try catch	2024-06-03 03:02:00 -07:00
Nick Sweeting	c63917a22d	fix more CHROME_USER_DATA_DIR issues	2024-06-03 02:51:11 -07:00
Nick Sweeting	1cd62ecc61	add filesizes and stray files in snapshot dir to snapshot_live ui	2024-06-03 02:31:35 -07:00
Nick Sweeting	729f05ab93	fix snapshot_live iframe scrollTo not working due to origin sandbox rules	2024-06-02 19:15:12 -07:00
Nick Sweeting	11a24d9640	merge migrations between branches	2024-06-02 19:14:58 -07:00
Nick Sweeting	b3f2a71c93	bump requirements and version to 0.8.1	2024-06-02 19:14:45 -07:00
Nick Sweeting	07d80db5aa	fix CHROME_USER_DATA_DIR split str error	2024-06-02 19:14:20 -07:00
Nick Sweeting	f72bae8eec	autodiscover plugins on startup	2024-05-17 20:48:11 -07:00
Nick Sweeting	48becde9b4	add new pydantic-based plugin system	2024-05-17 20:13:54 -07:00
Nick Sweeting	e4176dbf7a	change live snapshot preview iframe sandbox rules	2024-05-17 20:12:37 -07:00
Nick Sweeting	774ce3fda7	fix singlefile extractor exception when result is none	2024-05-17 20:12:18 -07:00
Nick Sweeting	29c7aa26bc	show original section titles in config admin ui	2024-05-17 20:11:54 -07:00
Nick Sweeting	acfd346440	make abids searchable in the admin ui	2024-05-17 20:11:31 -07:00
Nick Sweeting	a1afd0211f	fix abid calculation	2024-05-17 20:11:00 -07:00
Nick Sweeting	fdf6f465db	add docstrings	2024-05-14 18:46:38 -07:00
Nick Sweeting	406f57031a	add API support for obj.pk .uuid .abid	2024-05-13 08:52:02 -07:00
Nick Sweeting	a4cc10d7f8	add migrations for third round of field changes	2024-05-13 07:50:22 -07:00
Nick Sweeting	241a7c6ab2	add created, modified, updated, created_by and update django admin	2024-05-13 07:50:07 -07:00
Nick Sweeting	1ba8215072	only use domain part of uri for hash	2024-05-13 07:49:36 -07:00
Nick Sweeting	206e7e74b3	add migrations to create and populate ABIDField and UUIDField values	2024-05-13 05:13:42 -07:00
Nick Sweeting	0420662174	switch everywhere to use Snapshot.pk and ArchiveResult.pk instead of id	2024-05-13 05:12:12 -07:00
Nick Sweeting	9733b8d04c	remove accidentally commited db	2024-05-13 02:38:02 -07:00
Nick Sweeting	4f9f22e024	create abid_utils with new ABID type for ArchiveBox IDs	2024-05-13 02:37:48 -07:00
Nick Sweeting	f896e5dbeb	switch from monkey patching WebhookModel to using swappable	2024-05-13 02:36:15 -07:00
Nick Sweeting	e97d779cd3	move monkey patches to dedicated file	2024-05-13 02:35:19 -07:00
Nick Sweeting	b5ad134264	dont wait for ipython history saver thread before shell exit	2024-05-12 19:25:55 -07:00
Nick Sweeting	ce833e8ead	automatically create storage directories and symlinks based on ulid	2024-05-12 05:21:58 -07:00
Nick Sweeting	33bc4622a0	add ulid and typeid to Snapshot and ArchiveResult	2024-05-12 04:45:34 -07:00
Nick Sweeting	0529099639	create cache dir and sqlite db	2024-05-12 01:42:34 -07:00
Nick Sweeting	e1a04729b3	wait for bg threads to finish before exiting	2024-05-12 01:42:20 -07:00
Nick Sweeting	0438924491	remove unused template vars	2024-05-11 22:43:26 -07:00
Nick Sweeting	2b4b6e5b3a	select single snapshot row automatically when hotlinking to admin actions	2024-05-11 22:33:39 -07:00
Nick Sweeting	27809f2976	fix python3.10 nested fstring quotes	2024-05-11 22:33:27 -07:00
Nick Sweeting	baf24d2d6a	change wording on 404 snapshot detail page	2024-05-11 22:33:17 -07:00
Nick Sweeting	8841e8b181	add new live django template for snapshot detail page	2024-05-11 22:33:02 -07:00
Nick Sweeting	457c42bf84	load EXTRACTORS dynamically using importlib.import_module	2024-05-11 22:28:59 -07:00
Nick Sweeting	c7f55fc3ba	make CHROME_USER_DATA_DIR error quieter	2024-05-11 22:16:10 -07:00
Nick Sweeting	102e87578c	re-lock packages for python3.10 because we have to support it still	2024-05-11 15:06:19 -07:00
Nick Sweeting	913590ee39	explain weird use of ellipses magic value	2024-05-11 15:02:43 -07:00
Nick Sweeting	b90ba6c909	change header auth to use X-ArchiveBox-API-Key so it doesnt conflict with other auth headers	2024-05-08 20:02:47 -07:00
Nick Sweeting	4c5a3fba8b	more fixes for wget_output_path	2024-05-07 05:38:29 -07:00
Nick Sweeting	f2729c9dc7	revert wget character limit back to windows, ascii is allows MORE not less	2024-05-07 05:24:16 -07:00
Nick Sweeting	9b21ce490e	add workaround logic to catch paths that are too long or contain unprintable characters	2024-05-07 05:03:23 -07:00
Nick Sweeting	f62cb5fb43	change wget to use stricter ascii filepath normalization	2024-05-07 05:03:01 -07:00
Nick Sweeting	f770bba3cf	fix OSError 36 caused by checking for path that is too long to exist	2024-05-07 04:12:07 -07:00
Nick Sweeting	ef856e8051	Merge branch 'dev' into issue1316	2024-05-06 23:14:16 -07:00
Nick Sweeting	27d5d1ddc8	revert queryset intersection back to union for search results	2024-05-06 23:13:52 -07:00
Nick Sweeting	f472705d10	Change phrasing (#1419)	2024-05-06 23:11:26 -07:00
Nick Sweeting	3095265880	fix inner quote not escaped	2024-05-06 23:10:59 -07:00
Brandl	8ccd606973	Fix quotation ... Fixes: => ERROR [stage-0 22/23] RUN "/app"/bin/docker_entrypoint.sh version 2>&1 | tee -a /VERSION.txt 1.7s ------ > [stage-0 22/23] RUN "/app"/bin/docker_entrypoint.sh version 2>&1 | tee -a /VERSION.txt: 1.665 Traceback (most recent call last): 1.665 File "/usr/local/bin/archivebox", line 5, in <module> 1.665 from archivebox.cli import main 1.665 File "/app/archivebox/cli/__init__.py", line 83, in <module> 1.665 SUBCOMMANDS = list_subcommands() 1.665 ^^^^^^^^^^^^^^^^^^ 1.665 File "/app/archivebox/cli/__init__.py", line 43, in list_subcommands 1.665 module = import_module('.archivebox_{}'.format(subcommand), __package__) 1.665 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1.665 File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module 1.665 return _bootstrap._gcd_import(name[level:], package, level) 1.665 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 1.666 File "/app/archivebox/cli/archivebox_add.py", line 11, in <module> 1.666 from ..main import add 1.666 File "/app/archivebox/main.py", line 233 1.666 f'COMMIT_HASH={COMMIT_HASH[:7] if COMMIT_HASH else 'unknown'}', 1.666 ^^^^^^^ 1.666 SyntaxError: f-string: expecting '}'	2024-05-06 21:04:14 +02:00
Nick Sweeting	027c029316	redact passwords, keys, and secret tokens in admin UI	2024-05-06 11:06:42 -07:00
Nick Sweeting	8667ed29f1	improve API webhooks helptext and change app_label to API	2024-05-06 08:11:01 -07:00
Evan Boehs	f998647350	change phrasing	2024-05-06 10:32:36 -04:00
Nick Sweeting	d782bafe2e	fix storages missing stackfiles error	2024-05-06 07:14:01 -07:00
Nick Sweeting	47666ec26b	show webhooks config in django admin	2024-05-06 07:13:54 -07:00
Nick Sweeting	f067451267	fix django timezone.utc removed in 5.0	2024-05-06 07:13:25 -07:00
Nick Sweeting	c7fc9c004f	add django-signal-webhooks	2024-05-06 06:58:03 -07:00
Nick Sweeting	3805a1730d	add 0002 api migration	2024-04-30 21:45:02 -07:00
Nick Sweeting	e20eb52f15	fix COMMIT_HASH missing error	2024-04-30 21:43:22 -07:00
Nick Sweeting	18a5b6e99c	move EXTERNAL_LOCATIONS to data locations in version output	2024-04-25 21:36:43 -07:00
Nick Sweeting	6a6ae7468e	fix lint errors	2024-04-25 21:36:11 -07:00
Nick Sweeting	4a5ad32040	add django-requests-tracker	2024-04-25 18:02:01 -07:00
Nick Sweeting	af669d2f37	rename api files for clarity	2024-04-25 05:55:47 -07:00
Nick Sweeting	75153252dc	big overhaul of REST API, split into auth, core, and cli methods	2024-04-25 03:56:22 -07:00
Nick Sweeting	e5aba0dc2e	fix urljoin bug causing multiple slashes to be merged into one	2024-04-24 19:41:31 -07:00
Nick Sweeting	6cb357e76c	fix fix_url_from_markdown assertion to be valid url	2024-04-24 19:41:11 -07:00
Nick Sweeting	128419f991	expand comment about markdown url trailing paren trimming	2024-04-24 17:50:18 -07:00
Nick Sweeting	beb3932d80	replace uses of URL_REGEX with find_all_urls to handle markdown better	2024-04-24 17:45:45 -07:00
Nick Sweeting	c6d644be29	Merge branch 'dev' into issue1316	2024-04-24 16:24:16 -07:00
Nick Sweeting	98c5e69203	bump lockfiles	2024-04-24 14:38:21 -07:00
Nick Sweeting	8dcfa93ec6	Merge branch 'main' into dev	2024-04-24 14:32:07 -07:00
Nick Sweeting	17f40f3ada	Merge branch 'dev' into fix-URL_REGEX	2024-04-23 19:53:58 -07:00
Nick Sweeting	c6f8a33a63	Update util.py	2024-04-23 19:53:18 -07:00
Nick Sweeting	63fc317229	minor pylint fixes in logging_util	2024-04-23 17:45:36 -07:00
Nick Sweeting	756e159dfe	add new bin/lock_pkgs.sh to generate package lockfiles consistently	2024-04-23 17:45:36 -07:00
longzai	e4dc2701ef	fix URL_REGEX 2	2024-04-11 15:51:55 +08:00
Brandl	5f9aac18f2	api v1	2024-04-10 01:29:24 +02:00
longzai	4ae765ec27	fix the URL_REGEX used in generic_html parsers ... Signed-off-by: longzai <437172242@qq.com>	2024-04-08 04:53:05 +08:00
Nick Sweeting	e48159b8a0	cleanup docker-compose by storing crontabs in data dir	2024-03-26 15:24:05 -07:00
Nick Sweeting	ac73fb5129	merge fixes	2024-03-26 15:22:40 -07:00
Nick Sweeting	b4c3aa5097	Merge branch 'main' into dev	2024-03-26 15:01:36 -07:00
Nick Sweeting	a4453b6f87	fix PERSONAS PERSONAS_DIR typo	2024-03-26 14:19:25 -07:00
jim winstead	8b1b01e508	Update to Django 4.2.x, now in LTS until April 2026	2024-03-25 17:46:01 -07:00
Nick Sweeting	c0b5dbcecb	create new data/personas dir to hold cookies and chrome profiles	2024-03-18 14:41:39 -07:00
Nick Sweeting	c5bb99dce1	explicitly use Default profile inside user data dir	2024-03-18 14:40:40 -07:00
Nick Sweeting	1fc5d7c5c8	add USER_AGENT config option to set all USER_AGENTs at once	2024-03-18 14:39:09 -07:00
jim winstead	5478d13d52	Add generic_jsonl parser ... Resolves #1369	2024-03-14 15:42:29 -07:00
Nick Sweeting	ca2c484a8e	Add _EXTRA_ARGS for various extractors (#1360)	2024-03-14 01:55:09 -07:00
Nick Sweeting	48f4b12ae2	Use COOKIES_FILE to fetch page titles (#1364)	2024-03-14 01:52:44 -07:00
Nick Sweeting	099f7d00fe	Use feedparser for RSS parsing (#1362) ... Fixes #1171 Fixes #870 (probably, would need to test against a Wallabag Atom file to Fixes #135 Fixes #123 Fixes #106	2024-03-14 01:51:45 -07:00
Nick Sweeting	3512dc7e60	Disable searching for existing chrome user profiles by default	2024-03-14 00:58:45 -07:00
Ben Muthalaly	f4deb97f59	Add ARGS and EXTRA_ARGS for Mercury extractor	2024-03-05 21:15:38 -06:00
Ben Muthalaly	d8cf09c21e	Remove unnecessary variable length args for dedupe	2024-03-05 21:13:45 -06:00