472 commits

Author	SHA1	Message	Date
Nick Sweeting	295c5c46e0	add new crawl model	2024-10-01 21:47:16 -07:00
Nick Sweeting	f46d62a114	add py-machineid lib for new machine app	2024-10-01 21:46:35 -07:00
Nick Sweeting	276a505cae	fix extractor path calculation	2024-10-01 21:44:56 -07:00
Nick Sweeting	d21bc86075	finish migrating almost all config to new system	2024-09-30 23:21:34 -07:00
Nick Sweeting	2a1645ba27	fix import errors	2024-09-30 19:32:57 -07:00
Nick Sweeting	b913e6f426	rename OUTPUT_DIR to DATA_DIR	2024-09-30 17:44:18 -07:00
Nick Sweeting	363a499289	move util.py into misc folder	2024-09-30 17:25:15 -07:00
Nick Sweeting	dfca4b13b2	move system.py into misc folder	2024-09-30 17:13:55 -07:00
Nick Sweeting	3e5b6ddeae	move config into dedicated global app	2024-09-30 15:59:05 -07:00
Nick Sweeting	8d3f45b720	merge plugantic and abx, all praise be to praise our glorious pluggy gods	2024-09-27 01:26:12 -07:00
Nick Sweeting	8ed3155ec5	migrate plugin loading process to new pluggy-powered system based on djp	2024-09-26 02:43:12 -07:00
Nick Sweeting	0cfcabf6f4	fix admin data view configs type rendering	2024-09-26 02:41:22 -07:00
Nick Sweeting	ed45f58758	use constants in more places	2024-09-26 02:41:09 -07:00
Nick Sweeting	eb360f188a	remove old insecure index.json url serving from root	2024-09-26 02:38:59 -07:00
Nick Sweeting	d8a9dca0f6	use constants in more places	2024-09-26 02:38:45 -07:00
Nick Sweeting	bb65b2dbec	move almost all config into new archivebox.CONSTANTS	2024-09-25 05:10:09 -07:00
Nick Sweeting	6742888278	setup rich tracebacks width properly	2024-09-25 00:39:27 -07:00
Nick Sweeting	e99260feb2	fix rich logging issues	2024-09-24 21:17:07 -07:00
Nick Sweeting	bde0bf8263	load ipython rich extension in archivebox shell	2024-09-24 19:37:05 -07:00
Nick Sweeting	64c7100cf9	speed up startup time, add rich startup progressbar, split logging and checks into misc, fix search index import backend bug	2024-09-24 19:04:38 -07:00
Nick Sweeting	97695bda5e	more settings loading tweaks and improvements	2024-09-24 15:13:54 -07:00
Nick Sweeting	c9c163efed	begin migrating search backends to new plugin system	2024-09-24 02:13:01 -07:00
Nick Sweeting	e8f1264954	rename plugins dirs	2024-09-24 01:34:27 -07:00
Nick Sweeting	a9a97c013d	split plugin dirs, created new cleaner import path for plugin config in settings.py	2024-09-24 01:25:55 -07:00
Nick Sweeting	8f38f70e4a	define PACKAGE_DIR and DATA_DIR in settings.py directly	2024-09-22 19:26:26 -07:00
Nick Sweeting	3bbf8f69ab	cleanup settings.py sqlite settings more	2024-09-13 03:27:38 -07:00
Nick Sweeting	eae11cba19	add recommended SQLite db connection settings to avoid single-writer lock contention	2024-09-11 16:50:44 -07:00
Nick Sweeting	cecca8d169	allow deleting results from list page	2024-09-10 03:09:43 -07:00
Nick Sweeting	1ce09b88d7	show logs and workers in Django Admin data views	2024-09-10 00:05:45 -07:00
Nick Sweeting	60154fba5f	add django_huey, huey_monitor, and replace Threads with huey tasks	2024-09-10 00:05:45 -07:00
Nick Sweeting	db5428a486	increase maximum form field POST param size	2024-09-10 00:05:45 -07:00
Nick Sweeting	3f76e0a87f	fix migrations import errors	2024-09-06 03:48:52 -07:00
Nick Sweeting	f6ca48835a	hide more 404 and 304 errors from daphne logs	2024-09-06 02:54:53 -07:00
Nick Sweeting	52386d9c16	run all blocking commands in background threads and show nice UI messages as confirmation	2024-09-06 02:54:22 -07:00
Nick Sweeting	b56b1cac35	cleanup plugantic and pkg apps, make BaseHook actually create its own settings	2024-09-06 01:48:18 -07:00
Nick Sweeting	2e1e1945f2	add django-object-actions to provide Regenerate ABID button	2024-09-05 23:19:21 -07:00
Nick Sweeting	00aa7dc19f	setup daphne and django channels to replace runserver	2024-09-05 21:45:43 -07:00
Nick Sweeting	2c48dabfab	add preliminary support for ASGI/daphne serving	2024-09-05 21:43:42 -07:00
Nick Sweeting	c76c50e71f	add HTTP byte range request support to media file serving	2024-09-05 21:41:49 -07:00
Nick Sweeting	ba6c1fd69b	minor formatting and fixes	2024-09-05 04:39:46 -07:00
Nick Sweeting	44669fab73	add BaseHook concept to underlie all Plugin hooks	2024-09-05 03:36:18 -07:00
Nick Sweeting	ed5357cec9	add migrations for datetime field renames	2024-09-04 23:44:13 -07:00
Nick Sweeting	0fef2357c7	change default datetime template rendering to include seconds	2024-09-04 23:43:25 -07:00
Nick Sweeting	0c236b034b	allow accessing admin change pages by .abid as well as .id	2024-09-04 23:43:07 -07:00
Nick Sweeting	cbf2a8fdc3	rename datetime fields to _at, massively improve ABID generation safety and determinism	2024-09-04 23:42:36 -07:00
Nick Sweeting	68a39b7392	remove .old_id entirely and make ABID generation only happen once on initial save	2024-09-04 16:40:15 -07:00
Nick Sweeting	9072f4a999	fix ABID added to default to bookmarked value	2024-09-04 02:34:27 -07:00
Nick Sweeting	4427869ae8	fix ABID generation by chopping ts_src precision to consistent length	2024-09-04 02:02:29 -07:00
Nick Sweeting	d060eaa499	abid gradual improvements, some regrets	2024-09-04 00:08:14 -07:00
Nick Sweeting	ae13f1811f	better ABID display in admin UI	2024-09-03 17:11:10 -07:00
Nick Sweeting	f915ef6c5e	try to improve django console log 404 and 200 error filtering	2024-09-03 14:17:02 -07:00
Nick Sweeting	01094ecb03	fix REST API CSRF and auth handling	2024-09-03 14:16:44 -07:00
Nick Sweeting	9af260df16	BasePlugin system expanded and registration system improved	2024-09-03 00:58:50 -07:00
Nick Sweeting	9be7e1a0bc	change Snapshot admin action UI button text	2024-09-02 01:14:06 -07:00
Nick Sweeting	da76a84c45	rename pkgs app to pkg	2024-08-30 03:32:36 -07:00
Nick Sweeting	1e73a06ba0	change ABIDModel.created to use AutoTimeField seeded on .save instead of auto_now_add so that ts_src for ABID is available on creation before DB row is created	2024-08-28 03:02:37 -07:00
Nick Sweeting	e558d71b10	fix uuid checks on save	2024-08-28 03:01:19 -07:00
Nick Sweeting	b789be098d	fix type hint	2024-08-28 00:26:37 -07:00
Nick Sweeting	d0fefc0279	add chunk_size=500 to more iterator calls	2024-08-27 19:28:00 -07:00
Nick Sweeting	d7e333b5cf	fix settings.py for prod	2024-08-26 20:20:33 -07:00
Nick Sweeting	24fe958ff3	massively improve Snapshot admin list view query performance	2024-08-26 20:16:43 -07:00
Nick Sweeting	917af4b1d7	add pydantic logfire and improve settings for debugging	2024-08-26 20:14:47 -07:00
Nick Sweeting	5fe3edd79a	new plugin loading system	2024-08-23 02:02:34 -07:00
Nick Sweeting	34389e5e7c	improve CSRF_TRUSTED_ORIGINS loading logic	2024-08-23 02:01:40 -07:00
Nick Sweeting	2c2d034d6d	move to new vendoring fallback logic	2024-08-23 02:01:02 -07:00
Nick Sweeting	9c35f3ddb7	add new CSRF_TRUSTED_ORIGINS config option	2024-08-22 18:40:47 -07:00
Nick Sweeting	73a3e6aad0	handle tag with no slug or name	2024-08-22 18:25:15 -07:00
Nick Sweeting	80d7a30f13	improve version_str detection	2024-08-22 17:57:33 -07:00
Nick Sweeting	1d31b88fa3	fix migration failing when Tag name is empty	2024-08-22 16:30:25 -07:00
Nick Sweeting	09553d8340	hardcode EXTRACTOR_CHOICES to prevent nondeterministic migrations	2024-08-22 15:36:02 -07:00
Nick Sweeting	afe1307617	fix created_by field migration to create User properly if none exists	2024-08-22 15:20:36 -07:00
Nick Sweeting	9b1659c72f	make created_by_id autoapply to any ArchiveResults created under Snapshot	2024-08-20 19:43:07 -07:00
Nick Sweeting	c30ae1d2cb	add created_by_id to all Snapshot creation functions	2024-08-20 19:28:28 -07:00
Nick Sweeting	aa282daadf	show CONSTANTS in separate section of django admin config	2024-08-20 19:27:21 -07:00
Nick Sweeting	ea81f2fc14	use ABIDAdmin as base ModelAdmin	2024-08-20 19:26:40 -07:00
Nick Sweeting	0285aa52a0	config and attr access improvements	2024-08-20 18:31:21 -07:00
Nick Sweeting	4ae186dfca	fix ABID generation consistency when self._state.adding is True	2024-08-20 05:56:19 -07:00
Nick Sweeting	75018ed10b	Merge branch 'dev' into feat/search-fields	2024-08-20 03:37:56 -07:00
Nick Sweeting	267964881c	fix tags editor in Snapshot list view	2024-08-20 03:29:57 -07:00
Nick Sweeting	849b4963a1	add migrations	2024-08-20 01:58:44 -07:00
Nick Sweeting	52a813aa80	fix title display in admin UI and abid filter matching in urls	2024-08-20 01:58:36 -07:00
Nick Sweeting	9273db528e	fix abid generation migrations to be historically consistent	2024-08-20 01:58:19 -07:00
Nick Sweeting	506b3d28d4	fix admin UI TagInline and ArchiveResultInline form POST handling	2024-08-20 01:57:07 -07:00
Nick Sweeting	c4ef2993b2	update REST API and Admin UI to use new id and old_id exclusively	2024-08-19 20:00:04 -07:00
Nick Sweeting	344e902fc6	migrate SnapshotTag to use new snapshot id	2024-08-19 19:42:25 -07:00
Nick Sweeting	74c11d41be	dont allow modifying Snapshot.timestamp as it would break folder dirs	2024-08-19 19:42:01 -07:00
Nick Sweeting	cf2faecf61	add migrations for SnapshotTag through model	2024-08-19 18:36:20 -07:00
Nick Sweeting	2c157f0b5b	increase max POST field limit for admin	2024-08-19 18:34:52 -07:00
Nick Sweeting	18ebaed7ed	fix admin UI display of tags	2024-08-19 18:34:01 -07:00
Nick Sweeting	57d31b2b14	fix snapshot uuid	2024-08-18 01:07:21 -07:00
Nick Sweeting	8c50257fe9	move snapshot id to old_id	2024-08-18 00:24:38 -07:00
Nick Sweeting	a271bcb4ce	use new ids for ArchiveResult API	2024-08-17 23:03:43 -07:00
Nick Sweeting	f72debfdb2	migrate ArchiveResult.id to old_id, and make uuid main id	2024-08-17 22:58:35 -07:00
Nick Sweeting	033ec08d0c	save snapshot ids during migration	2024-08-17 21:56:45 -07:00
Nick Sweeting	4d0bbfccfc	add uri salt and fix api url namespaces	2024-08-17 21:56:23 -07:00
Nick Sweeting	3a87a7fb8c	re-arrange snapshot detail page preference and add opus support	2024-08-17 19:31:34 -07:00
Nick Sweeting	5e0cc926f1	show ulid in archivebox admin	2024-08-17 19:31:18 -07:00
Lucas Schwiderski	10f0c4a077	Implement searching individual fields ... So far only for the public view, since we already have a custom search form there, where this is easy to add. This initial implementation supports the common set of metadata fields that the searchbar placeholder also mentions, but adding more fields is trivial.	2024-06-19 16:50:16 +02:00
Nick Sweeting	de489d3c60	minor snapshot details ui fixes and migrations log msg improvements	2024-06-04 04:17:32 -07:00
Nick Sweeting	1cd62ecc61	add filesizes and stray files in snapshot dir to snapshot_live ui	2024-06-03 02:31:35 -07:00