487 commits

Author	SHA1	Message	Date
Nick Sweeting	2e1e1945f2	add django-object-actions to provide Regenerate ABID button	2024-09-05 23:19:21 -07:00
Nick Sweeting	00aa7dc19f	setup daphne and django channels to replace runserver	2024-09-05 21:45:43 -07:00
Nick Sweeting	2c48dabfab	add preliminary support for ASGI/daphne serving	2024-09-05 21:43:42 -07:00
Nick Sweeting	c76c50e71f	add HTTP byte range request support to media file serving	2024-09-05 21:41:49 -07:00
Nick Sweeting	ba6c1fd69b	minor formatting and fixes	2024-09-05 04:39:46 -07:00
Nick Sweeting	44669fab73	add BaseHook concept to underlie all Plugin hooks	2024-09-05 03:36:18 -07:00
Nick Sweeting	ed5357cec9	add migrations for datetime field renames	2024-09-04 23:44:13 -07:00
Nick Sweeting	0fef2357c7	change default datetime template rendering to include seconds	2024-09-04 23:43:25 -07:00
Nick Sweeting	0c236b034b	allow accessing admin change pages by .abid as well as .id	2024-09-04 23:43:07 -07:00
Nick Sweeting	cbf2a8fdc3	rename datetime fields to _at, massively improve ABID generation safety and determinism	2024-09-04 23:42:36 -07:00
Nick Sweeting	68a39b7392	remove .old_id entirely and make ABID generation only happen once on initial save	2024-09-04 16:40:15 -07:00
Nick Sweeting	9072f4a999	fix ABID added to default to bookmarked value	2024-09-04 02:34:27 -07:00
Nick Sweeting	4427869ae8	fix ABID generation by chopping ts_src precision to consistent length	2024-09-04 02:02:29 -07:00
Nick Sweeting	d060eaa499	abid gradual improvements, some regrets	2024-09-04 00:08:14 -07:00
Nick Sweeting	ae13f1811f	better ABID display in admin UI	2024-09-03 17:11:10 -07:00
Nick Sweeting	f915ef6c5e	try to improve django console log 404 and 200 error filtering	2024-09-03 14:17:02 -07:00
Nick Sweeting	01094ecb03	fix REST API CSRF and auth handling	2024-09-03 14:16:44 -07:00
Nick Sweeting	9af260df16	BasePlugin system expanded and registration system improved	2024-09-03 00:58:50 -07:00
Nick Sweeting	9be7e1a0bc	change Snapshot admin action UI button text	2024-09-02 01:14:06 -07:00
Nick Sweeting	da76a84c45	rename pkgs app to pkg	2024-08-30 03:32:36 -07:00
Nick Sweeting	1e73a06ba0	change ABIDModel.created to use AutoTimeField seeded on .save instead of auto_now_add so that ts_src for ABID is available on creation before DB row is created	2024-08-28 03:02:37 -07:00
Nick Sweeting	e558d71b10	fix uuid checks on save	2024-08-28 03:01:19 -07:00
Nick Sweeting	b789be098d	fix type hint	2024-08-28 00:26:37 -07:00
Nick Sweeting	d0fefc0279	add chunk_size=500 to more iterator calls	2024-08-27 19:28:00 -07:00
Nick Sweeting	d7e333b5cf	fix settings.py for prod	2024-08-26 20:20:33 -07:00
Nick Sweeting	24fe958ff3	massively improve Snapshot admin list view query performance	2024-08-26 20:16:43 -07:00
Nick Sweeting	917af4b1d7	add pydantic logfire and improve settings for debugging	2024-08-26 20:14:47 -07:00
Nick Sweeting	5fe3edd79a	new plugin loading system	2024-08-23 02:02:34 -07:00
Nick Sweeting	34389e5e7c	improve CSRF_TRUSTED_ORIGINS loading logic	2024-08-23 02:01:40 -07:00
Nick Sweeting	2c2d034d6d	move to new vendoring fallback logic	2024-08-23 02:01:02 -07:00
Nick Sweeting	9c35f3ddb7	add new CSRF_TRUSTED_ORIGINS config option	2024-08-22 18:40:47 -07:00
Nick Sweeting	73a3e6aad0	handle tag with no slug or name	2024-08-22 18:25:15 -07:00
Nick Sweeting	80d7a30f13	improve version_str detection	2024-08-22 17:57:33 -07:00
Nick Sweeting	1d31b88fa3	fix migration failing when Tag name is empty	2024-08-22 16:30:25 -07:00
Nick Sweeting	09553d8340	hardcode EXTRACTOR_CHOICES to prevent nondeterministic migrations	2024-08-22 15:36:02 -07:00
Nick Sweeting	afe1307617	fix created_by field migration to create User properly if none exists	2024-08-22 15:20:36 -07:00
Nick Sweeting	9b1659c72f	make created_by_id autoapply to any ArchiveResults created under Snapshot	2024-08-20 19:43:07 -07:00
Nick Sweeting	c30ae1d2cb	add created_by_id to all Snapshot creation functions	2024-08-20 19:28:28 -07:00
Nick Sweeting	aa282daadf	show CONSTANTS in separate section of django admin config	2024-08-20 19:27:21 -07:00
Nick Sweeting	ea81f2fc14	use ABIDAdmin as base ModelAdmin	2024-08-20 19:26:40 -07:00
Nick Sweeting	0285aa52a0	config and attr access improvements	2024-08-20 18:31:21 -07:00
Nick Sweeting	4ae186dfca	fix ABID generation consistency when self._state.adding is True	2024-08-20 05:56:19 -07:00
Nick Sweeting	75018ed10b	Merge branch 'dev' into feat/search-fields	2024-08-20 03:37:56 -07:00
Nick Sweeting	267964881c	fix tags editor in Snapshot list view	2024-08-20 03:29:57 -07:00
Nick Sweeting	849b4963a1	add migrations	2024-08-20 01:58:44 -07:00
Nick Sweeting	52a813aa80	fix title display in admin UI and abid filter matching in urls	2024-08-20 01:58:36 -07:00
Nick Sweeting	9273db528e	fix abid generation migrations to be historically consistent	2024-08-20 01:58:19 -07:00
Nick Sweeting	506b3d28d4	fix admin UI TagInline and ArchiveResultInline form POST handling	2024-08-20 01:57:07 -07:00
Nick Sweeting	c4ef2993b2	update REST API and Admin UI to use new id and old_id exclusively	2024-08-19 20:00:04 -07:00
Nick Sweeting	344e902fc6	migrate SnapshotTag to use new snapshot id	2024-08-19 19:42:25 -07:00
Nick Sweeting	74c11d41be	dont allow modifying Snapshot.timestamp as it would break folder dirs	2024-08-19 19:42:01 -07:00
Nick Sweeting	cf2faecf61	add migrations for SnapshotTag through model	2024-08-19 18:36:20 -07:00
Nick Sweeting	2c157f0b5b	increase max POST field limit for admin	2024-08-19 18:34:52 -07:00
Nick Sweeting	18ebaed7ed	fix admin UI display of tags	2024-08-19 18:34:01 -07:00
Nick Sweeting	57d31b2b14	fix snapshot uuid	2024-08-18 01:07:21 -07:00
Nick Sweeting	8c50257fe9	move snapshot id to old_id	2024-08-18 00:24:38 -07:00
Nick Sweeting	a271bcb4ce	use new ids for ArchiveResult API	2024-08-17 23:03:43 -07:00
Nick Sweeting	f72debfdb2	migrate ArchiveResult.id to old_id, and make uuid main id	2024-08-17 22:58:35 -07:00
Nick Sweeting	033ec08d0c	save snapshot ids during migration	2024-08-17 21:56:45 -07:00
Nick Sweeting	4d0bbfccfc	add uri salt and fix api url namespaces	2024-08-17 21:56:23 -07:00
Nick Sweeting	3a87a7fb8c	re-arrange snapshot detail page preference and add opus support	2024-08-17 19:31:34 -07:00
Nick Sweeting	5e0cc926f1	show ulid in archivebox admin	2024-08-17 19:31:18 -07:00
Lucas Schwiderski	10f0c4a077	Implement searching individual fields ... So far only for the public view, since we already have a custom search form there, where this is easy to add. This initial implementation supports the common set of metadata fields that the searchbar placeholder also mentions, but adding more fields is trivial.	2024-06-19 16:50:16 +02:00
Nick Sweeting	de489d3c60	minor snapshot details ui fixes and migrations log msg improvements	2024-06-04 04:17:32 -07:00
Nick Sweeting	1cd62ecc61	add filesizes and stray files in snapshot dir to snapshot_live ui	2024-06-03 02:31:35 -07:00
Nick Sweeting	f72bae8eec	autodiscover plugins on startup	2024-05-17 20:48:11 -07:00
Nick Sweeting	48becde9b4	add new pydantic-based plugin system	2024-05-17 20:13:54 -07:00
Nick Sweeting	29c7aa26bc	show original section titles in config admin ui	2024-05-17 20:11:54 -07:00
Nick Sweeting	acfd346440	make abids searchable in the admin ui	2024-05-17 20:11:31 -07:00
Nick Sweeting	a1afd0211f	fix abid calculation	2024-05-17 20:11:00 -07:00
Nick Sweeting	fdf6f465db	add docstrings	2024-05-14 18:46:38 -07:00
Nick Sweeting	a4cc10d7f8	add migrations for third round of field changes	2024-05-13 07:50:22 -07:00
Nick Sweeting	241a7c6ab2	add created, modified, updated, created_by and update django admin	2024-05-13 07:50:07 -07:00
Nick Sweeting	206e7e74b3	add migrations to create and populate ABIDField and UUIDField values	2024-05-13 05:13:42 -07:00
Nick Sweeting	0420662174	switch everywhere to use Snapshot.pk and ArchiveResult.pk instead of id	2024-05-13 05:12:12 -07:00
Nick Sweeting	4f9f22e024	create abid_utils with new ABID type for ArchiveBox IDs	2024-05-13 02:37:48 -07:00
Nick Sweeting	f896e5dbeb	switch from monkey patching WebhookModel to using swappable	2024-05-13 02:36:15 -07:00
Nick Sweeting	ce833e8ead	automatically create storage directories and symlinks based on ulid	2024-05-12 05:21:58 -07:00
Nick Sweeting	33bc4622a0	add ulid and typeid to Snapshot and ArchiveResult	2024-05-12 04:45:34 -07:00
Nick Sweeting	0529099639	create cache dir and sqlite db	2024-05-12 01:42:34 -07:00
Nick Sweeting	0438924491	remove unused template vars	2024-05-11 22:43:26 -07:00
Nick Sweeting	27809f2976	fix python3.10 nested fstring quotes	2024-05-11 22:33:27 -07:00
Nick Sweeting	baf24d2d6a	change wording on 404 snapshot detail page	2024-05-11 22:33:17 -07:00
Nick Sweeting	8841e8b181	add new live django template for snapshot detail page	2024-05-11 22:33:02 -07:00
Nick Sweeting	457c42bf84	load EXTRACTORS dynamically using importlib.import_module	2024-05-11 22:28:59 -07:00
Nick Sweeting	913590ee39	explain weird use of ellipses magic value	2024-05-11 15:02:43 -07:00
Nick Sweeting	ef856e8051	Merge branch 'dev' into issue1316	2024-05-06 23:14:16 -07:00
Nick Sweeting	27d5d1ddc8	revert queryset intersection back to union for search results	2024-05-06 23:13:52 -07:00
Nick Sweeting	f472705d10	Change phrasing (#1419)	2024-05-06 23:11:26 -07:00
Nick Sweeting	3095265880	fix inner quote not escaped	2024-05-06 23:10:59 -07:00
Nick Sweeting	027c029316	redact passwords, keys, and secret tokens in admin UI	2024-05-06 11:06:42 -07:00
Nick Sweeting	8667ed29f1	improve API webhooks helptext and change app_label to API	2024-05-06 08:11:01 -07:00
Evan Boehs	f998647350	change phrasing	2024-05-06 10:32:36 -04:00
Nick Sweeting	d782bafe2e	fix storages missing stackfiles error	2024-05-06 07:14:01 -07:00
Nick Sweeting	47666ec26b	show webhooks config in django admin	2024-05-06 07:13:54 -07:00
Nick Sweeting	c7fc9c004f	add django-signal-webhooks	2024-05-06 06:58:03 -07:00
Nick Sweeting	6a6ae7468e	fix lint errors	2024-04-25 21:36:11 -07:00
Nick Sweeting	4a5ad32040	add django-requests-tracker	2024-04-25 18:02:01 -07:00
Nick Sweeting	75153252dc	big overhaul of REST API, split into auth, core, and cli methods	2024-04-25 03:56:22 -07:00
Nick Sweeting	c6d644be29	Merge branch 'dev' into issue1316	2024-04-24 16:24:16 -07:00
Nick Sweeting	8dcfa93ec6	Merge branch 'main' into dev	2024-04-24 14:32:07 -07:00
Brandl	5f9aac18f2	api v1	2024-04-10 01:29:24 +02:00
jim winstead	8b1b01e508	Update to Django 4.2.x, now in LTS until April 2026	2024-03-25 17:46:01 -07:00
Nick Sweeting	e8772513ca	Update archivebox/core/auth_ldap.py ... add log line	2024-01-24 11:00:42 -08:00
Vladimir D	914df75ee7	Merge branch 'dev' into feature/is_staff-is_superuser-flags-defaults	2024-01-24 22:25:27 +04:00
Vladimir D	95580ee743	populate is_staff and is_superuser flags at LDAP authentication	2024-01-24 22:18:02 +04:00
Nick Sweeting	c7cab7cadc	Show the upgrade notification only in specific views (#1314)	2024-01-23 10:29:14 -08:00
Neel Suthar	279883d6bb	Text Search and Filters don't work at the same time in the web UI #1316 ... Making sure to return distinct results. Changing set operation to '&' to show the matching results from filters AND search term	2024-01-21 17:34:22 -06:00
Neel Suthar	aca0512926	Fix for Bug: Search sometimes shows the same snapshot twice #1193 ... Making sure the search results are unique	2024-01-19 17:03:51 -06:00
Neel Suthar	702b81f9e6	Fixes #1193 Bug: Search sometimes shows the same snapshot twice ... Making sure user gets the distinct set of search results	2024-01-19 13:49:09 -06:00
Nick Sweeting	d0cd84a2af	Merge branch 'dev' into specific-version-banner	2024-01-19 04:01:32 -08:00
Nick Sweeting	ef1a697588	fix sorting by Size or by Files to sort by number of archive results	2024-01-19 03:47:38 -08:00
Ben Muthalaly	5de45dbf30	Show upgrade notification in admin snapshot view	2024-01-08 22:55:30 -06:00
Nick Sweeting	1532186283	Merge branch 'main' into dev	2024-01-04 12:30:46 -08:00
Nick Sweeting	c26d9dc1bb	link to exact build commit hash in public index footer	2024-01-03 20:09:04 -08:00
Nick Sweeting	fcdc41a1ab	re-enable autocomplete tag field for now as its breaking action buttons display	2024-01-03 19:09:14 -08:00
Nick Sweeting	4af9beedd1	disable GLOBAL_CONTEXT used by version upgrade banner as not all admin views accept extra_context kwarg	2024-01-03 16:11:27 -08:00
Nick Sweeting	168cdac2c4	rename VERSION_RELEASES to VERSIONS_AVAILABLE	2023-12-19 09:58:42 -08:00
Nick Sweeting	480eff52db	Update urls.py	2023-12-19 09:58:15 -08:00
Nick Sweeting	14f10a0461	Merge branch 'dev' into version-banner	2023-12-19 09:57:08 -08:00
Nick Sweeting	5805e18380	Update settings.py	2023-12-18 19:25:08 -08:00
Nick Sweeting	7d5a7f80ca	Update admin.py	2023-12-18 19:24:48 -08:00
Nick Sweeting	7d06836443	add WIP comment	2023-12-18 19:24:18 -08:00
Vladimir D	37bdcbf950	make local auth working along with ldap auth so that local superuser could manage ldap users	2023-12-06 16:16:51 +04:00
Ben Muthalaly	a3fd8a8ecd	Move version banner logic from JS to Python ... Also adds CSS styling to banner.	2023-11-27 22:56:30 -06:00
Ben Muthalaly	7599dbb79d	Add notification about upgrade to admin page	2023-11-23 03:04:31 -06:00
Nick Sweeting	6761ed368c	wip pre-0.7.1 side fixes	2023-11-14 02:09:58 -08:00
Nick Sweeting	a680724367	Merge branch 'dev' into search_index_extract_html_text	2023-10-27 23:09:28 -07:00
Ross Williams	310b4d1242	Add htmltotext extractor ... Saves HTML text nodes and selected element attributes in `htmltotext.txt` for each Snapshot. Primarily intended to be used for search indexing.	2023-10-23 21:42:32 -04:00
Nick Sweeting	63ad43f46c	Merge branch 'dev' into method_allow_deny	2023-10-20 04:25:44 -07:00
Nick Sweeting	76f9b91ed3	dockerfile fixes	2023-10-20 04:08:38 -07:00
Nick Sweeting	85f8583d62	catch ldap configuration and packaging errors and make them non fatal	2023-10-19 18:05:59 -07:00
Nick Sweeting	ffe2968e4f	improve some comments	2023-09-14 02:41:27 -07:00
root	23f086aa40	add LDAP support	2023-08-17 19:51:02 -05:00
Ross Williams	46e80dd509	Rename URL_(WHITE|BLACK)LIST to URL_(ALLOW|DENY)LIST ... Retain aliases for old configuration files	2023-08-02 09:31:48 -04:00
Nick Sweeting	2538b170c7	Merge branch 'dev' into feat/reverse-proxy-auth	2023-01-09 18:20:45 -08:00
Joseph Turian	22d8e57637	Add missing migration 0021	2022-09-14 09:36:17 +00:00
Nick Sweeting	375ba9d135	Update settings.py	2022-06-08 20:00:29 -07:00
Igor Rzegocki	d4f534e612	add LOGOUT_REDIRECT_URL	2022-03-31 21:40:14 +02:00
Igor Rzegocki	05de1c9fe6	healthcheck endpoint	2021-10-03 19:12:03 +02:00
Igor Rzegocki	95cf85f8cf	Support for Reverse Proxy authentication backends (like authelia)	2021-09-30 17:40:13 +02:00
TJ Horner	cdcfb7fa44	Exempt /add route from CSRF	2021-07-01 20:55:51 -04:00
Nick Sweeting	217952c8d0	sigh linters	2021-04-11 12:51:50 -04:00
Nick Sweeting	bfefe042bf	fix noqa	2021-04-11 03:57:04 -04:00
Nick Sweeting	4524129965	dont prevent search when backend throws an exception or times out	2021-04-11 00:49:35 -04:00
Nick Sweeting	6a6ef63775	enforce utf8 on more file reads	2021-04-10 07:22:32 -04:00
Nick Sweeting	193df5c8d3	add video subtitles and description to full-text index	2021-04-10 07:22:20 -04:00
Nick Sweeting	6949803395	enforce new models to use uuid keys	2021-04-10 06:32:45 -04:00
Nick Sweeting	1b67dd5c45	fix issues with django 3.2	2021-04-10 06:22:07 -04:00
Nick Sweeting	89158d5dc7	allow full-text search from public index	2021-04-10 05:14:02 -04:00