1984 commits

Author	SHA1	Message	Date
Nick Sweeting	684a394cba	add HOSTNAME to config.permissions	2024-11-16 02:45:58 -08:00
Nick Sweeting	b4a5da3ffd	update archivebox add CLI command to use new actor system	2024-11-16 02:45:37 -08:00
Nick Sweeting	43514da0d0	add crawl and seed endpoints to REST API	2024-11-16 02:45:11 -08:00
Nick Sweeting	48bb634b75	fix orchestrator startup and add exit_on_idle option	2024-11-16 02:44:57 -08:00
Nick Sweeting	c3d692b5d5	fix minor actor erros around CLAIM_ATOMIC	2024-11-16 02:44:33 -08:00
Nick Sweeting	7c0e3dcc21	load crawls,seeds,actors apps as pluggy plugins	2024-11-16 02:44:11 -08:00
Nick Sweeting	ed43f1d027	better docstrings and comments	2024-11-15 23:21:40 -08:00
Nick Sweeting	ec100bfe29	fix docs build for vendored pkgs	2024-11-12 23:53:34 -08:00
Nick Sweeting	57852fd89e	fix sphinx docs build	2024-11-12 22:20:11 -08:00
Nick Sweeting	a9a3b153b1	more StateMachine, Actor, and Orchestrator improvements	2024-11-04 07:08:39 -08:00
Nick Sweeting	1148cadd7a	Update __init__.py	2024-11-03 16:12:29 -05:00
Nick Sweeting	b6ab4e298e	merge dev	2024-11-03 12:56:44 -08:00
Nick Sweeting	758c0c6774	add user providable PLAYWRIGHT cache dir	2024-11-03 12:54:04 -08:00
Nick Sweeting	50a85ec97b	Update archivebox/plugins_pkg/playwright/binproviders.py	2024-11-03 15:47:00 -05:00
Andrew Dunham	49c520914c	playwright: support PLAYWRIGHT_BROWSERS_PATH environment variable ... This environment variable is used by Playwright to configure where to install browsers. If specified, the given directory is used instead of the OS-specific cache folder. For compatibility with existing Playwright installations, and better control over where these binaries are installed, check the same environment variable in PlaywrightBinProvider. Signed-off-by: Andrew Dunham <andrew@du.nham.ca>	2024-11-03 13:59:20 -05:00
Nick Sweeting	48f8416762	add new core and crawsl statemachine manager	2024-11-03 00:41:11 -07:00
Nick Sweeting	41efd010f0	add wip crawl actor spec	2024-11-02 19:54:37 -07:00
Nick Sweeting	2337f874ad	better actor atomic claim	2024-11-02 19:54:25 -07:00
Nick Sweeting	9b24fe7390	merge dev	2024-11-02 17:34:33 -07:00
Nick Sweeting	dbe5c0bc07	more orchestrator and actor improvements	2024-11-02 17:25:51 -07:00
Nick Sweeting	721427a484	hide progress bar on startup	2024-10-31 07:11:15 -07:00
Nick Sweeting	17faa5a507	improvements to new actor and orchestrators	2024-10-31 07:11:15 -07:00
Nick Sweeting	9c2eac4e47	add new actors and orchestrators	2024-10-31 07:11:14 -07:00
Nick Sweeting	f56cdd2da5	add chrome flag to fix long screenshots getting cut off	2024-10-29 14:51:31 -07:00
Nick Sweeting	5efeb9d347	add get_SCOPE_CONFIG	2024-10-29 00:33:14 -07:00
Nick Sweeting	eb721bd514	tweak parser imports	2024-10-28 20:16:46 -07:00
Nick Sweeting	dee4eb7992	rename vendor dir to pkgs	2024-10-28 20:05:20 -07:00
Nick Sweeting	001056f292	remove vendored copy of pydantic-pkgr	2024-10-28 20:00:21 -07:00
Nick Sweeting	6530d1f4bf	remove vendored copy of pocket and add [debug] group of pkgs for running with DEBUG=True	2024-10-28 19:56:49 -07:00
Nick Sweeting	70926f1d9f	replace os.access with os.path.isdir	2024-10-28 19:06:38 -07:00
Nick Sweeting	d93aa46949	fix django.forms.JSONField does not exist 500 error	2024-10-28 18:47:45 -07:00
Nick Sweeting	d47d429e9d	add placeholder pyproj	2024-10-28 16:12:43 -07:00
Nick Sweeting	b3c1cb716e	move abx plugins inside vendor dir	2024-10-28 04:07:35 -07:00
Ben Muthalaly	4213d7dc27	Fix API crash	2024-10-26 01:53:49 -05:00
Nick Sweeting	5d9a32c364	wip	2024-10-25 01:06:12 -07:00
Nick Sweeting	4b6f08b0fe	swap more direct settings.CONFIG access to abx getters	2024-10-24 15:42:19 -07:00
Nick Sweeting	b61f6ff8d8	rename system_tasks queue to commands queue	2024-10-24 15:41:43 -07:00
Nick Sweeting	60f0458c77	rename configfile to collection	2024-10-24 15:40:24 -07:00
Nick Sweeting	b6e60e63bc	bump pydantic pkgr	2024-10-24 15:12:44 -07:00
Nick Sweeting	9e40dd69a4	more config improvements, move away from settings GLOBALS to getters	2024-10-24 14:50:07 -07:00
Nick Sweeting	b9492f7736	silence ABID drift warning	2024-10-24 14:50:07 -07:00
Nick Sweeting	082930b2ef	add wip states	2024-10-22 02:25:54 -07:00
Nick Sweeting	72f777059f	minor plugin improvements	2024-10-22 02:25:20 -07:00
Nick Sweeting	1dff8bae40	dont try to install brew casks on linux	2024-10-21 03:44:45 -07:00
Nick Sweeting	cd3b937469	fix syntax error	2024-10-21 03:21:48 -07:00
Nick Sweeting	657eec479b	fix CONSTANTS.LIB_DIR old style access	2024-10-21 03:20:20 -07:00
Nick Sweeting	9b7ad2cae6	fix Dockerfile and INSTALLER_BIN_ABSPATH when not available on host	2024-10-21 03:16:07 -07:00
Nick Sweeting	8a2b38b46c	tweak version output order	2024-10-21 03:12:48 -07:00
Nick Sweeting	312e40b95b	finally get rid of config/legacy in favor of configfile.py and django.py	2024-10-21 03:06:19 -07:00
Nick Sweeting	b3107ab830	move final legacy config to plugins and fix archivebox config cmd and add search opt	2024-10-21 02:56:00 -07:00
Nick Sweeting	5e743b6aee	create Chrome profile on first run automagically	2024-10-21 01:33:08 -07:00
Nick Sweeting	267fde0138	call validate functions at AppConfig.ready time manually	2024-10-21 01:32:53 -07:00
Nick Sweeting	7a6f1f36d2	trigger abx.pm.hook.ready from core.AppConfig.ready	2024-10-21 01:31:02 -07:00
Nick Sweeting	dd40f252ed	dont warn about update_in_place config more than once per key	2024-10-21 01:30:42 -07:00
Nick Sweeting	5023bdba2f	only list admins that are not system user	2024-10-21 00:38:36 -07:00
Nick Sweeting	a97cc82979	fix Crawl models	2024-10-21 00:38:29 -07:00
Nick Sweeting	d48a99ab64	only load django huey config when not getting help or version to avoid creating queue.sqlite3	2024-10-21 00:38:17 -07:00
Nick Sweeting	02a1fc3049	rename sessions app in INSTALLED_APPS to personas	2024-10-21 00:37:57 -07:00
Nick Sweeting	82db227561	move sessions app to personas to avoid app_label conflict	2024-10-21 00:36:17 -07:00
Nick Sweeting	a211461ffc	fix LIB_DIR and TMP_DIR loading when primary option isnt available	2024-10-21 00:35:56 -07:00
Nick Sweeting	35dd5911d8	WIP add new effects, reads, writes, to abx interface	2024-10-15 22:33:22 -07:00
Nick Sweeting	0abbc11a6b	add get_scope_config method to merge config down to scope	2024-10-15 22:32:36 -07:00
Nick Sweeting	c9f88f5875	add sessions and seeds models	2024-10-15 22:32:13 -07:00
Nick Sweeting	0866f4aaf3	add architecture mockup	2024-10-15 01:03:17 -07:00
Nick Sweeting	80d8a6b667	split archivebox.use into archivebox.reads and archivebox.writes	2024-10-15 01:03:01 -07:00
Nick Sweeting	aaf069fab0	remove tags field from Machine admin	2024-10-15 01:02:13 -07:00
Nick Sweeting	df79b8e038	rename config sections to match old sections	2024-10-15 01:01:34 -07:00
Nick Sweeting	7671813aa5	enable relevant hookspecs	2024-10-15 01:01:23 -07:00
Nick Sweeting	c04e0b2f39	add abx function to get plugin hooks	2024-10-15 01:01:10 -07:00
Nick Sweeting	8d87378a46	better docstrings in abx	2024-10-15 01:00:57 -07:00
Nick Sweeting	536e3ac3c4	make textarea django-jsonfield look nicer	2024-10-15 01:00:42 -07:00
Nick Sweeting	654bd22bd0	re-author plugins	2024-10-15 01:00:21 -07:00
Nick Sweeting	01ba6d49d3	new vastly simplified plugin spec without pydantic	2024-10-14 21:50:47 -07:00
Nick Sweeting	abf75f49f4	add tentative handling for Hooks as modules instead of BaseHook subclasses	2024-10-14 17:42:52 -07:00
Nick Sweeting	d69df359ea	remove Crawl migration in favor of separate app	2024-10-14 17:41:07 -07:00
Nick Sweeting	943f843189	add core/admin_site	2024-10-14 17:40:35 -07:00
Nick Sweeting	bb9c3fda14	fix makemigrations being blocked by check_migrations func	2024-10-14 17:40:06 -07:00
Nick Sweeting	2ebd28aebd	fix ABID and uniqueness for new Seed models	2024-10-14 17:39:43 -07:00
Nick Sweeting	59b669691f	fix Admin data view for Config to render both sections and individual values	2024-10-14 17:39:14 -07:00
Nick Sweeting	1d7f0ab20d	fix Tag creation via admin erroring because slug field is not filled	2024-10-14 17:38:53 -07:00
Nick Sweeting	c0b7887fd7	fix admin registration using abx hooks	2024-10-14 17:38:38 -07:00
Nick Sweeting	30923c340f	move mixins.py for search into search app	2024-10-14 17:37:30 -07:00
Nick Sweeting	8cff6ddfc6	move ldap_auth setup into LDAP plugin	2024-10-14 17:36:45 -07:00
Nick Sweeting	86380a1ef2	fix .archivebox_id being created outside collection dir	2024-10-14 17:35:43 -07:00
Nick Sweeting	9a04ed7c76	move serve_static and shell_welcome_message into misc	2024-10-14 17:35:28 -07:00
Nick Sweeting	5ac941cf2e	resolve circular import in core/views	2024-10-14 15:43:53 -07:00
Nick Sweeting	a0bef4e27b	move crawl model out of core	2024-10-14 15:42:36 -07:00
Nick Sweeting	f75ae805f8	comment out Crawl api methods temporarily	2024-10-14 15:41:58 -07:00
Nick Sweeting	536a5ea745	clear up Machine models cache vars	2024-10-14 15:38:32 -07:00
Nick Sweeting	b3b2c551c2	fix get_sock_file symlink process	2024-10-14 15:37:48 -07:00
Nick Sweeting	a7c19a5da0	add register_admin to abx hookspec	2024-10-14 15:35:44 -07:00
Nick Sweeting	4634cd6c8e	make should_extract take config and uri as args	2024-10-14 15:35:20 -07:00
Nick Sweeting	518c46b4ab	fix circular import and show log of plugins loading on startup	2024-10-14 15:35:03 -07:00
Nick Sweeting	ad4657861f	add new crawls app	2024-10-11 19:47:05 -07:00
Nick Sweeting	1768f147d4	fix playwright and puppeteer install methods on linux	2024-10-11 02:21:15 -07:00
Nick Sweeting	38cb1ac87b	add apt install method for chrome	2024-10-11 02:20:48 -07:00
Nick Sweeting	f12efbdb45	use new dry run system	2024-10-11 02:20:37 -07:00
Nick Sweeting	bffa9ee78d	fix puppeteer install using npx stalling waiting for y user confirmation	2024-10-11 01:03:34 -07:00
Nick Sweeting	146cdff7d9	fix apt not installing npm package as separate package	2024-10-11 01:03:21 -07:00
Nick Sweeting	6e7071bd19	add new binproviders and binaries args to install and version, bump pydantic-pkgr version	2024-10-11 00:45:59 -07:00