1901 commits

Author	SHA1	Message	Date
Nick Sweeting	86380a1ef2	fix .archivebox_id being created outside collection dir	2024-10-14 17:35:43 -07:00
Nick Sweeting	9a04ed7c76	move serve_static and shell_welcome_message into misc	2024-10-14 17:35:28 -07:00
Nick Sweeting	5ac941cf2e	resolve circular import in core/views	2024-10-14 15:43:53 -07:00
Nick Sweeting	a0bef4e27b	move crawl model out of core	2024-10-14 15:42:36 -07:00
Nick Sweeting	f75ae805f8	comment out Crawl api methods temporarily	2024-10-14 15:41:58 -07:00
Nick Sweeting	536a5ea745	clear up Machine models cache vars	2024-10-14 15:38:32 -07:00
Nick Sweeting	b3b2c551c2	fix get_sock_file symlink process	2024-10-14 15:37:48 -07:00
Nick Sweeting	a7c19a5da0	add register_admin to abx hookspec	2024-10-14 15:35:44 -07:00
Nick Sweeting	4634cd6c8e	make should_extract take config and uri as args	2024-10-14 15:35:20 -07:00
Nick Sweeting	518c46b4ab	fix circular import and show log of plugins loading on startup	2024-10-14 15:35:03 -07:00
Nick Sweeting	ad4657861f	add new crawls app	2024-10-11 19:47:05 -07:00
Nick Sweeting	1768f147d4	fix playwright and puppeteer install methods on linux	2024-10-11 02:21:15 -07:00
Nick Sweeting	38cb1ac87b	add apt install method for chrome	2024-10-11 02:20:48 -07:00
Nick Sweeting	f12efbdb45	use new dry run system	2024-10-11 02:20:37 -07:00
Nick Sweeting	bffa9ee78d	fix puppeteer install using npx stalling waiting for y user confirmation	2024-10-11 01:03:34 -07:00
Nick Sweeting	146cdff7d9	fix apt not installing npm package as separate package	2024-10-11 01:03:21 -07:00
Nick Sweeting	6e7071bd19	add new binproviders and binaries args to install and version, bump pydantic-pkgr version	2024-10-11 00:45:59 -07:00
Nick Sweeting	5e2dfb5baa	totally ignore sonic unless its the selected search backend	2024-10-09 19:12:47 -07:00
Nick Sweeting	8ce7313325	ignore puppeteer and playwright install stdout lines that arent chrome bin abspath	2024-10-09 19:12:35 -07:00
Nick Sweeting	0c29e08f73	avoid creating collection id file on every startup since its not needed	2024-10-09 19:12:08 -07:00
Nick Sweeting	b1a17689cd	dont load sonic binary if sonic backend is not enabled	2024-10-09 18:57:24 -07:00
Nick Sweeting	c25ced0c2a	fix Node binary to install with apt as nodejs	2024-10-09 18:40:50 -07:00
Nick Sweeting	228bb7bd52	disable LDAP_BINARY if LDAP_ENABLED=False	2024-10-09 18:39:45 -07:00
Nick Sweeting	fdc720b13d	remove checks import and bump pydantic pkgr version	2024-10-09 18:28:05 -07:00
Nick Sweeting	04d2316800	remove CHECKS feature, not needed	2024-10-09 18:06:00 -07:00
Nick Sweeting	de7ab65f11	ignore errors when chowning at initial startup	2024-10-09 04:48:09 -07:00
Nick Sweeting	ad675a8e7c	properly handle chowning DATA_DIR on init when using sudo	2024-10-09 04:39:09 -07:00
Nick Sweeting	2f68a1d476	fix ldap lib loading after apt install	2024-10-09 04:03:02 -07:00
Nick Sweeting	1b7aca130b	properly detect sudo UID	2024-10-09 04:02:46 -07:00
Nick Sweeting	31c66a1068	fix playwright and puppeteer browser detection after install	2024-10-09 03:46:28 -07:00
Nick Sweeting	f4f9ae2f28	fix LIB_NPM provider loading and euid	2024-10-09 03:19:31 -07:00
Nick Sweeting	861b9cd16f	fix LDAP_LIB loading from venv and other site packages dirs	2024-10-09 03:19:09 -07:00
Nick Sweeting	afc24e802a	tweak version log output	2024-10-09 03:18:22 -07:00
Nick Sweeting	db65af898b	correctly update environment HOME and USER vars when dropping permissions	2024-10-09 03:18:04 -07:00
Nick Sweeting	48409d8a5a	run env BinProvider as ArchiveBox User	2024-10-09 03:17:50 -07:00
Nick Sweeting	613caec8eb	improve install flow with sudo, check package managers, and fix docker build	2024-10-09 00:41:16 -07:00
Nick Sweeting	e4f543f54a	add new plugin order sigil file feature	2024-10-09 00:39:46 -07:00
Nick Sweeting	f6446a141f	also try installing ldap with apt if its not available	2024-10-08 19:27:02 -07:00
Nick Sweeting	7c34f2bc90	hide errors if user is just getting help or version info	2024-10-08 19:20:03 -07:00
Nick Sweeting	9f274cf9f4	remove platformdirs dependency	2024-10-08 19:17:18 -07:00
Nick Sweeting	0b4cbb6415	clear out stale supervisord state in between runs	2024-10-08 18:59:48 -07:00
Nick Sweeting	584abe8548	never attempt to create system venv, install ldap in lib automatically, and setup binproviders before bins	2024-10-08 18:52:02 -07:00
Nick Sweeting	3e4a846488	fix more installer bugs	2024-10-08 18:06:57 -07:00
Nick Sweeting	4b34b729ab	fuck it go back to nested lib and tmp dirs with supervisord sock workaround	2024-10-08 17:48:59 -07:00
Nick Sweeting	1888691ee8	try creating shared libs as 777 when running as root	2024-10-08 17:10:56 -07:00
Nick Sweeting	35c7019772	handle failure on tmp_dir and lib_dir detection better	2024-10-08 16:56:25 -07:00
Nick Sweeting	a33da44492	more attempts to fix euid permissions issues on ubuntu	2024-10-08 16:56:24 -07:00
Nick Sweeting	216e885b85	bump pydantic-pkgr	2024-10-08 03:53:41 -07:00
Nick Sweeting	8d32508581	only show data dir info if one is active	2024-10-08 03:04:38 -07:00
Nick Sweeting	de2ab43f7f	switch .is_dir and .exists for os.access to avoid PermissionError on startup	2024-10-08 03:02:34 -07:00
Nick Sweeting	611a2b7c1b	fix a few small nits	2024-10-08 02:10:08 -07:00
Nick Sweeting	3fb5b6eb94	exit archivebox version with failure status if any subdependencies are not installed	2024-10-08 01:52:04 -07:00
Nick Sweeting	5e351f6ba6	more docker dependency tweaks	2024-10-08 01:47:38 -07:00
Nick Sweeting	46c0463539	safer import handling	2024-10-08 00:51:58 -07:00
Nick Sweeting	d9fef4cd84	fix import order	2024-10-08 00:15:53 -07:00
Nick Sweeting	397ae1a99b	fix Docker build and import issues	2024-10-08 00:12:09 -07:00
Nick Sweeting	cf1ea8f80f	improve config loading of TMP_DIR, LIB_DIR, move to separate files	2024-10-07 23:45:11 -07:00
Nick Sweeting	7a895d9285	make sure monkey patches and vendor libs come before first constants import	2024-10-05 16:42:51 -07:00
Nick Sweeting	430e974719	add pydantic-pkgr back as vendored lib submodule for now	2024-10-05 16:39:20 -07:00
Nick Sweeting	ccdc3e1c47	add pydantic-pkgr back as vendored lib becauase pypi is misbehaving	2024-10-05 16:38:55 -07:00
Nick Sweeting	55e286972d	fix timeout check showing regardless of value	2024-10-05 04:24:07 -07:00
Nick Sweeting	db10a2142e	remove extra files from repo root and move package.json into etc	2024-10-05 03:53:23 -07:00
Nick Sweeting	66a785bb35	only use system tmp dirs because of socket path length restrictions	2024-10-05 03:16:27 -07:00
Nick Sweeting	35446ce742	include sonic-client by default and allow ldap to be installed at runtime	2024-10-05 03:11:48 -07:00
Nick Sweeting	ce2e19a429	switch to uv builds and rc1 versioning system	2024-10-04 23:48:25 -07:00
Nick Sweeting	0876cc78d9	remove no longer needed vendored libs	2024-10-04 23:35:34 -07:00
Nick Sweeting	80e052b166	fix pip binary loading	2024-10-04 23:26:49 -07:00
Nick Sweeting	beefe69b74	fix CHROME_TIMEOUT causing hanging on some platforms	2024-10-04 21:49:09 -07:00
Nick Sweeting	ac96cc62fc	fix CUSTOM_TEMPLATES_DIR loading	2024-10-04 21:40:36 -07:00
Nick Sweeting	0c7d7a2225	fix archivebox init colors and dir status checking	2024-10-04 21:34:19 -07:00
Nick Sweeting	5323953f94	handle Ctrl+C more gracefully	2024-10-04 21:33:46 -07:00
Nick Sweeting	026169a8e2	fix rich colors	2024-10-04 21:09:29 -07:00
Nick Sweeting	d747cf7f31	fix SYSTEM_TMP_DIR and SYSTEM_LIB_DIR in docker	2024-10-04 21:03:02 -07:00
Nick Sweeting	811f9a8d93	move queue db name into constants and fix file detection at startup	2024-10-04 19:38:36 -07:00
Nick Sweeting	3f986f09cc	fix relative dir calculation in extraactor hook	2024-10-04 19:24:01 -07:00
Nick Sweeting	73e69ccb8b	fixes for docs generation	2024-10-04 19:16:46 -07:00
Nick Sweeting	f76bdc4332	fix old wsgi.py	2024-10-04 16:08:57 -07:00
Nick Sweeting	da274fd8e8	remove dead code	2024-10-04 14:48:20 -07:00
Nick Sweeting	396a7ffcd8	move tmp dir to machine-id scoped dir	2024-10-04 03:24:15 -07:00
Nick Sweeting	12f32c4690	fix tmp data dir resolution when running help or version outside data dir	2024-10-04 01:40:41 -07:00
Nick Sweeting	f321d25f4c	fallback to reading binaries from filesystem when theres no db	2024-10-04 01:00:09 -07:00
Nick Sweeting	152b530249	scope LIB_DIR by os, arch, and docker status	2024-10-04 00:08:44 -07:00
Nick Sweeting	c84ea81c5a	more ldap lib optimization	2024-10-03 18:25:35 -07:00
Nick Sweeting	89a066da0b	remove django-url-tools in favor of core_tags snippet	2024-10-03 18:25:20 -07:00
Nick Sweeting	3f1a19dd35	fix type imports	2024-10-03 18:24:57 -07:00
Nick Sweeting	c5da3c1f22	fix docker build	2024-10-03 18:24:27 -07:00
Nick Sweeting	0619750ffa	add django-url-tools to fix pagination and search on public index	2024-10-03 17:39:55 -07:00
Nick Sweeting	1492c02bfa	lazy-load loadfire and ldap lib for faster startup time	2024-10-03 17:39:39 -07:00
Nick Sweeting	563e4de678	unwinding circular dependencies	2024-10-03 17:05:49 -07:00
Nick Sweeting	9241a45bb8	Update base_configset.py	2024-10-03 11:42:27 -04:00
Nick Sweeting	de09867f87	ignore lib/bin symlinking errors	2024-10-03 04:10:52 -07:00
Nick Sweeting	aae9deccc2	add machine migrations	2024-10-03 04:06:28 -07:00
Nick Sweeting	b072fd8ef4	load all binaries from cache by default	2024-10-03 04:06:17 -07:00
Nick Sweeting	0f37abb657	fix symlinking to lib when conflicting file already exists	2024-10-03 03:56:45 -07:00
Nick Sweeting	490e5ba11d	fallback to localhost if detecting dnsserver fails	2024-10-03 03:53:50 -07:00
Nick Sweeting	b36e89d086	relocate LIB_DIR and TMP_DIR inside docker so it doesnt clash with outside docker	2024-10-03 03:43:02 -07:00
Nick Sweeting	f4f1d7893c	fix CUSTOM_TEMPLATES_DIR config load and chrome symlinking	2024-10-03 03:20:25 -07:00
Nick Sweeting	29fc14dff4	dont build docker container twice during release	2024-10-03 03:12:18 -07:00
Nick Sweeting	9728d81fee	add puppeteer to docker requirements for easier browser fetching	2024-10-03 03:12:06 -07:00
Nick Sweeting	697d0a3566	nicer version and help pretty printing with rich	2024-10-03 03:11:23 -07:00