Nick Sweeting
|
ad675a8e7c
|
properly handle chowning DATA_DIR on init when using sudo
|
2024-10-09 04:39:09 -07:00 |
|
Nick Sweeting
|
2f68a1d476
|
fix ldap lib loading after apt install
|
2024-10-09 04:03:02 -07:00 |
|
Nick Sweeting
|
1b7aca130b
|
properly detect sudo UID
|
2024-10-09 04:02:46 -07:00 |
|
Nick Sweeting
|
31c66a1068
|
fix playwright and puppeteer browser detection after install
|
2024-10-09 03:46:28 -07:00 |
|
Nick Sweeting
|
f4f9ae2f28
|
fix LIB_NPM provider loading and euid
|
2024-10-09 03:19:31 -07:00 |
|
Nick Sweeting
|
861b9cd16f
|
fix LDAP_LIB loading from venv and other site packages dirs
|
2024-10-09 03:19:09 -07:00 |
|
Nick Sweeting
|
afc24e802a
|
tweak version log output
|
2024-10-09 03:18:22 -07:00 |
|
Nick Sweeting
|
db65af898b
|
correctly update environment HOME and USER vars when dropping permissions
|
2024-10-09 03:18:04 -07:00 |
|
Nick Sweeting
|
48409d8a5a
|
run env BinProvider as ArchiveBox User
|
2024-10-09 03:17:50 -07:00 |
|
Nick Sweeting
|
613caec8eb
|
improve install flow with sudo, check package managers, and fix docker build
|
2024-10-09 00:41:16 -07:00 |
|
Nick Sweeting
|
e4f543f54a
|
add new plugin order sigil file feature
|
2024-10-09 00:39:46 -07:00 |
|
Nick Sweeting
|
f6446a141f
|
also try installing ldap with apt if its not available
|
2024-10-08 19:27:02 -07:00 |
|
Nick Sweeting
|
7c34f2bc90
|
hide errors if user is just getting help or version info
|
2024-10-08 19:20:03 -07:00 |
|
Nick Sweeting
|
9f274cf9f4
|
remove platformdirs dependency
|
2024-10-08 19:17:18 -07:00 |
|
Nick Sweeting
|
0b4cbb6415
|
clear out stale supervisord state in between runs
|
2024-10-08 18:59:48 -07:00 |
|
Nick Sweeting
|
584abe8548
|
never attempt to create system venv, install ldap in lib automatically, and setup binproviders before bins
|
2024-10-08 18:52:02 -07:00 |
|
Nick Sweeting
|
3e4a846488
|
fix more installer bugs
|
2024-10-08 18:06:57 -07:00 |
|
Nick Sweeting
|
4b34b729ab
|
fuck it go back to nested lib and tmp dirs with supervisord sock workaround
|
2024-10-08 17:48:59 -07:00 |
|
Nick Sweeting
|
1888691ee8
|
try creating shared libs as 777 when running as root
|
2024-10-08 17:10:56 -07:00 |
|
Nick Sweeting
|
35c7019772
|
handle failure on tmp_dir and lib_dir detection better
|
2024-10-08 16:56:25 -07:00 |
|
Nick Sweeting
|
a33da44492
|
more attempts to fix euid permissions issues on ubuntu
|
2024-10-08 16:56:24 -07:00 |
|
Nick Sweeting
|
216e885b85
|
bump pydantic-pkgr
|
2024-10-08 03:53:41 -07:00 |
|
Nick Sweeting
|
8d32508581
|
only show data dir info if one is active
|
2024-10-08 03:04:38 -07:00 |
|
Nick Sweeting
|
de2ab43f7f
|
switch .is_dir and .exists for os.access to avoid PermissionError on startup
|
2024-10-08 03:02:34 -07:00 |
|
Nick Sweeting
|
611a2b7c1b
|
fix a few small nits
|
2024-10-08 02:10:08 -07:00 |
|
Nick Sweeting
|
3fb5b6eb94
|
exit archivebox version with failure status if any subdependencies are not installed
|
2024-10-08 01:52:04 -07:00 |
|
Nick Sweeting
|
5e351f6ba6
|
more docker dependency tweaks
|
2024-10-08 01:47:38 -07:00 |
|
Nick Sweeting
|
46c0463539
|
safer import handling
|
2024-10-08 00:51:58 -07:00 |
|
Nick Sweeting
|
d9fef4cd84
|
fix import order
|
2024-10-08 00:15:53 -07:00 |
|
Nick Sweeting
|
397ae1a99b
|
fix Docker build and import issues
|
2024-10-08 00:12:09 -07:00 |
|
Nick Sweeting
|
cf1ea8f80f
|
improve config loading of TMP_DIR, LIB_DIR, move to separate files
|
2024-10-07 23:45:11 -07:00 |
|
Nick Sweeting
|
7a895d9285
|
make sure monkey patches and vendor libs come before first constants import
|
2024-10-05 16:42:51 -07:00 |
|
Nick Sweeting
|
430e974719
|
add pydantic-pkgr back as vendored lib submodule for now
|
2024-10-05 16:39:20 -07:00 |
|
Nick Sweeting
|
ccdc3e1c47
|
add pydantic-pkgr back as vendored lib becauase pypi is misbehaving
|
2024-10-05 16:38:55 -07:00 |
|
Nick Sweeting
|
55e286972d
|
fix timeout check showing regardless of value
|
2024-10-05 04:24:07 -07:00 |
|
Nick Sweeting
|
db10a2142e
|
remove extra files from repo root and move package.json into etc
|
2024-10-05 03:53:23 -07:00 |
|
Nick Sweeting
|
66a785bb35
|
only use system tmp dirs because of socket path length restrictions
|
2024-10-05 03:16:27 -07:00 |
|
Nick Sweeting
|
35446ce742
|
include sonic-client by default and allow ldap to be installed at runtime
|
2024-10-05 03:11:48 -07:00 |
|
Nick Sweeting
|
ce2e19a429
|
switch to uv builds and rc1 versioning system
|
2024-10-04 23:48:25 -07:00 |
|
Nick Sweeting
|
0876cc78d9
|
remove no longer needed vendored libs
|
2024-10-04 23:35:34 -07:00 |
|
Nick Sweeting
|
80e052b166
|
fix pip binary loading
|
2024-10-04 23:26:49 -07:00 |
|
Nick Sweeting
|
beefe69b74
|
fix CHROME_TIMEOUT causing hanging on some platforms
|
2024-10-04 21:49:09 -07:00 |
|
Nick Sweeting
|
ac96cc62fc
|
fix CUSTOM_TEMPLATES_DIR loading
|
2024-10-04 21:40:36 -07:00 |
|
Nick Sweeting
|
0c7d7a2225
|
fix archivebox init colors and dir status checking
|
2024-10-04 21:34:19 -07:00 |
|
Nick Sweeting
|
5323953f94
|
handle Ctrl+C more gracefully
|
2024-10-04 21:33:46 -07:00 |
|
Nick Sweeting
|
026169a8e2
|
fix rich colors
|
2024-10-04 21:09:29 -07:00 |
|
Nick Sweeting
|
d747cf7f31
|
fix SYSTEM_TMP_DIR and SYSTEM_LIB_DIR in docker
|
2024-10-04 21:03:02 -07:00 |
|
Nick Sweeting
|
811f9a8d93
|
move queue db name into constants and fix file detection at startup
|
2024-10-04 19:38:36 -07:00 |
|
Nick Sweeting
|
3f986f09cc
|
fix relative dir calculation in extraactor hook
|
2024-10-04 19:24:01 -07:00 |
|
Nick Sweeting
|
73e69ccb8b
|
fixes for docs generation
|
2024-10-04 19:16:46 -07:00 |
|
Nick Sweeting
|
f76bdc4332
|
fix old wsgi.py
|
2024-10-04 16:08:57 -07:00 |
|
Nick Sweeting
|
da274fd8e8
|
remove dead code
|
2024-10-04 14:48:20 -07:00 |
|
Nick Sweeting
|
396a7ffcd8
|
move tmp dir to machine-id scoped dir
|
2024-10-04 03:24:15 -07:00 |
|
Nick Sweeting
|
12f32c4690
|
fix tmp data dir resolution when running help or version outside data dir
|
2024-10-04 01:40:41 -07:00 |
|
Nick Sweeting
|
f321d25f4c
|
fallback to reading binaries from filesystem when theres no db
|
2024-10-04 01:00:09 -07:00 |
|
Nick Sweeting
|
152b530249
|
scope LIB_DIR by os, arch, and docker status
|
2024-10-04 00:08:44 -07:00 |
|
Nick Sweeting
|
c84ea81c5a
|
more ldap lib optimization
|
2024-10-03 18:25:35 -07:00 |
|
Nick Sweeting
|
89a066da0b
|
remove django-url-tools in favor of core_tags snippet
|
2024-10-03 18:25:20 -07:00 |
|
Nick Sweeting
|
3f1a19dd35
|
fix type imports
|
2024-10-03 18:24:57 -07:00 |
|
Nick Sweeting
|
c5da3c1f22
|
fix docker build
|
2024-10-03 18:24:27 -07:00 |
|
Nick Sweeting
|
0619750ffa
|
add django-url-tools to fix pagination and search on public index
|
2024-10-03 17:39:55 -07:00 |
|
Nick Sweeting
|
1492c02bfa
|
lazy-load loadfire and ldap lib for faster startup time
|
2024-10-03 17:39:39 -07:00 |
|
Nick Sweeting
|
563e4de678
|
unwinding circular dependencies
|
2024-10-03 17:05:49 -07:00 |
|
Nick Sweeting
|
9241a45bb8
|
Update base_configset.py
|
2024-10-03 11:42:27 -04:00 |
|
Nick Sweeting
|
de09867f87
|
ignore lib/bin symlinking errors
|
2024-10-03 04:10:52 -07:00 |
|
Nick Sweeting
|
aae9deccc2
|
add machine migrations
|
2024-10-03 04:06:28 -07:00 |
|
Nick Sweeting
|
b072fd8ef4
|
load all binaries from cache by default
|
2024-10-03 04:06:17 -07:00 |
|
Nick Sweeting
|
0f37abb657
|
fix symlinking to lib when conflicting file already exists
|
2024-10-03 03:56:45 -07:00 |
|
Nick Sweeting
|
490e5ba11d
|
fallback to localhost if detecting dnsserver fails
|
2024-10-03 03:53:50 -07:00 |
|
Nick Sweeting
|
b36e89d086
|
relocate LIB_DIR and TMP_DIR inside docker so it doesnt clash with outside docker
|
2024-10-03 03:43:02 -07:00 |
|
Nick Sweeting
|
f4f1d7893c
|
fix CUSTOM_TEMPLATES_DIR config load and chrome symlinking
|
2024-10-03 03:20:25 -07:00 |
|
Nick Sweeting
|
29fc14dff4
|
dont build docker container twice during release
|
2024-10-03 03:12:18 -07:00 |
|
Nick Sweeting
|
9728d81fee
|
add puppeteer to docker requirements for easier browser fetching
|
2024-10-03 03:12:06 -07:00 |
|
Nick Sweeting
|
697d0a3566
|
nicer version and help pretty printing with rich
|
2024-10-03 03:11:23 -07:00 |
|
Nick Sweeting
|
161afc7297
|
add health stats counters to machine models
|
2024-10-03 03:11:04 -07:00 |
|
Nick Sweeting
|
3b9e48ead8
|
show deprecation warning for archivebox setup command
|
2024-10-03 03:10:36 -07:00 |
|
Nick Sweeting
|
e315905721
|
add new InstalledBinary model to cache binaries on host machine
|
2024-10-03 03:10:22 -07:00 |
|
Nick Sweeting
|
32167de936
|
add daemonize flag to archivebox server
|
2024-10-02 19:46:48 -07:00 |
|
Nick Sweeting
|
035a14b6ea
|
better help text output
|
2024-10-02 19:46:31 -07:00 |
|
Nick Sweeting
|
968adf64da
|
small easter eggs
|
2024-10-02 14:17:28 -07:00 |
|
Nick Sweeting
|
95043e5a07
|
bump js subdeps versions
|
2024-10-01 21:48:07 -07:00 |
|
Nick Sweeting
|
295c5c46e0
|
add new crawl model
|
2024-10-01 21:47:16 -07:00 |
|
Nick Sweeting
|
f46d62a114
|
add py-machineid lib for new machine app
|
2024-10-01 21:46:35 -07:00 |
|
Nick Sweeting
|
4a19051f4a
|
change BaseExtractor to use new extract hookspec
|
2024-10-01 21:45:18 -07:00 |
|
Nick Sweeting
|
276a505cae
|
fix extractor path calculation
|
2024-10-01 21:44:56 -07:00 |
|
Nick Sweeting
|
8498ca5c64
|
add abx.archivebox extract hookspec
|
2024-10-01 21:44:19 -07:00 |
|
Nick Sweeting
|
81d16e96fd
|
fix toml_util circular import in abx
|
2024-10-01 21:43:35 -07:00 |
|
Nick Sweeting
|
716df44b7a
|
fix created_by_id access error on abid creation
|
2024-10-01 21:43:11 -07:00 |
|
Nick Sweeting
|
5697ecefad
|
fix index SEARCH_BACKENDS import_backend to load via ABX instead of settings
|
2024-10-01 00:19:19 -07:00 |
|
Nick Sweeting
|
dac134dfca
|
improve default chrome cli launch args used for archiving
|
2024-10-01 00:18:57 -07:00 |
|
Nick Sweeting
|
94123ca68c
|
fix archive_dot_org repsonse parsing bytes vs str bug
|
2024-10-01 00:18:38 -07:00 |
|
Nick Sweeting
|
71d215367b
|
add types to abx use api
|
2024-10-01 00:18:21 -07:00 |
|
Nick Sweeting
|
8fbfa10df3
|
fix missing check_migrations import
|
2024-09-30 23:52:48 -07:00 |
|
Nick Sweeting
|
18474f452b
|
move config moved out of legacy files and better version output
|
2024-09-30 23:52:00 -07:00 |
|
Nick Sweeting
|
d21bc86075
|
finish migrating almost all config to new system
|
2024-09-30 23:21:34 -07:00 |
|
Nick Sweeting
|
4b6a2a3e50
|
add git plugin
|
2024-09-30 23:20:03 -07:00 |
|
Nick Sweeting
|
8c3342afe5
|
rename archivebox setup to archivebox install
|
2024-09-30 23:19:46 -07:00 |
|
Nick Sweeting
|
4334c74548
|
change archivebox setup to install ALL binaries by default
|
2024-09-30 21:44:23 -07:00 |
|
Nick Sweeting
|
f6176ae05e
|
move curl into plugin
|
2024-09-30 21:43:54 -07:00 |
|
Nick Sweeting
|
69522da4bb
|
move wget and mercury into plugins
|
2024-09-30 21:43:45 -07:00 |
|
Nick Sweeting
|
c4e040f11a
|
add WgetPlugin with WgetExtractor, WarcExtractor, WgetBinary
|
2024-09-30 19:33:30 -07:00 |
|
Nick Sweeting
|
2a1645ba27
|
fix import errors
|
2024-09-30 19:32:57 -07:00 |
|
Nick Sweeting
|
51fe4c38c2
|
speed up version command by checking if quiet is passed
|
2024-09-30 18:33:43 -07:00 |
|
Nick Sweeting
|
31ce490321
|
fix help command output docstrings and more CLI log coloring
|
2024-09-30 18:29:17 -07:00 |
|
Nick Sweeting
|
7489663ff3
|
use pretty printing for config CLI output
|
2024-09-30 18:14:43 -07:00 |
|
Nick Sweeting
|
c909c00123
|
improve archivebox version cli output
|
2024-09-30 18:13:05 -07:00 |
|
Nick Sweeting
|
66cd711df9
|
improve version detection
|
2024-09-30 18:12:48 -07:00 |
|
Nick Sweeting
|
b913e6f426
|
rename OUTPUT_DIR to DATA_DIR
|
2024-09-30 17:44:18 -07:00 |
|
Nick Sweeting
|
363a499289
|
move util.py into misc folder
|
2024-09-30 17:25:15 -07:00 |
|
Nick Sweeting
|
dfca4b13b2
|
move system.py into misc folder
|
2024-09-30 17:13:55 -07:00 |
|
Nick Sweeting
|
7a41b6ae46
|
remove ConfigSectionName and add type hints to CONSTANTS
|
2024-09-30 16:50:36 -07:00 |
|
Nick Sweeting
|
3e5b6ddeae
|
move config into dedicated global app
|
2024-09-30 15:59:05 -07:00 |
|
Nick Sweeting
|
ee7f73bd7b
|
bump version to 0.8.5
|
2024-09-27 01:26:12 -07:00 |
|
Nick Sweeting
|
8d3f45b720
|
merge plugantic and abx, all praise be to praise our glorious pluggy gods
|
2024-09-27 01:26:12 -07:00 |
|
Nick Sweeting
|
4f42eb0313
|
move ini_to_toml into misc
|
2024-09-27 01:26:11 -07:00 |
|
Nick Sweeting
|
6f7b6c6bde
|
move unused ansible folder
|
2024-09-27 01:26:11 -07:00 |
|
Nick Sweeting
|
7b6a491ae0
|
exclude dunder vars from constants
|
2024-09-27 01:26:10 -07:00 |
|
Nick Sweeting
|
0589ff2b5d
|
move loading of vendor libs import archivebox init
|
2024-09-27 01:26:10 -07:00 |
|
Nick Sweeting
|
8ed3155ec5
|
migrate plugin loading process to new pluggy-powered system based on djp
|
2024-09-26 02:43:12 -07:00 |
|
Nick Sweeting
|
efd341d8ad
|
add DIR_OUTPUT_PERMISSIONS to STORAGE_CONFIG and fix ripgrep constants import
|
2024-09-26 02:42:50 -07:00 |
|
Nick Sweeting
|
7b85ba7fd8
|
fix log line view in admin data when bytes are not utf8
|
2024-09-26 02:41:45 -07:00 |
|
Nick Sweeting
|
0cfcabf6f4
|
fix admin data view configs type rendering
|
2024-09-26 02:41:22 -07:00 |
|
Nick Sweeting
|
ed45f58758
|
use constants in more places
|
2024-09-26 02:41:09 -07:00 |
|
Nick Sweeting
|
eb360f188a
|
remove old insecure index.json url serving from root
|
2024-09-26 02:38:59 -07:00 |
|
Nick Sweeting
|
d8a9dca0f6
|
use constants in more places
|
2024-09-26 02:38:45 -07:00 |
|
Nick Sweeting
|
24a9f432c9
|
fix archivebox manage command not passing args correctly
|
2024-09-26 02:37:44 -07:00 |
|
Nick Sweeting
|
6ec5925b7f
|
fix readability plugin name
|
2024-09-26 02:37:26 -07:00 |
|
Nick Sweeting
|
45736036e0
|
simplify archivebox.constants to just use benedict instead of kludgy NamedTuple
|
2024-09-26 02:36:59 -07:00 |
|
Nick Sweeting
|
80d3def206
|
improve archivebox.__init__ and load vendor libs at very beginning
|
2024-09-26 02:36:34 -07:00 |
|
Nick Sweeting
|
446b38dc41
|
add favicon and archivedotorg plugins
|
2024-09-26 02:32:10 -07:00 |
|
Nick Sweeting
|
c950271bc3
|
fix more constants / config loading
|
2024-09-25 05:12:34 -07:00 |
|
Nick Sweeting
|
bb65b2dbec
|
move almost all config into new archivebox.CONSTANTS
|
2024-09-25 05:10:09 -07:00 |
|
Nick Sweeting
|
f5e8d99fdf
|
update archivebox setup to use new binprovider install methods
|
2024-09-25 01:15:15 -07:00 |
|
Nick Sweeting
|
bc08bb04a2
|
archivebox version show when binary is not loaded correctly
|
2024-09-25 01:15:00 -07:00 |
|
Nick Sweeting
|
0ef3a0913b
|
check python encoding in SHELL_CONFIG validation
|
2024-09-25 01:14:48 -07:00 |
|
Nick Sweeting
|
e0eb3119b7
|
bump pydantic-pkgr to 0.3.7
|
2024-09-25 01:13:34 -07:00 |
|
Nick Sweeting
|
a5ffd4e9d3
|
move pdf, screenshot, dom, singlefile, and ytdlp extractor config to new plugin system
|
2024-09-25 00:42:26 -07:00 |
|
Nick Sweeting
|
a2a586e369
|
fix system.run not using text arg
|
2024-09-25 00:41:55 -07:00 |
|
Nick Sweeting
|
5b6cf68d98
|
move system startup checks to pip and plugins_sys config model validation
|
2024-09-25 00:41:24 -07:00 |
|
Nick Sweeting
|
2fd837f254
|
setup rich tracebacks width properly in monkey patched exception handler
|
2024-09-25 00:40:37 -07:00 |
|
Nick Sweeting
|
ee5bec6a10
|
flip link_archive exception throw order so real exception is easier to read at the bottom
|
2024-09-25 00:39:49 -07:00 |
|
Nick Sweeting
|
6742888278
|
setup rich tracebacks width properly
|
2024-09-25 00:39:27 -07:00 |
|
Nick Sweeting
|
5e4b78d9e0
|
change supervisord to always start non-daemonized by default
|
2024-09-24 22:22:03 -07:00 |
|
Nick Sweeting
|
de2ba890ea
|
add ArchiveBox binary
|
2024-09-24 22:01:28 -07:00 |
|
Nick Sweeting
|
3dacec3f5b
|
prevent redundant supervisord starts
|
2024-09-24 22:01:18 -07:00 |
|
Nick Sweeting
|
b117484de7
|
add new Snapshot.archive method powered by huey task
|
2024-09-24 21:17:51 -07:00 |
|
Nick Sweeting
|
e99260feb2
|
fix rich logging issues
|
2024-09-24 21:17:07 -07:00 |
|
Nick Sweeting
|
0dffbf1bb4
|
fix rich autodetection of TTY, USE_COLOR, SHOW_PROGRESS
|
2024-09-24 19:37:29 -07:00 |
|
Nick Sweeting
|
bde0bf8263
|
load ipython rich extension in archivebox shell
|
2024-09-24 19:37:05 -07:00 |
|
Nick Sweeting
|
07eff77c0a
|
bump pydantic-pkgr submodule
|
2024-09-24 19:05:09 -07:00 |
|