Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-29 23:50:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
3932 commits 6 branches 78 tags 19 MiB
Commit graph

242 commits

Author SHA1 Message Date
Ben Muthalaly
a3fd8a8ecd Move version banner logic from JS to Python 
Also adds CSS styling to banner.
 2023-11-27 22:56:30 -06:00
Nick Sweeting
6761ed368c wip pre-0.7.1 side fixes 2023-11-14 02:09:58 -08:00
Nick Sweeting
f0033f75d0 config.py lint fixes 2023-11-14 02:07:35 -08:00
Nick Sweeting
0b950b3253 make get_commit_hash more precisce by using exact ref 2023-11-14 02:04:57 -08:00
Nick Sweeting
6188dcdcbd return placeholder version during docs build 2023-11-14 00:38:37 -08:00
Nick Sweeting
d3d05f5f64 add fallback version detection mode 2023-11-14 00:21:09 -08:00
Nick Sweeting
f5739506f6
Merge pull request #1244 from overhacked/search_index_extract_html_text 2023-11-08 23:34:44 -08:00
Nick Sweeting
7377d9ebc3 bump version to 0.7.1 and fix version autodetection code 2023-11-03 20:07:39 -07:00
Nick Sweeting
db733151f7 bump user agent browser version 2023-10-31 19:16:15 -07:00
Nick Sweeting
166573d417 autodetect qemu and warn about broken chrome in emulated containers 2023-10-31 00:23:19 -07:00
Nick Sweeting
62e077a5bc
Merge pull request #1241 from overhacked/fts5-search 2023-10-30 18:49:52 -07:00
Nick Sweeting
a680724367
Merge branch 'dev' into search_index_extract_html_text 2023-10-27 23:09:28 -07:00
Ross Williams
310b4d1242 Add htmltotext extractor 
Saves HTML text nodes and selected element attributes in
`htmltotext.txt` for each Snapshot. Primarily intended to be used
for search indexing.
 2023-10-23 21:42:32 -04:00
Nick Sweeting
63ad43f46c
Merge branch 'dev' into method_allow_deny 2023-10-20 04:25:44 -07:00
Nick Sweeting
53355be46a
Merge branch 'dev' into fts5-search 2023-10-20 04:23:50 -07:00
Nick Sweeting
d4ca2d1154 update mercury bin path to postlight parser 2023-10-20 02:46:41 -07:00
Ben Muthalaly
9e6a87114b Fix formatting, logging, logic issues 2023-10-18 12:07:36 -05:00
Ben Muthalaly
521ea70e0c Add check for existing user, change varable names 2023-10-18 03:07:54 -05:00
Ross Williams
8fe5faf4d0 Introduce SQLite FTS5-powered search backend 
Use SQLite's FTS5 extension to power full-text search without any
additional dependencies. FTS5 was introduced in SQLite 3.9.0,
[released][1] in 2015 so should be available on most SQLite
installations at this point in time.

[1]: https://www.sqlite.org/changes.html#version_3_9_0
 2023-10-16 13:23:26 -04:00
Ben Muthalaly
11d473e536 Add config options to add admin user on first run 2023-10-14 00:38:04 -05:00
Ross Williams
d8aa84ac98 Make extracting text for indexing optional 
Add a configuration option to enable/disable HTML text extraction
for indexing
 2023-10-12 13:14:39 -04:00
Nick Sweeting
5c1a14e4f2 ignore errors while getting system user name 2023-09-14 03:39:44 -07:00
Nick Sweeting
cd9f228b2f
Merge pull request #1214 from DanielBatteryStapler/DanielBatteryStapler-patch-1 2023-09-03 21:25:12 -07:00
Nick Sweeting
16d278fbdb
Merge pull request #1168 from mAAdhaTTah/add-readwise-reader 2023-09-03 21:24:49 -07:00
Nick Sweeting
110a22ee32
Merge branch 'dev' into DanielBatteryStapler-patch-1 2023-08-31 15:20:46 -07:00
Nick Sweeting
73a5f74d38
update default YOUTUBEDL_ARGS to fix subs and filesize 2023-08-31 15:17:45 -07:00
root
23f086aa40 add LDAP support 2023-08-17 19:51:02 -05:00
Ross Williams
b44f7e68b1 Add URL-specific method allow/deny lists 
Allows enabling only allow-listed extractors or disabling specific
deny-listed extractors for a regular expression matched against an added
site's URL.
 2023-08-02 09:36:40 -04:00
Ross Williams
46e80dd509 Rename URL_(WHITE|BLACK)LIST to URL_(ALLOW|DENY)LIST 
Retain aliases for old configuration files
 2023-08-02 09:31:48 -04:00
Ross Williams
9d9872d325 bin_version means to modify, not replace environ 
the `bin_version` function means to modify the environment,
not replace it entirely. Fixes bugs that occur when it wipes out the
PATH environment variable, such as when running in a virtual
environment.
 2023-07-31 11:36:34 -04:00
mAAdhaTTah
181501fd36
Add Readwise Reader API parser 
Implemented similar to the Pocket API.
 2023-07-02 11:20:58 -04:00
Micah R Ledbetter
1e50ca243e Add FAVICON_PROVIDER option for custom favicon service 2023-05-05 20:42:36 -05:00
Nick Sweeting
9f42a3bf29
fix whitespace 2023-03-15 16:01:02 -07:00
ふぁ
d77c770c47
add CHROME_TIMEOUT args 
Signed-off-by: ふぁ <yuki@yuki0311.com>
 2023-03-14 20:29:41 +09:00
Nick Sweeting
dca69933eb
Update archivebox/config.py 
Co-authored-by: dugite-code <dugite-code@users.noreply.github.com>
 2023-01-09 18:22:01 -08:00
Nick Sweeting
2538b170c7
Merge branch 'dev' into feat/reverse-proxy-auth 2023-01-09 18:20:45 -08:00
Nick Sweeting
0cbeeb4346
Merge pull request #1021 from renaisun/dev 2023-01-09 18:17:39 -08:00
Joseph Turian
30947aeb07 yt-dlp flag cleanup 2022-09-14 06:29:57 +02:00
Joseph Turian
f729bbe122 yt-dlp fixes 2022-09-14 06:27:58 +02:00
Joseph Turian
dba423a568 A few more youtube-dl tweaks 2022-09-12 20:36:23 +00:00
Joseph Turian
f5f7aff3b4 Added yt-dlp everywhere 2022-09-12 20:34:02 +00:00
Nick Sweeting
03eb7e5875
Update config.py 2022-06-09 01:04:55 -07:00
renaisun
8899fe0b92
Add SINGLEFILE_ARGS to control single-file arguments 2022-06-09 14:35:48 +08:00
Nick Sweeting
319ea481b8
Update config.py 2022-06-08 20:17:38 -07:00
Nick Sweeting
2bbc742017
typo fix 2022-06-08 20:16:08 -07:00
Nick Sweeting
e2fa68dba6
resolve config paths before using 2022-06-08 20:15:22 -07:00
Nick Sweeting
9f90a2d60d
disable unused sqlite3 stuff 2022-06-08 20:12:55 -07:00
Nick Sweeting
c78a2edc42
add is_mount and COMMIT_HASH to config.py 2022-06-08 20:04:01 -07:00
Nick Sweeting
0c6d4c82c3
Update config.py 2022-06-08 19:11:02 -07:00
Nick Sweeting
f9c5808940
Update config.py 2022-06-08 19:09:11 -07:00
Nick Sweeting
31d5fbbf17
Update config.py 2022-06-08 19:04:06 -07:00
Nick Sweeting
6b019da3e9
Update config.py 2022-06-08 19:01:55 -07:00
Nick Sweeting
f9c82841ad
fix sqlite option detection 2022-06-08 18:58:15 -07:00
Nick Sweeting
3e3c011f86
enforce UTC timezone on server 2022-06-08 18:40:48 -07:00
Nick Sweeting
e06717419c
fix sqlite3 version detection 2022-06-08 18:35:31 -07:00
Nick Sweeting
d0f129295f
move sqlite3 checks up a level 2022-06-08 18:29:53 -07:00
Nick Sweeting
89175ccb22
check SQLite3 version and enabled extensions on startup 2022-06-08 18:24:17 -07:00
Nick Sweeting
c245d36e44
add PUID and PGID as config options in archivebox 2022-06-08 17:42:52 -07:00
Nick Sweeting
5e4ddbbf25 fix mercury bin parsing back 2022-05-09 21:58:17 -07:00
Nick Sweeting
e96c1bcf13 bump mercury parser to git head version 2022-05-09 21:48:41 -07:00
Nick Sweeting
d05510f844 fix version string parsing on macOS in some cases where LANG C is not supported 2022-05-09 21:21:08 -07:00
Nick Sweeting
8ebf3e2f93 add config option PREVIEW_ORIGINALS to hide original iframes in snapshot detail pages 2022-05-09 19:31:41 -07:00
Nick Sweeting
6e66863871 add max 5s writing delay for concurrent writers and flush WAL slower 2022-05-09 18:36:40 -07:00
Nick Sweeting
57df65f28f use yt-dlp for media archiving instead of youtube-dl 2022-04-21 07:11:35 -07:00
Nick Sweeting
eb81d41f84 bump Dockerfile base image version and install yt-dlp 2022-04-21 07:11:35 -07:00
Igor Rzegocki
d4f534e612
add LOGOUT_REDIRECT_URL 2022-03-31 21:40:14 +02:00
Pellaeon Lin
5e9d05483e Fix bin_version: set LANG=C when calling executables to avoid parsing 
localized output.
 2022-02-24 17:01:00 +08:00
Nick Sweeting
666ab20df5
Update archivebox/config.py 2022-01-10 20:42:09 -05:00
hannah98
fc3d2bb4dc rename TAG_SEPARATORS to TAG_SEPARATOR_PATTERN 2022-01-06 14:14:41 +00:00
hannah98
049f88def9 Added TAG_SEPARATORS option to supply a regex of characters to use when splitting tags 2021-12-30 20:19:48 +00:00
Mika Tuupola
f14a861605 Change logfile open to write mode only 2021-12-19 23:17:33 +02:00
Igor Rzegocki
95cf85f8cf
Support for Reverse Proxy authentication backends (like authelia) 2021-09-30 17:40:13 +02:00
Nick Sweeting
5a2c78e14b add proper support for URL_WHITELIST instead of using negation regexes 2021-07-06 23:42:00 -04:00
Nick Sweeting
c2d1a57581 fix umask dir permissions 2021-06-01 00:50:18 -04:00
Nick Sweeting
8230f88d80 change default OUTPUT_PERMISSIONS to disallow execution except on dirs 2021-05-31 19:31:51 -04:00
Nick Sweeting
1112526543 add option ENFORCE_ATOMIC_WRITES to allow disabling forced FSYNC writes on network drives 2021-05-31 19:31:51 -04:00
Nick Sweeting
79e19ecd47 shield pwd import for windows 2021-04-24 03:51:38 -04:00
Nick Sweeting
208f866fc4 fix user detection on freebsd always returning root 2021-04-24 02:52:09 -04:00
Nick Sweeting
9563e74f7c enforce minimum django version 2021-04-23 15:02:26 -04:00
Nick Sweeting
54cab113f3 allow static_index.json in dir root from old static exports 2021-04-12 17:06:32 -04:00
Nick Sweeting
6a6ef63775 enforce utf8 on more file reads 2021-04-10 07:22:32 -04:00
Nick Sweeting
193df5c8d3 add video subtitles and description to full-text index 2021-04-10 07:22:20 -04:00
Nick Sweeting
1e3c1e3c83 also write autogenerated subtitles 2021-04-10 06:56:19 -04:00
Nick Sweeting
c5a9b77f4f update user agents to mimic latest chrome 2021-04-10 05:45:02 -04:00
Nick Sweeting
a9986f1f05 add timezone support, tons of CSS and layout improvements, more detailed snapshot admin form info, ability to sort by recently updated, better grid view styling, better table layouts, better dark mode support 2021-04-10 04:21:36 -04:00
Nick Sweeting
06c58ac42a fix unreachable config backup clear 2021-04-09 12:15:47 -04:00
Nick Sweeting
32764347ce add new SEARCH_BACKEND_TIMEOUT config option defaulted to 90sec 2021-04-09 00:27:12 -04:00
Nick Sweeting
2e57df917e handle BaseExceptions properly 2021-04-08 06:08:17 -04:00
Nick Sweeting
2c07d8f1f3 fix lgtm alerts 2021-04-06 13:47:26 -04:00
Nick Sweeting
e92db03f5f fallback to using bin_path to find archivebox binary if sys.argv is empty 2021-04-06 02:45:57 -04:00
Nick Sweeting
e4823cc5d7 add back CUSTOM_TEMPLATES_DIR config option 2021-04-06 00:57:20 -04:00
Nick Sweeting
5a9f27204a dont use chrome when its not available on windows systems 2021-04-05 23:33:08 -04:00
Nick Sweeting
bbebc33b7a better autodetection of node modules dependencies 2021-04-05 21:14:06 -04:00
Nick Sweeting
ef76f7c490 require running setup in existing data dir 2021-04-05 19:59:33 -04:00
Nick Sweeting
3449bac5bc add new setup command to autoinstall dependencies 2021-04-05 19:51:57 -04:00
Nick Sweeting
0e39a2098d split migrations and data dir check 2021-04-01 14:00:07 -04:00
Nick Sweeting
7e7c134ec0 use exc 2021-03-27 05:32:07 -04:00
Nick Sweeting
bd6d9c165b enforce utf8 on literally all file operations because windows sucks 2021-03-27 01:16:29 -04:00
Nick Sweeting
185d2f9f9b check for JSON1 extesnion in Sqlite on startup 2021-03-27 00:27:58 -04:00
Nick Sweeting
62089fdb22 close sqlite3 connections if unused 2021-02-28 22:53:34 -05:00