Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-10 06:34:16 +00:00
Code Issues Projects Releases Packages Wiki Activity
3084 commits 6 branches 77 tags 18 MiB
Commit graph

1260 commits

Author SHA1 Message Date
Nick Sweeting
03eb7e5875
Update config.py 2022-06-09 01:04:55 -07:00
renaisun
8899fe0b92
Add SINGLEFILE_ARGS to control single-file arguments 2022-06-09 14:35:48 +08:00
Nick Sweeting
d586a8babc
show mount paths with at symbol in version output 2022-06-08 20:22:58 -07:00
Nick Sweeting
319ea481b8
Update config.py 2022-06-08 20:17:38 -07:00
Nick Sweeting
01555dfe34
Update main.py 2022-06-08 20:17:31 -07:00
Nick Sweeting
2bbc742017
typo fix 2022-06-08 20:16:08 -07:00
Nick Sweeting
e2fa68dba6
resolve config paths before using 2022-06-08 20:15:22 -07:00
Nick Sweeting
ccce4a6a2f
use new is_mount and COMMIT_HASH config options 2022-06-08 20:13:22 -07:00
Nick Sweeting
9f90a2d60d
disable unused sqlite3 stuff 2022-06-08 20:12:55 -07:00
Nick Sweeting
c78a2edc42
add is_mount and COMMIT_HASH to config.py 2022-06-08 20:04:01 -07:00
Nick Sweeting
375ba9d135
Update settings.py 2022-06-08 20:00:29 -07:00
Nick Sweeting
ae5c8f2bf8
fix newline included in commit hash 2022-06-08 19:57:38 -07:00
Nick Sweeting
cb3ebbe69a
fix git commit hash location 2022-06-08 19:52:48 -07:00
Nick Sweeting
413aa2ef04
fix commit hash detection 2022-06-08 19:51:46 -07:00
Nick Sweeting
33ec2117e9
Update main.py 2022-06-08 19:50:45 -07:00
Nick Sweeting
dd29e1bf78
clean up first line of CLI version output for easier downstream parsing 2022-06-08 19:46:09 -07:00
Nick Sweeting
0c6d4c82c3
Update config.py 2022-06-08 19:11:02 -07:00
Nick Sweeting
f9c5808940
Update config.py 2022-06-08 19:09:11 -07:00
Nick Sweeting
5509b5cd8b
Update main.py 2022-06-08 19:08:33 -07:00
Nick Sweeting
19b88d30b2
fix missing brace 2022-06-08 19:06:03 -07:00
Nick Sweeting
31d5fbbf17
Update config.py 2022-06-08 19:04:06 -07:00
Nick Sweeting
6b019da3e9
Update config.py 2022-06-08 19:01:55 -07:00
Nick Sweeting
c752c7053d
Update main.py 2022-06-08 18:59:08 -07:00
Nick Sweeting
f9c82841ad
fix sqlite option detection 2022-06-08 18:58:15 -07:00
Nick Sweeting
1fd5830f58
enforce UTC timezone on server 2022-06-08 18:41:22 -07:00
Nick Sweeting
3e3c011f86
enforce UTC timezone on server 2022-06-08 18:40:48 -07:00
Nick Sweeting
e06717419c
fix sqlite3 version detection 2022-06-08 18:35:31 -07:00
Nick Sweeting
d0f129295f
move sqlite3 checks up a level 2022-06-08 18:29:53 -07:00
Nick Sweeting
0c7d7deb32
add missing brace 2022-06-08 18:26:42 -07:00
Nick Sweeting
ca16c88a3d
show PUID, PGID, ENFORCE_ATOMIC_WRITES, and OUTPUT_PERMISSIONS in version output header 2022-06-08 18:24:58 -07:00
Nick Sweeting
89175ccb22
check SQLite3 version and enabled extensions on startup 2022-06-08 18:24:17 -07:00
Nick Sweeting
c245d36e44
add PUID and PGID as config options in archivebox 2022-06-08 17:42:52 -07:00
Nick Sweeting
c5fc3e1e65 --ammend 2022-05-09 23:59:27 -07:00
Nick Sweeting
0b4df768ba hack to check for generator type cause too lazy to import 2022-05-09 23:50:56 -07:00
Nick Sweeting
5e4ddbbf25 fix mercury bin parsing back 2022-05-09 21:58:17 -07:00
Nick Sweeting
e96c1bcf13 bump mercury parser to git head version 2022-05-09 21:48:41 -07:00
Nick Sweeting
d581a5081f correctly handle bytes strings in hints 2022-05-09 21:29:37 -07:00
Nick Sweeting
a6767671fb append content of referenced files to imports 2022-05-09 21:21:39 -07:00
Nick Sweeting
f6d6a06c78 always show all totals in log output 2022-05-09 21:21:26 -07:00
Nick Sweeting
d05510f844 fix version string parsing on macOS in some cases where LANG C is not supported 2022-05-09 21:21:08 -07:00
Nick Sweeting
4b8b17e788 add update flag support to archivebox schedule 2022-05-09 20:18:43 -07:00
Nick Sweeting
8cfe6f4afb cleanup update flag handling and show better logging to clarify when its working 2022-05-09 20:15:55 -07:00
Nick Sweeting
38e54b93fe allow parsing to continue even when fetching URL contents fails 2022-05-09 19:56:24 -07:00
Nick Sweeting
ecbcb6a1b3 fix bracing in template tag for PREVIEW_ORIGINALS 2022-05-09 19:56:08 -07:00
Nick Sweeting
8ebf3e2f93 add config option PREVIEW_ORIGINALS to hide original iframes in snapshot detail pages 2022-05-09 19:31:41 -07:00
Nick Sweeting
acd53c854d handle new wallabag export format with newlines mid-tag attributes 2022-05-09 19:07:48 -07:00
Nick Sweeting
950b5cbbb6
Merge pull request #924 from prnake/dev 
improve title extractor
 2022-05-09 18:38:12 -07:00
Nick Sweeting
6e66863871 add max 5s writing delay for concurrent writers and flush WAL slower 2022-05-09 18:36:40 -07:00
Nick Sweeting
57df65f28f use yt-dlp for media archiving instead of youtube-dl 2022-04-21 07:11:35 -07:00
Nick Sweeting
eb81d41f84 bump Dockerfile base image version and install yt-dlp 2022-04-21 07:11:35 -07:00
Ross
c63822a5e5
Fix missing input redirection in a hint text 2022-04-19 22:25:49 +01:00
Igor Rzegocki
d4f534e612
add LOGOUT_REDIRECT_URL 2022-03-31 21:40:14 +02:00
Pellaeon Lin
5e9d05483e Fix bin_version: set LANG=C when calling executables to avoid parsing 
localized output.
 2022-02-24 17:01:00 +08:00
prnake
011bd104cb
remove unused import 2022-02-09 10:48:51 +08:00
papersnake
de8e22efb7 improve title extractor 2022-02-08 23:17:52 +08:00
Nick Sweeting
666ab20df5
Update archivebox/config.py 2022-01-10 20:42:09 -05:00
hannah98
fc3d2bb4dc rename TAG_SEPARATORS to TAG_SEPARATOR_PATTERN 2022-01-06 14:14:41 +00:00
hannah98
049f88def9 Added TAG_SEPARATORS option to supply a regex of characters to use when splitting tags 2021-12-30 20:19:48 +00:00
Nick Sweeting
d7f01922f3
fix direct assignment of tags to many-to-many set 2021-12-23 12:29:17 -05:00
Nick Sweeting
b1b7ee2b85
Update sql.py 2021-12-23 12:17:55 -05:00
hannah98
4b8962b60b Fix #725 - correctly parse tags on json import 2021-12-20 08:58:58 -06:00
Mika Tuupola
f14a861605 Change logfile open to write mode only 2021-12-19 23:17:33 +02:00
TheCakeIsNaOH
decab91ea2
(#847) Decode error output hints to string if needed 2021-12-16 16:46:12 -06:00
Nick Sweeting
44f5338470
fix typo in pocket_api articl variable name 2021-11-12 19:23:47 -05:00
Nick Sweeting
8878dcc5e8
Merge pull request #843 from bltavares/patch-1 2021-11-12 15:57:19 -08:00
Nick Sweeting
d5240f1a1d
Merge pull request #885 from adamwolf/safari-admin-actions 2021-11-12 08:56:31 -08:00
Adam Wolf
18e1fb0d96 Fixes Add button behavior on Safari 
Previously, when you clicked the Add button, the page wouldn't change.

It looked like it wasn't doing anything, as noted by @rcarmo
(https://github.com/ArchiveBox/ArchiveBox/issues/658#issuecomment-948300055)

I didn't track it down the exact reason why. It may be that Safari
didn't like the two opening <h3>s, but I was able to find a bunch of
people complaining about Safari being very finicky with innerHTML
and actually repainting the page, enough that I decided to try just
extending the block hide/show behavior already done with the
delay-warning, and it works for me now in both Chrome and Safari.

For #658.
 2021-10-28 22:31:54 -05:00
Adam Wolf
83731f5a68 Tweak JS so Safari can choose admin actions 
I noticed that Safari was submitting both the empty option and the
selected options back to the server.

Digging into it, I was able to get Safari to deselect the ---------
option by using '[selected]' as the selector.

For #658
 2021-10-28 22:22:46 -05:00
Igor Rzegocki
05de1c9fe6
healthcheck endpoint 2021-10-03 19:12:03 +02:00
Igor Rzegocki
95cf85f8cf
Support for Reverse Proxy authentication backends (like authelia) 2021-09-30 17:40:13 +02:00
Bruno Tavares
bb2a2e758a
Avoid KeyError on Pocket API parser 
When trying to import my pocket library I got a lot of ` KeyError`  on Python. Pocket API has a few idiosyncrasies, such as sometimes returning the keys on json, sometimes not.

` ` ` sh
archivebox add --parser pocket_api pocket://my_username
` ` ` 

Gave me this errors
` ` ` 
  File "/app/archivebox/parsers/pocket_api.py", line 54, in link_from_article
    title = article['resolved_title'] or article['given_title'] or url
KeyError: 'resolved_title'
` ` ` 

This commit are the patches I've changed to successfully import my library
 2021-09-07 21:53:36 -03:00
Ross Williams
f6cf35a45d Fix Pinboard RSS parsing valid links as None 
`item.find(p)` returns either an `ElementTree.Element` or `None`.  The
[lambda on line 24][lambda] coerces the return value to a bool, which is
`False` if the `<link>` element has no children (see
[`ElementTree.py` line 207][etbooldef]), so the lambda returns `None`.

Further, returning a `Link` with `url=None` violates
[an assertion in `index/schema.py`][assertion], which crashes
the `archivebox add` command.

[lambda]: 3d54b1321b/archivebox/parsers/pinboard_rss.py (L24)
[etbooldef]: 3d8993a744/Lib/xml/etree/ElementTree.py (L207)
[assertion]: 3d54b1321b/archivebox/index/schema.py (L165)
 2021-08-04 10:13:37 -04:00
Inndy
0e81a0722e Discard Referer header from iframe and link to original URL 2021-07-19 21:48:01 +08:00
Nick Sweeting
5a2c78e14b add proper support for URL_WHITELIST instead of using negation regexes 2021-07-06 23:42:00 -04:00
Nick Sweeting
e4974d3536 support negation patterns by checking both re.search and re.match 2021-07-06 23:17:05 -04:00
TJ Horner
cdcfb7fa44 Exempt /add route from CSRF 2021-07-01 20:55:51 -04:00
Nick Sweeting
e0a2b2e252 ominous warnings 2021-06-01 03:03:42 -04:00
Nick Sweeting
aa53fe653c fix use of uneeded perms arg 2021-06-01 02:58:36 -04:00
Nick Sweeting
c2d1a57581 fix umask dir permissions 2021-06-01 00:50:18 -04:00
Nick Sweeting
4715ace7dd ignore BaseException lgtm errors 2021-05-31 20:59:05 -04:00
Nick Sweeting
9f1470cf03 fix output permissions tests 2021-05-31 20:57:46 -04:00
Nick Sweeting
8230f88d80 change default OUTPUT_PERMISSIONS to disallow execution except on dirs 2021-05-31 19:31:51 -04:00
Nick Sweeting
1112526543 add option ENFORCE_ATOMIC_WRITES to allow disabling forced FSYNC writes on network drives 2021-05-31 19:31:51 -04:00
Nick Sweeting
49faec8f6d
add no-zygote and single-process args to try and prevent orphan chrome processes after exit 2021-05-13 05:04:23 -04:00
Nick Sweeting
eb4d3bca9d
Update readability.py 2021-05-13 00:13:32 -04:00
Nick Sweeting
c3d009e4ec fix python file execution checking 2021-04-24 04:43:02 -04:00
Nick Sweeting
79e19ecd47 shield pwd import for windows 2021-04-24 03:51:38 -04:00
Nick Sweeting
3db77fd1a2 fix lint errors 2021-04-24 03:49:01 -04:00
Nick Sweeting
3992e0fee3 auto prepend python binary to args when running system command with python file as first argument 2021-04-24 03:29:22 -04:00
Nick Sweeting
fa84f749ff run archivebox version using python binary 2021-04-24 03:29:22 -04:00
Nick Sweeting
226e26852c dont try to autodetect whether node, chrome, etc are needed during setup 2021-04-24 03:29:22 -04:00
Nick Sweeting
f129b9f443 allow executing main 2021-04-24 03:11:06 -04:00
Nick Sweeting
208f866fc4 fix user detection on freebsd always returning root 2021-04-24 02:52:09 -04:00
Nick Sweeting
06f6084e3b ignore stdin when passed instead of throwing an error 2021-04-24 00:09:52 -04:00
Nick Sweeting
eb80dc26a1 cleanup config files 2021-04-23 22:58:44 -04:00
Nick Sweeting
4f91af0ce9 TODO create admin user from env vars 2021-04-23 15:50:34 -04:00
Nick Sweeting
9563e74f7c enforce minimum django version 2021-04-23 15:02:26 -04:00
Kunwoo Ahn
b6ee7d291e
fix bookmarklet script 
Fix '&' bug in parameter
 2021-04-16 21:24:51 +09:00
Nick Sweeting
63552be0e3 whoops 2021-04-13 04:03:47 -04:00
Nick Sweeting
7d1f8cb496 no longer recommend snap chromium, just use playwright 2021-04-13 03:59:17 -04:00