Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-29 23:50:22 +00:00
Code Issues Projects Releases Packages Wiki Activity
3280 commits 6 branches 78 tags 19 MiB
Commit graph

3280 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Sweeting
ffe2968e4f improve some comments 2023-09-14 02:41:27 -07:00
Nick Sweeting
f809efce4d
Merge pull request #996 from barthalion/dev 2023-09-03 21:40:49 -07:00
Nick Sweeting
aaca74f6a8
only start parsing json after the first open brace 2023-09-03 21:40:12 -07:00
Nick Sweeting
cd9f228b2f
Merge pull request #1214 from DanielBatteryStapler/DanielBatteryStapler-patch-1 2023-09-03 21:25:12 -07:00
Nick Sweeting
16d278fbdb
Merge pull request #1168 from mAAdhaTTah/add-readwise-reader 2023-09-03 21:24:49 -07:00
Nick Sweeting
110a22ee32
Merge branch 'dev' into DanielBatteryStapler-patch-1 2023-08-31 15:20:46 -07:00
Nick Sweeting
73a5f74d38
update default YOUTUBEDL_ARGS to fix subs and filesize 2023-08-31 15:17:45 -07:00
Nick Sweeting
86366d5640
Update logging_util.py to fix generator subscripting error 2023-08-31 15:12:43 -07:00
Nick Sweeting
a837f870af
Merge pull request #1221 from spresse1/update-singlefile 2023-08-29 17:04:04 -07:00
spresse1
c8597a7fa1 Update singlefile to the latest version and switch it to single-file-cli. Unfortunately, this requires a rewrite of NPM dependency files. 2023-08-29 20:28:48 +02:00
Nick Sweeting
62fb56354b
Merge pull request #1219 from spresse1/chrome-cleanup 2023-08-28 20:02:22 -07:00
spresse1
603ce7ec10 After a timeout, chrome will leave behind a SingletonLock, which prevents future instances of chrome from starting. When an extractor fails due to a timeout, remove this file. 2023-08-28 17:27:03 +02:00
Nick Sweeting
0b6064b7dd
Update docker_entrypoint.sh to use /bin/bash 2023-08-22 16:35:43 -07:00
root
23f086aa40 add LDAP support 2023-08-17 19:51:02 -05:00
Nick Sweeting
00ecf57b0f
Merge pull request #1211 from DanielBatteryStapler/DanielBatteryStapler-patch-1 2023-08-15 21:34:53 -07:00
DanielBatteryStapler
94dacc49c7
Fix archive_org icon "exists" 2023-08-15 23:49:54 -04:00
Nick Sweeting
68e936e7c2
Merge pull request #1186 from ArchiveBox/dependabot/npm_and_yarn/word-wrap-1.2.4 2023-08-13 16:43:09 -07:00
Nick Sweeting
a7d7644dca
Merge pull request #1205 from overhacked/fix_url_regex_hyphen 2023-08-09 14:20:07 -07:00
Ross Williams
c039ef05b3 Fix hyphen placement in util.URL_REGEX 
Incorrect hyphen placement in `URL_REGEX` was allowing it to match more
characters than intended. In a regex character class, a literal hyphen
can only appear as the first character in the class, or it will be
interpreted as the delimiter of a range of characters.

The issue fixed here caused the range of characters from `[$-_]`
be treated as valid URL characters, instead of the intended set of three
characters `[-_$]`. The incorrect range interpretation inadvertantly
included most ASCII punctuation, most importantly the angle brackets,
square brackets, and single quote that the expression uses
to mark the end of a match.

This causes the expression to match a URL that has a "hostname" portion
beginning with one of the intended "stop parsing" characters. For
example:

```
https://<b>www</b>.example.com/  # MATCHES but should not
https://[for example]            # MATCHES but should not
scheme='https://'                # MATCHES, including final quote, but should not
```

Some test cases have been added to the `URL_REGEX` assert in
archivebox.parsers to cover this possibility.
 2023-08-08 15:24:16 -04:00
Ross Williams
2076474252 Drop use of TypeAlias to maintain Python 3.9 compat 
TypeAlias annotation was introduced in Python 3.10, and is not strictly
necessary. Drop use of it to maintain Python 3.9 compatibility.
 2023-08-02 10:56:48 -04:00
Ross Williams
b44f7e68b1 Add URL-specific method allow/deny lists 
Allows enabling only allow-listed extractors or disabling specific
deny-listed extractors for a regular expression matched against an added
site's URL.
 2023-08-02 09:36:40 -04:00
Ross Williams
46e80dd509 Rename URL_(WHITE|BLACK)LIST to URL_(ALLOW|DENY)LIST 
Retain aliases for old configuration files
 2023-08-02 09:31:48 -04:00
Nick Sweeting
b773041952
Merge pull request #1199 from overhacked/chrome_version_detection_fix 2023-08-01 10:14:18 -07:00
Nick Sweeting
5b7ecfc872
Merge pull request #1197 from overhacked/bin_version_env_fix 2023-08-01 10:12:59 -07:00
Ross Williams
d0e65eba7f More reliably detect Google Chrome version number 
Previous method was splitting on the first whitespace, and missing the
version number when it appeared as `"Google Chrome 115.0.234.2342"`
instead of, i.e. `"Chromium 115.0.234.8283"`.

This commit changes the version detection to regex search for
whitespace, then one or more digits followed by a period, then at least
one more digit. Only the first sequence of digits is captured. Unless
Chrome radically changes their version numbering, this should capture
the first group of digits after the reported browser name, which would
be the major version.
 2023-07-31 15:34:58 -04:00
Ross Williams
9d9872d325 bin_version means to modify, not replace environ 
the `bin_version` function means to modify the environment,
not replace it entirely. Fixes bugs that occur when it wipes out the
PATH environment variable, such as when running in a virtual
environment.
 2023-07-31 11:36:34 -04:00
Nick Sweeting
3e5e9c7a41
Merge pull request #1194 from wogong/patch-1 2023-07-29 09:29:34 -07:00
Zhen
3e9e221232
Fix Instapaper export link in README.md 
Original link to Instapaper export `https://www.instapaper.com/user/export` is broken: `405: Method Not Allowed`.
 2023-07-28 07:58:58 +08:00
dependabot[bot]
0bf739b736
Bump word-wrap from 1.2.3 to 1.2.4 
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-19 02:03:00 +00:00
Nick Sweeting
40ddd33602
Update README.md 2023-07-07 09:37:42 -07:00
mAAdhaTTah
181501fd36
Add Readwise Reader API parser 
Implemented similar to the Pocket API.
 2023-07-02 11:20:58 -04:00
Nick Sweeting
0d26538a4b
Update README.md example commands to use new docker compose format 2023-06-13 17:46:32 -07:00
Nick Sweeting
37d238cd31
Update README.md 2023-06-13 17:43:40 -07:00
Nick Sweeting
2580f76a2e
Update README.md 2023-06-13 17:41:08 -07:00
Nick Sweeting
571131d5f3
Update README.md to simplify intro instructions 2023-06-13 17:35:00 -07:00
Nick Sweeting
733dbfa1f3
Update scheduler to persist single shared contab via volume instead of requiring separate container for each job 2023-06-13 17:13:55 -07:00
Nick Sweeting
58d784cdd8
limit nginx config to only serve archive directory instead of main data folder root 2023-06-13 16:43:37 -07:00
Nick Sweeting
0e0b06bef1
Merge pull request #1159 from ArchiveBox/pirate-patch-1 2023-06-13 05:49:58 -07:00
Nick Sweeting
406e2b681d
Update docker-compose.yml scheduled task image and container name 2023-06-13 05:49:22 -07:00
Nick Sweeting
347b8d977d
Merge pull request #1154 from sissbruecker/fix/oneshot_exit_code 2023-05-30 17:34:25 -07:00
Sascha Ißbrücker
7bf4f40da0 just use out_dir 2023-05-29 10:03:49 +02:00
Sascha Ißbrücker
40c122515a fix: make oneshot command return successful exist code 2023-05-29 10:01:27 +02:00
Nick Sweeting
32eef51081
Merge pull request #1153 from ArchiveBox/pirate-patch-1 
Add links to new social media profiles under Further Reading
 2023-05-27 06:08:25 -07:00
Nick Sweeting
d6780d1ab7
Add links to new social media profiles under Further Reading 2023-05-27 06:08:04 -07:00
Nick Sweeting
dbe1cd8dc0
Merge pull request #1152 from jess-sol/patch-1 2023-05-26 21:34:15 -07:00
J
aa5533b80f
Update export_browser_history.sh 
Add tags to export of Firefox bookmarks. Ignore invalid bookmarks such as 'javascript:' or 'place:'
 2023-05-25 09:08:13 -06:00
Nick Sweeting
081ba7e81b
Merge pull request #1147 from mrled/set-app-permissions-dockerfile 2023-05-06 00:20:50 -07:00
Micah R Ledbetter
3edb319ba9 Set /app permissions in the Dockerfile 
Add a note in readme to be careful of permissions with --reload
 2023-05-05 23:47:02 -05:00
Nick Sweeting
5bbc5228cc
Merge pull request #1146 from mrled/add-favicon-provider 2023-05-05 19:00:15 -07:00
Micah R Ledbetter
1e50ca243e Add FAVICON_PROVIDER option for custom favicon service 2023-05-05 20:42:36 -05:00