Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-22 20:23:12 +00:00
Code Issues Projects Releases Packages Wiki Activity
3270 commits 6 branches 78 tags 19 MiB
Commit graph

3270 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Sweeting
62fb56354b
Merge pull request #1219 from spresse1/chrome-cleanup 2023-08-28 20:02:22 -07:00
spresse1
603ce7ec10 After a timeout, chrome will leave behind a SingletonLock, which prevents future instances of chrome from starting. When an extractor fails due to a timeout, remove this file. 2023-08-28 17:27:03 +02:00
Nick Sweeting
0b6064b7dd
Update docker_entrypoint.sh to use /bin/bash 2023-08-22 16:35:43 -07:00
root
23f086aa40 add LDAP support 2023-08-17 19:51:02 -05:00
Nick Sweeting
00ecf57b0f
Merge pull request #1211 from DanielBatteryStapler/DanielBatteryStapler-patch-1 2023-08-15 21:34:53 -07:00
DanielBatteryStapler
94dacc49c7
Fix archive_org icon "exists" 2023-08-15 23:49:54 -04:00
Nick Sweeting
68e936e7c2
Merge pull request #1186 from ArchiveBox/dependabot/npm_and_yarn/word-wrap-1.2.4 2023-08-13 16:43:09 -07:00
Nick Sweeting
a7d7644dca
Merge pull request #1205 from overhacked/fix_url_regex_hyphen 2023-08-09 14:20:07 -07:00
Ross Williams
c039ef05b3 Fix hyphen placement in util.URL_REGEX 
Incorrect hyphen placement in `URL_REGEX` was allowing it to match more
characters than intended. In a regex character class, a literal hyphen
can only appear as the first character in the class, or it will be
interpreted as the delimiter of a range of characters.

The issue fixed here caused the range of characters from `[$-_]`
be treated as valid URL characters, instead of the intended set of three
characters `[-_$]`. The incorrect range interpretation inadvertantly
included most ASCII punctuation, most importantly the angle brackets,
square brackets, and single quote that the expression uses
to mark the end of a match.

This causes the expression to match a URL that has a "hostname" portion
beginning with one of the intended "stop parsing" characters. For
example:

```
https://<b>www</b>.example.com/  # MATCHES but should not
https://[for example]            # MATCHES but should not
scheme='https://'                # MATCHES, including final quote, but should not
```

Some test cases have been added to the `URL_REGEX` assert in
archivebox.parsers to cover this possibility.
 2023-08-08 15:24:16 -04:00
Ross Williams
2076474252 Drop use of TypeAlias to maintain Python 3.9 compat 
TypeAlias annotation was introduced in Python 3.10, and is not strictly
necessary. Drop use of it to maintain Python 3.9 compatibility.
 2023-08-02 10:56:48 -04:00
Ross Williams
b44f7e68b1 Add URL-specific method allow/deny lists 
Allows enabling only allow-listed extractors or disabling specific
deny-listed extractors for a regular expression matched against an added
site's URL.
 2023-08-02 09:36:40 -04:00
Ross Williams
46e80dd509 Rename URL_(WHITE|BLACK)LIST to URL_(ALLOW|DENY)LIST 
Retain aliases for old configuration files
 2023-08-02 09:31:48 -04:00
Nick Sweeting
b773041952
Merge pull request #1199 from overhacked/chrome_version_detection_fix 2023-08-01 10:14:18 -07:00
Nick Sweeting
5b7ecfc872
Merge pull request #1197 from overhacked/bin_version_env_fix 2023-08-01 10:12:59 -07:00
Ross Williams
d0e65eba7f More reliably detect Google Chrome version number 
Previous method was splitting on the first whitespace, and missing the
version number when it appeared as `"Google Chrome 115.0.234.2342"`
instead of, i.e. `"Chromium 115.0.234.8283"`.

This commit changes the version detection to regex search for
whitespace, then one or more digits followed by a period, then at least
one more digit. Only the first sequence of digits is captured. Unless
Chrome radically changes their version numbering, this should capture
the first group of digits after the reported browser name, which would
be the major version.
 2023-07-31 15:34:58 -04:00
Ross Williams
9d9872d325 bin_version means to modify, not replace environ 
the `bin_version` function means to modify the environment,
not replace it entirely. Fixes bugs that occur when it wipes out the
PATH environment variable, such as when running in a virtual
environment.
 2023-07-31 11:36:34 -04:00
Nick Sweeting
3e5e9c7a41
Merge pull request #1194 from wogong/patch-1 2023-07-29 09:29:34 -07:00
Zhen
3e9e221232
Fix Instapaper export link in README.md 
Original link to Instapaper export `https://www.instapaper.com/user/export` is broken: `405: Method Not Allowed`.
 2023-07-28 07:58:58 +08:00
dependabot[bot]
0bf739b736
Bump word-wrap from 1.2.3 to 1.2.4 
Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4.
- [Release notes](https://github.com/jonschlinkert/word-wrap/releases)
- [Commits](https://github.com/jonschlinkert/word-wrap/compare/1.2.3...1.2.4)

---
updated-dependencies:
- dependency-name: word-wrap
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-07-19 02:03:00 +00:00
Nick Sweeting
40ddd33602
Update README.md 2023-07-07 09:37:42 -07:00
mAAdhaTTah
181501fd36
Add Readwise Reader API parser 
Implemented similar to the Pocket API.
 2023-07-02 11:20:58 -04:00
Nick Sweeting
0d26538a4b
Update README.md example commands to use new docker compose format 2023-06-13 17:46:32 -07:00
Nick Sweeting
37d238cd31
Update README.md 2023-06-13 17:43:40 -07:00
Nick Sweeting
2580f76a2e
Update README.md 2023-06-13 17:41:08 -07:00
Nick Sweeting
571131d5f3
Update README.md to simplify intro instructions 2023-06-13 17:35:00 -07:00
Nick Sweeting
733dbfa1f3
Update scheduler to persist single shared contab via volume instead of requiring separate container for each job 2023-06-13 17:13:55 -07:00
Nick Sweeting
58d784cdd8
limit nginx config to only serve archive directory instead of main data folder root 2023-06-13 16:43:37 -07:00
Nick Sweeting
0e0b06bef1
Merge pull request #1159 from ArchiveBox/pirate-patch-1 2023-06-13 05:49:58 -07:00
Nick Sweeting
406e2b681d
Update docker-compose.yml scheduled task image and container name 2023-06-13 05:49:22 -07:00
Nick Sweeting
347b8d977d
Merge pull request #1154 from sissbruecker/fix/oneshot_exit_code 2023-05-30 17:34:25 -07:00
Sascha Ißbrücker
7bf4f40da0 just use out_dir 2023-05-29 10:03:49 +02:00
Sascha Ißbrücker
40c122515a fix: make oneshot command return successful exist code 2023-05-29 10:01:27 +02:00
Nick Sweeting
32eef51081
Merge pull request #1153 from ArchiveBox/pirate-patch-1 
Add links to new social media profiles under Further Reading
 2023-05-27 06:08:25 -07:00
Nick Sweeting
d6780d1ab7
Add links to new social media profiles under Further Reading 2023-05-27 06:08:04 -07:00
Nick Sweeting
dbe1cd8dc0
Merge pull request #1152 from jess-sol/patch-1 2023-05-26 21:34:15 -07:00
J
aa5533b80f
Update export_browser_history.sh 
Add tags to export of Firefox bookmarks. Ignore invalid bookmarks such as 'javascript:' or 'place:'
 2023-05-25 09:08:13 -06:00
Nick Sweeting
081ba7e81b
Merge pull request #1147 from mrled/set-app-permissions-dockerfile 2023-05-06 00:20:50 -07:00
Micah R Ledbetter
3edb319ba9 Set /app permissions in the Dockerfile 
Add a note in readme to be careful of permissions with --reload
 2023-05-05 23:47:02 -05:00
Nick Sweeting
5bbc5228cc
Merge pull request #1146 from mrled/add-favicon-provider 2023-05-05 19:00:15 -07:00
Micah R Ledbetter
1e50ca243e Add FAVICON_PROVIDER option for custom favicon service 2023-05-05 20:42:36 -05:00
Nick Sweeting
a1e2fce405
Update debian CI builds to use ubuntu 22.04 and actions/checkout@v3 2023-04-18 22:17:58 -07:00
Nick Sweeting
32fac5999e
Merge pull request #1131 from dcalano/dev 2023-03-30 13:14:17 -06:00
David Calano
f48e48e6da
Fix for Issue #1008 
- Added missing decode() when setting pkg_path variable
 2023-03-29 01:48:12 -04:00
Nick Sweeting
f86891bc33
Merge pull request #1128 from tejr/dev 2023-03-26 18:46:06 -06:00
Tom Ryder
53af810ff8 Add missing closing quote to style attribute 2023-03-27 10:54:04 +13:00
Nick Sweeting
829e377890
Merge pull request #1123 from fa0311/dev 2023-03-18 08:07:23 -06:00
ふぁ
44a5a5ed7e
add explicitly specify --headless=new 
Signed-off-by: ふぁ <yuki@yuki0311.com>
 2023-03-17 19:30:14 +09:00
Nick Sweeting
f5e631dbfa
Merge pull request #1120 from fa0311/dev 2023-03-15 17:05:46 -06:00
Nick Sweeting
9f42a3bf29
fix whitespace 2023-03-15 16:01:02 -07:00
ふぁ
d77c770c47
add CHROME_TIMEOUT args 
Signed-off-by: ふぁ <yuki@yuki0311.com>
 2023-03-14 20:29:41 +09:00