Mirrors/ArchiveBox
2
0
Fork 0
mirror of https://github.com/ArchiveBox/ArchiveBox synced 2024-11-25 05:30:23 +00:00
Code Issues Projects Releases Packages Wiki Activity
4553 commits 6 branches 78 tags 19 MiB
Commit graph

176 commits

Author SHA1 Message Date
Nick Sweeting
60154fba5f
add django_huey, huey_monitor, and replace Threads with huey tasks 2024-09-10 00:05:45 -07:00
Nick Sweeting
c76c50e71f
add HTTP byte range request support to media file serving 2024-09-05 21:41:49 -07:00
Nick Sweeting
b5ad134264
dont wait for ipython history saver thread before shell exit 2024-05-12 19:25:55 -07:00
Nick Sweeting
e1a04729b3
wait for bg threads to finish before exiting 2024-05-12 01:42:20 -07:00
Nick Sweeting
8ab17f0561 add tag support to schedule command 2024-01-03 20:31:14 -08:00
Nick Sweeting
4b8b17e788 add update flag support to archivebox schedule 2022-05-09 20:18:43 -07:00
Nick Sweeting
8cfe6f4afb cleanup update flag handling and show better logging to clarify when its working 2022-05-09 20:15:55 -07:00
Nick Sweeting
8b4f84959a new setup flag on init command to autosetup on init 2021-04-05 21:15:32 -04:00
Nick Sweeting
211bf86f4a fix bug checking value in subcommand_args when None 2021-04-05 21:13:49 -04:00
Nick Sweeting
05c17af4dc fix lint errors 2021-04-05 20:00:16 -04:00
Nick Sweeting
ef76f7c490 require running setup in existing data dir 2021-04-05 19:59:33 -04:00
Nick Sweeting
8c1d12020f fix lint errors 2021-04-05 19:53:42 -04:00
Nick Sweeting
3449bac5bc add new setup command to autoinstall dependencies 2021-04-05 19:51:57 -04:00
Nick Sweeting
0e39a2098d split migrations and data dir check 2021-04-01 14:00:07 -04:00
Nick Sweeting
005da31f8f add overwrite flag option to schedule cmd 2021-03-31 11:29:51 -04:00
Nick Sweeting
36f0646501
Merge pull request #669 from FliegendeWurst/fix-issue-235 
add command: --parser option (fixes #235)
 2021-03-31 00:53:47 -04:00
Nick Sweeting
2656e59215
change list style 2021-03-31 00:47:42 -04:00
FliegendeWurst
60bd9a902e add command: --parser option 2021-03-28 10:09:11 +02:00
Nick Sweeting
fea0b89dbe add tag cli option 2021-03-27 03:57:05 -04:00
Nick Sweeting
bd6d9c165b enforce utf8 on literally all file operations because windows sucks 2021-03-27 01:16:29 -04:00
Nick Sweeting
6aef1dd98d dont check db if init is pending 2021-02-28 22:53:23 -05:00
Nick Sweeting
33df9c1ebe fix after and before in remove tests 2021-02-18 06:21:44 -05:00
Nick Sweeting
106f6adc59 fix lgtm alerts 2021-02-18 04:29:24 -05:00
Nick Sweeting
74c882ef16 add nothreading arg to archivebox server for perf profiling 2021-02-16 15:46:58 -05:00
Nick Sweeting
49939f3eaa only accept stdin if args are not passed, fix stdin hang in docker 2021-02-16 01:20:47 -05:00
Nick Sweeting
9ce3bd5bdc use index.LINK_FILTERS to validate filter-type args instead of hardocding them twice 2021-02-15 20:43:36 -05:00
Nick Sweeting
3c3bae02d2 add quick-init option to skip reimporting all snapshot dirs on init 2021-02-15 14:52:10 -05:00
Nick Sweeting
0407d03b6b add cli tests file back 2021-02-15 13:39:49 -05:00
Nick Sweeting
c4b02be24d remove dead tests code 2021-02-01 05:14:43 -05:00
Nick Sweeting
39ec77e46c add createsuperuser flag to server command 2021-01-28 22:27:02 -05:00
Nick Sweeting
02bdb3bdeb fix DATABASE_NAME posixpath 2021-01-20 18:42:12 -05:00
Cristian
a57a5b6b83 refactor: call setup_django with the check_db attribute for the commands that actually need the database 2020-12-11 18:02:56 -05:00
Nick Sweeting
9fa70b3452 add extractors arg to oneshot command and bump version to v0.5.1 2020-12-11 15:48:46 +02:00
Cristian
275ad22db7 refactor: Remove skip_index from archive related functions 2020-12-08 18:42:25 -05:00
Cristian
9aac09a5e1 feat: Patch setup_django so we can use an inmemory db in specific commands 2020-12-08 18:42:25 -05:00
Cristian
35a5700c73 fix: Move the setup_django command to a place where we already know what the actual subcommand is 2020-12-08 18:42:25 -05:00
JDC
273c9d91c6 Add tag filter to update command 2020-12-06 01:13:39 +02:00
JDC
8484bdb973 Fix add search filter to update 2020-12-06 01:13:39 +02:00
JDC
70cc0c1950 Add search filter-type 2020-12-06 01:13:39 +02:00
JDC
0ed53cc117 Add search filter type for update 2020-12-06 01:13:39 +02:00
JDC
0f7dba07df feat: add search filter-type to list command 2020-12-06 01:13:37 +02:00
Cristian
35389608d1 feat: Add --extract flag to update command 2020-12-05 12:20:47 -05:00
Nick Sweeting
1b22f8eeef
Merge pull request #515 from cdvv7788/POC-setup-django-on-init 2020-11-27 23:56:37 -05:00
Nick Sweeting
fdd4effc92
Merge pull request #535 from cdvv7788/extractors-flag 2020-11-13 14:53:17 -05:00
Nick Sweeting
257d3f2a98
Update archivebox/cli/archivebox_add.py 2020-11-13 14:52:21 -05:00
JDC
d54c3eec9d Add tag filter argument to remove command 2020-11-13 14:16:48 -05:00
Cristian
54df0a035b fix: Move csv split to the add function to avoid optional nullable argument 2020-11-13 13:10:17 -05:00
Cristian
1ec8276514 fix: Use a comma separated input instead of nargs for the extract flag 2020-11-13 13:01:11 -05:00
JDC
cbb3d04c12 Allow list filtering by tag name 2020-11-13 12:06:12 -05:00
Cristian
44eede96e5 feat: Add extract flag to add command 2020-11-13 09:24:34 -05:00
Nick Sweeting
cac3912439 small type fixes 2020-10-30 04:50:14 -04:00
Cristian
a6bee5f111 feat: Move setup_django to an inner module 2020-10-26 08:02:04 -05:00
Angel Rey
3e26ab3ce3 Replaced os.path in clic tests 2020-10-02 15:46:39 -05:00
Angel Rey
fa364ed728 Replaced od.path in init cli 2020-10-02 15:46:39 -05:00
apkallum
1aa7bac85b fix oneshot command type signature 2020-09-17 09:09:52 -05:00
Cristian
fa622d3e14 refactor: Replace --index with --with-headers in the list command to make it more explicit. Change it so it affects the csv output too. 2020-09-15 08:05:46 -05:00
Cristian
404f333e17 feat: Refactor get_invalid_folders to work with a queryset instead of a list of links 2020-09-15 08:05:46 -05:00
Cristian
a77d6dc235 feat: list command fails when --index is used without --json or --html 2020-09-15 08:05:46 -05:00
Cristian
885ff50449 feat: Add html export to list command 2020-09-15 08:05:46 -05:00
Cristian
aab8f96520 feat: Add flag to list command to support index like output 2020-09-15 08:05:46 -05:00
Nick Sweeting
73408fb035 fix version parsing and attempt to npm install during pip post_install 2020-08-18 15:58:05 -04:00
Nick Sweeting
718d39e242 add common code extensions to default blacklist 2020-08-18 08:12:10 -04:00
Nick Sweeting
b681a477ae add overwrite flag to add command to force re-archiving 2020-08-18 04:37:54 -04:00
Nick Sweeting
da671532a4 fix schedule cmd defaulting to day and remove week 2020-08-18 04:35:59 -04:00
Nick Sweeting
494be09bc2 add depth flag to schedule cmd 2020-08-18 01:58:54 -04:00
Nick Sweeting
a0901ba474 use BIND_ADDR config default for runserver 2020-08-13 23:21:37 -04:00
Cristian Vargas
b2a318c5eb
fix: Update error message for oneshot command 
Co-authored-by: Nick Sweeting <git@sweeting.me>
 2020-07-31 10:51:54 -05:00
Cristian
8bcb171e74 fix: Remove support for multiple urls in oneshot command 2020-07-31 09:05:40 -05:00
Cristian
c073ea141d feat: Initial oneshot command proposal 2020-07-29 11:19:06 -05:00
Nick Sweeting
9e7330cc14 add init flag to server and fix SHOW_PROGRESS config being ignored 2020-07-28 05:57:34 -04:00
Cristian
6006b4f93b refactor: Organize code to remove flake8 issues 2020-07-24 12:25:25 -05:00
Cristian
a5550b2105 fix: Rename logging folder to avoid naming conflicts (and circular import issues) 2020-07-22 11:02:13 -05:00
Cristian
f4d1b5121e refactor: Move logging.py to main module to avoid circular import issues 2020-07-17 18:00:04 -05:00
Nick Sweeting
ae208435c9 fix the add links form 2020-07-13 12:21:37 -04:00
Nick Sweeting
d159e674e1 write stderr instead of stdout for version info 2020-07-13 11:41:18 -04:00
Nick Sweeting
d3bfa98a91 fix depth flag and tweak logging 2020-07-13 11:26:34 -04:00
Cristian
4ebf929606 refactor: Change wording on CLI help 2020-07-08 08:30:07 -05:00
Cristian
f12bfeb322 refactor: Change add() to receive url and depth instead of import_str and import_path 2020-07-08 08:17:47 -05:00
Cristian
c1d8a74e4f feat: Make input sent via stdin behave the same as using args 2020-07-07 15:49:40 -05:00
Cristian
b68c13918f feat: Disable stdin from archivebox add 2020-07-07 12:39:36 -05:00
Cristian
a6940092bb feat: Make sure that depth can only be either 1 or 0 2020-07-07 10:25:02 -05:00
Cristian
32e790979e feat: Enable depth=1 functionality 2020-07-07 10:07:44 -05:00
Cristian
2db0324539 feat: depth=0 crawls the current page only 2020-07-07 09:49:28 -05:00
Cristian
8b22a2a7dd feat: Enable --depth flag (still does nothing) 2020-07-07 09:10:36 -05:00
Nick Sweeting
7c428f40c8 fix stdin link importing 2020-07-02 13:31:05 -04:00
Nick Sweeting
0c48449aa6 fix subcommand and args not being passed 2020-07-02 04:00:51 -04:00
Nick Sweeting
322be6b292 move main into cli init and remove circular import layer 2020-07-02 03:53:39 -04:00
Nick Sweeting
df593dea0a fix missing imports 2020-06-30 05:55:34 -04:00
Nick Sweeting
44bd6866ac colorize hints in CLI output more consistently 2020-06-30 01:48:41 -04:00
Nick Sweeting
1ecb94fe09 check PYTHON_ENCODING correclty when lowercase 2020-06-30 01:07:38 -04:00
Nick Sweeting
b9e17fa0d1 rename archivebox-info to archivebox-status 2020-06-25 23:32:01 -04:00
Nick Sweeting
f60b5ed867 better stdin handling 2019-05-01 02:28:12 -04:00
Nick Sweeting
ad3898add6 allow forcing init in dirty directory 2019-05-01 02:27:50 -04:00
Nick Sweeting
cb2dd1ee28 rename model Page to Snapshot 2019-04-30 23:44:51 -04:00
Nick Sweeting
95007d9137 split up utils into separate files 2019-04-30 23:13:04 -04:00
Nick Sweeting
daf5951897 allow passing debug flag to archivebox server 2019-04-30 23:11:41 -04:00
Nick Sweeting
0ba821b351 better return status from list cmd 2019-04-30 23:11:07 -04:00
Nick Sweeting
158f145d9a move docstrings to main.py out of cli files 2019-04-30 23:10:48 -04:00
Nick Sweeting
1b8abc0961 move everything out of legacy folder 2019-04-27 17:26:24 -04:00
Nick Sweeting
1edb14b229 wip config reset command 2019-04-26 14:43:23 -04:00