Commit graph

50 commits

Author SHA1 Message Date
Sascha Ißbrücker
4220ea0b4c
Fix website loader content encoding detection (#482) 2023-05-30 22:04:54 +02:00
Sascha Ißbrücker
43115fd8f2
Add notes to bookmarks (#472)
* Add basic bookmark notes

* Add bookmark list JS to shared bookmarks page

* Allow testing through ngrok

* Improve CSS

* Set notes through API

* Improve notes editing

* Improve notes icon

* Remove transitions for now

* Update keyboard shortcut

* Add bookmark list tests

* Add setting for showing notes permanently

* Add test for toggling notes

* Update API docs

* Allow searching for notes content

* Skip test
2023-05-20 11:54:26 +02:00
Sascha Ißbrücker
74134d3896
Escape texts in exported HTML (#429) 2023-02-18 18:25:54 +01:00
Sascha Ißbrücker
6b4664117b
Fix favicon being cleared by web archive snapshot task (#405) 2023-01-22 14:07:06 +01:00
Sascha Ißbrücker
814401be2e
Add option for showing bookmark favicons (#390)
* Implement favicon loader

* Implement load favicon task

* Show favicons in bookmark list

* Add missing migration

* Load missing favicons on import

* Automatically refresh favicons

* Add enable favicon setting

* Update uwsgi config to host favicons

* Improve settings wording

* Fix favicon loader test setup

* Document LD_FAVICON_PROVIDER setting

* Add refresh favicons button
2023-01-21 16:36:10 +01:00
Sascha Ißbrücker
30da1880a5
Cache website metadata to avoid duplicate scraping (#401)
* Cache website metadata to avoid duplicate scraping

* fix test setup
2023-01-20 22:28:44 +01:00
Sascha Ißbrücker
021d1cd673
Fix bookmark website metadata not being updated when URL changes (#400) 2023-01-20 20:59:09 +01:00
Sascha Ißbrücker
43d52642a6 Fix website loader test 2023-01-14 12:26:04 +01:00
Sascha Ißbrücker
4f9170c48d Improve website loader logging 2023-01-14 11:24:09 +01:00
Luca
c2d8cde86b
Trim website metadata title and description (#383)
* feat: trim fetched metadata placeholders

* feat: implement trimming serverside

* Add website loader tests

* Address review comments

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2023-01-12 21:06:36 +01:00
Sascha Ißbrücker
2fd7704816
Limit document size for website scraper (#354)
Limits the size of scraped HTML documents to prevent out of memory errors. The scraper will stop reading from the response when it encounters the closing head tag, or if the read content's size exceeds a max limit.

Fixes #345
2022-10-07 21:18:18 +02:00
Sascha Ißbrücker
b94eaee833 Setup logging for background tasks 2022-09-11 07:50:08 +02:00
Sascha Ißbrücker
1b35d5b5ef
Prevent rate limit errors in wayback machine API (#339)
The Wayback Machine Save API only allows a limited number of requests within a timespan. This introduces several changes to avoid rate limit errors:
- There will be max. 1 attempt to create a new snapshot
- If a new snapshot could not be created, then attempt to use the latest existing snapshot
- Bulk snapshot updates (bookmark import, load missing snapshots after login) will only attempt to load the latest snapshot instead of creating new ones
2022-09-10 20:43:15 +02:00
Sascha Ißbrücker
1b67081773
Skip updating website metadata on edit unless URL has changed (#318)
* Skip updating website metadata on edit unless URL has changed

* Prevent form fetching metadata when editing existing bookmark
2022-08-13 11:21:26 +02:00
Sascha Ißbrücker
fec966f687
Add bookmark sharing (#311)
* Allow marking bookmarks as shared

* Add basic share view

* Ensure tag names in tag cloud are unique

* Filter shared bookmarks by user

* Add link for filtering by user

* Prevent n+1 queries when rendering bookmark list

* Prevent empty query params in return URL

* Fix user select template tag name

* Create shared bookmarks through API

* List shared bookmarks through API

* Show bookmark suggestions for shared view

* Show unique tags in search suggestions

* Sort user options

* Add bookmark sharing feature flag

* Add test for share setting default

* Simplify settings view
2022-08-04 19:37:16 +02:00
Sascha Ißbrücker
e6718be53b
Update unread flag when saving duplicate URL (#306)
Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-07-26 00:13:41 +02:00
Sascha Ißbrücker
13ff9ac4f8
Add read it later functionality (#304)
* Allow marking bookmarks as unread

* Restructure navigation to include preset filters

* Add mark as read action

* Improve description

* Highlight unread bookmarks visually

* Mark bookmarks as read by default

* Add tests

* Implement toread flag in importer

* Implement admin actions

* Add query tests

* Remove untagged link

* Update api docs

* Reduce height of description textarea

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-07-23 22:17:20 +02:00
Sascha Ißbrücker
b618a8b10b Do not associate tags if bookmark was not imported 2022-07-03 14:44:16 +02:00
Dustin Blackman
b53bd9f112
Bump waybackpy to 3.0.6 (#281)
* fix wayback

* fix tests

* Reuse user agent from website loader

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-07-03 06:26:16 +02:00
wahlm
0829d00e5f
no duplication of imported tags (#289)
* no duplication of imported tags (#287)

* Add importer test

* Revert settings test

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-07-03 05:34:40 +02:00
Sascha Ißbrücker
e08bf9fd03
Fake request headers to reduce bot detection (#263)
Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-05-21 13:25:32 +02:00
Sascha Ißbrücker
f4e3d724f0
Improve import performance (#261)
* Run import in batches, cache tags

* Use bulk operations for bookmarks and assigning tags

* Improve naming

* Restore bookmark validation

* Add logging

* Bulk create tags

* Use HTMLParser for parsing bookmarks

* add parser tests

* Add more importer tests

* Add more importer tests

* Remove pyparsing dependency

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-05-21 09:27:30 +02:00
Sascha Ißbrücker
f92c3dd403
Make Internet Archive integration opt-in (#250)
* Make web archive integration opt-in

* Add toast message about web archive integration opt-in

* Improve wording for web archive setting

* Add toast admin

* Fix toast clear button visited styles

* Add test for redirect

* Improve wording

* Ensure redirects to same domain

* Improve wording

* Fix snapshot test

Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@gmail.com>
2022-05-14 09:46:51 +02:00
Sascha Ißbrücker
82b4268a26
Ensure tag names don't contain spaces (#184) 2021-12-12 22:54:22 +01:00
Sascha Ißbrücker
d87dde6bae
Create snapshots on web.archive.org for bookmarks (#150)
* Implement initial background tasks concept

* fix property reference

* update requirements.txt

* simplify bookmark null check

* improve web archive url display

* add background tasks test

* add basic supervisor setup

* schedule missing snapshot creation on login

* remove task locks and clear task history before starting background task processor

* batch create snapshots after import

* fix script reference in supervisord.conf

* add option to disable background tasks

* restructure feature overview
2021-09-04 22:31:04 +02:00
Sascha Ißbrücker
e47c00bd07
Add support for micro-, nanosecond timestamps in importer (#151) 2021-08-26 12:33:54 +02:00
Taku Izumi
937858cf58
Fix website scraper decoding content incorrectly (#126)
* Avoid stall on web scraping

This patch fixes stall on web scraping.
I encountered a stall (scraping never ends) when adding
a bookmark of some site.
To avoid this case, adding a timeout parameter at requests.get()
function is a solution.

Signed-off-by: Taku Izumi <admin@orz-style.com>

* Avoid character corruption of scraping some Japanese sites

This patch fixes character corruption of scraping some Japanese
sites. To avoid character corruption, I use r.content instead
of r.text in load_page function.

The reason of character corruption is encoding problem, I think.
r.text handles data as unicode encoded text, so if scraping
web site's charset is not unicode encoded, character corruption
occurs. r.content handles data as str[], we can avoid encoding
problem.

Signed-off-by: Taku Izumi <admin@orz-style.com>

* use charset_normalizer to determine response encoding

Co-authored-by: Taku Izumi <admin@orz-style.com>
Co-authored-by: Sascha Ißbrücker <sascha.issbruecker@googlemail.com>
2021-08-25 10:16:23 +02:00
Sascha Ißbrücker
8047ba6c63
Fix importer not validating bookmark models (#149) 2021-08-25 09:20:01 +02:00
Sascha Ißbrücker
3e48b22095 Add settings view tests 2021-05-14 23:34:53 +02:00
Sascha Ißbrücker
0c1c21c8d1
Implement bulk edit (#101) 2021-03-29 00:43:50 +02:00
Sascha Ißbrücker
c61e8ee2cd
Implement archive feature (#73)
* Implement archive function (#46)

* Implement archive view (#46)

* Filter tags for archived/unarchived (#46)

* Implement archived bookmarks endpoint (#46)

* Implement archive mode for search component (#46)

* Move bookmarklet to settings (#46)

* Update modified timestamp on archive/unarchive (#46)

* Fix bookmarklet (#46)
2021-02-14 18:00:22 +01:00
Sascha Ißbrücker
70953a52b9
Fix duplicate tag error (#65) 2021-01-12 22:42:56 +01:00
Sascha Ißbrücker
9df270557f
Make tag search and assignment case insensitive (#56)
* Make tag assignment and search case-insensitive (#45)

* Add tests for tag case-sensitivity and deduplication (#45)

Co-authored-by: Sascha Ißbrücker <sissbruecker@lyska.io>
2021-01-02 11:30:20 +01:00
Sascha Ißbrücker
779de41b65
Implement custom netscape file parser (#51)
* Implement custom Netscape file parser (#50)

* Add environment variable to configure request timeouts (#50)

Co-authored-by: Sascha Ißbrücker <sissbruecker@lyska.io>
2020-12-31 07:02:28 +01:00
Sascha Ißbrücker
477fed284d
#47 Fix description import (#48)
Co-authored-by: Sascha Ißbrücker <sissbruecker@lyska.io>
2020-12-29 13:14:10 +01:00
Sascha Ißbrücker
e497bcb5c0
#24 Implement REST API (#32)
* #24 Implement readonly bookmark API

* #24 Implement create/update bookmark API

* #24 Fix title, description not allowing blank values

* #24 Code cleanup

* #24 Add modification dates to response

* #24 Add API docs

* #24 Implement delete bookmark API

* #24 Fix API docs link

* #24 Fix API docs link

* #24 Implement tag API

Co-authored-by: Sascha Ißbrücker <sissbruecker@lyska.io>
2020-09-27 09:34:56 +02:00
Sascha Ißbrücker
348a536aa3
#23 Prevent bookmark duplicates
* Show hint if URL is already bookmarked

* Remove hint if URL belongs to edited bookmark

* Fix query param encoding

* Update bookmark instead of duplicating it

Co-authored-by: Sascha Ißbrücker <sissbruecker@lyska.io>
2020-09-13 08:46:07 +02:00
Sascha Ißbrücker
10fd3d89be Add error handling and logging for importer 2020-06-07 14:15:44 +02:00
Arpit
9275baabd2 Use system timezone for bookmark's "Added date" 2020-06-06 23:05:57 +05:30
Arpit
81d1fd1e6a Use a default value if attribute missing from tag 2020-06-06 23:05:42 +05:30
Sascha Ißbrücker
991025d13e Implement bookmark export 2019-12-26 13:45:12 +01:00
Sascha Ißbrücker
d57ea27ba3 Create docker image 2019-07-03 17:18:29 +02:00
Sascha Ißbrücker
d2f896966a Cleanup 2019-07-02 02:05:09 +02:00
Sascha Ißbrücker
e07da529f1 Preview website title + description in bookmark form
Fix unnecessary selects when rendering bookmarks
2019-07-02 01:28:02 +02:00
Sascha Ißbrücker
0e872c754b Edit bookmark tags 2019-07-01 22:05:38 +02:00
Sascha Ißbrücker
4a16e4d64c Implement tag model 2019-06-30 07:15:46 +02:00
Sascha Ißbrücker
9eefd479c6 Implement basic importer 2019-06-29 08:42:54 +02:00
Sascha Ißbrücker
451a049d46 Implement fetching website metadata 2019-06-29 02:01:26 +02:00
Sascha Ißbrücker
37ee00077a Implement edit bookmark route 2019-06-29 00:27:20 +02:00
Sascha Ißbrücker
c653206dd3 Implement add bookmark route 2019-06-28 19:37:41 +02:00