Commit graph

943 commits

Author SHA1 Message Date
kim
abe9447d28
fix cache startup (#1414)
Signed-off-by: kim <grufwub@gmail.com>
2023-02-03 20:16:11 +00:00
kim
33aee1b1e9
[chore] reformat GetAccount() functionality, support updating accounts based on last_fetch (#1411)
* reformat GetAccount() functionality, and add UpdateAccount() function.

* use fetched_at instead of last_webfingered_at

* catch local "not found" errors. small formatting / error string changes

* remove now unused error type

* return nil when wrapping nil error

* update expected error messages

* return correct url for foss satan webfinger

* add AP model for Some_User

* normalize local domain

* return notretrievable where appropriate

* expose NewErrNotRetrievable

* ensure webfinger for new accounts searched by uri

* update local account short circuit

* allow enrich to fail for already-known accounts

* remove unused LastWebfingeredAt

* expose test maps on mock http client

* update Update test

* reformat GetAccount() functionality, and add UpdateAccount() function.

* use fetched_at instead of last_webfingered_at

* catch local "not found" errors. small formatting / error string changes

* remove nil error checks (we shouldn't be passing nil errors to newError() initializers)

* remove mutex unlock on transport init fail (it hasn't yet been locked!)

* woops add back the error wrapping to use ErrNotRetrievable

* caches were never being started... 🙈

---------

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-02-03 20:03:05 +00:00
f0x52
a59dc855d9
[feature/frogend] (Mastodon) domain block CSV import (#1390)
* checkbox-list styling with taller <p> element

* CSV import/export, UI/UX improvements to import-export interface

* minor styling tweaks

* csv export, clean up export type branching

* abstract domain block entry validation

* foundation for PSL check + suggestions

* Squashed commit of the following:

commit e3655ba4fb
Author: f0x <f0x@cthu.lu>
Date:   Tue Jan 31 15:19:10 2023 +0100

    let debug depend on env (prod/debug) again

commit 79c792b832
Author: f0x <f0x@cthu.lu>
Date:   Tue Jan 31 00:34:01 2023 +0100

    update checklist components

commit 4367960fe4
Author: f0x <f0x@cthu.lu>
Date:   Mon Jan 30 23:46:20 2023 +0100

    checklist performance improvements

commit 204a4c02d1
Author: f0x <f0x@cthu.lu>
Date:   Mon Jan 30 20:05:34 2023 +0100

    checklist field: use reducer for state

* remove debug logging

* show and use domain block suggestion

* restructure import/export buttons

* updating suggestions

* suggestion overview

* restructure check-list behavior, domain import/export
2023-02-03 12:07:40 +01:00
Autumn!
49beb17a8f
[chore] Text formatting overhaul (#1406)
* Implement goldmark debug print for hashtags and mentions

* Minify HTML in FromPlain

* Convert plaintext status parser to goldmark

* Move mention/tag/emoji finding logic into formatter

* Combine mention and hashtag boundary characters

* Normalize unicode when rendering hashtags
2023-02-03 11:58:58 +01:00
tobi
271da016b9
[bugfix] Read Bookwyrm Articles more thoroughly (#1410) 2023-02-02 16:41:02 +01:00
tobi
382512a5a6
[feature] Implement /api/v2/instance endpoint (#1409)
* interim: start adding /api/v2/instance

* finish up
2023-02-02 14:08:13 +01:00
tobi
4ee4cd2da1
[chore/performance] use only 1 sqlite db connection regardless of multiplier (#1408) 2023-02-01 11:55:34 +01:00
tobi
b80be48fed
[chore] Use 'immediate' lock for sqlite transactions (#1404)
* [chore] Use 'immediate' lock for sqlite transactions

* allow 1 connection regardless of cpu amount
2023-01-31 13:46:45 +01:00
Daniele Sluijters
eccb3800d0
[chore] Silence maxprocs logging (#1402)
This switches the logger to Debug. By default the library logs things
internally despite returning errors. Due to the way we had it hooked up
this resulted in scary error level logs being emitted for innocuous
errors. Now those errors will only be visible in debug mode.

Fixes: #1398
Relates to: #1336
2023-01-31 09:39:46 +01:00
ketan-vijayvargiya
dc766f9e6f
[docs] Add an example on how to setup redirect with Traefik (#1395) 2023-01-30 10:47:03 +01:00
dependabot[bot]
356e238793
[chore]: Bump github.com/go-playground/validator/v10 (#1400)
Bumps [github.com/go-playground/validator/v10](https://github.com/go-playground/validator) from 10.11.1 to 10.11.2.
- [Release notes](https://github.com/go-playground/validator/releases)
- [Commits](https://github.com/go-playground/validator/compare/v10.11.1...v10.11.2)

---
updated-dependencies:
- dependency-name: github.com/go-playground/validator/v10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-30 10:45:34 +01:00
dependabot[bot]
7bcdf35cc1
[chore]: Bump github.com/microcosm-cc/bluemonday from 1.0.21 to 1.0.22 (#1399)
Bumps [github.com/microcosm-cc/bluemonday](https://github.com/microcosm-cc/bluemonday) from 1.0.21 to 1.0.22.
- [Release notes](https://github.com/microcosm-cc/bluemonday/releases)
- [Commits](https://github.com/microcosm-cc/bluemonday/compare/v1.0.21...v1.0.22)

---
updated-dependencies:
- dependency-name: github.com/microcosm-cc/bluemonday
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-30 10:45:19 +01:00
Mina Galić
43cbe3b45c
[docs] Simplify Apache httpd proxy documentation (#1396)
since 2.4.47 (released April 22nd 2021), Apache httpd can ProxyPass to
websockets on the same URL, without mod_rewrite (and, without
mod_proxy_wstunnel).
2023-01-30 10:45:01 +01:00
tobi
3283900b0d
[feature] Federate reports to remote instance as Flag (if desired) (#1386)
* reports federate out, we did it lxds

* fix optional line start (should be optional slash)
2023-01-27 14:48:11 +01:00
tobi
c59ec6f2a4
[docs] Add Flag documentation to federation docs (#1393) 2023-01-27 14:39:28 +01:00
tobi
1fa574f148
[docs] Tidy up federation docs into 'federating with gotosocial' section (#1392) 2023-01-27 12:32:23 +01:00
tobi
8d18888db6
[chore/docs] add instance-expose-suspended-web to instance docs (#1391) 2023-01-27 09:23:54 +01:00
f0x52
08f8feaec5
[feature/frontend] filterable local emoji list (#1385) 2023-01-27 09:09:26 +01:00
tobi
782169da76
[chore] set max open / idle conns + conn max lifetime for both postgres and sqlite (#1369)
* [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite

* reduce cache size default to 8MiB, reduce connections to 2 * cpu

* introduce max open conns multiplier, tune sqlite and pg separately

* go fmt
2023-01-26 15:12:48 +01:00
f0x52
17eecfb6d9
[feature] Public list of suspended domains (#1362)
* basic rendered domain blocklist (unauthenticated!)

* style basic domain block list

* better formatting for domain blocklist

* add opt-in config option for showing suspended domains

* format/linter

* re-use InstancePeersGet for web-accessible domain blocklist

* reword explanation, border styling

* always attach blocklist handler, update error message

* domain blocklist error message grammar
2023-01-25 18:06:41 +01:00
tobi
993aae5e48
[feature] Accept incoming federated Flag activity (#1382)
* start working on handling incoming Flag activity

* interim commit

* federate Flag in successfully
2023-01-25 11:12:27 +01:00
tobi
faeb7ded3b
[feature] Implement reports admin API so admins can view + close reports (#1378)
* add admin report api endpoints + tests

* [chore] remove funky duplicate attachment in testrig
2023-01-25 11:12:17 +01:00
f0x52
27d4e364e0
[chore] Settings refactor fix4 (#1383)
* fix error handling behavior in emoji overview and FormWithData components

* css: long domain cutoff

* unused require

* eslint vscode task
2023-01-25 09:47:55 +01:00
tobi
36f62d6e60
[chore] remove funky duplicate attachment in testrig (#1379)
* [chore] remove funky duplicate attachment in testrig

* whoops
2023-01-23 13:39:01 +01:00
tobi
e9747247d5
[feature] Implement /api/v1/reports endpoints on client API (#1330)
* start adding report client api

* route + test reports get

* start report create endpoint

* you can create reports now babyy

* stub account report processor

* add single reportGet endpoint

* fix test

* add more filtering params to /api/v1/reports GET

* update swagger

* use marshalIndent in tests

* add + test missing Link info
2023-01-23 13:14:21 +01:00
tobi
605dfca1af
[chore] bump go version to 1.19.5 (#1377) 2023-01-23 11:58:58 +01:00
dependabot[bot]
98a09b5633
[chore]: Bump github.com/spf13/viper from 1.14.0 to 1.15.0 (#1375)
Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/spf13/viper/releases)
- [Commits](https://github.com/spf13/viper/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: github.com/spf13/viper
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 10:24:00 +01:00
dependabot[bot]
3e4dc6bff3
[chore]: Bump github.com/abema/go-mp4 from 0.9.0 to 0.10.0 (#1374)
Bumps [github.com/abema/go-mp4](https://github.com/abema/go-mp4) from 0.9.0 to 0.10.0.
- [Release notes](https://github.com/abema/go-mp4/releases)
- [Commits](https://github.com/abema/go-mp4/compare/v0.9.0...v0.10.0)

---
updated-dependencies:
- dependency-name: github.com/abema/go-mp4
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-23 10:23:29 +01:00
Kelson Vibber
73be24401a
[feature] Add RSS autodiscovery on profiles that enable RSS (#1373) 2023-01-23 10:23:07 +01:00
즈눅
13ec15d2bb
[chore] extending maximumPasswordLength to 256 (#1372)
* [chore] extending maximumPasswordLength to 256

Resolves: #1365

* Make long test longer
2023-01-22 23:25:11 +01:00
tobi
0ceacd7b1d
[chore] bump db dependencies (#1366) 2023-01-22 12:26:47 +01:00
f0x52
b375d3b5d6
[chore] Add name to instance field for autosuggestion (#1359) 2023-01-19 17:37:42 +01:00
f0x52
747683ba54
[chore] Settings refactor fix 2 (#1357)
* fix emoji query tagging

* fix proxy url for gts instance

* fix: don't flash callback error on authorize flow
2023-01-18 16:41:00 +01:00
f0x52
13e3aaaed1
[chore] Fix new emoji preview title/alt text (#1354)
* fix new emoji preview title/alt text

* empty commit to start Drone pr build
2023-01-18 15:16:29 +01:00
f0x52
9b139b6320
[frogend] Settings refactor (#1318)
* yakshave new form field structure

* fully refactor user profile settings form

* use rtk query api for profile settings

* refactor user post settings

* refactor password change form

* refactor admin settings

* FormWithData structure for user forms

* admin actions refactor

* whitespace

* fix user settings data prop

* remove superfluous logging

* cleanup old code

* refactor federation/suspend (overview, detail)

* mostly abstracted (emoji) checkbox list

* refactor parse-from-toot

* refactor custom-emoji, progress on federation bulk

* loading icon styling to prevent big spinny

* refactor federation import-export interface

* cleanup old files

* [chore] Update/add license headers for 2023

* redux fixes

* text-field exports

* appease the linter

* refactor authentication with RTK Query

* fix login/logout state transition weirdness

* fixes/cleanup

* small linter-related fixes

* add eslint license header check, fix existing files

* remove old code, clarify comment

* clarify suspend on subdomains

* collapse if/else

* fa-fw width info comment
2023-01-18 14:45:14 +01:00
tobi
974ec80a20
[chore] Change default sqlite busy timeout to 5m (#1352) 2023-01-18 14:37:52 +01:00
Daniele Sluijters
acc333c40b
[feature] Inherit resource limits from cgroups (#1336)
When GTS is running in a container runtime which has configured CPU or
memory limits or under an init system that uses cgroups to impose CPU
and memory limits the values the Go runtime sees for GOMAXPROCS and
GOMEMLIMIT are still based on the host resources, not the cgroup.

At least for the throttling middlewares which use GOMAXPROCS to
configure their queue size, this can result in GTS running with values
too big compared to the resources that will actuall be available to it.

This introduces 2 dependencies which can pick up resource contraints
from the current cgroup and tune the Go runtime accordingly. This should
result in the different queues being appropriately sized and in general
more predictable performance. These dependencies are a no-op on
non-Linux systems or if running in a cgroup that doesn't set a limit on
CPU or memory.

The automatic tuning of GOMEMLIMIT can be disabled by either explicitly
setting GOMEMLIMIT yourself or by setting AUTOMEMLIMIT=off. The
automatic tuning of GOMAXPROCS can similarly be counteracted by setting
GOMAXPROCS yourself.
2023-01-17 20:59:04 +00:00
tobi
627b8eeae6
[feature] Tune sqlite pragmas (#1349)
* sqlite pragma tuning

* use formatuint

* add sqlite busy timeout

* fix incorrect cache size format

* update envparsing test

* add sqlite tuning flags to cli

* set sqlite timeout to 30s default
2023-01-17 12:29:44 +00:00
dependabot[bot]
a6c6bdb34a
[chore]: Bump codeberg.org/gruf/go-errors/v2 from 2.0.2 to 2.1.1 (#1346)
Bumps codeberg.org/gruf/go-errors/v2 from 2.0.2 to 2.1.1.

---
updated-dependencies:
- dependency-name: codeberg.org/gruf/go-errors/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-17 11:25:13 +00:00
tobi
d4cddf460a
[bugfix] Parse video metadata more accurately; allow Range in fileserver (#1342)
* don't serve unused fields for video attachments

* parse video bitrate + duration more accurately

* use ServeContent where appropriate to respect Range

* abstract temp file seeker into its own function
2023-01-16 16:19:17 +01:00
dependabot[bot]
fe3e9ede52
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.44 to 7.0.47 (#1348)
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go) from 7.0.44 to 7.0.47.
- [Release notes](https://github.com/minio/minio-go/releases)
- [Commits](https://github.com/minio/minio-go/compare/v7.0.44...v7.0.47)

---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 10:29:47 +01:00
dependabot[bot]
2a46980082
[chore]: Bump golang.org/x/oauth2 from 0.3.0 to 0.4.0 (#1347)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.3.0 to 0.4.0.
- [Release notes](https://github.com/golang/oauth2/releases)
- [Commits](https://github.com/golang/oauth2/compare/v0.3.0...v0.4.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-16 10:29:19 +01:00
Phil Hagelberg
6b15b83bf1
[docs] Remove videos from the list of missing features in the FAQ. (#1344)
These actually work now! Yay.
2023-01-16 10:28:18 +01:00
tobi
132c73883f
[bugfix] Mount bookmarks endpoint correctly (#1338) 2023-01-13 17:07:52 +01:00
tobi
eafd73c292
[chore] Remove omitempty on account source; refactor tests to use prettyprint json (#1337)
* remove omitEmpty tag on account source items

* update tests
2023-01-13 17:02:45 +01:00
kim
5318054808
[performance] media processing improvements (#1288)
* media processor consolidation and reformatting, reduce amount of required syscalls

Signed-off-by: kim <grufwub@gmail.com>

* update go-store library, stream jpeg/png encoding + use buffer pools, improved media processing AlreadyExists error handling

Signed-off-by: kim <grufwub@gmail.com>

* fix duration not being set, fix mp4 test expecting error

Signed-off-by: kim <grufwub@gmail.com>

* fix test expecting media files with different extension

Signed-off-by: kim <grufwub@gmail.com>

* remove unused code

Signed-off-by: kim <grufwub@gmail.com>

* fix expected storage paths in tests, update expected test thumbnails

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* fix cached presigned s3 url fetching

Signed-off-by: kim <grufwub@gmail.com>

* fix tests

Signed-off-by: kim <grufwub@gmail.com>

* fix test models

Signed-off-by: kim <grufwub@gmail.com>

* update media processing to use sync.Once{} for concurrency protection

Signed-off-by: kim <grufwub@gmail.com>

* shutup linter

Signed-off-by: kim <grufwub@gmail.com>

* fix passing in KVStore GetStream() as stream to PutStream()

Signed-off-by: kim <grufwub@gmail.com>

* fix unlocks of storage keys

Signed-off-by: kim <grufwub@gmail.com>

* whoops, return the error...

Signed-off-by: kim <grufwub@gmail.com>

* pour one out for tobi's code <3

Signed-off-by: kim <grufwub@gmail.com>

* add back the byte slurping code

Signed-off-by: kim <grufwub@gmail.com>

* check for both ErrUnexpectedEOF and EOF

Signed-off-by: kim <grufwub@gmail.com>

* add back links to file format header information

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2023-01-11 12:13:13 +01:00
Sleep
3512325e46
[feature] Add local user and post count to nodeinfo responses (#1325)
* Add local user and post count to nodeinfo responses

This fixes #1307 (at least partially). The nodeinfo endpoint should now
return the total users on an instance, along with their post count.

* Update NodeInfoUsers docstring and swagger yaml file
2023-01-11 11:36:36 +01:00
tobi
d6487933c7
[feature] Implement Report database model and utility functions (#1310)
* implement report database model

* implement report cache + config changes

* implement report database functions

* report uri / regex functions

* update envparsing test

* remove unnecessary uri index

* remove unused function + cache lookup

* process error when storing report
2023-01-10 14:19:05 +00:00
dependabot[bot]
36aa6854bd
[chore] Bump json5 from 1.0.1 to 1.0.2 in /web/source (#1308)
Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-09 10:24:49 +01:00
dependabot[bot]
ac562fa79e
[chore]: Bump github.com/coreos/go-oidc/v3 from 3.4.0 to 3.5.0 (#1322)
Bumps [github.com/coreos/go-oidc/v3](https://github.com/coreos/go-oidc) from 3.4.0 to 3.5.0.
- [Release notes](https://github.com/coreos/go-oidc/releases)
- [Commits](https://github.com/coreos/go-oidc/compare/v3.4.0...v3.5.0)

---
updated-dependencies:
- dependency-name: github.com/coreos/go-oidc/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-01-09 08:23:49 +00:00