Commit graph

106 commits

Author SHA1 Message Date
tobi
efd1a4f717
[bugfix] Use better plaintext representation of status for filtering (#3301)
* [bugfix] Use better plaintext representation of status for filtering

* add new deps to readme

* lint

* update tests

* update regexes

* address review comments

* remove now unused xxhash

* whoops, wrong logger

* Merge branch 'main' into status_filtering_bugfix

* put cache in caches struct

* pain
2024-09-16 14:00:23 +02:00
kim
d842069985
[docs/chore] update list of library deps on readme (#3288) 2024-09-11 10:59:55 +02:00
tobi
ad700c5029
[chore/frontend] Use smaller webp images for logo and default avatars (#3119) 2024-07-20 14:02:22 +01:00
kim
f5a44ccad4
[docs] update README with our corporate sponsorship policy (#3112)
* update README with our corporate sponsorship policy

* move location of corporate sponsorship

* tweaks after discussion

* update to maintain consistency
2024-07-19 14:14:07 +02:00
kim
cde2fb6244
[feature] support processing of (many) more media types (#3090)
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg

* specify the video codec to use when generating static image from emoji

* update go-storage library (fixes incompatibility after updating go-iotools)

* maintain image aspect ratio when generating a thumbnail for it

* update readme to show go-ffmpreg

* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility

* remove extra debug from error message

* fix up incorrect function signatures

* update PutFile to just use regular file copy, as changes are file is on separate partition

* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe

* update more tests, add more code comments

* add utilities to generate processed emoji / media outputs

* fix remaining tests

* add test for opus media file, add license header to utility cmds

* limit the number of concurrently available ffmpeg / ffprobe instances

* reduce number of instances

* further reduce number of instances

* fix envparsing test with configuration variables

* update docs and configuration with new media-{local,remote}-max-size variables
2024-07-12 09:39:47 +00:00
tobi
ee6e9b2795
[docs] Rework README a bit, import into docs (#3006) 2024-06-13 23:21:41 +02:00
kim
3d3e99ae52
[performance] update storage backend and make use of seek syscall when available (#2924)
* update to use go-storage/ instead of go-store/v2/storage/

* pull in latest version from codeberg

* remove test output 😇

* add code comments

* set the exclusive bit when creating new files in disk config

* bump to actual release version

* bump to v0.1.1 (tis a simple no-logic change)

* update readme

* only use a temporary read seeker when decoding video if required (should only be S3 now)

* use fastcopy library to use memory pooled buffers when calling TempFileSeeker()

* update to use seek call in serveFileRange()
2024-05-22 11:46:24 +02:00
Daenney
39b3a27c82
[docs] Remove last references to RPi (#2885)
This updates the documentation to remove the last stray references to
the copaganda Pi. It now uses the the term single-board computer. GtS
can run fine on all kinds of SBCs and isn't limited to that one
particular fruit version.
2024-04-30 14:27:44 +01:00
kim
c9c0773f2c
[performance] update remaining worker pools to use queues (#2865)
* start replacing client + federator + media workers with new worker + queue types

* refactor federatingDB.Delete(), drop queued messages when deleting account / status

* move all queue purging to the processor workers

* undo toolchain updates

* code comments, ensure dereferencer worker pool gets started

* update gruf libraries in readme

* start the job scheduler separately to the worker pools

* reshuffle ordering or server.go + remove duplicate worker start / stop

* update go-list version

* fix vendoring

* move queue invalidation to before wipeing / deletion, to ensure queued work not dropped

* add logging to worker processing functions in testrig, don't start workers in unexpected places

* update go-structr to add (+then rely on) QueueCtx{} type

* ensure more worker pools get started properly in tests

* fix remaining broken tests relying on worker queue logic

* fix account test suite queue popping logic, ensure noop workers do not pull from queue

* move back accidentally shuffled account deletion order

* ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete()

* silently drop deletes from accounts not permitted to

* don't warn log on forwarded deletes

* make if else clauses easier to parse

* use getFederatorMsg()

* improved code comment

* improved code comment re: requesting account delete checks

* remove boolean result from worker start / stop since false = already running or already stopped

* remove optional passed-in http.client

* remove worker starting from the admin CLI commands (we don't need to handle side-effects)

* update prune cli to start scheduler but not all of the workers

* fix rebase issues

* remove redundant return statements

* i'm sorry sir linter
2024-04-26 13:50:46 +01:00
kim
adf345f1ec
[chore] bump go structr cache version -> v0.6.0 (#2773)
* update go-structr library -> v0.6.0, add necessary wrapping types + code changes to support these changes

* update readme with go-structr package changes

* improved wrapping of the SliceCache type

* add code comments for the cache wrapper types

* remove test.out 😇

---------

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2024-04-02 12:03:40 +02:00
tobi
b614d33c40
[feature] Try HTTP signature validation with and without query params for incoming requests (#2591)
* [feature] Verify signatures both with + without query params

* Bump to tagged version
2024-01-31 14:15:28 +00:00
kim
07207e71e9
[performance] cache library performance enhancements (updates go-structr => v0.2.0) (#2575)
* update go-structr => v0.2.0

* update readme

* whoops, fix the link
2024-01-26 12:14:10 +00:00
Vyr Cossont
726b599a1e
[docs] Update Feditext URL (#2568)
Fixes #2567
2024-01-25 15:29:57 +01:00
tobi
60d7060895
[docs] use latest cavage link for http signatures (#2565) 2024-01-22 15:29:01 +01:00
tobi
ebf550b7c1
[chore] Move to codeberg's exif-terminator (#2536) 2024-01-15 20:54:32 +00:00
tobi
9607b482cc
[chore/docs] Replace specific year range of copyright notice (#2520) 2024-01-13 16:33:53 +01:00
tobi
21a101ebc4
[docs] Update README.md (#2279) 2023-10-19 12:57:37 +02:00
tobi
183eaa5b29
[feature] Implement explicit domain allows + allowlist federation mode (#2200)
* love like winter! wohoah, wohoah

* domain allow side effects

* tests! logging! unallow!

* document federation modes

* linty linterson

* test

* further adventures in documentation

* finish up domain block documentation (i think)

* change wording a wee little bit

* docs, example

* consolidate shared domainPermission code

* call mode once

* fetch federation mode within domain blocked func

* read domain perm import in streaming manner

* don't use pointer to slice for domain perms

* don't bother copying blocks + allows before deleting

* admonish!

* change wording just a scooch

* update docs
2023-09-21 12:12:04 +02:00
tobi
42bb352d53
[feature] Add snapshot binary builds + uploads (#2119)
* [feature] Add snapshot binary builds + uploads

* Update docs to include info on snapshot builds

* review comments

* little tweaks
2023-08-15 18:48:17 +02:00
Daenney
9df4d38c43
[chore] Add Feditext as recommended client (#2081)
With Feditext now accepting beta users, this adds it as the third
client to recommend so we have web and the dominant mobile platforms
covered.

This also removes the screenshots from the README, because it became a
mess trying to add a third one. Either the cells become very narrow, or
the table doubles in height. As the UI may also change over time, it
might be better to point folks at the apps instead who'll hopefully have
up to date screenshots in their storefronts.
2023-08-08 12:19:41 +02:00
kim
00adf18c24
[feature] simpler cache size configuration (#2051)
* add automatic cache max size generation based on ratios of a singular fixed memory target

Signed-off-by: kim <grufwub@gmail.com>

* remove now-unused cache max-size config variables

Signed-off-by: kim <grufwub@gmail.com>

* slight ratio tweak

Signed-off-by: kim <grufwub@gmail.com>

* remove unused visibility config var

Signed-off-by: kim <grufwub@gmail.com>

* add secret little ratio config trick

Signed-off-by: kim <grufwub@gmail.com>

* fixed a word

Signed-off-by: kim <grufwub@gmail.com>

* update cache library to remove use of TTL in result caches + slice cache

Signed-off-by: kim <grufwub@gmail.com>

* update other cache usages to use correct interface

Signed-off-by: kim <grufwub@gmail.com>

* update example config to explain the cache memory target

Signed-off-by: kim <grufwub@gmail.com>

* update env parsing test with new config values

Signed-off-by: kim <grufwub@gmail.com>

* do some ratio twiddling

Signed-off-by: kim <grufwub@gmail.com>

* add missing header

* update envparsing with latest defaults

Signed-off-by: kim <grufwub@gmail.com>

* update size calculations to take into account result cache, simple cache and extra map overheads

Signed-off-by: kim <grufwub@gmail.com>

* tweak the ratios some more

Signed-off-by: kim <grufwub@gmail.com>

* more nan rampaging

Signed-off-by: kim <grufwub@gmail.com>

* fix envparsing script

Signed-off-by: kim <grufwub@gmail.com>

* update cache library, add sweep function to keep caches trim

Signed-off-by: kim <grufwub@gmail.com>

* sweep caches once a minute

Signed-off-by: kim <grufwub@gmail.com>

* add a regular job to sweep caches and keep under 80% utilisation

Signed-off-by: kim <grufwub@gmail.com>

* remove dead code

Signed-off-by: kim <grufwub@gmail.com>

* add new size library used to libraries section of readme

Signed-off-by: kim <grufwub@gmail.com>

* add better explanations for the mem-ratio numbers

Signed-off-by: kim <grufwub@gmail.com>

* update go-cache

Signed-off-by: kim <grufwub@gmail.com>

* library version bump

Signed-off-by: kim <grufwub@gmail.com>

* update cache.result{} size model estimation

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-08-03 11:34:35 +02:00
tobi
b401bd1ccb
[chore] update latest deps, ensure readme up to date (#1873)
* [chore] update latest deps, ensure readme up to date

* remove double entry
2023-06-05 10:15:05 +02:00
Daenney
186e849dbf
[docs] Add Repology stats (#1859)
Instead of a manually curated list of distribution packages, this adds
the Repology[1] widget instead.

This also separates out the distribution packages from the
self-hosting/deployment options into their own sections.

[1]: https://repology.org/project/gotosocial/versions
2023-06-04 11:40:32 +02:00
tobi
36fcd2e604
[docs] clarify other federation modes not yet implemented (#1849) 2023-05-31 10:35:55 +02:00
tobi
1f06914007
[docs] Update + simplify roadmap, revise beta estimate (#1826) 2023-05-26 18:00:30 +02:00
Julian-Samuel Gebühr
9c24dee01f
[chore] Replace pinafore with semaphore (#1801)
* Replace pinafore with semaphore

* Typo
2023-05-21 22:40:43 +02:00
0hlov3
ae9d432f13
[docs] Migrates Chart Location (#1708) 2023-04-24 13:49:44 +02:00
tobi
1603a7fd48
[chore] License sloth logo + default avatars under CC by-sa 4.0 (#1651) 2023-03-27 15:36:08 +02:00
kim
2af33d3fb5
Update README.md (#1533) 2023-02-19 16:10:40 +00:00
kim
83d8a44020
[chore] update gruf's libs on readme (#1520)
also groups them into a single block to improve readability
2023-02-18 16:45:43 +01:00
tobi
98edd75f1b
[docs] Rewrite sponsorship + funding section, add NLnet (#1305)
* [docs] Rewrite sponsorship + funding section, add NLnet

* use just one small nlnet logo
2023-01-08 12:37:10 +01:00
tobi
0dbe6c514f
[chore] Update/add license headers for 2023 (#1304) 2023-01-05 12:43:00 +01:00
tobi
2bbc64be43
[feature] Enable basic video support (mp4 only) (#1274)
* [feature] basic video support

* fix missing semicolon

* replace text shadow with stacked icons

Co-authored-by: f0x <f0x@cthu.lu>
2022-12-17 04:38:56 +00:00
tobi
610c2708ca
[docs] Update CONTRIBUTING.md, add pull request templates (#1216)
* [docs] Update contributing.md

- Add Pull Request process and guidelines.
- Add feature/bug issue process.
- Rearrange some sections for clarity.
- Add overview of package structure.

* [docs] Add build from source links

* [chore] add pull request templates

These link to the new CONTRIBUTING.md document, and include a checklist to validate that contributors have read the guidelines.

* [docs] Put existing stub CoC in separate doc

* update web related stuff in CONTRIBUTING.md

Co-authored-by: f0x <f0x@cthu.lu>
2022-12-09 11:53:58 +01:00
Marie Axelsson
aea16bbf3c
[docs] Update README.md (#1126)
added myself to the team, tyvm.
2022-12-07 21:51:01 +01:00
Phil Hagelberg
b153808472
[docs] Add first draft of FAQ (#1086)
* Clarify that gotosocial doesn't use recommendation algorithms

It uses lots of algorithms.

    ~/src/gotosocial $ rgrep algorithm . | wc
        856    8971  143395

* Add first draft of the FAQ.

* Suggested FAQ changes.

Mention backfill of posts as not yet being implemented; link to FAQ
from readme/index.

Mention that GtS does serve up some UI; just not most interaction.
2022-11-21 13:19:51 +01:00
kim
0c2971a567
[chore] update readme with kim's links (#1093)
* add kim's code and fediverse profiles

Signed-off-by: kim <grufwub@gmail.com>

* fix words

Signed-off-by: kim <grufwub@gmail.com>

* undo my accidental inclusion of custom version string...

Signed-off-by: kim <grufwub@gmail.com>

Signed-off-by: kim <grufwub@gmail.com>
2022-11-21 13:18:25 +01:00
tobi
e2e2bedd42
[docs] Use correct Friendica link in README (#1046) 2022-11-14 13:58:17 +01:00
tobi
298a7ad21b
[docs] add note about Alpha status right at the top (#971) 2022-11-06 12:18:52 +01:00
tobi
2311c8e96e
[docs] add matrix space to readme (#952) 2022-11-04 12:58:25 +01:00
tobi
80663061d8
[feature] Add opt-in RSS feed for account's latest Public posts (#897)
* start adding rss functionality

* add gorilla/feeds dependency

* first bash at building rss feed
still needs work, this is an interim commit

* tidy up a bit

* add publicOnly option to GetAccountLastPosted

* implement rss endpoint

* fix test

* add initial user docs for rss

* update rss logo

* docs update

* add rssFeed to frontend

* feed -> feed.rss

* enableRSS

* increase rss logo size a lil bit

* add rss toggle

* move emojify to text package

* fiddle with rss feed formatting

* add Text field to test statuses

* move status to rss item to typeconverter

* update bun schema for enablerss

* simplify 304 checking

* assume account not rss

* update tests

* update swagger docs

* allow more characters in title, trim nicer

* update last posted to be more consistent
2022-10-08 14:00:39 +02:00
kim
1d999712e6
[feature] update config types to use bytesize.Size (#828)
* update config size types to use bytesize.Size

* submit unchecked-out file ... 🤦

* fix bytesize config var decoding

* bump bytesize version

* update kim's libraries in readme

* update envparse.sh to output more useful errors

* improve envparse.sh

* remove reliance on jq

* instead, use uint64 for bytesize flag types

* remove redundant type

* fix viper unmarshaling

* Update envparsing.sh

* fix envparsing test

Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-09-29 21:50:43 +01:00
nya1
bee8458a2d
[feature] add rate limit middleware (#741)
* feat: add rate limit middleware

* chore: update vendor dir

* chore: update readme with new dependency

* chore: add rate limit infos to swagger.md file

* refactor: add ipv6 mask limiter option

Add IPv6 CIDR /64 mask

* refactor: increase rate limit to 1000

Address https://github.com/superseriousbusiness/gotosocial/pull/741#discussion_r945584800

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-08-31 12:06:14 +02:00
Callum Macdonald
47152c2706
[docs] Fix link to open collective (#770) 2022-08-29 11:07:13 +02:00
Martijn de Boer
2ca234f42e
[docs] Textual updates on markdown files (#756)
* A few spelling and grammer fixes on readme

Changes a few wording, some punctuation and grammar.

* Grammar and punctuation on Roadmap

Mostly grammar and punctuation on the roadmap

* Update README.md

Restore British English spelling of favourite, because it's used on the API endpoints in Roadmap as well.

* Grammar and punctuation on Contributing

Mainly grammar and punctuation on Contributing file.
2022-08-18 11:44:17 +02:00
tobi
91c8d5d20d
[bugfix] Fix thumbnails not taking exif rotation into account (#746)
* use disintegration/imaging instead of nfnt/resize

* update tests

* use disintegration lib for thumbing (if necessary)
2022-08-10 14:05:14 +02:00
tobi
879b4abde7
[bugfix] Markdown formatting updates (#743)
* add minify dependency specifically for markdown

* rearrange markdown formatting

* update markdown tests
2022-08-07 18:19:16 +02:00
tobi
136799aefe
[docs] Update sponsorship + funding sections (#721) 2022-07-20 11:14:00 +02:00
tobi
c84384e660
[bugfix] html escape special characters in text instead of totally removing them (#719)
* remove minify dependency

* tidy up some tests

* remove pre + postformat funcs

* rework sanitization + formatting

* update tests

* add some more markdown tests
2022-07-19 15:21:17 +02:00
kim
098dbe6ff4
[chore] use our own logging implementation (#716)
* first commit

Signed-off-by: kim <grufwub@gmail.com>

* replace logging with our own log library

Signed-off-by: kim <grufwub@gmail.com>

* fix imports

Signed-off-by: kim <grufwub@gmail.com>

* fix log imports

Signed-off-by: kim <grufwub@gmail.com>

* add license text

Signed-off-by: kim <grufwub@gmail.com>

* fix package import cycle between config and log package

Signed-off-by: kim <grufwub@gmail.com>

* fix empty kv.Fields{} being passed to WithFields()

Signed-off-by: kim <grufwub@gmail.com>

* fix uses of log.WithFields() with whitespace issues and empty slices

Signed-off-by: kim <grufwub@gmail.com>

* *linter related grumbling*

Signed-off-by: kim <grufwub@gmail.com>

* gofmt the codebase! also fix more log.WithFields() formatting issues

Signed-off-by: kim <grufwub@gmail.com>

* update testrig code to match new changes

Signed-off-by: kim <grufwub@gmail.com>

* fix error wrapping in non fmt.Errorf function

Signed-off-by: kim <grufwub@gmail.com>

* add benchmarking of log.Caller() vs non-cached

Signed-off-by: kim <grufwub@gmail.com>

* fix syslog tests, add standard build tags to test runner to ensure consistency

Signed-off-by: kim <grufwub@gmail.com>

* make syslog tests more robust

Signed-off-by: kim <grufwub@gmail.com>

* fix caller depth arithmatic (is that how you spell it?)

Signed-off-by: kim <grufwub@gmail.com>

* update to use unkeyed fields in kv.Field{} instances

Signed-off-by: kim <grufwub@gmail.com>

* update go-kv library

Signed-off-by: kim <grufwub@gmail.com>

* update libraries list

Signed-off-by: kim <grufwub@gmail.com>

* fuck you linter get nerfed

Signed-off-by: kim <grufwub@gmail.com>

Co-authored-by: tobi <31960611+tsmethurst@users.noreply.github.com>
2022-07-19 10:47:55 +02:00