* reformat GetAccount() functionality, and add UpdateAccount() function.
* use fetched_at instead of last_webfingered_at
* catch local "not found" errors. small formatting / error string changes
* remove now unused error type
* return nil when wrapping nil error
* update expected error messages
* return correct url for foss satan webfinger
* add AP model for Some_User
* normalize local domain
* return notretrievable where appropriate
* expose NewErrNotRetrievable
* ensure webfinger for new accounts searched by uri
* update local account short circuit
* allow enrich to fail for already-known accounts
* remove unused LastWebfingeredAt
* expose test maps on mock http client
* update Update test
* reformat GetAccount() functionality, and add UpdateAccount() function.
* use fetched_at instead of last_webfingered_at
* catch local "not found" errors. small formatting / error string changes
* remove nil error checks (we shouldn't be passing nil errors to newError() initializers)
* remove mutex unlock on transport init fail (it hasn't yet been locked!)
* woops add back the error wrapping to use ErrNotRetrievable
* caches were never being started... 🙈
---------
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
* Implement goldmark debug print for hashtags and mentions
* Minify HTML in FromPlain
* Convert plaintext status parser to goldmark
* Move mention/tag/emoji finding logic into formatter
* Combine mention and hashtag boundary characters
* Normalize unicode when rendering hashtags
* [chore] set max open / idle conns + conn max lifetime for both postgres and sqlite
* reduce cache size default to 8MiB, reduce connections to 2 * cpu
* introduce max open conns multiplier, tune sqlite and pg separately
* go fmt
* start adding report client api
* route + test reports get
* start report create endpoint
* you can create reports now babyy
* stub account report processor
* add single reportGet endpoint
* fix test
* add more filtering params to /api/v1/reports GET
* update swagger
* use marshalIndent in tests
* add + test missing Link info
* don't serve unused fields for video attachments
* parse video bitrate + duration more accurately
* use ServeContent where appropriate to respect Range
* abstract temp file seeker into its own function
* media processor consolidation and reformatting, reduce amount of required syscalls
Signed-off-by: kim <grufwub@gmail.com>
* update go-store library, stream jpeg/png encoding + use buffer pools, improved media processing AlreadyExists error handling
Signed-off-by: kim <grufwub@gmail.com>
* fix duration not being set, fix mp4 test expecting error
Signed-off-by: kim <grufwub@gmail.com>
* fix test expecting media files with different extension
Signed-off-by: kim <grufwub@gmail.com>
* remove unused code
Signed-off-by: kim <grufwub@gmail.com>
* fix expected storage paths in tests, update expected test thumbnails
Signed-off-by: kim <grufwub@gmail.com>
* remove dead code
Signed-off-by: kim <grufwub@gmail.com>
* fix cached presigned s3 url fetching
Signed-off-by: kim <grufwub@gmail.com>
* fix tests
Signed-off-by: kim <grufwub@gmail.com>
* fix test models
Signed-off-by: kim <grufwub@gmail.com>
* update media processing to use sync.Once{} for concurrency protection
Signed-off-by: kim <grufwub@gmail.com>
* shutup linter
Signed-off-by: kim <grufwub@gmail.com>
* fix passing in KVStore GetStream() as stream to PutStream()
Signed-off-by: kim <grufwub@gmail.com>
* fix unlocks of storage keys
Signed-off-by: kim <grufwub@gmail.com>
* whoops, return the error...
Signed-off-by: kim <grufwub@gmail.com>
* pour one out for tobi's code <3
Signed-off-by: kim <grufwub@gmail.com>
* add back the byte slurping code
Signed-off-by: kim <grufwub@gmail.com>
* check for both ErrUnexpectedEOF and EOF
Signed-off-by: kim <grufwub@gmail.com>
* add back links to file format header information
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* Add local user and post count to nodeinfo responses
This fixes#1307 (at least partially). The nodeinfo endpoint should now
return the total users on an instance, along with their post count.
* Update NodeInfoUsers docstring and swagger yaml file
* launch websocket streaming in goroutine to allow upgrade handler to return
* don't send any message on ping, improved close check on failed read
* use context to signal wsconn close, ensure canceled in read goroutine
Signed-off-by: kim <grufwub@gmail.com>
* [feature] Add throttling middleware to AP endpoints
* refactor a lil bit
* use config setting, start updating docs
* doc updates
* use relative links in faq doc
* small docs fixes
* return code 503 instead of 429 when throttled
* throttle other endpoints too
* simplify token channel prefills
* interim commit: start refactoring middlewares into package under router
* another interim commit, this is becoming a big job
* another fucking massive interim commit
* refactor bookmarks to new style
* ambassador, wiz zeze commits you are spoiling uz
* she compiles, we're getting there
* we're just normal men; we're just innocent men
* apiutil
* whoopsie
* i'm glad noone reads commit msgs haha :blob_sweat:
* use that weirdo go-bytesize library for maxMultipartMemory
* fix media module paths
* start messing about with different mp4 metadata extraction
* heyyooo it works
* add test cow
* move useful multierror to gtserror package
* error out if video doesn't seem to be a real mp4
* test parsing mkv in disguise as mp4
* tidy up error handling
* remove extraneous line
* update framerate formatting
* use float32 for aspect
* fixy mctesterson
* close pipereader on failed data function
* gently slurp the bytes
* readability updates
* go fmt
* tidy up file server tests + add more cases
* start moving io wrappers to separate iotools package. Remove use of buffering while piping recache stream
Signed-off-by: kim <grufwub@gmail.com>
* add license text
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
* for domain block lookups, lookup along subdomain parts
Signed-off-by: kim <grufwub@gmail.com>
* only lookup up to a max of 5 domain parts to prevent DOS, limit inserted domains to max of 5 subdomains
Signed-off-by: kim <grufwub@gmail.com>
* add test for domain block wildcarding
Signed-off-by: kim <grufwub@gmail.com>
* check cached status first, increase cached domain time
Signed-off-by: kim <grufwub@gmail.com>
* fix domain wildcard part building logic
Signed-off-by: kim <grufwub@gmail.com>
* create separate domain.BlockCache{} type to hold all domain blocks in memory
Signed-off-by: kim <grufwub@gmail.com>
* remove unused variable
Signed-off-by: kim <grufwub@gmail.com>
* add docs and test to domain block cache, check for domain == host in domain block getter funcs
Signed-off-by: kim <grufwub@gmail.com>
* add license text
Signed-off-by: kim <grufwub@gmail.com>
* check order in which we check primary cache
Signed-off-by: kim <grufwub@gmail.com>
* add better documentation of how domain block checking is performed
Signed-off-by: kim <grufwub@gmail.com>
* change
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* simplify pruneRemote, remove unncecessary media trace logging, update RemoteOlderThan() to include headers/avis
Signed-off-by: kim <grufwub@gmail.com>
* cleanup pruneallmeta, add remote header to pruneremote tests
Signed-off-by: kim <grufwub@gmail.com>
* fix olderthan duration additions
Signed-off-by: kim <grufwub@gmail.com>
* fix broken test now that test model header changed
Signed-off-by: kim <grufwub@gmail.com>
* instead use new remote test account for new header model
Signed-off-by: kim <grufwub@gmail.com>
* use newer generated ULID for remote_account_3 to ensure it is sorted last
Signed-off-by: kim <grufwub@gmail.com>
* reorganize serialized keys to match expected test account model order
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
* update config generator to support nested structs, add cache configuration options
* update envparsing test
* add cache configuration to config parse tests
* set cache configuration in testrig
* move caches to sub-cache "gts" namespace, update envparsing, add cache config docs to example config
Signed-off-by: kim <grufwub@gmail.com>
* [chore] Move ShortcodeDomain to its own little util func
* [feature] Add RefetchEmojis function to media manager
* [feature] Expose admin media refresh via admin API
* update following review feedback
- change/fix log levels
- make sure not to try to refetch local emojis
- small style refactoring + comments
* log on emoji refetch start
Signed-off-by: kim <grufwub@gmail.com>
Co-authored-by: kim <grufwub@gmail.com>
* Implement Bookmarks
* Update based on review comments
* Update swagger doc
* Fix argument passing to status.Bookmark
* Update changed test
* Updates based on latest PR review
* move caches to a separate State{} structure
Signed-off-by: kim <grufwub@gmail.com>
* fix call to log.Panic not using formatted call
Signed-off-by: kim <grufwub@gmail.com>
* move caches to use interfaces, to make switchouts easier in future
Signed-off-by: kim <grufwub@gmail.com>
* fix rebase issue
Signed-off-by: kim <grufwub@gmail.com>
* improve code comment
Signed-off-by: kim <grufwub@gmail.com>
* fix further issues after rebase
Signed-off-by: kim <grufwub@gmail.com>
* heh
Signed-off-by: kim <grufwub@gmail.com>
* add missing license text
Signed-off-by: kim <grufwub@gmail.com>
Signed-off-by: kim <grufwub@gmail.com>
