Commit graph

2166 commits

Author SHA1 Message Date
kegsay
6de29c1cd2
bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)
* Fix flakey sytest 'Local device key changes get to remote servers'

* Debug logs

* Remove internal/test and use /test only

Remove a lot of ancient code too.

* Use FederationRoomserverAPI in more places

* Use more interfaces in federationapi; begin adding regression test

* Linting

* Add regression test

* Unbreak tests

* ALL THE LOGS

* Fix a race condition which could cause events to not be sent to servers

If a new room event which rewrites state arrives, we remove all joined hosts
then re-calculate them. This wasn't done in a transaction so for a brief period
we would have no joined hosts. During this interim, key change events which arrive
would not be sent to destination servers. This would sporadically fail on sytest.

* Unbreak new tests

* Linting
2022-05-17 13:23:35 +01:00
kegsay
cd82460513
Add docs which explain how to calculate coverage (#2468) 2022-05-17 10:45:50 +01:00
Till
05607d6b87
Add roomserver tests (3/4) (#2447)
* Add Room Aliases tests

* Add Rooms table test

* Move StateKeyTuplerSorter to the types package

* Add StateBlock tests
Some optimizations

* Add State Snapshot tests
Some optimization

* Return []int64 and convert to pq.Int64Array for postgres

* Move []types.EventNID back to rows.Next()

* Update tests, rename SelectRoomIDs
2022-05-16 19:33:16 +02:00
Neil Alexander
6af35385ba
Version 0.8.5 (#2461)
* Version 0.8.5

* Update changelog

* Update changelog
2022-05-13 13:17:15 +01:00
Neil Alexander
b40b548432
The Pinecone gobind demo must listen on localhost for baseURL to be correct 2022-05-13 12:06:47 +01:00
Neil Alexander
1698c39579
Update README.md 2022-05-13 11:52:42 +01:00
Neil Alexander
be9be2553f
Resolve over old and new extremities (#2457)
* Feed existing state into state res when calculating state from new extremities

* Remove duplicates

* Fix bug

* Sort and unique

* Update to matrix-org/gomatrixserverlib#308

* Trim the slice properly

* Update gomatrixserverlib again

* Update to matrix-org/gomatrixserverlib#308
2022-05-13 11:52:04 +01:00
Neil Alexander
cafc2d2c10
Update NATS Server to version 2.8.2 (#2460) 2022-05-13 11:36:04 +01:00
Till Faelligen
b57fdcc82d Only try to get OTKs if the context isn't done yet 2022-05-13 10:28:00 +02:00
Till
870f9b0c3f
Shuffle config Verify/Defaults a bit around (#2459) 2022-05-13 09:33:55 +02:00
Neil Alexander
fc670f03a2
Separate sample configs for monolith and polylith (#2456)
* Update sample configs

* Update references

* Remove sections that are dead in the monolith sample
2022-05-12 12:05:55 +01:00
Till
0d1505a4c1
Fix create-account with global database settings (#2455)
* Fix create-account with global database settings

* Avoid warning about open registration
2022-05-12 11:35:35 +02:00
Kegan Dougal
3437adf597 Wait 100ms for events to be processed by syncapi 2022-05-12 10:11:46 +01:00
Till
58af7f61b6
Fix OTK upload spam (#2448)
* Fix OTK spam

* Update comment

* Optimize selectKeysCountSQL to only return max 100 keys

* Return CurrentPosition if the request timed out

* Revert "Return CurrentPosition if the request timed out"

This reverts commit 7dbdda9641.

Co-authored-by: kegsay <kegan@matrix.org>
2022-05-11 17:15:18 +01:00
Neil Alexander
24f7be968d
Fix link 2022-05-11 15:46:45 +01:00
Neil Alexander
19a9166eb0
New documentation: https://matrix-org.github.io/dendrite/ 2022-05-11 15:39:36 +01:00
kegsay
9599b3686e
More syncapi tests (#2451)
* WIP tests for flakey create event

* Uncomment all database test
2022-05-11 13:44:32 +01:00
kegsay
c15bfefd0d
Add RoomExists flag to QueryMembershipForUser (#2450)
Fixes https://github.com/matrix-org/complement/pull/369
2022-05-11 11:29:23 +01:00
Till
6db08b2874
Add roomserver tests (2/?) (#2445)
* Add invite table tests; move variable declarations

* Add Membership table tests

* Move variable declarations

* Add PrevEvents table tests

* Add Published table test

* Add Redactions tests
Fix bug in SQLite markRedactionValidatedSQL

* PR comments, better readability for invite tests
2022-05-10 14:41:12 +02:00
Neil Alexander
1897e2f1c0
Version 0.8.4 2022-05-10 12:44:29 +01:00
Neil Alexander
e2a932ec0b
Add indexes to syncapi_output_room_events table that satisfy the filters (#2446) 2022-05-10 11:23:36 +01:00
Neil Alexander
77722c5a4f
Back out matrix-org/dendrite#2421 by restoring http.Clients
This creates problems with non-HTTPS endpoints and should fix #2444.
2022-05-10 11:08:10 +01:00
database64128
1b3fa9689c
🏗️ mediaapi/thumbnailer: fix build with bimg (#2440)
Co-authored-by: kegsay <kegan@matrix.org>
2022-05-09 17:51:30 +01:00
Neil Alexander
6b3c183396
Version 0.8.3 (#2431)
* Version 0.8.3

* Update changelog
2022-05-09 17:31:14 +01:00
kegsay
236b16aa6c
Begin adding syncapi component tests (#2442)
* Add very basic syncapi tests

* Add a way to inject jetstream messages

* implement add_state_ids

* bugfixes

* Unbreak tests

* Remove now un-needed API call

* Linting
2022-05-09 17:23:02 +01:00
Neil Alexander
a443d1e5f3
Don't store invites in sync API that aren't relevant to local users (#2439) 2022-05-09 16:25:22 +01:00
Neil Alexander
79da75d483
Federation consumer adds_state_event_ids tweak (#2441)
* Don't ask roomserver for events we already have in federation API

* Check number of events returned is as expected

* Preallocate array

* Improve shape a bit
2022-05-09 16:19:35 +01:00
Neil Alexander
1a7f4c8aa9
Don't try to re-fetch the event if it is listed in adds_state_event_ids (#2437)
* Don't try to re-fetch the event in the output message

* Try that again

* Add the initial event into the set
2022-05-09 15:22:33 +01:00
Till
f69ebc6af2
Add roomserver tests (1/?) (#2434)
* Add EventJSONTable tests

* Add eventJSON tests

* Add EventStateKeysTable tests

* Add EventTypesTable tests

* Add Events Table tests
Move variable declaration outside loops
Switch to testify/assert for tests

* Move variable declaration outside loop

* Remove random data

* Fix issue where the EventReferenceSHA256 is not set

* Add more tests

* Revert "Fix issue where the EventReferenceSHA256 is not set"

This reverts commit 8ae34c4e5f.

* Update GMSL

* Add tests for duplicate entries

* Test what happens if we select non-existing NIDs

* Add test for non-existing eventType

* Really update GMSL
2022-05-09 15:30:32 +02:00
Neil Alexander
09d754cfbf
One NATS instance per BaseDendrite (#2438)
* One NATS instance per `BaseDendrite`

* Fix roomserver
2022-05-09 14:15:24 +01:00
Neil Alexander
79e2fbc663
Update to matrix-org/gomatrixserverlib#307 2022-05-09 13:53:51 +01:00
Neil Alexander
4c15c73b3a
Add (user_id, device_id) index on OTK table (#2435) 2022-05-09 11:13:04 +01:00
Neil Alexander
633ca06eb9
Version 0.8.3rc1 2022-05-06 16:34:52 +01:00
Neil Alexander
6bc6184d70
Simplify calculateLatest (#2430)
* Simplify `calculateLatest`

* Comments
2022-05-06 15:52:44 +01:00
Neil Alexander
85c00208c5
Fix power level event auth bugs (update to matrix-org/gomatrixserverlib#306) 2022-05-06 15:41:16 +01:00
Till
6493c0c0f2
Move LL cache (#2429) 2022-05-06 15:33:34 +02:00
Neil Alexander
507f63d0fc
Add PolylithMode base config option (#2428)
* Add `PolylithMode` base config option

* Polylith mode always uses HTTP APIs
2022-05-06 13:51:48 +01:00
Neil Alexander
a1a5357f79
Produce more useful event auth errors (update to matrix-org/gomatrixserverlib#305) 2022-05-06 12:46:01 +01:00
kegsay
85704eff20
Clean up interface definitions (#2427)
* tidy up interfaces

* remove unused GetCreatorIDForAlias

* Add RoomserverUserAPI interface

* Define more interfaces

* Use AppServiceInternalAPI for consistent naming

* clean up federationapi constructor a bit

* Fix monolith in -http mode
2022-05-06 12:39:26 +01:00
Till
4705f5761e
Add FAQ entry for anonymous stats (#2419) 2022-05-06 08:26:24 +02:00
kegsay
9957752a9d
Define component interfaces based on consumers (2/2) (#2425)
* convert remaining interfaces

* Tidy up the userapi interfaces
2022-05-05 19:30:38 +01:00
Neil Alexander
e4da04e75b
Update to matrix-org/gomatrixserverlib#303 2022-05-05 14:06:05 +01:00
Neil Alexander
42f35a57ac
Update table names for user API stats table 2022-05-05 13:42:12 +01:00
Neil Alexander
530fd488a9
Don't log consumer errors on shutdown 2022-05-05 13:29:39 +01:00
kegsay
506de4bb3d
Define component interfaces based on consumers (1/2) (#2423)
* Specify interfaces used by appservice, do half of clientapi

* convert more deps of clientapi to finer-grained interfaces

* Convert mediaapi and rest of clientapi

* Somehow this got missed
2022-05-05 13:17:38 +01:00
Neil Alexander
d9e71b93b6
Use gomatrixserverlib.Client instead of http.Client (#2421)
* Update to matrix-org/gomatrixserverlib#303

* Use `gomatrixserverlib.Client` for phone-home stats

* Use `gomatrixserverlib.Client` for push notifications

* Use `gomatrixserverlib.Client` for appservices

* Use `gomatrixserverlib.Client` for three-PID invites
2022-05-05 11:33:16 +01:00
Till
1bfe87aa56
Fix user already joined when using server notices (#2364) 2022-05-05 12:01:28 +02:00
Till
658e82f8bc
Don't use in-memory db for userapi tests (#2417)
* Don't use in-memory db

* Use WithAllDatabases where possible
2022-05-05 12:00:18 +02:00
kegsay
5c37f165ae
Errors from createdb are non-fatal (#2420)
As they are expected if the database already exists.
2022-05-05 10:53:52 +01:00
kegsay
d86dcbef66
syncapi: define specific interfaces for internal HTTP communications (#2416)
* syncapi: use finer-grained interfaces when making the syncapi

* Use specific interfaces for syncapi-roomserver interactions

* Define query access token api for shared http auth code
2022-05-05 09:56:03 +01:00