Commit graph

495 commits

Author SHA1 Message Date
Till
da7bca0224
Some tweaks for the device list updater (#3251)
This makes the following changes:
- Adds two new metrics observing the usage of the `DeviceListUpdater`
workers
- Makes the number of workers configurable
- Adds a 30s timeout for DB requests when receiving a device list update
over federation
2023-10-31 16:39:45 +01:00
Till
4fa8512d57
Check event is not rejected (#3243)
Companion PR to https://github.com/matrix-org/gomatrixserverlib/pull/421
2023-10-25 09:47:21 +02:00
Till
8b3adaf244
Fix state resets (#3231)
Needs https://github.com/matrix-org/gomatrixserverlib/pull/419

May fix: https://github.com/matrix-org/dendrite/issues/2508,
https://github.com/matrix-org/dendrite/issues/1760
2023-10-23 15:17:21 +02:00
Till
f02d998253
Remove the creator field when upgrading to v11 (#3210)
Minor oversight
2023-09-28 07:36:57 +02:00
Till
05a8f1ede3
Support for room version v11 (#3204)
Fixes #3203
2023-09-27 08:27:08 +02:00
devonh
16d922de70
Complement fixes for pseudoIDs (#3206) 2023-09-26 17:44:49 +00:00
devonh
8245b24100
Update gmsl to use new validated RoomID on PDUs (#3200)
GMSL returns a `spec.RoomID` when calling `PDU.RoomID()`
2023-09-15 14:39:06 +00:00
Sam Wedgwood
9b5be6b9c5
[pseudoIDs] More pseudo ID fixes - Part 2 (#3181)
Fixes include:
- Translating state keys that contain user IDs to their respective room
keys for both querying and sending state events
- **NOTE**: there may be design discussion needed on what should happen
when sender keys cannot be found for users
- A simple fix for kicking guests from rooms properly
- Logic for boundary history visibilities was slightly off (I'm
surprised this only manifested in pseudo ID room versions)

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-24 16:43:51 +01:00
Sam Wedgwood
9a12420428
[pseudoID] More pseudo ID fixes (#3167)
Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-15 12:37:04 +01:00
Sam Wedgwood
35804f8493
Add config key for default room version (#3171)
This PR adds a config key `room_server.default_config_key` to set the
default room version for the room server.

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-08 14:20:05 +01:00
Sam Wedgwood
c7193e24d0
Use *spec.SenderID for QuerySenderIDForUser (#3164)
There are cases where a dendrite instance is unaware of a pseudo ID for
a user, the user is not a member of that room. To represent this case,
we currently use the 'zero' value, which is often not checked and so
causes errors later down the line. To make this case more explict, and
to be consistent with `QueryUserIDForSender`, this PR changes this to
use a pointer (and `nil` to mean no sender ID).

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-02 11:12:14 +01:00
Sam Wedgwood
af13fa1c75
[pseudoIDs] Fixes for room alias tests (#3159)
Some (deceptively) simple fixes for some bugs that caused room alias
tests to fail (sytext `tests/30rooms/05aliases.pl`). Each commit has
details about what it fixes.

Sytest results:

- Sytest before (79d4a0e):
https://gist.github.com/swedgwood/972ac4ef93edd130d3db0930703d6c82
- Sytest after (4b09bed):
https://gist.github.com/swedgwood/504b00ac4ee892acb757b7fac55fa28a

Room aliases go from `8/15` to `15/15`, but looks like these fixes also
managed to fix about `4` other tests, which is a nice bonus :)

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-07-31 14:39:41 +01:00
Till Faelligen
79d4a0e399
Restore old behaviour of PurgeRoom 2023-07-26 09:09:04 +02:00
devonh
c809e95335
Fix event federation with pseudoID rooms (#3156) 2023-07-21 16:08:40 +00:00
Sam Wedgwood
9582827493
de-MSC-ifying space summaries (MSC2946) (#3134)
- This PR moves and refactors the
[code](https://github.com/matrix-org/dendrite/blob/main/setup/mscs/msc2946/msc2946.go)
for
[MSC2946](https://github.com/matrix-org/matrix-spec-proposals/pull/2946)
('Space Summaries') to integrate it into the rest of the codebase.
- Means space summaries are no longer hidden behind an MSC flag
- Solves #3096

Signed-off-by: Sam Wedgwood <sam@wedgwood.dev>
2023-07-20 15:06:05 +01:00
Till
297479ea49
Use pointer when passing the connection manager around (#3152)
As otherwise existing connections aren't reused.
2023-07-19 13:37:04 +02:00
Till
5267cc0f54
Optimise getting local members and membership counts (#3150)
The previous version was getting **ALL** membership events (as
`ClientEvents`, so going through `NewEventFromTrustedJSONWithID`) for a
given room.
Now we are querying only locally joined users as `ClientEvents`, which
should **significantly** reduce allocations.

Take for example a large room with 2k membership events, but only 1
local user - avoiding 1999 `NewEventFromTrustedJSONWithID` calls just to
calculate the `roomSize` which we can also query by other means.

This is also getting called for every `OutputRoomEvent` in the userAPI.

Benchmark with 1 local user and 100 remote users.
```
pkg: github.com/matrix-org/dendrite/userapi/consumers
cpu: 12th Gen Intel(R) Core(TM) i5-12500H
                    │   old.txt   │               new.txt               │
                    │   sec/op    │   sec/op     vs base                │
LocalRoomMembers-16   375.9µ ± 7%   327.6µ ± 6%  -12.85% (p=0.000 n=10)

                    │    old.txt    │               new.txt                │
                    │     B/op      │     B/op      vs base                │
LocalRoomMembers-16   79.426Ki ± 0%   8.507Ki ± 0%  -89.29% (p=0.000 n=10)

                    │   old.txt   │              new.txt               │
                    │  allocs/op  │ allocs/op   vs base                │
LocalRoomMembers-16   1015.0 ± 0%   277.0 ± 0%  -72.71% (p=0.000 n=10)
```
2023-07-13 14:19:08 +02:00
Till
eb9e90379d
Add event size checks similar to Synapse (#3140)
Companion to https://github.com/matrix-org/gomatrixserverlib/pull/400
This tries to mimic the logic found in Synapse, as dropping events can
break rooms (and we may end up in endless loops..)
2023-07-07 20:37:23 +02:00
devonh
d507c5fc95
Add pseudoID compatibility to Invites (#3126) 2023-07-06 15:15:24 +00:00
Till Faelligen
5a87c703fa
Fix metrics.. 2023-07-05 12:34:53 +02:00
Till
4c3a526e1b
Fix adding state events to the database (#3133)
When we're adding state to the database, we check which eventNIDs are
already in a block, if we already have that eventNID, we remove it from
the list. In its current form we would skip over eventNIDs in the case
we already found a match (we're decrementing `i` twice)
My theory is, that when we later get the state blocks, we are receiving
"too many" eventNIDs (well, yea, we stored too many), which may or may
not can result in state resets when comparing different state snapshots.
(e.g. when adding state we stored a eventNID by accident because we
skipped it, later we add more state and are not adding it because we
don't skip it)
2023-07-04 17:15:44 +02:00
Till Faelligen
939ee325f8
Actually use the parameter 2023-06-29 18:02:11 +02:00
Till
23cd7877a1
Add MXIDMapping for pseudoID rooms (#3112)
Add `MXIDMapping` on membership events when
creating/joining rooms.
2023-06-28 20:29:49 +02:00
Till
a734b112c6
Fix backfilling (#3117)
This should fix two issues with backfilling:
1. right after creating and joining a room over federation, we are doing
a `/backfill` request, which would return redacted events, because the
`authEvents` are empty. Even though the spec states that, in the absence
of a history visibility event, it should be handled as `shared`.
2. `gomatrixserverlib: unsupported room version ''` - because, well, we
were never setting the `roomInfo` field..
2023-06-20 16:52:29 +02:00
Devon Hudson
8cf6c381e2
Fix senderID/key conversion unit tests 2023-06-14 17:11:27 +01:00
Devon Hudson
3f4df25b31
Add missing dep 2023-06-14 17:04:19 +01:00
Devon Hudson
5aaa539e3e
Fix senderID/key conversions 2023-06-14 16:42:09 +01:00
devonh
e4665979bf
Merge SenderID & Per Room User Key work (#3109) 2023-06-14 14:23:46 +00:00
Till
7a2e325d10
Add AssignRoomNID to pre-assign roomNIDs (#3111) 2023-06-13 16:28:41 +02:00
Till
2c87972a3a
Create user room key if needed (#3108) 2023-06-13 14:19:31 +02:00
devonh
77d9e4e93d
Cleanup remaining statekey usage for senderIDs (#3106) 2023-06-12 11:19:25 +00:00
Till
832ccc32f6
Add initial support for storing user room keys (#3098) 2023-06-12 12:45:42 +02:00
devonh
8ea1a11105
Use SenderID Type (#3105) 2023-06-07 17:14:35 +00:00
devonh
7a1fd7f512
PDU Sender split (#3100)
Initial cut of splitting PDU Sender into SenderID & looking up UserID where required.
2023-06-06 20:55:18 +00:00
Till
725ff5567d
Make StrictValidityChecking a function (#3092)
Companion PR to https://github.com/matrix-org/gomatrixserverlib/pull/388
2023-06-06 15:16:55 +02:00
Till
d11da6ec7c
Fix newly found linter issues (#3099)
Fixes the issues found in
https://github.com/matrix-org/dendrite/actions/runs/5155539352/jobs/9285342056#step:5:22.
Only naked returns in longer functions.
2023-06-02 15:48:04 +02:00
devonh
ea6b368ad4
Move Invite logic to GMSL (#3086)
This is both the federation receiving & sending side logic (which were
previously entangeld in a single function)
2023-05-31 16:33:49 +00:00
devonh
cbdc601f1b
Move CreateRoom logic to Roomserver (#3093)
Move create room logic over to roomserver.
2023-05-31 15:27:08 +00:00
Till
61341aca50
Add tests for the UpDropEventReferenceSHAPrevEvents migration (#3087)
... as they could fail if there are duplicate events in
`roomserver_previous_events`.
This fixes the migration by trying to combine the `event_nids` if
possible (same room) as mentioned by @kegsay in
https://github.com/matrix-org/dendrite/pull/3083#discussion_r1195508963
2023-05-30 18:05:48 +02:00
Till
f956a8c1d9
Docs restructure (#2953)
Needs to be merged into `gh-pages` later on.
2023-05-30 10:02:53 +02:00
Till
11b557097c
Drop reference_sha column (#3083)
Companion PR to https://github.com/matrix-org/gomatrixserverlib/pull/383
2023-05-24 12:14:42 +02:00
devonh
2eae8dc489
Move SendJoin logic to GMSL (#3084)
Moves the core matrix logic for handling the send_join endpoint over to
gmsl.
2023-05-19 16:27:01 +00:00
Devon Hudson
027a9b8ce0
Fix bug with nil interface return & add test 2023-05-18 13:41:47 -06:00
devonh
67d6876857
Move MakeJoin logic to GMSL (#3081) 2023-05-17 00:33:27 +00:00
devonh
0489d16f95
Move json errors over to gmsl (#3080) 2023-05-09 22:46:49 +00:00
devonh
a49c9f01e2
Only require room version instead of room info for db.Events() (#3079)
This reduces the API requirements for the Events database to align with
what is actually required.
2023-05-08 19:25:44 +00:00
kegsay
2b34f88fde
Use ProtoEvent where needed instead of EventBuilder (#3075)
They are fundamentally different concepts, so should be represented as
such. Proto events are exchanged in /make_xxx calls over federation, and
made as "fledgling" events in /createRoom and general event sending.
*Building* events is a reasonably complex VERSION SPECIFIC process which
needs amongst other things, auth event providers, prev events, signing
keys, etc.

Requires https://github.com/matrix-org/gomatrixserverlib/pull/379
2023-05-04 11:17:42 +01:00
kegsay
6284790f98
Use PDU in even more places (#3074)
- No longer rely on *Event returning from NewEventFrom... functions
 
Requires https://github.com/matrix-org/gomatrixserverlib/pull/377
2023-05-03 10:21:27 +01:00
kegsay
f5b3144dc3
Use PDU not *Event in HeaderedEvent (#3073)
Requires https://github.com/matrix-org/gomatrixserverlib/pull/376

This has numerous upsides:
 - Less type casting to `*Event` is required.
- Making Dendrite work with `PDU` interfaces means we can swap out Event
impls more easily.
 - Tests which represent weird event shapes are easier to write.

Part of a series of refactors on GMSL.
2023-05-02 15:03:16 +01:00
Devon Hudson
696cbb70b8
Pass federation API to roomserver in PurgeRoom tests 2023-05-01 21:28:10 -06:00