2
0
Fork 0
mirror of https://github.com/matrix-org/dendrite synced 2024-12-14 15:22:50 +00:00
Commit graph

1262 commits

Author SHA1 Message Date
Kegsay
fb56bbf0b7
Generate stream IDs for locally uploaded device keys ()
* Breaking: add stream_id to keyserver_device_keys table

* Add tests for stream ID generation

* Fix whitelist
2020-08-03 17:07:06 +01:00
Kegsay
ffcb6d2ea1
Produce OTK counts in /sync response ()
* Add QueryOneTimeKeys for /sync extensions

* Unbreak tests

* Produce OTK counts in /sync response

* Linting
2020-08-03 12:29:58 +01:00
Kegsay
b5cb1d1534
Fix edge cases around device lists ()
* Fix New users appear in /keys/changes

* Create blank device keys when logging in on a new device

* Add PerformDeviceUpdate and fix a few bugs

- Correct device deletion query on sqlite
- Return no keys on /keys/query rather than an empty key

* Unbreak sqlite properly

* Use a real DB for currentstateserver integration tests

* Race fix
2020-07-31 14:40:45 +01:00
Kegsay
a7e67e65a8
Notify clients when devices are deleted ()
* Recheck device lists when join/leave events come in

* Add PerformDeviceDeletion

* Notify clients when devices are deleted

* Unbreak things

* Remove debug logging
2020-07-30 18:00:56 +01:00
Neil Alexander
292a9ddd82
Fix Yggdrasil demo builds 2020-07-30 15:47:44 +01:00
Kegsay
a2174d3294
Implement /keys/changes ()
* Implement /keys/changes

And refactor QueryKeyChanges to accept a `to` offset.

* Unbreak tests

* Sort keys when serialising log tokens
2020-07-30 14:52:21 +01:00
Kegsay
9355fb5ac8
Hook up device list updates to the sync notifier ()
* WIP hooking up key changes

* Fix import cycle, get tests passing and binary compiling

* Linting and update whitelist
2020-07-30 11:15:46 +01:00
Kegsay
0fdd4f14d1
Add support for logs in StreamingToken ()
* Add support for logs in StreamingToken

Tokens now end up looking like `s11_22|dl-0-123|ab-0-12224`
where `dl` and `ab` are log names, `0` is the partition and
`123` and `12224` are the offsets.

* Also test reserialisation

* s/|/./g so tokens url escape nicely
2020-07-29 19:00:04 +01:00
Kegsay
9a5fb489c5
Add QueryKeyChanges ()
Hook some things up to call it as well.
2020-07-28 18:25:16 +01:00
Kegsay
adf7b59294
Persist partition|offset|user_id in the keyserver ()
* Persist partition|offset|user_id in the keyserver

Required for a query API which will be used by the syncapi which
will be called when a `/sync` request comes in which will return
a list of user IDs of people who have changed their device keys
between two tokens.

* Add tests and fix maxOffset bug

* s/offset/log_offset/g because 'offset' is a reserved word in postgres
2020-07-28 17:38:30 +01:00
Neil Alexander
acc8e80a51
User directory ()
* User directory

* Fix syncapi unit test

* Make user directory only show remote users you know about from your joined rooms

* Update sytest-whitelist

* Review comments
2020-07-28 10:53:17 +01:00
Kegsay
c632867135
Modify /state/{eventType}/{stateKey} to return the event at the time the user left ()
* Modify /state/{eventType}/{stateKey} to return the event at the time the user left

Or live, depending on their current state. Hopefully fixes some sytests!

* Linting

* Set HasBeenInRoom

* Fix cases for world-readable history visibility

* Fix bug in finding the requested state event

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-07-28 10:09:10 +01:00
Henrik Sölver
83f038e12b
Don't use more than 999 variables in SQLite querys. ()
Closes 

Signed-off-by: Henrik Sölver <henrik.solver@gmail.com>

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-07-27 13:19:30 +01:00
Kegsay
c8d476a3cc
Return HTTP errors when trying to kick invalid users ()
Room integrity was never compromised as GMSL does auth checks,
but we would incorrectly 200 OK the request instead of 403ing.
2020-07-27 09:20:09 +01:00
Kegsay
61963a74ae
Add logic for determining when device lists have changed due to membership changes ()
* Flesh out structure for handling device list updates for room membership changes

* First cut untested algorithm

* Add tests for determining changed/left device lists

* Linting

* Unbreak tests

* Sigh.. linting
2020-07-27 09:19:55 +01:00
Kegan Dougal
abef9bc04f Put this in the right category 2020-07-24 17:11:16 +01:00
Kegan Dougal
b63fa7b880 Update AWSY list 2020-07-24 16:17:33 +01:00
Kegsay
af5b4d1f6b
Modify QuerySharedUsers to handle counts/include/exclude ()
* Modify QuerySharedUsers to handle counts/include/exclude

We will need this functionality when working out whether to
send device list changes to users who have joined/left a room.

* Linting
2020-07-24 10:33:41 +01:00
Kegsay
98f2f09bb4
keyserver: produce key change events ()
* Produce kafka events when keys are added

* Consume key changes in syncapi with TODO markers for handling them and catching up

* unbreak tests

* Linting
2020-07-23 16:41:36 +01:00
Kegsay
7b862384a7
currentstate: Add QuerySharedUsers ()
This will be used to determine who to send device list updates to. It
can also be used to determine who to send presence info to.
2020-07-23 12:26:31 +01:00
Neil Alexander
cfeb1b2f42
Add UNIQUE constraint to blacklist table () 2020-07-23 10:22:23 +01:00
Kegan Dougal
9684efebd7 Fix gobind 2020-07-22 17:06:19 +01:00
Kegsay
541a23f712
Handle inbound federation E2E key queries/claims ()
* Handle inbound /keys/claim and /keys/query requests

* Add display names to device key responses

* Linting
2020-07-22 17:04:57 +01:00
Neil Alexander
1e71fd645e
Persistent federation sender blacklist ()
* Initial persistence of blacklists

* Move statistics folder

* Make MaxFederationRetries configurable

* Set lower failure thresholds for Yggdrasil demos

* Still write events into database for blacklisted hosts (they can be tidied up later)

* Review comments
2020-07-22 17:01:29 +01:00
Kegsay
470933789b
Perform outbound federation hits for querying/claiming E2E keys ()
* Perform outbound federation hits for querying/claiming E2E keys

Untested currently because we need the receiving end to work
before sytest will be happy.

* Linting
2020-07-21 17:46:47 +01:00
Neil Alexander
f54689f862
Fix lint error 2020-07-21 17:21:48 +01:00
Neil Alexander
b6bc132485
Use TransactionWriter in other component SQLite ()
* Use TransactionWriter on other component SQLites

* Fix sync API tests

* Fix panic in media API

* Fix a couple of transactions

* Fix wrong query, add some logging output

* Add debug logging into StoreEvent

* Adjust InsertRoomNID

* Update logging
2020-07-21 15:48:21 +01:00
Kegsay
1d72ce8b7a
Implement claiming one-time keys locally ()
* Add API shape for claiming keys

* Implement claiming one-time keys locally

Fairly boring, nothing too special going on.
2020-07-21 14:47:53 +01:00
Neil Alexander
d76eb1b994
Use TransactionWriter in roomserver SQLite () 2020-07-21 10:48:49 +01:00
Neil Alexander
489f34fed7
Remove debug lines 2020-07-20 17:03:20 +01:00
Neil Alexander
11a39fe3b5
Deduplicate FS database, EDU persistence table ()
* Deduplicate FS database, add some EDU persistence groundwork

* Extend TransactionWriter to use optional existing transaction, use that for FS SQLite database writes

* Fix build due to bad keyserver import

* Working EDU persistence

* gocyclo, unsurprisingly

* Remove unused

* Update copyright notices
2020-07-20 16:55:20 +01:00
Creed Haymond
f3c482b078
Adding the e2ekey database to the create_db script and docs ()
Signed-off-by: Creed Haymond <creedasaurus@gmail.com>
2020-07-20 09:18:48 +01:00
Neil Alexander
986498f5a5
Don't block Yggdrasil switch 2020-07-16 15:19:26 +01:00
Neil Alexander
fbf139f20e
Small fix to gobind build 2020-07-16 14:26:04 +01:00
Neil Alexander
e5208c2ec9
Yggdrasil demo updates ("Bare QUIC")
Squashed commit of the following:

commit 86c2388e13ffdbabdd50cea205652dccc40e1860
Merge: b0a3ee6c f5e7e751
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 16 13:47:10 2020 +0100

    Merge branch 'master' into neilalexander/yggbarequic

commit b0a3ee6c5c063962384bb91c59ec753ddc8cfe5f
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 16 13:42:22 2020 +0100

    Add support for broadcasting wake-up EDUs to known hosts

commit 8a5c2020b3a4b705b5d5686a9e71990a49e6d471
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 16 13:42:10 2020 +0100

    Bare QUIC demo working

commit d3939b3d6568cf4262c0391486a5203873b68bfc
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Wed Jul 15 11:42:43 2020 +0100

    Support bare Yggdrasil sessions with encrypted QUIC
2020-07-16 13:52:08 +01:00
Kegsay
f5e7e7513c
Implement /keys/query locally ()
* Implement /keys/query locally

* Fix sqlite tests and close rows
2020-07-15 18:40:41 +01:00
Kegsay
df8d6823ee
Fix panic when uploading keys ()
We don't return OTK counts when the client doesn't upload OTKs.
2020-07-15 15:26:04 +01:00
Neil Alexander
84387e5249
Protect User API SQLite calls with mutexes ()
* Protect SQLite calls with mutexes (replaces )

* Remove locks on reads
2020-07-15 15:14:59 +01:00
Kegsay
9dd2ed7f65
Implement key uploads ()
* Add storage layer for postgres/sqlite

* Return OTK counts when inserting new keys

* Hook up the key DB and make a test pass

* Convert postgres queries to be sqlite queries

* Blacklist test due to requiring rejected events

* Unbreak tests

* Update blacklist
2020-07-15 12:02:34 +01:00
Neil Alexander
b4c07995d6
Create FUNDING.yml 2020-07-15 11:30:32 +01:00
Kegsay
7daa3bf098
Implement logic for key uploads ()
* begin work on storing keys

* Finish rough impl of the internal key API

* Linting
2020-07-14 12:59:07 +01:00
Kegsay
37db60f4d4
Update README/docs ()
* Update README

* Update docs
2020-07-14 12:58:00 +01:00
Neil Alexander
72b3160776
Send-to-device messages over federation ()
* Initial work to send send-to-device messages over federation

* Wire up send-to-device consumer, message formatting

* Generate random message ID

* Review comments, update sytest whitelist
2020-07-14 12:33:37 +01:00
Kegan Dougal
5355c289b9 Bump GMSL version to fix sytest 2020-07-14 10:47:34 +01:00
Kegsay
396219ef53
Add boilerplate for key server APIs ()
Also add a README which outilnes how things will work.
2020-07-13 16:02:35 +01:00
Neil Alexander
3178afde2c
Update go.mod/go.sum 2020-07-13 09:38:40 +01:00
Neil Alexander
08e9d996b6
Yggdrasil demo updates
Squashed commit of the following:

commit 6c2c48f862c1b6f8e741c57804282eceffe02487
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 10 16:28:09 2020 +0100

    Add README.md

commit 5eeefdadf8e3881dd7a32559a92be49bd7ddaf47
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 10 10:18:50 2020 +0100

    Fix wedge in federation sender

commit e2ebffbfba25cf82378393940a613ec32bfb909f
Merge: 0883ef88 abf26c12
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 10 09:51:23 2020 +0100

    Merge branch 'master' into neilalexander/yggdrasil

commit 0883ef8870e340f2ae9a0c37ed939dc2ab9911f6
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 10 09:51:06 2020 +0100

    Adjust timeouts

commit ba2d53199910f13b60cc892debe96a962e8c9acb
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 9 16:34:40 2020 +0100

    Try to wake up from peers/sessions properly

commit 73f42eb494741ba5b0e0cef43654708e3c8eb399
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 9 15:43:38 2020 +0100

    Use TransactionWriter to reduce database lock issues on SQLite

commit 08bfe63241a18c58c539c91b9f52edccda63a611
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 9 12:38:02 2020 +0100

    Un-wedge federation

    Squashed commit of the following:

    commit aee933f878
    Author: Neil Alexander <neilalexander@users.noreply.github.com>
    Date:   Thu Jul 9 12:22:41 2020 +0100

        Un-goroutine the goroutines

    commit 478374e5d1
    Author: Neil Alexander <neilalexander@users.noreply.github.com>
    Date:   Thu Jul 9 12:09:31 2020 +0100

        Reduce federation sender wedges

commit 40cc62c54d9e3a863868214c48b7c18e522a4772
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Thu Jul 9 10:02:52 2020 +0100

    Handle switching in/out background more reliably
2020-07-10 16:28:18 +01:00
Kegsay
abf26c12f1
Add User-Interactive Authentication ()
* Add User-Interactive Authentication

And use it when deleting a device. With tests.

* Make remaining sytest pass

* Linting

* 403 not 401 on wrong user/pass
2020-07-10 00:39:44 +01:00
Neil Alexander
9cc52f47f3
Use TransactionWriter to reduce database lock issues on SQLite () 2020-07-09 17:48:56 +01:00
Neil Alexander
99b50f30a0
Reduce federation sender wedges ()
* Reduce federation sender wedges

* Un-goroutine the goroutines
2020-07-09 15:39:35 +01:00