Commit graph

173 commits

Author SHA1 Message Date
kegsay
1432743d1a
Use PDU in more places (#3072) 2023-04-28 16:00:22 +01:00
kegsay
6171310307
Use PDU interface (#3070)
We only use it in a few places currently, enough to get things to
compile and run. We should be using it in much more places.

Similarly, in some places we cast []PDU back to []*Event, we need to not
do that. Likewise, in some places we cast PDU to *Event, we need to not
do that. For now though, hopefully this is a start.
2023-04-27 16:35:19 +01:00
kegsay
b189edf4f4
Remove gmsl.HeaderedEvent (#3068)
Replaced with types.HeaderedEvent _for now_. In reality we want to move
them all to gmsl.Event and only use HeaderedEvent when we _need_ to
bundle the version/event ID with the event (seriailsation boundaries,
and even then only when we don't have the room version).

Requires https://github.com/matrix-org/gomatrixserverlib/pull/373
2023-04-27 12:54:20 +01:00
Till
2475cf4b61
Add some roomserver UTs (#3067)
Adds tests for `QueryRestrictedJoinAllowed`, `IsServerAllowed` and
`PerformRoomUpgrade`. Refactors the `QueryRoomVersionForRoom` method to
accept a string and return a `gmsl.RoomVersion` instead of req/resp
structs.
Adds some more caching for `GetStateEvent`

This should also fix #2912 by ignoring state events belonging to other
users.
2023-04-27 08:07:13 +02:00
devonh
ed19efc5d7
Move fedclient interface over to gmsl (#3061)
Companion PR: https://github.com/matrix-org/gomatrixserverlib/pull/366
2023-04-24 16:23:25 +00:00
kegsay
72285b2659
refactor: update GMSL (#3058)
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/364

Read this commit by commit to avoid going insane.
2023-04-19 15:50:33 +01:00
kegsay
ca63b414da
Update GMSL: use static Check functions (#3052)
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/359 

A nice side effect is that we don't need to re-parse the events in some
cases.
2023-04-14 12:32:42 +01:00
kegsay
0db43f13a6
refactor: use latest GMSL which splits fed client from matrix room logic (#3051)
Part of a series of refactors on GMSL.
2023-04-06 09:55:01 +01:00
kegsay
3691423626
Move GMSL client types to Dendrite (#3045)
GMSL is intended for Federation only. Sister PR to
https://github.com/matrix-org/gomatrixserverlib/pull/357
2023-04-04 17:16:53 +00:00
Boris Rybalkin
985298cfc4
app service unix socket support (#3022)
This is the last part of unix socket support to talk to app servers, go
based app services already support unix sockets:

5a68173fe3
```
appservice:
  # The address that the homeserver can use to connect to this appservice.
  address: unix:///var/snap/matrix/current/whatsapp.socket

  # The hostname and port where this appservice should listen.
  hostname: /var/snap/matrix/current/whatsapp.socket
  port: 0
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-04-04 09:42:46 +02:00
Till
5e85a00cb3
Remove BaseDendrite (#3023)
Removes `BaseDendrite` to, hopefully, make testing and composing of
components easier in the future.
2023-03-22 09:21:32 +01:00
Till
5579121c6f
Preparations for removing BaseDendrite (#3016)
Preparations to actually remove/replace `BaseDendrite`.
Quite a few changes:
- SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed
from `BaseDendrite`)
- Caches are removed from `BaseDendrite`
- Introduces a `Router` struct (likely to change)
  - also fixes #2903
- Introduces a `sqlutil.ConnectionManager`, which should remove
`base.DatabaseConnection` later on
- probably more
2023-03-17 11:09:45 +00:00
Boris Rybalkin
d88f71ab71
simplify unix socket permission format (#3014)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-03-16 08:51:21 +01:00
Till
689b5ee72f
Change default stats reporting endpoint (#3007)
It's the same instance we report to, only using the subdomain.
2023-03-10 12:27:08 +01:00
Boris Rybalkin
6b1c9eafa9
unix socket support (#2974)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`

I need this for Syncloud project (https://github.com/syncloud/platform)
where I run multiple apps behind an nginx on the same RPi like device so
unix socket is very convenient to not have port conflicts between apps.
Also someone opened this Issue:
https://github.com/matrix-org/dendrite/issues/2924

---------

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-01 22:57:30 +01:00
Till
6c20f8f742
Refactor StoreEvent, add MaybeRedactEvent, create an EventDatabase (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
David Schneider
e6aa0955ff
Unify logging by using logrus for jetstream logs (#2976)
I guess tests for the logging is rather unusual so I omitted tests for
this change.

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `David Schneider <dsbrng25b@gmail.com>`

---------

Signed-off-by: David Schneider <dsbrng25b@gmail.com>
2023-02-24 08:56:53 +01:00
Till
4594233f89
Merge keyserver & userapi (#2972)
As discussed yesterday, a first draft of merging the keyserver and the
userapi.
2023-02-20 14:58:03 +01:00
Till
11d9b9db0e
Remove polylith/API mode (#2967)
This removes most of the code used for polylith/API mode.

This removes the `/api` internal endpoints entirely. 

Binary size change roughly 5%: 
```
51437560 Feb 13 10:15 dendrite-monolith-server # old
48759008 Feb 13 10:15 dendrite-monolith-server # new
```
2023-02-14 12:47:47 +01:00
devonh
63df85db6d
Relay integration to pinecone demos (#2955)
This extends the dendrite monolith for pinecone to integrate the s&f
features into the mobile apps.
Also makes a few tweaks to federation queueing/statistics to make some
edge cases more robust.
2023-01-28 23:27:53 +00:00
Lukas
80738cc2a0
Added Landing Page (#2885)
I have added/copied a landing page like Synpase does.
Recently I have installed Dendrite and was wondering why it´s not
working. After some troubleshooting I figured out there is no landing
page like synpase has, so the Server was running just fine.

Hopefuly this PR can fix this problem and may help other users who run
into this issue.
I have not written any unit tests, because it´s just a simple landing
page with a redirect to a static site.


### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Lukas Huida<lukas@leucali.net>`

Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-01-26 16:25:17 +01:00
devonh
5b73592f5a
Initial Store & Forward Implementation (#2917)
This adds store & forward relays into dendrite for p2p.
A few things have changed:
- new relay api serves new http endpoints for s&f federation
- updated outbound federation queueing which will attempt to forward
using s&f if appropriate
- database entries to track s&f relays for other nodes
2023-01-23 17:55:12 +00:00
Till
48fa869fa3
Use t.TempDir for SQLite databases, so tests don't rip out each others databases (#2950)
This should hopefully finally fix issues about `disk I/O error` as seen
[here](https://gitlab.alpinelinux.org/alpine/aports/-/jobs/955030/raw)

Hopefully this will also fix `SSL accept attempt failed` issues by
disabling HTTP keep alives when generating a config for CI.
2023-01-23 13:17:15 +01:00
Till
ce2bfc3f2e
Make tests more reliable (#2948)
When using `testrig.CreateBase` and then using that base for other
`NewInternalAPI` calls, we never actually shutdown the components.
`testrig.CreateBase` returns a `close` function, which only removes the
database, so still running components have issues connecting to the
database, since we ripped it out underneath it - which can result in
"Disk I/O" or "pq deadlock detected" issues.
2023-01-20 12:45:56 +01:00
Andrew Morgan
eeeb3017d6
Switch the default config option values for recaptcha_sitekey_class and recaptcha_form_field (#2939)
Attempting to use the [web auth fallback
mechanism](https://spec.matrix.org/v1.5/client-server-api/#fallback) for
Google ReCAPTCHA with the default setting for
`client_api.recaptcha_sitekey_class` of "g-recaptcha-response" results
in no captcha being rendered:


![image](https://user-images.githubusercontent.com/1342360/212482321-14980045-6e20-4d59-adaa-59a01ad88367.png)

I cross-checked the captcha code between [dendrite.matrix.org's fallback
page](https://dendrite.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
and [matrix-client.matrix.org's
one](https://matrix-client.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
(which both use the same captcha public key) and noticed a discrepancy
in the `class` attribute of the div that renders the captcha.
[ReCAPTCHA's docs
state](https://developers.google.com/recaptcha/docs/v3#automatically_bind_the_challenge_to_a_button)
to use "g-recaptcha" as the class for the submit button.

I noticed this when user `@parappanon:parappa.party` reported that they
were also seeing no captcha being rendered on their Dendrite instance.
Changing `client_api.recaptcha_sitekey_class` to "g-recaptcha" caused
their captcha to render properly as well.

There may have been a change in the class name from ReCAPTCHA v2 to v3?
The [docs for
v2](https://developers.google.com/recaptcha/docs/display#auto_render)
also request one uses "g-recaptcha" though.

Thus I propose changing the default setting to unbreak people's
recaptcha auth fallback pages. Should fix dendrite.matrix.org as well.
2023-01-16 12:52:30 +01:00
Till
d579ddb8e7
Add simplified helm chart (#2905)
As discussed yesterday, a simplified version of [my
helm](https://github.com/S7evinK/dendrite-helm) which deploys a monolith
with internal NATS and an optionally enabled PostgreSQL server. If the
PostgreSQL dependency is not enabled, a user specified connection string
is constructed.

Co-authored-by: kegsay <kegan@matrix.org>
2023-01-06 15:44:10 +01:00
Till
f762ce1050
Add clientapi tests (#2916)
This PR
- adds several tests for the clientapi, mostly around `/register` and
auth fallback.
- removes the now deprecated `homeserver` field from responses to
`/register` and `/login`
- slightly refactors auth fallback handling
2022-12-23 14:11:11 +01:00
Till
5eed31fea3
Handle guest access [1/2?] (#2872)
Needs https://github.com/matrix-org/sytest/pull/1315, as otherwise the
membership events aren't persisted yet when hitting `/state` after
kicking guest users.

Makes the following tests pass:
```
Guest users denied access over federation if guest access prohibited
Guest users are kicked from guest_access rooms on revocation of guest_access
Guest users are kicked from guest_access rooms on revocation of guest_access over federation
```

Todo (in a follow up PR):
- Restrict access to CS API Endpoints as per
https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-22 13:05:59 +01:00
Neil Alexander
934056f21f
Fix dendrite-demo-pinecone, /_dendrite namespace setup 2022-12-01 10:45:15 +00:00
Neil Alexander
1990c154e9
Update configuration 2022-11-29 11:11:08 +00:00
Neil Alexander
8299da5905
Fix registration for virtual hosting 2022-11-18 13:24:02 +00:00
Neil Alexander
607819f425
Fix /key/v2/server, add HTTP Host matching 2022-11-17 09:26:56 +00:00
Neil Alexander
a916b041b1
Detect consumer being deleted in JetStreamConsumer 2022-11-16 10:28:22 +00:00
Neil Alexander
6650712a1c
Federation fixes for virtual hosting 2022-11-15 15:05:23 +00:00
Till
d35a5642e8
Deny guest access on several endpoints (#2873)
Second part for guest access, this adds a `WithAllowGuests()` option to
`MakeAuthAPI`, allowing guests to access the specified endpoints.
Endpoints taken from the
[spec](https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14)
and by checking Synapse endpoints for `allow_guest=true`.
2022-11-11 10:52:08 +01:00
X. Ding
0782011f54
Add hcaptcha support besides Google ReCaptcha (#2834)
### Pull Request Checklist
This PR add support for hcaptcha.com as an alternative to Google
ReCaptcha. It also makes possible for user to customize ReCaptcha URL
when needed. (Such as use recaptcha.net instead of www.google.com)

This feature needs manual test cuz it involves 3rd party _captcha_.

Signed-off-by: `Simon Ding <dxl@plotbridge.com>`

Co-authored-by: dxl <dxl@plotbridge.com>
2022-10-28 11:25:01 +01:00
Neil Alexander
f6dea712d2
Initial support for multiple server names (#2829)
This PR is the first step towards virtual hosting by laying the
groundwork for multiple server names being configured.
2022-10-26 12:59:19 +01:00
Neboer
2a4c7f45b3
Add support for config "auto_join_rooms" (#2823)
Add support for config "auto_join_rooms". Now new accounts can join the
rooms in config file automatically.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
2022-10-26 11:04:53 +02:00
Till Faelligen
0843bd776e
Fix wrong config key 2022-10-24 07:10:50 +02:00
Neil Alexander
d605d928bc
Allow specifying old signing keys with the public key and key ID only (#2770)
If the private key file is lost, it's often possible to retrieve the
public key from another server elsewhere, so we should make it possible
to configure it in that way.
2022-10-06 11:56:00 +01:00
Till
8c0c3441d8
Add RoomEventType nats.Header to avoid unneeded unmarshalling (#2765) 2022-10-05 12:12:42 +02:00
Neil Alexander
3da182212e
Track reasons why the process is in a degraded state 2022-10-04 13:02:41 +01:00
Neil Alexander
34993717fd
Update search docs 2022-09-27 17:10:47 +01:00
Till
87be32ca26
Fulltext implementation using Bleve (#2675)
Based on #2480

This actually indexes events based on their event type. They are removed
from the index if we receive a `m.room.redaction` event on the
`OutputRoomEvent` stream.
An admin endpoint is added to reindex all existing events.


Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 18:06:49 +02:00
Neil Alexander
6c67552bf9
Return M_UNRECOGNIZED for unknown CS API endpoints/actions (#2740)
Fixes #2739.
2022-09-27 15:50:22 +01:00
Till
249b32c4f3
Refactor notifications (#2688)
This PR changes the handling of notifications
- removes the `StreamEvent` and `ReadUpdate` stream
- listens on the `OutputRoomEvent` stream in the UserAPI to inform the
SyncAPI about unread notifications
- listens on the `OutputReceiptEvent` stream in the UserAPI to set
receipts/update notifications
- sets the `read_markers` directly from within the internal UserAPI

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 15:01:34 +02:00
Neil Alexander
d8b19c857f
HTTP connection keepalives (#2730)
Beforehand we disabled HTTP keepalives to prevent ambient system
resources from being used by excess idle connections. Now that we've
fixed some bugs in the federation API and device list updater, this
situation is now much better and we don't open so many remote
connections anyway.

Keepalives allow us to not have to handshake TLS so often (which is
quite expensive) and reusing an idle connection is much faster than
having to open a new one. This can help with response times when talking
to remote federated servers.

This PR also adds a new option to disable keepalives if needed:

```
  # Disable HTTP keepalives, which also prevents connection reuse. Dendrite will typically
  # keep HTTP connections open to remote hosts for 5 minutes as they can be reused much
  # more quickly than opening new connections each time. Disabling keepalives will close
  # HTTP connections immediately after a successful request but may result in more CPU and
  # memory being used on TLS handshakes for each new connection instead.
  disable_http_keepalives: false
```
2022-09-20 17:17:44 +01:00
Neil Alexander
bd39748b5c
Update dependencies (#2729)
This updates Dendrite dependencies.
2022-09-20 15:01:19 +01:00
Till Faelligen
af9a204cc0 Only verify if fulltext indexing is enabled 2022-09-09 17:19:29 +02:00
Till
d5876abbe9
Fulltext implementation incl. config (#2480)
This adds the main component of the fulltext search.
This PR doesn't do anything yet, besides creating an empty fulltextindex
folder if enabled. Indexing events is done in a separate PR.
2022-09-07 18:15:54 +02:00