Commit graph

2599 commits

Author SHA1 Message Date
Till
ce2bfc3f2e
Make tests more reliable (#2948)
When using `testrig.CreateBase` and then using that base for other
`NewInternalAPI` calls, we never actually shutdown the components.
`testrig.CreateBase` returns a `close` function, which only removes the
database, so still running components have issues connecting to the
database, since we ripped it out underneath it - which can result in
"Disk I/O" or "pq deadlock detected" issues.
2023-01-20 12:45:56 +01:00
Neil
738686ae68
Add /_dendrite/admin/purgeRoom/{roomID} (#2662)
This adds a new admin endpoint `/_dendrite/admin/purgeRoom/{roomID}`. It
completely erases all database entries for a given room ID.

The roomserver will start by clearing all data for that room and then
will generate an output event to notify downstream components (i.e. the
sync API and federation API) to do the same.

It does not currently clear media and it is currently not implemented
for SQLite since it relies on SQL array operations right now.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-01-19 21:02:32 +01:00
genofire
67f5c5bc1e
fix(helm): extract image tag to value (and use as default from Chart.… (#2934)
improve image tag handling on the default helm way.
with usage of appVersion from:

0995dc4822/helm/dendrite/Chart.yaml (L4)

maybe you like to review @S7evinK ?

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Geno <geno+dev@fireorbit.de>`
2023-01-18 08:45:34 +01:00
Till
b55a7c238f
Version 0.10.9 (#2942) 2023-01-17 19:04:02 +01:00
Till
0d0280cf5f
/sync performance optimizations (#2927)
Since #2849 there is no limit for the current state we fetch to
calculate history visibility. In large rooms this can cause us to fetch
thousands of membership events we don't really care about.
This now only gets the state event types and senders in our timeline,
which should significantly reduce the amount of events we fetch from the
database.

Also removes `MaxTopologicalPosition`, as it is an unnecessary DB call,
given we use the result in `topological_position < $1` calls.
2023-01-17 10:08:23 +01:00
Umar Getagazov
8582c7520a
Omit state field from /messages response if empty (#2940)
The field type is `[ClientEvent]` in the
[spec](https://spec.matrix.org/v1.5/client-server-api/#get_matrixclientv3roomsroomidmessages),
but right now `null` can also be returned. Omit the field completely if
it's empty. Some clients (rightfully) assume it's either not present at
all or it's of the right type (see
https://github.com/matrix-org/matrix-react-sdk/pull/9913).

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
    * The PR is a simple struct tag fix
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Umar Getagazov <umar@handlerug.me>`

Signed-off-by: Umar Getagazov <umar@handlerug.me>
2023-01-17 09:07:42 +01:00
Andrew Morgan
eeeb3017d6
Switch the default config option values for recaptcha_sitekey_class and recaptcha_form_field (#2939)
Attempting to use the [web auth fallback
mechanism](https://spec.matrix.org/v1.5/client-server-api/#fallback) for
Google ReCAPTCHA with the default setting for
`client_api.recaptcha_sitekey_class` of "g-recaptcha-response" results
in no captcha being rendered:


![image](https://user-images.githubusercontent.com/1342360/212482321-14980045-6e20-4d59-adaa-59a01ad88367.png)

I cross-checked the captcha code between [dendrite.matrix.org's fallback
page](https://dendrite.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
and [matrix-client.matrix.org's
one](https://matrix-client.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
(which both use the same captcha public key) and noticed a discrepancy
in the `class` attribute of the div that renders the captcha.
[ReCAPTCHA's docs
state](https://developers.google.com/recaptcha/docs/v3#automatically_bind_the_challenge_to_a_button)
to use "g-recaptcha" as the class for the submit button.

I noticed this when user `@parappanon:parappa.party` reported that they
were also seeing no captcha being rendered on their Dendrite instance.
Changing `client_api.recaptcha_sitekey_class` to "g-recaptcha" caused
their captcha to render properly as well.

There may have been a change in the class name from ReCAPTCHA v2 to v3?
The [docs for
v2](https://developers.google.com/recaptcha/docs/display#auto_render)
also request one uses "g-recaptcha" though.

Thus I propose changing the default setting to unbreak people's
recaptcha auth fallback pages. Should fix dendrite.matrix.org as well.
2023-01-16 12:52:30 +01:00
Devon Hudson
477a44faa6
Always initialize statistics server map 2023-01-12 09:23:03 -07:00
Till
0491a8e343
Fix room summary returning wrong heroes (#2930)
This should fix #2910.
Probably makes Sytest/Complement a bit upset, since this not using
`sort.Strings` anymore.
2023-01-12 10:06:03 +01:00
Devon Hudson
25dfbc6ec3
Extend cypress test timeout in ci 2023-01-11 10:47:37 -07:00
Devon Hudson
6ae1dd565c
Revert "Add cypress cloud recording"
This reverts commit b297ea7379.
2023-01-11 10:46:52 -07:00
Devon Hudson
b297ea7379
Add cypress cloud recording 2023-01-11 10:40:38 -07:00
Devon Hudson
8fef692741
Edit cypress config before running tests 2023-01-11 10:10:24 -07:00
Devon Hudson
11a07d855d
Initial attempt at adding cypress tests to ci 2023-01-11 09:52:58 -07:00
Devon Hudson
97ebd72b5a
Add FAQs based on commonly asked questions from the community 2023-01-10 16:26:41 -07:00
devonh
7482cd2b47
Handle DisplayName field in admin user registration endpoint (#2935)
`/_synapse/admin/v1/register` has a `displayname` field that we were
previously ignoring.
This handles that field and adds the displayname to the new user if one
was provided.
2023-01-10 18:09:25 +00:00
Till
b0c5af6674
Fix /login issue causing wrong device list updates (#2922)
Fixes https://github.com/matrix-org/dendrite/issues/2914 and possibly
https://github.com/matrix-org/dendrite/issues/2073?
2023-01-10 17:02:38 +01:00
Devon Hudson
0995dc4822
Add curl to dendrite-demo-pinecone docker container 2023-01-06 12:02:43 -07:00
Devon Hudson
54b47a98e5
Add curl to dendrite docker containers 2023-01-06 11:49:59 -07:00
Till Faelligen
3fd95e60cc
Try that again 2023-01-06 15:54:04 +01:00
Till Faelligen
002310390f
Output to docs folder, hopefully 2023-01-06 15:51:07 +01:00
Till
d579ddb8e7
Add simplified helm chart (#2905)
As discussed yesterday, a simplified version of [my
helm](https://github.com/S7evinK/dendrite-helm) which deploys a monolith
with internal NATS and an optionally enabled PostgreSQL server. If the
PostgreSQL dependency is not enabled, a user specified connection string
is constructed.

Co-authored-by: kegsay <kegan@matrix.org>
2023-01-06 15:44:10 +01:00
Till
2e1fe58937
Fix backfilling (#2926)
This should fix https://github.com/matrix-org/dendrite/issues/2923
2023-01-05 09:24:00 +01:00
Till
e449d174cc
Add possibility to run complement with coverage enabled (#2901)
This adds the possibility to run Complement with coverage enabled.
In combination with https://github.com/matrix-org/complement/pull/566 we
should then be able to extract the coverage logs, combine them with
https://github.com/wadey/gocovmerge (or similar) and upload them to
Codecov (with different flags, depending on SQLite, HTTP etc.)
2022-12-23 14:28:15 +01:00
Till
f762ce1050
Add clientapi tests (#2916)
This PR
- adds several tests for the clientapi, mostly around `/register` and
auth fallback.
- removes the now deprecated `homeserver` field from responses to
`/register` and `/login`
- slightly refactors auth fallback handling
2022-12-23 14:11:11 +01:00
Till
f47515e38b
Pushrule tweaks, make pattern non-optional on EventMatchCondition (#2918)
This should fix https://github.com/matrix-org/dendrite/issues/2882
(Tested with FluffyChat 1.7.1)
Also adds tests that the predefined push rules (as per the spec) is what
we have in Dendrite.
2022-12-23 12:52:47 +01:00
Till
5eed31fea3
Handle guest access [1/2?] (#2872)
Needs https://github.com/matrix-org/sytest/pull/1315, as otherwise the
membership events aren't persisted yet when hitting `/state` after
kicking guest users.

Makes the following tests pass:
```
Guest users denied access over federation if guest access prohibited
Guest users are kicked from guest_access rooms on revocation of guest_access
Guest users are kicked from guest_access rooms on revocation of guest_access over federation
```

Todo (in a follow up PR):
- Restrict access to CS API Endpoints as per
https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-22 13:05:59 +01:00
Till Faelligen
09dff951d6
More flakey tests 2022-12-22 13:04:32 +01:00
Till
d1d2d16738
Fix reset password endpoint (#2921)
Fixes the admin password reset endpoint.
It was using a wrong variable, so could not detect the user.
Adds some more checks to validate we can actually change the password.
2022-12-22 11:54:03 +01:00
Till Faelligen
beea2432e6
Fix flakey test 2022-12-22 11:31:54 +01:00
Till
d3db542fbf
Add federation peeking table tests (#2920)
As the title says, adds tests for inbound/outbound peeking federation
table tests.

Also removes some unused code
2022-12-22 10:56:20 +01:00
Kento Okamoto
76db8e90de
Dendrite Documentation Fix (#2913)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->
I was reading through the Dendrite documentation on
https://matrix-org.github.io/dendrite/development/contributing and
noticed the installation link leads to a 404 error. This link works fine
if it is viewed directly from
[docs/CONTRIBUTING.md](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md)
but this might not be very obvious to new contributors who are reading
through the [contribution
page](https://matrix-org.github.io/dendrite/development/contributing)
directly.

This PR is mainly a small re-organization of the online documentation
mainly in the
[Development](https://matrix-org.github.io/dendrite/development) tab
along with any links throughout the doc that may be impacted by the
change. This does not contain any Go unit tests as this does not
actually touch core dendrite functionality.

* [ ] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Kento Okamoto <kentokamoto@proton.me>`
2022-12-12 16:46:37 +00:00
Till
7d2344049d
Cleanup stale device lists for users we don't share a room with anymore (#2857)
The stale device lists table might contain entries for users we don't
share a room with anymore. This now asks the roomserver about left users
and removes those entries from the table.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-12-12 08:20:59 +01:00
Till Faelligen
aaf4e5c865
Use older sytest-dendrite image 2022-12-09 18:45:42 +01:00
dependabot[bot]
8846de7312
Bump nokogiri from 1.13.9 to 1.13.10 in /docs (#2909)
Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.13.9
to 1.13.10.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/sparklemotion/nokogiri/releases">nokogiri's
releases</a>.</em></p>
<blockquote>
<h2>1.13.10 / 2022-12-07</h2>
<h3>Security</h3>
<ul>
<li>[CRuby] Address CVE-2022-23476, unchecked return value from
<code>xmlTextReaderExpand</code>. See <a
href="https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-qv4q-mr5r-qprj">GHSA-qv4q-mr5r-qprj</a>
for more information.</li>
</ul>
<h3>Improvements</h3>
<ul>
<li>[CRuby] <code>XML::Reader#attribute_hash</code> now returns
<code>nil</code> on parse errors. This restores the behavior of
<code>#attributes</code> from v1.13.7 and earlier. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2715">#2715</a>]</li>
</ul>
<hr />
<p>sha256 checksums:</p>

<pre><code>777ce2e80f64772e91459b943e531dfef387e768f2255f9bc7a1655f254bbaa1
nokogiri-1.13.10-aarch64-linux.gem
b432ff47c51386e07f7e275374fe031c1349e37eaef2216759063bc5fa5624aa
nokogiri-1.13.10-arm64-darwin.gem
73ac581ddcb680a912e92da928ffdbac7b36afd3368418f2cee861b96e8c830b
nokogiri-1.13.10-java.gem
916aa17e624611dddbf2976ecce1b4a80633c6378f8465cff0efab022ebc2900
nokogiri-1.13.10-x64-mingw-ucrt.gem
0f85a1ad8c2b02c166a6637237133505b71a05f1bb41b91447005449769bced0
nokogiri-1.13.10-x64-mingw32.gem
91fa3a8724a1ce20fccbd718dafd9acbde099258183ac486992a61b00bb17020
nokogiri-1.13.10-x86-linux.gem
d6663f5900ccd8f72d43660d7f082565b7ffcaade0b9a59a74b3ef8791034168
nokogiri-1.13.10-x86-mingw32.gem
81755fc4b8130ef9678c76a2e5af3db7a0a6664b3cba7d9fe8ef75e7d979e91b
nokogiri-1.13.10-x86_64-darwin.gem
51d5246705dedad0a09b374d09cc193e7383a5dd32136a690a3cd56e95adf0a3
nokogiri-1.13.10-x86_64-linux.gem
d3ee00f26c151763da1691c7fc6871ddd03e532f74f85101f5acedc2d099e958
nokogiri-1.13.10.gem
</code></pre>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md">nokogiri's
changelog</a>.</em></p>
<blockquote>
<h2>1.13.10 / 2022-12-07</h2>
<h3>Security</h3>
<ul>
<li>[CRuby] Address CVE-2022-23476, unchecked return value from
<code>xmlTextReaderExpand</code>. See <a
href="https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-qv4q-mr5r-qprj">GHSA-qv4q-mr5r-qprj</a>
for more information.</li>
</ul>
<h3>Improvements</h3>
<ul>
<li>[CRuby] <code>XML::Reader#attribute_hash</code> now returns
<code>nil</code> on parse errors. This restores the behavior of
<code>#attributes</code> from v1.13.7 and earlier. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2715">#2715</a>]</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4c80121dc3"><code>4c80121</code></a>
version bump to v1.13.10</li>
<li><a
href="85410e3841"><code>85410e3</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2715">#2715</a>
from sparklemotion/flavorjones-fix-reader-error-hand...</li>
<li><a
href="9fe0761c47"><code>9fe0761</code></a>
fix(cruby): XML::Reader#attribute_hash returns nil on error</li>
<li><a
href="3b9c736bee"><code>3b9c736</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2717">#2717</a>
from sparklemotion/flavorjones-lock-psych-to-fix-bui...</li>
<li><a
href="2efa87b49a"><code>2efa87b</code></a>
test: skip large cdata test on system libxml2</li>
<li><a
href="3187d6739c"><code>3187d67</code></a>
dep(dev): pin psych to v4 until v5 builds in CI</li>
<li><a
href="a16b4bf14c"><code>a16b4bf</code></a>
style(rubocop): disable Minitest/EmptyLineBeforeAssertionMethods</li>
<li>See full diff in <a
href="https://github.com/sparklemotion/nokogiri/compare/v1.13.9...v1.13.10">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nokogiri&package-manager=bundler&previous-version=1.13.9&new-version=1.13.10)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-08 10:19:55 +00:00
Till
c136a450d5
Fix newly joined users presence (#2854)
Fixes #2803 
Also refactors the presence stream to not hit the database for every
user, instead queries all users at once now.
2022-12-08 08:25:03 +01:00
Till
0351618ff4
Add UserAPI util tests (#2907)
This adds some `userapi/util` tests.
2022-12-08 08:24:24 +01:00
Till
27a1dea522
Fix issue with multiple/duplicate log entries during tests (#2906) 2022-12-08 08:24:06 +01:00
Neil Alexander
ba2ffb7da9
Repeatable reads for /sync (#2783)
This puts repeatable reads into all sync streams.

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-06 18:16:17 +00:00
Till Faelligen
ded43e0f2d
Fix issue with sending presence events to invalid servers 2022-12-06 13:27:33 +01:00
kegsay
7583478305
Update contributing guidelines (#2904) 2022-12-05 16:54:01 +00:00
Till Faelligen
b99349b18c
Use test.WithAllDatabases 2022-12-05 16:00:02 +01:00
Till Faelligen
3dc06bea81
Differentiate between project and patch 2022-12-05 15:49:11 +01:00
Till Faelligen
0e6d94757b
Enforce coverage 2022-12-05 15:24:36 +01:00
Till Faelligen
07e8ed13f6
Fix CI and test.WithAllDatabases 2022-12-05 15:09:59 +01:00
Till
e245a26f6b
Enable/Disable internal metrics (#2899)
Basically enables us to use `test.WithAllDatabases` when testing
internal HTTP APIs, as this would otherwise result in Prometheus
complaining about already registered metric names.
2022-12-05 13:53:36 +01:00
Till
b65f89e61e
Add tests for the AS internal API (#2898) 2022-12-02 16:42:23 +01:00
Till
9a46d8d95c
Test and CI related changes (#2896)
In an attempt to:
- make on-boarding a bit easier (`go test ./...` should now not need
additional postgres setup)
- get code coverage faster, not only scheduled at night
- test the `create-account` binary
2022-12-02 11:44:20 +01:00
Neil Alexander
934056f21f
Fix dendrite-demo-pinecone, /_dendrite namespace setup 2022-12-01 10:45:15 +00:00
Neil Alexander
1be0afa181
Expose /_dendrite and /_synapse on the P2P demo HTTP muxes 2022-12-01 10:24:17 +00:00