This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
### Pull Request Checklist
<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->
* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
---
I do not know, how you run helm-docs ....
otherwise i would like to add somewhere:
````markdown
* Works well with [Prometheus Operator](https://prometheus-operator.dev/) ([Helmchart](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack)) and there setup of [Grafana](https://grafana.com/grafana/), by enabling following values:
```yaml
prometheus:
servicemonitor:
enabled: true
labels:
release: "kube-prometheus-stack"
rules:
enabled: true # will deploy alert rules
additionalLabels:
release: "kube-prometheus-stack"
grafana:
dashboards:
enabled: true # will deploy default dashboards
```
PS: The labels `release=kube-prometheus-stack` is setup with the helmchart of the Prometheus Operator. For Grafana Dashboards it maybe need scan enable to scan in correct namespaces (or ALL), enabled by `sidecar.dashboards.searchNamespace` in [Helmchart of grafana](https://artifacthub.io/packages/helm/grafana/grafana) (which is part of PrometheusOperator, so `grafana.sidecar.dashboards.searchNamespace`)
````
Maybe also put somewhere the Screenshot of that Grafana Dashboard:
https://grafana.com/grafana/dashboards/13916-dendrite/
---
@S7evinK do you take a look?
Signed-off-by: genofire <geno+dev@fireorbit.de>
This PR changes a few things:
- It pulls out the creation of several NIDs from the `StoreEvent`
function to make the functions more reusable
- Uses more caching when using those NIDs to avoid DB round trips
I guess tests for the logging is rather unusual so I omitted tests for
this change.
* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
Signed-off-by: `David Schneider <dsbrng25b@gmail.com>`
---------
Signed-off-by: David Schneider <dsbrng25b@gmail.com>
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to
0.7.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8e2b117aee"><code>8e2b117</code></a>
http2/hpack: avoid quadratic complexity in hpack decoding</li>
<li><a
href="547e7edf38"><code>547e7ed</code></a>
http2: avoid referencing ResponseWrite.Write parameter after
returning</li>
<li><a
href="39940adcaa"><code>39940ad</code></a>
html: parse comments per HTML spec</li>
<li><a
href="87ce33ecb4"><code>87ce33e</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="415cb6d518"><code>415cb6d</code></a>
all: fix some comments</li>
<li><a
href="7e3c19ca52"><code>7e3c19c</code></a>
all: correct typos in comments</li>
<li><a
href="296f09aa38"><code>296f09a</code></a>
http2: case insensitive handling for 100-continue</li>
<li><a
href="f8411da775"><code>f8411da</code></a>
nettest: fix tests on dragonfly and js/wasm</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.5.0...v0.7.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.1.0
to 0.5.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e6c2a4cdd5"><code>e6c2a4c</code></a>
tiff: don't pre-allocate giant slices before reading</li>
<li><a
href="3db422c472"><code>3db422c</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="bb712eb67b"><code>bb712eb</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="9b8a3be425"><code>9b8a3be</code></a>
font/plan9font: limit the font image's dimensions</li>
<li><a
href="c5235ae222"><code>c5235ae</code></a>
font/plan9font: fix byteoffset for non-zero origin</li>
<li><a
href="0888fdd524"><code>0888fdd</code></a>
font/plan9font: fix bounds overflow</li>
<li><a
href="9fdfde75ec"><code>9fdfde7</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/image/compare/v0.1.0...v0.5.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This fixes an issue with device_id/session_ids.
If a `device_id` is reused, we would reuse the same `session_id`, since
we delete one device and insert a new one directly, resulting in the
query to get a new `session_id` to return the previous session_id.
(`SELECT count(access_token)`)
This removes most of the code used for polylith/API mode.
This removes the `/api` internal endpoints entirely.
Binary size change roughly 5%:
```
51437560 Feb 13 10:15 dendrite-monolith-server # old
48759008 Feb 13 10:15 dendrite-monolith-server # new
```
Should fix the following issues or make a lot less worse when using
Postgres:
The main issue behind #2911: The client gives up after a certain time,
causing a cascade of context errors, because the response couldn't be
built up fast enough. This mostly happens on accounts with many rooms,
due to the inefficient way we're getting recent events and current state
For #2777: The queries for getting the membership events for history
visibility were being executed for each room (I think 185?), resulting
in a whooping 2k queries for membership events. (Getting the
statesnapshot -> block nids -> actual wanted membership event)
Both should now be better by:
- Using a LATERAL join to get all recent events for all joined rooms in
one go (TODO: maybe do the same for room summary and current state etc)
- If we're lazy loading on initial syncs, we're now not getting the
whole current state, just to drop the majority of it because we're lazy
loading members - we add a filter to exclude membership events on the
first call to `CurrentState`.
- Using an optimized query to get the membership events needed to
calculate history visibility
---------
Co-authored-by: kegsay <kegan@matrix.org>
This adds Sytest and Complement coverage reporting to the nightly
scheduled CI runs.
Fixes a few API mode related issues as well, since we seemingly never
really ran them with Complement.
Also fixes a bug related to device list changes: When we pass in an
empty `newlyLeftRooms` slice, we got a list of all currently joined
rooms with the corresponding members. When we then got the
`newlyJoinedRooms`, we wouldn't update the `changed` slice, because we
already got the user from the `newlyLeftRooms` query. This is fixed by
simply ignoring empty `newlyLeftRooms`.
Bumps [activesupport](https://github.com/rails/rails) from 6.0.5 to
6.0.6.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/rails/rails/releases">activesupport's
releases</a>.</em></p>
<blockquote>
<h2>v6.0.6.1</h2>
<h2>Active Support</h2>
<ul>
<li>No changes.</li>
</ul>
<h2>Active Model</h2>
<ul>
<li>No changes.</li>
</ul>
<h2>Active Record</h2>
<ul>
<li>
<p>Make <code>sanitize_as_sql_comment</code> more strict</p>
<p>Though this method was likely never meant to take user input, it was
attempting sanitization. That sanitization could be bypassed with
carefully crafted input.</p>
<p>This commit makes the sanitization more robust by replacing any
occurrances of "/<em>" or "</em>/" with "/
<em>" or "</em> /". It also performs a
first pass to remove one surrounding comment to avoid compatibility
issues for users relying on the existing removal.</p>
<p>This also clarifies in the documentation of annotate that it should
not
be provided user input.</p>
<p>[CVE-2023-22794]</p>
</li>
</ul>
<h2>Action View</h2>
<ul>
<li>No changes.</li>
</ul>
<h2>Action Pack</h2>
<ul>
<li>No changes.</li>
</ul>
<h2>Active Job</h2>
<ul>
<li>No changes.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="28bb76d3ef"><code>28bb76d</code></a>
Version 6.0.6.1</li>
<li><a
href="91cf62e7b4"><code>91cf62e</code></a>
Version 6.0.6</li>
<li><a
href="c7d64e91b6"><code>c7d64e9</code></a>
Preparing for 6.0.5.1 release</li>
<li><a
href="c177e45858"><code>c177e45</code></a>
updating version and changelog</li>
<li>See full diff in <a
href="https://github.com/rails/rails/compare/v6.0.5...v6.0.6.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This extends the dendrite monolith for pinecone to integrate the s&f
features into the mobile apps.
Also makes a few tweaks to federation queueing/statistics to make some
edge cases more robust.
I have added/copied a landing page like Synpase does.
Recently I have installed Dendrite and was wondering why it´s not
working. After some troubleshooting I figured out there is no landing
page like synpase has, so the Server was running just fine.
Hopefuly this PR can fix this problem and may help other users who run
into this issue.
I have not written any unit tests, because it´s just a simple landing
page with a redirect to a static site.
### Pull Request Checklist
<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->
* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
Signed-off-by: `Lukas Huida<lukas@leucali.net>`
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
This adds store & forward relays into dendrite for p2p.
A few things have changed:
- new relay api serves new http endpoints for s&f federation
- updated outbound federation queueing which will attempt to forward
using s&f if appropriate
- database entries to track s&f relays for other nodes
This should hopefully finally fix issues about `disk I/O error` as seen
[here](https://gitlab.alpinelinux.org/alpine/aports/-/jobs/955030/raw)
Hopefully this will also fix `SSL accept attempt failed` issues by
disabling HTTP keep alives when generating a config for CI.
This PR implements
[MSC3904](https://github.com/matrix-org/matrix-spec-proposals/pull/3904).
This PR is almost identical to #2781 but this PR is also filed well
technically 1 day before the MSC passes FCP but well everyone knows this
MSC is expected to have passed FCP on monday so im refiling this change
today on saturday as i was doing prep work for monday. I assume that
this PR wont be counted as clogging the queue since by the next time i
expect to be a work day for this project this PR will be implementing an
FCP passed disposition merge MSC.
Also as for the lack of tests i belive that this simple change does not
need to pass new tests due to that these tests are expected to already
have been passed by the successful use of Dendrite with Room version 10
already.
### Pull Request Checklist
* [X] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [X] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)
Signed-off-by: Catalan Lover <catalanlover@protonmail.com>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: kegsay <kegan@matrix.org>
