Commit graph

318 commits

Author SHA1 Message Date
Kegsay
5caae6f3a0
sqlite: fixes from sytest (#872)
* bugfix: fix panic on new invite events from sytest

I'm unsure why the previous code didn't work, but it's
clearer, quicker and easier to read the `LastInsertID()` way.
Previously, the code would panic as the SELECT would fail
to find the last inserted row ID.

* sqlite: Fix UNIQUE violations and close more cursors

- Add missing `defer rows.Close()`
- Do not have the state block NID as a PRIMARY KEY else it breaks for blocks
  with >1 state event in them. Instead, rejig the queries so we can still
  have monotonically increasing integers without using AUTOINCREMENT (which
  mandates PRIMARY KEY).

* sqlite: Add missing variadic function

* Use LastInsertId because empirically it works over the SELECT form (though I don't know why that is)

* sqlite: Fix invite table by using the global stream pos rather than one specific to invites

If we don't use the global, clients don't get notified about any invites
because the position is too low.

* linting: shadowing

* sqlite: do not use last rowid, we already know the stream pos!

* sqlite: Fix account data table in syncapi by commiting insert txns!

* sqlite: Fix failing federation invite

Was failing with 'database is locked' due to multiple write txns
being taken out.

* sqlite: Ensure we return exactly the number of events found in the database

Previously we would return exactly the number of *requested* events, which
meant that several zero-initialised events would bubble through the system,
failing at JSON serialisation time.

* sqlite: let's just ignore the problem for now....

* linting
2020-02-20 09:28:03 +00:00
Kegsay
b6ea1bc67a
Support sqlite in addition to postgres (#869)
* Move current work into single branch

* Initial massaging of clientapi etc (not working yet)

* Interfaces for accounts/devices databases

* Duplicate postgres package for sqlite3 (no changes made to it yet)

* Some keydb, accountdb, devicedb, common partition fixes, some more syncapi tweaking

* Fix accounts DB, device DB

* Update naffka dependency for SQLite

* Naffka SQLite

* Update naffka to latest master

* SQLite support for federationsender

* Mostly not-bad support for SQLite in syncapi (although there are problems where lots of events get classed incorrectly as backward extremities, probably because of IN/ANY clauses that are badly supported)

* Update Dockerfile -> Go 1.13.7, add build-base (as gcc and friends are needed for SQLite)

* Implement GET endpoints for account_data in clientapi

* Nuke filtering for now...

* Revert "Implement GET endpoints for account_data in clientapi"

This reverts commit 4d80dff458.

* Implement GET endpoints for account_data in clientapi (#861)

* Implement GET endpoints for account_data in clientapi

* Fix accountDB parameter

* Remove fmt.Println

* Fix insertAccountData SQLite query

* Fix accountDB storage interfaces

* Add empty push rules into account data on account creation (#862)

* Put SaveAccountData into the right function this time

* Not sure if roomserver is better or worse now

* sqlite work

* Allow empty last sent ID for the first event

* sqlite: room creation works

* Support sending messages

* Nuke fmt.println

* Move QueryVariadic etc into common, other device fixes

* Fix some linter issues

* Fix bugs

* Fix some linting errors

* Fix errcheck lint errors

* Make naffka use postgres as fallback, fix couple of compile errors

* What on earth happened to the /rooms/{roomID}/send/{eventType} routing

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-13 17:27:33 +00:00
Kiril Vladimiroff
d5dbe546e4
Always defer *sql.Rows.Close and consult with Err (#844)
* Always defer *sql.Rows.Close and consult with Err

database/sql.Rows.Next() makes sure to call Close only after exhausting
result rows which would NOT happen when returning early from a bad Scan.
Close being idempotent makes it a great candidate to get always deferred
regardless of what happens later on the result set.

This change also makes sure call Err() after exhausting Next() and
propagate non-nil results from it as the documentation advises.

Closes #764

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

* Override named result parameters in last returns

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

* Do the same over new changes that got merged

Signed-off-by: Kiril Vladimiroff <kiril@vladimiroff.org>

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 14:12:21 +00:00
Neil Alexander
c20109a573
Implement room version capabilities in CS API (#866)
* Add wiring for querying the roomserver for the default room version

* Try to implement /capabilities for room versions

* Update copyright notices

* Update sytests, add /capabilities endpoint into CS API

* Update sytest-whitelist

* Add GetDefaultRoomVersion

* Fix cases where state package was shadowed

* Fix version formatting

* Update Dockerfile to Go 1.13.6

* oh yes types I remember

* And fix the default too
2020-02-05 18:06:39 +00:00
Neil Alexander
880d8ae024
Room version abstractions (#865)
* Rough first pass at adding room version abstractions

* Define newer room versions

* Update room version metadata

* Fix roomserver/versions

* Try to fix whitespace in roomsSchema
2020-02-05 16:25:58 +00:00
Neil Alexander
49f760a30b
CS API: Support for /messages, fixes for /sync (#847)
* Merge forward

* Tidy up a bit

* TODO: What to do with NextBatch here?

* Replace SyncPosition with PaginationToken throughout syncapi

* Fix PaginationTokens

* Fix lint errors

* Add a couple of missing functions into the syncapi external storage interface

* Some updates based on review comments from @babolivier

* Some updates based on review comments from @babolivier

* argh whitespacing

* Fix opentracing span

* Remove dead code

* Don't overshadow err (fix lint issue)

* Handle extremities after inserting event into topology

* Try insert event topology as ON CONFLICT DO NOTHING

* Prevent OOB error in addRoomDeltaToResponse

* Thwarted by gocyclo again

* Fix NewPaginationTokenFromString, define unit test for it

* Update pagination token test

* Update sytest-whitelist

* Hopefully fix some of the sync batch tokens

* Remove extraneous sync position func

* Revert to topology tokens in addRoomDeltaToResponse etc

* Fix typo

* Remove prevPDUPos as dead now that backwardTopologyPos is used instead

* Fix selectEventsWithEventIDsSQL

* Update sytest-blacklist

* Update sytest-whitelist
2020-01-23 17:51:10 +00:00
Neil Alexander
3bd023e74d
Federation fixes (#845)
* Update gomatrixserverlib to p2p commit 92c0338, other tweaks

* Update gomatrixserverlib to p2p commit e5dcc65

* Rewrite getAuthChain

* Update gomatrixserverlib in go.mod/go.sum

* Correct a couple of package refs for updated gmsl/gomatrix

* Update gomatrixserverlib ref in go.mod/go.sum

* Update getAuthChain comments following @babolivier review
2020-01-10 12:11:44 +00:00
Neil Alexander
9e489845eb Fix #842 - again... 2020-01-09 17:18:19 +00:00
Neil Alexander
714959126b
Fall back to postgres when database connection string parsing fails (#842)
* Fall back to postgres when parsing the database connection string for a URI schema fails

* Fix behaviour so that it really tries postgres when URL parsing fails and it complains about unknown schema if it succeeds
2020-01-09 17:03:36 +00:00
Neil Alexander
c28577ea25
Implement storage interfaces (#841)
* Implement interfaces for federationsender storage

* Implement interfaces for mediaapi storage

* Implement interfaces for publicroomsapi storage

* Implement interfaces for roomserver storage

* Implement interfaces for syncapi storage

* Implement interfaces for keydb storage

* common.PartitionStorer in publicroomsapi interface

* Update copyright notices
2020-01-03 14:07:05 +00:00
S7evinK
4f75e4febe Fix copy & paste error (#812) 2019-12-20 15:00:25 +00:00
Alex Chen
43308d2f3f
Associate transactions with session IDs instead of device IDs (#789) 2019-08-24 00:55:40 +08:00
Alex Chen
94ea325c93
Fix permission and 404 response for alias deletion - #654 (#706) 2019-08-07 11:00:58 +08:00
Alex Chen
f8d2860765 Replace membership and visibility values with constants (#774)
Signed-off-by: Alex Chen <minecnly@gmail.com>
2019-08-06 15:07:36 +01:00
Alex Chen
6106ec1399 Fix getAliasesForRoomID has no HTTP handler in aliasAPI (#705)
This PR adds back the HTTP handler for internal API GetAliasesForRoomID in roomserver, which seemed to be missing.
2019-07-09 17:13:43 +01:00
Serra Allgood
a0dec456c1 rommserver/alias: Do not call appserviceAPI in GetRoomIDForAlias if local alias found #631 (#702)
A conditional is added to wrap the call to appserviceAPI if a local alias is not found in the database.

Fixes #631

Signed-off-by: Serra Allgood <serra@allgood.dev>
2019-06-25 13:43:18 +01:00
Andrew Morgan
bc382bba46
Fix pipeline, emoji and syntax (#713)
Fixes #697

Switched to golangci-lint, fixes issues with buildkite and does some linting fixes to appease the new linters.
2019-06-19 14:05:03 +01:00
ruben
74827428bd use go module for dependencies (#594) 2019-05-21 21:56:55 +01:00