12 commits

Author	SHA1	Message	Date
Kegan Dougal	ed4097825b	Factor out StatementList to sqlutil and use it in userapi ... It helps with the boilerplate.	2021-07-28 18:30:04 +01:00
Neil Alexander	5ce1fe80de	State storage refactor (#1839) ... * Hash-deduplicated state storage (and migrations) for PostgreSQL and SQLite * Refactor droomserver database setup for migrations * Fix conflict statements * Update migration names * Set a boundary for old to new block/snapshot IDs so we don't rewrite them more than once accidentally * Create sequence if not exists * Fix boundary queries * Fix boundary queries * Use Query * Break out queries a bit * More sequence tweaks * Query parameters are not playing the game * Injection escaping may not work for CREATE SEQUENCE after all * Fix snapshot sequence name * Use boundaried IDs in SQLite too * Use IFNULL for SQLite * Use COALESCE in PostgreSQL * Review comments @Kegsay	2021-04-26 13:25:57 +01:00
Neil Alexander	91fc1f1c92	Fix bug in error handling in SQLite InsertPreviousEvent (#1456)	2020-10-01 10:55:00 +01:00
Neil Alexander	a06c18bb56	Soft-fail (#1364) ... * Initial work on soft-fail * Fix state block retrieval * Copy-pasta QueryLatestEventsAndState code * Fix state lookup * Clean up * Fix up failing sytest * Linting * Update previous events SQLite insert query * Update SQLite InsertPreviousEvent properly * Hopefully fix the event references updates Co-authored-by: Kegan Dougal <kegan@matrix.org>	2020-09-21 14:55:46 +01:00
Neil Alexander	b24747b305	Transaction writer changes, move roomserver writers (#1285) ... * Updated TransactionWriters, moved locks in roomserver, various other tweaks * Fix redaction deadlocks * Fix lint issue * Rename SQLiteTransactionWriter to ExclusiveTransactionWriter * Fix us not sending transactions through in latest events updater	2020-08-19 15:38:27 +01:00
Neil Alexander	3d58417555	Roomserver database-wide TransactionWriters (#1282) ... * Database-wide TransactionWriter * Fix deadlocking Sync API tests * Undo non-roomserver changes for now	2020-08-19 10:57:29 +01:00
Neil Alexander	d76eb1b994	Use TransactionWriter in roomserver SQLite (#1208)	2020-07-21 10:48:49 +01:00
Kegsay	ecd7accbad	Rehuffle where things are in the internal package (#1122) ... renamed: internal/eventcontent.go -> internal/eventutil/eventcontent.go renamed: internal/events.go -> internal/eventutil/events.go renamed: internal/types.go -> internal/eventutil/types.go renamed: internal/http/http.go -> internal/httputil/http.go renamed: internal/httpapi.go -> internal/httputil/httpapi.go renamed: internal/httpapi_test.go -> internal/httputil/httpapi_test.go renamed: internal/httpapis/paths.go -> internal/httputil/paths.go renamed: internal/routing.go -> internal/httputil/routing.go renamed: internal/basecomponent/base.go -> internal/setup/base.go renamed: internal/basecomponent/flags.go -> internal/setup/flags.go renamed: internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go renamed: internal/postgres.go -> internal/sqlutil/postgres.go renamed: internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go renamed: internal/sql.go -> internal/sqlutil/sql.go	2020-06-12 14:55:57 +01:00
Kegsay	a7f12bce79	Convert remaining roomserver tables (#1065) ... * Convert invites table * Convert membership table * Factor out remaining functions except for *Updater structs * Remove 'implements' comments from long-forgotten interfaces. Move those comments to storage.Database	2020-05-27 11:03:47 +01:00
Kegsay	c0c5d9452a	Convert room_aliases previous_events state_block and state_snapshot tables (#1064) ... * Convert state_snapshot and state_block tables * Convert room_aliases and previous_events tables * Add missing table	2020-05-27 09:36:09 +01:00
Kegsay	24d8df664c	Fix #897 and shuffle directory around (#1054) ... * Fix #897 and shuffle directory around * Update find-lint * goimports Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>	2020-05-21 14:40:13 +01:00
Kegsay	b6ea1bc67a	Support sqlite in addition to postgres (#869) ... * Move current work into single branch * Initial massaging of clientapi etc (not working yet) * Interfaces for accounts/devices databases * Duplicate postgres package for sqlite3 (no changes made to it yet) * Some keydb, accountdb, devicedb, common partition fixes, some more syncapi tweaking * Fix accounts DB, device DB * Update naffka dependency for SQLite * Naffka SQLite * Update naffka to latest master * SQLite support for federationsender * Mostly not-bad support for SQLite in syncapi (although there are problems where lots of events get classed incorrectly as backward extremities, probably because of IN/ANY clauses that are badly supported) * Update Dockerfile -> Go 1.13.7, add build-base (as gcc and friends are needed for SQLite) * Implement GET endpoints for account_data in clientapi * Nuke filtering for now... * Revert "Implement GET endpoints for account_data in clientapi" This reverts commit 4d80dff458. * Implement GET endpoints for account_data in clientapi (#861) * Implement GET endpoints for account_data in clientapi * Fix accountDB parameter * Remove fmt.Println * Fix insertAccountData SQLite query * Fix accountDB storage interfaces * Add empty push rules into account data on account creation (#862) * Put SaveAccountData into the right function this time * Not sure if roomserver is better or worse now * sqlite work * Allow empty last sent ID for the first event * sqlite: room creation works * Support sending messages * Nuke fmt.println * Move QueryVariadic etc into common, other device fixes * Fix some linter issues * Fix bugs * Fix some linting errors * Fix errcheck lint errors * Make naffka use postgres as fallback, fix couple of compile errors * What on earth happened to the /rooms/{roomID}/send/{eventType} routing Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>	2020-02-13 17:27:33 +00:00