Our search was using a linear search over the message text data.
This is very inefficient for large databases and barely usable.
We can add an FTS index, trading storage for speed.
As it's setup, this only supports English, but then we get fancy
stemming so that say "work" matches itself as well as "working".
This could be reduced to just stripping funny chars, with less good
search in the English case.
If we specify the spec in the config file, we can't manually
specify a specific test file from the cli.
This is annoying, as the alternative is copying out the full
package.json blurb into the shell.
Rather, give the spec in the invocation and add a helper
that makes testing a specific file simple.
With this `yarn test:nospec test/plugins/link.ts` will only run
tests within that file
Tests should run the tests, not the coverage.
Frequently one is debugging a test, the coverage won't change
between runs but it delays the cycle considerably.
Rather, if one wants to look at the coverage, one should use
the "coverage" command
Introduce the ability to clean up old messages from the sqlite db.
The StoragePolicy can be chosen by the user. Currently there's
two versions, delete everything based on age is the obvious.
The other is for the data hoarders among us. It'll only delete
message types which can be considered low value... Types with
a time aspect like away / back... joins / parts etc.
It tries to do that in a sensible way, so that we don't block
all other db writers that are ongoing.
The "periodically" interval is by design not exposed to the user.
Make the cleaner available to users by exposing it as a subcommand
to thelounge storage.
This is recommended to be run whenever the storage policy significantly
changes in a way that makes many messages eligible for deletion.
The cleaner would cope, but it'll be inefficient and can take many hours.
Due to how storage works in sqlite, the space would not actually be
given back to the OS, just marked for future writes.
Hence this also runs a vacuum to compact the DB as much as it can.
Once this is getting hooked up, it'll periodically delete old
messages.
The StoragePolicy can be chosen by the user, currently there's
two versions, delete everything based on age is the obvious.
The other is for the data hoarders among us. It'll only delete
message types which can be considered low value... Types with
a time aspect like away / back... joins / parts etc.
It tries to do that in a sensible way, so that we don't block
all other db writers that are ongoing.
The "periodically" interval is by design not exposed to the user.
This is laying the foundation to build a cleaning task that's
sort of database agnostic.
All calls are done by acting on a "DeletionRequest" so interpretation
of the config will go through a single point
A user reported in the IRC chan that installing packages fails with
```
2023-12-13 20:02:34 [INFO] Installing thelounge-theme-solarized v1.1.9...
undefined:1
(node:3329) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
^
SyntaxError: Unexpected token '(', "(node:3329"... is not valid JSON
```
Now, this happens as yarn helpfully prints a deprecation warning
that is shown in the stack trace.
Let's assume that we may get non json messages and log them at debug, as we
don't know their severity.
