Clean up error handling, and report a better error message
on search and flush if FTS5 tables haven't yet been created.
Add some mypy comments to clean up type-checking errors.
Use SQLite's FTS5 extension to power full-text search without any
additional dependencies. FTS5 was introduced in SQLite 3.9.0,
[released][1] in 2015 so should be available on most SQLite
installations at this point in time.
[1]: https://www.sqlite.org/changes.html#version_3_9_0
Add space after any close tag to ensure that
tokens that would be rendered separate in HTML
get extracted as separate tokens in text.
Example:
`<p>First</p><p>Second</p>` --> `First Second`
NOT `FirstSecond`
singlefile.html contains a lot of large strings in the form of `data:`
URLs, which can be unnecessarily stored in full-text indices. Also,
large chunks of JavaScript shouldn't be indexed, either, as they pollute
search results for searches about JS functions, etc.
This commit takes a blanket approach of parsing singlefile.html as it is
read and only outputting text and selected textual attributes (like
`alt`) for indexing.
This slightly modifies the HTML_TITLE_REGEX to fix two parsing errors.
The first occurred when title tags were empty (e.g. "<title></title>")
which was parsed as "</title". The second occurred when titles were a
single character (e.g. "<title>A</title>") which was not matched by the
regex, and so would fall back to link.base_url.
Now when tags are empty, it falls back to link.base_url, and single
character titles are parsed correctly.
The way the regex works now is still a bit wonky for some edge cases.
I couldn't find any cases of incorrect behavior, but it still might be
worth reworking more completely for robustness.
Most frustratingly, the outdated docker-compose link prompts users to
download an older version of the docker-compose.yml file, which packages
a broken YouTube retrieval method.
