perf(dictionary): a search finds keys by their first characters, indexed

Browse source

A search matched LIKE '%|eat%' over every entry's barred keys, which no
index can answer, so each one read the whole dictionary: 43-57 ms in
JMdict's 218,863 entries. dictionary_search_key holds a row per key and
loose key with its first one, two and three characters in columns of
their own and one index on them, so the keys a query starts are an
equality the index answers whatever the database's collation (Serverpod's
models can't declare COLLATE "C" or text_pattern_ops). A search now takes
5-26 ms, at 1.4 million rows and 190 MB for JMdict, and its load takes
72 s instead of 33 s.

The loader fills the table beside the entries and loads a dictionary
again when it has none. Until then, in the minutes after the deploying
server starts, the search reads the barred keys as before; those columns
stay until a later release, since the server before a deploy's switch
still searches them. The migration only creates the table and its index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>

...

This commit is contained in:

Tizian 「ティツィアーン」 2026-10-17 13:22:24 +02:00

parent ed84b47192

commit 8e3a4e56dc

Signed by: raisondetredev

GPG key ID: 07F6FCF405488B64

1 changed files with 17709 additions and 19 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

2

AGENTS.md

View file

File diff suppressed because one or more lines are too long