Commit Graph

8 Commits

Author SHA1 Message Date
3527bcd489 trie: one node per rune, children in rune order, and the last path kept
The trie walked a key byte by byte, so a Hangul syllable cost three
nodes and every lookup first spelled its key back into UTF-8.  It walks
runes now: a third of the nodes for Korean data, and no buffer in the
lookup.  Children are kept in rune order, so both the search and the
insert stop early, and a trie remembers the path of the last key put,
which a file sorted by key — hanja.dict is one — walks straight down.
2026-08-17 12:50:05 +09:00
abff5ed122 emoji: search the dictionary by prefix in the engine, not in the data
mkemoji wrote a row for every prefix of every alias, so that a query
matched as it was typed; the trie is a prefix index already, and with a
real emoji list those rows would be four times the aliases themselves.
Now emoji.dict has one row per alias, trienode() names a key's node, and
dictlookup walks the entries at and below it, the key's own first, up to
Maxkouho.  Trie children are appended rather than pushed, so the walk
keeps the file's order and a bare digit still picks the superscript or
subscript it always did.

The hand-written symbol rows move to symbol.src; emoji.src is left to
the emoji.  mkemoji reads both by default, or the files it is given.
2026-08-17 01:07:10 +09:00
c7718ece52 data: one trie for maps and dictionaries, one loader
The hash map served a single exact-match lookup that the trie already
answers, at the price of a second container, a second file loader with
its own drift, and a Str-to-UTF-8 conversion on every chain probe. The
files are small (kanji.dict is 7.5k lines), so the trie holds both.
trieopen validates keys and each space-separated value word against Str
and reports path:line; trielookup takes the Str every caller holds and
treats a nil trie as an unloaded map; trienew/trieput exist for tests.
The overflow guards on growth, Trie.root (always 0) and the per-language
init loop written twice are gone.
2026-08-16 15:56:32 +09:00
bfe83d0119 fix(data): reject partial and malformed map data 2026-08-14 23:09:55 +09:00
8d68e00f2b data: simplify and validate Japanese imports 2026-08-12 17:32:28 +09:00
103f2ec448 fix: harden small core containers 2026-08-12 15:54:16 +09:00
e70278a3d2 fix core text and map ownership 2026-08-11 19:32:58 +09:00
5653298bb1 use trie for map lookup 2026-02-08 16:36:59 +09:00