Hojun-Cho/strans
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

data(hanja): words, not only syllables

Browse Source 
The import kept only readings of one syllable, so the Hanja search could
convert 한 but never 한자, 학교, or 대한민국 — the conversion every other
Korean input method offers.  libhangul's table has 187k readings; both
scripts now keep them all, and the search finds a word as readily as a
syllable.  The daemon pays for it: 24 MB instead of 12, and 170 ms to
start instead of 30.
This commit is contained in:
Hojun-Cho
2026-08-17 12:50:05 +09:00
parent 3527bcd489
commit 2e53627b7d
7 changed files with 424873 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
map/README
View File

@@ -3,18 +3,19 @@
## Korean Hanja data
`hanja.src` is the tracked, reviewable source for Korean Hanja conversion.
Every data row is exactly one Hanja character, a tab, and one modern Hangul
syllable:
Every data row is Hanja, a tab, and its modern Hangul reading, a character
or a word:
```
漢 한
漢字 한자
```
It contains no word rows such as `견출지` or `방학`. `mkhanja` validates this
contract and groups rows by their Hangul reading to produce the existing
runtime dictionary format. Candidate order follows source order. The source
keeps all retained pairs for review; the generated dictionary stores the first
128 candidates per reading because that is the engine's lookup limit.
`mkhanja` validates that contract and groups rows by their reading to
produce the runtime dictionary. Candidate order follows source order. The
source keeps all retained pairs for review; the generated dictionary stores
the first 128 candidates per reading because that is the engine's lookup
limit.
The source is derived from libhangul release tag `libhangul-0.2.0`. The
annotated tag object is `20afc38922e3595ee3ed5b186f2ea05afe663763`, its
@@ -30,11 +31,10 @@ map/libhangul2hanja upstream >map/hanja.src
map/mkhanja >map/hanja.dict
```
`libhangul2hanja` retains only rows whose reading is one modern Hangul
syllable and whose value is one Hanja character supported by the current
popup: U+3400–U+4DBF, U+4E00–U+9FFF, or U+F900–U+FAFF. Thus word readings,
multi-character values, jamo readings, and supplementary-plane ideographs
are omitted. The import preserves the upstream license header and row order.
`libhangul2hanja` retains only rows whose reading is modern Hangul syllables
throughout and whose value is Hanja the popup can draw: U+3400–U+4DBF,
U+4E00–U+9FFF, or U+F900–U+FAFF. Thus jamo readings, mixed values, and
supplementary-plane ideographs are omitted. The import preserves the upstream license header and row order.
The retained data is BSD 3-Clause licensed by Choe Hwanjin; the complete text
is in `LICENSES/BSD-3-Clause-libhangul-hanja.txt`.