data(hanja): a lone consonant is a reading too

Every Korean keyboard's 한자 key answers a lone consonant with the KS X
1001 symbol palette, and has since 한글 워드프로세서: ㅁ for ※ ○ △ ㈜, ㄴ
for the brackets, ㄹ for the units, ㅇ for the circled numbers.  strans
sends that key to the same search as a syllable -- Khanja is Ctrl+H at
strans.c:830, and startsearch seeds the query with whatever ko.c left
pending -- but every one of hanja.dict's 187286 readings is a syllable, so
the popup came up with a query in it and nothing to pick:

	ㅁ: 0 candidates
	ㄴ: 0 candidates
	ㄹ: 0 candidates
	한: 99 candidates 韓 漢 寒 限 閑 恨 旱 汗 翰 邯 罕 悍 澣 閒 瀚

libhangul ships that palette beside the Hanja table already imported here:
data/hanja/mssymbol.txt, same commit, same author, same BSD-3 terms, same
key:value:comment format -- and keyed by the compatibility jamo ko.c
already holds, U+3141 for ㅁ.  So the engine does not change at all; the
same dictlookup on the same trie now finds something:

	ㅁ: 75 candidates # & * @ § ※ ☆ ★ ○ ● ◎ ◇ ◆ □ ■ △ ▲ ▽ ▼
	ㄴ: 23 candidates " ( ) [ ] { } ‘ ’ “ ” 〔 〕 〈 〉 《 》 「 」
	ㄹ: 94 candidates $ % ₩ F ′ ″ ℃ Å ¢ £ ¥ ¤ ℉ ‰ € ㎕ ㎖ ㎗ ℓ
	한: 99 candidates 韓 漢 寒 限 閑 恨 旱 汗 翰 邯 罕 悍 澣 閒 瀚

Both scripts widen by one rule -- a syllable reading gives Hanja, a jamo
reading gives a symbol -- and hanja.src regenerates byte for byte as it
was, because upstream's own non-syllable readings are words like ㄱ자집
whose values were never Hanja and still fall out.  985 of mssymbol.txt's
987 rows survive: its ideographic space and its soft hyphen do not, since
a candidate the popup cannot draw is not a candidate, and the row format
separates candidates with a space besides.

The two keyspaces cannot collide -- one is syllables, one is single jamo --
so the 187286 existing rows are unchanged, byte for byte, and 18 rows join
them.  mkhanja takes a source list as mkemoji already does, and keeps each
upstream header, which is why the licence text now appears twice.

89 unit, check-live, check-stress and valgrind all clean.  The five new
assertions were checked by breaking the change five ways: dropping
mssymbol.src from SOURCES, letting issymbol keep a formatting character,
letting a jamo reading keep Hanja, widening isjamo to the vowels, and
making mkhanja reject jamo readings.  Each fails only the tests that exist
for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-18 01:56:23 +09:00
parent 82078a60e7
commit 62ddc2177b
8 changed files with 1211 additions and 55 deletions

View File

@@ -1,11 +1,13 @@
#!/usr/bin/env python3
"""Write hanja.dict from a Hanja-per-row UTF-8 source."""
"""Write hanja.dict from result-per-row UTF-8 sources."""
import sys
import unicodedata
from pathlib import Path
SOURCE = Path(__file__).with_name("hanja.src")
SOURCES = [Path(__file__).with_name(name)
for name in ("hanja.src", "mssymbol.src")]
MAXCANDIDATES = 128
@@ -13,17 +15,29 @@ def ishangul(s):
return s != "" and all(0xAC00 <= ord(c) <= 0xD7A3 for c in s)
def isjamo(s):
return len(s) == 1 and 0x3131 <= ord(s) <= 0x314E
def ishanja(s):
return s != "" and all(0x3400 <= ord(c) <= 0x4DBF
or 0x4E00 <= ord(c) <= 0x9FFF
or 0xF900 <= ord(c) <= 0xFAFF for c in s)
def read(path):
def issymbol(s):
"""One rune the popup can draw and a candidate row can carry: not a
space, which the row separates candidates with, and not a formatting
character, which would leave a blank candidate to pick."""
return (len(s) == 1 and not s.isspace() and not ishanja(s)
and not unicodedata.category(s).startswith("C"))
def read(path, table, seen):
"""Adds path's rows to table, keyed by reading, and returns its header."""
comments = []
table = {}
seen = set()
leading = True
found = False
with path.open(encoding="utf-8") as src:
for lineno, raw in enumerate(src, 1):
line = raw.rstrip("\r\n")
@@ -36,31 +50,36 @@ def read(path):
leading = False
fields = line.split("\t")
if len(fields) != 2:
raise ValueError(f"{path}:{lineno}: need Hanja<TAB>reading")
hanja, reading = fields
if not ishanja(hanja):
raise ValueError(f"{path}:{lineno}: need BMP Hanja")
if not ishangul(reading):
raise ValueError(f"{path}:{lineno}: need Hangul syllables")
pair = (hanja, reading)
raise ValueError(f"{path}:{lineno}: need result<TAB>reading")
value, reading = fields
if ishangul(reading):
if not ishanja(value):
raise ValueError(f"{path}:{lineno}: need BMP Hanja")
elif isjamo(reading):
if not issymbol(value):
raise ValueError(f"{path}:{lineno}: need one symbol rune")
else:
raise ValueError(f"{path}:{lineno}: need a syllable or jamo "
"reading")
pair = (value, reading)
if pair in seen:
raise ValueError(f"{path}:{lineno}: duplicate Hanja reading")
raise ValueError(f"{path}:{lineno}: duplicate reading")
seen.add(pair)
table.setdefault(reading, []).append(hanja)
if not seen:
raise ValueError(f"{path}: no Hanja readings")
return comments, table
found = True
table.setdefault(reading, []).append(value)
if not found:
raise ValueError(f"{path}: no readings")
return comments
def main():
sys.stdout.reconfigure(encoding="utf-8")
sys.stderr.reconfigure(encoding="utf-8")
if len(sys.argv) > 2:
print(f"usage: {sys.argv[0]} [hanja.src]", file=sys.stderr)
return 2
path = Path(sys.argv[1]) if len(sys.argv) == 2 else SOURCE
paths = [Path(arg) for arg in sys.argv[1:]] or SOURCES
table = {}
seen = set()
try:
comments, table = read(path)
comments = [line for path in paths for line in read(path, table, seen)]
for line in comments:
print(line)
if comments: