data(emoji): every Japanese alias in both kana

A query typed in the Hiragana mode is hiragana; CLDR's Japanese
keywords are mostly katakana (スマイル, ハート), so the one could not
find the other.  mkemoji now writes each alias that has kana in both
scripts.
This commit is contained in:
2026-08-17 01:44:29 +09:00
parent 0c4271b8b5
commit fcdce7dc16
3 changed files with 3345 additions and 37 deletions

View File

@@ -45,8 +45,10 @@ so that a bare `1`-`9` picks the matching superscript or subscript from a
`^` or `_` prefix search. `emoji.src` is generated: every fully-qualified
emoji of Unicode's `emoji-test.txt` except the skin-tone variants, with
its CLDR names and keywords in English, Korean, and Japanese. `mkemoji`
folds the aliases and writes one row per alias to `emoji.dict`; the engine
searches that dictionary by prefix, so the rows carry no prefixes.
folds the aliases, adds each Japanese one in the other kana so a query
typed in either mode finds it, and writes one row per alias to
`emoji.dict`; the engine searches that dictionary by prefix, so the rows
carry no prefixes.
The sources are Emoji 17.0 and CLDR 48.2.0. Regenerate them from the
repository root with:

File diff suppressed because it is too large Load Diff

View File

@@ -12,11 +12,21 @@ MAXRUNES = 64
MAXCANDIDATES = 128
HIRA = {c: c + 0x60 for c in range(0x3041, 0x3097)}
KATA = {c: c - 0x60 for c in range(0x30A1, 0x30F7)}
def fold(s):
s = "".join(chr(ord(c) + 32) if "A" <= c <= "Z" else c for c in s)
return unicodedata.normalize("NFC", s)
def kana(alias):
"""The alias, and in the other kana where it has any: a query typed
in either Japanese mode finds it."""
return {alias, alias.translate(HIRA), alias.translate(KATA)}
def hascontrol(s):
return any(unicodedata.category(c) == "Cc" for c in s)
@@ -42,7 +52,7 @@ def read(path):
if (not alias or len(alias) > MAXRUNES or hascontrol(alias)
or alias != alias.strip() or alias.startswith(";")):
raise ValueError(f"{path}:{lineno}: bad alias")
entries.append((result, alias))
entries.extend((result, a) for a in sorted(kana(alias)))
return entries