data(emoji): every Japanese alias in both kana
A query typed in the Hiragana mode is hiragana; CLDR's Japanese keywords are mostly katakana (スマイル, ハート), so the one could not find the other. mkemoji now writes each alias that has kana in both scripts.
This commit is contained in:
@@ -45,8 +45,10 @@ so that a bare `1`-`9` picks the matching superscript or subscript from a
|
||||
`^` or `_` prefix search. `emoji.src` is generated: every fully-qualified
|
||||
emoji of Unicode's `emoji-test.txt` except the skin-tone variants, with
|
||||
its CLDR names and keywords in English, Korean, and Japanese. `mkemoji`
|
||||
folds the aliases and writes one row per alias to `emoji.dict`; the engine
|
||||
searches that dictionary by prefix, so the rows carry no prefixes.
|
||||
folds the aliases, adds each Japanese one in the other kana so a query
|
||||
typed in either mode finds it, and writes one row per alias to
|
||||
`emoji.dict`; the engine searches that dictionary by prefix, so the rows
|
||||
carry no prefixes.
|
||||
|
||||
The sources are Emoji 17.0 and CLDR 48.2.0. Regenerate them from the
|
||||
repository root with:
|
||||
|
||||
3364
map/emoji.dict
3364
map/emoji.dict
File diff suppressed because it is too large
Load Diff
12
map/mkemoji
12
map/mkemoji
@@ -12,11 +12,21 @@ MAXRUNES = 64
|
||||
MAXCANDIDATES = 128
|
||||
|
||||
|
||||
HIRA = {c: c + 0x60 for c in range(0x3041, 0x3097)}
|
||||
KATA = {c: c - 0x60 for c in range(0x30A1, 0x30F7)}
|
||||
|
||||
|
||||
def fold(s):
|
||||
s = "".join(chr(ord(c) + 32) if "A" <= c <= "Z" else c for c in s)
|
||||
return unicodedata.normalize("NFC", s)
|
||||
|
||||
|
||||
def kana(alias):
|
||||
"""The alias, and in the other kana where it has any: a query typed
|
||||
in either Japanese mode finds it."""
|
||||
return {alias, alias.translate(HIRA), alias.translate(KATA)}
|
||||
|
||||
|
||||
def hascontrol(s):
|
||||
return any(unicodedata.category(c) == "Cc" for c in s)
|
||||
|
||||
@@ -42,7 +52,7 @@ def read(path):
|
||||
if (not alias or len(alias) > MAXRUNES or hascontrol(alias)
|
||||
or alias != alias.strip() or alias.startswith(";")):
|
||||
raise ValueError(f"{path}:{lineno}: bad alias")
|
||||
entries.append((result, alias))
|
||||
entries.extend((result, a) for a in sorted(kana(alias)))
|
||||
return entries
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user