data(emoji): every emoji, with its CLDR names in English, Korean, and Japanese

emoji.src was a seed of thirty emoji.  cldr2emoji now generates it from
Unicode's emoji-test.txt (Emoji 17.0) and CLDR 48.2.0's annotations: the
1914 fully-qualified emoji without their skin-tone variants, each with
its names and keywords in the three languages, so that a search finds
what fcitx5's emoji picker finds.  The data is under the Unicode License,
added to LICENSES.

LICENSES/README.md

@@ -6,4 +6,8 @@
 - `GPL-2.0-or-later.txt` contains GNU GPL version 2.  The header of
   `map/kanji.dict` grants the option to use GPL version 2 or any later
   version.
+- `Unicode-3.0.txt` contains the Unicode License v3, the terms for the
+  emoji names and keywords in `map/emoji.src` and generated
+  `map/emoji.dict`, derived from Unicode's `emoji-test.txt` and CLDR's
+  annotations.
 No repository-wide license has been declared for original strans source.

LICENSES/Unicode-3.0.txt

@@ -0,0 +1,39 @@
+UNICODE LICENSE V3
+
+COPYRIGHT AND PERMISSION NOTICE
+
+Copyright © 1991-2026 Unicode, Inc.
+
+NOTICE TO USER: Carefully read the following legal agreement. BY
+DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
+SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
+TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
+DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.
+
+Permission is hereby granted, free of charge, to any person obtaining a
+copy of data files and any associated documentation (the "Data Files") or
+software and any associated documentation (the "Software") to deal in the
+Data Files or Software without restriction, including without limitation
+the rights to use, copy, modify, merge, publish, distribute, and/or sell
+copies of the Data Files or Software, and to permit persons to whom the
+Data Files or Software are furnished to do so, provided that either (a)
+this copyright and permission notice appear with all copies of the Data
+Files or Software, or (b) this copyright and permission notice appear in
+associated Documentation.
+
+THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
+KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
+THIRD PARTY RIGHTS.
+
+IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
+BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
+OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
+WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
+ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
+FILES OR SOFTWARE.
+
+Except as contained in this notice, the name of a copyright holder shall
+not be used in advertising or otherwise to promote the sale, use or other
+dealings in these Data Files or Software without prior written
+authorization of the copyright holder.

README.md

@@ -37,7 +37,10 @@ syllable and goes on to the application, so it still leaves insert mode.
 Hiragana mode composes a complete reading before offering Kanji candidates.
 Katakana mode does not perform Kanji conversion. Emoji and Hanja searches
 return to the previous mode after use; switching language during a search
-commits the shown query. `Ctrl+H` takes the syllable being composed as its
+commits the shown query. The Emoji search matches the typed keys, and what
+they spell in the current language, against a prefix of every emoji's
+CLDR name and keywords in English, Korean, and Japanese, and against ASCII
+symbol aliases such as `->` and `<=`. `Ctrl+H` takes the syllable being composed as its
 query; the Hanja dictionary lists one modern Hangul syllable at a time.
 
 ## Preedit and candidates

map/README

@@ -38,6 +38,33 @@ are omitted.  The import preserves the upstream license header and row order.
 The retained data is BSD 3-Clause licensed by Choe Hwanjin; the complete text
 is in `LICENSES/BSD-3-Clause-libhangul-hanja.txt`.
 
+## Emoji and symbol data
+
+`symbol.src` is hand-written: a symbol, a tab, and its ASCII aliases, kept
+so that a bare `1`-`9` picks the matching superscript or subscript from a
+`^` or `_` prefix search.  `emoji.src` is generated: every fully-qualified
+emoji of Unicode's `emoji-test.txt` except the skin-tone variants, with
+its CLDR names and keywords in English, Korean, and Japanese.  `mkemoji`
+folds the aliases and writes one row per alias to `emoji.dict`; the engine
+searches that dictionary by prefix, so the rows carry no prefixes.
+
+The sources are Emoji 17.0 and CLDR 48.2.0.  Regenerate them from the
+repository root with:
+
+```sh
+curl -L https://www.unicode.org/Public/17.0.0/emoji/emoji-test.txt -o emoji-test.txt
+printf '%s  %s\n' 1d8a944f88d7952f7ef7c5167fef3c67995bcae24543949710231b03a201acda emoji-test.txt | sha256sum -c -
+for l in en ko ja; do
+	curl -L https://raw.githubusercontent.com/unicode-org/cldr-json/48.2.0/cldr-json/cldr-annotations-full/annotations/$l/annotations.json -o $l.json
+	curl -L https://raw.githubusercontent.com/unicode-org/cldr-json/48.2.0/cldr-json/cldr-annotations-derived-full/annotationsDerived/$l/annotations.json -o $l-derived.json
+done
+map/cldr2emoji emoji-test.txt en.json en-derived.json ko.json ko-derived.json ja.json ja-derived.json >map/emoji.src
+map/mkemoji >map/emoji.dict
+```
+
+The data is under the Unicode License v3; the complete text is in
+`LICENSES/Unicode-3.0.txt`.
+
 ## Japanese romaji
 
 `hira.map` and `kata.map` are Mozc's default romaji table, one row per key,

map/cldr2emoji

@@ -0,0 +1,50 @@
+#!/usr/bin/env python3
+"""Write emoji.src: every fully-qualified emoji of Unicode's emoji-test.txt
+but the skin-tone variants, each with its CLDR names and keywords in the
+languages given, as result-first TAB-separated rows for mkemoji."""
+
+import json
+import sys
+from pathlib import Path
+
+
+def emoji(path):
+    out = []
+    for line in path.open(encoding="utf-8"):
+        if "; fully-qualified" not in line:
+            continue
+        cps = [int(cp, 16) for cp in line.split(";")[0].split()]
+        if any(0x1F3FB <= cp <= 0x1F3FF for cp in cps):
+            continue
+        out.append("".join(map(chr, cps)))
+    return out
+
+
+def annotations(path):
+    """CLDR annotations.json, plain or derived: emoji -> names, keywords."""
+    root = json.load(path.open(encoding="utf-8"))
+    table = root[next(iter(root))]["annotations"]
+    return {e: a.get("tts", []) + a.get("default", []) for e, a in table.items()}
+
+
+def main():
+    if len(sys.argv) < 3:
+        print(f"usage: {sys.argv[0]} emoji-test.txt annotations.json...",
+              file=sys.stderr)
+        return 2
+    tables = [annotations(Path(arg)) for arg in sys.argv[2:]]
+    print("# Result first, then its CLDR names and keywords; see map/README.")
+    for e in emoji(Path(sys.argv[1])):
+        aliases = []
+        for table in tables:
+            for alias in table.get(e.replace("️", ""), table.get(e, [])):
+                alias = alias.strip()
+                if alias and alias not in aliases:
+                    aliases.append(alias)
+        if aliases:
+            print(e + "\t" + "\t".join(aliases))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())