Iconvex ICU

iconvex_icu is the current-ICU extension for Iconvex. It provides 212 native Elixir codecs backed by 191 package-owned mapping tables. Runtime conversion uses no ICU library, NIF, Port, operating-system iconv, or external process.

The package is deliberately separate from iconvex_icu_archive: this leaf owns current ICU 78.3 converter profiles and native implementations; the archive package owns 1,050 historical revision snapshots.

What is included

Family	Codecs	Notes
ICU UCM single-byte	135	Exact SBCS revision names, aliases, precision flags, and canonical inverse choices
ICU multibyte	30	Complete table-driven MBCS/DBCS profiles, including IBM, EUC, Windows, CNS 11643, and JIS X 0212 mappings
ICU EBCDIC stateful	10	SI/SO mixed SBCS/DBCS converters with canonical shift framing
ICU S/390 `swaplfnl`	16	EBCDIC variants that exchange the LF and NL byte assignments
JIS7/JIS8	2	ICU ISO-2022 Japanese implementation profiles
LMBCS	12	Groups 1, 2, 3, 4, 5, 6, 8, 11, 16, 17, 18, and 19
ICU Unicode variants	6	Versioned UTF-16 plus platform/opposite-endian UTF-16 and UTF-32 profiles
X11 Compound Text	1	ICU's compound-text converter behavior

The checked-in surface manifest is authoritative for all 212 canonical registrations, aliases, implementation modules, table IDs, and source artifacts.

Installation

def deps do
  [
    {:iconvex, "~> 0.1.1"},
    {:iconvex_icu, "~> 0.1.0"}
  ]
end

Starting the OTP application registers the entire package atomically. A failed name or table-provider claim rolls the operation back; stopping the application withdraws exactly its routes and providers.

Examples

All codecs use the normal Iconvex pipeline:

{:ok, encoded} =
  Iconvex.convert("AあいB", "UTF-8", "ibm-930_P120-1999")

<<0xC1, 0x0E, 0x44, 0x81, 0x44, 0x82, 0x0F, 0xC2>> = encoded

{:ok, "AあいB"} =
  Iconvex.convert(encoded, "ibm-930_P120-1999", "UTF-8")

ICU's S/390 newline surface keeps the LF/NL distinction explicit:

{:ok, <<0x15, 0x25>>} =
  Iconvex.convert(<<0x0A::utf8, 0x85::utf8>>, "UTF-8",
    "ibm-1047_P100-1995,swaplfnl"
  )

LMBCS profiles are separate encodings because their optimization groups choose different canonical byte sequences:

{:ok, lmbcs} = Iconvex.convert("Lotus Ελληνικά", "UTF-8", "LMBCS-6")
{:ok, "Lotus Ελληνικά"} = Iconvex.convert(lmbcs, "LMBCS-6", "UTF-8")

Versioned and host-endian names are preserved exactly:

{:ok, bytes} = Iconvex.convert("A😀", "UTF-8", "UTF-16,version=2")
{:ok, "A😀"} = Iconvex.convert(bytes, "UTF-16,version=2", "UTF-8")

Provenance and verification

The mapping corpus is pinned to ICU release 78.3, Git revision 21d1eb0f306e1141c10931e914dfc038c06121da. Tests verify every source digest, independently reconstruct directional mappings from the UCM files, and execute:

every slot in all single-byte and swaplfnl tables;
all 689,977 generated decoder-domain cases;
all 691,381 canonical generated encoder mappings;
every stateful SI/SO source mapping;
original ICU-derived JIS, LMBCS, Unicode-variant, and Compound Text suites;
complete registration/provider stop, restart, and collision rollback.

The full Unicode scalar fixture is also pinned for LMBCS and Unicode-variant coverage. See BENCHMARKS.md for the executable ICU-oracle performance gates: all 24 LMBCS encode/decode paths remain below the project's 30x ceiling.

Generated tables are loaded lazily and cached in persistent_term; the package shares Core's optimized table and stateful engines rather than duplicating a second conversion runtime.

Licensing

Original Iconvex source is licensed under LGPL-2.1-or-later. ICU/Unicode data and derived mapping tables are covered by the Unicode License v3. See LICENSE, LICENSE.UNICODE, and NOTICE.