Mishka Slug

📖 What is Mishka Slug?

mishka_slug turns a title into a URL slug in its own writing system. Persian stays Persian, Hindi stays Hindi, Japanese stays Japanese: nothing is transliterated. Latin text slugs the way WordPress slugs it, accents removed. Most slug libraries either turn every title into ASCII or keep Unicode with a regex that deletes the vowel signs of Hindi and Thai and turns Arabic and Hebrew marks into hyphens. This one keeps whole words and takes out only what is unsafe or optional.

🚀 Getting started

{:mishka_slug, "~> 0.0.1"}

MishkaSlug.slugify("Café Crème")
#=> {:ok, "cafe-creme"}

MishkaSlug.slugify("Café Crème", latin: :keep)
#=> {:ok, "café-crème"}

MishkaSlug.slugify("Straße", language: "de")
#=> {:ok, "strasse"}

MishkaSlug.slugify("snake_case_name")
#=> {:ok, "snake_case_name"}

MishkaSlug.slugify("snake_case_name", keep: [])
#=> {:ok, "snake-case-name"}

MishkaSlug.slugify("__init__ my_post_")
#=> {:ok, "__init__-my_post_"}

MishkaSlug.slugify("__init__ my_post_", trim_kept: true)
#=> {:ok, "init-my_post"}

MishkaSlug.slugify("آموزش React در ۱۰ دقیقه", locales: ["fa", "en"])
#=> {:ok, "آموزش-react-در-۱۰-دقیقه"}

MishkaSlug.slugify("كتاب يك", language: "fa")
#=> {:ok, "کتاب-یک"}

MishkaSlug.slugify("😀🚀")
#=> {:error, :empty}

MishkaSlug.slugify("قیمت", locales: ["en"])
#=> {:error, :no_allowed_script}

MishkaSlug.slugify!("Привет, мир!")
#=> "привет-мир"

slugify/2 is idempotent: a slug slugged again with the same options is unchanged. A CMS can re-slug an incoming URL parameter and redirect when the result differs, which also folds the variants people paste (uppercase, Arabic ي for Persian ی, decomposed accents).

English and other Latin text, like WordPress:

• Accents are removed: Café Crème gives cafe-creme, Tiếng Việt gives tieng-viet, Łódź gives lodz, Straße gives strase.
• The :language adds its own spelling: de writes Straße Öl as strasse-oel, da writes Ærø as aeroe, ca writes Col·lecció as colleccio, sr and bs write Đorđe as djordje.
• latin: :keep leaves the accents as typed: Café Crème gives café-crème.
• Only Latin changes: Greek, Cyrillic, Arabic and every other script keep their letters and marks.
• _ stays in a word like WordPress: snake_case_name gives snake_case_name, and so does __init__ (__init__). keep: [] makes it a break: snake-case-name. trim_kept: true removes kept characters at a word's ends: __init__ gives init.
• . separates words unless it is in :keep: a_b.c gives a_b-c.
• & is dropped (Q&A gives q-a), an apostrophe inside a word is removed (don't gives dont), and 1,000 gives 1-000.
• :max_length cuts at whole words: hello world foo with max_length: 13 gives hello-world.

⚙️ Options

Options are checked by MishkaSlug.Options (a guarded_struct). An unknown option or a value outside the accepted ones raises ArgumentError.

The built-in table knows about 85 languages. For any other code, or to follow CLDR, plug in a resolver. It is called as apply(module, function, [code | args]) and returns {:ok, [script]} or :error (which gives {:error, {:unknown_locale, code}}). MishkaSlug.Scripts.from_iso15924/1 turns a CLDR script code ("Arab", "Cyrl", "Jpan") into the script atoms. With the localize package:

defmodule MyApp.LocaleScripts do
  def scripts(code) do
    with {:ok, tag} <- Localize.LanguageTag.parse(code),
         {:ok, tag} <- Localize.LanguageTag.add_likely_subtags(tag) do
      MishkaSlug.Scripts.from_iso15924(tag.script)
    else
      _ -> :error
    end
  end
end

MishkaSlug.slugify("Сәлам", locales: ["tt"], script_resolver: {MyApp.LocaleScripts, :scripts, []})
#=> {:ok, "сәлам"}

Japanese (Han with kana), Korean (Hangul with Han) and Chinese with Bopomofo count as one script, so their words are never cut.

🛡️ Safety

These are always on, not options:

• Invisible characters are removed without leaving a break: zero-width spaces and joiners, bidi controls and isolates (U+202E), soft hyphens, byte-order marks, variation selectors, control, private-use and unassigned code points.
• A word never mixes scripts, so a homoglyph cannot pass for a Latin letter inside a word.
• A slug never holds / ? # % \ : or whitespace, and is never . or ...
• The text is NFKC-normalised first and the slug is NFC, so fullwidth letters, ligatures and Arabic presentation forms become their plain letters.
• Arabic harakat, tatweel and Hebrew points are dropped as optional spelling. Every other mark is kept, because in Indic and Thai scripts the marks spell the word.

Uniqueness and reserved words stay the job of the resource that stores the slug.

🧱 In an Ash resource

With Ash in the host, MishkaSlug.Change writes the slug in an action:

create :create do
  accept [:title, :language]

  change {MishkaSlug.Change,
          from: :title,
          into: :slug,
          language: :language,
          locales: {MyApp.Sites, :languages, []},
          keep: [".", "_"]}
end

language: :language reads the record's language, and locales: takes a list or {module, function, args} called with the changeset first, so the site's languages can come from its tenant. A title with no slug is an error on :title; the slug is never set to nil. Without :max_length the max_length of the slug attribute is used.

🤖 Usage rules for AI agents

The package ships usage_rules files (usage-rules.md and usage-rules/) so an agent knows the options, the return shapes and the do's and don'ts. Add {:usage_rules, "~> 1.2", only: :dev}, set usage_rules: [file: "CLAUDE.md", usage_rules: [{:mishka_slug, sub_rules: ["ash", "configuration"]}]] in project/0, and run mix usage_rules.sync.

🗂️ Layout

lib/
├── mishka_slug.ex          slugify/2 and slugify!/2, the options and the pipeline
└── mishka_slug/
    ├── options.ex          the options schema, shared by slugify/2 and the change
    ├── scripts.ex          site languages to scripts, and one script per word
    └── change.ex           the Ash change (compiled when Ash is present)
usage-rules.md, usage-rules/ rules for AI agents (examples checked by a test)
test/mishka_slug/           real titles per language, safety, options and the Ash change

🔭 Planned

• A confusable check for slugs that look like another slug
• A byte limit beside :max_length
• Replacements such as & to and

🧩 Part of Mishka CMS

Mishka Slug makes the addresses of posts and pages in Mishka CMS: Open-Source Multi-Site CMS, Mishka Blog: Open-Source Blog Engine and Mishka Document: Versioned Docs Engine.

🤝 Contributing

mix deps.get
mix ci

💬 Community and support

• Join the Mishka Discord to ask questions and meet the team.
• Report a bug or ask a question in GitHub issues.
• Read about new releases on the Mishka blog.
• For custom work or priority support, hire the Mishka team.

💖 Support the project

Mishka CMS and its libraries are free and open source. You can support their development through GitHub Sponsors or Buy Me a Coffee.

📜 License

Apache License 2.0. See LICENSE.