Muninn

A fast, full-text search engine for Elixir, powered by Tantivy

CIPrecomp NIFsHex.pmDocs

Named after Odin's raven who gathers information from across the nine worlds.

Why "Muninn"?

In Norse mythology, Odin—the Allfather—possessed two ravens named Huginn (thought) and Muninn (memory). Each dawn, they would fly across the Nine Realms, observing everything that transpired in the world. At dusk, they returned to perch upon Odin's shoulders and whisper all they had learned into his ears.

Of the two, Odin feared more for Muninn:

"Huginn and Muninn fly each dayover the spacious earth.I fear for Huginn, that he come not back,yet more anxious am I for Muninn." — Grímnismál, Poetic Edda

Memory, after all, is what transforms raw observation into wisdom.

This library embodies that spirit: it flies through your documents, indexes what it finds, and returns with perfect recall—no matter how vast your data grows. Fast, reliable, and always remembering.

Features

Installation

Add muninn to your mix.exs:

def deps do
  [
    {:muninn, "~> 0.4.0"}
  ]
end

Requirements:

Quick Start

1. Define a Schema

alias Muninn.Schema

schema = Schema.new()
  |> Schema.add_text_field("title", stored: true, indexed: true)
  |> Schema.add_text_field("body", stored: true, indexed: true)
  |> Schema.add_u64_field("views", stored: true, indexed: true)
  |> Schema.add_bool_field("published", stored: true, indexed: true)

2. Create and Index Documents

alias Muninn.{Index, IndexWriter}

{:ok, index} = Index.create("/path/to/index", schema)

IndexWriter.add_document(index, %{
  "title" => "Getting Started with Elixir",
  "body" => "Elixir is a functional programming language...",
  "views" => 1523,
  "published" => true
})

IndexWriter.commit(index)

3. Search Documents

alias Muninn.{IndexReader, Searcher}

{:ok, reader} = IndexReader.new(index)
{:ok, searcher} = Searcher.new(reader)

# Simple search
{:ok, results} = Searcher.search_query(
  searcher,
  "elixir programming",
  ["title", "body"]
)

# Field-specific search
{:ok, results} = Searcher.search_query(
  searcher,
  "title:elixir AND published:true",
  ["title", "body"]
)

# Search with highlighted snippets
{:ok, results} = Searcher.search_with_snippets(
  searcher,
  "functional programming",
  ["title", "body"],
  ["body"],
  max_snippet_chars: 150
)

# Autocomplete/prefix search
{:ok, results} = Searcher.search_prefix(
  searcher,
  "title",
  "eli",
  limit: 10
)

# Range queries
{:ok, results} = Searcher.search_query(
  searcher,
  "views:[1000 TO 5000]",
  ["title"]
)

# Programmatic range queries
{:ok, results} = Searcher.search_range_u64(
  searcher,
  "views",
  1000,
  5000,
  inclusive: :both
)

# Fuzzy search (handles typos)
{:ok, results} = Searcher.search_fuzzy(
  searcher,
  "title",
  "elixr",  # Typo for "elixir"
  distance: 1
)

# Fuzzy prefix (autocomplete with typo tolerance)
{:ok, results} = Searcher.search_fuzzy_prefix(
  searcher,
  "author",
  "jse",  # Typo while typing "jose"
  distance: 1,
  limit: 10
)

Search Features

Query Parser Syntax

Highlighted Snippets

Returns HTML snippets with matching words wrapped in <b> tags:

{:ok, results} = Searcher.search_with_snippets(
  searcher,
  "elixir",
  ["title", "content"],
  ["content"],
  max_snippet_chars: 200
)

# Result contains:
# "snippets" => %{
#   "content" => "<b>Elixir</b> is a functional programming language..."
# }

Prefix Search (Autocomplete)

Perfect for search-as-you-type functionality:

{:ok, results} = Searcher.search_prefix(searcher, "title", "pho", limit: 10)
# Matches: "Phoenix Framework", "Photography", "Photoshop", etc.

Range Queries

Filter numeric fields with flexible boundary control:

# QueryParser syntax - inclusive range [100, 1000]
{:ok, results} = Searcher.search_query(searcher, "views:[100 TO 1000]", ["title"])

# Programmatic API with boundary control
{:ok, results} = Searcher.search_range_u64(
  searcher,
  "views",
  100,
  1000,
  inclusive: :both    # :both, :lower, :upper, :neither
)

# Range queries work for all numeric types
Searcher.search_range_u64(searcher, "views", 0, 1000)        # Unsigned integers
Searcher.search_range_i64(searcher, "temperature", -10, 30)  # Signed integers
Searcher.search_range_f64(searcher, "price", 9.99, 99.99)    # Floating point

# Combine with text search
{:ok, results} = Searcher.search_query(
  searcher,
  "title:elixir AND views:[1000 TO *]",
  ["title"]
)

Fuzzy Search (Typo Tolerance)

Handle spelling errors and typos automatically using Levenshtein distance:

# Basic fuzzy search (distance=1: one character difference)
{:ok, results} = Searcher.search_fuzzy(
  searcher,
  "title",
  "elixr",  # Typo for "elixir"
  distance: 1
)

# More tolerant search (distance=2: two character differences)
{:ok, results} = Searcher.search_fuzzy(
  searcher,
  "content",
  "phoneix",  # Typo for "phoenix"
  distance: 2
)

# Fuzzy prefix search (autocomplete with typo tolerance)
{:ok, results} = Searcher.search_fuzzy_prefix(
  searcher,
  "author",
  "jse",  # User typing "jose" with typo
  distance: 1,
  limit: 10
)

# Fuzzy search with highlighted snippets
{:ok, results} = Searcher.search_fuzzy_with_snippets(
  searcher,
  "content",
  "elixr",
  ["content"],
  distance: 1,
  max_snippet_chars: 150
)

# Transposition handling (character swaps count as 1 edit)
{:ok, results} = Searcher.search_fuzzy(
  searcher,
  "title",
  "elixer",  # "i" and "x" swapped
  distance: 1,
  transposition: true  # default
)

Performance Notes:

Field Types

Type Description Example Use Case
text Full-text searchable strings Titles, descriptions, content
u64 Unsigned 64-bit integers Counts, IDs, timestamps
i64 Signed 64-bit integers Scores, offsets, differences
f64 64-bit floating point Prices, ratings, coordinates
bool Boolean values Flags, states (published, active)

Field Options:

Defaults:stored: false, indexed: true

Examples

See the examples/ directory for complete working examples:

Run any example:

mix run examples/complete_search_demo.exs

API Reference

Core Modules

Search Methods

Basic Term Search - Simple, direct term matching:

query = Query.term("field", "value")
Searcher.search(searcher, query, limit: 10)

Query Parser - Natural syntax with boolean operators:

Searcher.search_query(searcher, "field:value AND other", ["field", "other"])

With Snippets - Highlighted search results:

Searcher.search_with_snippets(searcher, query, search_fields, snippet_fields, opts)

Prefix Search - Autocomplete functionality:

Searcher.search_prefix(searcher, "field", "prefix", limit: 10)

Range Queries - Numeric filtering with flexible boundaries:

Searcher.search_range_u64(searcher, "views", 100, 1000, inclusive: :both)
Searcher.search_range_i64(searcher, "temperature", -10, 30)
Searcher.search_range_f64(searcher, "price", 10.0, 100.0)

Fuzzy Search - Error-tolerant matching with Levenshtein distance:

Searcher.search_fuzzy(searcher, "title", "elixr", distance: 1)
Searcher.search_fuzzy_prefix(searcher, "author", "jse", distance: 1)
Searcher.search_fuzzy_with_snippets(searcher, "content", "elixr", ["content"])

Architecture

Elixir Application
      ↓
  Muninn API (lib/)
      ↓
  Native NIFs (Rustler)
      ↓
  Tantivy (Rust)

Performance

Testing

# Run all tests
mix test

# Run with coverage
mix test --cover

# Run specific test file
mix test test/muninn/searcher_test.exs

Test Coverage: 175+ tests covering:

Documentation

Generate documentation:

mix docs

View at doc/index.html

Development Status

Current: Phase 7 Complete - Fuzzy Matching and Typo Tolerance

Implemented:

Roadmap:

License

Apache 2.0 - See LICENSE for details.

Credits