LanceDB for Elixir

This package embeds LanceDB in Elixir through Rustler. Use local files or S3-compatible object storage. A database server is not necessary. Supply your own vector embeddings.

The Elixir package version is 0.2.0. The Rust software development kit (SDK) is pinned to 0.37.1. Each native build has a separate immutable ID. The selected native build passed public download and production release checks on both supported Linux targets, including supervised table caching, background FTS warming, and shared cache reuse.

Requirements

• Elixir 1.20 or a later 1.x version. CI uses OTP 29.
• Linux x86_64 with glibc 2.35 or later, or Linux ARM64 with glibc 2.39 or later.

Native archives use public GitHub releases. GitHub CLI, credentials, Rust, and Cargo are not necessary for normal installation.

Installation

Add the package to mix.exs:

defp deps do
  [
    {:lancedb_ex, "~> 0.2.0"}
  ]
end

Run these commands from your application directory:

mix deps.get
mix compile

Mix downloads the native archive and checks the committed checksum. The package supplies a default artifact URL. Direct database operations need no supervision tree entry. The optional table cache is application-supervised.

Normal installation does not compile Rust or create checksums. Missing configuration, unsupported platforms, download failures, and incorrect checksums stop compilation.

See the installation guide for URL overrides, offline builds, application releases, and S3 storage options.

Quick start

Use a new data directory for this example:

alias LanceDB.{Connection, Query, Table}

{:ok, db} = LanceDB.connect("./data/vectors")

{:ok, table} =
  Connection.create_table(db, "documents", [
    %{id: "1", text: "Elixir and OTP", vector: [1.0, 0.0, 0.2]},
    %{id: "2", text: "Sourdough bread", vector: [0.0, 1.0, 0.8]}
  ])

{:ok, matches} =
  Query.search(table, [0.9, 0.1, 0.2], limit: 5, distance: :cosine)

IO.inspect(matches)

{:ok, %LanceDB.WriteResult{inserted: 1}} =
  Table.add(table, [
    %{id: "3", text: "Phoenix LiveView", vector: [0.9, 0.1, 0.3]}
  ])

{:ok, 3} = Table.count_rows(table)

Result rows have string keys. Vector search adds a numeric "_distance" field. Database operations return {:ok, value} or {:error, %LanceDB.Error{}}. Builders and in-memory inspection helpers return their values directly. Incorrect API arguments raise ArgumentError.

Connections and table opens support deadlines, cache budgets on the initial connection, and Table.cache_stats/1. See table-open controls and cache statistics for session sharing, option ranges, and examples.

Supervised table caching

Version 0.2.0 includes an optional cache for applications that retain handles and warm indexes in the background. Add it to your supervision tree:

{LanceDB.TableCache,
 name: MyApp.TableCache,
 uri: "./data/vectors",
 max_tables: 64,
 index_cache_size_bytes: 112 * 1024 * 1024,
 metadata_cache_size_bytes: 16 * 1024 * 1024,
 read_consistency_interval_ms: 0}

{:ok, :scheduled} =
  LanceDB.TableCache.prewarm(MyApp.TableCache, "documents", indexes: ["text_idx"])

{:ok, table} = LanceDB.TableCache.fetch(MyApp.TableCache, "documents")
{:ok, rows} = LanceDB.Query.full_text_search(table, "Elixir")

The application creates tables and indexes and decides when to warm them. The cache shares concurrent opens and index warming, retains one session, and limits handles and background work. Fetches and queries can proceed during warming. See table caching for status, invalidation, capacity controls and consistency guarantees.

Documentation

The source repository is preciz/lancedb_ex. This package uses the MIT license.