Elixir

Extract text, tables, images, metadata, and code intelligence from 107 file formats and 371 programming languages including PDF, Office documents, images, and audio/video transcripts where native transcription is available. Elixir bindings with native BEAM concurrency, OTP integration, and idiomatic Elixir API.

What This Package Provides

• Document intelligence core — extract text, tables, images, metadata, entities, keywords, code intelligence, and transcripts in builds that enable transcription.
• Format coverage — PDF, Office, images, HTML/XML, email, archives, notebooks, citations, scientific formats, plain text, and audio/video formats in builds that enable transcription.
• OCR choices — Tesseract, PaddleOCR, Candle where supported, VLM OCR through liter-llm, and plugin hooks for custom backends.
• Same engine as every binding — Rust, Python, Node.js, Go, Java, PHP, Ruby, .NET, Elixir, WASM, Kotlin Android, Swift, Dart, Zig, and C FFI share the same Rust implementation.
• BEAM package — Rustler NIF binding for OTP pipelines.

Installation

Package Installation

Add to your mix.exs dependencies:

def deps do
  [
    {:xberg, "~> 1.1.3"}
  ]
end

Then run:

mix deps.get

System Requirements

• Elixir 1.14+ and Erlang/OTP 26+ required
• Pre-compiled NIFs bundled via rustler_precompiled for macOS (arm64, x64), Linux (x64, arm64), and Windows (x64)
• Optional: ONNX Runtime version 1.24+ for ORT-dependent inference features
• Optional: Tesseract OCR for OCR functionality

Quick Start

Basic Extraction

Extract text, metadata, and structure from any supported document format:

# Basic document extraction workflow
# Load file -> extract -> access results

{:ok, output} = Xberg.extract(input: %Xberg.ExtractInput{kind: :uri, uri: "document.pdf"}, config: nil)

result = List.first(output.results)
IO.puts("Extracted Content:")
IO.puts(result.content)

IO.puts("\nMetadata:")
IO.puts("Format: #{inspect(result.metadata.format)}")
IO.puts("Tables found: #{length(result.tables)}")

Common Use Cases

Extract with Custom Configuration

Most use cases benefit from configuration to control extraction behavior:

With OCR (for scanned documents):

alias Xberg.ExtractionConfig

config = %ExtractionConfig{
  ocr: %{"enabled" => true, "backend" => "tesseract"}
}

{:ok, output} = Xberg.extract(input: %Xberg.ExtractInput{kind: :uri, uri: "scanned_document.pdf"}, config: config)

result = List.first(output.results)
content = result.content
IO.puts("OCR Extracted content:")
IO.puts(content)
IO.puts("Metadata: #{inspect(result.metadata)}")

Table Extraction

See Configuration Guide for table extraction options.

Processing Multiple Files

inputs = [
  %{"kind" => "uri", "uri" => "report.pdf"},
  %{"kind" => "uri", "uri" => "notes.txt"}
]

case Xberg.extract_batch(inputs: inputs) do
  {:ok, output} -> Enum.each(output.results, &IO.puts(&1.content))
  {:error, reason} -> IO.puts(:stderr, "Extraction failed: #{inspect(reason)}")
end

Async Processing

For non-blocking document processing:

input = %Xberg.ExtractInput{kind: :uri, uri: "document.pdf"}

{:ok, output} = Xberg.extract(input: input, config: nil)

IO.inspect(output.summary)

Next Steps

• Installation Guide - Platform-specific setup
• API Documentation - Complete API reference
• Examples & Guides - Full code examples and usage guides
• Configuration Guide - Advanced configuration options

Features

Supported File Formats (107 formats · 141 file extensions · 56 MIME aliases)

107 formats across 140 unique file extensions, with 56 compatibility MIME aliases, intelligent format detection, and comprehensive metadata extraction.

Office Documents

Images (OCR-Enabled)

Audio & Video

Web & Data

Email & Archives

Academic & Scientific

Code Intelligence (371 Languages)

Powered by tree-sitter-language-pack — documentation.

Complete Format Reference

Key Capabilities

• Text Extraction - Extract all text content with position and formatting information
• Metadata Extraction - Retrieve document properties, creation date, author, etc.
• Table Extraction - Parse tables with structure and cell content preservation
• Image Extraction - Extract embedded images and render page previews
• Audio/Video Transcription - Extract speech transcripts from MP3, M4A, WAV, WebM, and MP4 inputs when the native transcription feature is available
• OCR Support - Integrate multiple OCR backends for scanned documents
• Async/Await - Non-blocking document processing with concurrent operations
• Plugin System - Extensible post-processing for custom text transformation
• Embeddings - Generate vector embeddings using ONNX Runtime models or provider-hosted services
• Batch Processing - Efficiently process multiple documents in parallel
• Memory Efficient - Stream large files without loading entirely into memory
• Language Detection - Detect and support multiple languages in documents
• Code Intelligence - Extract structure, imports, exports, symbols, and docstrings from 371 programming languages via tree-sitter
• Configuration - Fine-grained control over extraction behavior
• Six Output Formats - Plain text, Markdown, Djot, HTML, JSON tree structure, or Docling DocTags

OCR Support

Xberg supports multiple OCR backends for extracting text from scanned documents and images:

• Tesseract

• Paddleocr

• Sceptre

OCR Configuration Example

alias Xberg.ExtractionConfig

config = %ExtractionConfig{
  ocr: %{"enabled" => true, "backend" => "tesseract"}
}

{:ok, output} = Xberg.extract(input: %Xberg.ExtractInput{kind: :uri, uri: "scanned_document.pdf"}, config: config)

result = List.first(output.results)
content = result.content
IO.puts("OCR Extracted content:")
IO.puts(content)
IO.puts("Metadata: #{inspect(result.metadata)}")

Async Support

This binding provides full async/await support for non-blocking document processing:

input = %Xberg.ExtractInput{kind: :uri, uri: "document.pdf"}

{:ok, output} = Xberg.extract(input: input, config: nil)

IO.inspect(output.summary)

Plugin System

Xberg supports extensible post-processing plugins for custom text transformation and filtering.

For detailed plugin documentation, visit Plugin System Guide.

Plugin Example

alias Xberg.Plugin

# Word Count Post-Processor Plugin
# This post-processor automatically counts words in extracted content
# and adds the word count to the metadata.

defmodule MyApp.Plugins.WordCountProcessor do
  @behaviour Xberg.Plugin.PostProcessor
  require Logger

  @impl true
  def name do
    "WordCountProcessor"
  end

  @impl true
  def processing_stage do
    :post
  end

  @impl true
  def version do
    "1.0.0"
  end

  @impl true
  def initialize do
    :ok
  end

  @impl true
  def shutdown do
    :ok
  end

  @impl true
  def process(result, _options) do
    content = result["content"] || ""
    word_count = content
    |> String.split(~r/\s+/, trim: true)
    |> length()

    # Update metadata with word count
    metadata = Map.get(result, "metadata", %{})
    updated_metadata = Map.put(metadata, "word_count", word_count)

    {:ok, Map.put(result, "metadata", updated_metadata)}
  end
end

# Register the word count post-processor
Plugin.register_post_processor(:word_count_processor, MyApp.Plugins.WordCountProcessor)

# Example usage
result = %{
"content" => "The quick brown fox jumps over the lazy dog. This is a sample document with multiple words.",
"metadata" => %{
"source" => "document.pdf",
"pages" => 1
}
}

case MyApp.Plugins.WordCountProcessor.process(result, %{}) do
  {:ok, processed_result} ->
  word_count = processed_result["metadata"]["word_count"]
  IO.puts("Word count added: #{word_count} words")
  IO.inspect(processed_result, label: "Processed Result")

  {:error, reason} ->
  IO.puts("Processing failed: #{reason}")
end

# List all registered post-processors
{:ok, processors} = Plugin.list_post_processors()
IO.inspect(processors, label: "Registered Post-Processors")

Embeddings Support

Generate vector embeddings for extracted text using the built-in ONNX Runtime support. Requires ONNX Runtime installation.

Embeddings Guide

Batch Processing

Process multiple documents efficiently:

inputs = [
  %{"kind" => "uri", "uri" => "report.pdf"},
  %{"kind" => "uri", "uri" => "notes.txt"}
]

case Xberg.extract_batch(inputs: inputs) do
  {:ok, output} -> Enum.each(output.results, &IO.puts(&1.content))
  {:error, reason} -> IO.puts(:stderr, "Extraction failed: #{inspect(reason)}")
end

Configuration

For advanced configuration options including language detection, table extraction, OCR settings, and more:

Configuration Guide

Documentation

• Official Documentation
• API Reference
• Examples & Guides

Contributing

Contributions are welcome! See Contributing Guide.

Part of Xberg.io

• Xberg — the open-source content-intelligence engine: text, tables, and metadata from 107 formats (141 file extensions), with OCR, transcription, and code intelligence. MIT.
• Xberg Pro — a complete self-hosted content-intelligence backend in a single container. Commercial.
• Xberg Enterprise — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
• crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
• html-to-markdown — fast, lossless HTML→Markdown engine.
• liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
• tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
• alef — the polyglot binding generator that produces this README and all per-language bindings.
• Discord — community, roadmap, announcements.

License

MIT License — see LICENSE for details.

Support

• Discord Community: Join our Discord
• GitHub Issues: Report bugs
• Discussions: Ask questions