LLM

Lightweight Elixir client for LLM APIs.

It provides a normalized interface for text generation, streaming responses, tool calls, and provider-specific wire formats across:

• OpenAI Chat Completions
• OpenAI Responses
• Anthropic Messages
• Gemini
• OpenRouter (OpenAI-compatible)

Installation

Add llm to your dependencies:

def deps do
  [
    {:llm, "~> 0.1.0"}
  ]
end

Configuration

Configure API keys in application config:

config :llm, :providers,
  openai: [api_key: "sk-..."],
  anthropic: [api_key: "sk-ant-..."],
  gemini: [api_key: "AIza..."],
  openrouter: [api_key: "sk-or-..."]

Or set them at runtime:

LLM.put_key(:openai, "sk-...")
LLM.put_key(:anthropic, "sk-ant-...")

Quick start

Generate text

{:ok, response} =
  LLM.generate("What is Elixir?",
    provider: :openai,
    model: "gpt-4"
  )

response.message.content

Stream responses

{:ok, stream} =
  LLM.stream("Tell me a story",
    provider: :anthropic,
    model: "claude-sonnet-4-5-20250514"
  )

{:ok, response} = LLM.Stream.collect(stream, on_chunk: &IO.write/1)

Use provider modules

You can use provider modules directly for cleaner configuration:

# Using provider module (reads API key from config)
{:ok, response} =
  LLM.generate("Hello",
    provider: LLM.Provider.OpenAI,
    model: "gpt-4"
  )

# With explicit API key via tuple
{:ok, response} =
  LLM.generate("Hello",
    provider: {LLM.Provider.OpenAI, api_key: "sk-..."},
    model: "gpt-4"
  )

# Anthropic
{:ok, response} =
  LLM.generate("Hello",
    provider: LLM.Provider.Anthropic,
    model: "claude-sonnet-4-5-20250514"
  )

# Gemini
{:ok, response} =
  LLM.generate("Hello",
    provider: LLM.Provider.Gemini,
    model: "gemini-2.5-flash"
  )

# OpenRouter
{:ok, response} =
  LLM.generate("Hello",
    provider: LLM.Provider.OpenRouter,
    model: "anthropic/claude-3-opus"
  )

Use a custom provider

{:ok, response} =
  LLM.generate("Hello",
    provider: %{
      adapter: LLM.Adapter.Anthropic,
      base_url: "https://my-proxy.com",
      api_key: "sk-ant-..."
    },
    model: "claude-sonnet-4-5-20250514"
  )

Use tools

{:ok, response} =
  LLM.generate("Read mix.exs",
    provider: :openai,
    model: "gpt-4",
    tools: [MyApp.ReadFileTool]
  )

Providers

A provider can be:

• A preset atom: :openai, :anthropic, :gemini, :openrouter, :openai_responses
• A provider module: LLM.Provider.OpenAI, LLM.Provider.Anthropic, etc.
• A tuple {module, opts} for runtime API key: {LLM.Provider.OpenAI, api_key: "sk-..."}
• A map with :adapter, :base_url, and optionally :api_key

List the built-in presets with:

LLM.providers()

Development

• mix compile
• mix test
• mix test test/llm_generate_test.exs
• mix test test/llm_generate_test.exs:11
• mix format --check-formatted