LlmCall
A call-oriented SDK for LLM APIs: a canonical IR, dialect codecs, and a one-call client on Finch whose JSON layer you can replace.
defmodule MyApp.OpenAI do
use LlmCall.Provider,
base_url: "https://api.openai.com",
auth: :bearer,
codecs: [chat: LlmCall.Dialect.OpenAI.Chat]
end
client = MyApp.OpenAI.client(api_key: System.fetch_env!("OPENAI_API_KEY"))
request = %LlmCall.Request{
model: "gpt-4o",
messages: [%LlmCall.Message{role: :user, content: [%LlmCall.Content.Text{text: "Hello"}]}]
}
{:ok, response} = LlmCall.complete(client, request)
LlmCall.Response.text(response)
complete/3 validates, encodes, sends, and decodes. stream/3 does the same and
returns a lazy Enumerable of events. generate/3 and poll/3 cover dedicated
image and video endpoints, including long-running jobs.
Installation
def deps do
[
{:llm_call, "~> 0.1.0"}
]
end
Requires Elixir 1.18+ on OTP 27+. Runtime dependencies are Finch and
mint_web_socket; everything else is dev/test only.
Providers
A provider is a module you write: endpoint, auth shape, and the codecs it serves.
There is no built-in catalog — examples/ carries reference definitions.
defmodule MyApp.Groq do
use LlmCall.Provider,
base_url: "https://api.groq.com/openai",
auth: :bearer,
codecs: [chat: LlmCall.Dialect.OpenAI.Chat]
end
client = MyApp.Groq.client(api_key: key)
codecs: is a named map because one endpoint can serve several request shapes.
An endpoint that returns text and images is still one chat codec
(modalities: [:text, :image]); a different request shape — Imagen's predict,
OpenAI Images — is a second codec selected per call:
{:ok, %LlmCall.Media.Response{assets: assets}} =
LlmCall.generate(client, media_request, codec: :images)
Auth shapes: :none, :bearer, :x_api_key, {:header, name}, or
{module, function}. The key is always passed in; nothing reads the environment.
query: adds URL parameters to every request, for vendors that key on one —
MiniMax's GroupId, Azure OpenAI's api-version. A codec's own parameter still
wins on the same key, and the Responses WebSocket handshake gets them too. A
query already written into base_url is folded into query: when the client is
built, so pasting https://…/openai?api-version=2024-02-01 works.
defmodule MyApp.MiniMax do
use LlmCall.Provider,
base_url: "https://api.minimax.io",
auth: :bearer,
codecs: [video: LlmCall.Media.Dialect.MiniMax.Video]
end
client =
MyApp.MiniMax.client(
api_key: System.fetch_env!("MINIMAX_API_KEY"),
query: %{"GroupId" => System.fetch_env!("MINIMAX_GROUP_ID")}
)
The declaration only takes literals, so a value read at runtime belongs on
client/1.
Streaming
{:ok, stream} = LlmCall.stream(client, %{request | stream: true})
stream
|> Stream.each(&IO.inspect({&1.type, &1.delta}))
|> Stream.run()
The library parses SSE and provider terminators. A failure while the body is being
read raises LlmCall.HTTP.Error, whose error/1 returns the canonical
%LlmCall.Error{} with the HTTP status; {:halt, acc} from a reducer closes the
connection. complete/3 rejects stream: true rather than silently flipping it.
To observe events without owning the reduce, pass on_event::
{:ok, stream} =
LlmCall.stream(client, request, on_event: &MyApp.Audit.stream_event/1)
It runs before the reducer, including for finish_stream/2 output; the return
value is ignored, and it is not called after the reducer halts. LlmCall.HTTP is
the only HTTP transport, so raw bytes are not exposed as a hook.
A reasoning block spans several events. The text arrives incrementally, and a
provider that signs its thinking blocks sends the signature in its own event, with
an empty text — Anthropic on every block, Gemini on the thought parts before a
function call. Keep both when you fold the stream into an assistant turn, or the
block cannot be replayed in the next request:
def merge(%LlmCall.Content.Thinking{} = block, %LlmCall.StreamDelta.Thinking{} = delta) do
%{block | text: block.text <> delta.text, signature: delta.signature || block.signature}
end
Responses over WebSocket
The Responses protocol also has a multiplexed WebSocket connection:
named stream_id lanes, decoded events, and reconnect before the 60-minute
limit.
{:ok, connection} = LlmCall.Dialect.OpenAI.Responses.Connection.start_link(client: client)
{:ok, stream} = LlmCall.Dialect.OpenAI.Responses.Connection.create(connection, request, stream_id: "main")
Enum.to_list(stream)
An in-flight response cannot resume across a reconnect; its stream raises
LlmCall.WS.Error. The connection reconnects with exponential backoff and does
not stop on failure: while disconnected, create/3 returns a :transport error
and Connection.reconnect/1 retries immediately. Connection.response_id/2
returns the provider id for previous_response_id.
Tool-call fragments are assembled into complete calls by a pure helper, which never runs them:
assembler = LlmCall.StreamAssembler.new(json_decoder: &JSON.decode/1)
{:ok, assembler, calls} =
Enum.reduce(events, {:ok, assembler, []}, fn event, {:ok, acc, calls} ->
{:ok, acc, new_calls} = LlmCall.StreamAssembler.feed(acc, event)
{acc, calls ++ new_calls}
end)
Media generation
A completion can ask for image output; generated parts arrive in
Response.content:
request = %LlmCall.Request{
model: "gemini-2.5-flash-image",
modalities: [:text, :image],
messages: [%LlmCall.Message{role: :user, content: [%LlmCall.Content.Text{text: "a cat"}]}]
}
{:ok, response} = LlmCall.complete(client, request)
# response.content may contain %LlmCall.Content.Image{}
Dedicated endpoints take a prompt and media parameters instead of messages:
media = %LlmCall.Media.Request{
modality: :image,
model: "gpt-image-1",
prompt: "a cat, watercolor",
params: %{n: 2, size: "1024x1024"}
}
{:ok, %LlmCall.Media.Response{assets: assets}} =
LlmCall.generate(client, media, codec: :images)
Asynchronous video returns an %LlmCall.Operation{} whose next is the request
to execute; poll/3 runs it and returns the next state:
{:ok, %LlmCall.Media.Response{operation: op}} = LlmCall.generate(client, video, codec: :video)
{:ok, %LlmCall.Operation{status: :completed} = done} = LlmCall.poll(client, op, codec: :video)
next_kind: :poll means "execute next and poll"; next_kind: :asset means the
body is media bytes — use LlmCall.fetch/2 for it. The library never polls,
waits, or downloads on its own.
LlmCall.media_capabilities/3 reports what a codec accepts: max_n: :unsupported means no n, max_references is the reference count. A knob the
codec cannot express is a validation error, never a silent drop.
Replaceable JSON layer
HTTP is Finch, one implementation. JSON is a module resolved at client-build
time: the call option, the provider declaration, the compile-time application
default, then Elixir's built-in JSON.
| Layer | Module | Default |
|---|---|---|
| JSON | LlmCall.JSON facade |
Elixir JSON |
# config/config.exs of the host application
config :llm_call,
json_library: Torque,
capture_raw: :response,
http_opts: [name: MyApp.Finch],
timeout: 30_000
Torque, Jason and Poison expose tagged encode/1/decode/1; Elixir
JSON exposes encode!/1/decode!/1. The LlmCall.JSON facade accepts either
shape and normalizes it, so no adapter module is needed. json_library: is the
single configuration key, matching Phoenix.
json_library and capture_raw are read with Application.compile_env/3;
changing them requires recompiling the library. Per-client json_library:,
capture_raw: (through LlmCall.Config.new/1) and provider declarations still
override them without touching the compiled default. An explicit
%LlmCall.Config{} uses the compiled default for both.
LlmCall.Application starts a Finch pool named LlmCall.Finch; pass
http_opts: [name: MyPool] to use your own. If that name is already taken when
:llm_call starts, the existing pool is used and no second one is started.
http_opts accepts only :name, :pool_timeout, :request_timeout and
:pool_strategy; timeout: is the read timeout, and receive_timeout in
http_opts is rejected instead of ignored.
timeout: is Finch's :receive_timeout: each body chunk must arrive within it,
not the whole call, so a large response on a slow link can outlive it. Add
http_opts: [request_timeout: ...] when you want a wall-clock cap.
Tools
Tool data is wire data only: LlmCall.ToolSpec, LlmCall.ToolCall and
LlmCall.ToolResult. The library does not derive schemas or dispatch calls — the
tool_call package does that well, so use it
directly and map its provider-neutral descriptions into canonical specs:
defmodule MyApp.Tools.Weather do
use ToolCall
@doc """
Get the forecast for a city.
## Parameters
* `city` - city name
"""
@spec get_weather(city :: String.t()) :: {:ok, %{temp: float()}}
def get_weather(city), do: {:ok, %{temp: 21.5}}
end
tools =
for tool <- ToolCall.describe(MyApp.Tools.Weather) do
%LlmCall.ToolSpec{
name: tool.name,
description: tool.description,
parameters: tool.parameters
}
end
request = %{request | tools: tools}
{:ok, response} = LlmCall.complete(client, request)
for call <- LlmCall.Response.tool_calls(response) do
{:ok, result} = ToolCall.run(MyApp.Tools.Weather, String.to_existing_atom(call.name), call.arguments)
# build the %LlmCall.ToolResult{} and send the next request yourself
end
Approvals, audits and timeouts stay in tool_call, and the follow-up request stays
with you: the library does not run an agent loop.
What a decoded turn can be sent back as depends on the protocol: text and tool
calls always, thinking blocks and Content.Raw only where the request shape
allows it. docs/design.md §8.2 has the table, and
LlmCall.Codec.Conformance.assert_replays/3 pins it for an adapter.
Provider-hosted tools (web search, grounding, server tools) are builtin tools whose
name is the vendor's own identifier; the codec forwards them verbatim when it
declares builtin_tools: :passthrough:
%LlmCall.ToolSpec{kind: :builtin, name: "web_search_20250305", provider_options: %{"max_uses" => 5}}
Dialect adapters
Four reference adapters — the dialect families the IR is written against:
| Dialect | Module |
|---|---|
| OpenAI Chat Completions | LlmCall.Dialect.OpenAI.Chat |
| OpenAI Responses | LlmCall.Dialect.OpenAI.Responses |
| Anthropic Messages | LlmCall.Dialect.Anthropic.Messages |
| Gemini GenerateContent | LlmCall.Dialect.Gemini.GenerateContent |
And four vendored media adapters, which ship with the library but are not part
of the stable API surface (see LlmCall.Media):
| Dialect | Module |
|---|---|
| OpenAI Images | LlmCall.Media.Dialect.OpenAI.Images |
Gemini Imagen predict |
LlmCall.Media.Dialect.Gemini.Predict |
| Seedance (BytePlus ModelArk) | LlmCall.Media.Dialect.Seedance.Video |
| MiniMax Hailuo | LlmCall.Media.Dialect.MiniMax.Video |
The library does not promise provider coverage. An adapter outside this repository
is a module that uses LlmCall.Dialect.Adapter (or LlmCall.Media.Dialect.Adapter) and
supplies @path, @headers, a private body/2, and the protocol's own
validate_request/2; share validation through LlmCall.Dialect.Helpers instead
of reimplementing it. See CONTRIBUTING.md.
OpenAI-compatible endpoints (Groq, DeepSeek, Together, vLLM, Ollama's OpenAI
surface, ...) use the OpenAI Chat codec with their own base URL and
provider_options; their capabilities are not uniform.
Two levels
The one-call client is the default. The encode/decode functions stay available for callers that already own a transport:
{:ok, encoded} = LlmCall.encode(LlmCall.Dialect.Anthropic.Messages, request)
# transport: encoded.method to base_url <> encoded.path, JSON-encode encoded.body
{:ok, response} = LlmCall.decode_response(LlmCall.Dialect.Anthropic.Messages, payload)
Design
See docs/design.md for the canonical IR, the codec contract,
the streaming model, the client, and the failure and compatibility policy.
docs/roadmap.md records what 0.1 shipped and the order in which
further operation families (embeddings, rerank, speech, transcription) would arrive.
Adapters check themselves with LlmCall.Codec.Conformance or
LlmCall.Media.Conformance (test support published from lib/, not a runtime
dependency). The vendored media adapters run the media suite in this repository's
own mix test.