Inference

Inference

MIT LicenseGitHub

inference provides reusable Elixir contracts for semantic model inference: requests, responses, clients, adapters, capabilities, trace metadata, redaction, and adapter conformance tests.

It is intentionally small. The package gives application code one stable shape for prompts, responses, client configuration, trace summaries, and adapter contracts. Provider SDKs, local agent runtimes, governed execution systems, and transport stacks remain outside the core contract.

It is not an Execution Plane wrapper. Execution Plane remains the lower runtime substrate. inference is the product-facing provider/model boundary used by standalone libraries such as trinity_coordinator and gepa_ex.

Installation

Add :inference to the application that wants the shared contract:

def deps do

  [

    {:inference, "~> 0.1.0"}

  ]

end

Provider-specific dependencies are opt-in. For example:

def deps do

  [

    {:inference, "~> 0.1.0"},

    {:gemini_ex, "..."},

    {:agent_session_manager, "..."}

  ]

end

The initial package ships adapter modules, not separate adapter packages:

Provider-specific dependencies are opt-in dependencies in the consuming application. GeminiEx is the direct Gemini API adapter: Gemini API users add both :inference and :gemini_ex; core/mock users add only :inference. Gemini CLI is retired. Antigravity is the current Google coding-agent SDK and is reached through the explicitly admitted ASM agent-session boundary, not the Gemini API adapter.

Jido governed execution is owned by jido_integration. The Jido-owned adapter implements Inference.Adapter from that repository and translates shared requests into governed control-plane execution.

That governed lane is required release scope for platforms that use universal auth authority. It remains outside the core :inference package so direct standalone adapters stay reusable, while governed deployments carry authority refs, credential handles or leases, target grants, and redacted trace evidence through the Jido-owned adapter.

Usage

client =

  Inference.Client.new!(

    adapter: Inference.Adapters.Mock,

    provider: :mock,

    model: "mock-fast",

    adapter_opts: [response_text: "hello"]

  )



{:ok, response} = Inference.complete(client, "Say hello")

Inference.Response.text(response)

Requests can also be built explicitly:

{:ok, request} =

  Inference.Request.from_messages([

    %{role: :system, content: "Be concise."},

    %{role: :user, content: "Summarize the result."}

  ])



{:ok, response} = Inference.complete(client, request)

Design Rules

Guides