Managoat.Broker
An egress credential proxy for sandboxed agents. The sandbox holds a placeholder where a credential used to be, plus a proxy address with a session token in it. Every outbound HTTP request goes through this proxy, which looks the token up in a session store the host implements, gets back the rules the host prepared for that session, and attaches the real credential to each request that matches one. The agent process never holds the credential; the proxy is the only host it may reach, so a placeholder is worthless off the box.
# The host's supervision tree:
children = [
{Managoat.Broker,
port: 14322,
store: MyApp.BrokerSessions, # a Managoat.Broker.Store
ca_seed: MyApp.broker_ca_seed(), # 32 bytes, the same on every replica
allow_private_upstreams: false}
]
# The sandbox's environment, prepared by the host:
# HTTPS_PROXY=http://<token>:<label>@broker.example.com:14322
# GITHUB_TOKEN=__github_token__
# and the root from Managoat.Broker.ca_pem/0 in its trust store.
What the proxy does
One plaintext HTTP listener (TLS toward the sandbox, if any, is the ingress's job) speaking the two things a forward proxy speaks:
CONNECT host:443for HTTPS. The proxy opens the upstream TLS connection first (an unreachable or untrusted origin is a502before any tunnel exists), answers200, then completes a TLS handshake with the sandbox using a leaf certificate for that host signed by the listener's own CA. Inside the tunnel it reads each request head, rewrites the headers per the session's rules, and forwards head and body to the origin. Bytes coming back are relayed untouched, so a streaming model reply streams; they are also framed, alongside the relay rather than in front of it, so each request's event can say what status it got and how long it took. Keep-alive works; every request on the tunnel is rewritten. A WebSocket upgrade is injected like any other request, after which the tunnel is a byte pipe, unless the session opts into HTTP-only enforcement.- Absolute-form requests (
GET http://host/path) for plain HTTP. The sandbox's connection is kept alive and may carry as many requests as it likes, each decided afresh; the origin's connection is this request's alone. The response head is the one thing not relayed byte for byte, becauseConnectiondescribes the hop it arrived on.
Either form may name an IPv6 literal, bracketed: CONNECT [::1]:8443 and
GET http://[::1]:8080/x. Names are resolved over A and AAAA, and the
vetted addresses are dialed IPv4 first — every host that worked before
takes the address it took before, and IPv6 is a path for hosts that
previously had none. A leaf for a literal carries an iPAddress SAN
rather than a dNSName one, so it validates as a client verifying an
address expects.
The client authenticates with Proxy-Authorization: Basic base64(token:label), which is what an HTTP client sends for a proxy URL
with userinfo. Inside a tunnel the token is looked up once per client
connection (the unit a sandbox's HTTP client pools on, and the unit a
tunnel's fixed destination makes sensible); on the absolute-form path it is
looked up per request, since each request carries its own and trusting the
first would serve a token the proxy never checked.
proxy-authorization never reaches the origin. A missing, unknown or
expired token is 407.
Two guards protect the operator's network and the tenant's intent:
- SSRF. An origin that resolves into a private, loopback, link-local
(including the cloud metadata address), CGNAT, unique-local, multicast or
unspecified address is refused with
403before any connection, in both address families, and the dial goes to a vetted address rather than the name, so a rebinding DNS answer between check and dial changes nothing. Every answer is vetted and one blocked answer refuses the host, so the decision never depends on resolver ordering. The IPv6 forms that embed an IPv4 address —::ffff:,::,2002:and64:ff9b::— are decoded and judged by the IPv4 policy, since each is otherwise a spelling of a blocked address that a range check would call public.allow_private_upstreams: trueturns the guard off for a test rig whose origins are on localhost. deny. A session whoseunmatched_host_policyis:denyrefuses a host no rule names atCONNECT, before a tunnel or handshake exists, and refuses a request no rule matches inside a tunnel with403. That is how an allowlist is enforced: one:passthroughrule per allowed host.
The Store behaviour
@callback lookup(token :: binary()) :: {:ok, Managoat.Broker.Session.t()} | :error
For lookup-only sessions, that is all the proxy needs: the raw token in, a session
with its rules (credentials already resolved) out. Creating, releasing and
sweeping sessions are the host's business; they touch its tables and its
key hierarchy, and the proxy never needs any of it. Hashing the token
before storing it is the host's choice inside lookup/1; the library
passes the raw token from the header.
A store with several instances (one per listener in a test) implements
lookup/2 instead and is configured as store: {Module, instance}.
Managoat.Broker.Store.Memory is the reference store, an Agent holding a
map, for the library's tests and for a consumer without a database.
A Managoat.Broker.Session has rules, unmatched_host_policy
(:passthrough or :deny), expires_at and an opaque meta map the host
fills for its own logging. A Managoat.Broker.Rule has a pattern
(host[:port][/path], wildcards allowed; an IPv6 literal is bracketed,
[::1] or [::1]:8443, since otherwise there is no telling which colon
is the port separator), a scheme and the fields the scheme needs:
scheme |
fields | effect on a matched request |
|---|---|---|
:bearer |
credential |
Authorization: Bearer <credential> replaces any Authorization |
:basic |
credential as {username, password} |
Authorization: Basic base64(username:password) |
:api_key |
header (default Authorization), prefix, credential |
<header>: <prefix><credential> |
:custom |
template (%{header => "text {{ KEY }}"}), credential (%{"KEY" => value}) |
each header rendered from its template |
:substitute |
placeholder, credential |
every header value and the request target have the placeholder replaced by the credential |
:passthrough |
none | forwarded untouched; under deny, how a host is allowed |
When several rules match, the most specific one that sets a header
does: an exact host beats a *. wildcard (whatever their paths), then a
pinned port beats any port, then the longest literal path prefix wins, and
declaration order breaks what is left — so defaults written first with
overrides appended work the way they read, and a list of equally-specific
rules resolves exactly as declaration order alone would. These are Agent
Vault's tiers. A :passthrough rule never displaces a rule that injects,
however specific it is: it is how a host is allowed under deny, not a way
to suppress injection. Every matched :substitute rule applies, in
declaration order, to the header values and to the request target. A credential goes into the target byte for byte — nothing
is percent-encoded on the way in, because the proxy cannot know which URI
component a placeholder sits in nor what the origin expects, and the
canonical case says so: a bot token is <digits>:<rest> in a path
segment, where : is legal unencoded and %3A is a different URL. A
credential holding a control character or a space would split the request
line, so it is refused with 403 rather than written into a target.
A rule the host could not put a credential in — credential left nil, or
holding something other than the shape its scheme needs — has no header to
build. Every request it matches is refused with 502, carrying
error: :credential_missing on the request event, rather than being sent
without the header: the broker failed to obtain a credential, which is not
the agent doing anything wrong, and 502 is what tells it to retry once
the credential is provisioned. 403 would say it is not allowed, which is
a different and misleading thing. Inside a tunnel the request is refused
without ending the tunnel, unless the refused request left a body behind it
in the stream or an earlier response is still being relayed. :substitute
and an unfilled {{ KEY }} are the exceptions described above: a
placeholder the origin can see is the clearer failure there.
A placeholder must be distinctive enough to be one: four characters or
more, holding a letter or digit, and carrying a boundary — __ at either
end, or a character outside [A-Za-z0-9_]. Substitution is a literal
find-and-replace, so id would rewrite every id in a path and
account_sid is a real field name that appears in URLs.
Managoat.Broker.Injector.valid_placeholder?/1 is the check; call it when
building a session, so a bad rule fails where it is written rather than on
every request it would have matched.
Rules match against the target the client sent, and telemetry is derived from that same original, so a placeholder in a path is logged as the placeholder and one in a query is not logged at all.
Per-request authorization
A host can opt a session into fresh authorization by setting its
authorization field to a non-secret, server-controlled reference,
for example a session ID plus credential owner, grant ID and generation.
nil is the default and preserves lookup-only sessions. The reference is
not request telemetry and must never be taken from sandbox headers.
The configured store then implements an additional callback:
@callback authorize(authorization :: term(), request :: Managoat.Broker.Store.request()) ::
{:ok, [Managoat.Broker.Rule.t()]} | {:error, :denied | :unavailable}
Instance stores implement authorize(instance, authorization, request).
The request is %{scheme: :http | :https, host: host, port: port, method: method, target: target}. Host, port and scheme describe the actual
proxy destination, independently of the sandbox's Host header; target is
the original request target, before substitution, including any query.
Do not log this map wholesale.
The callback runs before injection and forwarding of every HTTP
request, including each request inside an existing CONNECT tunnel. It
must check durable authority for the pinned reference, serialized with
revocation, and resolve the current credentials from that same read.
Returned rules replace the initial rules for this request only. The
original reference, expiry, unmatched-host policy and telemetry metadata
remain pinned; no returned credentials are cached for later requests.
Initial rules still check CONNECT reachability under :deny and can
contain patterns with no credential. They never provide a fallback after
an authorization failure.
Denied authority (including a locally expired opted-in session) returns
403 with error: :authorization_denied. Unavailable authority, a missing
callback, an exception/exit/throw or a malformed result returns 503 with
error: :authorization_unavailable. On the absolute-form path both close
the client connection; inside a tunnel a bodyless 403 keeps the tunnel (see
"Connections, and who decides them") and the next request is authorized
again, while a 503 closes it. Callback payloads and exception messages are
neither logged nor returned.
A missing grant or session must be denied by the host, even if its initial
lookup succeeded on a different node before revocation.
The callback runs synchronously in the connection handler. Hosts must bound their database/provider waits and release any locks before returning. Admission is that host's serialized authority check: a request admitted before revocation may finish, including its streamed response. The next request must check again, regardless of missed invalidation notifications.
Set http_only: true alongside authorization when later operations must
remain HTTP requests. The callback alone does not authorize operations
inside an upgraded byte pipe. Protected credentials additionally require
Session.protected, described below, and hosts must drain legacy connections
before adopting the custody boundary.
HTTP-only sessions
Session.http_only defaults to false, preserving ordinary WebSocket and
other upgrade behavior. A server-controlled true pins the session to
HTTP request/response traffic, independently of which rules the request
matches or what its authorization callback returns.
- Reject any client
Upgradeheader (including empty values), anyConnectionorProxy-Connectionoccurrence containing theupgradetoken, and a nested CONNECT inside the intercepted tunnel. Check before credential resolution, then check the effective headers after templates and substitution. Refusals return 403 witherror: :protocol_upgradeand close, even inside a tunnel where other bodyless denials keep it; no rejected request is forwarded to the origin. Invalid header names and value control characters also return 403 witherror: :unsafe_request, so a template cannot manufacture hidden wire headers with a newline in an unrelated value. Reject ambiguous request lengths/codings and any rule-induced change to body framing, so body bytes cannot become an unchecked next request at the origin. Ordinary proxy CONNECT for TLS interception remains supported. - Hold each upstream response head until it parses and passes the policy.
An unexpected 101 closes both sides without forwarding its handshake
or trailing frames. Its event has
error: :upstream_upgradeand no status, because no final response head was accepted. Informational responses do not exempt the following final head from the check. - Bound response heads at 64 KiB and reject malformed/ambiguous framing, including duplicate Content-Length, conflicting length/transfer coding, and transfer codings other than chunked. This stricter parsing applies only to opted-in sessions. A rejected packet is discarded in full, potentially truncating safe bytes that preceded the bad head in it.
- Preserve accepted header bytes inside CONNECT and stream response bodies as they arrive. HEAD, chunked responses/trailers, informational responses, and keep-alive still work. Plain HTTP retains its hop-specific Connection rewrite and keep-alive. This does not cap stream duration or recall a request admitted before revocation.
Hosts must set the policy when issuing sessions. Existing cached sessions and already-upgraded sockets do not acquire it retroactively: invalidate and drain them on every serving node before adopting the protected path. The library does not provide that deployment-wide drain mechanism.
Protected credentials
A host may pin one protected bearer per session, separately from ordinary rules. The policy is non-secret and server-controlled; neither account owners nor sandbox headers may construct it or widen it. There are no provider-specific defaults in the library:
%Managoat.Broker.Session{
authorization: pinned_session_and_grant_reference,
http_only: true,
protected: %Managoat.Broker.ProtectedRule{
name: "subscription",
host: "provider.example",
port: 443,
paths: ["/responses", "/models/"],
methods: ["GET", "POST"],
identity: "account-123",
identity_header: "x-account-id",
allowed_headers: ["content-type", "accept"],
# The default. `:allow` forwards a query string under the bearer.
query: :refuse
},
rules: ordinary_rules
}
The store's existing authorize/2 or instance callback receives the original
request with protected: true, added by the proxy, and must return
{:ok, %Managoat.Broker.ProtectedCredential{bearer: fresh_token, identity: "account-123"}}.
It must check the durable session/owner/grant/generation binding and obtain
both values from that same authorized read. The identity must equal the
policy's pinned identity. Denied authority is 403; an unavailable, malformed,
wrong-shaped or mismatched-identity result is 503. The credential has redacted
Inspect and no JSON encoder. It never enters an ordinary rule, template map,
substitution, session cache, or telemetry record. Do not put its fields there
in host code either.
The proxy enforces the policy independently of what ordinary rules say:
-
Protected requests use the exact HTTPS host/port and approved HTTP methods. TRACE and CONNECT are never permitted. Paths match exactly; a trailing slash explicitly permits that subtree. Encoded/ambiguous paths, dot segments, backslashes and fragments are denied.
-
A query string on a protected route is refused by default (
query: :refuse): any target holding a?, a barepath?included, is a403witherror: :protected_query, decided before the credential is resolved, so nothing is injected and nothing reaches the origin. A route is pinned so that the bearer reaches one operation; a query the sandbox wrote is a parameter to that operation which nobody pinned. A host whose routes do take a query setsquery: :allow, and it is then forwarded unchanged, never templated. -
Per-route policy.
paths,methodsandqueryhold jointly: every method and the query policy apply on every path. Where one route must be narrower than another under the same bearer, setroutesinstead, and leavepathsandmethodsunset:routes: [ %{path: "/backend-api/codex/responses", methods: ["POST"], query: :refuse}, %{path: "/backend-api/codex/models", methods: ["GET"], query: {:only, ["client_version"]}} ]A request is admitted when one route matches its path, its method and its query; overlapping routes are a union. A route's
querydefaults to:refuse.{:only, names}admits a query only when it isname=valuepairs whose every name is one ofnames, each at most once, compared as raw bytes (no percent-encoded or otherwise respelled name matches), with values limited to unreserved characters, well-formed%XXescapes and+ , : @ ! $ ' ( ) * /, and never decoding to a control character.;, a repeated name, an empty pair and a barepath?are refused. An admitted query is forwarded byte for byte. A pinned name set gives the sandbox the values of parameters the host chose, which is what it already writes in the body, rather than any parameter the origin understands; pin only parameters whose every value is harmless. A policy withroutesmust not also setpaths,methodsor a non-defaultquery, and one withoutroutes(including one persisted by 0.15) behaves exactly as before. -
A matching ordinary injection rule is a conflict (403) before credential resolution; passthrough rules are harmless. Rules for other destinations still work. That ordinary path receives no protected credential, and a typed credential mistakenly returned there is rejected.
-
The proxy supplies Host, Authorization and the pinned identity header. Client values cannot override them. Only allowlisted headers and Content-Length survive; cookies, routing overrides and hop headers are excluded. Protected request bodies must have a Content-Length or no body; chunked requests/trailers are refused to prevent identity changes in trailers. Response bodies still stream. Hosts must verify routes, methods, headers and request framing against their supported client before adoption.
-
Protected TLS always verifies the certificate chain and actual destination name. Listener options cannot disable verification, replace the hostname check/SNI, or resume a previously weaker TLS session. Configured trust roots remain the host's responsibility. The proxy never follows redirects; a client's request to the redirected destination gets a separate decision and cannot carry the injected bearer through the ordinary rule path.
The bearer in a response
The sandbox never holds the bearer, so the one way left for it to get there is for a response to repeat it: an origin that echoes the request, an error page from whatever sits in front of it, a debug endpoint. Only the proxy knows the value at that moment, so only the proxy can look. Every response to a protected request is searched for the bearer that request was sent with, and refused if it holds it. This is on for every protected rule and there is no option to turn it off. Responses to every other request are not searched and take the path they always did.
It refuses rather than scrubs. Rewriting a framed stream means re-framing it (a
Content-Length that is no longer true, chunk sizes to recompute), and a
scrubber that misses one spelling still ships the rest of a response the origin
should never have produced. A cut connection is a failure an agent already
handles.
- The request asks for no coding. A protected request goes upstream with
Accept-Encoding: identity, whatever the client sent and whether or notallowed_headersnames the header, because a compressed body cannot be searched. A response that arrives with any otherContent-Encodinganyway is refused unread (error: :protected_response_encoded). - The head is searched whole, status line and every header, before any of
it is released. Informational (
1xx) heads are searched too. - The body is searched as it streams and is never accumulated. It is
searched decoded: chunk framing is stepped over, so a chunk boundary in
the middle of the value does not hide it, and trailers are searched like
body. Bytes are released only once they cannot be the start of the value:
the longest suffix in hand that is a proper prefix of the bearer waits for
the next read, so a value split across reads or chunks is caught before
its first half is forwarded. Ordinarily that suffix is empty and nothing
waits; at most
byte_size(bearer) - 1bytes ever do, and they are released when the body ends or the origin closes. A chunk extension on a searched response is refused as:malformed_responserather than searched. - On a match (
error: :credential_reflected) nothing more is forwarded, the packet that held the value is dropped whole, and both connections close. If the sandbox had been sent none of that response yet (the value was in the head, or in a body that arrived with its head) it gets a fixed502with a fixed body instead; mid-stream there is no status left to send, so the stream is cut. The request event carrieserror, the route (method,host,path), theruleand the session'smeta, withstatus: 502where that was sent and the origin's status where the body had begun. A warning is logged with the same fields. Neither carries the bearer or the bytes that matched.
What is recognised is the bearer's exact bytes, which any longer string
holding it (Bearer <value>, a JSON document quoting it) contains, plus its
JSON spelling with / written \/. What is not recognised: base64, hex,
percent-encoding or any other transformation of the value; a truncated or
partial copy; a body that is compressed or encoded without a
Content-Encoding saying so; a copy in the response to some other request
or on another connection; and any secret other than this bearer (an ordinary
rule's credential is not searched for). It narrows what an accidental echo can
do. It is not a defence against an origin that sets out to exfiltrate.
ProtectedRule.valid_session?/1 checks persisted policy without credentials.
Invalid policies, missing authorization references, or disabled HTTP-only mode
are rejected at initial lookup. The rule supplies CONNECT reachability under
:deny, but the HTTP request still needs fresh authorization. Protected
request events use scheme: :protected_bearer; :protected_destination,
:protected_query and :protected_conflict identify policy refusals, and
:credential_reflected and :protected_response_encoded a refused response,
all without secret payloads.
The host must still enforce write-time policy ownership, prevent reserved credential aliases in its compiler, fence issuance/updates with revocation, and drain legacy sessions on every node. Fountain's protected compiler, provider-specific routes/client fixture, durable broker fencing and drain integration are not part of this library release.
Connections, and who decides them
Absolute-form plain HTTP keeps the sandbox's connection alive, so apt and
its kin get one connection rather than one per request. Each request on it
is authenticated, host-checked, matched against the session's rules and
given its own read deadline again — nothing is carried over, so the second
request on a connection is decided exactly as the first was, and a deny
session refuses a host it does not allow whether it is asked first or
tenth.
The two hops are decided separately, which is what a proxy is supposed to
do with a hop-by-hop header. The origin is asked to close and its
connection is dropped after the response; the sandbox is told what this
hop is doing, which means the response head is re-emitted with the proxy's
own Connection rather than the origin's. That head is the one place a
response is not relayed byte for byte — bodies still are, which is what
keeps a streamed reply a stream. A connection ends after one request when
the client asked for that, when it speaks HTTP/1.0, when the response ends
only at the origin's close (there is no boundary to follow), or on any
refusal.
CONNECT tunnels keep alive too. Rule processing, and authorization for
opted-in sessions, run for each HTTP request inside the tunnel.
A refusal inside a tunnel refuses the request, not the tunnel. A 403 (a
denial: deny policy, a protected route or query, denied authority, a
conflict) or a 502 (:credential_missing) is answered without
connection: close and the tunnel goes on to the next request, which is
decided afresh — so a client refused one route of a host can reach an
allowed route of it without a new TCP and TLS connection. The tunnel still
closes when the refused request carried a body (the proxy does not read a
body it is refusing, so the next request could not be found in the stream),
when an earlier response on the tunnel is still being relayed (the proxy's
own reply would otherwise land inside it), on a 503, and on the HTTP-only
refusals of a request's own shape (:protocol_upgrade, :unsafe_request).
Absolute-form plain HTTP still closes on any refusal, as above.
The child spec
{Managoat.Broker, port: 14322, store: Module, ca_seed: <32 bytes>, allow_private_upstreams: false}. Every option but the last is required,
and a missing one raises at start naming the option. Optional:
upstream_ssl_options (merged over the :ssl options the proxy dials
origins with; a test origin's cacerts), max_cached_leaves (default
1024; see The CA) and name (default Managoat.Broker, for
several listeners in one VM). There is no
configuration module reading an otp_app: the listener is started by the
host with values the host computed at boot, and a library that is not
started serves nothing.
Once up: Managoat.Broker.port/1, running?/1 and ca_pem/1.
The CA
A brokered sandbox trusts one root, and the proxy presents a leaf for each
host the sandbox CONNECTs to, signed by that root. Every replica of the
host must present leaves the sandbox trusts, whichever one the ingress
hands a connection to, and a sandbox that survived a restart must still
trust what a fresh replica signs. So the root is not generated and stored:
it is derived from the 32-byte ca_seed with HKDF-SHA256, reduced into
P-256's scalar field, and the certificate's subject, serial and validity
are fixed, so every replica computes the same key, subject and serial from
the same seed. Rotating the seed rotates the CA; nothing else does.
The root's self-signature bytes vary per derivation, because ECDSA signing
is randomised, so two replicas' PEMs differ byte for byte. That is
harmless: a client matches a trust anchor by subject and public key and
never verifies a root's own signature. ca_test.exs proves a leaf signed
after a re-derivation chains to the first root.
The seed is the host's to derive, and it must not be a key the host uses for anything else: derive it from a master key with a fixed info string, so the CA key is never the storage key. Leaves live thirty days, are cached per host in an ETS table the listener owns, and are re-signed after twenty-nine.
That cache is bounded, because its key is the host from a sandbox's own
CONNECT line: under :passthrough an agent browses wherever it likes,
and a wildcard DNS record aimed at one address makes every
*.attacker.example a distinct name that resolves, connects and would
otherwise be cached for the life of the listener. It holds at most
max_cached_leaves of them — 1024 by default — and the least recently
used goes first, so a host still being visited keeps its leaf and a busy
listener does not become a re-signing treadmill. Going over the cap costs
one ECDSA signature the next time a fallen-off host is seen; it refuses
nothing, which is why it is an option with a default rather than a decision
the host has to make.
A host is validated before any of that. The name off the request line
reaches :inet.getaddrs, the TLS server_name_indication, that cache's
key and the subject and SAN of a certificate this proxy signs, so a host
longer than 253 bytes, or holding a control character, whitespace or any of
@ / \ ? # %, or beginning or ending with a dot, is 400 before it is
resolved, dialed, cached or signed. It is not sanitised: forwarding a name
the client did not ask for is worse than refusing the one it did. This is
Agent Vault's brokercore.IsValidHost less its DNS-name blocklist
(localhost, metadata.google.internal and two more), which is not worth
porting — the SSRF guard above works on the addresses a name resolves to,
so it catches every name that reaches a blocked range rather than the four
anyone thought to write down.
Telemetry
Every request the proxy decides about emits [:managoat, :broker, :request] with the measurements %{count: 1, duration: <native units>}
and the metadata method, host, path, outcome (:injected,
:passthrough or :denied), rule (the applied rule's name, or nil),
scheme (that rule's scheme, or nil), status, error and meta (the
session's, unchanged). Never a header, never a body. The host attaches a
handler and writes its log line with whatever meta carries; the library
logs only refusals, which have no session to attribute.
outcome answers "did a rule apply", not "was a credential
attached", and the two are not the same question. :passthrough means
no rule matched and the session's policy let the request through, so
it never appears under unmatched_host_policy: :deny. A matched
:passthrough rule — the documented way a host is allowed under deny —
is outcome: :injected with that rule's name, because a rule did apply;
nothing was attached to the request.
scheme is what separates them, and it is the field an audit log should
read. In a default-deny session the two rules a consumer most wants to
tell apart are exactly the two that outcome collapses:
| rule | outcome |
rule |
scheme |
|---|---|---|---|
api.stripe.com, :bearer |
:injected |
"stripe" |
:bearer |
registry.npmjs.org, :passthrough |
:injected |
"npm" |
:passthrough |
no rule matched, policy :passthrough |
:passthrough |
nil | nil |
| refused | :denied |
nil | nil |
The event is terminal: one per request, emitted when the request is over rather than when it starts.
- For an upstream response, it fires once the response body has completed
or failed.
statusis the status the origin sent;durationis monotonic, in native time units (System.convert_time_unit/3turns it into milliseconds), and covers the whole request through the end of the response body — not time to first byte. - For a refusal the proxy makes itself, it fires immediately, with the
status the proxy sent (
403). erroris nil when the request completed. Otherwise it is one of:upstream_send_failed,:upstream_read_failed,:malformed_response,:upstream_closedor:client_closed; or, for a protected request,:credential_reflectedor:protected_response_encoded(see "The bearer in a response"). A response whose head arrived and whose body then failed carries both itsstatusand itserror.
A consequence worth planning for: a long-lived request is not recorded until it ends, so a streaming reply appears in a host's audit log when the stream finishes. That matches Agent Vault's total-duration semantics and avoids a second event and a row-update protocol. If immediate visibility for long-lived requests is ever needed, that is correlated start/stop events, not more meaning packed into this one.
For ordinary sessions, framing only observes the relay: every byte is written to the sandbox before the framer sees it. HTTP-only sessions add a response-head gate before the write, so malformed framing can close the connection. Accepted body bytes still stream in both modes.
path is the URL path and nothing else. A query string never appears in
it, on either request path, because a query can already hold a credential
this proxy never brokered — a signed URL is one in itself, and ?key= is
a shape clients use. The origin receives the request target unchanged;
only the event is narrowed. This is Agent Vault's contract too: its
request log recorded r.URL.Path. A CONNECT names an authority rather
than a path, and is reported as it was sent.
Deviations from Agent Vault
This proxy replaced Infisical's Agent Vault behind the same interface. The
parity suite (test/managoat/broker/agent_vault_parity_test.exs) replays
the upstream tests it stands in for and lists what was not ported.
Each of these is a decision rather than a backlog item. Agent Vault is deleted from the cluster and from Fountain's codebase, so the A/B that settled the last round — the same request against both proxies, compared on the wire — no longer exists. A row reopened here has to be argued from the upstream tests in the parity suite, from Agent Vault v0.39.1's source, or from the protocol; it cannot be measured.
Deliberate, and expected to stay that way
-
No auth-failure rate limiting on the proxy port. Agent Vault had one, and having it caused a production incident: the limiter counted per source address, every sandbox behind one NAT egress shared an address, and one misconfigured client locked out unrelated tenants. Not having it is the fix, not the gap.
The assumption that replaces it is operational, and a host has to hold up its end: the listener is reachable only through the intended ingress or network boundary. If that ever stops being true, the answer is not to restore the limiter as it was — it is to key one on something better than the peer address.
-
The label half of the proxy credential is not checked. Agent Vault refused a valid token presented with another vault's name. Here the per-session token is random and is the whole binding, so checking the label would add no authority — there is nothing a wrong label could protect. The label exists only because some clients (git) refuse a proxy URL with a username and no password.
-
No WebSocket frame rewriting. The upgrade request is injected like any other; the frames after it are piped as bytes. Rewriting them is possible without buffering a whole WebSocket, but it replaces a byte pipe with a protocol implementation that has to get masking, fragmentation, control-frame interleaving and negotiated compression right before a substitution is even correct. Nothing sends a credential inside a frame today. The simple byte pipe is worth keeping until something does.
Deliberate for now, with a condition attached
-
One origin connection per absolute-form request, where Agent Vault pooled them. The sandbox's connection is kept alive and may carry as many requests as it likes; the origin connection is dialed fresh for each one, asked to close, and closed after the response. Agent Vault ran its outbound requests through Go's
http.Transport, which keeps a pool keyed by host with a 90-second idle timeout.A pool would save a handshake per request and cost the race that comes with one: a pooled socket the origin closed while it was idle. The usual answer is to redial and retry, and that answer is not available here — the proxy streams a request body upstream rather than buffering it, so there is nothing left to send again. Dialing per request is always correct, and it is a local TCP handshake against traffic that is package managers.
-
Body caps are configurable, with Agent Vault's defaults.
max_request_bytesdefaults to 1 GiB, matching Agent Vault'sDefaultMaxRequestBytes;max_response_bytesdefaults to:infinity, matching itsDefaultMaxResponseBytesof 0. So a consumer that names neither gets Agent Vault's behaviour.A request whose declared
Content-Lengthexceeds the cap is refused with413before the origin is told anything. A chunked body, which declares no length, is counted as it streams and the connection ends when it passes the cap — the origin already holds a partial body by then, so there is nothing honest left to say. The count includes chunk framing, which makes the cap very slightly conservative rather than parsing a body this proxy has no business reading.A response cap can only end a response, never prevent one: every byte reaches the sandbox before the framer sees it, which is what keeps a stream a stream. So an over-long response arrives up to roughly the cap and the connection is then torn down. Agent Vault ends the same way, by aborting mid-stream.
-
One request-read deadline rather than Agent Vault's timeout matrix. Agent Vault set
ReadHeaderTimeout(10s),ReadTimeout(60s),WriteTimeout(30 min) andIdleTimeout(2 min) on the server inside the tunnel, andTLSHandshakeTimeout(10s),ResponseHeaderTimeout(5 min) andIdleConnTimeout(90s) on the upstream transport.Here the head is bounded at 30s, the gap between reads at 300s, the upstream dial at 10s, and one whole request read — head plus body — by
request_read_timeout, which defaults to five minutes and is the only one of them a host can name. That last is the one that closes the connection-occupancy hole: the others are per operation, so a client sending one byte at a time never trips any of them, and the byte cap bounds volume rather than time.The response side is deliberately not bounded, which is where this differs. A stalled origin is bounded only indirectly on the absolute-form path (by the 300s gap between reads) and not at all inside a tunnel, where the relay waits on origin bytes with no deadline. Agent Vault's
ResponseHeaderTimeoutis the thing we lack, and adding it means a per-request timer in the relay rather than a timeout argument. Nothing has needed it: a stalled origin costs a socket, and the sandbox's own client gives up on its own. If that stops being true it gets its own issue. -
Set-Cookieis relayed, where Agent Vault stripped it. ItsShouldStripResponseHeaderdropped hop-by-hop headers andSet-Cookiefrom every response, "to prevent the upstream from planting cookies in the agent's jar" — a real if small channel: a cookie an origin sets is sent back to it on later requests, from a sandbox that may outlive the task that visited it.Agent Vault rebuilt every response head anyway, so the strip cost it nothing. Here a tunnelled response is relayed byte for byte, and stripping means parsing and re-emitting every head on the path that carries the streams — the trade
Managoat.Broker.Response's moduledoc sets out, spent for something smaller than it costs. Most agent HTTP clients keep no cookie jar at all, and where one does, a session cookie is sometimes how an origin's own auth works, so stripping could break a flow a:passthroughrule was meant to allow.The condition: if response heads are ever re-emitted inside a tunnel for another reason, stripping becomes nearly free and this should be revisited rather than inherited. The parity suite pins the current behaviour so that change has to look at this decision. Doing it on the absolute-form path alone — where heads are re-emitted — would be half a fix whose asymmetry needed explaining every time anyone noticed it.
-
No body substitution. The
:substituterule reaches header values and the request target — a placeholder in a path (/bot<token>/send) or a query (?key=<token>) is replaced on both request paths — but a request body is forwarded as bytes.
Two operational traps a host must handle
Both were found in production with the previous broker and are about the sandbox and the ingress, not the proxy, so this library cannot fix them.
sudostrips the proxy environment. A sandbox that runssudo apt-getlosesHTTPS_PROXYand the rest, and the install fails against a closed network with an error naming apt, not the proxy. Provisioning has to keep the proxy variables acrosssudo(Defaults env_keepin a sudoers drop-in).- An unknown CA leaf, and a shared-NAT peer. A sandbox that does not
trust the root sees every brokered host fail TLS with an "unknown
issuer" error the tool attributes to the origin. The root has to reach
the operating system trust store (
update-ca-certificates) and the toolchains that carry their own roots (NODE_EXTRA_CA_CERTSfor Node,SSL_CERT_FILE/REQUESTS_CA_BUNDLE/CARGO_HTTP_CAINFOpointed at the full system bundle,UV_NATIVE_TLS) before anything else runs. The same incident's other half was a per-IP auth rate limiter at the vendor's proxy port tripping on many sandboxes behind one NAT address; this proxy has none, so nothing in front of it needs to allow for that.
Licence
Apache-2.0. See LICENSE.