Know what you're spending.
Conduit is a local proxy that sits between your code and your LLM providers. It records every call's model, tokens, and cost. Point your SDK's base URL at it — your application doesn't change.
MIT licensed. Single Rust binary.
Most teams know their total LLM bill. Very few know what each feature actually costs.
The invoice arrives. Someone in finance asks which feature is driving the spend. You don’t know — and the engineers don’t know either, because nobody instrumented the SDK calls when they were shipping. A month later you’re still trying to reconstruct it from CloudWatch logs and provider dashboards.
The fix doesn’t have to be a wrapper library, a managed service, or a new SDK your team has to adopt. It can be a proxy that sits in front of the calls they’re already making.
The way teams track LLM costs today
- Wrapper libraries that need to be installed in every service, kept in sync, and added to every new client SDK release.
- Provider dashboards that show totals but can't break down by feature, environment, customer, or team.
- Hand-rolled logging middleware that catches some calls but misses retries, streaming responses, or anything that doesn't go through your one HTTP client.
What a local proxy gets you instead
- Install Conduit once. Set a base URL env var. Every SDK in every language routes through it the same way.
- Tag requests with whatever metadata you want — feature, environment, customer, request ID. Slice the data later.
- Conduit sees the raw HTTP traffic, so it catches every call your code makes, regardless of which SDK or wrapper it goes through.
How it works
Install, point your SDK at it, watch the data come in.
Install the binary
cargo install valohai-conduit, or grab a prebuilt binary from GitHub Releases. Conduit is a single Rust binary that runs anywhere — your laptop, a sidecar in a container, a shared dev box.
Point your SDK at it
Set OPENAI_BASE_URL=http://localhost:8080/openai/v1 (or the equivalent for Anthropic). Most SDKs pick the env var up automatically. No code changes needed, no wrapper library to maintain.
Inspect usage
Run conduit dashboard for a terminal-based view of every request, with model, token counts, and cost. Slice by provider, by environment, by anything you put in the request metadata. Export the data into your own systems when you need to.