A switchboard.
Every call to any model goes through one desk. It checks keys, budgets and limits, writes everything down, and reroutes when a line is down.
- theloop vs LiteLLM
- 2-minute read
LiteLLM routes the call.
theloop runs the conversation.
LiteLLM is a gateway: one API in front of 100+ model providers, with keys, budgets, fallbacks and logs. theloop sits one level up and runs the whole loop: model, then tools, then model again until the answer is done. Different jobs, and they stack.
In plain words.
The person on the call.
Talks to the model, fetches what it asks for from your tools, comes back, and repeats until the answer is done. Then hangs up and forgets.
Side by side.
| Question | LiteLLM | theloop |
|---|---|---|
| What is it? | A Python SDK and a self-hosted proxy, the AI gateway | A stateless HTTP service that runs the agent loop |
| Main job | One OpenAI-style API over many providers | Model → tools → model until a final answer |
| Providers | 100+ LiteLLM wins | Anthropic, OpenAI, DeepSeek, GLM, GigaChat, MiniMax and anything OpenAI-compatible |
| Your tools | Can run tools from MCP servers registered in the gateway | Calls your own HTTPS endpoints, through a host allowlist |
| Risky actions | Set per MCP server. With approval set to never, the gateway runs tools itself |
Write tools never run: they answer deferred, and your system approves |
| Keys and budgets | Virtual keys, team budgets, rate limits, fallbacks, caching | A key per request, per-key queues, token and cost budgets per run |
| Cost tracking | Spend per key, team and user | Usage per provider attempt, failed ones included |
Which one is yours?
Pick LiteLLM if…
- Many teams call many models, and you need central keys, budgets and logs
- You want fallbacks between providers when one goes down
- You need model calls, not an agent loop
Pick theloop if…
- You need the agent loop itself, with your own tools
- Writes must wait for approval in your system
- You bill customers per run and need usage per attempt
Better together. Put LiteLLM behind theloop's openai-compatible provider. LiteLLM handles routing and budgets, theloop runs the loop. It just has to be reachable over HTTPS on an allowlisted host.