What Is the OpenAI-Compatible API? How One Key Works
Why changing one base_url line switches you between GPT, Claude, Gemini and DeepSeek: how the compatibility layer works, and when you need native endpoints.
"OpenAI-compatible" became the de facto standard of the LLM industry — every gateway, local runtime and client claims it. But how much does the compatibility layer actually cover, and when does it distort behaviour? Five minutes well spent.
The core of the OpenAI protocol
Narrowly defined, it's three endpoints:
POST /v1/chat/completions— chat, 95% of all usagePOST /v1/embeddings— vectorisationPOST /v1/images/generations— image generation
A chat request is a list of role-tagged messages (system / user / assistant) plus sampling parameters (temperature, max_tokens), returning the assistant reply and token usage. The protocol is deliberately thin — which is exactly why it won: cheap to implement, cheaper to switch.
What a gateway does in the middle
When your request hits https://api.zaimod.com/v1/chat/completions with model claude-sonnet-5, the gateway does three things:
- Protocol translation: OpenAI messages become Anthropic
/v1/messagesformat (the system prompt is pulled out of the message list into its own field) - Channel routing: the request routes to a live channel for that model and group, with automatic failover
- Metering: token usage is recorded and billed as model price × group multiplier
The response is translated back to OpenAI format. Your code never notices — the only configuration is base_url and key.
Where translation is lossy
Use native protocols when you need:
- Claude extended thinking: the thinking block structure has no OpenAI-format equivalent
- Gemini-native multimodal: video understanding and grounding parameters can drop in translation
- Vendor-only parameters: e.g. Anthropic prompt-caching controls
zaimod exposes all three protocols:
| Protocol | Endpoint | Use when |
|---|---|---|
| OpenAI | https://api.zaimod.com/v1 | General, tooling integration |
| Anthropic | https://api.zaimod.com/v1/messages | Full Claude features |
| Gemini | https://api.zaimod.com/v1beta/models/{model}:generateContent | Gemini-native features |
Streaming and tool calling
Two areas to watch:
Streaming (SSE): deltas are forwarded chunk by chunk, so typewriter UX is preserved. But if a gateway aborts mid-stream, your client must tolerate incomplete JSON.
Tool calling: parallel calls and strict mode differ across vendors. The compatibility layer maps what it can; complex multi-turn tool chains deserve a small-scale validation first.
Practical guidance
- Existing OpenAI codebase, want more models → compatibility layer, change base_url
- Deep in one vendor's features → native endpoint
- Cross-vendor A/B testing → compatibility layer with model-name switching, then freeze the winner
"Thin protocol, lossy translation" — with those two ideas you can always tell which path your tooling should take.