LLM Save
Setting up Pump LLM Save
Pump LLM Save is a drop-in gateway. You keep your existing OpenAI or Anthropic SDK, your request shapes, and your own provider account. You change one line — the base URL — and use a Pump API key in place of your provider key.
Pump then sits in the request path and gives you per-request cost and token logging, response caching, automatic model routing, and spend visibility across every app and team sharing the key.
your app ──► https://api.pump.co/ai ──► OpenAI / Anthropic / Azure / Vertex
│
└── logs, cost, cache, routingRequest and response bodies are unchanged. Streaming, tool use, prompt caching, and provider beta headers pass through untouched.
Time to first request: about five minutes.
Step 1 — Create a Pump API key
In the Pump app: LLM → Onboarding (or Security → AI → Virtual Keys) → Create key.
The key is shown once. Copy it and store it as PUMP_API_KEY.
export PUMP_API_KEY=pk_your_key_hereStep 2 — Connect your provider key
In the Pump app: Integrations → BYOK → add a credential for each provider you call.
OpenAI
sk-... API key
Anthropic
sk-ant-... API key
Google Vertex
GCP service-account JSON
Your provider keys are stored encrypted and are only decrypted inside the gateway at request time. Model spend is billed to your provider account — Pump is not reselling tokens.
Until a credential exists for the provider you call, every request is rejected with:
This is the most common first-request failure. If you see it, you skipped this step.
Step 3 — Point your SDK at Pump
OpenAI SDK — base URL https://api.pump.co/ai/v1
Node
Python
curl
Anthropic SDK — base URL https://api.pump.co/ai
Node
Python
curl
Why the two base URLs differ. The Anthropic SDK appends
/v1/messagesitself, so its base URL ends at/ai. The OpenAI SDK does not, so its base URL includes/v1. Raw curl always uses the full path:https://api.pump.co/ai/v1/....
Step 4 — Verify the setup
Save this as pump-check.sh and run PUMP_API_KEY=pk_... bash pump-check.sh. It tests auth, both protocols, streaming, and caching, and prints the Pump response headers so you can confirm your traffic is actually being metered.
Expected results: steps 1–4 pass, and step 5 prints miss on the first request and exact on the second.
Note that /health on its own is not a setup check — it answers before authentication runs, so it succeeds even with an invalid key or a missing provider credential. Steps 2 and 3 are the real test.
Your requests appear in the Pump app under LLM → Logs within seconds, with model, tokens, cost, cache tier, latency, and any tags you attach.
Supported models
Call these by the exact model id. Prices are the provider’s list price per 1M tokens (input / output) — Pump does not mark them up.
OpenAI
gpt-5
frontier
1.25
10
o3
frontier
2
8
gpt-4.1
frontier
2
8
gpt-4o
balanced
2.5
10
gpt-5-mini
balanced
0.25
2
o4-mini
balanced
1.1
4.4
gpt-4.1-mini
value
0.4
1.6
gpt-4o-mini
value
0.15
0.6
gpt-4.1-nano
value
0.1
0.4
gpt-5-nano
value
0.05
0.4
Anthropic
claude-opus-4-5
frontier
5
25
claude-sonnet-4-5
balanced
3
15
claude-3-7-sonnet
balanced
3
15
claude-haiku-4-5
value
1
5
Google Vertex
Gemini models use the OpenAI surface (/v1/chat/completions); Claude-on-Vertex models use the Anthropic surface (/v1/messages).
vertex/gemini-3.1-pro
frontier
2
12
vertex/gemini-2.5-pro
frontier
1.25
10
vertex/gemini-3.5-flash
balanced
1.5
9
vertex/gemini-3-flash
balanced
0.5
3
vertex/gemini-2.5-flash
balanced
0.3
2.5
vertex/gemini-3.1-flash-lite
value
0.25
1.5
vertex/gemini-2.5-flash-lite
value
0.1
0.4
vertex/claude-opus-4-8
frontier
5
25
vertex/claude-opus-4-1
frontier
15
75
vertex/claude-sonnet-4-5
balanced
3
15
vertex/claude-haiku-4-5
value
1
5
Azure OpenAI / Foundry
azure/gpt-5-mini
balanced
0.25
2
azure/o4-mini
balanced
1.1
4.4
azure/mistral-large
balanced
4
12
azure/gpt-4.1-mini
value
0.4
1.6
azure/llama-3.3-70b
value
0.71
0.71
Open-weight
glm-5p2.
New models are added on the Pump side — there is nothing to install or upgrade. When a model ships, start using its id.
How Pump picks the provider
In priority order:
The
x-pump-providerheader, if set (openai,anthropic,azure,vertex)The model-id prefix —
vertex/,azure/,openai/,anthropic/, or anyclaude*idThe API path you called —
/v1/messagesmeans Anthropic,/v1/chat/completionsmeans OpenAI
Standard SDK usage resolves automatically. The vertex/ and azure/ prefixes are required to reach those hosts — gemini-3.1-pro will not route, vertex/gemini-3.1-pro will. Use the header only for ambiguous cases:
Optional features
All opt-in, all via request headers. Omit them and you get plain passthrough.
x-pump-cache
true, read-only, write-only
Enable response caching for this request. Exact match is tried first, then semantic match (0.92 similarity). 24h TTL.
x-pump-cache-scope
company (default), key
Share cache entries across the org, or isolate them per key.
x-pump-cache-tags
"prod,checkout" or a JSON array
Partition the cache further.
x-pump-cache-disable
1
Hard-off for this request; overrides everything above.
x-pump-tags
"team-a,batch-job"
Tag the request in logs and cost reports.
x-pump-metadata
JSON object
Attach arbitrary metadata (user id, feature, environment) to the logged event.
Caching applies to chat/completions and messages JSON requests. Every other endpoint passes straight through.
Every response carries:
x-pump-cache
exact or semantic (served from cache), miss, or bypass
x-pump-credential-source
which credential served the request
x-pump-provider-credential-id
which of your provider credentials was used
These are the fastest way to confirm the gateway is in your path and metering is live.
Automatic model routing
Set "model": "pump/auto" and Pump chooses the model per request from the pool you configure in LLM → Routing, optimizing for cost or quality. Works on /v1/chat/completions, /v1/responses, and /v1/messages; the model actually chosen is recorded in your logs.
Configure routing preferences in the app first — otherwise the request returns 400 Configure routing preferences in the Pump app to use automatic routing.
Reliability
Transient upstream failures (408, 409, 429, and 5xx) are retried up to three times with exponential backoff and jitter, honoring Retry-After. Caller errors (400, 401, 404, 422) are returned to you unchanged — Pump relays the provider’s own error body, so your existing SDK error handling keeps working.
Command-line tools
Claude Code
Codex CLI, or any OpenAI-compatible tool — set the tool’s base URL to https://api.pump.co/ai/v1 and its API key to your Pump key.
Troubleshooting
401
missing pump key
No Authorization: Bearer or x-api-key header was sent.
401
invalid pump key
Key is wrong or was rotated. Create a new one in the Pump app.
403
key disabled
The key was disabled in the Pump app — re-enable it under Virtual Keys, or create a new key.
400
please set up BYOK keys for this to work successfully
Add a provider credential (Step 2) for the provider you are calling.
400
could not determine provider: ...
Set x-pump-provider, or use a recognized model prefix or provider-native path.
400
Configure routing preferences in the Pump app...
You sent pump/auto before configuring routing.
503
auth backend unavailable
Transient. Retry.
Anything else is your upstream provider’s own error, relayed verbatim.
What Pump can see
Pump logs each request’s model, token counts, cost, latency, status, cache tier, and any tags or metadata you attach, along with the prompt and response payloads used for caching and cost attribution. Your provider API keys are stored encrypted and are never returned to any client.
Last updated
Was this helpful?

