LLM Save

Setting up Pump LLM Save

Pump LLM Save is a drop-in gateway. You keep your existing OpenAI or Anthropic SDK, your request shapes, and your own provider account. You change one line — the base URL — and use a Pump API key in place of your provider key.

Pump then sits in the request path and gives you per-request cost and token logging, response caching, automatic model routing, and spend visibility across every app and team sharing the key.

your app ──► https://api.pump.co/ai ──► OpenAI / Anthropic / Azure / Vertex
                     │
                     └── logs, cost, cache, routing

Request and response bodies are unchanged. Streaming, tool use, prompt caching, and provider beta headers pass through untouched.

Time to first request: about five minutes.

---

Step 1 — Create a Pump API key

In the Pump app: LLM → Onboarding (or Security → AI → Virtual Keys) → Create key.

The key is shown once. Copy it and store it as PUMP_API_KEY.

export PUMP_API_KEY=pk_your_key_here

---

Step 2 — Connect your provider key

In the Pump app: Integrations → BYOK → add a credential for each provider you call.

Provider	What to paste
OpenAI	sk-... API key
Anthropic	sk-ant-... API key
Google Vertex	GCP service-account JSON

Your provider keys are stored encrypted and are only decrypted inside the gateway at request time. Model spend is billed to your provider account — Pump is not reselling tokens.

Until a credential exists for the provider you call, every request is rejected with:

{ "error": { "message": "please set up BYOK keys for this to work successfully", "type": "config" } }

This is the most common first-request failure. If you see it, you skipped this step.

---

Step 3 — Point your SDK at Pump

OpenAI SDK — base URL https://api.pump.co/ai/v1

Node

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.PUMP_API_KEY,
  baseURL: "https://api.pump.co/ai/v1", // ← only line you change
});

const res = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(res.choices[0].message.content);

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PUMP_API_KEY"],
    base_url="https://api.pump.co/ai/v1",  # ← only line you change
)

res = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(res.choices[0].message.content)

curl

curl https://api.pump.co/ai/v1/chat/completions \
  -H "Authorization: Bearer $PUMP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Why is the sky blue?"}]
  }'

Anthropic SDK — base URL https://api.pump.co/ai

Node

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.PUMP_API_KEY,
  baseURL: "https://api.pump.co/ai", // ← only line you change
});

const msg = await client.messages.create({
  model: "claude-haiku-4-5",
  max_tokens: 1024, // required by the Anthropic API
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(msg.content[0].text);

Python

import os
from anthropic import Anthropic

client = Anthropic(
    api_key=os.environ["PUMP_API_KEY"],
    base_url="https://api.pump.co/ai",  # ← only line you change
)

msg = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,  # required by the Anthropic API
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(msg.content[0].text)

curl

curl https://api.pump.co/ai/v1/messages \
  -H "x-api-key: $PUMP_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-haiku-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Why is the sky blue?"}]
  }'

Why the two base URLs differ. The Anthropic SDK appends /v1/messages itself, so its base URL ends at /ai. The OpenAI SDK does not, so its base URL includes /v1. Raw curl always uses the full path: https://api.pump.co/ai/v1/....

---

Step 4 — Verify the setup

Save this as pump-check.sh and run PUMP_API_KEY=pk_... bash pump-check.sh. It tests auth, both protocols, streaming, and caching, and prints the Pump response headers so you can confirm your traffic is actually being metered.

#!/usr/bin/env bash
# Pump LLM Save — connectivity check.
# Usage: PUMP_API_KEY=pk_... bash pump-check.sh
set -uo pipefail

BASE="https://api.pump.co/ai"
: "${PUMP_API_KEY:?set PUMP_API_KEY first}"

pass() { printf '  \033[32mPASS\033[0m %s\n' "$1"; }
fail() { printf '  \033[31mFAIL\033[0m %s\n' "$1"; }

echo "1. Gateway reachable"
code=$(curl -s -o /dev/null -w '%{http_code}' "$BASE/health")
[ "$code" = "200" ] && pass "/health -> 200" || fail "/health -> $code"

echo "2. OpenAI surface (chat/completions)"
out=$(curl -s -D /tmp/pump_h1 -o /tmp/pump_b1 -w '%{http_code}' \
  "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PUMP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Reply with the word: ok"}],"max_tokens":5}')
if [ "$out" = "200" ]; then
  pass "200 $(grep -i '^x-pump-' /tmp/pump_h1 | tr -d '\r' | paste -sd' ' -)"
else
  fail "$out $(cat /tmp/pump_b1)"
fi

echo "3. Anthropic surface (messages)"
out=$(curl -s -D /tmp/pump_h2 -o /tmp/pump_b2 -w '%{http_code}' \
  "$BASE/v1/messages" \
  -H "x-api-key: $PUMP_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-haiku-4-5","max_tokens":16,"messages":[{"role":"user","content":"Reply with the word: ok"}]}')
if [ "$out" = "200" ]; then
  pass "200 $(grep -i '^x-pump-' /tmp/pump_h2 | tr -d '\r' | paste -sd' ' -)"
else
  fail "$out $(cat /tmp/pump_b2)"
fi

echo "4. Streaming passthrough"
frames=$(curl -s -N "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PUMP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"count to three"}],"stream":true}' \
  | grep -c '^data:')
[ "$frames" -gt 1 ] && pass "$frames SSE frames" || fail "no stream received"

echo "5. Cache round-trip (opt-in header)"
for i in 1 2; do
  tier=$(curl -s -D - -o /dev/null "$BASE/v1/chat/completions" \
    -H "Authorization: Bearer $PUMP_API_KEY" \
    -H "Content-Type: application/json" \
    -H "x-pump-cache: true" \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"pump cache probe"}],"max_tokens":16}' \
    | grep -i '^x-pump-cache:' | tr -d '\r' | awk '{print $2}')
  echo "  request $i -> x-pump-cache: ${tier:-none}"
done

echo
echo "Done. Requests should now appear in the Pump app under LLM -> Logs."

Expected results: steps 1–4 pass, and step 5 prints miss on the first request and exact on the second.

Note that /health on its own is not a setup check — it answers before authentication runs, so it succeeds even with an invalid key or a missing provider credential. Steps 2 and 3 are the real test.

Your requests appear in the Pump app under LLM → Logs within seconds, with model, tokens, cost, cache tier, latency, and any tags you attach.

---

Supported models

Call these by the exact model id. Prices are the provider’s list price per 1M tokens (input / output) — Pump does not mark them up.

OpenAI

Model	Tier	$/1M in	$/1M out
gpt-5	frontier	1.25	10
o3	frontier	2	8
gpt-4.1	frontier	2	8
gpt-4o	balanced	2.5	10
gpt-5-mini	balanced	0.25	2
o4-mini	balanced	1.1	4.4
gpt-4.1-mini	value	0.4	1.6
gpt-4o-mini	value	0.15	0.6
gpt-4.1-nano	value	0.1	0.4
gpt-5-nano	value	0.05	0.4

Anthropic

Model	Tier	$/1M in	$/1M out
claude-opus-4-5	frontier	5	25
claude-sonnet-4-5	balanced	3	15
claude-3-7-sonnet	balanced	3	15
claude-haiku-4-5	value	1	5

Google Vertex

Gemini models use the OpenAI surface (/v1/chat/completions); Claude-on-Vertex models use the Anthropic surface (/v1/messages).

Model	Tier	$/1M in	$/1M out
vertex/gemini-3.1-pro	frontier	2	12
vertex/gemini-2.5-pro	frontier	1.25	10
vertex/gemini-3.5-flash	balanced	1.5	9
vertex/gemini-3-flash	balanced	0.5	3
vertex/gemini-2.5-flash	balanced	0.3	2.5
vertex/gemini-3.1-flash-lite	value	0.25	1.5
vertex/gemini-2.5-flash-lite	value	0.1	0.4
vertex/claude-opus-4-8	frontier	5	25
vertex/claude-opus-4-1	frontier	15	75
vertex/claude-sonnet-4-5	balanced	3	15
vertex/claude-haiku-4-5	value	1	5

Azure OpenAI / Foundry

Model	Tier	$/1M in	$/1M out
azure/gpt-5-mini	balanced	0.25	2
azure/o4-mini	balanced	1.1	4.4
azure/mistral-large	balanced	4	12
azure/gpt-4.1-mini	value	0.4	1.6
azure/llama-3.3-70b	value	0.71	0.71

Open-weight

glm-5p2.

New models are added on the Pump side — there is nothing to install or upgrade. When a model ships, start using its id.

How Pump picks the provider

In priority order:

1. The x-pump-provider header, if set (openai, anthropic, azure, vertex)

2. The model-id prefix — vertex/, azure/, openai/, anthropic/, or any claude* id

3. The API path you called — /v1/messages means Anthropic, /v1/chat/completions means OpenAI

Standard SDK usage resolves automatically. The vertex/ and azure/ prefixes are required to reach those hosts — gemini-3.1-pro will not route, vertex/gemini-3.1-pro will. Use the header only for ambiguous cases:

-H "x-pump-provider: anthropic"

---

Optional features

All opt-in, all via request headers. Omit them and you get plain passthrough.

Header	Values	Effect
x-pump-cache	true, read-only, write-only	Enable response caching for this request. Exact match is tried first, then semantic match (0.92 similarity). 24h TTL.
x-pump-cache-scope	company (default), key	Share cache entries across the org, or isolate them per key.
x-pump-cache-tags	"prod,checkout" or a JSON array	Partition the cache further.
x-pump-cache-disable	1	Hard-off for this request; overrides everything above.
x-pump-tags	"team-a,batch-job"	Tag the request in logs and cost reports.
x-pump-metadata	JSON object	Attach arbitrary metadata (user id, feature, environment) to the logged event.

Caching applies to chat/completions and messages JSON requests. Every other endpoint passes straight through.

Every response carries:

Response header	Meaning
x-pump-cache	exact or semantic (served from cache), miss, or bypass
x-pump-credential-source	which credential served the request
x-pump-provider-credential-id	which of your provider credentials was used

These are the fastest way to confirm the gateway is in your path and metering is live.

Automatic model routing

Set "model": "pump/auto" and Pump chooses the model per request from the pool you configure in LLM → Routing, optimizing for cost or quality. Works on /v1/chat/completions, /v1/responses, and /v1/messages; the model actually chosen is recorded in your logs.

Configure routing preferences in the app first — otherwise the request returns 400 Configure routing preferences in the Pump app to use automatic routing.

Reliability

Transient upstream failures (408, 409, 429, and 5xx) are retried up to three times with exponential backoff and jitter, honoring Retry-After. Caller errors (400, 401, 404, 422) are returned to you unchanged — Pump relays the provider’s own error body, so your existing SDK error handling keeps working.

---

Command-line tools

Claude Code

export ANTHROPIC_BASE_URL="https://api.pump.co/ai/"
export ANTHROPIC_AUTH_TOKEN="$PUMP_API_KEY"
claude

Codex CLI, or any OpenAI-compatible tool — set the tool’s base URL to https://api.pump.co/ai/v1 and its API key to your Pump key.

---

Troubleshooting

Status	Message	Fix
401	missing pump key	No Authorization: Bearer or x-api-key header was sent.
401	invalid pump key	Key is wrong or was rotated. Create a new one in the Pump app.
403	key disabled	The key was disabled in the Pump app — re-enable it under Virtual Keys, or create a new key.
400	please set up BYOK keys for this to work successfully	Add a provider credential (Step 2) for the provider you are calling.
400	could not determine provider: ...	Set x-pump-provider, or use a recognized model prefix or provider-native path.
400	Configure routing preferences in the Pump app...	You sent pump/auto before configuring routing.
503	auth backend unavailable	Transient. Retry.

Anything else is your upstream provider’s own error, relayed verbatim.

---

What Pump can see

Pump logs each request’s model, token counts, cost, latency, status, cache tier, and any tags or metadata you attach, along with the prompt and response payloads used for caching and cost attribution. Your provider API keys are stored encrypted and are never returned to any client.