> For the complete documentation index, see [llms.txt](https://help.pump.co/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://help.pump.co/llm-save.md).

# LLM Save

## Setting up Pump LLM Save

Pump LLM Save is a drop-in gateway. You keep your existing OpenAI or Anthropic SDK, your request shapes, and your own provider account. You change one line — the base URL — and use a Pump API key in place of your provider key.

Pump then sits in the request path and gives you per-request cost and token logging, response caching, automatic model routing, and spend visibility across every app and team sharing the key.

```
your app ──► https://api.pump.co/ai ──► OpenAI / Anthropic / Azure / Vertex
                     │
                     └── logs, cost, cache, routing
```

Request and response bodies are unchanged. Streaming, tool use, prompt caching, and provider beta headers pass through untouched.

**Time to first request:** about five minutes.

***

### Step 1 — Create a Pump API key

In the Pump app: **LLM → Onboarding** (or **Security → AI → Virtual Keys**) → **Create key**.

The key is shown once. Copy it and store it as `PUMP_API_KEY`.

```bash
export PUMP_API_KEY=pk_your_key_here
```

***

### Step 2 — Connect your provider key

In the Pump app: **Integrations → BYOK** → add a credential for each provider you call.

| Provider      | What to paste            |
| ------------- | ------------------------ |
| OpenAI        | `sk-...` API key         |
| Anthropic     | `sk-ant-...` API key     |
| Google Vertex | GCP service-account JSON |

Your provider keys are stored encrypted and are only decrypted inside the gateway at request time. Model spend is billed to **your** provider account — Pump is not reselling tokens.

Until a credential exists for the provider you call, every request is rejected with:

```json
{ "error": { "message": "please set up BYOK keys for this to work successfully", "type": "config" } }
```

This is the most common first-request failure. If you see it, you skipped this step.

***

### Step 3 — Point your SDK at Pump

#### OpenAI SDK — base URL `https://api.pump.co/ai/v1`

**Node**

```ts
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.PUMP_API_KEY,
  baseURL: "https://api.pump.co/ai/v1", // ← only line you change
});

const res = await client.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(res.choices[0].message.content);
```

**Python**

```python
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["PUMP_API_KEY"],
    base_url="https://api.pump.co/ai/v1",  # ← only line you change
)

res = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(res.choices[0].message.content)
```

**curl**

```bash
curl https://api.pump.co/ai/v1/chat/completions \
  -H "Authorization: Bearer $PUMP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role": "user", "content": "Why is the sky blue?"}]
  }'
```

#### Anthropic SDK — base URL `https://api.pump.co/ai`

**Node**

```ts
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic({
  apiKey: process.env.PUMP_API_KEY,
  baseURL: "https://api.pump.co/ai", // ← only line you change
});

const msg = await client.messages.create({
  model: "claude-haiku-4-5",
  max_tokens: 1024, // required by the Anthropic API
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(msg.content[0].text);
```

**Python**

```python
import os
from anthropic import Anthropic

client = Anthropic(
    api_key=os.environ["PUMP_API_KEY"],
    base_url="https://api.pump.co/ai",  # ← only line you change
)

msg = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,  # required by the Anthropic API
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(msg.content[0].text)
```

**curl**

```bash
curl https://api.pump.co/ai/v1/messages \
  -H "x-api-key: $PUMP_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-haiku-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Why is the sky blue?"}]
  }'
```

> **Why the two base URLs differ.** The Anthropic SDK appends `/v1/messages` itself, so its base URL ends at `/ai`. The OpenAI SDK does not, so its base URL includes `/v1`. Raw curl always uses the full path: `https://api.pump.co/ai/v1/...`.

***

### Step 4 — Verify the setup

Save this as `pump-check.sh` and run `PUMP_API_KEY=pk_... bash pump-check.sh`. It tests auth, both protocols, streaming, and caching, and prints the Pump response headers so you can confirm your traffic is actually being metered.

```bash
#!/usr/bin/env bash
# Pump LLM Save — connectivity check.
# Usage: PUMP_API_KEY=pk_... bash pump-check.sh
set -uo pipefail

BASE="https://api.pump.co/ai"
: "${PUMP_API_KEY:?set PUMP_API_KEY first}"

pass() { printf '  \033[32mPASS\033[0m %s\n' "$1"; }
fail() { printf '  \033[31mFAIL\033[0m %s\n' "$1"; }

echo "1. Gateway reachable"
code=$(curl -s -o /dev/null -w '%{http_code}' "$BASE/health")
[ "$code" = "200" ] && pass "/health -> 200" || fail "/health -> $code"

echo "2. OpenAI surface (chat/completions)"
out=$(curl -s -D /tmp/pump_h1 -o /tmp/pump_b1 -w '%{http_code}' \
  "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PUMP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Reply with the word: ok"}],"max_tokens":5}')
if [ "$out" = "200" ]; then
  pass "200 $(grep -i '^x-pump-' /tmp/pump_h1 | tr -d '\r' | paste -sd' ' -)"
else
  fail "$out $(cat /tmp/pump_b1)"
fi

echo "3. Anthropic surface (messages)"
out=$(curl -s -D /tmp/pump_h2 -o /tmp/pump_b2 -w '%{http_code}' \
  "$BASE/v1/messages" \
  -H "x-api-key: $PUMP_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{"model":"claude-haiku-4-5","max_tokens":16,"messages":[{"role":"user","content":"Reply with the word: ok"}]}')
if [ "$out" = "200" ]; then
  pass "200 $(grep -i '^x-pump-' /tmp/pump_h2 | tr -d '\r' | paste -sd' ' -)"
else
  fail "$out $(cat /tmp/pump_b2)"
fi

echo "4. Streaming passthrough"
frames=$(curl -s -N "$BASE/v1/chat/completions" \
  -H "Authorization: Bearer $PUMP_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"count to three"}],"stream":true}' \
  | grep -c '^data:')
[ "$frames" -gt 1 ] && pass "$frames SSE frames" || fail "no stream received"

echo "5. Cache round-trip (opt-in header)"
for i in 1 2; do
  tier=$(curl -s -D - -o /dev/null "$BASE/v1/chat/completions" \
    -H "Authorization: Bearer $PUMP_API_KEY" \
    -H "Content-Type: application/json" \
    -H "x-pump-cache: true" \
    -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"pump cache probe"}],"max_tokens":16}' \
    | grep -i '^x-pump-cache:' | tr -d '\r' | awk '{print $2}')
  echo "  request $i -> x-pump-cache: ${tier:-none}"
done

echo
echo "Done. Requests should now appear in the Pump app under LLM -> Logs."
```

Expected results: steps 1–4 pass, and step 5 prints `miss` on the first request and `exact` on the second.

Note that `/health` on its own is **not** a setup check — it answers before authentication runs, so it succeeds even with an invalid key or a missing provider credential. Steps 2 and 3 are the real test.

Your requests appear in the Pump app under **LLM → Logs** within seconds, with model, tokens, cost, cache tier, latency, and any tags you attach.

***

### Supported models

Call these by the exact `model` id. Prices are the provider’s list price per 1M tokens (input / output) — Pump does not mark them up.

#### OpenAI

| Model          | Tier     | $/1M in | $/1M out |
| -------------- | -------- | ------- | -------- |
| `gpt-5`        | frontier | 1.25    | 10       |
| `o3`           | frontier | 2       | 8        |
| `gpt-4.1`      | frontier | 2       | 8        |
| `gpt-4o`       | balanced | 2.5     | 10       |
| `gpt-5-mini`   | balanced | 0.25    | 2        |
| `o4-mini`      | balanced | 1.1     | 4.4      |
| `gpt-4.1-mini` | value    | 0.4     | 1.6      |
| `gpt-4o-mini`  | value    | 0.15    | 0.6      |
| `gpt-4.1-nano` | value    | 0.1     | 0.4      |
| `gpt-5-nano`   | value    | 0.05    | 0.4      |

#### Anthropic

| Model               | Tier     | $/1M in | $/1M out |
| ------------------- | -------- | ------- | -------- |
| `claude-opus-4-5`   | frontier | 5       | 25       |
| `claude-sonnet-4-5` | balanced | 3       | 15       |
| `claude-3-7-sonnet` | balanced | 3       | 15       |
| `claude-haiku-4-5`  | value    | 1       | 5        |

#### Google Vertex

Gemini models use the OpenAI surface (`/v1/chat/completions`); Claude-on-Vertex models use the Anthropic surface (`/v1/messages`).

| Model                          | Tier     | $/1M in | $/1M out |
| ------------------------------ | -------- | ------- | -------- |
| `vertex/gemini-3.1-pro`        | frontier | 2       | 12       |
| `vertex/gemini-2.5-pro`        | frontier | 1.25    | 10       |
| `vertex/gemini-3.5-flash`      | balanced | 1.5     | 9        |
| `vertex/gemini-3-flash`        | balanced | 0.5     | 3        |
| `vertex/gemini-2.5-flash`      | balanced | 0.3     | 2.5      |
| `vertex/gemini-3.1-flash-lite` | value    | 0.25    | 1.5      |
| `vertex/gemini-2.5-flash-lite` | value    | 0.1     | 0.4      |
| `vertex/claude-opus-4-8`       | frontier | 5       | 25       |
| `vertex/claude-opus-4-1`       | frontier | 15      | 75       |
| `vertex/claude-sonnet-4-5`     | balanced | 3       | 15       |
| `vertex/claude-haiku-4-5`      | value    | 1       | 5        |

#### Azure OpenAI / Foundry

| Model                 | Tier     | $/1M in | $/1M out |
| --------------------- | -------- | ------- | -------- |
| `azure/gpt-5-mini`    | balanced | 0.25    | 2        |
| `azure/o4-mini`       | balanced | 1.1     | 4.4      |
| `azure/mistral-large` | balanced | 4       | 12       |
| `azure/gpt-4.1-mini`  | value    | 0.4     | 1.6      |
| `azure/llama-3.3-70b` | value    | 0.71    | 0.71     |

#### Open-weight

`glm-5p2`.

New models are added on the Pump side — there is nothing to install or upgrade. When a model ships, start using its id.

#### How Pump picks the provider

In priority order:

1. The `x-pump-provider` header, if set (`openai`, `anthropic`, `azure`, `vertex`)
2. The model-id prefix — `vertex/`, `azure/`, `openai/`, `anthropic/`, or any `claude*` id
3. The API path you called — `/v1/messages` means Anthropic, `/v1/chat/completions` means OpenAI

Standard SDK usage resolves automatically. **The `vertex/` and `azure/` prefixes are required** to reach those hosts — `gemini-3.1-pro` will not route, `vertex/gemini-3.1-pro` will. Use the header only for ambiguous cases:

```bash
-H "x-pump-provider: anthropic"
```

***

### Optional features

All opt-in, all via request headers. Omit them and you get plain passthrough.

| Header                 | Values                            | Effect                                                                                                                |
| ---------------------- | --------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| `x-pump-cache`         | `true`, `read-only`, `write-only` | Enable response caching for this request. Exact match is tried first, then semantic match (0.92 similarity). 24h TTL. |
| `x-pump-cache-scope`   | `company` (default), `key`        | Share cache entries across the org, or isolate them per key.                                                          |
| `x-pump-cache-tags`    | `"prod,checkout"` or a JSON array | Partition the cache further.                                                                                          |
| `x-pump-cache-disable` | `1`                               | Hard-off for this request; overrides everything above.                                                                |
| `x-pump-tags`          | `"team-a,batch-job"`              | Tag the request in logs and cost reports.                                                                             |
| `x-pump-metadata`      | JSON object                       | Attach arbitrary metadata (user id, feature, environment) to the logged event.                                        |

Caching applies to `chat/completions` and `messages` JSON requests. Every other endpoint passes straight through.

**Every response carries:**

| Response header                 | Meaning                                                        |
| ------------------------------- | -------------------------------------------------------------- |
| `x-pump-cache`                  | `exact` or `semantic` (served from cache), `miss`, or `bypass` |
| `x-pump-credential-source`      | which credential served the request                            |
| `x-pump-provider-credential-id` | which of your provider credentials was used                    |

These are the fastest way to confirm the gateway is in your path and metering is live.

#### Automatic model routing

Set `"model": "pump/auto"` and Pump chooses the model per request from the pool you configure in **LLM → Routing**, optimizing for cost or quality. Works on `/v1/chat/completions`, `/v1/responses`, and `/v1/messages`; the model actually chosen is recorded in your logs.

Configure routing preferences in the app first — otherwise the request returns `400 Configure routing preferences in the Pump app to use automatic routing`.

#### Reliability

Transient upstream failures (408, 409, 429, and 5xx) are retried up to three times with exponential backoff and jitter, honoring `Retry-After`. Caller errors (400, 401, 404, 422) are returned to you unchanged — Pump relays the provider’s own error body, so your existing SDK error handling keeps working.

***

### Command-line tools

**Claude Code**

```bash
export ANTHROPIC_BASE_URL="https://api.pump.co/ai/"
export ANTHROPIC_AUTH_TOKEN="$PUMP_API_KEY"
claude
```

**Codex CLI, or any OpenAI-compatible tool** — set the tool’s base URL to `https://api.pump.co/ai/v1` and its API key to your Pump key.

***

### Troubleshooting

| Status | Message                                                 | Fix                                                                                          |
| ------ | ------------------------------------------------------- | -------------------------------------------------------------------------------------------- |
| 401    | `missing pump key`                                      | No `Authorization: Bearer` or `x-api-key` header was sent.                                   |
| 401    | `invalid pump key`                                      | Key is wrong or was rotated. Create a new one in the Pump app.                               |
| 403    | `key disabled`                                          | The key was disabled in the Pump app — re-enable it under Virtual Keys, or create a new key. |
| 400    | `please set up BYOK keys for this to work successfully` | Add a provider credential (Step 2) for the provider you are calling.                         |
| 400    | `could not determine provider: ...`                     | Set `x-pump-provider`, or use a recognized model prefix or provider-native path.             |
| 400    | `Configure routing preferences in the Pump app...`      | You sent `pump/auto` before configuring routing.                                             |
| 503    | `auth backend unavailable`                              | Transient. Retry.                                                                            |

Anything else is your upstream provider’s own error, relayed verbatim.

***

### What Pump can see

Pump logs each request’s model, token counts, cost, latency, status, cache tier, and any tags or metadata you attach, along with the prompt and response payloads used for caching and cost attribution. Your provider API keys are stored encrypted and are never returned to any client.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://help.pump.co/llm-save.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.