Optimising AI / LLM Cost

After proxying your AI agent's LLM calls through MockServer, you can export a structured optimisation brief — a pre-framed Markdown document that you paste directly into any LLM to get concrete, costed advice on reducing inference spend. No extra context needed; the brief contains everything the downstream LLM requires to reason about your traffic.

MockServer analyses the captured traffic offline, computing nine deterministic optimisation signals (repeated system prompts, low cache-hit rates, oversized tool results, unused tool schema, and more) along with token counts and estimated USD costs from provider pricing tables. MockServer never calls an LLM itself — every number in the report is deterministic and computed locally. An in-product verdict (A–F grade and a "$X recoverable" headline) summarises the findings at a glance.

How It Works

1. Start MockServer as a proxy and capture LLM traffic — including real traffic from a headless OpenCode run
2. Export the optimisation brief — via the dashboard, MCP tool, or REST endpoint
3. Read the verdict — instant A–F grade and "$X recoverable" headline, no LLM required
4. Act on the detected opportunities — nine signal types, each mapped to a concrete lever

1. Start MockServer as a Proxy and Capture Traffic

Any MockServer instance can act as an HTTPS proxy — no special mode is needed. See AI Traffic Inspection for the full setup, including how to trust the MockServer CA certificate and configure standard environment variables for Node.js, Python, and other tools.

The quick-start for local use:

docker run -d --rm -p 1080:1080 mockserver/mockserver

export HTTPS_PROXY=http://localhost:1080
export NODE_EXTRA_CA_CERTS=/path/to/mockserver-ca.pem   # Node.js tools
export SSL_CERT_FILE=/path/to/mockserver-ca.pem          # Python tools

# Now run your agent — its LLM calls are captured automatically
claude   # or: opencode, python my_agent.py, etc.

MockServer captures every LLM request/response pair (including SSE-streamed completions) in its event log, and these are the source the optimisation report is built from. Both proxied/forwarded traffic (to a real provider) and mocked LLM responses served by MockServer itself are analysed — so you can optimise against real captured runs or against mocked conversations (for example, the data created by npm run demo). LLM traffic is recognised by request shape (the provider's API path), so it works regardless of the upstream host.

Generate real traffic with OpenCode

The offline npm run demo dataset includes a crafted, deterministic agent run that fires every optimisation signal — great for a first look. To analyse your own agent's real spend instead, proxy a headless OpenCode run through MockServer and pointed at a real provider.

Heads-up: this generates real, non-deterministic LLM traffic — it needs network access and your own provider API key, and the captured run varies each time. It is not wired into npm run demo, which stays offline. Never commit or share API keys.

The repository ships a helper script, mockserver-ui/scripts/demo-opencode-proxy.sh, that starts MockServer as an HTTPS proxy with a machine-local CA, extracts that CA, and prints the exact environment to run OpenCode through it:

# Start the proxy and print the run instructions
./mockserver-ui/scripts/demo-opencode-proxy.sh

# …or start the proxy AND run a one-shot OpenCode prompt through it
./mockserver-ui/scripts/demo-opencode-proxy.sh "summarise the README and suggest one improvement"

The equivalent done by hand, so you can run OpenCode in your own shell:

# 1. Start MockServer as an HTTPS proxy with a unique, machine-local CA
#    (the default CA private key is public — never trust it for real traffic).
docker run -d --rm --name mockserver-proxy -p 1080:1080 mockserver/mockserver \
  -serverPort 1080 \
  -Dmockserver.dynamicallyCreateCertificateAuthorityCertificate=true \
  -Dmockserver.directoryToSaveDynamicSSLCertificate=/dynamic-certs

# 2. Trigger CA generation, then copy the CA out of the container
curl -sk https://localhost:1080/ >/dev/null
docker exec mockserver-proxy cat /dynamic-certs/CertificateAuthorityCertificate.pem > mockserver-ca.pem

# 3. Route OpenCode through the proxy and trust the CA
export HTTPS_PROXY=http://localhost:1080
export NODE_EXTRA_CA_CERTS=$PWD/mockserver-ca.pem   # OpenCode is a Node.js tool
export SSL_CERT_FILE=$PWD/mockserver-ca.pem          # any Python helpers

# 4. Point OpenCode at a REAL provider with YOUR OWN API key, then run it headless
export OPENAI_API_KEY=sk-...            # or ANTHROPIC_API_KEY=sk-ant-...
opencode run "summarise the README and suggest one improvement"

OpenCode's LLM calls are now captured by MockServer. Open the dashboard LLM Optimise tab (immediately after Chaos in the navigation bar), or curl the report, to see signals computed from your real run:

curl -s "http://localhost:1080/mockserver/llm/optimisationReport?format=markdown"

2. Export the Optimisation Brief

Three ways to export — choose whichever fits your workflow:

Dashboard — LLM Optimise Screen

Open the MockServer dashboard and click LLM Optimise in the navigation bar (it sits immediately after Chaos):

http://localhost:1080/mockserver/dashboard

The LLM Optimise screen shows:

• A session picker — pick a single upstream host, or All captured LLM traffic (the default)
• A verdict banner — grade letter (A–F, colour-coded) with the "$X recoverable (Y% of spend)" headline and a one-line rationale
• Hero cards — total estimated cost, input/output tokens, call count, average latency, cache-hit rate, and one-shot rate
• A Detected Opportunities panel listing each signal with severity chip, estimated saving, and structured fix guidance
• A per-call table
• Copy verdict — copies a compact plain-text grade + fix summaries to the clipboard (built from the loaded JSON, no extra fetch)
• Copy optimisation brief — copies the full Markdown brief to the clipboard
• Download bundle — saves the structured JSON report

MCP Tool

If your AI agent is connected to MockServer's MCP control plane (see MCP Setup), use the export_optimisation_report tool:

{
  "method": "tools/call",
  "params": {
    "name": "export_optimisation_report",
    "arguments": {
      "format": "markdown"
    }
  }
}

{
  "method": "tools/call",
  "params": {
    "name": "export_optimisation_report",
    "arguments": {
      "format": "json",
      "host": "api.openai.com"
    }
  }
}

REST Endpoint

Call GET /mockserver/llm/optimisationReport directly. This is a MockServer control-plane endpoint; CORS is enabled so the dashboard UI can call it even when the dashboard and control plane are on different hosts or ports.

curl -s "http://localhost:1080/mockserver/llm/optimisationReport?format=markdown"

curl -s "http://localhost:1080/mockserver/llm/optimisationReport?format=json" | python3 -m json.tool

# Only OpenAI traffic
curl -s "http://localhost:1080/mockserver/llm/optimisationReport?format=markdown&host=api.openai.com"

# Only Anthropic traffic, JSON
curl -s "http://localhost:1080/mockserver/llm/optimisationReport?format=json&provider=ANTHROPIC"

# A specific named session
curl -s "http://localhost:1080/mockserver/llm/optimisationReport?format=markdown&session=host%3Aapi.openai.com"

Query Parameters

If no LLM traffic has been captured yet, the endpoint returns HTTP 200 with an empty report (JSON) or a brief that says "no LLM traffic captured" (Markdown).

What the Markdown Brief Looks Like

The brief is structured in a fixed order so the downstream LLM can reason about it efficiently:

1. Framing preamble — tells the LLM it is a cost-optimisation expert and what to do
2. Verdict — A–F grade, rationale, estimated recoverable spend and token count, cache-hit rate, one-shot rate
3. Run summary — provider(s), model(s), call count, token totals, estimated cost, latency, tool-call count, cache-hit rate, one-shot rate
4. Per-call table — one row per call: model, input/output tokens, cost, latency, tools, finish reason. For proxied/forwarded traffic the per-call latencyMs is the measured upstream round-trip time (full-stream duration for streaming responses). It is 0 only when the upstream time could not be captured for that call.
5. Detected opportunities — each detected signal as a section with title, detail, affected call indices, estimated saving, recommendation, and fix guidance (action, config snippet or example expectation, and a docs link where available)
6. Conversations and tool definitions (appendix) — redacted messages and tool schemas so the LLM has the raw material to propose concrete edits

Once you have the brief, paste it into any LLM. No additional context is needed.

3. The Verdict

Every report includes a deterministic verdict — an A–F grade and a "$X recoverable" headline — computed from the detected signals without any LLM call. The grade is your quick answer to "how much is there to fix here?"

The "recoverable" figure is an estimate of how much spend could be saved by acting on the detected signals. It is calculated using per-call MAX attribution: each call's contribution is the maximum saving across all signals that affect it (not a sum), so the total can never exceed the actual session spend. Treat it as a directional estimate, not an invoice.

The rationale line explains the grade in plain English, for example: "Grade C — an estimated 18% of spend ($1.42) is recoverable across 3 findings (1 high, 2 medium)."

Session KPIs

Two new headline metrics appear in the hero cards and run summary:

• Cache hit rate — the fraction of input tokens served from cache (cachedInputTokens / inputTokens). A rate below 50% on a session with repeated prompts suggests caching is not enabled.
• One-shot rate — the fraction of calls that were not retries. A windowed retry detector (window of 3) identifies calls that duplicate a recent request; a low one-shot rate points to retry loops or duplicate request patterns worth de-duplicating.

4. The Nine Optimisation Signals

MockServer analyses the captured calls and emits up to nine deterministic signals, sorted by urgency (a combination of severity and how many calls are affected). Each signal names the problem, quantifies it in tokens and estimated USD, and provides structured fix guidance — including a copy-paste config snippet or example expectation where relevant.

Redaction

The optimisation report always strips sensitive headers before including them in either the JSON bundle or the Markdown brief:

• Authorization (Bearer tokens, Basic auth)
• x-api-key / api-key
• Cookie / Set-Cookie
• Proxy-Authorization

Body fields are redacted according to the mockserver.fixtureBodyRedactFields configuration property — the same setting used by record_llm_fixtures. The report includes a redaction object describing what was stripped, so you know exactly what the downstream LLM will not see.

Body content is not automatically redacted beyond the configured field list. Review the Markdown brief before pasting it into an external LLM if your prompts or tool results contain sensitive data not covered by fixtureBodyRedactFields.

Configuration

Current Limitations

• Cost estimates — costs are computed from MockServer's built-in provider pricing tables (LlmPricing), which may lag provider price changes. The costIsEstimated field in the JSON bundle is true when the provider did not return real usage tokens and MockServer estimated them from the decoded text; it is false when the provider returned real usage data. Treat all cost figures as directional estimates.
• Non-LLM traffic — only traffic where MockServer can detect a supported LLM provider (OpenAI, Anthropic, Gemini, Bedrock, Azure OpenAI, Ollama), by API path shape, is included — whether mocked or proxied. Other traffic is ignored.
• Session grouping — in v1, the optimisation report groups proxied LLM traffic by upstream host, so the dashboard LLM Optimise session picker offers host-based sessions (and "All captured LLM traffic"). The session REST parameter accepts either the composite host:<host> key or the bare host.

Related Pages

• AI Traffic Inspection — proxy setup, CA trust, and how traffic is captured
• MCP Setup — connect your AI agent to MockServer's MCP control plane to use the export_optimisation_report tool
• MCP Tools Reference — full parameter documentation for export_optimisation_report and related tools
• LLM Response Mocking — mock LLM API responses for deterministic, offline testing
• Configuration Properties — mockserver.llmOptimisationMaxCalls and mockserver.fixtureBodyRedactFields

AI Integration — See Also

• MCP Setup — connect Claude Code, Cursor, Windsurf, Cline, or OpenCode to MockServer's built-in MCP endpoint
• MCP Tools Reference — full documentation of all MCP tools, parameters, and resources
• Debugging with AI — workflows for using AI assistants to debug HTTP traffic via MCP
• AI Traffic Inspection — inspect and record LLM/MCP traffic for debugging and deterministic replay
• OpenAPI Contract Verification — verify recorded traffic and run contract/resiliency tests against an OpenAPI spec
• OpenAPI for AI — use MockServer's OpenAPI spec as a fallback for AI tools without MCP support
• AI Protocol Mocking (MCP & A2A) — mock MCP servers and A2A agents your AI application depends on
• LLM Response Mocking — mock LLM API responses from OpenAI, Anthropic, Gemini, Bedrock, Azure OpenAI, and Ollama with provider-correct formatting, streaming, conversations, and chaos
• LLM Cost Optimisation — export a one-click optimisation brief (Markdown) or JSON bundle from captured LLM traffic to find ways to cut inference cost
• llms.txt — machine-readable index of MockServer documentation for AI assistants and LLMs