Mock Drift Detection

What is mock drift?

Mock drift occurs when a real upstream service evolves (adding fields, changing response codes, modifying headers) but the mock stubs used in your tests remain unchanged. This leads to tests that pass against stale stubs but fail against the real service.

MockServer's drift detection automatically compares forwarded (real) upstream responses against your stub expectations and flags structural differences, giving you early warning that your mocks need updating.

How it works

When MockServer forwards a request to an upstream service (in proxy mode) and there are also response-type stub expectations matching the same request pattern, MockServer compares the real response against each matching stub and records any structural drifts. The analysis runs asynchronously and never slows down the response path.

Structural drift detection (status, schema and header differences) is automatic and requires no configuration — simply have both forwarding expectations and response-type stub expectations for the same request patterns. The two enrichments described later on this page are opt-in: semantic drift analysis (LLM-powered severity classification) and performance drift detection each need to be turned on with a configuration property.

What is detected

Non-semantic headers (date, x-request-id, content-length, transfer-encoding, connection, keep-alive, server) are automatically excluded from comparison.

Retrieving drift records

Use the GET /mockserver/drift endpoint to retrieve detected drifts:

curl http://localhost:1080/mockserver/drift

Response:

{
  "count": 2,
  "drifts": [
    {
      "expectationId": "abc-123",
      "driftType": "STATUS",
      "field": "statusCode",
      "expectedValue": "200",
      "actualValue": "422",
      "confidence": 1.0,
      "epochTimeMs": 1717145600000
    },
    {
      "expectationId": "abc-123",
      "driftType": "SCHEMA_FIELD_ADDED",
      "field": "$.newField",
      "confidence": 0.9,
      "epochTimeMs": 1717145600000
    }
  ]
}

Filtering

Filter by expectation ID:

curl "http://localhost:1080/mockserver/drift?expectationId=abc-123"

Limit the number of results (default 50, max 500):

curl "http://localhost:1080/mockserver/drift?limit=10"

The limit parameter applies only to unfiltered queries (most-recent-first). When you filter by expectationId, all matching records are returned and limit is ignored.

Clearing drift records

To clear all recorded drifts:

curl -X PUT http://localhost:1080/mockserver/drift/clear

Drift records are also cleared automatically when MockServer is reset via PUT /mockserver/reset.

Storage

Drift records are stored in an in-memory LRU (least recently used) buffer with a maximum capacity of 1000 entries. When the buffer is full, the oldest records are evicted to make room for new ones.

Semantic drift analysis (LLM-powered)

When you have a runtime LLM backend configured (see Configuration Properties), you can enable semantic drift analysis to automatically classify each structural drift as BREAKING, WARNING, or INFORMATIONAL.

Enable it with:

-Dmockserver.driftSemanticAnalysisEnabled=true

When enabled, MockServer sends the drift details along with truncated stub and real response bodies to your configured LLM. The LLM classifies each drift and provides a one-sentence explanation. These are added to the drift records as semanticSeverity and semanticExplanation fields.

Semantic analysis is best-effort: if the LLM is unavailable or returns an error, drift records are stored with their original structural data only.

Example response with semantic fields

{
  "count": 1,
  "drifts": [
    {
      "expectationId": "abc-123",
      "driftType": "SCHEMA_FIELD_REMOVED",
      "field": "$.role",
      "expectedValue": "admin",
      "confidence": 0.95,
      "epochTimeMs": 1717145600000,
      "semanticSeverity": "BREAKING",
      "semanticExplanation": "Removing the role field will break clients that depend on it for authorization"
    }
  ]
}

Performance drift detection

MockServer can track response times per expectation and alert you when the p95 (95th percentile) response time exceeds a threshold. This helps detect performance regressions in upstream services.

Enable it by setting a threshold in milliseconds:

-Dmockserver.driftResponseTimeThresholdMs=500

When the p95 response time for any expectation exceeds 500ms, a PERFORMANCE drift record is emitted. MockServer uses a sliding window of the last 100 response time observations per expectation to calculate percentiles.

Set to 0 (the default) to disable performance drift detection.

Not the same as LLM cassette drift

The drift detection described on this page is for HTTP mock stubs — it compares forwarded upstream responses against your stub expectations while MockServer proxies live traffic. It is separate from the LLM-specific detect_llm_drift MCP tool, which is a different mechanism: that tool replays a recorded LLM cassette (fixture) against the live LLM provider and reports structural drift (new/removed fields, type changes) in the provider's responses. It requires a configured runtime LLM backend and is intended for an opt-in / scheduled CI lane rather than the per-commit build. If you are validating recorded LLM/MCP traffic against a real provider, use detect_llm_drift; for everything else, use the /mockserver/drift endpoint documented here.