Skip to main content
Schema drift happens when an upstream API changes its response shape — a field is renamed, a field is removed, a type changes — without your agent knowing. The API returns 200. Everything looks fine. But your agent is reading user.email from a field that no longer exists, or building a prompt around total_price that is now called amount. This is one of the hardest failures to catch because it is silent. The agent does not error — it just produces wrong output. With GoDizzy, you can mock the drifted schema before it ships, run your agent against it, and find the breakage in a controlled environment.

​
Simulate a missing or renamed field

1

Identify the field that changed

Pick a field your agent depends on. Common examples:
  • user.email removed from a user profile response
  • total_price renamed to amount in an order response
  • A required session_token field dropped from an auth response
2

Create a routing rule for the tool endpoint

In your route collection, create a new routing rule:
  • Method: GET
  • Path: /orders/:id
Adjust the method and path to match the endpoint your agent calls.
3

Set the action to mock

Choose Mock as the action. Set Status to 200 — the API is responding successfully. The only thing that changed is the schema.
4

Configure the mock body with the drifted schema

Use the before and after bodies below as a guide. Paste the after body (the drifted version) into the Body field.Before (what your agent currently expects):
before — original schema
{
  "order": {
    "id": "ord_7f3a91bc",
    "status": "fulfilled",
    "total_price": 4999,
    "currency": "usd",
    "customer": {
      "id": "cus_abc123",
      "name": "Jordan Lee",
      "email": "jordan@example.com"
    },
    "items": [
      {
        "sku": "WIDGET-001",
        "quantity": 2,
        "unit_price": 2499
      }
    ]
  }
}
After (the drifted schema — total_price renamed to amount, customer.email removed):
after — drifted schema
{
  "order": {
    "id": "ord_7f3a91bc",
    "status": "fulfilled",
    "amount": 4999,
    "currency": "usd",
    "customer": {
      "id": "cus_abc123",
      "name": "Jordan Lee"
    },
    "items": [
      {
        "sku": "WIDGET-001",
        "quantity": 2,
        "unit_price": 2499
      }
    ]
  }
}
Notice: total_price is gone (replaced by amount), and customer.email is gone entirely. The response is still 200.
5

Save and run your agent or eval suite

Save the rule. Your agent now receives the drifted schema on every request to this endpoint. Run your agent and observe what happens.

​
What the agent sees

GET /orders/ord_7f3a91bc HTTP/1.1
Host: your-collection.godizzy.dev

HTTP/1.1 200 OK
Content-Type: application/json

{
  "order": {
    "id": "ord_7f3a91bc",
    "status": "fulfilled",
    "amount": 4999,
    "currency": "usd",
    "customer": {
      "id": "cus_abc123",
      "name": "Jordan Lee"
    },
    "items": [
      {
        "sku": "WIDGET-001",
        "quantity": 2,
        "unit_price": 2499
      }
    ]
  }
}

​
What to test

  • Does the agent crash or handle the missing field gracefully? If it tries to read order.total_price and the field is absent, does it throw an exception, return null, or skip the field?
  • Is the missing field required for the agent’s prompt or output? If the agent’s prompt includes Total: {{order.total_price}}, what does the output look like when that value is undefined?
  • Does the agent’s output change in an unacceptable way? A missing email might mean the agent silently skips sending a confirmation. A missing price might mean a financial summary is wrong. Look for silent degradation, not just crashes.
  • Does your code have schema validation? If you parse the response into a typed model or validate with a schema, does it surface an error? This is where schema drift is easiest to catch — if you have validation in place.
The most dangerous schema drift is the kind that does not crash anything. If total_price becomes amount and your agent reads total_price as null without erroring, it may produce a subtly wrong answer that looks correct at a glance.

​
Use drifted fixtures in regression testing

Schema drift is most valuable as part of a regression test suite. The workflow:
  1. Create a mock rule with the original schema. Run your evals. Establish a baseline.
  2. Update the mock body to the drifted schema (missing or renamed field). Run the same evals.
  3. Compare outputs. Any eval that fails or produces different output is a dependency on the changed field.
This lets you catch breakage before the upstream API ships the change — not after.
GoDizzy stores versioned mock responses per rule. Create one version with the original schema and one with the drifted schema, then revert between them using the version history to compare your agent’s behavior side by side. No need to edit the body manually each time.

​
Switch back to the real API

When you are done testing, flip the rule from Mock to Proxy in the dashboard. Your agent goes back to hitting the real endpoint. Same URL, no code changes.