Demo Script Catches Production Bug on First Execution
A demo script for an MCP server that wraps Google's Gemini image model failed on its second step, exposing a bug that had been lurking in the codebase despite a fully passing test suite. The issue: the server's local validation allowed thinking levels that the real API did not support, including a default value that guaranteed a 400 error on every live call that didn't explicitly override it.
The Bug: Mismatched Allowed Values
The server, which exposes image generation and editing tools via the Model Context Protocol, maintained a set of four allowed thinking levels: `minimal`, `low`, `medium`, and `high`. The API for the Gemini model, however, only accepts `low` and `high`. The default value was `medium`, meaning every call using default parameters—the most common case for agents and users—would fail with an HTTP 400 error. The bug was invisible to the test suite because all unit tests mocked the API client, returning success regardless of the value passed.
How the Tests Missed It
The test suite consisted of 10 unit tests that mocked the API client to verify the server's internal logic: argument handling, file operations, and error formatting. While these tests validated that the server correctly forwarded the thinking level to the mocked API, they could never detect that the remote contract differed from the local allowlist. The mock returned a successful response for `medium` even though the real API would reject it. Additionally, prior live demos had all used explicit values like `high`, masking the broken default and the two phantom values.
The Fix and Verification
The fix was straightforward: remove the unsupported values from the allowlist and change the default to `low`, a valid option. Beyond the code change, the team updated all documentation—tool signatures, docstrings, README, API cheat-sheet—and added a regression test that explicitly checks that `medium` is rejected. A live call with pure default parameters was run to confirm the fix. Because the server ships as a Docker image, a rebuild and push were necessary to ensure the published artifact no longer contained the bug.
Lessons Learned
The incident highlighted several key lessons for API-facing code. Mocked tests verify your code's logic but cannot verify the contract with a remote service; at least one live smoke test is essential. A local allowlist of remote-owned values is a drift time bomb—if you must pre-validate, treat the list as a cache of someone else's truth and pin regression tests to observed API behavior. Defaults should be tested explicitly, as they are the values nobody explicitly passes and therefore often untested. Finally, a demo script that exercises the real happy path is the cheapest end-to-end test you can write; it found a production bug on its first run, before any audience saw it.