Every rule in our engineering harness started as a specific failure.
Not a hypothetical. Not a best practice from a conference talk. A specific day, a specific product, a specific amount of time lost because we missed something the AI could not see on its own.
Part 1 of this series introduced the concept. The model is the brain. The harness is the engineering you wrap around it. This article shows where the harness came from.
Here are three of those failures.
Failure one: the database that was never there
We were building GoviHub, an agricultural intelligence platform. The AI built the frontend. It built the data displays, the components, the screens showing market prices and supply chain data. Everything looked right on screen.
The data was not real.
Ten database tables had failed to create during the migration. Silently. The migration script returned success. The tables were not there. The frontend had a fallback, and the fallback filled every screen with substitute data without raising an alarm.
We spent five days debugging what looked like a data processing problem. We ran three rounds of fixes. Each one patched a display symptom. Each one left the root cause untouched. A five-minute database audit would have caught it on day one.
The AI did not create this fallback carelessly. The fallback existed for good reasons in development. But it was never switched off for production, and the AI had no way to tell the difference between real data and substitute data on a screen that looked correct.
Two rules went into the harness after that week.
First: after every database migration, count the tables. The expected count is a number in the spec, not an approximation. If the count does not match, stop. Do not deploy. Fix the migration.
Second: no substitute data in any non-development environment. If an API fails, show an error. A screen that looks right while the backend is broken is worse than a screen that shows a problem. One hides the failure. The other surfaces it.
Failure two: the certificate the external world could not verify
We deployed a new product. The AI built the MCP server, configured the Traefik routing, set up the proxy, and ran the smoke tests. Every test passed. We shipped.
Zero external traffic. For days.
The AI had run its checks against localhost with curl -k. That flag skips TLS certificate chain validation. The tests confirmed the process was running. They did not confirm the product was reachable by a real client over a real connection.
The problem was the TLS certificate chain. The server sent only the leaf certificate, not the full chain including the CA bundle. Strict TLS clients, which is most production software and every MCP tool, drop connections when the chain is incomplete. They do so without raising an obvious error. From the server side, everything looked healthy. From the client side, the connection simply did not happen.
One command catches this:
openssl s_client -connect host:443 -servername host
The fix was one line: cat leaf ca-bundle > fullchain.pem, then a Traefik reload.
The rule: every smoke test replicates real external client conditions. No -k flags. No localhost shortcuts. curl -k confirms the process runs. It does not confirm the product works. Every new HTTPS endpoint gets an openssl chain validation before the deployment is called complete.
Failure three: the specification that left too much open
This one was quieter. It recurred more often than the other two.
We gave the AI a goal like "build the admin dashboard for this product." The AI built something complete. Navigation, data tables, action buttons, status indicators. It looked like an admin dashboard.
It was not the admin dashboard we had in mind.
The AI made dozens of small decisions we never specified. Which fields to show. Which actions to expose. How to handle empty states. What to do when a query returned no data. Every gap in our instructions became an assumption. The assumptions stacked. The result was coherent but wrong, and wrong in ways that looked right until we used it in context.
The cost was not the wasted build. The cost was the time spent finding and unwinding decisions we had not noticed the AI was making. You do not see the assumptions until you see the output. By then the code is written.
The rule: break every problem into pieces small enough that the AI has no room to assume. One prompt, one task, one verification step. Before any build prompt runs, the spec for that piece must answer every question a reasonable engineer would ask before starting. If the spec leaves a question open, the AI will answer it. You want to answer it first.
What this adds up to
Three failures. Three rules. Each rule is now a gate in our process.
Before a migration ships: count the tables.
Before a deployment is called done: run openssl on the new endpoint, no -k flags.
Before a build prompt runs: close every open question in the spec.
The next article shows how these rules, and others like them, work together as a system, and how this whole approach has made our shipping process faster and more reliable, not slower.
The harness is not documentation. It is not a checklist sitting in a folder no one reads. It is the process, the rules, and the gates that direct the AI to do one thing at a time inside a defined boundary. The AI has no lane outside it.
That is what thirty years of shipping software teaches you. Not which technology to use. Where things break.