A report comes in. A senior engineer reads it, opens the code, and ninety seconds later says the most dangerous sentence in our profession: "oh, I know what that is."
They're almost always right about where the problem lives. They're wrong more often than they'd like about why it happens. Out of that gap comes the fix that treats the symptom, closes the ticket, and returns the same bug wearing a different hat six weeks later.
We have a rule that cuts the path off at the root: every bug starts with a test that reproduces the failure and fails against the unfixed code. No test, no fix.
Not a recommendation. Not an aspirational best practice. An entry requirement.
The trouble with "I know what that is" has nothing to do with arrogance. It has to do with a verification loop that's broken in a way that feels good.
When you fix first and verify after, verification means running the scenario and watching it work. But you never established that the scenario failed for the reason you believe. If the bug depended on state you didn't reproduce, your scenario passed before the change too. You've just written code that fixes nothing and convinced yourself otherwise with evidence that can't tell your two hypotheses apart.
The failing test is, quite literally, the only proof that you understood the problem. It's an experiment with a control. Without it, you're guessing in good handwriting.
1. Repro. Exact conditions first: input, prior state, sequence. If we can't reproduce it, that's information in itself, and we'll come back to it below.
2. The failing test. We write it against untouched code and watch it fail. If it passes on the first run, we haven't reproduced the bug; we've reproduced something else. Back to step 1.
3. Root cause. Exact file and line. Not "something in the session module", but the point where behavior diverges from what we expected.
4. Minimal diff. The smallest change that makes the test pass. A bug is not an invitation to refactor the module. If the surrounding code is begging for cleanup, that's a separate PR, and this is where our one-bug-per-PR rule earns its keep.
5. Full suite. Run everything to catch regressions. The new test proves you fixed what you meant to; the suite proves you didn't break what you didn't.
6. Bug doc. Repro, root cause, fix strategy, rejected alternatives and why, test plan, rollback plan.
This is where the rule stopped being an inherited best practice and became the thing that saves our week.
We work with AI agents inside the flow. Describe a bug to an agent and you get a plausible fix in seconds, and that's exactly the problem: plausible is the operative word. The code comes out well formatted, with sensible names, with a comment explaining what it does. Every surface signal a human reviewer uses as a proxy for quality ships included. The one thing that doesn't ship included is any guarantee that it went after the cause rather than the symptom.
The failing test is the filter that can't be fooled by form. Write the test before you ask the agent for anything, and you have an objective criterion the fix either meets or doesn't, so review stops depending on whether the code "looks right".
We also invert the order people assume. We ask the agent for the test first, not the fix. Reproducing a failure from a written repro is a task models are good at and where the risk is low: if the test is wrong, it fails or passes when it shouldn't, and you find out immediately. Writing the fix blind is the task where they're wrong expensively and quietly.
Hura, our technical assessment platform, generates tests from a job description and evaluates answers using two models in parallel. That breaks the silent assumption underneath almost every test suite: that the same input produces the same output.
You can't write assert answer == "expected text" against a model. And by our own rule, not being able to write the test meant not touching the code. So we had to change what the test asserts.
What we do now:
Test the contract, not the prose. That the evaluation returns every required field, that the score lands inside the valid range, that each criterion from the role carries its justification, that the ordering of candidates is consistent with the scores.
Test the business invariants. Two identical answers must receive the same grade. An empty answer can never pass. Switching language can't change the outcome, since everything a user sees ships in both Spanish and English.
Treat disagreement between models as a signal, not an error. When the two evaluators diverge more than expected, that's a case deserving human review, and that condition is tested too.
The principle survives intact: if you can't write a check that would have failed with the bug present, you don't understand the bug yet. What changes is that the check asserts properties instead of literals.
This case is the interesting one, because the rule turns a frustration into a diagnosis.
If you can't reproduce the bug in a test, you're almost always in one of three situations:
You don't understand the trigger yet. A missing state variable, an event ordering, a concurrency condition. Being unable to write the test is the early warning that any fix you write right now is a bet.
The code isn't testable at that point. Hard dependencies, wall-clock time, side effects you can't reach. That's a finding about the design, and it's often the real bug.
The failure isn't where you think. It's in configuration, in the data, or at the boundary with an external system.
In all three the useful response is the same: keep working on the reproduction, not the fix. It's slower and it feels worse. It's also why we no longer have a category called "ghost bugs that sometimes come back".
And yes, this isn't free. There are trivial bugs where the full ritual feels disproportionate, and teams with slow suites where running everything on each fix genuinely hurts. Our position is that the ritual scales with risk, but step 2, the failing test, is never negotiable, not even on the trivial ones. Trivial is precisely where "I know what that is" is wrong more often than it feels.
Step 6 draws the most resistance up front and pays the most over time.
We keep a living log of every triaged bug. It isn't an archive of closed tickets, which everyone has and nobody reads. It's a decision record: what broke, why it actually broke, what we considered doing, and why we chose otherwise.
That record changes two very specific conversations. When something reappears with a familiar symptom, the opening question stops being "does anyone remember whether we've seen this?" and becomes a search. And in onboarding, the fastest way for someone to understand a system isn't the architecture doc, it's reading how the thing broke.
There's a third reader we didn't plan for: the agent. The bug doc is the context that stops it from proposing, for the second time, the alternative we rejected eight months ago.
Make "write the failing test before touching the code" an entry requirement rather than a good habit. And if you have AI agents writing fixes, promote it further: ask the agent for the test before the fix, because generated code arrives with every surface signal of quality and no guarantee it understood the cause. When the system isn't deterministic, change what the test asserts, not whether you write one.
About Aztia. We're a software development firm. We build Hura, our technical assessment platform (huraapp.com), holding ourselves to the same process we describe in this series. More at aztia.co.
Thirty minutes, no pitch. Tell us what you're building.
Talk to us →