We take responsibility for the correctness of our code by testing it thoroughly.
We do not tolerate preventable defects. —from the 8th Light Principles of Well-Crafted Software

Bugs. Defects. Errors, whatever you want to call them they are the scourge of our industry, and they will continue to be so until we take them seriously.

"But Eric!" you say, "We already take our defects seriously! We categorize them, prioritize them, describe them. We have a huuuge database and a gigantic QA department. How can you say we don't take our defects seriously?"

Simple: If you took your defects seriously you would fix them!

Think of all the things I just mentioned. Huge QA departments running manual test scripts, bug tracking databases, bug prioritization meetings, etc. all exist because as an industry we have accepted the premise that software will have a huge number of defects. So many that we can't possibly fix all of them. Instead we track them, and fix only the "important" ones.

At 8th Light we reject this premise. It is not that we believe we don't make mistakes, but we work as hard as we can to prevent them and take responsibility for fixing them.

We Test Thoroughly

We practice Test Driven Development and it's not negotiable. If you want to "save time" by eliminating this, you'll get a straightforward answer.

No.

Any 8th Light team will immediately begin writing tests first, and will write tests for any untested code they encounter. But TDD alone will not produce a quality product.

Each team will add higher-level testing suitable to the context of the project. We may use Fitnesse, Cucumber, Selenium and others. We'll write integration tests as necessary. When we find a defect we will write a test to make sure it never happens again.

Finally we will manually test, and in a special way. In the third edition of this series Paul detailed our demo process. He mentioned that the demo is practiced, which is something we developed on one of our earliest projects. There are many reasons to practice the demo but one of the biggest benefit is catching missed requirements. I couldn't list the number of times one team member would demonstrate a feature only to realize they had left something out or made a simple mistake, usually before the person they were demonstrating to could even catch it. The simple act of walking through the feature slowly proved immensely beneficial.

We Take Responsibility

Have you ever been in a meeting, or meetings, trying to determine the root cause of an expensive defect? Development, QA and management are all represented, and then somebody will ask QA:

"So how come YOU didn't catch this bug?"

It's said with an accusatory emphasis on YOU because after all it's QA's job to catch bugs and if they aren't catching them, then what are they doing? The poor QA lead fumbles around, mentions they'll add a test for it so it won't happen next time, and development goes on doing the same thing they've always done.

Of course the question should be:

"Why did you write that bug?" or better "Why did you ALL miss this bug?"

And the one and only correct answer to this question is really quite simple:

"We screwed up."

You may substitute your own colorful phrasing if you like, but the point is the same. If a mistake was made it's because development made it, not because QA didn't see it, and ideally it's more than one person on the development team. 8th Light backs this up this responsibility in the most clear way possible, financially. On the craftsmanship page of 8th Light's website you'll find the following quote:

If we make a mistake, we'll fix it quickly—for free. It's that simple.

I've told other people we do this. I've told them stories about us working for free for days or weeks at a time while we correct large errors. Usually they claim we're idiots, that we'll lose all our money fixing defects for free, because they accept the premise that large amounts of defects are inevitable.

As evidence to the contrary I point out that we've had this policy for as long as I've been an employee, and in that time we've grown from 5 employees to 40. It seems to be working out just fine.

We Do Not Tolerate Preventable Defects

Why do we "manage" defects? The Wikipedia entry comparing issue-tracking systems has 53 entries in it. Do you know what the best issue tracking system is? A piece of paper, with the defects written on it. Cross them off as you fix them. The end. Why the complications of classification, prioritization, assignment and all that?

It only makes sense if we aren't going to fix them all. We're going to tolerate many of them, but not "too" many, so we have complicated systems to track them all. I find the notion offensive both to my client and to the end user.

A simple todo list works just fine if you actually fix the defects. On my projects I've found the best system is to stick defects in the next iteration. Sometimes you can fix them all in a given week, sometimes you can't, and if you find you don't regularly get to 0 known defects then you've got to bite the bullet and and do nothing but fix defects for a while. Yes this will delay features shipping occasionally, but working in nearly bug-free code will mean you work faster the overwhelming majority of the time.

I've worked on products with hundreds, even thousands of open defects. These defects were usually tolerated because individually they were small, but collectively they made for extremely slow development and angry customers. When you've reached that number of defects you really only have one option, a complete rewrite. I assure that's far more expensive than occasionally moving out a ship date by a week or two to fix bugs.

Real World

Whenever I get on my soapbox about defects people start giving me reasons why it can't work.

• "You'd never ship!"
• "What about performance?" or "memory?" or "a feature they didn't like but works?"
• "Your customers will take advantage of you!"

The truth is most of the time these problems take care of themselves. We can argue all day on the edges of it, in particular what it means for something to be a bug, and someday I may write about it, but my answer now is to say I work for a company that fixes every defect. And it works.