When I join a new team, I am fascinated to watch how quality issues are handled. Application users complain about bugs in production, testers report issues during testing, and developers try to fix ill-defined bugs… or argue that there is no bug! Most organizations have a defect tracking mechanism, whether it is email, a spreadsheet, Slack, or a formal issue tracking tool. However, I find that many don’t have a simple approach for how to report bugs. They either have no process or a scary-long template with lots of fields that don’t typically matter.
In both cases, a lot of time is wasted for multiple people from ineffective bug reports. How many times do you see a report like “the app is broken”? This is followed by wasted time going back and forth:
“what is broken?”
"the inventory app"
“what page are you on?”
"the item page"
“it works for me”
"the picture doesn't display"
“which item are you looking at?”
If the discussion is in a shared messaging app, then dozens of people waste their time reading that back and forth!
Another problem comes up when the bug reporter thinks the issue is obvious, and just reports:
“Fix this! http://ourdomain.com/page_with_issue”
This kind of report fails if the issue isn’t the default behavior of the page, or just isn't obvious. It also fails if time passes. Looking at the page three months later, one can’t tell what the report is about, as the original issue has since been fixed. Conscientious people scrutinize the page for all manner of issues that aren’t actually issues any more, wasting time they could have spent looking for Waldo (with equal business value).
On the other end of the spectrum, there are “IRS 1040” style defect reports that take forever to fill out. I’ve seen organizations where folks perform extensive workarounds to deal with a series of little application problems instead of reporting them! As they continue working around the issue, often bigger data integrity problems arise than the original bugs. Execution is more costly, training is more costly, upgrading is harder, and overall people don’t feel good about doing their work.
Often, once a developer can actually see the defect, fixing it is very straightforward. If the software team is following good principles, the defect report can probably be fixed with ease. As such, an effective issue reporting approach might be one that optimizes how easy it is to “see the issue”.
An Effective Issue Report
-
SIMPLE, not so hard that people avoid reporting minor issues
-
FAST, don’t waste time on details that don’t matter 90% of the time
-
EFFECTIVE, write so anyone can easily reproduce
Simple Issue Report
Template:
START: [URL …or… mobile OS + app]
STEPS:
1. [list the exact steps it takes to reproduce the problem]
2. [use numbers to show each step in order]
3. [include multiple apps if necessary]
4. [write it so a 10 year old could reproduce]
ACTUAL: [identify the thing that is broken, screenshots help]
EXPECTED: [explain what you expected to see]
URGENCY: [only for critical production issue reports]
Example:
START: http://devserver.ourcompany.com/search?name=boo&status=hoo
STEPS:
1. notice that the first 100 results are visible
2. scroll to the bottom of the page
3. click “Next” button
ACTUAL: page 2 with header & footer around empty results section
EXPECTED: page 2 with header & footer with results 101 and above
Following this pattern allows reporters to quickly communicate all of the critical details of a bug report, while providing the minimum information to reproduce and understand the defect 90% of the time.
For the 10% of the time when this pattern isn’t sufficient, it is enough detail for the person reading to know exactly what questions they need to ask, and makes for a very easy conversation to close the loop.
So, 90% of the time, total understanding, 10% of the time, an efficient discussion. What isn’t to love?
But, But, But!
But… you aren’t including crucial information, like OS version, app version, environment, severity, priority, etc.!
Frequently, these details simply don’t matter. For most internal applications, everyone is using the same OS and browser anyway. For mobile apps, often knowing iOS or Android is all you need to know (although an Android app might need OS version). When something else does matter, the reader can ask the author.
But… the EXPECTED value is obviously the opposite of the ACTUAL! Why include both?
Well, frequently the bug is something that you don’t see, or a calculated value that a programmer wouldn’t necessarily understand. By explicitly stating the EXPECTED functionality, all ambiguity is removed.
But… we just have to report the priority. And the severity, too! How will anyone know how bad the bug is?
Just because someone found a bug, they can’t necessarily tell how important or severe a bug report is. For end users, testers, or developers, it is often hard to understand the business priority or impact. As such, it is reasonable for product owners or someone with business knowledge to manage the PRIORITY and SEVERITY designations after the report is submitted.
But… what about the build and environment?
This is typically obvious from the START, as a URL typically points directly at the environment. Otherwise, if important, it is easy to add as a STEP.
But… what about the data conditions! How can you reproduce if you don’t know the data?
If the START and the STEPS are specific enough, the data conditions will be encountered by the reader of the report.
The easier it is for people to reproduce the report, the easier it is to understand, prioritize, and resolve. It is important to foster an environment where people report bugs so they can be fixed instead of creating damaging or time-consuming workarounds.
One More Thing…
As a bonus, a Simple Issue Report is also a great Simple Enhancement Report, just change the word EXPECTED to NEW. That’s it!
Enjoy all the time your team will save!
If you’d like to work with software developers who always write an automated test for each feature they implement, give us a call at 8th Light. One of our core principles is to deliver quality through well-crafted software.