Agile Software Development: Dealing with Bugs

Using Agile methodologies for Software Development can be tricky. The concepts of agile are slightly abstract, and that can make them difficult to understand, and difficult to relate back to concrete concepts like time, number of people, and cost. But dealing with software bugs in an agile way is particularly difficult given the very nature of the bugs themselves. So let’s start there.

What’s A Bug?

First, let’s be clear. A “bug” is a particular malfunction of an application; a specific way in which a specific feature performs incorrectly given some precondition. If a test is defined in the traditional given-when-then structure (given a set of conditions is satisfied, when event X happens, then result Y is observed), a bug is defined as the failure of that test. A bug is, by definition, unexpected. You’ve programmed a piece of software to perform a task, and you believe it does so… until someone discovers some conditions under which it doesn’t. There are two general types of bugs: Progression, and Regression. Progression bugs are logged against new features or functionality that has just been developed. Progression bugs can be caused by a lack of understanding of the feature or poor grooming; For example, an edge case in a new feature that wasn’t outlined properly in the definition of work, or maybe just a failure of the developer to do adequate testing. Regression bugs are bugs that are introduced in existing features that previously worked to spec. These types of bugs might have been created as a side effect of other work. Developers might not always know all of the consequences of changing things in the code, depending on the complexity of the product, and every change represents some non-zero risk of introducing regression bugs.

The Problem

As always, good engineering methodology dictates that before we develop a solution, we exhaustively identify and define the problem. In this case, the problem is how to account for bugs within the agile framework of planning, estimation, and measurement. More specifically, how do we measure our capacity and velocity for fixing bugs, and how do we measure our performance? The problem arises because of the very nature of the bugs themselves. Bugs represent holes in our understanding of how the feature or the existing product works, or is supposed to work. Since bugs are unexpected results we didn’t plan for or intend, the work to fix them is difficult to estimate, since we don’t always know what’s causing the bug in the first place. The complexity of the bug is wildly unpredictable, depending on the underlying cause, and part of the complexity of the bug is, in fact, determining that cause. Estimating bug-fix work, and tracking the velocity therefore becomes problematic.

Schools of Thought

According to my Scrum Master colleague Annaliese Johnsen, there are two general approaches to dealing with bugs… you either estimate, groom and track them, or… you don’t. There are pros and cons to each approach. If you choose to try to estimate the complexity of your bugs, your grooming will take much longer, and you’ll probably need more people to do it, since the developers will have to do some amount of digging on the cause of the bug in order to have any idea how complex it will be to fix. And even with that extra time, unless you dig out the detailed cause of each bug in grooming, estimation of fix-complexities can still be dramatically inaccurate. However, the “optics” of planning and tracking bug capacity and velocity is great for your team. Engineering Leadership often makes fixing bugs a priority, and by accounting for and tracking bugs in your overall process, you pad the numbers of what your team has accomplished when leadership takes a look. On the other hand, if you don’t estimate or track bugs, you don’t really get “credit” for fixing them, even though your team is spending capacity on the effort. The flip side of that coin is that, if you give yourselves the stat-boost of planning and tracking bugs, in can be argued that you’re, in effect, rewarding your team for introducing those bugs in the first place. Also, if you don’t bother estimating the bugs, your grooming time will be much shorter than if you do.

The Solution

As with most software dev process problems, there’s no silver bullet solution here. But something that can probably help, no matter which school of thought you subscribe to, is to reserve some predetermined capacity for fixing bugs in every sprint. This works both ways. If you’re estimating your bugs, you can maintain separate capacities for feature work vs bug fixing work, and one overall capacity that’s a combination of the two. If you aren’t estimating your bugs, you simply lower your overall capacity by whatever percentage of the sprint you want to spend fixing bugs, and only plan that much feature work. For example, if you want to spend 50% of you sprint fixing bugs, and you know that you can historically close 100 points of complexity in a given sprint, then you only plan to do 50 points worth of feature work, and spend the excess fixing bugs. By reserving capacity for bug-fix, regardless of whether you estimate and track or don’t, you tune the process to account for the time you want to spend hardening, regardless of the complexity. If you’re grooming bugs, you’ll have a mechanism for tracking your performance and the quality of your estimation, and if you don’t groom bugs, you’ll still have built time into the sprint schedule for hardening, without any causing any detriment to your velocity or capacity.

Personally, I find that the exercise of trying to estimate bugs and then award your team points for closing them is a low-value proposition. The nature of bugs defies estimation without a significant amount of work on the part of the developers, and that work would have to occur inside the current sprint, in order to be able to plan for fixing those bugs in the next sprint. The whole enterprise, in my opinion, just adds unnecessary complexity to the grooming process, which ought to be a tool for you to plan and execute a work strategy, rather than a way to prove your team’s worth to the uppity ups.

But no matter how you prefer to work, reserving a little capacity for bugs is a tiny process change that goes a long way. It will give you the breathing room to fix your defects in an agile-consistent way, no matter whether you estimate their complexity, or not.