saveenr: Pragmatic Bug Modeling or When will all the bugs be fixed?

[Blog bot]

Источник: http://blogs.msdn.com/b/saveenr/arch...-be-fixed.aspx
==============

 

When your job is to lead a team, you will often be asked to explain when your team will be finished fixing the bugs in their area. Some find it odd that we could answer such a question – how could one predict the future – but I assure you that far from guesswork there is a practical and reasonable approach to addressing this question. I will show you now.

I use this technique when communicating the status of the teams I run. As a manager, the analysis inherent in the method is something I expect when interacting with leaders from other teams.

 

SCENE 1 ACT 1

A project review is taking place in a large meeting room. Joe, the program manger is representing his team’s comprehensive status.

GENERAL MANAGER: Joe, can you explain to me how your team will get to zero bugs?

JOE: We’re are going to try really hard and fix as many as we can every day. I’m sure we can get to zero by the end of the milestone.

This review didn’t go well for Joe. He’s earnest, he may be accurate, but it doesn’t show that he’s thought through the problem. As a result, how can the general manager trust Joe?

 

HOW WE WILL SHOW THE NUMBERS

First, we will decide to explain the bug numbers on a week-by-week basis instead of a day-by-day basis. This lets us deal with fewer values to plot as well as let’s us not have to deal with how weekends are handled.

To compute the compute the bugs for week N is just to take the previous week’s number and add to that the net effect of all activities that affected the bug count. Simple!

 



How we compute (“model”) those activities indicated the strength (or weakness) behind the analysis.

 

WHEN EVERYONE IS ON VACATION

We’ll start with a simple case – everyone is on vacation. Nobody’s entering new bugs, and nobody is fixing them.

 





The cells red text with gray background indicate calculated values – to distinguish them from values one will enter manually.

 

INCOMING BUGS

Every week the team has to deal with people filing new bugs. This we will call the INCOMING RATE. For now I will simply state that 40 new bugs are filed every week.



 



 

BUT HOW MANY FROM THE INCOMING ARE KEPT?

In reality when a team looks at the new bugs filed every day, not every bug is kept. Some percentage are dismissed immediately.

The reasons can be any of the following:

• It’s a duplicate of an existing bug
• It was recently fixed
• The person who filed the bug has a misunderstanding about the issue or performed incorrect steps
• The bug has nothing to do with the code – it is not actionable as by a developer so it shouldn’t count as a “bug” (for example, maybe it is a documentation issue)

It’s not uncommon that 50% of incoming bugs are discarded for reasons like those above. We describe this number as the KEEP RATIO – so if we keep half of all bugs filed then then ration is 0.5. and we APPLY it to the weekly incoming.



As you can see instead of keeping 40 every day we are now only increasing by 20. The situation is bad, but better than before.



 

WHAT IS OUR CAPACITY TO FIX?

Finally we get the developers involved.

All we need to do is take into the account the ability for the developers to fix bugs. Although this might seem low to start off our estimation we simply consider that a developer can fix 1 bug in 1 day or 5 in one week.

With a team of 6 developers our capacity to fix bugs is 6 * 5 = 30 in one week.

Now we can include that number to our calculations and see how our trends look. I’ve added a net change row so that it is obvious what the end result is for a specific week. As you can see we are simply going down by 10 bugs every day.





The situation doesn’t seem as dire now, clearly the trend is favorable.

 

NOW COMES THE MODELING

If you look at my numbers, you’ll notice that only some cells are calculated: the Bugs for weeks 1 through 6 and the Net Change at the bottom. What this means is that we can directly modify incoming, keep ratio, fix capacity based on the weekly conditions.

And this is where the modeling really occurs. Because we are going to take into account the realities of what the team will be doing week-by-week. And this is the level of analysis we need from feature teams – don’t simply look at the bugs as numbers. Apply contextual knowledge about the organization’s behavior and see how that affects the data.

 

EXAMPLE: THE “TEST PASS”

During this period some teams may schedule a “test pass” – essentially a time where the test team will run through every set of test cases and scenarios they can think of.

Let’s suppose that our “test pass” occurs on week 4. Based on some historical data (which would be best) or some guesswork (which is acceptable) we can say that the incoming rate will be 2x the normal one and we will keep 75% of the bugs in that week..

 



 



 

EXAMPLE: RAMPING DOWN DEV & TEST ACTIVITIES

In a given phase of the project not everyone is always doing the same thing. In particular, after the test pass the test team may not be looking as actively for bugs – they may be moving on to planning activities for the next phase of the project. We adjust for that effect by reducing the incoming rate in week 5. Likewise dev may be moving from fixing bugs to writing design documents for the next milestone so we need to account for that as well – in this case we’ll use the example of a single developer moving from bug fixes to planning.

 



 

 

EXAMPLE: ASKING FOR HELP

Our modeling reveals that given what we know and anticipate about the project, we aren’t going to hit zero bugs zoon. Typically at this point we might make a request for developers to be temporarily reallocated from other parts of the product to this one. The general manger will rightly ask how much of a difference will this make. And fortunately because we have modeled the bugs – we can show the effect.

Let’s suppose we asked for two additional developers for w0 and w1, and only 1 for w2, w3, w4. It would affect the numbers like this …



As you can see by the net change in week 6, we will have 2 bugs starting on week 7 – so we have a reasonably expectation that with additional help we will be done on week 7.



 

PARTING THOUGHTS

• Where do the numbers come from? You’ve got to look back to the history of the project – there will always some guesswork and estimation involved. As you collect data week by week you can refine your models as needed.
• This a simple model. As presented I only wanted to give you a flavor of how to start your own analysis and modeling.
• It isn’t always necessary. This as an analytic tool to help teams think about their project. It may be too much overhead for some teams. But having overseen multiple teams for many years at Microsoft, I think it can bring some useful rigor to project execution.

Источник: http://blogs.msdn.com/b/saveenr/arch...-be-fixed.aspx