Lasse's weblog

Planning Poker 101

Tue, 22 Apr 2008 04:04:57 GMT

Ever heard of planning poker? If you haven't, continue reading. If you know what planning poker is and don't feel like reading yet another intro, jump over to The Beef. No, wait. I think you should read the whole thing. If for no other reason than to see where you and I differ in opinion or style. You might even be inspired to post a comment or a trackback or whatever folks nowadays do in the blogosphere.

The Intro

Planning poker is a group estimation technique popularized by Mike Cohn. And it's focused around playing cards - hence the name. It's not really poker but you often see teams using a deck of playing cards just like the ones you use for ripping off your mates in Texas Hold'em. At gunpoint, I might identify one similarity between planning poker and hold'em, though. You don't know what cards are the other players holding.

Ok, enough with the mystery. Here's how it works.

You're in a planning meeting for the next iteration. You have a bunch of features to implement and you need to figure out which or how many of them you can implement in your two-week iteration. There's plenty of things you and your team needs to estimate and a limited amount of time to spend on it. Let's say you're using story points. So, you pull out a deck of cards.

You deal out sets of six cards (A,2,3,5,8,K) to each team member - everyone gets an Ace and a King and cards of rank 2, 3, 5 and 8. Next, you name the feature you're estimating and maybe read out loud some details such as a short description, the acceptance criteria, or something like that. A brief discussion ensues where team members ask clarifying questions about the feature. Then, one by one, the team members place their hand on the table, holding one of their cards face down. Once everyone has their hand (and card) on the table, you count to three and turn the cards over. The rank your card has expresses your estimate for whatever you're estimating. The Ace is a "1", the King is "too big to estimate". Everything else is just that - the rank.

Some teams find that plain old playing cards cramp their style and order specially designed planning poker cards from Mike Cohn, Crisp, Nordija or PlanningPokerCards.com. Or, if you're working with us, just ask and we'll bring along a couple of decks or our cards. Our cards are obviously better looking than the rest but they all work quite nicely.

As you look around the table and your team members' cards, you see one of three things: a consensus, numbers all over the range, or something in between. If there's a consensus you write down the estimate - you all agree so there's no point in going through motions. In any other case, the highest and lowest estimate explain the rationale behind their estimate - why did they give that particular estimate? A discussion ensues and, after some 53 seconds of exchanging thoughts, people start putting their hands on the table again - with a card face down, indicating that they're ready to have another go at estimating whatever you were just estimating. The odds are, you now either get consensus or don't. (Yes, I did figure that out on my own.) If you get consensus, write down the estimate and move on. If you don't, have another go at arguing for the high/low estimate and re-estimate. If you still don't get consensus (or near consensus), you either take the highest estimate or the average. It doesn't really matter all that much which approach you take - just decide the protocol with the team before the planning session.

There. You've got all the items estimated and there's still 12 minutes before someone takes over the meeting room. And, lo and behold, those estimates included not just the input of Jack and Bill who are always keen to voice their opinion about anything and everything but also that of Steve and Melinda who usually keep their mouth shut while Jack and Bill argue about effort estimates. This planning poker stuff isn't just fast and lightweight. It's also helping our team commit to its promises simply by involving the whole team in the estimation process!

That's planning poker. Simple. Fast. Effective. However, when people get introduced to planning poker and go about their first planning session using it, something happens that I really dislike. It has to do with what you do when you don't immediately get consensus. Let me explain.

The Beef [with Average and Median]

My beef is with people intuitively beginning to use an "average" or "median" algorithm for resolving differences between team members' estimates when they first see those cards coming up without a consensus.

The phenomenon of thinking ahead, trying to find a "solution" in parallel to receiving information is human nature. You can see it in, for example, a software developer starting to imagine what the system architecture will be already after reading the first two pages of the requirements specification.

This, I understand. I do it all the time.

But why average and why median? Why do we take these as some kind of a "safe bet", trusting that the truth must be close to the majority? Isn't it possible that the one who's in the minority knows more than the others?

Using an average or median algorithm for resolving differences (rather than trying to find a consensus through discussion) is like imagining the system architecture based on the first section of the spec. It's simply too soon! Just like the requirements specification always hides something you didn't quite take into consideration with your immediate vision of the architecture, the few minority votes behind those poker cards always include some very useful information that's worth unrevealing.

Really? Always? Isn't it the case that every now and then the minority vote was due to lack of knowledge about the domain, the code base, or an oversight of some kind?

Yes.

And that's why you should bring it up.

It's not just about getting the best possible estimate with the available knowledge. It's also about sharing that knowledge and improving those estimates in the future.

So bring up those arguments and try to get that consensus before resorting to the last word of average or median.

On Story Points

Wed, 16 Apr 2008 21:33:06 GMT

I've mentioned this before but most of my work these days revolves around helping organizations and software development teams adopt agile methods or otherwise restructure their operations. Specifically, a lot of our clients (like most teams at Reaktor) are using or looking at Scrum so I get to see a lot of teams take their first steps in that direction. And I get asked similar questions over and over again. While there's really no single "correct" answer to most of those questions, some of them I tend to respond to with an almost identical answer or suggestion.

Earlier today I found an old index card with a note sketched vertically along the side saying, "[empty tick box] story points for product backlog", with the word "product" underlined. I don't really remember what I was trying to say to myself with that note (that happens a lot with notes I've made more than a few weeks ago) but I interpret it to mean that the team I had been consulting at that time had decided to use story points for estimating tasks for their sprint backlogs - and I wanted to bring forth some thoughts about that at the next suitable moment.

Now, the use of story points has been the topic of many of those recurring questions I get asked so I decided to (gasp) blog about. I haven't done this in a while so it's kind of exciting. Anyway, here's a short introduction to story points and, related to that, what I often suggest to teams I coach or consult with.

STORY POINTS - THE DIRECTORLASSE'S CUT

Scene 1: Relative estimation

Story points are an abstract unit of effort (or complexity but I'll stick with effort for now). They're not hours. They're not days. They're not quarters-of-a-day. They have no defined relationship with any time unit you can think of. What they do have in common is the scale - it's linear and proportional. In other words, the distance between "1 story point" and "2 story points" is the same as the distance between "2" and "3", between "3" and "4", and so forth. The measures are relative and can be compared with each other. For example, 2 story points is twice the size of 1 story point.

So we know that story points are an abstract unit of effort. What do we measure in story points, then? We use story points for estimating the effort of implementing user stories (or whatever format you prefer for expressing requirements or desired features for your product). In other words, we assign story points (or just "points") to features based on how much effort we estimate them to require. And, remember, story points are not a measure of time but effort - not even a measure of effort in time. A measure of effort. Period.

The advantage of estimating in story points (compared to, say, effective engineering hours or calendar days) is speed. With story points we're estimating relatively, comparing items of work to each other and placing them into virtual buckets representing our chosen scale. Items of size "1" go with other items of size "1". Items of size "3" go with other items of size "3", and so forth. When we encounter a story that's almost twice as much work as an item in the 3-point bucket, we throw that in the 5-point bucket. It's all relative. And that's what makes it fast - we (humans) are good at comparing things. We're not that good at estimating how long it will take to make that point-of-sale system support an additional type of campaign pricing scheme. With relative estimation, we can plow through many more backlog items with equal accuracy (and I mean accuracy, not precision) compared to breaking them all up to smaller tasks, estimating them, adding them up, and slapping some buffer on top.

So where does this take us? We have a technique for quickly generating estimates by comparing the items relatively to each other but those estimates are presented in abstract units of effort that have no relationship to the Gregorian calendar or time - let alone the project's deadline. How do we know how long it will take to build the product or finish the project?

Well, only the almighty god would know the answer - on a good day - but we do have the means for producing a useful prediction. It's called an experiment. We simply start working on a couple of those items and see how much we could accomplish in a given period of time - how many story points worth of features did we implement?

If we completed 18 story points in the first week and the product backlog has a total of 440 story points remaining, we can quite confidently say that the project will finish within 4-18 months. After the second week, with 14 more story points completed, we can quite confidently say that the project will finish within 8-16 months. After the third week, with 15 more story points completed, we can quite confidently say that the project will finish within 7-12 months. After the fourth week, we're again that much more confident that our velocity - the pace at which we complete features - is representative of the rest of the project and yields a reliable prediction for the overall completion date.

We won't be 100% certain before all features have been delivered. And that goes for any other estimation techniques, too.

Scene 2: Calibrating the scale

Now this is all fine and dandy but what if we need to make a prediction about the completion date without the luxury of working on the project first for a few weeks? That's where calibration comes into the picture.

Calibration, referring to calibrating the story point scale, is an activity where a small number of items are analyzed more closely - the traditional work breakdown structure approach - in an effort to produce a time-based estimate, for example, in effective engineering hours (also referred to as ideal engineering hours) or calendar days.

For example, you could take items that are roughly 10 hours, 20 hours, and 30 hours of effort in effective engineering hours. Then you would assign these items a story point-based estimate, effectively anchoring your story point scale to time. For instance, those three items you picked could be assigned 3, 5, and 8 story points, respectively. It's not a mathematically solid translation but it's close enough.

Now, if you need to make that prediction about the completion date, you estimate the product backlog relatively in story points, add it all up, and translate back to calendar time with whatever formula you used for calibration, not forgetting to translate from effective engineering hours to calendar time. What you should forget at this point is the formula – it has served its purpose and is now just mental baggage, seducing our feeble minds to continue translating back and forth between hours and points.

But doesn't this mean that the story points do relate to time after all? Where's the catch?

Yes, there is a relation between story points and time. I apologize for claiming otherwise. I would like to point out, however, that I did it with good intentions. You see, whenever we deal with story points we should think in terms of story points and relative estimates, not making the mental translation back and forth between hours and points. When we start doing that, we start losing the advantage of relative estimation. The neurolinguistic programming we do with "points" versus "hours" is needed in order to help us get to (and stay in) the relative estimation mindset.

Scene 3: Scales come in different shapes

In the calibration example above, I suggested that the scale was set with 3, 5 and 8 story points for items of an estimated 10, 20, and 30 effective engineering hours, respectively. Why that mapping? Wouldn't it be easier to just keep the numbers and rename the unit to "story points"?

First of all, that whole NLP stuff isn't just fluff. "Just" calling them story points doesn't do the trick. It's not enough to turn that mental switch from hours to points. Some kind of a (preferably non-trivial) translation is necessary. Second, the larger the numbers get the more difficult it becomes for our relative comparison to work effectively. We're really good at telling whether item A is twice or three times the size of item B. We suck at telling whether item C is 9 or 42 times the size of item D. We need a small scale to make this work. As a rule of thumb, anything with two digits is already too big.

The mapping from 10 to 3, 20 to 5, and 30 to 8 was not purely coincidental either. I could've just divided everything by four and gotten 2.5, 5 and 7.5 – nice and small numbers between 1 and 10. Floating-point numbers are too complicated, though, so that wouldn't work. I also could've divided by four and rounded down to, say, 2, 5, and 8. But I didn't. Why?

First of all, that would've been quite all right. I just happen to like the practice of making all story point estimates fit into the Fibonacci sequence-based scale of 1, 2, 3, 5 and 8. It might seem like a small difference but I've found that a more limited scale like this (compared to the full scale from 1-8) helps me think in relative terms when I can't assign a "4" or a "6" on a story. Besides, the increasing gap between the available values quite nicely represents the increase in uncertainty when estimating bigger items. Many other teams and consultants have found this scale useful, too. It's not something I'd fight for but I like it and recommend it.

My second favorite would be something like a scale limited to 1, 2, 4 and 8. This would be simpler but for some reason I'm not sure I'm fully aware of, I prefer the Fibonacci sequence. It probably has to do with what my colleague Jukka pointed out – the Fibonacci sequence has slightly smaller gaps and, given a backlog item to estimate, it’s that much less likely to fall ‘in the between’ the available values compared to the 1-2-4-8 scale. In other words, the gaps of the Fibonacci sequence just feel right for me.

Scene 4: The note

Remember that old note on an index card I mentioned in the beginning of this rather lengthy blog post? It related to a team using story points for estimating their sprint backlog and I had something to say about that practice. Here's that something.

Most consultants and practitioners recommend using story points for estimating the product backlog and effective engineering hours for estimating the sprint backlog. Myself included. Why is that? As I mentioned earlier, one of the major advantages of estimating relatively in story points is that we can do it quickly for even a large list of backlog items because we're not entrenching ourselves in the nitty-gritty details of what a given backlog item entails in terms of implementation - the rough estimates in the product backlog are quite sufficient because the all of those variances cancel each other out in the long run.

For a sprint backlog of, say, two weeks of work, that canceling out doesn't happen quite as effectively as it does for several months of work. In other words, when we're estimating the sprint backlog - trying to figure out whether we can deliver a given set of backlog items in those two weeks or not - we benefit from more detail, more analysis, more effort put into the estimation. While the story point estimates for our product backlog items indicate how much we tend to deliver in two weeks, time-based estimates increase the certainty of that tendency. For many teams, this increase feels significant enough to justify the higher effort.

Some teams, however, are doing great by using story points for estimating their sprint backlog, too. Having broken down backlog items into tasks and established (anchored) a scale for technical tasks, they routinely run through the sprint backlog, throwing tasks into buckets according to their "task point" scale. That scale is typically different from the one used for the product backlog, mind you. Using the same scale would practically push the story point values beyond the comfort zone of single-digit estimates and could possibly bias the estimates given for technical tasks. We (humans) are good at making the world fit a pattern. That's also true for making task estimates add up to story point estimates so I recommend sticking to a different scale.

While I do prefer and recommend keeping story points and relative estimation in the realm of the product backlog, I have seen relative, point-based estimation work quite well for sprint backlogs, too. Just ensure that the points are small enough – most tasks should be doable in less than a day. I know it can work and if it does, it's quite a breeze to do sprint planning (assuming that there's no bottleneck in, say, access to information about the problem domain or in our understanding of the code base). Still, I acknowledge that it's more intuitive and probably - on average - more accurate to estimate the technical tasks of a sprint backlog in terms of engineering hours.

There. I think I've said everything I wanted to. Except the things I forgot already. Well, that's what the blog comments are for...