Lean Tracking
Part 2 of 3.
Second of three in the series Agile in the Wild, this article elaborates on how we track our progress at KingdomWorks Studios, a film studio in Stuart, Florida. Here's the first one if you missed it.
Let’s jump in.
Tracking progress
At the end of each sprint, as scrum-master, I take photos of the whiteboard and collect all story cards that made it to done. That data is loaded into a spreadsheet which reveals insights not intuitively gleaned from the whiteboard alone. Every two weeks I spend 15-30 minutes to compute all of our statistics.
This cross-section of data isn’t voluminous. It is not sufficient to answer every conceivable question. It is though the sweet-spot that returns the biggest bang for our buck. It’s lightweight on the time invested away from production work, yet heavy weight for consistent progress tracking. We track for the production team total points started, points left undone, velocity as points done, velocity as percentage, number of stories done, the average point size done, number of stories done which were added after the sprint planning meeting, number of cancelled stories, and total points done per workday.
The Infrastructure Team uses Kanban and we use the same two-week rhythm for evaluating team progress, as we do for the scrum-based Production Team. Kanban requires fewer data points, so we track only velocity as points done, number of stories done, average point size done, and points done per workday.
Thumbnail graphs are produced and provide visual indicators on our short and long term direction. This mini-dashboard drives focused discussions that lead toward calibration and improvements.
(Image intentionally blurred for confidentiality.)
Make the rules as you go.
There are other ways we are different from traditional agile. Maybe the biggest is that we're intentionally non-dogmatic. Meaning, we start with only a wire-frame process and as conditions warrant, layer in lightweight enhancements. Examples:
Sprint slips - We don’t track this per se since it’s self-tracked via the strike-through notation atop each affected card. We intentionally use a handful of key measures to drive the team, this is not one of them. We talk through root-cause analysis in broad strokes. Rosy planning or hasty story card writing is more often the topic.
Total cards - A count of one for each card is useful in determining other metrics. I used to be a fan of card-counts vs. story-points, but for the amount of effort required to generate a story point, we get a good ROI. Story points tell us a lot. None the less it’s easy to count both the number of stories and the number of points, so we do.
Total points - We’ll look at the number committed at the start of the sprint, vs the number in total at the end of the sprint. We look most closely though at the done (aka our velocity) vs the undone and this is how we calculate efficiency.
Average story point size - We tend to have a series of very small deliverables and sometimes that can be habit forming to where we generate small-point stories disproportionately. Since the exercise of writing a card takes effort, we encourage right-sized deliverables vs just small ones. Average story point size helps us see which way we’re leaning.
Cancels – Sometimes legit “stuff happens” but most of the time cancels are pure waste, usually a symptom of rushed planning. If what we thought was important turns into the opposite within a two-week period, something in our planning process is broken. We don’t have many but each one gets our attention.
Adds – Adding a new card after the sprint has started is something we allow. Hard-line agile consultants shudder. We don’t encourage it but we track frequency. We borrowed this notion from Kanban where cards are continually added to the backlog, day to day. If something hot comes up, or if we forgot about a sub-dependent deliverable, we’ll add a card with a star drawn at top center. It’s not ideal, but is a necessary tradeoff since we have chosen to spend, at most, an hour with our sprint planning meeting. Law of diminishing returns having a longer sprint planning process tells us that our best investment is spending an hour or less.
Burn-down - We use the white board’s done column and eye-balling view of the board as a whole. If we’re half-way through the sprint we should see somewhere near half of the cards in the Done column. More specifically, we should be seeing some fairly natural distribution of the cards within their day-of-the-sprint row (or swim lane). Meaning, the first Monday of the two-week sprint should have a few one-pointer stories in it. The first Tuesday should have perhaps a few more ones and twos, and so on. Coupled with the daily scrum meeting, it’s easy to see if we’re distributing our work somewhat evenly across the two-week period. This technique reveals the essence of a horizontal bar chart and daily trend.
Roles - What we don’t do is track by individual person. Great things come from great teams. We track performance at the macro. We don’t have a full-time product manager, customer liaison, or scrum-master. Our executive wears multiple hats, and team members provide plenty of input to make those roles easier. Using our story color card convention, it’s pretty easy to scan the whiteboard and sense how each person is contributing.
That’s how we roll.
In the next and final article of this series, we’ll see how a team journeyed to a point where some atrophy started taking place. That is, how the team, not unlike any other, started off strong but over time started to take agile disciplines for granted. We’ll show you how we handled and implemented one more technique to ensure ongoing success.
---
About the author: Tim Arthur, PMP, PMI-ACP, CSM, MSMOT is an agile and business consultant. He was with for IBM for 17 years as a manufacturing software manager for the Personal Computer lines of business, then with SAS Institute for another 17 in a variety of management and supervisory roles. His last position there was as Distinguished Agile Project & Program Manager. Since retiring in 2016, he’s worked in paid and non-paid positions to help select companies make our world a better place. He’s the full-time Executive Director at Light of the World Charities, a non-profit dental clinic that provides a for-free dental home to under-served children.
