Commitment considered harmful

Disclaimer: This content falls into the realm of Systems Thinking. Systems which include humans are complex and changes to them often result in unintended consequences. As this post covers ‘productivity’ metrics we should tread carefully and remember the warning from Eliyahu M. Goldratt, “Tell me how you measure me, and I will tell you how I will behave.”

Estimating the amount of work that will be completed has always been a controversial topic. In 2011 the Scrum guide was updated and the term commitment was replaced with forecast. Regardless of the terminology, most Scrum teams make a guess as to how much they will do in a sprint, and this is typically reported to management.

There are generally three different techniques teams use for deciding on the forecast . A team can use a ‘gut feeling’ and take an educated guess as to how much will be completed. Alternatively a team may take an average of the last three to five sprints and use that as the forecast. Thirdly, there is a technique called yesterdays weather. This is when the actual points delivered in the previous sprint are used as the forecast for the next sprint. I have concerns about all three of these approaches. I feel that all of these approaches lead a team into overcommitting.

When teams use their ‘gut feeling’ they are subject to a number of biases. There is the planning fallacy which is the tendency for people to underestimate how long it will take to complete a task, even when they have done similar tasks before. Hofstadter’s Law states that “it always takes longer than you expect, even when you take into account Hofstadter’s Law.” On scrumplop.org they point out that motivated teams naturally set higher goals for themselves, however this does not always result in an improvement.

It’s human nature that individuals and teams with self-esteem set increasingly higher goals for themselves. By trying to achieve these goals, teams learn. Sometimes such strivings leads to immediate improvement, particularly when the team challenges itself to improve through use of a new-found technique or technology. Sometimes, however, the new technique doesn’t pan out and the team’s performance remains the same or even gets worse.

Sometimes, the team raises the bar just to raise the bar, either to test the limits or to unconsciously publish its bravado. Even these self-challenges can pay off, especially in the short term. It is more common that these higher levels are unsustainable in the long term.

Therefore: In most cases, the number of Estimation Points completed in the last Sprint is the most reliable predictor of how many Estimation Points will be completed in the next Sprint.

The quote above ends with scrumplop.org recommending yesterdays weather as the most reliable way to forcast the work that will be done in the next sprint. Unlike using your ‘gut feeling’ with ‘’yesterdays weather’ the team can’t simply choose a high velocity (potentially hiding from reality). In order to get a high commitment, you need to have actually delivered a high amount in the previous sprint.

This may work for mature teams with stable platforms, it may however be less successful when there are many disruptions e.g. changes in personal, many technical unknowns, unpredictable development and test environments e.t.c. The following diagram is data taken from a real project. It shows actual delivery plotted against a retrofitted forecast based on yesterdays weather. In this case the team would have met the goal ~52% of the time. This may not be consistent enough for the teams planning needs and this success rate may negatively affect the teams motivation over time.

ForecastYesterdays Weather

Forecast based on ‘Yesterdays Weather’

Note: The forecastscommitments have been retrofitted to the graph. The team may have had different results had they used yesterdays weather during planning.

If the actual velocity is varying enough to make yesterdays weather unreliable, a team may decide to take the average of the last few sprints (typically 3 – 6 sprints). If we consider the definition of average, this means that about half of the actuals will fall above the average and about half will be below. Statistically speaking we only have a 50% chance of meeting the forecast if we base this on the average.

A retrofitted forecast based on the total average

A retrofitted forecast based on the total average, means that the commitment is met 50% of the time

A retrofitted forecast based on a rolling average of 5 sprints

A retrofitted forecast based on a rolling average of 5 sprints, means that the commitment is met 46% of the time

Due to the reasons stated above I believe that these, well intentioned, forecasting techniques may lead a team into over committing on a regular basis. Not only does this make release planning and budgeting difficult, overtime this can have a detrimental effect on a team. Team members may stop caring about meeting the commitment and sprint goal. This can result in a general lack of motivation. If your team is suffering from this, you may want to try using a control chart to help you set realistic forecasts.

Control Charts1

Control Charts are typically used to determine if a manufacturing or business process is in a state of statistical control. Carl Berardinelli explains that there are four process states (seen above). He states that “every process falls into one of these states at any given time, but will not remain in that state. All processes will migrate toward the state of chaos. Companies typically begin some type of improvement effort when a process reaches the state of chaos (although arguably they would be better served to initiate improvement plans at the brink of chaos or threshold state). Control charts are robust and effective tools to use as part of the strategy used to detect this natural process degradation”.

ControlCharts2

The following link shows children explaining how Flowcharts and Control Charts are used at their kindergarten.

Above is a Control Chart derived from a teams velocity

Above is a Control Chart derived from a teams velocity. The dark blue line is the actual velocity over the past 29 sprints. The grey (gray?) line represents the average actual velocity. The red lines show the Upper and Lower control limits. These are plotted at one standard deviation from the average.

Continue reading

In Theory and In Practice

We recently completed a small but interesting project at work. The team consisted of Tim Weeks, Lee Walton and I played a supporting role. We shared the lessons learned in an experience report at the Agile Practitioners group. The talk was filmed and can be seen below.

On this project, the developers tried a number of Agile and XP practices. BDD and Pair Programming were tried for the first time, in the video they explain what this was like.

In addition to the Agile aspects, the talk shows what Lean (kanban) principles we followed e.g. making decisions at the last responsible moment.

There is a large section on using code reviews and code metrics to help improve code quality.

I got off to a shaky start, but things seemed to improve as the video goes on.

Part 1:

  • Overview of the project
  • Developers discuss how they felt on the project
  • Discussion on pair programming for the first time
  • The importance of putting people first
  • NUMMI case study (GM and Toyota)
  • Start of project – beginning with the end in mind
  • Re-writing stories, to remove implementation details
  • Simplicity – the art of not doing work
  • Cost of defects
  • Concurrent development
  • Making decisions at the last responsible moment

Part 2:

  • Burn down charts
  • Cumulative flow diagram
  • Walking skeleton or tracer bullet
  • Lean thinking – limiting work in process
  • Pull System
  • Dealing with blocked stories
  • Code Reviews (brief discussion)
  • Relationship with PO – developers having access to users
  • Task switching – cost of task switching on projects

Continue reading

The Buffering Law observed in Software Engineering

Railway buffer at the top of Pikes Peak, Colorado Springs CO

There is an informative slide deck on the FactoryPhysics website entitled A Fast Cycle Time Story. This slide deck was created by two Intel Products employees namely Tim Skowronski and Joan Tafoya.

The slides cover many aspects of lean manufacturing, one of the topics covered is The Buffering Law

Systems with variability must be buffered by some combination of:

  • Inventory
  • Capacity
  • Time

Tafoya and Skowronski explain the implication of this law

If you cannot pay to reduce variability, you will pay in terms of high WIP, under-utilized capacity, or reduced customer service i.e. lost sales, long lead times, and/or late deliveries.

The following variability buffering examples are provided:

  • Ballpoint Pens:
  • can’t buffer with time (who will backorder a cheap pen?)
  • can’t buffer with capacity (too expensive, and slow)
  • must buffer with inventory
  • Ambulance Service:
    • can’t buffer with inventory (an inventory of trips to hospitals?)
    • can’t buffer with time (response time is the key measure)
    • must buffer with capacity
  • Organ Transplants:
    • can’t buffer with WIP (perishable – very short usable life)
    • can’t buffer with capacity (we cannot ethically increase capacity)
    • must buffer with time

    Continue reading

    The X Penny Game

    The X Penny Game is a simulation game exploring the effects of WIP limits. It is a combination of Karen Greaves modified Scrum Penny game with Karl Scotland’s version of the Ball Flow Game.

    This game is geared to show the importance of limiting the work in progress and to explore the following formula (implied by Little’s Law)

    Flow Time = WIP/Throughput

    • Flow Time (Cycle Time, Lead Time) – average amount of time it takes to fully complete a unit of work
    • WIP (Work In Process) – is the average amount of units in the system
    • Throughput – average number of units being completed within a given time frame

    The game is designed to work with 6 to 12 people – we had 8 players and 1 facilitator

    The team divides into the following roles:

    • 1 – Customer
    • 5 – Workers
    • The rest are efficiency experts

    We had a total of 8 players (6 workers, 2 efficiency experts).

    The team organises themselves around a table. The image below shows how our team arranged themselves. Each worker has an empty area on the table directly in from of them referred to as a workspace.

    Continue reading

    Avatars and not acting like a manager

    Recently we started using avatars on our Kanban board. We are certainly not the first to do this, however I was surprised by how excited and rejuvenated the team became. If you have never tried this I recommend you suggest it to your team and see if they go for it.

    Not only is it fun, but it is also practical. With one look at our board we can see exactly who is working on what and who they are working with. Each person has one avatar on the board, therefore they can only work on one task at a time. Occasionally someone will start work without updating the board, but this is very seldom.

    We went for the classic South Park characters, each team member got to draw themselves, here is the result:

    You may have noticed that my character (Daryn) is a bit stressed out. This is a reminder to myself, that as the ScrumMaster I should not get overly concerned about everything. I need to trust in the team and leave them alone to get the job done. I need to remember that a ScrumMaster is a facilitator and not a manager. For me, not acting like a manager is the most difficult part about being a ScrumMaster.