• Improvement evaluation


Critical success factors of process improvements are feedback loops. How are we doing in getting improvement suggestions realised? Do the improvement suggestions actually help meeting the improvement objectives? Same thing for the business objectives: are we getting any closer while implementing the improvements? What to do when answers to these questions are negative?

Improvement evaluation

Acceptance criteria
How do we know that the improvements are successfully implemented?

Feedback loop one: on success of improvement suggestions.

  • How to check that improvements have been implemented successfully?
  • During retrospective meetings, a team evaluates the results of improvement suggestions.
  • A ‘definition of done’ of improvements provides useful criteria.
  • Cycle time of this feedback loop best is between days and a couple of weeks (in case of scrum: sprint length).

What if improvements are not successful?

  • Check if the improvement suggestion needs to be adjusted
  • Reschedule for the next cycle

Feedback loop two: on meeting the improvement objectives.

  • The improvement suggestions are based on heuristics, which might work but also might fail in meeting the improvement objectives.
  • The improvement architect measures the effect of the improvements on its objectives on a regular basis.
  • The effect of improvements may not be measurable immediately after completion of an improvement, this takes some time.
  • The cycle time of this feedback loop is longer that the first one, try a couple of sprints.

What if improvements are not helping meeting the improvement objectives?

  • Understand why:
    • maybe the improvement suggestions require adjustments or
    • maybe prerequisites outside the scope prevent positive effects
  • Identify follow up with improvement owner
    • this could be a re-assessment

Feedback loop three: on meeting the business objectives.
The world changes. The time from the start of an improvement project until finishing the last improvement suggestions typically is months, sometimes more than a year. Regular checks that the improvements still fit the business objectives prevent divestment of time and resources. This is done with the improvement owner and senior management. Cycle time is to be decided with the involved stakeholders. Making this shorter than the second feedback loop can be useful if the environment changes fast and influences the business objectives.

What if there is a mismatch?

  • Understand why:
    • failing improvement project
    • changing business objectives
  • Maybe the improvement project needs to be discontinued
  • Maybe the improvement architecture needs to be updated; this could mean a restart from square one