• Assessment approach

 

The improvement architecture (Business objectives, Improvement objectives, Scope, Context and Constraints) is used for determining an appropriate approach for the assessment.

Assessment approach

Oracles
Oracles help us recognizing the (detailed) problems that reduce or limit the ability of an organization to meet its objectives. Models are often used for this purpose.  The experience of the improvement architect and other involved persons also provide important oracles during an assessment. (Refer to www.developsense.com/resources/Oracles.pdf [Michael Bolton] for an explanation of oracles in the context of testing).

Improvement models
Models are useful tools for improvement projects. During an assessment improvement models are used as oracles. Most approaches in the table below are centred around a dedicated model for a certain context or scope. Go to the model page for an explanation on typical features of an improvement model.

  • I4Agile contains a model that covers the complet software development process for Agile teams.
  • TI4Agile uses parts of the I4Agile model and focuses on testing in Agile context.
  • TI4Automation contains a model that addresses test automation.
  • TI4Mobile provides a model that focusses in detail on testing mobile apps.
  • TPI®Next is built around a model fit for testing  in organizations that have testing organized as separate discipline.

People to interview
The most important information comes directly from talking with people, often during (group) interviews. More interviews means more information, more details, more chances for double checking, but also the chance to increase improvement buy in.

Documents to study
The importance of documents vary with context. Studying relevant documents (and maybe irrelevant ones too..) provide useful information for an assessment.

Activities to observe
The process in operations tells an important story too. Observing the work done gives information that is not get from interviews (because it was not discussed) and not from documents either (because it is not written down, or not according to documentation).

Assessment approach heuristics

All assessment approaches are exploratory. Even when models are used, exploration is the primary way to get information about strengths and weaknesses or to find root causes for problems. Exploration results in better problem detection.

Size the assessment approach to the right scale. A powerful, compact assessment approach is a brainstorm by stakeholders under the leadership of an improvement architect. Compact TI is an example of a assessment approach that produces an improvement work plan in typically one day.

Models tend to focus on checking. Checkpoints in models are like test cases: they tend to focus on checking. When used as ‘investigation ideas’ they can be the starting point for exploration which result into better problem detection.

Use hybrid approaches. Models help covering the area of concern with sufficient detail. Apply oracles that are free format (not model based) to prevent blind spots and to support exploration outside model boundaries.

Scoring. Certain organizations are focussed on ‘scoring’. They want their organization to be assessed on a maturity level, for instance for marketing purposes. Assessment models with a sufficient level of formality and industry recognized authority are needed.

Benchmarks. Organizations that want to have a comparison with similar organizations ask for benchmarks. Apply assessment models with sufficient level of formality and find representative assessment results of organizations to compare with.

Standards. Organizations that need to comply to certain standards require these standards to be used in combination with assessment oracles that are useful in the context.

Industry standard. It can be important for stakeholders that the improvement project makes use of methods that are broadly used in the industry.

Comparison. Organizations that need a consistent improvement approach in different parts of the organziation (e.g. different agile teams) need oracles with a high level of objectivity, like models.

Repetition. Organizations that want to visualise progress require an approach that is repeatable. A context specific model is an obvious choice.

Understandability. People involved in the improvement project need to understand what is happening; certain models are easier to comprehend than others.

Representativeness – coverage. The sample size (the coverage) applied in the assessment must be big enough to build up a representative basis for improvement suggestions.

Change readiness. Organizations that are eager to change can be approached with opportunism, that is: during assessment dialogues potential changes are explored. This results into a more vigourous improvement approach.
Organizations that are reluctant to change have more benefit from an assessment approach that focusses on building buy in for future changes. This results into a more careful improvement approach.

Partner with a representative of the organisation. Partnership between the improvement architect and a representative of the organisation during the assessment can have advantages:

  • knowledge gained by the representative can be easily transferred to the improvement implementation phase
  • the partner knows the organisation, which makes logistics more easy (making appointments, finding convenient locations)
  • the partner could assess whether answers and behaviour of people are representative enough

DO NOT partner with a representative of the organisation. Partnership has disadvantages too:

  • people may be less open to tell the complete truth when a colleague is part of the assessment team
  • an appointed partner may have interest in the outcome of the assessment, which put independence and integrity at risk