What is Evidence-Based Management?

Evidence-Based Management in a nutshell

Evidence-Based Management (EBM) is an approach to organization-wide continuous improvement that was created by Scrum.org with the intent to improve the value generation in an agile organization based on defined measurements.¹

In the context of Scrum.org as one of the two main certifying bodies of Scrum (next to Scrum Alliance), EBM appears slightly alien.

It is not really about Scrum to the degree that the EBM guide does not even mention the term “Scrum” outside the footnotes.² The main inspiration from Scrum that goes into EBM is the concept of empiricism and the trinity of inspection, adaptation and transparency.

Apart from that, EBM flies on quite a different flight level than Scrum or other team-based or even “team of teams” based agile approaches, looking at the organization as a whole and in a longer term, strategic perspective.

This makes it even more interesting and worth taking a closer look at why you should consider using it, how it works, what it needs to be implemented and in which contexts it may be useful.

Why Evidence-Based Management?

Benefits of EBM according to Scrum.org are that EBM

• Improving customer outcomes
• Develop organizational capabilities
• Increase business results

EBM starts on the organizational (strategic) level and is designed specifically for situations of uncertainty where there is a need for incremental approaches and experimentation.

As the official EBM guide puts it:

Evidence-Based Management (EBM) is an empirical approach that helps organizations to continuously improve customer outcomes, organizational capabilities, and business results under conditions of uncertainty. It provides a framework for organizations to improve their ability to deliver value in an uncertain world, seeking a path toward strategic goals.

EBM Guide (2020)

EBM in action

Looking at how Evidence-Based Management works in practice, the two key concepts are states and goals.

• The relevant states to record and measure are the starting state and the current state.
• Goals can be strategic, intermediate or immediate (tactical)

EBM measures the progression from the starting state via the current state towards the intermediate goal (i.e. the envisioned next concrete future state) which is a step towards the large and long term strategic goal.

Hypotheses and experiments

The progress towards the intermediate goal happens not in a plan-driven fashion but in an incremental and iterative way.

Every step along the way is an experiment to test a hypothesis which is done by way of adaptation and inspection (true again to the Scrum concept of empiricism (a.k.a. empirical process control).

Examples:

• A strategic goal may be something like: “Build a fully autonomous, self-learning humanoid robot”.
• An intermediate goal could be: “Successfully simulate a machine that independently learns and performs the task of manual dish washing in a virtual kitchen environment using simulated sensor input”
• One tactical goal could be: “Prove that the dishwashing skill can be learned using a learning algorithm”

The higher level goal structure provides direction for tactical experimentation and building of increments. After each tactical step the intermediate goals may be adjusted

It’s all about value

The central concept of Evidence Based Management is value. Value is broken down into four so-called Key Value Areas (KVA).

These Key Value Areas are:

• Unrealized Value (UV)
• Current Value (CV)
• Time-to-Market (T2M)
• Ability to Innovate (A2I)

The four value areas are roughly split into two outcome-related spheres: Market Value and Organizational Capability. And it all revolves around the central factor: Business Value which somehow goes hand in hand with agility (the concrete explanation how this works is left open by the EBM guide).

As shown in this diagram:

Measuring what matters

Evidence-Based Management breaks down the measurements and metrics of value into three useful categories. These are:

• Activities
• Outputs
• Outcomes

In the end, value is about outcomes. At the same time, outcomes are typically the most difficult thing to measure. This is why EBM – without going into greater depth – suggests to use the other types of measures in context of and as being related to the generation of outcomes.

Examples to make this more clear:

• Activity metrics can be something like the number of tasks completed, lines of code written, reports produced per time period.
• Output metrics relate to delivery and intermediate results – such as features delivered, product increments launched, websites deployed.
• Outcome metrics measure the impact of activities and outputs, namely things such as customer experience, impact for the end user (how does the product improve a user’s life), overall business impact.

EBM thereby more or less implicitly assumes a hierarchy of metrics going from meassuring activities and outputs up to outcomes as the main measure of value and business success. This provides a conceptual framework for an agile KPI framework.

Key Value Areas and Measures

Where things get interesting: Each Key Value Area (KVA) is accompanied by a corresponding Key Value Measures (KVM).

EBM as a management framework is all about managing the Key Value Areas by using Key Value Measures.

Taking a closer look at the Key Value Areas and their measurements:

Current Value (CV)

Current Value refers to “the value that an organization delivers to customers and stakeholders at the present time”. Its scope becomes more clear when looking at exemplary Key Value Measures proposed by the EBM guide. These are e.g.:

• Customer satisfaction – gauging the sentiment of customers in terms of paramaters such as happiness (with a product or service), engagement, loyalty.
• Employee satisfaction – how happy, engaged, motivated is your workforce?
• Product cost ratio – how well do products currently perform in terms of cost vs revenue
• Revenue per employee – how does the size of the current workforce perform in relation to revenue?

Measuring current value helps understand the current situation in relation to how it has developed in the past and how it can be developed in the future.

Unrealized Value (UV)

Unrealized value is potential value that could be realized in the future. It is captured in the future goals, from strategic down to tactical.

Examples how Unrealized Value may be measured:

• Market share – indicating the potential for growth within the total market
• Customer / user satisfaction gap – the delta between desired and actual customer or user satisfaction
• Desired customer experience or satisfaction – how can customer experience and satisfaction be improved.

Unrealized value is the younger (or future) sibling of Current Value. It represents the potential outcome improvements that the organization may realize through new experiments.

Time to Market (T2M)

Time to market covers the whole duration or lead time from start to end – i.e. from beginning to create a new capability, service or product to its delivery.

As a core measurement for all lean and agile methodologies, time to market is also a critical factor in EBM.

The way to measure time to market may vary extensively across different domains and contexts. Some more general examples provided in the EBM guide:

• Build and integration frequency – measuring the number of fully integrated builds the organization can create per time interval (reaching up to continuous integration)
• Release frequency – the number of releases that can be delivered by time unit (up to continuous delivery)
• Customer cycle time – the time it takes for a release from the time work on it starts to production
• Lead time for changes – the time it takes for a technical change to go to production
• Time to learn – how long does it take to generate learnings about a new idea or feature all the way from ideation to gathering live feedback from real users
• Time to pivot – how fast can an organization shift their feature delivery priorities based on new market insights or opportunities

Managing time to market in relevant areas helps speed up the rate of improvement of Current Value (CV) and is thereby a major competitive factor.

Ability to Innovate (A2I)

Ability to Innovate represents how effectively an organization can deliver new capabilities that meet customer needs.

It is measured in terms of e.g.:

• Innovation rate – how much does the organization invest in innovation as a fraction of overall cost or efforts invested?
• Change failure rate – what percentage of changes lead to defects and service degradation? (i.e. how costly is it for you to roll out new / innovative functionality)?
• Technical debt – how big is the part of product designs / code that contain problematic quality and may cause trouble when changing it?
• On-product index – what percentage of time is spent by teams working on product and value (rather than non value-adding activities)?
• Context-switching time – how much time is wasted on switching between tasks as well as on “overhead” activities such as meetings, calls, unplanned support activities etc.

From the examples it can be seen that the ability to effectively innovate is measured to a large degree by metrics that measure the ability to focus on activities that add value and lead to rolling out new features and functionality.

How effective is Evidence-Based Management?

EBM is all about measuring and optimizing value in an iterative-incremental way by continuously inspecting and adapting the progress towards goals. Measurements / metrics play a crucial role in controlling this process and measuring its performance.

But how do we know how EBM itself performs as a model or framework in contrast to other approaches?

Frankly, this is not easy to say since EBM does not provide any measurements for how EBM performs. There is also not a lot of data available from implementations of EBM out in the field.

Being a highly abstract model and more a template for concrete systems or frameworks than a concrete guideline (such as the Scrum Guide), it is – quite ironically – hard to measure.

This may be one reason why so far it has not become more popular despite its potential.

Pros and Cons of Evidence-Based Management

Conclusion: Who should use EBM?

Evidence-Based Management is a highly generalistic model and should be treated as such. It takes a holistic view of an organization that wants to combine agility with defined measurements of value generation.

This lends a wide range of applications while on the other hand makes it not easily applicable to a specific context without extensive customization.

EBM may be used as a universal model providing a conceptual framework to implement more concrete models for a specific domains and scenarios.

If one is looking for a similarly scoped model or framework combining the core intuitions of empiricism and adaptive working up to the (or starting from) a level of organizational strategy, that is more defined in terms of implementation strategies, templates, practices and tools, alternatives to consider would be e.g.:

• Kanban Enterprise Services Planning
• The Scaled Agile Framework (SAFe)
• The Flight Levels model

Still, EBM has its specific virtues in providing a clear structure for how an organization can benefit from being at the same time agile and adaptive and base decisions on objective, empirical measurement and data.

If you are willing to invest the resources to set it up, it may be well worth an organizational experiment. Just don’t expect it to be “plug and play”.

Get the EBM Guide

PS: The good news: EBM is free to use under a Creative Commons license.

You can download it either from Scrum.org – or directly here:

---

Learn more

Contact us if you need support in implementing Evidence-Based Management to improve business agility in your organization!

1. For disambiguation, the same term (evidence-based management) has been used also in contexts outside of Scrum / Agile, but this is not the scope of this article. See e.g.: https://hbr.org/2006/01/evidence-based-management, for the Toyota Production System (TPS) as a form of EBM – see https://www.intrafocus.com/2023/04/evidence-based-management-why-toyota-succeeded/ ↩︎
2. The only traceable connection to Scrum is the reference to Scrum’s “empiricism” including the trinity of “transparency”, “inspection” and “adaptation” – see https://scrumguides.org/scrum-guide.html#scrum-theory ↩︎