Wednesday, September 18, 2024
Prioritization
Product management often moves at a very fast pace, leaving us with ruthless prioritization decisions to be made. One commonly used framework to achieve this is the impact vs effort matrix—a tool that helps teams decide which tasks to tackle based on their potential impact and the effort required to complete them. Popular product management tools like Productboard and Aha! have integrated this matrix to assist teams in making these decisions. Whilst this framework can have it’s merits, it lacks in several key areas for prioritization.
The basic premise is simple: plot tasks on a two-dimensional graph where one axis represents impact (the value or benefit the task brings) and the other represents effort (the resources or time required). Tasks that promise high impact with low effort are prioritized, while those with low impact and high effort are deprioritized.
While this matrix seems straightforward, it has significant limitations. In many ways it’s straightforwardness IS its limitation. It doesn't allow product teams to factor in all the different types of risks and data (both qualitative and quantitative) associated with a task. This oversimplification can lead to a suboptimal understanding of risks and how to mitigate them. In the following sections, we'll delve deeper into these specific limitations.
'Impact' is bigger than just one number
Impact isn't a one-dimensional metric that can be neatly captured by a single number. It encompasses a multitude of factors:
Customer value: How crucial is this feature or improvement to our customers? Does it solve significant pain points or substantially enhance user experience?
Market potential: Will this task differentiate us from competitors? How does it affect our position in the market?
Business alignment: Does it align with our strategic goals? Will it drive revenue growth, increase engagement, or improve retention?
By reducing impact to a solitary score on the matrix, we fail to acknowledge these nuances. This simplification can lead to misguided priorities that don't fully leverage opportunities or address critical needs. Teresa Torres, a renowned product discovery coach, emphasizes the importance of breaking down opportunities into detailed ‘factors’ in her articles on Product Talk, advocating for a more nuanced approach for prioritization. These factors are:
Customer factors: How important is this opportunity to our customers? For example, how many customers want this, and how much of a problem is it.
Market factors; How will this affect our position in the market? For example, will this make us more differentiated.
Company factors; How will this impact our strategic business objectives? For example, does this drive any of our primary product targets.
Effort is hard to gauge early in the process
Estimating effort isn't as straightforward as assigning a T-shirt size after a quick chat with the engineering team. Early in the development process, many core assumptions haven't been tested:
Engineering risk assessment: Establishing the level of engineering risk is essential. Some opportunities can be expedited into development, while others may require more testing on core assumptions.
Unknown complexities: Hidden technical challenges may not become apparent until deeper investigation.
Resource availability: Team bandwidth and skill sets can significantly affect how much effort a task truly requires.
By not thoroughly understanding these factors, the effort estimation on the impact vs. effort matrix becomes a rough guess at best, leading to potential delays and resource misallocation.
Effort also stretches far beyond engineering, there are also efforts involved with marketing, selling and supporting certain features we build. For example a product enhancement targeting an established user group or geographic market, would have less GTM effort than an enhancement targeting a newer market. The marketing effort would be greater.
These nuances are also not picked up by a generic ‘effort’ number, but you can be damn sure they affect the company resources!
Subjectivity and bias in scoring
Another significant limitation of scoring impact vs effort is the inherent subjectivity and bias in scoring. Assigning accurate impact and effort scores isn't just a quick estimation; it requires deep insights into market trends, customer needs, and competitive landscapes. However, obtaining these insights can be time-consuming:
Don't settle for random scores: Relying on gut feelings leads to inconsistent evaluations. Instead, gather sufficient data to inform your scores.
Need for qualitative customer research: Truly understanding customer needs requires qualitative methods like interviews and user testing. These provide valuable insights but are resource-intensive.
Deep understanding of business goals: Ensure you have a clear understanding of the company's strategic objectives to score effectively.
The pressure to move quickly can lead teams to rely on assumptions or incomplete data, introducing bias into the scoring process on the matrix. This subjectivity can result in inconsistent evaluations and misaligned priorities.
Type caption (optional)
Where the impact vs effort matrix has its place
Despite its limitations, this framework isn't without merit. It serves as a great initial filtering tool to weed out less attractive opportunities you uncover from your product discovery:
Filtering out low-value tasks: The matrix can help quickly identify tasks that are low impact and high effort, allowing teams to focus their attention elsewhere.
Starting point for discussion: It provides a visual representation that can facilitate team discussions about priorities.
Resource allocation: Helps in making quick decisions when resources are limited.
However, while it's useful for an initial pass, the matrix may not be enough to justify a detailed roadmap. To create a robust plan, you also need clarity on the desirability and usability of features:
Desirability: Understand whether customers actually want the feature. This requires customer interviews and feedback loops beyond what the matrix captures.
Usability: Assess how easily users can interact with the new feature. Usability testing and user experience research are necessary to ensure the feature meets customer expectations.
In essence, the impact vs. effort matrix can be a helpful tool in your toolkit but should not be the sole method for prioritization. It's the starting point, not the finish line.
Solution
To overcome these limitations, adopting a more nuanced approach than the traditional impact vs. effort matrix is essential.
Break down impact into detailed components
Instead of assigning a single score to impact, divide it into specific categories:
Customer value: Assess how the opportunity addresses customer needs and pain points by collecting direct feedback and understanding user behavior.
Market potential: Evaluate how the opportunity positions your product in the market by analyzing trends and competitor offerings.
Business alignment: Determine how the opportunity aligns with your strategic objectives and business goals.
By separately evaluating these factors, you can create a more comprehensive impact assessment. This approach ensures that all facets of impact are considered, leading to better-informed prioritization decisions than a simple matrix can provide.
Refine effort estimation by assessing engineering risk
For effort, it's crucial to establish the level of engineering risk associated with each opportunity:
Identify core assumptions: Determine what needs to be true for the solution to succeed. If core assumptions haven't been validated, the opportunity may require further testing.
Expedite low-risk opportunities: Fast-track opportunities with lower engineering risk and validated assumptions into development.
Delay high-risk opportunities: If an opportunity involves untested assumptions or significant uncertainties, delay assigning an effort score to save engineering resources.
This approach ensures that engineering resources are allocated efficiently, focusing on tasks with a clearer path to success and minimizing wasted effort on high-risk, unvalidated projects.
Mitigate subjectivity and bias with data-driven scoring
To reduce subjectivity and bias in the impact vs. effort matrix:
Gather sufficient data: Don't settle for random scores. Collect quantitative and qualitative data, especially direct feedback from customers.
Deeply understand business goals: Ensure that everyone involved in the scoring process has a clear understanding of the company's strategic objectives.
Standardize scoring criteria: Establish clear, consistent criteria for evaluating impact and effort, helping to align the team's perspectives.
By grounding scores in solid data and well-understood goals, you enhance the accuracy and reliability of your prioritization decisions, making the matrix more effective.
Conclusion
The traditional impact vs. effort matrix falls short by oversimplifying complex factors that are critical for effective prioritization. While it has its place as a filtering tool to eliminate less attractive opportunities, relying solely on it isn't enough to justify a roadmap. You also need clarity on the desirability and usability of features.
By acknowledging the multifaceted nature of impact and the challenges in estimating effort, product managers can adopt a more nuanced approach. Dividing impact into customer value, market potential, and business alignment provides a clearer understanding of an opportunity's true value. Refining effort estimation by assessing engineering risk ensures efficient use of resources. Mitigating subjectivity and bias through data-driven scoring leads to more accurate and consistent prioritization.
We're working on a solution to address these challenges, providing a framework that factors in the detailed nuances of impact and effort, along with risk assessments. If you're interested in improving how you prioritize work as Product Manager, sign up for our waitlist to stay updated on our progress.