If you’ve taken an Agile training course, you’ve probably heard the term “fail fast.” The goal is to be intentional and efficient about your decisions, some of which may lead to failure. When do you learn? How do you learn? How are learning and failure connected? Can you shift learning and failing earlier in the product lifecycle, decreasing the cost and impact of learning? This article explores the process of learning and failing – both integral to product creation.
Why do we talk about failing and not just learning? Isn’t failing a bad thing? In product development, you can sometimes simply learn what you need to know. If you want to learn how many Customer Relationship Management systems exist and what they do, that is directly learnable. However, you can’t directly learn if people will embrace your new CRM product without having your potential customers respond to something. It can be a prototype. It can be a minimal product that addresses a portion of a customer challenge. But after the analysis is done, you do something, and the result you get may be different from what you expected.
In science, this happens all the time. There is a hypothesis, it is tested, and the results may be what was expected or not. If you are in the uncertain business of product development, sometimes things will work as expected, and sometimes things will not. We think a better expression is “Fail Fast, Fail Safely, Fail Efficiently.” This phrase communicates that a thoughtful approach to product discovery can save you time, cost, and reputation. We will use “fail fast” as a shorthand in this article.
When creating products for internal or external customers, the goal is to maximize the return on investment and create something people love and use. Every product idea stems from a set of assumptions about where people currently encounter challenges, how your product could offer a solution, why customers might choose it, and how it will generate revenue or optimize costs. These assumptions may be accurate, or they may not. "Failing fast" essentially means testing our most crucial assumptions as early and cost-effectively as possible to determine whether further investment in our idea is warranted. It also suggests that, depending on what we discover, we may need to pivot or even abandon our product idea completely. This article is dedicated to promoting rapid learning, especially in:
Each topic mentioned could be elaborated on extensively, perhaps enough to fill a book. The purpose of this article is to help you begin this journey.
Anchoring better product outcomes in feedback and rapid learning begins with cultivating a product learning culture. Below are fundamental perspectives encompassed in that culture:
Learning in product development is an ongoing, continuous process. It can occur anywhere in the product lifecycle, from the time you have an initial idea to when you’ve completed the project. Engaging with potential or existing customers—whether through interviews, UX testing, or analyzing product usage data—yields fresh insights to guide your next steps.
Product Managers and Product Owners foster a product learning culture by openly discussing key hypotheses, assumptions, and plans for testing. They incorporate these hypotheses into high-level backlog items, making them visible. They engage with stakeholders and the team, discussing hypotheses and using data to substantiate their learnings, making this data accessible for feedback. They understand that learning is a collaborative effort and is most effective when individuals with diverse perspectives interact and learn together.
The culture within an organization is vital for effective product development. When a company is open to constructively questioning its strategy and product direction, many missteps can be prevented, and capital can be directed toward optimizing business outcomes.
Product Discovery is a key technique for spending a little time to learn quickly instead of spending a lot of time on something that doesn’t produce the desired results. Eric Reiss said this in his book The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses.
“I call the riskiest elements of a startup’s plan, the parts on which everything depends, leap-of-faith assumptions. The two most important assumptions are the value hypothesis and the growth hypothesis. These give rise to tuning variables that control a startup’s engine of growth. Each iteration of a startup is an attempt to rev this engine to see if it will turn.”[1]
While the quote emphasizes “startup,” any time you are investing in a significant new product or update to an existing product, there are likely assumptions that should be tested before going all-in on development. And once you ship the first Minimal Viable Product (MVP), you want to continue learning with rapid feedback loops. When we work with our clients, we help them identify their key assumptions and discuss strategies for economically exploring those assumptions.
Let’s consider an example of a product company trying to reduce customer calls, improve customer satisfaction, and reduce product service costs. Today, they have a chatbot that provides customers with simple options to solve their problems without having to interact with a staff member. Only about 7% of the customer issues get solved via automated prompts. In addition, customers report feeling frustrated having to go through multiple generic prompts before they can get to a person. The Product Management team would like to improve how this works by using a Large Language Model AI to interact with the customer instead. What might be some assumptions to test?
Developing the list of assumptions will often be more effective with multiple people working collaboratively. For example, a product manager might not think about the challenge of hiring or training people to do the work, whereas a technologist might.
Assumptions usually fit into a few categories:
If you don’t identify and investigate key product assumptions, you may pursue a nonproductive product path, wasting valuable capital. This capability is integral to product success. Are you good at identifying assumptions? Where could you improve? Where will you improve?
Prioritizing which assumptions to test is important. Testing assumptions means spending a smaller amount of money now to perform an experiment and learn quickly. This saves money in the long run as well as time, capacity, and potential reputation hits by reducing the likelihood of building the wrong thing. The investments you make testing product assumptions should be balanced by the potential risk of not learning. The assumptions you typically focus on are ones where:
The assumptions you are testing go onto your Product Backlog. As you learn, you refine and reprioritize that backlog based on what you now know. Prioritizing assumptions is important because, inevitably, there will be more assumptions than you have time or capital to test. Addressing the most important ones is key to reducing product risk.
Testing assumptions are usually done via an experiment that provides an indication of how things will evolve after a product is released to the market. Testing assumptions is a vast topic. Here are some examples:
There are many books, articles, videos, and courses on this topic, for example, Bland, David J, and Alexander Osterwalder. Testing Business Ideas. Hoboken, New Jersey, John Wiley & Sons, Inc, 2019.
Interviews are a popular tactic, but they are challenging to execute. Asking a prospective customer, “Would you use this feature?” is unlikely to get a useful response. A “yes” might just mean the prospect was trying to be polite. For someone to do something new, they must have significant frustration with what they do today and be willing to overcome the inertia of moving from their current approach to a new approach. There must be a significant struggle. If you don’t dig into the struggle and really understand it at the root, you might not understand what product they need and would adopt. See Moesta, Bob, and Greg Engle. Demand-Side Sales 101. Lioncrest Publishing, 22 Sept. 2020., for several examples of in-depth interviews.
Testing assumptions is critical to reducing product risk. Designing and executing tests for assumptions is a core competency in effective product culture. If this isn’t an area of focus for your organization, start testing your most critical initiatives. Practice is core to improving.
Gathering and evaluating data on your product assumptions is not just critical; it's a skill as crucial as learning from the outcomes. Yet extracting meaningful insights from experiments often proves more complex than anticipated. Human biases about customer preferences can cloud judgment, making it surprisingly simple to overlook evidence that challenges our expectations.
Consider the case of an airline exploring new service enhancements, such as luggage pickup and delivery or food pre-ordering options. The team conducted interviews with passengers in the waiting areas but didn't encounter much enthusiasm for the proposed ideas. Despite this lukewarm reception during their review session, the team remained confident in the value of their original ideas.
Faced with such a dilemma, should the team have set aside their ideas or refined them for further customer evaluation? Eric Ries's concept of “pivot or persevere” applies here: the path forward isn't always obvious post-experiment. It demands that teams conduct a rigorous analysis of the data. At times, another round of experimentation may be the key to unraveling deeper insights and making informed decisions about the product's future.
When you have gathered and reviewed data and reflected on what you have learned, you must be able to decide on the next steps. Should you launch a limited version of the product, discard the product idea altogether, or conduct further testing?
An effective product culture is one that can decisively abandon product ideas that lack the desired value. However, this can be more challenging than expected. Here are a few reasons why:
Do you currently utilize data to inform adjustments to your plans? Where is this data discussed, and who participates in the discussion? Does this lead to actionable changes? Are you willing to forsake product ideas when they no longer align with your objectives? Is the ability to abandon a product concept based on data valued?
“Learning Fast” and “Failing Fast” encapsulate much more than their names suggest. Throughout this article, we've delved into the nuances of these concepts to shed light on their true meaning. At its heart, “learning fast” and “failing fast” is about rigorously identifying and testing crucial assumptions, actively learning from the outcomes, and nimbly adapting based on those insights. This cycle not only boosts the return on investment but also avoids the risk of sinking funds into unproductive product initiatives that don't yield valuable returns.
[1] Ries, Eric (2011-09-13). The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses (p. 76). Random House, Inc.. Kindle Edition.
Written by: Rita Emmons and Bob Fischer