In today’s world, the only thing you can count on is that things keep changing. Evolution is on permanent fast-forward as everything from consumer behavior to technology to business models exists in a perpetual state of flux. For product management, this is both a frustrating challenge and an exciting opportunity. Well-researched plans and strategies have a short shelf life, as their underlying assumptions fluctuate and transform.
The only way to have confidence in the validity of a pre-set agenda is by committing to product innovation and continually testing hypotheses and confirming beliefs. It requires a culture of experimentation and investigation, which can run counter to stakeholders’ desire for a reliable path forward.
But how do you create an environment where colleagues embrace an ongoing commitment to assailing the status quo? Can you grow and innovate if you’re constantly checking the temperature before diving in?
Rethink Product Innovation: Making Experimentation and Learning a Fact of Life
Once your organization gets on board with routine experimentation, they’ll soon see the team is making smarter, informed decisions with better outcomes and fewer misfires. Here’s how to make it happen.
Defeat Doubts
Lobbying for experimentation means admitting that you don’t know everything. No one knows it all, so there should be no shame in owning this truth. But for product managers who strive to build credibility with stakeholders and establish themselves as subject matter experts within their company and industry, this can be jarring.
But as a product manager, your credibility isn’t built on your inherent encyclopedic knowledge and talents for prognostication. Rather, its foundation lies in the data you use to build and support your positions. Experimentation is merely another way to gather that data.
During the very early stages of product development, testing and experimentation are embraced as part of the minimum viable product process. The whole point is to see what works and achieve market validation for the principle assumptions of its value proposition.
Building things we think people want isn’t the same as building things people actually want. No matter how slick, sexy, and functional they are, it doesn’t matter unless there’s true demand.
Make Space for Learning
Some experiments don’t require a lot of resources. You can see which verbiage gets a better response, test out some messaging, or survey customers without tapping engineering resources or messing with the codebase.
But to try things out on a large swath of users, you’ll need support from technical and UX teams. Convincing these cohorts that this isn’t “throwaway work” because you’re bad at market research or can’t make up your mind is essential to instituting an experimentation-friendly climate.
No one wants to work on something destined for the rubbish bin. Impart the value of learning from these tests upon these cohorts. Part of this is letting them into the thought process behind these tests.
They are your fellow scientists (or at least lab assistants), so they should be fully aware of why you’re experimenting and what you hope to learn. It gets them invested in the process instead of feeling like they’re always working on lost causes destined to be scrapped more often than not.
It’s how pharmaceutical companies can keep their teams motivated despite a failure rate, often exceeding 90%. Success isn’t usually an easy, foregone conclusion.
While startups usually thrive on such tests, once a company gets some traction, experimentation can fall by the wayside as maintenance and minor, iterative enhancements are driven by customer demand take over. Cordoning off part of the organization to testing new things out is essential to avoiding stagnation.
Encourage Failure (you don’t learn from succeeding)
“Fail fast” has been a favorite axiom of Silicon Valley startups that realized the only way to know what works is to find out what doesn’t. This mindset has spread across industries and continents as a love of learning replaced fears of failure.
But learning from our mistakes doesn’t always come naturally. Mistakes are frustrating, ego-bruising experiences. Your company is probably full of Type A, overachieving folks priding themselves on their accomplishments and successes.
To overcome this stigma, see failure as a goal. Every dead end identified is a path you won’t pursue further, ultimately leading to the more efficient and productive deployment of resources. Crossing a shiny object off the list should be lauded just as much as proving the efficacy of something else.
Once you have a few experimentation wins under your belt, it’s important to socialize this information throughout the organization. It might seem odd to widely advertise something’s failure, but it’s all about context.
Here’s an experiment we spent a few days of development on, and the results showed it was not going to deliver the expected results. We did this instead of spending a few weeks or months building out the full feature, so those resources were used on other things with better returns.
Operationalize Experimentation
Scientists don’t “wing it” every time they want to see what happens. There’s a long-established culture of what’s required for results to be considered statistically accurate and found worthy by their peers.
While you won’t be sending off your A/B landing page test results to the New England Journal of Medicine, you can borrow a few tips from industries that espouse the benefits of trial and error. To turn your company into an experimentation factory, you must first establish the protocols of how to conduct these tests. Then, you measure and apply the results.
One way to make experimentation and testing more appealing to the organization is including a heavy dose of quantitative data gathering and analysis to power the decision-making process. While surveys, interviews, and anecdotes are the qualitative color that humanizes the endeavor, there’s nothing like reams of hard data to set stakeholders at ease.
Creating a playbook for experimentation keeps everyone from trying to reinvent the wheel. Identify what qualifies as test worthy, adequate sample sizes, success measures and methodologies, and feedback mechanisms.
The more routine it becomes, the less strenuous the task. It reduces internal friction and makes it standard operating procedure.
Make Space on the Roadmap
Product roadmaps and an uncertain future seem incompatible. The whole point of the product roadmap is to tell the audience what you’re going to do. However, you can’t provide specifics if you’re continually changing course based on the latest findings.
This is where a theme-based product roadmap can be a game-changer. Unlike traditional product roadmaps detailing every specific feature and delivery date, a feature-less product roadmap prioritizes goals and objectives while leaving the implementation unspecified. Its gives you room to test and learn while still communicating what you’re trying to achieve and where the product is heading at a high level.
If roadmapping also includes budgeting and resource allocation, carving out a consistent chunk for these activities is a must. Once those get locked in, it’s nearly impossible to scrounge up the dollars and cycles needed to conduct experiments. You’re essentially pulling those away from revenue-generating activities.
Product Innovation: Invisible Rewards
The Road Not Taken is more than just a poem we had to memorize in middle school. It represents the opportunity costs of the choices product teams make. We can only choose one route, and each turn we take costs both dollars and sense and reputational capital.
The faster you realize you’ve made a wrong turn, the quicker you can turn around or course correct. Better yet, via experimentation, you can deploy an advance scout before your whole caravan suffers the same fate.
So celebrate failure! Cheer the discovery of dead ends and duds! Rejoice in the knowledge you’ve avoided a big mistake by making a small one. Rethink product innovation and make experimentation and learning a fact of life.
For a deeper dive into building a culture of experimentation, check out the Product Manager to Product Leader ebook for more lessons and tips from industry experts.
