Modern teams balance speed of launching features with the need to validate hypotheses. How should teams invest their own limited time? As well as customers’ time?
As PMs, we continuously weigh a million factors: Should the focus be fixing stuff, usability, or adding new features? Jeanette laughs and says “Nirvana is do all 3 at once.”
“Researchers’ unique strength is increasing rigor"
Researchers take a methodological approach to uncovering user pain. They have time to do the analysis. If clarity is low, and the risk of getting it wrong is high, you need in-depth analysis.
At Pendo, Jeanette and her team use the product itself as a channel to accelerate research. She says “our UX team leverages in-app guides and strategically-defined segmentation to poll customers about product ideas, and recruit them to participate in qualitative research activities.” Researchers study ethnography and make deep observations on how users engage with a product. They understand that “nothing is binary” and know the tools and tricks to limit biases in customer interviews.
Still, it's difficult to know when to do research. That’s why Jeanette came up with a new framework to help prioritize resources in the UXR function.
In this framework, problem clarity and risk are inputs that determine how many resources should be allocated to user research. Qualitative and quantitative data point to crisp and clear customer problems. A common mistake is to focus on business problems instead of customer problems. Jeanette argues “business goals are achieved by ensuring that they solve deep customer pain.”
Risk, on the other hand, refers to the risk of getting it wrong, and is measured by cost or sometimes a loss of users.
“Problem: PMs don’t have time to validate every assumption.”
The framework indicates a scenario in which moving forward on intuition, and keeping assumptions as… assumptions is most efficient. Examples might include competitive strategy with feature parity, design systems, or heuristics as Nielsen Norman Group defines it. It’s important to understand that aesthetics can build trust in a product, and gathering this evidence can distract from solving the right problems, often times leading to analysis paralysis and decline in shipping velocity.
I ask Jeanette how she balances sync & a-sync decisions. She mentions the need to allow people the opportunity to iterate, give space for cross-functional alignment, and give time to observe what’s working well, and what is not. There may be situations where you can test something and have results in hours - for instance, feature flagging or A/B testing language on an email.
Jeanette finds that a hybrid model works best for her team. There needs to be a dedicated space for everyone to align on the problem, have checkpoints for share-outs, and make decisions. “People probably won’t look at your idea unless they are required to do so” she says. Without synchronous, there’s solving the wrong problem can have a cpompounding effect.
Even though it’s hard to imagine if anyone is different than us… we still need to have empathy. Jeanette recommends involving engineers in the empathy journey, and having artifacts to help build the case.
Ask yourself as a PM, can I prove this will move a top-level company metric? Reach out to your biggest customers and ask for feedback. Prototypes are a happy path. By showing your customers the real product as it’s being built, you’ll end up getting better feedback.
“Listen for where you’re wrong, not where you’re right.”
By listening to things you don’t agree with, you form a mental model for testing your assumptions, which allows you to clarify the problem faster.
The more risks you take, the faster you can ship, and faster you can learn. The impact of this could be the difference in achieving product-market fit or not. However, with more clarity of the problem, you potentially waste less resources. The key is to balance these.
