The global AI market is booming, with businesses investing billions in machine learning models, predictive analytics, and intelligent automation. Data has become the foundation of modern product development, shaping how businesses deliver value and make strategic decisions. Yet, despite its power, data alone does not guarantee success.
Every product that has data as its core, is built on a set of assumptions – about data quality, data availability, user behaviour, and business impact – that, if proven false, can result in failure.
Without a structured approach to testing assumptions early, teams risk building solutions that are useless, biased, or impossible to scale. As product managers, data scientists, and business leaders, we've all experienced that moment when an exciting new idea emerges, but uncertainty lingers. Is this the right path? Will it deliver real value?
Lean experimentation provides a systematic way to validate ideas before making costly commitments. By testing assumptions early, teams can avoid expensive missteps, accelerate learning, and build data and AI products that truly deliver value in the real world. This article will provide an overview of the common assumptions we make and how lean experimentation can help minimise risk and maximise return on investment which means more investment in data and AI products.
When building products that rely on data, especially those that consume and use data produced externally to the product itself (whether AI-powered personal assistants or analytical platforms), we rely on numerous assumptions about the data we need to use. If any of these assumptions turn out to be false, the product is at risk of failure – leading to wasted time, effort, and money. Such failures can erode leadership’s trust, making it harder to secure future investments.
These assumptions often relate to how data is collected, processed, interpreted, and used to drive decisions or generate insights. However, if they prove incorrect, the consequences can be severe, including poor adoption, missed business opportunities, and an overall lack of impact.
These are some of the common high risk assumptions:
Validating assumptions early in a project is crucial for reducing risk, saving time and resources, and increasing the chances of success. Lean experimentation is one of the most effective ways to validate assumptions in these projects, ensuring we are on the right path.
We applied this approach while working with a global asset management firm that had invested heavily in data platforms and solutions that leverage and are built on top of that platform. They initially assumed their data was of a high enough quality to meet their needs and that their technology choices were well-suited to their needs.
However, instead of committing to a full-scale solution upfront, we used lean experimentation to systematically test these assumptions through small, focused experiments. This approach not only prevented months of wasted effort but also ensured that the final product was effective and aligned with their business goals.
Lean experimentation minimises waste and maximises learning by testing assumptions quickly and efficiently. It is an approach to learn with the least amount of resources available whether an idea/initiative will deliver its anticipated value.
Assumptions are the beliefs or hypotheses that underpin product decisions, such as, “Users will find this feature valuable” or “ Our data's quality attributes are suitable for what we are trying to achieve” or “Our AI model will accurately predict outcomes.” If these assumptions are incorrect, the product will fail. It’s essential to identify these assumptions early in the process.
Tools for Uncovering Assumptions:
Once assumptions are identified, the next step is to design experiments to validate or invalidate these assumptions. The goal is to test the most critical assumptions that will have the largest impact on the future product.
Experiment Design Principles
This is the critical stage where we analyse the results of our experiment and determine the next steps—whether to move forward, pivot, or stop altogether.
Now is the time to combine data with real user insights. We look at quantitative metrics like user engagement, conversion rates, data quality, or AI model accuracy, alongside qualitative feedback from users, to assess whether our assumption holds true. Based on this analysis, we decide whether to refine our approach, rethink our assumptions, or proceed with confidence.
Working with data creates additional uncertainties to product development, beyond those around user adoption and value. Data quality, data availability and model performance are just as unpredictable. We tend to assume these factors will align perfectly—but in reality, they rarely do. Lean experimentation helps mitigate these risks by enabling teams to test critical assumptions early, ensuring that resources are invested in the right solutions.
By embracing an iterative, hypothesis-driven approach, companies can build resilient, user-centric AI and data products that effectively solve real-world problems.
Building products with data at their core is inherently uncertain. By proactively identifying and testing assumptions, teams can reduce the risk of failure, avoid wasted resources, and create solutions that genuinely meet user needs while delivering business value.
Lean experimentation enhances confidence in decision-making, saves time and money, and ensures solutions are fit for purpose. The three-step process—uncovering assumptions, designing experiments, and analysing results—is essential for success. Additionally, recognising and addressing high-risk assumptions early in the development cycle is crucial to building resilient, effective data-driven products.
.jpg)
