Data as a product is a truly game-changing idea. Adopting a product mindset towards data enables us to identify and share valuable data sets that meet the needs of data consumers across an organisation. This is a contrast to the traditional "data as a technical byproduct" view that has been prevalent for many years, hindering the value derived from data and limiting its reach.
Data as a product is a core concept in data mesh, a strategic approach to data management that focuses on delivering value from data at scale.
The key to creating successful data products is adopting a product-focused mindset, which involves aligning outcomes with user needs through a user-centric, exploratory, and iterative approach and processes, such as user research and product canvas definition. Product thinking has been successfully adopted by many organisations for all sorts of digital products.
However, the simplicity of data products is misleading. Implementing them at scale can be difficult because it challenges conventional data management practices. To increase our chances of success, we need to steer clear of common pitfalls that can increase complexity.
In this blog post, I want to share 5 common pitfalls that I have observed from my experience.
Fundamentally, data products result from applying product thinking to data.
Product thinking prioritises the design and development of products that meet the needs of the end user, which are the data consumers in the case of data products.
However, it's common in organisations adopting the data mesh paradigm for the initiative to start within the part of the business that owns the data, often the technology department.
Despite good intentions, there is a risk that old data ownership dynamics will persist during the adoption journey. This can lead to a top-down, architectural approach to data products, instead of a user-centric approach. This approach attempts to predict user needs and plan for them ahead of time, without iterative validation or incremental value delivery. The result is a large number of technical "data products" that are not aligned with user needs and have limited chances of success or widespread adoption.
To ensure success, adopt a user-centric approach from the start by using product thinking practices. This includes creating a backlog of data product opportunities, developing a clear product plan with a product canvas, reducing uncertainty through targeted research, and defining and building a minimum viable product (MVP) to quickly deliver value and gather feedback.
Maintaining a product-focused mindset is key. Start with a limited number of data products and validate the approach, then gradually scale.
The first question that we should ask ourselves when exploring data product opportunities is what type of outcomes are we trying to enable with this approach?
Many organisations I have worked with primarily use data for operational reporting purposes but face difficulties obtaining reliable data even for that purpose. In other words, they use data to support human-driven business processes with descriptive reporting. These reports are crucial to the functioning of their business and without them, the organisation would not be able to operate. It's understandable in this context to see data products as a comprehensive and immediate solution to all data issues.
However, a rapid and widespread adoption of data products can be a risky move. If the focus is solely on making small improvements to the existing data capabilities through an approach that hasn’t been proven, it is unlikely to gain enough traction or excitement from the business, which is crucial for successful adoption. Furthermore, it will also disrupt established ways of working, leading to even more resistance if there is no clear business outcome being sought.
As with any new approach we need to find use cases that will demonstrate its value and will provide a solid basis on top of which we can scale. Data products are powerful enablers for more advanced forms of analytics and data use cases.
Identify use cases that will clearly benefit from the approach and extend that to other areas when initial success and buy-in have been secured. These are typically within the space of data science, machine learning and AI. Or complex reporting requiring a high level of accuracy.
What are the strategic priorities of your organisation? And if you know, is this knowledge shared across the team, the department, or the business?
The issue of strategic priorities and value comes into play when planning for a data product, usually starting with a domain mapping exercise.
Domain mapping is a crucial step in designing a data mesh and figuring out how to allocate and prioritise data products. Each domain can clearly define its data requirements and the data products it makes available for the rest of the organisation.
It may be intuitive to organise domains to match the current business structure, but this approach is not necessarily useful in determining how data products can support strategic priorities. The big priorities that many businesses are dealing with today, such as customer innovation, environmental impact and regulatory reporting, tend to require the whole business to collaborate and improve its data sharing capabilities. Existing business and organisational structures typically fall behind the strategic priorities and are difficult to change flexibly. Key business concepts such as “customer” are naturally multi-faceted and cannot be siloed within one business unit.
Domain mapping that focuses mainly on mirroring the organisational structure in place could make strategic priorities more difficult to implement, rather than easier, by reinforcing silos and increasing the need for low-level data exchange across them.
Engage in domain mapping with awareness of the organisation's structure but an open mind regarding the outcome.
Use this activity to elevate strategic priorities to data domains and identify the data products needed to achieve their goals, as well as any new data products that can be shared and that will add value to the organisation.
Building data products is winning half the battle but it does not guarantee that the business will be able to consume them and utilise them effectively.
For many years, organisations have mainly consumed data through reports created and managed by dedicated BI teams. This dependency on these teams for data has resulted in a lack of adaptability and independence for data consumers in tailoring the data to their unique needs.
It may be tempting to continue providing data in a form that is already familiar to consumers, such as reports. But this approach has fundamental flaws, as it limits the ability to reuse and adapt the data for other purposes and perpetuates the dependency on a small group of data specialists. Breaking the tight coupling between data producers and consumers is key to unlocking the full potential of data products.
Consider consumption strategies early. Provide data products that can be fed into the tooling its users need, but that is not tied to a specific consumption method. This will establish a sustainable partnership between data producers and consumers, where each can focus on their own priorities independently.
Empower consumers to use data in their own way through self-service and providing the necessary tools, even if it may seem more effort initially.
The idea of data products is attractive because they can be reused across multiple use cases, maximising the return on investment. This is understandable - reusing data products is one way they deliver value over time.
However, the potential for reuse should not be the sole or the driving criterion for measuring their value and success. It is conceivable that some data products may have a limited user base but still deliver significant business impact, such as enabling a unique ML use case. Conversely, a data product may be highly reusable from the outset but lack specific application, making it less valuable on its own.
The challenge of early data products is not reuse. It is rather validating the idea, creating new business value and identifying the gaps in the current operating model. Prioritising broad reuse may delay value delivery by expanding the scope of the problem space and adding complexities that are not essential to the success of the approach.
Explore the potential value that a data product can offer by engaging by its target users and by using a product canvas to capture that value.
Concentrate on how this value can be verified and measured. Reusability is indeed a desirable outcome, but it stems from the data product being usable in the first place and interoperable with other data products.
At its core, data products are simply a new type of product with a different type of deliverable than other digital products, and a wider audience that encompasses various business stakeholders.
The key to success remains in applying a product-focused mindset methodically and from the outset, and in implementing effective structures and processes around it.
Focus on achieving early success and then scaling up.
Interested in seeing our latest blogs as soon as they get released? Sign up for our newsletter using the form below, and also follow us on LinkedIn.
