Sorry, couldn't make my comment earlier since I had very similar feelings about the definition of data products as you discussed and started to look into the relationships between data products and other datasets in data space. I'm recently working on a study on complexity analysis and design considerations for data products and Data-as-a-Service (DaaS). The initial study show we may need to do more work before we can effectively handle the complexity of the current and fast changing data operation and management environments in data space, which cannot be easily reduced can only be controlled and managed if we can find right enablers and methods.
We know not every dataset or data entity in data space needs to or should be a data product. But, each data product must have its associated dataset(s) after certain efforts in design and processes to move, transform, package and present it (or them) through using various tools, platforms and activities. This new and different form of data which has a special value or use cases such that we need to not only name or call it differently but also treat it differently in catalogs, metadata management, applications and management. And, we also need to make a distinction between the internal data products and the external data products.
Looking forwards to reading your following-up articles on data products taxonomy and approaches if they have not been shared here yet.
I have in recent years been looking for a clear definition of a data product, to date I have found a number definitions and while they made sense, they always seemed to be somewhat open ended. As data professionals we seem to like order and governance but openness at the same time. It feels a little like a get out of jail free card. I look forward to seeing where you take this.
Thanks Darren - and a heads up it is going to take me at least a week more to get to the conclusion that I planned. Trying to do lessons from the history of products in one article was too much - so it's being split in two.
Thanks for sharing, I found this really interesting, and always good to hear counterarguments.
I can see the point that a definition so broad is possibly useless. At the same time, I wonder if worrying too much about the definition risks missing the point, and that by saying 'only x is a data product' could make those not fitting to that definition reject some good practices.
To take the Apple example further, to the team working on Apple Watch straps, or the Apple Calculator app, or the weather APIs, that is the product they are building. Sure, it's not _the_ product Apple is going to put front and centre on the website, or talk about at their launch events, but applying product management as a discipline to the building of those projects is likely to lead to a better outcome.
To take the example even further, the teams building the platforms that iCloud is built on, or the SDKs developers use to build apps for the app stores, those are also products, and would benefit from the application of product management discipline.
SDKs might be closer to data than Apple Watch straps, and as a product they still have users, who have requirements, and part of product management is about knowing what those requirements are, how well you meet them, what you need to build next (prioritisation), what's stopping the adoption of that product (lack of docs, discovery, etc), knowing when that product needs to be deprecated, and so on.
Is it a problem that the definition is broad, if the ideas and discipline of product management are also broadly applicable?
Andrew, first thanks for the really thoughtful reply.
To me it is a problem and I could probably article why it is better.
For want of a better term I'm going to call my strict definition of products "true products" for the rest of this comment.
True product strategy is, rightly, a major thing. Launching a new product requires a lot of strategic thinking, board discussion and investment. I'm sure huge amount of work went into Apple deciding to open a new product line.
None of this is true for parts and accessories. The entire business case for parts and accessories comes from (and is often included in) the business case for true products.
It isn't a dis-service to say an accessory isn't a product. In fact in my last company it was widely known that accessories were one of the highest margin sales. Services were also lucrative sales. But those things are not true products.
So what?
I think one of the major challenges of the data world is the complexity we create and the lack of line of sight to value. True product simplify decision making and create a clear line of sight to value.
For a designer of a part (or engineer of a processed data source) you should know that the value of your work comes from the product. You should speak to the product manager to understand the requirements they have. If your part goes into multiple products, then you manage your part as a product and speak to multiple product owners to understand their requirements. You should even (when relevant) ask product owners to introduce you to real customers if the requirements are not clear.
We don't do this today. By calling a processed data source a product we end up in crazy conversations where we say the processed data source has value in and of itself.
Everyone in every company needs to understand that value comes from meeting the needs of the companies customers. We lack this in data.
We also lack the ability at times to distinguish when a product has potential to become a true product and to ask for appropriate levels of board engagement and investment. Better terminology wouldn't solve this - but it would help.
There are also complications. Data has to serve infrastructure needs and internal needs. This is the nuance I'm wrestling with. But I do think we need a clearer taxonomy to make it easier to identify where true (customer facing) requirements are, to prioritise those over internal needs and to get more investment when data becomes a true product.
Thanks for the detailed response! This makes sense to me, and I understand where you're coming from.
We certainly should be thinking about the true products more, and be focussed on how what we are doing ultimately delivers value by meeting the needs of our companies customers.
I also agree many data teams do not do this, and see the data transformations, dashboards, data catalogs, etc as the value they provide.
I'm thinking a lot about data products at the moment as I'm starting up a data engineering team in my company, and I'm considering introducing the term as part of that. If/when I do, I'd want to do it as you described in this comment as 'designers of a part'.
A lot of that is what others have been describing as a 'product mindset', and because I have more of a platform background I've been influenced by things like the Team Topologies book and talks like this: https://platformengineering.org/talks-library/platform-as-a-product
And I do think that taking that approach led to a better outcome. However, looking back, I also think you may have a point that by calling that the product, at times we may have lost sight of the companies products and how we were meeting those requirements.
It would be great to catch up and talk some more about this - thanks for the offer! I'll message you to follow up.
I totally agree on the point that almost all components are referred as a data product including the data lake, dashboard etc.
This definition:
"Actual products deliver value by fulfilling a need or solving a problem for customers, and crucially, they can be marketed, sold, and used independently"
can have bias as well though, what is customers are business analyst, they donuse datawarehouse tonquery, and yes data can be sold (might not be something company wants to do, but data can be sold as data as a service)
May you please make the definition more robust. (I don't have an answer)
Muaaz, in this definition of an actual (not necessarily data) product - the customer is external i.e. an actual customer of the organisation. So business analysts wouldn't count as a customer.
I think this works as a definition for an actual product but not always for a data product. I'm working towards that definition over the next few posts.
The way I will define "data product" based on my industry ecomm industry will be a data enabled product that moves the needle on the company's OKRs. Still vague I know, the problem with the definition you proposed is kinda customer facing, but in ecom, lets say order growth is a huge thing, and any data product that moves the needle there is actually a data product.
Muaaz, you're missing the point - I haven't shared a definition for data product yet. This was article one of four. The definition that you are seeing problems with is the definition that I gave for a product and, like you, I said this doesn't work for data products. In this article I was explaining the need for a better definition.
Sorry, couldn't make my comment earlier since I had very similar feelings about the definition of data products as you discussed and started to look into the relationships between data products and other datasets in data space. I'm recently working on a study on complexity analysis and design considerations for data products and Data-as-a-Service (DaaS). The initial study show we may need to do more work before we can effectively handle the complexity of the current and fast changing data operation and management environments in data space, which cannot be easily reduced can only be controlled and managed if we can find right enablers and methods.
We know not every dataset or data entity in data space needs to or should be a data product. But, each data product must have its associated dataset(s) after certain efforts in design and processes to move, transform, package and present it (or them) through using various tools, platforms and activities. This new and different form of data which has a special value or use cases such that we need to not only name or call it differently but also treat it differently in catalogs, metadata management, applications and management. And, we also need to make a distinction between the internal data products and the external data products.
Looking forwards to reading your following-up articles on data products taxonomy and approaches if they have not been shared here yet.
I have in recent years been looking for a clear definition of a data product, to date I have found a number definitions and while they made sense, they always seemed to be somewhat open ended. As data professionals we seem to like order and governance but openness at the same time. It feels a little like a get out of jail free card. I look forward to seeing where you take this.
Thanks Darren - and a heads up it is going to take me at least a week more to get to the conclusion that I planned. Trying to do lessons from the history of products in one article was too much - so it's being split in two.
Thanks for sharing, I found this really interesting, and always good to hear counterarguments.
I can see the point that a definition so broad is possibly useless. At the same time, I wonder if worrying too much about the definition risks missing the point, and that by saying 'only x is a data product' could make those not fitting to that definition reject some good practices.
To take the Apple example further, to the team working on Apple Watch straps, or the Apple Calculator app, or the weather APIs, that is the product they are building. Sure, it's not _the_ product Apple is going to put front and centre on the website, or talk about at their launch events, but applying product management as a discipline to the building of those projects is likely to lead to a better outcome.
To take the example even further, the teams building the platforms that iCloud is built on, or the SDKs developers use to build apps for the app stores, those are also products, and would benefit from the application of product management discipline.
SDKs might be closer to data than Apple Watch straps, and as a product they still have users, who have requirements, and part of product management is about knowing what those requirements are, how well you meet them, what you need to build next (prioritisation), what's stopping the adoption of that product (lack of docs, discovery, etc), knowing when that product needs to be deprecated, and so on.
Is it a problem that the definition is broad, if the ideas and discipline of product management are also broadly applicable?
Andrew, first thanks for the really thoughtful reply.
To me it is a problem and I could probably article why it is better.
For want of a better term I'm going to call my strict definition of products "true products" for the rest of this comment.
True product strategy is, rightly, a major thing. Launching a new product requires a lot of strategic thinking, board discussion and investment. I'm sure huge amount of work went into Apple deciding to open a new product line.
None of this is true for parts and accessories. The entire business case for parts and accessories comes from (and is often included in) the business case for true products.
It isn't a dis-service to say an accessory isn't a product. In fact in my last company it was widely known that accessories were one of the highest margin sales. Services were also lucrative sales. But those things are not true products.
So what?
I think one of the major challenges of the data world is the complexity we create and the lack of line of sight to value. True product simplify decision making and create a clear line of sight to value.
For a designer of a part (or engineer of a processed data source) you should know that the value of your work comes from the product. You should speak to the product manager to understand the requirements they have. If your part goes into multiple products, then you manage your part as a product and speak to multiple product owners to understand their requirements. You should even (when relevant) ask product owners to introduce you to real customers if the requirements are not clear.
We don't do this today. By calling a processed data source a product we end up in crazy conversations where we say the processed data source has value in and of itself.
Everyone in every company needs to understand that value comes from meeting the needs of the companies customers. We lack this in data.
We also lack the ability at times to distinguish when a product has potential to become a true product and to ask for appropriate levels of board engagement and investment. Better terminology wouldn't solve this - but it would help.
There are also complications. Data has to serve infrastructure needs and internal needs. This is the nuance I'm wrestling with. But I do think we need a clearer taxonomy to make it easier to identify where true (customer facing) requirements are, to prioritise those over internal needs and to get more investment when data becomes a true product.
Thoughts?
Also I'd love to catch up on and discuss.
Thanks for the detailed response! This makes sense to me, and I understand where you're coming from.
We certainly should be thinking about the true products more, and be focussed on how what we are doing ultimately delivers value by meeting the needs of our companies customers.
I also agree many data teams do not do this, and see the data transformations, dashboards, data catalogs, etc as the value they provide.
I'm thinking a lot about data products at the moment as I'm starting up a data engineering team in my company, and I'm considering introducing the term as part of that. If/when I do, I'd want to do it as you described in this comment as 'designers of a part'.
A lot of that is what others have been describing as a 'product mindset', and because I have more of a platform background I've been influenced by things like the Team Topologies book and talks like this: https://platformengineering.org/talks-library/platform-as-a-product
And I do think that taking that approach led to a better outcome. However, looking back, I also think you may have a point that by calling that the product, at times we may have lost sight of the companies products and how we were meeting those requirements.
It would be great to catch up and talk some more about this - thanks for the offer! I'll message you to follow up.
I totally agree on the point that almost all components are referred as a data product including the data lake, dashboard etc.
This definition:
"Actual products deliver value by fulfilling a need or solving a problem for customers, and crucially, they can be marketed, sold, and used independently"
can have bias as well though, what is customers are business analyst, they donuse datawarehouse tonquery, and yes data can be sold (might not be something company wants to do, but data can be sold as data as a service)
May you please make the definition more robust. (I don't have an answer)
Muaaz, in this definition of an actual (not necessarily data) product - the customer is external i.e. an actual customer of the organisation. So business analysts wouldn't count as a customer.
I think this works as a definition for an actual product but not always for a data product. I'm working towards that definition over the next few posts.
Thoughts?
The way I will define "data product" based on my industry ecomm industry will be a data enabled product that moves the needle on the company's OKRs. Still vague I know, the problem with the definition you proposed is kinda customer facing, but in ecom, lets say order growth is a huge thing, and any data product that moves the needle there is actually a data product.
Muaaz, you're missing the point - I haven't shared a definition for data product yet. This was article one of four. The definition that you are seeing problems with is the definition that I gave for a product and, like you, I said this doesn't work for data products. In this article I was explaining the need for a better definition.
Thanks for clarifying. Will be great once we have a gold standard definition for sure, looking forward to it.