Keynote - Data Mesh by Zhamak Dehghani 

Absolutely true , in lala land this might work but not in real world. when we propose any architecture something we should always think of how seamless can we operationalize it not how can we build it , not just that u have couple it with resourceing challanges .

Absolutely agree!

Me too in the same wavelength - I believe it started with new concept but ended with standard data governance process. The speaker should include examples how data product roles makes agile the modern data architecture governance .

How i can think of is if you have a client_id in one of the Data Product /Pod team revolving around Claims, the same client_id can have a Policy but Policy will be owned by another Data Pod/Product team. Now technically speaking, pretend you have table_policy in one workspace and table_claim owned by another team in another workspace, then you have to join them when you want to consume for analytics. This is one weave of the data between pods. This is how am envisioning it, more thoughts or ideas better than me would help me as well to learn. I may be wrong here but this how they are trying to move the ownership and splitting the laod now across different Data Pod teams. Then you simply join those information for your analytics when needed in reports. The technical concepts remains same but the accountability and responsibility are drafted to teams. This is how Mesh is created. Straigh away i can see this will create more dependency and costly to implment as well. Just like the front-end teams. I have a feel this will be too expensive to implement that is the only negative i see. Pretend you have 25 applications in an organization. How are you doing to split the ownership of those to the Data Pods ? I have those questions in my mind. Very interesting approach and am also curious as you to work in that space now and learn from it. It is a mindset and journey to embark on. A new paradigm hope this doesnt become a recruiter marketing gimmic. Another problem this can create a hugh dependency between networks of Data Pods. See this is not like K8s like you can bring the app in a flash, this is data with volumns which means it is massive in nature, hence if one Pod has issues then there will be ripple effect on other pods how are conusming that. So this is creating new problems in engineering. Trying to solve problem by creating a new one clearly not a good architecture imho. Sometime sticking to basics and planning properly solves the problem in simple.

If you look into data lake architectures, there is no importance given to domain of the input sources or consumers that they're serving. This is why data lakes are basically "single source of truth" for all of the data in the organization irrespective of the domain that they belong to. Now if you logically break down these into data products to their respective domain, then each of these data products are infact "single source of truth" for that particular domain. So instead of having "single source of truth" for all domains, we have "single source of truth" for each of the domains in the form of data products.

@@AnkitAnchan666 Thanks. That makes sense.

Good presentation! How we will manage cross domain data in this data mesh approach?

@@AnkitAnchan666 - this we are already doing it in refined layer or Gold Layer. I see the perspective now. Just organizing team to use the lake and start building information wearing a data product mindset (Ownership and Accountable is solved). Again perspective and from where you are looking at things matters. Situation, scneario and people matters when making this choise. With my Data Enterprise architect hat. @Anuruthan Thayaparan - This is not an architecture change it is just a mindset and management of people carving them to a particular scope. It is like creating TRUSTED DATA PRODUCT TEAMS in GOLD/ REFINED/ TRUSTED LAYER in the data lake world.

Hey I'm watching this now and seeing your name :) Small world, how are you?

@@ozlemakpinarsel2674 hehe small world indeed! All good here, hope you are doing great as well