The only things that can't be pushed back to the database are the dynamic calculations like ratios that need to be recalculated in real time depending on the user's interaction with the report
Totally agree! At my job we create all the business logic in the Gold Layer, there we can have it commented, keep track of changes, its way easier to debug problems, I've been the whole week checking some columns and updating them with the business team based on the way they want it changed. Data analysts just need to create reports and understand the data to present it to stakeholders/other teams, not that this is an easy task since they are essentially our bridge with the business team
Thanks for this video! Generally agree with all the points you mention, but what are your thoughts on allowing stakeholders to self-serve versus implementing controls such that most, if not all, data reporting should always be through the data team?
Can you do a follow up on how you might put some of these metrics into a database? I was under the impression that a semantic layer doesnt live in the traditional database and id love your guidance on where you might put it?
Great insights & very informative ! I have a question : so in my dbt project, i got the staging folder populated with functional models (transformations and aggregations in fuction of source data in the yml file) and modelized with entity relationship diagram, so what is the utility of marts layer essentially ? Can I just add constraints/relationships… on this folder for specific models ?
Interestingly, user are always excited to do it themselves and later on outsource to the IT/Dev to do the change because of getting loss into the report transformation. 😂 and in database give you track of change with code repository.
