What is Data Observability?

Подписаться 800 тыс.

Просмотров 35 тыс.

50% 1

Learn more about Databand → ibm.biz/IBM-databand
Check out IBM Analytics → ibm.biz/ibm-analytics
Data observability can help data teams and engineers better understand the health of data in their system and automatically identify, troubleshoot and resolve issues in near real-time.
In this video, Ryan Yackel talks about the difficulty in managing the health and quality of your data sets, what data observability is, and how it can help you fix issues with your data before it impacts your bottom-line.
Get started for free on IBM Cloud → ibm.biz/ibm-free-cloud-tier
Subscribe to see more videos like this in the future → ibm.biz/subscribe-now
#databand #observability #DataScience #MachineLearning #Deeplearning #ArtificialIntelligence #BigData

Опубликовано:

25 авг 2022

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 21

@Flankymanga Год назад

Hello guys. Thank you for this blackboard session. I have a suggestion: Theory is nice and it is the first step to get into the specific IT topic. But as they say: seeing is believing... could you please make a video with practical use case, how data observability practices are being utilized to solve data pipeline issues? In other words: How data observability looks and works in real life on real tools. That would be awesome to see. And if you could merge this topic with Data Governance that would be totally awesome! I am loving these blackboard explanations! Thank you very much!

@AISmallBusinessHub Год назад

Thanks for breaking this down clearly. It makes it easy my product designer brain to understand the concepts

@dineshshekhawat2021 Год назад

Very insightful, learned a very useful concept on a high level. Keep bringing such videos!

@billlabranche Год назад

Great illustration to the concept of Data Observability - thanks for making it simple enough to understand. PS... I'm very impressed with your "backwards writing skills"!!!!

@ladonwilliams1333 Год назад

Absolutely great overview. Appreciate the train analogy. I got a good understanding of the premise of Data Observability, plus the bonus of a concise understanding of data engineering and pipelines. Need this gentleman explaining more concepts!

@MsVikas99 4 месяца назад

Really great overview. Thank you

@robfletcher6274 Год назад

Great explanation, nice one

@alexpishvanov736 Год назад

I really like this idea to shift thinking paradigm from been software engineer to data engineer:)

@nikitashrestha684 Год назад

Hello, thanks for this explanation. I would like to see more videos on NLP domain. To be more specific using pre-trained tranformers for text analytics ( Transfer Learning )

@mustufabaig1483 Год назад

What I understand from this video is that data observability is just a grouping of "data quality", "data lineage", "ETL Logging" etc. into one umbrella. These are common concepts within data warehousing/engineering. Some teams solve them with frameworks/vendor tools, some through custom development and some use both approaches. Data observability is just a new name for it. good presentation though.

@paddyarunachalam7057 8 месяцев назад

Thank you. Great session. Could I ask what technology you use to put this together?

@rajarshisadhya8174 Год назад

Thanks much for the really cool explanation. Wondering whether there is any tool available that provides the capability to really ‘Observe’ any changes/incident in the data pipeline?

@thedatawhisperer Год назад

Great job!

@ARATHI2000 11 месяцев назад

Understood in theory..Thanks! Several years ago, one issue we used to run into was some of our Enterprise customers were missing sending data on defined schedule into our SaaS env. Our ETL data ingestion job (sort of data pipeline tool) won't help since it won't be kicked off. So we wrote something outside of the ETL job to ensure we can catch such data misses by the customer...could/should this be part of data observability?

@jorge-hernandez-ramirez Год назад

Good!!

@SoniaRaval 11 месяцев назад

Great overview and love the train analogy

@silicon6086 Год назад

Is Data Observability same as DataOps ?

@ladonwilliams1333 Год назад

I think that Data Observability is more of a technical implementation to identify/remediate issues in the data pipeline that is managed by Data Engineers and others potentially involved in the process due to cross-functional responsibilities. Whereas DataOps is a methodology to foster a culture of effective communication and collaboration between data stewards, data users, and executives/business process owners. So, Data Observability helps to ensures quality data so that DataOps can glean business value from data presented. In essence, Data Observability integrally folds into DataOps.

@vio4jesus Год назад

SoooOoooooo we've been moving data for years with ETL tools. Tools that generally don't require a super coder. Some of these tools have some level of monitoring built in, and some have data quality modules. BUT NOW..... let's go back to hand-writing code to move data. Hhhhhhmmm seems like a bad idea.

@davidmurphy563 Год назад

I think it would have been a better presentation if he had stuck to concrete examples more; like with the betting company. The train was cute, and I did get it, but he explained something abstract by making it even more abstract. Heading in the other direction and injecting some hard reality would have been more instructive. At the end of the day, this is not an academic exercise, it's there to solve real world problems.