Azure Synapse Analytics - Introduction to Azure Purview

Подписаться 32 тыс.

Просмотров 21 тыс.

50% 1

Yesterday we had the huge announcement that Azure Synapse Analytics is now Generally Available, but we also had a surprise announcement around Azure Purview, the new data governance & cataloguing tool from Microsoft, now in public preview.
In this video, Simon runs through the basics of getting set up with Purview, how to register data sources and start your data governance journey!
For more info on Azure Purview, see their overview page - azure.microsoft.com/en-us/ser...
As always, don't forget to hit like & subscribe if this video was useful, and stop by our website if you need any help on your Azure Analytics journey! - www.advancinganalytics.co.uk/

Опубликовано:

12 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 39

@CameronNeale 2 года назад

Love the videos! The way you present all of the details is very engaging. What a change from the dull videos we are used to watching from other channels!

@AdvancingAnalytics 2 года назад

Glad you enjoy it!

@benjaminbong9262 2 года назад

Great video! Very engaging and informative. Also clear and easy to understand. Thank you!

@VINITSANSARE 3 года назад

Beautiful demonstration of Azure Purview 👍

@karnasaurav 3 года назад

Thanks, man. Great stuff. The explanation is very informative. Appreciate it.

@carltonpatterson5539 3 года назад

I’m usually very lazy at making comments, but I feel I should say a big thanks for this excellent demonstration on Purview. Cheers Simon

@DarkMisley 3 года назад

Cheers Simon, hopefully they'll put more effort into this service than Data Catalog v1 (which never seemed to move beyond an MVP state)

@AdvancingAnalytics 3 года назад

Yeah... The Data Catalog that couldn't understand non-relational sources... Given Purview is Atlas-based that's already a good start. I imagine we'll see quite a few rough edges as it matures!

@28nov82 3 года назад

Very nice overview!

@PicaPauDiablo1 3 года назад

Thank you

@jinlinxu5109 3 года назад

Great video, Simon! Did you ever get the chance to look at the performance impact to the data provider?

@radekou 2 года назад

Where things stand right now - would you use Purview for your company's metadata management and governance or would you rather build something of your own (using API's, custom pipelines, dataverse, Power BI, ...)? I'd ask the same of the Databricks Unity Catalog.

@NeumsFor9 3 года назад

Simon, Thanks for showing this out of the box. It makes me miss consulting. As I search for a new position, these videos do a great job of helping people stay abreast. Does it feel as though this is a good step in the "DataOps" direction as part of the puzzle?

@AdvancingAnalytics 3 года назад

It does... but it's not perfect yet. There's been a big focus on control & security here (the name Purview alone speaks volumes), whereas for me, enabling DataOps is all about ease of data discovery and connectivity. It has features that align to this vision, and I think we'll get there fairly quickly, but the initial push is certainly more on the "govern & control" than "enable & encourage" path of data gov

@NeumsFor9 3 года назад

@@AdvancingAnalytics Maybe they can mix in the IDEAR tool from the team data science process.

@renatofreitas3877 3 года назад

Great introduction Simon, good to see Microsoft investing more in this field. With Atlas API's we might be able to use that as a schema registry for the data platform, to control the data that is ingested, process and the schema evolution. My thinking is correct?

@AdvancingAnalytics 3 года назад

Aha, I have a master plan along those lines, although I believe some of the auto-scanning functionality is disabled if you override entities manually... But there's certainly potential to use it as an engineering metastore!

@renatofreitas3877 3 года назад

Good to hear that. I'II play a litte bit with Purview. Thanks again Simon.

@iphadkegmail 3 года назад

Its a little weird. They just added schema registry feature directly into Event Hubs. Now, if they integrated that into Purview, they will be the absolutely killer use case!

@NeumsFor9 3 года назад

Dataverse, Purview, Synapse.....I am getting marketing rename fatigue, but at least metadata is getting more love. Of course, there is always Kimball's "Meta Meta Data Data" article to fall back on. I swear, sometimes I feel as though companies read his older articles...apply 55% of stated functionality.....wait for MVPs to fill in 20% more.....then, the final 25% comes from community pain and feedback. Ha. Rinse. Repeat.

@pini22ki 3 года назад

Thanks for the review. Is it all scans or workspace per subscription? Do we have an option to do it all over my account?

@AdvancingAnalytics 3 года назад

Hi! You currently set up scans on specific objects that you have connections to - this could be within the same subscription, other accounts you have access to, or even via service principals connecting to external sources. I don't believe there's a "scan my whole subscription" function built it, and I honestly don't know if they're going that direction, or if it's always going to be a manual inclusion process. Simon

@MarkVersteegh 3 года назад

@@AdvancingAnalytics Microsoft shows the option to register multiple sources at the same time, see ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-27bA4KFiEKk.html However, I have not been able to find the option myself (and I tried all the regions where purview is available), so I guess it's still private preview.

@rajeevsharma2664 3 года назад

Nice video. Does it mean Azure Data Catalogue is out? Second, in Azure SQL DB we've seen data classification/confidentiality setting? Will that be deprecated?

@AdvancingAnalytics 3 года назад

Yep, what was Data Catalog V2 is all wrapped into Purview. It's "out" as public preview, isn't fully out yet. It uses a lot of the same "scanning" technologies that you see with the auto-classification features in SQL, Cosmos etc, no idea about depreciation & merging of features in the future though!

@firstch7801 2 года назад

Could you please make content for purview about custom classification , I really need it :)

@jdr9861 2 года назад

Useful content -- thank you. Moving background is distracting and drives me crazy though.

@iphadkegmail 3 года назад

Thank you Simon.. One thing not very clear is, how are they scanning for data lineage? Scanning tables alone won't do that. Did not see a place to scan ADF flows or SSIS flows. Thoughts?

@AdvancingAnalytics 3 года назад

I'm not sure if it's changed since I looked at the preview - There's a fair bit about the ADF & Data Share setup in the docs, but when I had a quick look I couldn't see it - might not be in the public preview just yet! Expect it'll follow what's described here: docs.microsoft.com/en-gb/azure/purview/catalog-lineage-user-guide

@iphadkegmail 3 года назад

@@AdvancingAnalytics ty!

@NeumsFor9 3 года назад

You should check out the blog of Mr. Paul Andrew, who put together a nice metadata framework for Azure Data Factory in a similar way that sqlmetadata.codeplex.com did about 10 years ago.

@iphadkegmail 3 года назад

@@NeumsFor9 Can you share the link? Ty!

@somediver9925 3 года назад

You can also do API calls against ADF and I would suggest you store the data as key value pairs since there are a HUGE number of hierarchies. You can also do API calls within ADF just be aware of paging the result sets.

@saurabhkp89 3 года назад

I have been using Babylon data catalog for sometime but it's never stable , Hopefully with purview things will stable enough to do something.

@AdvancingAnalytics 3 года назад

Yeah, but that's the nature of internal/private preview - testing features not SLAs! Not to be cynical, but I imagine proper stability will come a few months after GA...