AZ-900 Episode 15 | Azure Big Data & Analytics Services | Synapse, HDInsight, Databricks

Подписаться 201 тыс.

Просмотров 255 тыс.

50% 1

Microsoft Azure Fundamental full course.
Skills Learned
- Describe Azure Big Data & Analytics Services such as
- Azure Synapse Analytics
- Azure HDInsight
- Azure Databricks
Study Materials: marczak.io/az-900/#ep15
* Episode Resources
* 📚 Study cheat sheet marczak.io/az-900/episode-15/...
* 🧠 Practice Test marczak.io/az-900/episode-15/...
* Study Guide
* Microsoft Azure: Azure Synapse Analytics azure.microsoft.com/en-us/ser...
* Microsoft Azure: Azure HDInsight azure.microsoft.com/en-us/ser...
* Microsoft Azure: Azure Databricks azure.microsoft.com/en-us/ser...
* 🌐 Wikipedia: Big Data en.wikipedia.org/wiki/Big_data
* Expand your knowledge - extra resources
* Microsoft Documentation: Azure Synapse Analytics docs.microsoft.com/en-us/azur...
* Microsoft Documentation: Azure HDInsight docs.microsoft.com/en-us/azur...
* Microsoft Documentation: Azure Databricks docs.microsoft.com/en-us/azur...
* 🌐 Apache Spark Homepage: Overview spark.apache.org/
* Azure Databricks Tutorial | Data transformations at scale • Azure Databricks Tutor...
Agenda
00:00 Episode introduction
00:30 What is big data?
01:42 Azure Synapse Analytics
04:15 Azure HDInsight
05:27 Azure Databricks
09:24 Summary of Big Data services
Want to connect?
- Blog marczak.io/
- Twitter / marczakio
- Facebook / marczakio
- LinkedIn / adam-marczak
- Site azure4everyone.com

Наука

Опубликовано:

25 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 185

@markcaneda7764 3 года назад

The channel name says it all, very intuitive visualizations and explanation. This is much better than udemy paid courses. Thank you for creating such contents!

@AdamMarczakYT 3 года назад

Wow, thanks! Appreciate your kind words Mark :)

@idougy18091809 Год назад

It's even better that Ucertify which a course leader was teaching from.

@kavinyudhitia Месяц назад

Let me also add... this is even better than Coursera courses (that was created by microsoft themselves).

@BijouBakson 2 года назад

Here I am, 2 years later and now an Admin associate, preparing for my Solutions Architect Expert AZ-305 exam... and still not entirely clear with Big Data. I am having to do the Data Engineering module to deal with it once and for all. These videos help clear things out. Thank you Adam.

@koduruchandu 3 года назад

@MarczakIO Hi, this is Chandra from Bangalore, India. Today i have successfully cleared AZ-900 exam by just going through the 37 youtube vedios you prepared. In spite of having prior fundamental knowledge on cloud computing, i could pass the exam with the help of your vedios on Azure. Hence, would like to thank you a lot 🙏 for making available such a great vedios which anyone can easily understand the Azure basics needed for exam.. Is a great social service..so nice of you😊

@AdamMarczakYT 3 года назад

Thank you, highly appreciated :) Glad you passed, keep it up!

@PawanKumar-tu6ti 3 года назад

Here we meet again, Adam! I passed my exam yesterday of az-900, thanks to Microsoft documentation and especially YOU, and your teachings.

@AdamMarczakYT 3 года назад

Fantastic! congrats, keep up the good work :)

@PawanKumar-tu6ti 3 года назад

@@AdamMarczakYT thanks, can you start making azure series on az-204? At least make a roadmap and prerequisites for it.

@oguzhankrcal357 3 года назад

Today, I've joined to Microsoft Azure Virtual Training Day: Fundamentals meeting. It has been finished yet and I missed you. ☺️ I came to here by running. Thanks man. Your English is understandable, your blackboard is awesome. They should learn how to present AZ-900 from you.

@AdamMarczakYT 3 года назад

Wonderful! Welcome back to where all the action is! :D

@sumitprakash6840 2 года назад

You are simply fantastic. I really love the way you describe each topic. Thanks for support and looking ahead to see you more to take advantage of new technology.

@abhisheksoni6012 Год назад

Thank you so much Adam for making these difficult-to-grasp concepts very easy to understand and interesting to learn. There are no words for your teaching skills. You are awesome. Thanks a lot Adam !

@simonkelso3447 3 года назад

Great Job Adam, this is forming the basis for my Azure learning. Keep up the good work !!

@AdamMarczakYT 3 года назад

Great to hear! Will do!

@benediktemydlandsolend8515 3 года назад

Thank you for your explanations. Love the way you illustrate and show us examples. Looking forward to your next video.

@AdamMarczakYT 3 года назад

More to come! thanks!

@ngophuthanh 3 года назад

Thanks a lot, Adam. As always, this episode is very well organized and presented. Diagrams and visualizations are excellent. Your deep knowledge and teaching skills are amazing.

@AdamMarczakYT 3 года назад

Many thanks! :)

@ilyasontube 3 года назад

all of the Adams videos are not only very clear and informative but also very inspirational, real MVP!

@AdamMarczakYT 3 года назад

Great to hear! thanks!

@SanatanYadav108 2 года назад

Hi Adam, you are fantastic mentor. Your presentation is extremely interesting. Thanks alot for your hard work behind these presentations.

@vitovito123 2 года назад

Adam, thank you so much. I passed my exam today.

@mmohan1234 Год назад

Excellent video on Azure's Big Data and Analytics products.

@jcb3883 2 года назад

I'm getting my certificate this friday. Wish me luck. Thank you for doing this series.

@csabafeher2632 3 года назад

I cannot describe how useful, professional and engaging your content is! Best of its kind! Btw, after couple of episodes I have realized you have cool T-Shirts! ;-)

@AdamMarczakYT 3 года назад

Wow, thanks! :)

@aurosish1 3 года назад

Thanks to you Adam, now I got a quick overview of Azure Big Data & Analytics Services.

@AdamMarczakYT 3 года назад

Fantastic! Thanks!

@thesloable 3 года назад

Thanks for creating concise and easy to understand materials. Hope to see more courses covered in this channel.

@AdamMarczakYT 3 года назад

Thanks, will do!

@binhngo12336 2 года назад

Adam. u are the king. best teacher I ever seen man. all the best for u man...

@clintonchristopher6532 3 года назад

Hi Adam, been following back to back on this video tutorial as i am about to take the azure fundamental exam, and i must say its been extremely helpful. Thanks so much ...

@AdamMarczakYT 3 года назад

Fantastic!

@shaunakv100 3 года назад

Thanks a ton for making this videos. You have not only simplified all concepts for better understanding but also kept it interesting till end. Your presentations, diagrams are also so clear. I learned a lot from your videos. I will be happy if I can support you in someway Thanks Again for helping us with these videos

@AdamMarczakYT 3 года назад

Awesome, thanks!

@scotmarkieboy 3 года назад

Hey Adam, thanks for putting this content together and making it available for free.

@AdamMarczakYT 3 года назад

My pleasure!

@alexluke_again Год назад

the least i can do i like each video and recommend to friends. thank you for doing this!

@spencerthomas9377 Год назад

Great stuff as always!

@prititiwari5183 3 года назад

You explained every topic thoroughly.

@AdamMarczakYT 3 года назад

Thank you! :D

@KDOERAK 2 года назад

another great overview!

@sanghvian 3 года назад

I am learning so much apart from just Azure !!! Thanks a lot Adam !

@AdamMarczakYT 3 года назад

My pleasure!

@timmusaka9577 Год назад

The best content is on RU-vid!

@gauravkumar796 3 года назад

Awesome.. I like the summarised comparison.

@AdamMarczakYT 3 года назад

Thank you! Appreciated :)

@alamzaibfarooq 2 года назад

The course is much better than Udemy courses. How is it possible that Adam is not charging anything for this course. He is amazing. The animation is slides are very easy to understand and eye catching.

@luism2754 Год назад

Dude for real, I have been paying a curse that is hours of hours of a guy saying the same thing over and over and just making everything more complicated. But thanks to Adam now I am understanding everything

@davidgates5189 Год назад

great as always

@franciscorossi5633 2 года назад

You are a genious, thanks mate!

@ruchijain2584 3 года назад

Thanks for amazing videos !!

@AdamMarczakYT 3 года назад

My pleasure!

@Lordxfx 3 года назад

Finally an insight to HDinsight :)

@AdamMarczakYT 3 года назад

😄

@jvstechtips Год назад

Very helpful, thanks

@stephaniebuadu3474 3 года назад

This was great! Thanks

@AdamMarczakYT 3 года назад

Glad it was helpful!

@thekbee5736 3 года назад

The explanations in your videos are the easiest to understand of everything else out there

@AdamMarczakYT 3 года назад

Thanks! :D

@farshidjamali 3 года назад

Thank you, Adam!

@AdamMarczakYT 3 года назад

My pleasure!

@toluagoro3232 3 года назад

You are amazing. The way you explain key concepts is top notch. I wish you can teach me for my security + certification

@AdamMarczakYT 3 года назад

Fantastic, thank you, best of luck! I wish I could teach everyone, that's why I created this channel!

@talentdlamini9326 3 года назад

you a life saver thanks Adam

@AdamMarczakYT 3 года назад

Thank you! :D

@saltnlightmalaysia 3 года назад

Wonderful material!!!

@AdamMarczakYT 3 года назад

Thank you! Cheers!

@denkozlov4220 2 года назад

thank you Adam

@debreczenytivadar1220 3 года назад

Hello Adam. Now I watch all your videos and everything is very professional. Here’s a little random sound clash around 9:28.

@AdamMarczakYT 3 года назад

Thanks. Yea some overlap happened there of two audio tracks, unfortunately youtube doesn't allow to reupload the videos. Would need to delete this.

@JagdishJ666 3 года назад

Outstanding explanation and easy to learn

@AdamMarczakYT 3 года назад

Thanks! :D

@jhosnatas 3 года назад

I've been aproved in az-900 thank you

@AdamMarczakYT 3 года назад

Congrats :) Keep it up!

@devanandapatro7889 3 года назад

Hi Adam, ur way of teaching is really awesome!! I attempted the az900 exam today and cleared with good marks.. Thanks for all your effort.. 😊😊

@devanandapatro7889 3 года назад

@@wiroee yes, I have watched all episodes and thereafter took a practice exam from Udemy..

@AdamMarczakYT 3 года назад

Great job! Thanks! :D Keep it up! :)

@samruddhikalsekar1038 3 года назад

@@devanandapatro7889 could you mention which practice exam exactly? There are multiple options on udemy

@swatikandarpatel1127 2 года назад

Thank you I cleared exam today.

@SpeoLori 2 года назад

This course is amazing. Congratulations. And there are deep-dive videos about specific topics. How did you created the drawings? They are so neat.

@vinayinfo15 2 месяца назад

Nice cover of az 900 study tipics

@PawanKumar-tu6ti 3 года назад

RU-vid knows how to suggest people, here we come again, thanks brother, your help means a lot, with every video I watch, I can't resist myself from leaving a comment!

@AdamMarczakYT 3 года назад

I appreciate that, thank you!

@dhawrylkiw Год назад

Thanks!

@rqn2274 3 года назад

great video as always

@AdamMarczakYT 3 года назад

Thanks again!

@jurijssolovjovs3575 3 года назад

great work thanks mate.

@AdamMarczakYT 3 года назад

No problem 👍

@tustinnrent8502 3 года назад

another great video! waiting for DP-900! :-)

@AdamMarczakYT 3 года назад

You and me both!

@tustinnrent8502 3 года назад

@@AdamMarczakYT JK, no pressure!!! Started doing it the old-fashioned way this week by reading MS docs. Many of your videos have already been an excellent complement!

@pankajkumar-wt2ro 3 года назад

Good content Adam ,I am waiting for next video

@AdamMarczakYT 3 года назад

More to come!

@dharangsharma6863 2 года назад

Hi Adam ! Just a quick question, Are your Practice Tests enough to clear the exam, or do I need to purchase some sample papers before appearing for the exam? Also want to thank you for the amazing content, will definitely recommend it to others.

@AjayVerma__0001 6 дней назад

Nice ❤

@mrindian7422 3 года назад

Nice explanation.. thanks

@AdamMarczakYT 3 года назад

Glad you liked it

@abdulkleit 3 года назад

Hello Adam, eagerly waiting for your remaining episodes from 32-39.

@AdamMarczakYT 3 года назад

Soon! :) I had some time of during x-mas and new year, back on schedule soon! :)

@abdulkleit 3 года назад

@@AdamMarczakYT Thank you Adam. Happy New Year!

@windmid39 3 года назад

This is good for DP-900 too. The comparison make thing more clear but not just list out the services and characteristic.

@AdamMarczakYT 3 года назад

Thanks! Yea some topics are mandatory for all fundamental exams like AZ-900, DP-900, AI-900 and even MS-900.

@RandyRieger 3 года назад

You need to stop being so awesome. Actually don't. Love how you explain everything

@AdamMarczakYT 3 года назад

Can't stop being awesome!

@shaileshsingh1445 3 года назад

you are awesome :)

@AdamMarczakYT 3 года назад

Thanks! :D

@sergiocoutinho6133 3 года назад

Congratulations for the videos .. your diagrams are excellent for assimilating the content and i am now confident to do AZ-900 soon ! But I confess that I get a little confused by the terms PIPELINE SYNAPSIS, DATA FACTORY or HDINSIGHTS .. sometimes they seem the same thing to me. I even thought that SYNAPSIS corresponded to the old SQL AZURE DATA WAREHOUSE, without the pipeline option. Best regards !

@AdamMarczakYT 3 года назад

Best of luck on the exam! You can do it! Those services serve similar purposes but the details are different. Only once you dive into each one of them, then you will be able to distinct them better! :)

@sahajrajmalla 3 года назад

I have been using Azure SQL and making API to query those databases but the query is taking very long time to execute. Now i want to add spark's distributed system and query through the database. Which azure service is better suite for me? Thank you for this wonderful free content Adam. I really appreciate it alot!

@TheLangmusi 3 года назад

Thank you for the summary. I wonder what if I want to save a data frame to Azure SQL DB in a Python Notebook like you demonstrate here? Moreover, can you explain which line on your Python block 1 register df as temp table in SQL? Thanks for your answer.

@AdamMarczakYT 3 года назад

Note that I registered data frame as Spark SQL view, it's not SQL DB view. Check out my azure databricks tutorial to learn a bit more on how to operate using databricks. Thanks for stopping by.

@TheLangmusi 3 года назад

@@AdamMarczakYT Thanks for the answer. Ok, I got it. I will watch your databricks tutorial for more info. Thanks agin.

@limsky8012 3 года назад

When I should choose the Synapse and when Databricks? What type of question I should answer to make a decision?

@AdamMarczakYT 3 года назад

It's a difficult choice. Maybe I'll do a video on this in the future, there are too many factors to explain in a simple comment.

@Abecedarianme 2 года назад

Very informative video. Hoping you could do for DP-203 too.

@AdamMarczakYT 2 года назад

Noted, thank you :)

@SiyamandRashid 3 года назад

Very good videos and the most beautiful part of it the way you prepared... what kind of tablet did you use please? Regards

@AdamMarczakYT 3 года назад

It's just my surface pro laptop :)

@SiyamandRashid 3 года назад

@@AdamMarczakYT thanks a lot for the reply, i just bought wacom small size but it is kind difficult to draw... Keep it up brother

@rs-tarxvfz 2 года назад

6:20 I think they have unified the creation of workspace within the Azure Portal itself. So no need to go to another platform

@sid0000009 3 года назад

Hello Adam, how do we choose between Databricks and HD Insight if we need a Spark Cluster for our use case?

@AdamMarczakYT 3 года назад

Thanks a tough question! But a very good one. I guess both can achieve pretty much the same so it's just matter of organizations/skillsets/etc. In my company we decided to use Databricks for all out data transformations projects. But I have few MVP colleagues which prefer using HD Insight. Some companies also use HD Insight because it's more secure.

@haivercarrillo8603 3 года назад

Hi Adam, great content as usual, can I ask you a favor, if you can explain the azure data explorer. Thank you!

@AdamMarczakYT 3 года назад

Thanks. ADX is definitely on the list :)

@BijouBakson 3 года назад

God, that was heavy! Oopf! fundamental huh!? :) I definitely didn't grasp this before I passed my exam, still don't understand it much. This course is a little rushed in my opinion. May it's because it's an entire complex field? I'm watching for the 3rd time now. Let's go. :)

@AdamMarczakYT 3 года назад

Well the services are quite similar so I understand how they can be confusing. They are competing offers for big data. But don't give up! :)

@BijouBakson 3 года назад

@@AdamMarczakYT Haha! my vocabulary is short of give-ups, luckily. I understand the overall idea better now! thank you :)

@timoose3960 2 года назад

Does anyone know what tools he uses to make the drawings/sketches on top of the video?

@cebabu 3 года назад

Generally we store enriched and transformed data sql dataware house, now callled synapse. Can I also store all raw data(structure ) in synapse.

@AdamMarczakYT 3 года назад

Those are just tools, use them however you like/need :)

@abhijitboda Год назад

THIS PART IS REMOVED FROM EXAM SINCE MAY 2022

@5siah530 3 месяца назад

Is this the only topic covered in the course that has been removed from the exam?

@jorgeregis5614 3 года назад

Your are awsome , also give us some tips for learn english hahaha , your accents is understandable.

@AdamMarczakYT 3 года назад

Thank you! 😃

@Mybigsissy 3 года назад

question 5 on your practice test: the question was about migration, so how does an analytics service perform the task?

@AdamMarczakYT 3 года назад

Migration is a process, not a tool :) Think of it from the perspective of a company.

@Search4Knowledge 3 года назад

Thanks for this awesome info! Just a question, is HDInsight still part of the DP200 certification? Also when to choose between HDInsight and Databricks as both are spark based?

@AdamMarczakYT 3 года назад

Well as you've seen in the video HDinsight is not spark based, Spark is one of several available engines for HDInsight. But more detailed comparisons can be made when using Spark here and in Databricks. In general Databricks is better optimized version of the spark as it was created by Spark creators.

@StephenBoesch 6 месяцев назад

Can these capabilities be scripted or configured via properties/json files? I prefer to use text editors to set things up *repeatably*, check in the config scripts to git, and be able to clone and tweak them.

@Paweljansienkiewicz 12 дней назад

Świetne kursy Adam, dziękuję :) Przerobiłem learning path na stronie Microsoftu i nie było tam informacji na temat IoT, Big Data czy AI. Nie znalazłem też tego w spisie wymaganych zagadnień. Źle szukałem czy te zagadnienia kryją się pod jakimś większym? Dzięki!

@MrDanesorensen 2 года назад

Struggling to understand the key differences between Azure Synapse Analytics & DH Insights. Anyone able to help clarify this for me?

@haiyingwang5987 3 года назад

Dear Adam, is it normal for free subscription to get "cloud provider launch failure" when trying to start a cluster in databricks? Is there anyway to get it starting somehow? Thank you!

@AdamMarczakYT 3 года назад

Possibly, free subs have 4 vCore limits. try selecting single node checkbox and select smallest available cluster size, which I think was F4s.

@sidarthg2477 Год назад

I think time for an update to the course. Coz I think this course was formed before DP-900 came into existence. So all these portions are now not under AZ-900 and instead it now emphasize more on networking and less on data analytics tolls

@hochun836 2 года назад

Hi Adam, is there a plan to make a video about synapse tutorial ? I like your tutorial !!

@AdamMarczakYT 2 года назад

Possibly! Thank you! :)

@shaunakv100 3 года назад

Do you have also similar videos on DP-900 as well?

@AdamMarczakYT 3 года назад

Not yet, maybe in the future :)

@rs-tarxvfz 2 года назад

Hi Adam, I din't get difference between *synapse pipeline vs data factory*. Pls..

@imsteven3044 2 года назад

What program do you use to make the notes of your videos?, such as the minute 1:23

@gocrow23 3 года назад

Can't find code, can you pls put the Databrick code piece you used for illustration?

@AdamMarczakYT 3 года назад

Hey, check my databricks tutorial. It’s much better if you want to learn it and it has samples attached. In this tutorial I just showcased databricks but I didn’t expect people to follow. I’m on holidays now but I can upload samples from this episode when I get back. Thanks for watching :)

@vr1908 3 года назад

Hi Mark, Can i have a script for "Connect to Microsoft Opendatset for Sample Data"

@thegreenroom5820 4 месяца назад

I am assuming this type of collaboration is extremely expensive in production

@massimo8672 3 года назад

how do we get the slides

@AdamMarczakYT 3 года назад

Sorry Massimo but I don't share my decks, thanks for the visit! :)

@Mauszinator666 2 года назад

I don't know if i'm the only one, but the sounds got a little messed up around 9:28. Maybe its just me, or otherwise i hope it is usefull feedback

@artak9439 3 года назад

What about Azure SQL Data Warehouse ?

@AdamMarczakYT 3 года назад

It's been renamed to as SQL pools in Synapse.

@artak9439 3 года назад

@@AdamMarczakYT Thanks

@CosmeJunior 3 года назад

These episode for me, the most difficult to memorize any tips from anyone ?

@AdamMarczakYT 3 года назад

Try my reference study materials and read Microsoft Learn, maybe a different perspective on the topic will help!

@thprodigy 3 года назад

what's the difference between SNOWFLAKE and Azure HDInsight?

@AdamMarczakYT 3 года назад

Closer comparison would be between Azure Synapse Analytics and Snowflake. They are both Massively Parallel Processing databases with unified workspace and ETL capabilities.

@BijouBakson 3 года назад

You missed to mention that Azure Synapse Analytics is the new name for Azure Data Warehouse.

@AdamMarczakYT 3 года назад

I didn't miss it. It was on purpose. Majority of the people who study Azure Fundamentals are not familiar with Azure so I rather avoid using already outdated terminology/names as it might introduce confusion. Of' course I will mention this in my upcoming Azure Synapse tutorials later this year.

@BijouBakson 3 года назад

@@AdamMarczakYT I understand now, thank you

@juggergabro 2 года назад

09:28

@Downstar 3 года назад

9:28 I thought that i accedently opend a video in a new tab or something hahah

@AdamMarczakYT 3 года назад

Hehe, small audio sync issue :)

@Downstar 3 года назад

@@AdamMarczakYT hahah no problem, i really ejoyed your content and i passed the exam yesterday! U have learned me alot, thanks!

@uskapyaar8852 3 года назад

i am wondering who would've disliked this video

@AdamMarczakYT 3 года назад

Me too :)

@uskapyaar8852 3 года назад

@@AdamMarczakYT you're just too nice at explaining all these concepts that too for free.. i am gonna stick with you.. 🙏 thanks for help i am feeling confident for my exam now ❤️