Olympic Data Analytics | Azure End-To-End Data Engineering Project

Подписаться 135 тыс.

Просмотров 277 тыс.

50% 1

Join My Data Engineer Courses - datavidhya.com/courses
➡️ Combo Package Python + SQL + Data warehouse (Snowflake) + Apache Spark: com.rpy.club/pdp/yYnEMzLOX?pl...
USE CODE: COMBO50 for 50% off
In this video, you will analyze Olympic data using various tools and technologies, including Azure Data Factory, Data Lake Gen 2, Synapse Analytics, and Azure Databricks.
Get more projects like this here - bit.ly/3saZfeo
Watch Part 2 here - • Olympic Data Analytics...
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
1. The dataset used in the video: github.com/darshilparmar/toky...
2. Azure account creation - portal.azure.com/
3. Original Dataset link - www.kaggle.com/datasets/arjun...
Errors? Join Data With Darshil Discord Server: / discord
Timestamps
0:00 Project Intro
1:30 Understand Architecture & Prerequisite
6:58 Understand Dataset & APIs
10:49 Azure Basics, Services Understanding, Account Setup
24:05 Complete Project Execution
👦🏻 My Linkedin - / darshil-parmar
📷 Instagram - / darshilparmarr
🎯Twitter - / parmardarshil07
🌟 Please leave a LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟
3 Books You Should Read
📈Principles: Life and Work: amzn.to/3HQJDyP
👀Deep Work: amzn.to/3IParkk
💼Rework: amzn.to/3HW981O
Tech I use every day
💻MacBook Pro M1: amzn.to/3CiFVwC
📺LG 22 Inch Monitor: amzn.to/3zk0Dts
🎥Sony ZV1: amzn.to/3hRpSMJ
🎙Maono AU-A04: amzn.to/3Bnu53n
⽴Tripod Stand: amzn.to/3tA7hu7
🔅Osaka Ring Light and Stand: amzn.to/3MtLAEG
🎧Sony WH-1000XM4 Headphone: amzn.to/3sM4sXS
🖱Zebronics Zeb-War Keyboard and Mouse: amzn.to/3zeF1yq
💺CELLBELL C104 Office Chair: amzn.to/3IRpiL2
👉Data Engineering Complete Roadmap: • Data Engineer Complete...
👉Data Engineering Project Series: • Data Engineering Proje...
👉Become Full-Time Freelancer: • Best Freelancer Series...
👉Data With Darshil Podcast: • Podcast Series - Data ...
✨ Tags ✨
azure data engineering, end-to-end azure data engineering project, azure data engineering project free
✨ Hashtags ✨
#azure #dataengineering #databricks

Опубликовано:

15 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 482

@hannahabigail909 5 месяцев назад

I've checked bunch of channels and series videos and it seems you know what you're doing and love what you're doing. Great great work! Subscribed. Keep up!

@DarshilParmar 10 месяцев назад

These many views but why LIKE counts are still less? Take one second to like and help this channel to grow :) please?

@Cussmem07 10 месяцев назад

Thank you for your effort, sir.

@viswanathtgr7966 10 месяцев назад

Thank u

@GATE_Education 10 месяцев назад

multiple copy activities can we use one for each activity instead of inside copy activities

@prabhatgupta6415 10 месяцев назад

@@GATE_Education yes u can use just pass the parameter at run time. have a textfile with name of all the data then read it one by one then pass it as a parameter

@HHHMED2009 10 месяцев назад

man, you are really amazing

@rameshbayanavenkata1305 10 месяцев назад

Thank you Darshil for providing such end-to-end data engineering projects for our viewers. The efforts you been putting on are so great. looking forward for more and more projects... thank you once again...

@DarshilParmar 10 месяцев назад

You are most welcome

@shrirambuchkul 4 месяца назад

java.lang.NullPointerException how to solve it@@DarshilParmar

@syedjunaid123 8 месяцев назад

I'm so glad I found this video before I started my Azure Data project. It saved me so much time and frustration. And I'm definitely subscribing to your channel and looking forward to more Data related tutorials.

@user-xp8gk2xv9g 8 месяцев назад

Hey Darshil, I wanted to thank you for the informative and useful session. Your content is great and very helpful. I appreciate that you are one of the few RU-vid channels that covers end-to-end data engineering projects.

@balive053 10 месяцев назад

Great video! very clear explanation without going into too much detail. Thanks for making it!

@SunilNagavelli 7 месяцев назад

Your content is gold!. I hope you keep doing such exciting work. Glad to have come across your channel!

@pakhijoshi3822 8 месяцев назад

Darshil, you are an amazing human being, giving such informative tutorials free of cost is quite rare these days so I really appreciate your efforts. Thank you so much for teaching with so much clarity and simplicity. You're the best!!❤

@DarshilParmar 8 месяцев назад

Thanks a ton

@ShahrukhHussain-rn7pg 7 месяцев назад

Amazing introductory course taught with adequate attention-to-detail, and well delivered too! Kudos man

@analyticsWithJay 6 месяцев назад

This is amazing, bro! Thank you for the detailed hands-on session. It's easy to understand.

@alanfung4516 7 месяцев назад

I have watched many data engineering videos on youtube, this is by far the best one I ever watched, thanks alot

@upper_cut5159 2 месяца назад

I had not idea what I was doing. But I followed your lead and completed the project. I'm sold on this journey. Thank you!

@shubhamgupta1632 10 месяцев назад

Whenever I watch your video, I gain new insights. Thank you; your efforts are truly appreciated.

@priyalverma234 2 месяца назад

Hi Darshil, This video helped me a lot in kickstarting my understanding of Azure Databricks. I am glad to have come across your channel. Thanks a ton for this and I hope your channel grows beyond expectations :)

@priyankadp5777 10 месяцев назад

This was awesome Darshil.. had a great time implementing this alongside you.. great explanation.

@RAHULRAHUL-xb7xd 8 месяцев назад

amazing Drashil, as you said in the video many people take thousands of money for this but you still explained it very easy and free of cost for the viewers, appreciated brother..... thank you

@kdiggity1 3 месяца назад

Love the structure of this walkthrough, the level of detail is just right amount.

@anonymousgal100 7 месяцев назад

Hey Darshil, I have completed first part, following along. A big thanks to you for putting so much efforts. The idea of explanation of services along with ongoing project is awesome. Keep doing such great work.

@yizmo5674 9 месяцев назад

This is my very first comment on RU-vid normally I just watch but ever since I got the idea of transitioning into a data engineer, I've been watching a lot of videos and I am 100 percent sure this is the best! Thanks, Darshil!

@data_with_michalis 10 месяцев назад

Bravo for your effort! Video quality has been improved a lot. Keep going!

@rajarshisikdar8800 6 месяцев назад

You are doing a very good service for the data engineering aspirants. Keep it up. Godspeed.

@vaibhavkamble3325 6 месяцев назад

Hey Darshil, Your video was a game-changer for me! I've always been curious about the inner workings of the industry, and your content came at just the right time. Your teaching method is incredibly effective, breaking down complex concepts into easy-to-understand chunks. You've given me the direction I needed to kickstart my learning journey in this non-tech realm. Now, I not only understand how it all works but also know where to focus my efforts. Your comprehensive approach for beginners is exactly what I was searching for. Expect me to be a regular viewer, soaking in all the valuable insights you share. Thanks a ton for being the guiding light I needed!

@piyushlakheda 9 месяцев назад

Great video! Not a Data person but really understand the concepts from it! Moving to next part now! Thanks a Lot for this! :)

@imehmood4839 3 месяца назад

Just finished this part!! Thank you for creating this valuable content.

@uchiha_sasuke_48 9 месяцев назад

By watching until the architecture part, you grabbed me as your subscriber. Looking forward to complete all the project playlists. 🎉🎉🎉

@Mark_Trail Месяц назад

Great video! Love your speed of pace.

@gauravmaheshwari1807 7 месяцев назад

Hey Darshil, Thanks a ton for such a valuable crisp amd concise knowledge transfer video! Really Appreciate your work!

@ranjitprakash1986 8 месяцев назад

Exactly what i was looking for in cloud computing using azure, thanks so much!

@mohamedmeqlad8064 10 месяцев назад

this the kind of projects I have been waiting for. I really want to thank you for your great content

@DarshilParmar 10 месяцев назад

My pleasure!

@pratikgugliya3771 10 месяцев назад

Thank you so much brother for providing such a beautiful insights on end to end data engineering project.

@sayan1231000 7 месяцев назад

just one word: WONDERFUL

@gowreeManohar 8 месяцев назад

a quick overview to understand the similarities between each of them! great video bro

@abhisheksivalingala6420 9 месяцев назад

Great ....you are just giving the valuable content for free....god bless❤️🙏

@BabaiChakraborty-ss8pt 10 месяцев назад

Absolutely fantastic Tutorial. Please continue to post such in future.

@aryamathew2273 5 месяцев назад

Thanks Darshil! I was able to complete the project, and I learned so much along the way!

@satyaprakashch6290 5 месяцев назад

Great content Darshil, After completion of data engineering internship, Iam looking for real time open source projects, this is one of the best project i have found it covers all major Tools/Technologies that i have learned in the internship. Thank you.

@dhruva24_ Месяц назад

Found the project helpful! Thanks a lot Darshil!

@abhijeetnayak9077 7 месяцев назад

Nice Explanation bro, experts generally cut of the thinking part and just show things in a mechanical way, but you tellin your though process while doing things makes learning very interesting and avoids distraction as we are not needed to serch for small small details at any other place.... Thanks man

@vaijayantigulve4883 9 месяцев назад

I liked the UI of Azure Synapse Analytics better than the others. Thank you so much for all the knowledge.

@funsoiyaju614 10 месяцев назад

Thank you Darshil for sharing this content. May God continue to bless you

@PerkAsh 22 дня назад

Thank you! This is so insightful!

@OmolaraFash 4 месяца назад

Thank you again for this video, I just concluded my version, and I am excited to make on research on what have learnt and keep practising

@bittencourt1869 25 дней назад

Congrats perfect explanation comparing to other channels now I can cleaar understand all the steps ingestion, transformation and storage

@gtanation4729 5 месяцев назад

Just completed my first project, Thanks alott Keep upload azure data engineering videoss

@ahmetalperenyildirim6184 8 месяцев назад

Awesome project, I'll do to add my portfolio!! Thank you for sharing!

@Mustafabatman611 9 месяцев назад

Thanks a lot, Mr. Darshil please Keep posting like these videos about data engineer projects again thank you a lot your videos help me a lot

@OmolaraFash 5 месяцев назад

I just completed the first part without any issues, welldone and thanks for this content, looking forward to more

@fernoss Месяц назад

As a Junior Data Engineer, had the opportunity to use Azure sandbox - bruh, this video is gold! Thanks for this, learnt so much and documented my process as a step-by-step guide to learn as I go. This will definitely go to my portfolio to showcase all these amazing tools that are actually used in real life, solid work here! Didn't have admin rights for app registration, so had a little workaround here and went with SAS-token, got it to work nevertheless! (to anyone who is wondering). Keep up the good work, all the love and support from here!

@ritwik99 28 дней назад

This video is just awesome to learn so many things and understand. Thank you so much for creating such helpful content and making things simple and clear. You are doing a great work, keep going, you just got a new subscriber :)

@akshaydeshmukh8602 4 месяца назад

You are doing grate work, thank you so much for providing such content which is full of knowlegde. 💙

@ramireddyvarun26 2 месяца назад

Thanks so much Darshil! Wonderful - it really helps with the jitters of understanding so many different tech stacks! :)

@randyvelasco5079 5 месяцев назад

Just finished this part without issues... easy to follow. Will do the second part a little later... Thanks Darshil.

@deepakrai6824 8 месяцев назад

Thanks Darshil for this amazing project.

@VetinCloud 9 месяцев назад

Thank you for putting this together from PR. Clear explanations and to the point... I like how you made mistakes and solve them on screen and explanation of services. Ill follow and await patiently while honking my horn on the parking lot :-) for more content. Quick question while going through it the col function was not there on data Bricks. is it common to install the library through the commands or should I install libraries at the beginning like I have seen on regular python scripts?

@happyheart9431 6 месяцев назад

I just want to say a million, thanks.. I just finish my Azure Synapse Analytics certification, and this video gave me so much confidence for my interview.

@preethamreddy787 3 месяца назад

Hey hi, I am thinking to start learning azure synapse. Could u pls guide me how can I start with and any suggestions for resources. Thanks in advance 😊

@mrmanualways 9 месяцев назад

Brother, this is great. Keep up the good work. Liked and subscribed

@SUMITKUMAR-ci3me 10 месяцев назад

Ya definately most wanted channel 🎉 your every video is most appreciated. Please continue this series and upload part2 of this project

@DarshilParmar 10 месяцев назад

Part 2 is already available, please check description

@user-jx3ms2pf9p 5 месяцев назад

Thank you for sharing your knowledge with us. Lots of support from everyone.

@terimaakichuth69 8 месяцев назад

Thank you for the detailed project. I'm going to learn and replicate it in my free time and on the weekend

@mohammedshoaib1769 10 месяцев назад

Great content. Completed implementing the same. It was smooth.! Waiting for second part.

@DarshilParmar 10 месяцев назад

It’s already there, check description

@tusharhatwar 10 месяцев назад

More Support to you brother ❤ All your videos are helpful And your Channel is DEFINITELY a GOLDMINE 😊

@DarshilParmar 10 месяцев назад

Thanks a ton

@wgsam82 9 месяцев назад

Awesome, appreciate your time and effort, thanks a lot.

@luisosorio4456 10 месяцев назад

Thank you so much! Best Data Engineering Platform!

@DarshilParmar 10 месяцев назад

You're very welcome!

@arfatahmedansari8916 Месяц назад

Thank you so much for creating and uploading this content.

@kindly_seven 9 месяцев назад

I jusr found a Gem today. Thank you Darshil sir 🎉

@eugene2653 10 месяцев назад

This is pure gold!

@user-zk8ce2gj1w 7 месяцев назад

With this kind of amazing tutorial and explanation gives interest to learn more

@oumaymaredissi5213 6 месяцев назад

Great video! thank you for the effort

@ajayigeorge1356 10 месяцев назад

Well done Darshil very intuitive video, appreciate the work. Can you do a video on how to schedule a python script on azure to run on a schedule, the python script could be extracting data from an API or something and then loading it to blob storage. Just giving ideas.

@AIMLDLNLP-TECH 7 месяцев назад

Very informative and clearly explained appropriated 🔥

@sivakumarrajabather1140 2 месяца назад

Really awesome explanation. it really helps. Thanks.

@thedataholics 5 месяцев назад

it was an awesome learning... thanks man for such wonderful work

@freetrainingvideos 6 месяцев назад

Excellent, and very well explained. Thanks

@abhimanyuwadhwa6335 10 месяцев назад

Thanks for such amazing content Darshil! Trying out Azure for the first time. :)

@DarshilParmar 10 месяцев назад

Best of luck!

@alexanderpotts8425 7 месяцев назад

"Have some water, stay hydrated". love it

@vaibhavisuresh04 7 месяцев назад

Thankyou, this video was very helpful!😊

@abdelghafourelgharbaoui3886 8 месяцев назад

clean work | thank you!

@bukunmiadebanjo9684 9 месяцев назад

Great content Darshil. Just concluded the first part of this project.

@atharvavandhye1358 Месяц назад

Good job darshil!

@Levy957 10 месяцев назад

Amazing Darshil

@Chan-ln3bv 2 месяца назад

Thank you so much.part1 was done successfully .. NXT -> moving to part 2

@rakeshverma6867 10 месяцев назад

Crisp and simple . Thanks @Darshil

@DarshilParmar 10 месяцев назад

You are welcome

@marianasimoes5891 4 месяца назад

Amazing! You helped me a lot 😄

@HakeemLawrence 4 дня назад

Nice Project Darshil!

@DATA_MACHINE22 7 месяцев назад

thanks Darshil for uploading free content, appreciate your efforts

@Sufi39 13 дней назад

Good tutorial, thanks a lot. It was very helpful and informative. More thanks for reminding us to stay hydrated :)

@niranjanadevi4839 Месяц назад

Good video and great explanation!

@nazmussakibsumon4199 9 месяцев назад

Very helpful. Thank you.

@venugopal-nc3nz 10 месяцев назад

Great work Darshil . It's reallly helpful for beginners who wanted to learn Azure

@DarshilParmar 10 месяцев назад

It's my pleasure

@tarukhmakhtoh9726 7 дней назад

You are of great help brother. Thank you very much.🙏🏻

@m.abdullahmansoor4925 3 дня назад

Thank u so much bro I copy pasted all of your video content and eventually got the best project in the class. ❤️

@jackmaguina8480 9 месяцев назад

Great video about end to end project in azure

@afzaalawan Месяц назад

Young man, you deserver a kiss on your forehead for this fantastic piece of presentation in wonderful way -- May God Bless you

@user-ku1bu6yq6m 10 месяцев назад

Hi Darshil, Great demos and tutorials, really appreciate your hard work , Please keep this going you are helping many... I had a quick question. : how do you do Increamental loads in the process also how do you handle SCD type 2 etc. Talking about the projects which are used to build the data lake/delta lake, and AWS projects for datawarehouse on redshift etc. could you please shed some light on both the processes

@HealthEdify675 10 месяцев назад

You can use a dataflow in ADF for incremental loading

@DeltaXML_Ltd 9 месяцев назад

Fantastic video!

@miladkhanlou6459 9 месяцев назад

You are awesome, Thanks for the content

@jaggyjut 5 месяцев назад

Thank you. Please keep creating similar projects. Something I’m struggling find is end to end machine learning project with front end web application to interact with the ml model.