How Data Engineering Works

Подписаться 52 тыс.

Просмотров 421 тыс.

50% 1

So, the sole purpose of data engineering is to take data from the source and save it to make it available for analysis. Sounds simple, but it’s the matter of the system that works under the hood.
Watch our video to find out more about data engineering:
00:00 Basic Data Infrastructure
02:07 ETL Pipeline
04:34 Data Warehouse
06:07 Data scientists and data engineers
08:05 Big Data
09:45 Data streaming
11:47 Distributed Computing
Sources:
[1] sre.google/sre-book/eliminati...
[2] www.statista.com/statistics/4...
[3] www.forbes.com/sites/bernardm...
To learn more check our articles:
1) What is Data Engineer: Role Description, Responsibilities, Skills, and Background www.altexsoft.com/blog/what-i...
2) Explaining the Data Pipeline, Data Warehouse, and Data Engineer Role www.altexsoft.com/blog/datasc...
3) Big Data Engineer: www.altexsoft.com/blog/big-da...
4) What is ETL Developer www.altexsoft.com/blog/datasc...
Learn more about AltexSoft: www.altexsoft.com/
Follow us on LinkedIn: / altexsoft
Follow us on Facebook: / altexsoft
Follow us on Twitter: / altexsoft
Follow us on Instagram: / altexsoftcom
Music in video by Kevin MacLeod (incompetech.filmmusic.io): Sneaky Snitch, Sheep May Safely Graze - BWV 208, Wholesome, Disco Sting, Krampuss Workshop, Thinking Music, Whiskey on the Mississippi, Hidden Agenda, Egmont Overture, Sheep May Safely Graze, Holiday Weasel, Brittle Rille, March of the spoons, Hidden Agenda, Fuzzball Parade, Gothamlicious, Light Sting.
License incompetech.filmmusic.io/stan...

Наука

Опубликовано:

26 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 554

@DistortedV12 3 года назад

This video deserves an Emmy award

@jesska.parker 3 года назад

Second that

@TheInternetFan 3 года назад

agreed

@Weallgoingtodiesomeday 2 года назад

I approve

@musicaansen5449 2 года назад

Seriously true...!!

@karthikanand1403 2 года назад

absolutely!

@siddharthsingh5031 3 года назад

I am a Data Engineer and now I know what I do :D

@codingwombat 3 года назад

You.. Whuuuuuuuuuuut?.. XD

@nandhithanathan432 2 года назад

What should someone learn or know before entering into Data Engineering job.In what platform should they prepare for that ??

@avnishdeo5165 2 года назад

me too

@tektauron 2 года назад

LOL XD

@abcrx32j 2 года назад

Only non programmers would be surprised by this comment

@Krousedp 2 года назад

After 8 years of trying to explain to my parents what I do, this does it in 14 minutes. Thanks!

@MrBoomBoom225 2 года назад

I can imagine how the initial conversation went with your parents “No dad I’m not help desk, I do things to move data”

@malcorub 2 года назад

My parents are super old school Mexican in their 80s. All they know is "I work with computers" 🤣 It's super hard to explain when they don't even know how to use smart phones or computers.

@victoriwuoha3081 2 года назад

Lol.... I can imagine.

@nitronesstalk4783 2 года назад

It’s frustrating job? does it pay well? Pls answer back, adding more details will even help.

@diwasyt 2 года назад

@@MrBoomBoom225 Dad: That's what I'm saying. You move data from one desk to another desk😂😂

@krishnacism 3 года назад

I have never seen such a beautiful video on Data Engineering topic. It covered everything what a 400 pages book can't do. Appreciate your efforts

@rqn2274 Год назад

I'm working as a data engineer for 15 years and this is the best explanation of my work I have ever seen :)

@rajageetha1936 11 месяцев назад

Hi sir iam new to this field... Can you explain me your project

@itsorcus649 10 месяцев назад

@@rajageetha1936 lmao how can he explain all that in one comment

@sreeyeshb 2 года назад

This video is the single malt of all youtube videos explaining data engineering. This deserves an award for sure.

@gmr409 8 месяцев назад

One of the best technical explanation videos I’ve ever seen in 25 years in the business. Bravo, well done.

@AltexSoft 8 месяцев назад

Thanks =)

@livyaharikachowdary644 Год назад

I am a Data Engineer and I cant tell you how much sense this video makes. Felt like we, Data Engineers got validation for the work we do! Next time some asks what do you do, I am going to send them this video link. Thank you :) Also special call for creativity and pun

@moverecursus1337 8 месяцев назад

On January of this year (2023) after I took a time off university I watched this video, loved so much and started studying this field with everyday from morning to night watching, reading, coding and adventuring on cloud. And I can say, after 9 Months, I got my first job on tech as Data Engineer. Thank You AltexSoft ❣ Without this I would be nothing 🙏

@AltexSoft 8 месяцев назад

Thanks for sharing. We are glad that our video helped you in your journey

@user-fk9zm4mj5b 8 месяцев назад

First of all, congratulations for your steady obsession which led you to your job.🎉🎉 Can you give us the rough roadmap of what you did during those precious 9 months ? Again, congratulations ❤

@moverecursus1337 7 месяцев назад

@@user-fk9zm4mj5bIt's a long story, I recommend taking a look at Data Engineering Zoomcamp, it will start January 15th and it talks about everything a Data Engineer need (free, on RU-vid)

@julesmules0901 2 года назад

I think I just fell in love with that video. It was truly mind-blowing. Not only did you guys manage to illustrate the different relationships between data systems and data discipline on a hight level but the way you visualised it made it so easy to comprehend that even people new to the subject would get it. Amazing! With topics like Data Engineering the true difficulty lies in simplifying things. And just just nailed that

@mahmoudfadaly8074 2 месяца назад

This might be the best video on the internet explaining data engineer role

@music4u98 3 года назад

These videos are like movies. Well scripted, picturized and to the point . Loved watching them thoroughly. I like the background music change as the topic of content changes

@everenjohn 3 года назад

Hi AltexSoft! I comment here not because I watched this video, but just to thank you for the amazing content! You've helped me a lot in understanding many things about software engineering in a much interesting manner compared to lecturers. I'm glad I found this channel regardless of the number of views because you making amazing content!

@goldenmilktea4992 3 года назад

Now I have a better understanding of data engineering! Thanks for the video!

@future_teknokrat7585 2 года назад

Video so clean and clear, I had to view it several times. I normally have to do that for stuff I don't quite catch. I understood everything here on the first go and just have to see it again and again. This was the perfect combination of audio and visual intake with just enough jargon.

@generativeresearch Год назад

This team who made this video truly deserves an award.

@sakinasaifee3118 3 года назад

One of the best videos I came across, with great explanation and animation. Enjoyed watching it! Thanks!

@shivnandanigarg9315 2 года назад

This is exactly how, lectures should be. This is ideal. Nothing can be enough when I want to appreciate it. Thanks

@priteshraka6408 3 года назад

Cant believe you cleared so many doubts at once in 15 min video! This was amazing work.

@shatandv 3 года назад

Amazing. A really good high-level explanation of Data Engineering!

@jacobjulag-ay5639 3 года назад

This is an amazing video, summarized everything I have been researching in just one video. Thank you!

@cidrisonly 2 года назад

In the middle of the night when you fetch your phone to make sure your decision on the Data Engineering boot camp is right, such videos are very reassuring. I don’t know how much I should appreciate this channel for this video in particular. You have earned a sincere Subscriber now.

@whymeiy 7 месяцев назад

The most concise, accessible, and beautiful video I've ever seen describe my job. Captured why data engineering matters, took us through a history of how pipelines have evolved over time, and illustrated the complexity we deal with without actually bogging down the viewer with said complexity. Absolutely wonderful! Thank you for this masterpiece! 💜

@ariesykes1432 2 года назад

Really interesting video! It definitely relates to what I am working on currently. I work in IT recruiting. I'm always looking for a data engineers for my jobs and this video helps me understand a little more about what data engineers have to face in their field. Thank you!

@swathik3117 3 года назад

I was searching for this kinda info almost since 3weeks, was so confused about the Big data DE tools how they fit into the system. Million thanks to you! :)

@caonhantran3531 2 года назад

The most meaningful explanation about Data Engineering I've ever seen.

@Eric-fv8ft 2 года назад

Amazing explanation!! Now I know what's the difference and the relationship between all these terms and professions! Thank you so much for the content producer!

@JuanCoolDad Год назад

I'm trying to transition from Data Analyst to Data Engineer. I already understand some of the concepts, but this video took it to the next level! Awesome content! 👌

@enesdogan1801 Год назад

Any news?

@enesdogan1801 Год назад

Any news?

@sonicbroom8522 2 года назад

This is honestly lightyears ahead of other videos on what DE is. Giant Kudos to you

@bernardogalvao85 Год назад

This video is amazing. I finally understand the place Spark and Kafka have. The more I learn about Data Engineering the more fascinated I get. Your video definitely contributed to that!

@user-vk6kk9wt9m Год назад

Great video! Very easy to understand explanations yet goes into enough depth for the general audience to understand the scope and complexity of data engineering. The graphics are great too, further helps with understanding the concept. Great work, thank you!

@ginnycheah5493 2 года назад

Concise and perfect! Soothing voice too! This video really deserves an award.

@terran008 9 месяцев назад

This video is amazing, so informative, I don’t remember the last time I watch such a short video with such valuable and clear content on data

@vaibhavgadewar5391 2 года назад

Never seen such impactful short video on the Data Engineering concepts which clearly show the role of each component and how they get added in progression with evolving concept.. Masterpiece.. Hats off to you guys for this simplicity 👏

@shubhamdandekar20 Год назад

I have never seen any video explain Data Engineering this good. Very easy to understand.

@calmrelaxationmusic8856 2 года назад

I have seen multiple videos on DE on youtube but this one sums up everything so well! All of you new folks need to check out this video. This deserves more traction.

@hormengyoong2054 Год назад

This is the best video that I have watched covering all aspects of data engineering in a simple, funny and engaging manner. It is a great work 👏 👍 👌

@jithinjawahar3744 2 года назад

So simple yet so profoundly explained. Thanks for your efforts

@Seeker_Winterfell Год назад

After watching this video , I feel like I have more experience in Data engineering now. Thank you

@TehUnbreakable 2 года назад

Such a great video ! Clear explanation with a good storytelling. Thank you!

@dravennow8948 2 года назад

great video! really appreciate your work. the visualization looks pretty clear. can you tell us what's software are you using for the visualization?

@robinalvarezp 2 года назад

This is the most clear video about Data and Analytics than I ever seen. Congratulations AltexSoft.

@bnee4313 2 года назад

In such a short time, video touches on every aspect of Data engineering. Great job

@supriyodatta3116 2 года назад

A wonderful way to explain data engineering! Provides a complete overview! Wow!

@denidota6393 Год назад

I am an ETL Developer, the more I learn about data, the more I feel that my knowledges very less . Thank you for amazing videos !

@syed_hussaini 2 года назад

I didn't know how data scientists & data engineers work together & also the usage of data lake and data warehouse before watching this video. A big thanks from your new subscriber.

@zma314125 2 года назад

Thank you guys for explaining Data Engineering in a way that I actually understood it.

@ShaileshKumar-lx6ov 2 года назад

The best video about Data Engineering ever. Kudos to the team who developed this.

@SamrudhVNayak 2 года назад

This video is just wow!! Simplified but captures the entire essence of Data Engineering.

@ihateorangecat 2 года назад

this video is Such a GOLD❤❤❤ no tutorial videos explained like this widely & beautifully. now everything makes sence to me about my study. no more watching 'day in life as a data scientist' videos hoping to grasp some knowledge. i ma start working now. Thanks ❤❤❤

@KarenLeos91 3 года назад

Im in love with this video, thank you for the thorough explanation

@aishwaryadhaigude3460 9 дней назад

You got my subscription just after this one video! Such rich content!

@himanshupandey4038 Год назад

This is one of the best videos on Data Engineering I was confused with all the other resources just using big buzz words and not explaining the core of the job Thank you for the simple explanation

@AltexSoft Год назад

Glad it helped you ♥

@satabdisatapathy433 3 года назад

This is all that I needed and was Searching for since a week ...thanks a ton... I usually never comment.. but this was a life saver!

@danielfulep6847 22 дня назад

If I've watched this yesterday morning, I would have nailed my trainee job interview in the afternoon. :D Amazing video! Thanks!

@AltexSoft 19 дней назад

Good luck in your job search

@farazqadir324 Год назад

probably the best video on the internet about data specific roles and their respective operations

@DarshilParmar 2 года назад

Very well explained

@divyaaggarwal7824 Год назад

What a lovely way to explain the concept of Data Engineering ! Kudos to the team 👏

@ossamahasni7739 2 года назад

This is absolutely one of the best videos I have seen in my life. PERFECTION

@CuriousDragonite Год назад

Thank You, Thank You, Thank You! Never seen such a perfectly crafted video talking about Data Engineering.

@AltexSoft Год назад

Thanks for your kind words, glad u liked our video

@mosesojonuba416 5 месяцев назад

Have been reading a lot about data Engineering but couldn't understand and place together all the concepts and terminologies.. But after watching this video animation and explanation, I can boldly say that i now understand the concept of data engineering. Thanks you

@ganapatihegde2320 2 года назад

Absolute beauty! Sheer brilliance to cover this in quarter of an hour.

@thibaultvialar2346 Год назад

This video is amazing! Simple and yet so precise. Thank you so much for your great work!

@RoryBarrett 2 года назад

This video has explained this incredibly well. I'm sharing this with my entire team.

@durgaprasada226 2 года назад

This is awesome. I hope all good things happen to you. I'm starting on big data and now i have better idea of what's going one.

@shivakumarcm9369 Год назад

Ive never seen a video that explains Data Engineering in such simpler manner. Thank you!

@AltexSoft Год назад

Thank you so much for watching and for your words ❤️

@ellouzesami5600 2 года назад

great vid. helped massively in preparation for an interview. keep up the good work!

@RZing 4 месяца назад

By far the best video to describe what data engineer is in a simple way,thank you!

@edsonsabino 2 года назад

The video is so instructive and fun at the same time

@ashishnikam3650 2 года назад

Best video about data engineering on RU-vid.

@amadousidibe2931 10 месяцев назад

Best ever , simple and clear

@lifedatatech6917 Год назад

I am super glad I found this video; I burst into tears with joy after having several aha moments while watching the video. Kudos to the video creators!

@deneb6139 3 года назад

Who ever design/produce/ideate this video deserves a Nobel Prize.

@tinarajan9886 Год назад

This is the most brilliant explanation I have watched of data engineering. The most worthwhile 15 mins of my life today 😅

@AltexSoft Год назад

Thanks, we'll continue to do our best ❤️

@muhammadawon8164 2 года назад

Thank you solving the mystery box, now I can just refer this video to the people who'd ask me what I'm doing. Great video 🙌

@sole008 2 года назад

wow best video so far about this topic and i spend the day looking at articles and videos of this subject

@daniellange7190 2 года назад

This video is so well produced and informative! Thanks!

@anikulapoidris8860 4 месяца назад

One of the best video I’ve ever watched on Data topics

@meowy2k 2 года назад

this is THE BEST DE summary i've seen so far

@ambhat3953 Год назад

One of the very few videos i have watched fully before going into comments section.

@jacquessteyn78 2 года назад

Wow - the absolute best explanation I've seen. Thank you for sharing!!

@user-yf3nw5vs5s 10 дней назад

You really made it so easy to understand! Huge thanks!

@alejandrogomez8766 2 года назад

Wow! this is the best way I have found to understand data engineering. Thank you so much.

@mohamedkhalaf6791 4 месяца назад

i don't think there is a better explanation than that. i am thinking about learning it and it made me love it.

@elyelena1002 2 года назад

This video is incredible. thank you for making it. Perfect explanation.

@abstract_machine 2 года назад

Hands down best video that explains this

@arigato1901 4 месяца назад

This is probably the best video I have seen to simply describe Data engineering! 👍

@devarapallivamsi7064 3 месяца назад

First thing that I have done as soon as I completed watching this video is subscribing so as to not miss out on such wonderful content. I think that's enough to explain it all.

@AltexSoft 3 месяца назад

♥♥♥