Top 5 Data Engineering Tools currently in the Job Market

Подписаться 29 тыс.

Просмотров 25 тыс.

50% 1

This video talks about best tools to work on for Data Engineers according to current job market. This will be super helpful while preparing for interviews or looking at market trend since best tools are evaluated using a Python code that takes data from LinkedIn jobs.
#dataengineering #datascience #technology #cloud
Vera by Board Infinity for AI based mock interviews: bit.ly/3Z5MzSp
For 1:1 mentoring sessions with me:
topmate.io/jash_radia
Follow me on Social Media for more frequent updates:
LinkedIn: / jashradia
Instagram: / jash.radia
0:00 Intro and Overview
1:55 Code Walkthrough
4:30 Result Sneak Peek
5:04 Number 5
7:35 Number 4
8:28 Number 3
9:56 Number 2
11:25 AI Interview Tool
12:18 Number 1
14:11 Honorable Mention
Code: github.com/jashradia/youtube_...

Опубликовано:

29 июн 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 107

@JashRadia 9 месяцев назад

Hope you enjoyed the video! Share your thoughts here and share it with people who might find it handy ♥ AI Interview tool, Vera by Board Infinity: bit.ly/3Z5MzSp

@vinayak6685 9 месяцев назад

Can you please make a similar code to extract the keywords used in job description which we can add in CV, so our CV doesn't get discarded in the automatic filter

@narasimharaodavuluri5321 9 месяцев назад

May i know why Azure data engineer tools were not added kn the search

@seekers_life 9 месяцев назад

Can we expect more such examples in coming days as it will really help people like me, who are learning data engineering as it provides real life example to learn data engineering.

@rshane1598 9 месяцев назад

Very nice job! Thanks for sharing.

@sirajansari2848 9 месяцев назад

Hardly I come across such videos where OP puts so much of their efforts. This is one of the most useful videos I saw today. Thanks for your hard work.!! :)

@JashRadia 9 месяцев назад

Thanks a lot for the shoutout! 😊

@seekers_life 9 месяцев назад

Very well stated, thanks for your hardwork, it is really helpful, thanks a lot.🥳🥳

@BillionDollerDream 10 дней назад

You never fell to amaze me...Thanks Jash

@saimahendrapatnaik1477 2 месяца назад

Excellent way of approach

@aishwaryaradia 9 месяцев назад

I know the hardwork that you've put in this video... Proud of you Brother 👏🏼🎉

@JashRadia 9 месяцев назад

Thank you! 😊

@sumedhpatil1474 9 месяцев назад

Great explanation.

@VivekDesai 9 месяцев назад

This is genius. No one can make such quality content, I have found a gem! Kudos!

@JashRadia 9 месяцев назад

Thank you! 😎😁

@anishdasgupta8949 9 месяцев назад

Very helpful video, have been working with my company's DEs to figure out a DE framework for data handling, have been using your videos for references.

@JashRadia 9 месяцев назад

Glad to know! Thanks a lot for watching 😊

@banking8675 9 месяцев назад

Great effort brother thanks for this

@JashRadia 9 месяцев назад

Thank you for watching! 😊

@Faz-Ahmed 9 месяцев назад

This is amazing, i wish you all the success in the world, for your efforts

@JashRadia 9 месяцев назад

Thank you for noticing! 😁

@guruprasadashridharhegde6792 3 месяца назад

Such an Amazing informative video till date. Thanks for sharing your valuable insights. Keep us updated..

@JashRadia 2 месяца назад

Thank you 😃

@artofheart2891 9 месяцев назад

Great Going Dear. Keep it up.. 👍🏻👍🏻👍🏻

@JashRadia 9 месяцев назад

Thank you! ♥️😊

@sathyamanikantabk4483 9 месяцев назад

Amazing and good information

@JashRadia 9 месяцев назад

Thanks!

@pramodreddy8151 9 месяцев назад

Content ❤

@pmmodi3583 9 месяцев назад

LOVE YOU BHAI❤

@JashRadia 9 месяцев назад

Love you too, Modi ji ♥️😅

@ritupornasengupta1391 5 месяцев назад

This is genuinely a very useful video and the work you have put to gather the stats is really recognized . I had one question -Are the stats that is pulled up is only for 'Canada' location or is it the complete list irrespective of the location ? While i get a reply on this would like to - Thank you for your videos I hope you keep making them and educating us .

@JashRadia Месяц назад

Glad you liked it! It is location agnostic

@aayushkumar561 9 месяцев назад

Please make a video to explain the setup and code you used to do the web scraping.

@JashRadia 9 месяцев назад

After installing packages mentioned in the code, you can run it in any Python environment.

@vinayak6685 9 месяцев назад

Can you please also share this code to us so, as the time changes we can run it ourselves and see the current top 5

@JashRadia 9 месяцев назад

It's already there in description

@prof.mangabhai 9 месяцев назад

hello jash , my company has provided me with the tech stack as aws glue, snowflake and pyspark , is this the best tech stack (or the one which is in demand) , I am a fresher who has just completed my btech , I want to switch jobs within a year and a half, so do I need to get familiar with top 7 tools you mentioned or just sticking up to my current tech stack is well enough, becuase you said redshift,bigquery and snowflake falls under same category ,so would I be eligible for jobs where they demand some other tool like reshift or bigquery but I will have only experience in snowflake

@JashRadia 9 месяцев назад

That is a pretty good tech stack. You'll learn a lot and it will be useful down the line, too. Good job!

@NagpurJobPortal 10 дней назад

I run the program however came across a problem how to display the data as you have shown to see insights

@babluprasad813 9 месяцев назад

Hey Jash can you share how long you prepared and how did you prepare for a job switch? I am trying to job switch but i am confused where to start preparation. Also where do i find what to prepare for interviews for data engineer and science roles.

@JashRadia 9 месяцев назад

If you're looking for a job switch then my roadmap video will be useful. For preparation, I prepared for a couple of months while working simultaneously.

@neerajrajput9977 9 месяцев назад

sir i am following your videos since couple of months,i love your hard work.apart from this i have a personal question.i have done my bsc(information technology) got passed in 2019 after that i prepeared for civil services exam for 4 year now i am thinking to study masters(data science) in germany .is this wise decision ?because study gap is very long so very difficult here to get a job

@JashRadia 9 месяцев назад

Thank you and call me Jash 😊 If you're doing masters then this gap wouldn't be a problem at all. As long as you can justify why you took it. Because after masters you will be tested on the skills you learned in uni. Yes, they would consider you a fresher but gap won't be a problem in most cases.

@NagpurJobPortal 10 дней назад

I run the program however got a problem on creating insights like u did

@RahulPal1998 9 месяцев назад

Hi jash, I have completed python and sql , and basically pandas in python , having project done on web scraping , so what you suggest that can I now start with pyspark or anything else for data engineering.

@JashRadia 9 месяцев назад

Yes, I think it is now good to start with pyspark and eventually you can learn real time data processing too down the line.

@ninaddeshpande4427 9 месяцев назад

Can you also make a video on how to look for jobs in data, if someone is transitioning from Mechanical Engineering field, like how to create a good resume for data, which ceertifications will be helpful ?

@JashRadia 9 месяцев назад

I'll try to address these topics thanks for suggestion

@Fresh_Career_Compass 5 месяцев назад

Is god tire road map for data science available?

@saurabhgavande6728 7 месяцев назад

how can we change the location from usa to india

@navaneethneeth6094 9 месяцев назад

Hi jash, Ho are you! In my company we have thing called academy by which we can move to technical domain, I chose Kafka and cleared its pre-assessment for further training. Will it be a good helping step or advantage for me by proceeding and working in that track(Kafka) to get into data engineering which is my ultimate goal? Or this kafka work experience won't be any added advantage for me to DE job? Please share your opinion, So I can decide to proceed or not with Kafka training :)

@JashRadia 9 месяцев назад

Kafka training will definitely helpful but it will only help you with real time ot near real time analysis. Many DE projects are batch analysis as well. Eventually you will have to learn that, too. But Kafka is a good skill to in DE with. All the best!

@mayurpatil8297 5 дней назад

Can you prepare save video for 2024?

@Pavan_Prabhu 9 месяцев назад

hi jash ,what about snowplow? if i am giving a chance which one should i pick snowflake or snowplow

@JashRadia 9 месяцев назад

It is very similar to snowflake but I think it's too new to make it into this list. Even if it is better, it would take time to adopt

@ramakrishnavanga1335 9 месяцев назад

In topmate please make price less na then it can reach then every one we are less financial

@MC-vw5le 9 месяцев назад

Hi Jash, to apply for intermediate level DE position at FAANG, do we need to be able to solve Dynamic Programming and Backtracking problems on Leetcode? Thanks!

@JashRadia 9 месяцев назад

Not required

@MC-vw5le 9 месяцев назад

@@JashRadia Thanks for getting back to me Jash! I assume linked lists, heap, queue and BST (DFS/BFS) are included?

@prakhargupta4347 9 месяцев назад

How to decide between DE and SWE(backend), I am in 2nd year. I have dipped my hands into both and both seem feasible to me. I have to finalise and pick one thing eventually. How should I go about it. Please guide.

@JashRadia 9 месяцев назад

Do you want to build apps or do you want to dig deeper into that app's data? What would you like to do more? There is no wrong option here and both roles are good. It depends on what do you like to do more.

@prakhargupta4347 9 месяцев назад

@@JashRadia I will have to spend some more time figuring out

@sivaprasad-ji4fi 9 месяцев назад

Hai jash tell me how u transfer to ai team do u need to learn ai and ml before shifting or after entering u learnt bcz it helps us when we also shift we do same as u did

@JashRadia 9 месяцев назад

Yes you need to learn ML concepts and it would also help if you can show it with a certification like GCP PMLE

@prasadpg6189 9 месяцев назад

Thanks for sharing these information DO YOU HAVE ANY PLANS TO START GCP DATA ENGINEERING COURSE please let us know?

@JashRadia 9 месяцев назад

Thanks for watching. And no, not anytime soon

@Nick-du9ss 9 месяцев назад

Hii jash good to see you back Visualisation tool is required for data engineering or not And i want to become azure data engineer so it is a good move for future If yes then which skills demands in market for azure data engineer

@JashRadia 9 месяцев назад

Visualization tool experience is not a must for most roles but It's good to have. I would always prefer being a cloud agnostic data engineer. You can start with Azure if you really want to but I don't see a reason why you shouldn't explore other cloud platforms after some experience.

@Nick-du9ss 9 месяцев назад

@@JashRadia i will explore other cloud but for right now I want to get entry in data engineering

@krish0804 9 месяцев назад

Great content.. Keep doing more such videos. .

@SushantShambharkar 9 месяцев назад

What do you think about future of Flink?

@JashRadia 9 месяцев назад

I have very little experience on Flink so I'm not sure I would be the right person to answer this. But I can say that I have never yet come across a project that requires Flink.

@amitpatel9670 9 месяцев назад

I am new to DE. I dont understand why spark is not there.

@JashRadia 9 месяцев назад

As mentioned in the video. It's not a tool... Databricks, EMR, Dataproc etc are tools that run Spark.

@ankitsharma8702 9 месяцев назад

Is learning a cloud necessary for data engineer as a fresher Or if I would learning a cloud what cloud should i learn as a fresher azure data engineering certification is good

@JashRadia 9 месяцев назад

Yes learning cloud is a must. You can learn any of GCP, AWS or Azure. The knowledge is easily transferrable and they all have distinct features as well.

@moyeenshaikh4378 9 месяцев назад

Sir i have heard kafka if we don't get real time project then its better not to learn that as it's hard

@JashRadia 9 месяцев назад

That is true. I myself didn't put a lot of time in learning Kafka like tools until I was working in a real time project but there are many roles out there that hire for this specific type of projects so learning it won't go to waste. I don't think it is hard though.

@mirirshadali33 8 месяцев назад

How did you transitioned from Data engineer to AI engineer? What are the steps to be followed if one want to transition to AI engineer from Data engineer?

@JashRadia 8 месяцев назад

I have made a separate video about this topic on this channel

@sivaprasad-ji4fi 9 месяцев назад

How long will it take to transfer to USA location as a ai/ml engineer what is the way please tell me

@JashRadia 9 месяцев назад

There is no definitive answer to this. Depends on company, projects and your skillset. Can take anywhere between 1 to 5 years if you really want to go.

@Onelife3468 9 месяцев назад

AWS vs azure which have more job and demand in market

@JashRadia 9 месяцев назад

AWS is the market leader and GCP is the fastest growing platform

@shivaramthallapally369 8 месяцев назад

Why can't you post data engineer full course on your channel.. It will be very helpful if you post.. Your way explanation is too good

@JashRadia 8 месяцев назад

Thanks for the suggestion. I usually don't have the bandwidth to do this but if I do, I'll definitely consider this!

@bgmiinsider3379 9 месяцев назад

Azure Data factory not even in top 26 ? Strange.

@JashRadia 9 месяцев назад

Yes that's what I thought, too. It is one of the main reasons people choose Azure 🤔

@pranavganesh1855 9 месяцев назад

Do ADF uses Apache airflow??

@JashRadia 9 месяцев назад

No that's a different tool. It does what airflow does and some more like even transformations on its own.

@sivaprasad-ji4fi 9 месяцев назад

Hai jash this is siva i want to know how you prepare for dsa and please hell me resources

@JashRadia 9 месяцев назад

There are only 2 resources needed for DSA. Geeksforgeeks for understanding concepts and leetcode for practice and looking at frequent questions.

@sivaprasad-ji4fi 9 месяцев назад

@@JashRadiathanks jash in interview what is the level of difficulty in DSA they asked And how you changed u from data engineer to ai engineer 1.u need to learn the course before joining the ai team or after joining ai team u learning 2.how long will it take to transfer to USA location please reply

@Ghost-of-uchihaaa 9 месяцев назад

Why ADF didn’t make it to the list?

@JashRadia 9 месяцев назад

No idea, surprising 🤷

@Fresh_Career_Compass 5 месяцев назад

Is God tire road map for data science available?😊

@JashRadia 2 месяца назад

Not yet, but may be soon 😁

@jhonsen9842 7 месяцев назад

Surprise where is Spark?

@Nick-du9ss 9 месяцев назад

So we can learn mongodb over Casandra

@JashRadia 9 месяцев назад

Both of their usecase are completely different. One is used for transactional but Cassandra is used as a columnar storage. More comparable to BigTable and DynamoDB. But anyway use of Cassandra is niche and s not an alternative to mongodb.

@Nick-du9ss 9 месяцев назад

@@JashRadia these top 7 tools you mentioned it is trending tech stack means if we learn these skills chances of getting job high Or what ?