God, I absolutely hate the AWS user interface! I struggle with it so much. Not to mention the annoying log in process that Amazon uses. The course is great, but not the tool. It was extremely frustrating to me that I seriously had to quit. This is the same with Tableau.. guess we all have our own prefs! Great content and I love the Data Engineer courses!
Very much agree about Mosh's python tutorials! Such a great resource... although I think they skipped over teaching the `luke_barousse` library 😂 Thanks for the shoutout, Ben!!
Thanks for the vod. Appreciate u clearly explaining different resources for multiple aspects of breaking into the data field. I have been making an interactive data system with student/admin portals with smartsheets (kinda like excel) for a private school students to make work easier for admin duties. I did not realize this was databasing work. This channel has been a sort of gateway to exploring this field further.
Been looking for a video like that and YES I just subscribed. Thank you so much Seattle, do you recommend that we start from course 1 going down or we can just pick any random course?
@@SeattleDataGuy Haven't planned anything but starting to look more into DE projects because I'm just finished my first Data Science internship. (kind of an MLE role) I concluded that ML isn't something I would like to do in the future and I would like to develop my technical skills further.
@@SeattleDataGuy No project ideas yet, but will start as soon as I review the basics of SQL, Python and Data Modelling as you recommended in your other videos. But scraping data and transforming it to something is the only thing I know for now haha
One thing I am curious about is that how do you balance learning new things with your workload? I assume there times when you are swamped with work for months how do continue to grow your skill set?
Awesome video. I am currently trying to make a shift of industry and I have sign up for the IBM professional data engineering in Coursera. Do you think this is a good route? Or should I look to other resources?
It depends on your general background. The biggest thing that will always help is experience. But certificates can be great to show your interest to some hiring managers
good video. but just to clarify.... if we're talkin about dockers.... are you sure that DE should have "generic" docker experience? Instead of this I'll recommend something like BlueData that is more reliable. Also dockers and Kubernetes... If you DE you rather will use something that we can run in distributed mode.
That makes sense. I actually have never had to use BlueData. But thats the wild thing about working as a DE. You could work a lifetime and never even touch 1/2 the tools that exist.
Hey there, would you recommend taking these courses, following your roadmap, or data camp? Looking to break into data engineering and learning all the skills to land a job in the field. Appreciate your insight.
Awesome video, really helpful. One qq, In most of the Job description I saw, Big data (and related tech) is basic requirement for DE. what's your thought on this? If someone has to pick between AWS certification and Big data one, which one should be priority?
In general you will need to have some understanding of big data. But what that means is so broad. I wouldn't worry too much if you're just getting started in your DE career. Learn about some distributed systems, but more from a high level.
I suspect Boris Paskhaver’s course at Udemy is the best one out there for absolute beginners to novice level programmers. Possibly for intermediate level too.
Data engineering vs python backend developer - what specialists are paid more? Why would python developers choose data engineering when on average they can be paid better as backend developers?
Hello! Thank you for your invaluable video! I find it extremely useful for beginners! I would like to ask about one thing regarding this map. I learnt Pandas in terms of Data Wrangling and Transformation. Therefore, how about Pandas for Data Engineers? Is it useful tool for ETL/ELT transformations? Obviously, the next step will be PySpark, but I would like to start learninig Pandas. It seems it is a good path for the next one. What do you think about it ? I would appreciate it if you could share your views about it.
So I'm still in my college so I will be glad ,if all you professionals once look into this I'm currently learning hadoop and spark and I'm confident in my java programming ,and after watching your videos I was planning to start learning SQL as well.So should I choose Mysql or postgreSQL or anyother and so finally this how my to-do list looks like Learn SQL Learn Scala Learn Hadoop Learn Spark Learn ETL(Extract,Transform,Load) .By the way I'm just a beginner in python ,so do you think i can start my carrer as data engineer with these and should I focus learning more on scala or python or just leave java as it is.
Hey, I am actually aiming for intern roles in Data Engineer domain. What skills should I have in my resume, ideally. I am a graduate student in CS and have knowledge in DBMS, SQL and Python. Can you please guide me?
Yes those skills are good. But also if you can get some ETL/ELT as well as at least a high level understanding of distributed systems that would also be good.
@@SeattleDataGuy This is what I'm struggling for. How can I learn about ETL and ELT and Datawarehouse? There is no course or tutorials on Udemy and RU-vid, and I feel like the only way is to get a job in that field but it requires the already-there experience to get the job!
What do you think about : Udemy - data engineering essentials using python sql and pyspark Then - Udemy - 50 hour of big data pyspark - aws , scala and scrapping
Is the Google cloud engineer cert for beginners with no prior knowledge of anything cloud related? I thought there were a few certs before that one. Love the channel btw your my go to guy for all data queries haha
I probably have some bias since I learned Java first. But overall I have personally seen more need for scala since a lot of people use it for Flink and Spark.
I have a question, I what about R programming language? Would it be beneficial to learn and what purpose does it serve in the Data Engineering world? I'm very new to this and trying to find some guidance on what language would be good to learn. I'm really interested in R though. For more clarification I want to go into Machine Learning and A.I. I would appreciate your insight
I don't think R is very helpful for DE roles. If you want to do data science and research sure. But implementation of a lot of models isn't often in R, sometimes not even python. So it depends on if you're interested in research, ML engineering, etc.
I want to pursue a career in DE and thanks to your videos I now have more clarity in what I need to do. I've got one question, would you recommend doing these courses whilst following your roadmap or afterwards?
It will depend on the company. Some companies expect you to know Dataframes others don't. For example, Facebook would prefer you have strong SQL skills.
I have strong SQL skills, which I used as part of my software QA career. I also know some C++, Java, and C#. I am definitely interested in data engimeering. What kept me away from data was the math and statistics part.
Thats awesome! DE isn't too much math. If it were data science, then yes. But you can probably just add some data warehousing and data pipelines and you're pretty much there
Hello, amazing content!!! Have you got any resources or courses for data engineering, ETLs, so on and so forth for JAVA developers not only for Python???? Thanks in advance
I think there are a lot of great options. I reference leetcode in this video ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-9vRfTZdbjtM.html. But also, I do find that there are some online study programs that have literally...somehow..gotten real questions or slightly modified questions from tech companies so if you split your time between leetcode, interviewquery and strata scratch, you're likely going to be good.
I'm so lost, i have basically no idea of programming and the skill sets i need to basically be able to work in a company like exxon mobil (which i eventually plan to get into) i have no programming background nothing, those videos are being very helpful thank you
Hello! Great video. I'm currently working as a data analyst, doing some data visualization. I want to do a fully transition to become Data Engineer... I have some knowledge in python(ML,pandas,numpy), SQL, No-SQL, Data visualization.... Please! Do you have any advise on which path to take to become a DE from my currently knowledge?
Gitflow (a strong knowlege of git and control of versions) , devops and/or MLops, Unix commands for automatization , jenkins (for testing and deploy automatization) and dockers, this will open the gates in a lot of companies, knowlege in a cloud technology will help too
1. I heard that to transition into Data Engineering from Analytics, I should also learn SDLC, CI/CD, and DevOps, do you have any recommendations where to learn those? 2. Is Alan Simons Udemy Data Modeling course good enough for me to learn normalization and stuff if I don’t take the SQL server/PostgreSQL Udemy course you recommended? I took Alan Simons Udemy Data Warehouse course you recommended and I love it.
1. I would say learn the terminology for SDLC and maybe hold off on CI/CD and devops, unless you're already a mid-level engineer. If you're early on enough you won't be expected to do as much of this. I don't have a specific course as I learned most of this on the job or in a few courses at school. I will have to check for courses 2. For normalization I would say check out this video ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-l5DCnCzDb8g.html. If you feel confident on this subject then focus on practicing building db models.
I was keen to learn . This is a great informative video. Then to my dismay I realised I earnt 6 figures in 1985 just with a degree in economics. Shows how the employers have turned the screws.
Can a beginner get job as a Data Engineer....? I have seen many openings for senior level but not for entry level . So learning these skills worth the time??
Hi I am working as a software engineer having 4.5 years of experience looking for career transition into data engineering. Is it possible to make career in that technology. Suggest me any good platform to learn to make my self ready to attend the interviews. Thanks
I'm an Automaton QA Engineer, who mostly done automation for front end. I want to transition to Data Engineer do you have any advise on which path to take ?
@@SeattleDataGuy I'm in a similar position, QA automation engineer with 6 months of experience in backend trading systems.. And i do have knowledge of python and SQL... could you suggest how to make a transition into Data Engineering?
gosh this is overwhelming, there's just so much to learn and not enough hours in a day to do so! perhaps DE isn't for me, perhaps a more humble DBA role is better suited to me.
It probably depends where you are and what companies you want to work for. I find that a lot of hadoop is being abstracted away or managed by infra teams. Then DEs just write SQL queries via other engines. Azure is great for large companies that use microsoft.
@@SeattleDataGuy currently I am working on Azure technology stack so I am thinking to focus more on Azure data engineering. Ok generally to become a data engineer what is the technology stack we have to learn ? Because on Azure we have data factory, synapse etc I don't have idea on how general data engineer technology stack look like Do you have any suggestions ?
@@SeattleDataGuy my Data Engineering Playlist would become like a goldmine. And collab with @Luke is dead crazy 🤪. Also seen the podcast with Shahshank amazing 👏🏼spot on💯 Also could you provide some reference or link for the Felipe Hoffa you mentioned in the video, as I couldn't find them in the description 😄
