What are other avenues getting into data engineering? If you don't have a good academic record for example? Is there any specific certification which would be helpful? Thanks
I personally love data engineering and find data science quite boring. In my experience, data engineers have better prospects. Most companies employ more data/BI analysts than they do data scientists, this is because of a constant need for descriptive analysis. Nonetheless, they're all excellent career choices.
Same here. I also love Data Engineering. Data Science was always boring to me. 80% of the time, what you are doing is descriptive analysis. Creating charts/dashboards and figuring how all the predictors are related to the dependent variable. Lastly the idea of waiting for hours just for a model to train really pisses me off and totally kills the mood I feel there are more diverse things you can do in data engineering compared to data science
@@charlesegbunike4596 I have no clue what you guys are talking about, i just find data engineering boring because the work is repetitive, I liked the aspect of data science where you can do root cause analysis, and forecasting, and figuring out which metrics to analyze to lead to greatest business growth.
I went the opposite way. Started with DS and then fell in love with data engineering and MLOps, it's probably thanks to my engineering background. I never had to work on weekends as a DE, and I've always felt recognized for my job, but I understand your reasoning and really big corporations are a bit different in that regards. Anyways, I'm glad you are doing something that you enjoy now! That's the hardest thing to achieve in this tech industry!
@@sahafhussain7448 well, we did have to learn a lot of programming in the uni though, so it wasn't that hard. I guess the most difficult thing was learning how to work in a team, using collaboration tools, the cloud, and the most important thing: good software developing practices
Hello Sundas! Me and my wife are making career changes into Data Science as well from our management positions. We're excited and nervous at the same! So happy to learn you're from Pakistan!
Thank you for being so honest Sundas! As someone working in data right now, I fully agree with your point on "wanting to be in the front end". I've had similar "analyst roles" that were more data engineering driven than the actual analysis part, and I definitely see myself more in data science than data engineering! At a time where data engineers are somewhat revered only because of total compensation, this video feels like a refreshing take
My opinion: "feature engineering" and "fetaure selection" are some of the most important activities that a data scientist must do right. Data engineering help me to understand what are the types of reasons why we have missing values, outliers, what are human errors etc. As a data scientist most of the time you blindly need to believe the data is clean and collected in the right way, which, depending on the area you are working on, will end up in misleading results and abandoned models that never survive the production pipeline. In conclusion, I like data engineering because lets me do better data science (or at least be part of the ds project).
Data Scientists are ~Consultants Data Engineers are ~Product Managers I wouldn't call Data Science as much of an engineering job but more of business consulting which is also very good. Creativity is with both but, in general, consulting is a need or requirement based creativity and engineering is more innovation based creativity!
This is not true. Product managers are quite non technical in most companies, they're in charge of the strategic road map of the product, often with a high level knowledge of what a feature is supposed to do. Data engineers are more like Software engineers
I don't think there should be a data scientist role specifically. I think a company should just have data engineers, data analysts, business analysts and machine learning engineers. A data analyst can use existing models and data for analysis, business analysts should utilize existing and new models created by machine learning engineers for forecasting and also work with real time data for predictive analytics. Machine learning engineers should focus on creating new models and implementation. Lastly, data engineers should be the bridge that supports these things to build and maintain the infrastructure. Maybe data engineering can evolve to be data ops to make more sense of the work.
Although we are in different silos of IT, your stories resonate with me. For the last 18+ years, I have been a Network Design Engineer in a pre-sales role at a major telecom. The only real growth was to change the number at the end of your title. Responsibilities really stayed the same you job got paid more. I recently decided to find a path outside this realm and just accepted a Principal Cyber Security Architect role at the same company I was at. I figured after over 20 years, I might as well find a role that I will love to finish out my career. I also have found that the title and job descriptions don't always align from company to company for most of the jobs I have looked at over the years. they also change over time due to newer technologies or needs of a company., I am very excited for Jan 1 to arrive and start this new journey. Keep putting these videos out. I have decided to self learn python because of some of your videos. I also think it will help in some of the cyber work I will be doing in the future. Thanks for all that you share!
Thanks for the video Sundas, I just want to stress on the core difference between the two roles which is basically the difference between the scientist and the engineer, Engineers creating/building things, Engineers are problem solvers move from initial point A to B, Scientist are more likely to try different unique things related to the problem trying to solve, they more likely to fail, but this is how scientists work, you don't expect as a scientist to accomplish 100% of the desired work, you don't expect everything would be perfectly fine. TO SUM UP: If you are from Engineering background make sure that you will be fine if you work as a scientist, and visa-versa.
This is solid. I have BS degree with data science concentration, yet see that landing a Data Scientist role can be quite difficult w/o a MS. My reason to finishing up a Data Engineering bootcamp and land a role is because it can still prepare you well for a Data Scientist role because you have an engineering mindset towards data, understand what complex challenges that can exist in the domain, etc. For me, I am still indecisive in regards to leveling up to a Sr. Data Engineer vs. landing a Data Scientist role later on.
Hey Sundas! Can you please make a video on what being a data scientist means at different companies? I've seen you talk about how it was a struggle for you to interview at different companies because what one company defined as a "data scientist" was completely different to what another company did! I just watched your video with Akriti and I think it would be really helpful to know what different companies emphasize on when it comes to data science interviews as well.
Hi Sundas, I'm currently working as a junior data engineer despite my degree in school is data science. My job is revolving around building BI dashboards and I can't decide what is my feeling about this job, whether I hate it or not. Thank you for your last sharing about what you love in data and data science, it made me realize some of what I have forgotten when I choose to learn data science in school.
yeah so I really agree with this, date engineering can be a little boring to do for a long period of time but it is also depends on what tools you are using in your job from python to cloud services it can be interesting but still the analytics rolls or the data scientists where you are the one in the front and you are the one who optimize the project you are the one who present the project to the stakeholders i think it depends on the person whether he likes to be in the front end or into the backend
As a DBA I’m thinking of changing my career to avoid being on call and working on weekends. So this video helped me a lot about the possibility to repeat the same afterhour work as a DE.
I love Data Engineering. I have been a Associate Director of Data and Analytics and missed the work. I love working with all the different tools and knowing I'm a integral part of the process. Being such a important part of the process is how I did get promoted to AD as well. So it is possible. That being said DS has been the sexy job in our realm for a while, especially with machine learning and AI. There is definitely more visability for a DS but you can get in front of the right people as a DE! 😉
It's subjective to the person in IT, on whether they favor data engineering or data science. For me, I'm currently an undergrad student in 2nd year studying at University. I was thinking of what kind of career path to go to, and I feel more interested with the path of a data scientist more than the others. It's because of one of the reasons that you highlighted, like the fact that you want to experiment, write a paper, and present the data to stakeholders (something you can't do as a data engineer or other engineer). Plus, I also am not that passionate of having to use some sort of engineering mindset in the tech field (seems like the software development lifecycle), where I have to define requirements based on what the customer wants, come up with the best possible design of the software (like the classes, methods, and possibly any data structures from the java libraries and/or custom built), implementing and testing the selected design, and such. Honestly, I like using programming skills more for analyzing data, preparing data to create AI/ML models (feature engineering; this seems fine, not like I'm doing something outside the box here), doing experiments, and presenting such a result to business stakeholders. I find that more interesting than the designing and testing of software (I don't hate this aspect of certain IT jobs; every computer science student, including me, starts from here). Plus, since I'm interested in the field of Artificial Intelligence, going into the data scientist career route seems like the best place to start and grow in (no need to worry about doing the software engineering part of it; unless I change my mind later down the line). I'm sure there are people who enjoy the job roles in IT where they get to design stuff, but I just don't have that desire.
Thanks alot Sundas. It's always insightful listening to you. I look forward to a mentorship from you as I am finding my way into the field of Data Science
When you say that your recognition is lost by the time leadership uses the data, does that nature of the position influence job security? Or longevity in the company?
I am currently starting out as a Data Engineer at a top firm. In the long run I want to shift to a PM role, so is it advisable to move from data engg to a Business Analyst kind of a role and then transition into PM or should I get an MBA?
Just to highlight a point made in the video here, so she said she used to work as data engineer 6-7 years back, when still all of the data fields were nascent. Nowadays, things are definitely different in data engineering from what it used to happen.
thanks Sundas for informative videos..i have a concerning question regarding choosing Data Science as non-tech background. with growing data science students would u think it will be still in demand in next 2-3 yrs or so, like if we start on this path it would take 2-3 yrs and by that time it might be outdated or there will be abundance of Data scientist? How about Videos on Data scientist as a freelancer. thanks again.
I recently received offer for Data Engineer job, I have not joined yet. Should I accept the offer and what's the main difference between Data Scientist and Data Engineer?
Data engineering is not data science but this can surely give you a path to get into data science eventually. Are you a fresher? I've been trying really hard to get a job as a data engineer or data analyst? It would be really nice if you can connect through linkedIn or email.
I've been working as a data engineer since 2020 and have come to realize that this career is not for me. As engineers are more in demand than scientists, I kind of got pushed into this career. Recently I decided to make this change.
That's sort of why I'm considering Data Engineering. I've been a physicist and astronomer previously but I know engineers are more in demand than scientists like you mentioned. Why do you say data engineering is not for you?
I need one career advice. I am Neil from India. I have done Computer Science Engineering Graduation from Tier-2 college in india. I passed out in 2018 and from then, I was doing preparation for civil services but I failed. Now, I want to get back into IT sector. My programming skills are not that good. I am trying to brush it up again. Any advice?
Hello Neil! I know a YT channel named as "DANIEL BOURKE" His video will help you out If you want to get started as Data Scientist Coursera has some great courses and also Udacity(free) I am switching to Data Science from Mechanical Engineer due to lack of job in Mecjanical domain in India These resources I find it very Useful!
Might be a bit late to this party, but starting your video about transitioning into data science with a shoutout to ask you all about data science rather than engineering which is where you came from feels quite offputting . Rest of the video was quite informative and interesting.
There are lot of people saying in 3-5 years the buzz of data science will get faded.. currently am a student doing bsc maths and simultaneously enrolled in online bsc degree program in data science and programming from a premium college.. getting a job in data science field would be difficult for me in coming 4-5 years (yes/no)?? Please answer
Im a student doing bachlors in statistics, currently im confused how to progress towards data science it feels overwhelming can you please guide me what should i do further
Not really, google stock symbol remained GOOG. The FAANG acronym was coined by an investor to represent big tech stocks. Facebook’s stock symbol would change..
thanks much sundas for this informations , i have a small question if you can help me with it please , i am doing a career shift as a fresh graduated civil engineering to data science and i was confused about which field would be better for me to become a data engineer or data analyst, which one would be easier for me to go with and would be better in the feature as someone who would start from scratch. any advice would be appreciated. thanks much
learn calculus first, all of the principal aspects (If you follow uni then calc I->II) learn basic linear algebra learn python after. learn advanced linear algebra (uni level 2) learn stats while you learn calc III && calc IIII, mostly stay in probability for the start learn data-structure theory (analysis) learn basic database technology like sql learn algorithmic theory learn algorithmic problem solving and use data structures often (leetcode helps) learn basic, basic ML theory become a data engineer while being a data engineer, learn the BI domain or whatever domain you want for a statistical background. as well, learn statistical theory, mathematical stats, design and analysis of experiments, statistical inference, testing, multivariate analysis, regression analysis, time series analysis, risk analysis, data mining, stochastic analysis, statistical visualization, storytelling, machine learning theory, machine learning inference, applications of machine learning, model development for machine learning, statistical analysis applied to machine learning, build an ML portfolio, etc
I am a computer engg on paper, I get salary 10 x lesser than a bigger earn there in your hood, I wanna get fire back, missing pal like u, wanna learn data science,... Don't know where to start, time running out, gotta make more money,
Hai,Iam interested in Data science and I want to become an Data scientist. Please make an video on learning path for Data scientist after completing my computer science in Engineering Degree Please , I Hope you will do what we are need full
Who is eligible for Data science, analyst. What degree need,Above 40 age. Eligible or not.pliz give me your suggestion. Road map to become data analyst.
I love your videos and also love u ❣️ too forever.. how can we contact chatting or call. I also want to start MS in Data engineer after 12 years of experince in digital marketing. But so many on youtube tells donot do fot it. I have done btech in CSE in 2011
Hi Sundas, I'm a data engineer in Microsoft(2 years). As of now, I'm happy that I'm getting to work on BIG Data. But I'm unsure, if I should pursue my masters in data science now or wait till I actually feel like I'm done with data engineering. Thanks in advance.
Hi Rashmika, if you are happy as data engineering then why not stick with it? If you are curious and wondering if you might like data science more, then try a few projects and see if you enjoy it. This way you’d know for sure before you commit to a degree.
Hi, I am working as a software engineer, I am trying for career transition to data engineering. Could any one suggest the better platform to learn data engineering.
I have 10 years experience as software engineer , basically development for front end and back end. What you think about shifting carrier to data engineering
@@prof.mangabhai can you share your idea about data engineer vs software engineer. bcz i m a software developer. but now i m doing my MSC data engineering. what is the best for future?
On Jesus second coming reward is non destructible body thats why i am telling Roman 13 (11 to 14 google , and no one is preaching that .People are born for car ,bike ,house ,husband wife like animals ........
Hello I'm new to this channel actually I need ur help i want to become a data scientist to but I'm not understanding how to start where to start I just passed 12th
In one video you say you left data science for data engineering in another you say the exact opposite. Are you really in any of these fields or you are just making videos.
Hello i see a lot of pm rols like vp or head of department like head of data analys, i consider what role that they have before like data engineer, data scientist or DA
That’s good to know. My experience was a few years back so the industry has evolved a bit. Though I still don’t see many head of data engineering. The SDM are topically managing DEs and SDEs
Mam Which place is better than for data science course and they give better placement after completion the course and they are excellent real industry project , they have excellent faculty plese answer me mam i am very unsatisfied for few months to not find out for better of me. plz msg to me mam