Will Data Engineering Exist In 5 Years - Is Data Engineering A Good Career Choice? 

THIIIISSSS! Yes...I should do a whole live just on this topic. I remember being a student, yeah it felt like everything was solved and everyone was doing ML...I promise most companies still don't do ML now. They just write articles about how they are doing it to stay relevant.

@@SeattleDataGuy demand in EU is high and AI doesn't have scaling capabilities or problem solving the human data engineer has

Absolutely 😄

That's for existing projects. But for fresh ones where AI or AI-assisted engineers would be responsible of developing the entire infrastructure, development will be conducted having a strong forsight and following good engineering approaches.

@@parlor3115This will take years. Implementing such an infrastructure is crazy expensive. Until it can be offered as a service up and down and across industries, we’re good 👌

Yes you should

@@rouenyu Yes, I noticed this as well. It seems, there are plenty roles for dedicated frontend developers and in those roles frontend, design skills, knowledge of APIs and browser side scripting with js are what’s expected and, honestly, that’s no surprise. Backend developers, as of late, are expected to know the full stack and be able to contribute on every layer (backend, architecture, cloud, storage, frontend, CI/CD, getting coffee and so on). Just a few years ago, I was able to look for a plain old backend developer role and I was set with the regular old java skills for backend. The industry evolves so fast and, it seems, you must with it while always be learning. My advice would be to set short term and long term goals: what I did was study frontend (react/vue) for 6 months until I can build full stack project. In that time I also studied cloud and architecture. The next 6 months I studied databases (Relational and Non Relational). I’m currently on miscellaneous technologies like Kafka, Flink, and streaming technologies). Luckily I got plenty of experience on my first job with CI/CD with Jenkins and such. Also, keep in mind, if you’re < 5 years in like myself, it just takes time to get familiar with ALL these technologies. As of late, there just seems to be more requirements on software engineers and an explosion of technologies. I don’t see the pace slowing down anytime soon. If not, then just ignore what I said haha. You’re probably extremely proficient in the technologies you’ve been using and could use that as leverage when you learn fullstack. You’ll crush it man!

Hii brother I'm interested in data engineering will I get job at this role at first place and what do they expect in skill sets?? Currently I'm doing my intern where I'm learning to build ML models

I think this is fair. I am thinking of how I might change the thumbnail. Whats kind of annoying is that usually my CTR is like 6% and this video is 13%

@@SeattleDataGuy I personally don’t think the title is that bad. You have to make a title that pops for the largest audience.

@@roncrudup2110 Did you see the old thumbnail? I just updated it...before it was like "It's Over For Data Engineers" with Fire. I think the Plain Bagel did an analysis and found those types of thumbnails got an average of like 3-4% more CTR which is massive.

@@SeattleDataGuy I think important thing is to create your channel community with robust, real subscribers. Not the ones coming with click bait titles. That’s what I think. Still I appreciate your videos and thank you for your effort 👌🏼

I clicked through when I saw the new thumbnail title. I think if I had seen the original "clickbait" title, I probably wouldn't have bothered checking it out.

Hmm, I have mixed feelings about masters in data science and computer science because i assume its better to just do..but i don't have a masters so maybe someone else would find it useful

yes!

I don’t know the market in Vietnamese, but I couldn’t imagine the competition being any less fierce than the competition in the US. I asked my mentor this question, his advice was to: 1. BUILD projects 2. Contribute to open source 3. Become more visible (blogging about popular technologies that many people use and will need help tutorial help with, blog about niche technologies you know that don’t have much resources, or blog about your own product and challenges you’ve faced, share those challenges because others might run into them). I wish you all the best, my friend!

There's too much supply of python DEs, what companies lack are framework/enterprise skilled DEs. Like example, they want to find people that excel in Talend, DataStage, Teradata, Informatica, Snowflake, to name a few. To land a job, use actual tools, not just python. For example, I only knew mysql back then, but landed an SSIS/SSAS/SSRS job. And with that, I landed an informatica and snowflake job. Now I get frequent calls for tools like DataStage and Talend which I don't use, but since I already have exp in other frameworks, they got no choice.

Hu đọc được cmt của a em cũng thấy hơi hoang mang vì tình trạng tìm việc của DE fresher ở việt nam , a cho em chút lời khuyên được không ạ

I am! You can look up Seattle Data Guy!

I think it'll change, I like what joe reis sad recently in his blog where he talked about DEs having to be more aware of data science and software so we can provide data for AI, data science, analytics, etc

entry level = data analyst. Microsoft Certified: Power BI Data Analyst Associate is the best course.

@@gavinkalaher7314 thanks

data engineer.

Backend developer is slowly becoming data engineer. Or is it other way ‘round?

@@idkwhy77why ? Any specific reason ?

😀

Because the business who creates that tech will become very rich 😅

@@SeattleDataGuy indeed, but at the expense of taking away people's career 😭

@@collinsnnadozie5092 google "luddite"

@@Zuranthus Are you the developer of AI Chat GPT ?

@@collinsnnadozie5092 no, i'm just your average day rational individual that understands advancements will be made and you will find some other job to do. or do you propose we all go back to tilling the land and raising barns? cause you know, that would make sure there's plenty of work to go around for everyone

Have you seen this video? ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-VtnQv-CN_NE.html

Love your name. 💚

@@SeattleDataGuy Thank you. You don’t discuss why you left in that video though…

@@WisomofHal Thanks. Someone called me aggressively NPC.

@@SaveThatMoney411 Was there any reasoning?

Hmm. Good point, but have you thought of other alternatives. If AI can do the work, it’ll be more productive and produce more value (more money). Why reduce the salary of the human if more value is being produced? Like you said, human still needs to monitor: if no human is willing to monitor then what’s the use of AI that can have many mistakes? Therefore human can, still, demand very good pay. It will still take deep knowledge to know what to look for, mistakes, corrections, additions and so on. We reduce human labor. Also, human labor has always been added to any work ticket, the cost of that labor. If we are making labor cheap, this can also make other aspects of life cheap while producing more output. As computation power and storage becomes less expensive. I believe the overall goal is to give people more time to do other things, things we actually enjoy, we spend so much time at work. Wouldn’t you want to be with family, reading a book, skiing, or playing music? Basically, it’s why we wanted slaves in the first place, only exchanged for AI. Now we’re talking ethics… it’s a complicated chat, my friend. The idea of AI is still young; imagine a world with AI powered robots who can build without rest, who can clean without breaks, who can drive you from A to B, who can give you all the answers, cool your meal, read your kid a book. Would the cost of new construction be more or less expensive and who’s incurring the cost if we humans are not working? If robots can construct non-stop, where will all the material come from. Say construction becomes so cheap, countries decide to build homes for everyone. Is that feasible? Now, we’re talking policy. If we go another route, your route, and even highly paid jobs are now paying nickels and dimes while less paid jobs have already been automated to AI/Robots and we’re still expected to pay these enormous bills then how do we pay the bills? Or will government step in and develop more policies. It’s a very sticky situation. Now, what is the value of money? If money is no longer the god, then what becomes god for sheep? AI, a new god, or do people go back to spend more time with their one true God? Just something to think about. In my opinion, I don’t see salaries going down in our lifetime.

@@WisomofHal That cannot work with capitalist system... And is polar opposite of current capitalist system

@@MeLLiFLuOuSViDeOs That’s my point exactly 🙂 Don’t you think AI will relate a world like this? A world where we do less, if anything at all? Where monetary value is significantly less important or not important at all. The assumption is AI will automate and takeover most, if not all, jobs. Wouldn’t the next logical step be that government will morph into something completely different and have to establish more and more policies? I don’t know. There seems to be multiple ways this whole AI thing can go. I guess, we’ll just have to wait and see. Why would I have a human build my deck when government begins to produce machines that can do it perfectly in 5% of the time it takes a human to. What are the ramifications of government becoming more powerful? Why is it that, in capitalist societies, the idea of socialism and socialist policies are appealing to more people, how is that leaders that promote these policies in the most capitalist societies are being elected as leaders and what’s the effect on society? Why have countries in the West promoted, and some implementing, less working days (4 day weeks, 10 hour work days) and will 10 hrs a day slowly decrease back to 8? Officially reducing productivity? Or are we more productive because of AI and automation?

​@WisomofHal your market salary isn't really determined by your added value, but by the factors of supply & demand on the labor market. It is profitable to hire you if and only if your market salary is lower than your added value, as with all products. Added value is, therefore, more like a salary ceiling. It can be the case that new technology enables you to be more productive and simultaneously reduces the skill floor of anyone wanting to do your job. In that case your market value actually goes down even though your added value goes up. We associate added value and productivity with higher salaries, because often if a machine tips over 3 out of 4 workers, the remaining 1/4 has to do less manual work and needs to work at a higher abstraction level. THAT translates into more salary, not the added value in itself.

I don't know if that's true. In many cases, as we automate work, it increased productivity which increased salaries. If AI acts as a tool, then I imagine it will be the same. Of course the ICs will see a decent jump in their pay but the biggest winner is always the corporations.