Which One Is Data Engineering?

Подписаться 15 тыс.

Просмотров 2,3 тыс.

50% 1

⏯RELATED VIDEOS⏯
Data Roles: • Don't Pick the Wrong D...
In this video, we're looking into comment wars on if data engineering is all about pipelines, or data modeling. Do you need to know the business side? Or just how to code? After that, we talk about pipelines vs data modeling, and what data engineers actually do. And what you need to know to advance your career.
________________________________________________
Data Podcast ►► open.spotify.c...
Website ►► www.nullquerie...
________________________________________________
For business inquiries please contact nullQueries@gmail.com

Опубликовано:

11 сен 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 11

@andynelson2340 Год назад

I definitely turned my nose up at SQL initially after learning python. I'm reformed now. I feel like SQL should be the lingua franca of data transformation/analytics/reporting. Whether you pipe your data around using scala, python, or java. Do all transformations and last mile with SQL.

@nullQueries Год назад

agree 100%

@brainsniffer Год назад

I’ve seen a few cases of people using ORMs getting bit because their tooling does bad sql. 99% of the time it’s fine, but those tools don’t always get indexing and set-based workloads that can dramatically improve performance and cost. At the same time, SQL can’t do a lot of what python does without complexity, because stored procedures just aren’t as powerful as methods, and a programmatic/library structure is much more manageable. It’s helpful to know enough about all the tools in your chain that you can see their strengths and weaknesses and know how to solve problems with them.

@dtae7855 Год назад

Love your content! The information is so concise & to the point. THANK YOU! New subscriber.

@smudgepost Год назад

As an economics, finance, Linux nerd with delusions of MLOps, your videos really help confirm and focus the data pipeline and the tools and people at play. Thanks!

@A_View_From_The_Shire 11 месяцев назад

I’m relatively new to the data world and just landed my first job, and this is something I’ve been thinking about a lot recently. I think more so because I want to make sure I’m learning the right skills and have longevity in this type of role. There seems to be a lot of divide between those who view Data Engineering as a subset or specialisation of Software Engineering and those who view Data Engineering as technical role, but one that is independent of SWE. As a newbie, I don’t really understand this need to put down Data Engineering as somehow inferior to SWE. Is it because SWE is a technically harder role? Maybe. I don’t see this same attitude towards system admin or Network Engineers roles.