Data Architects or Engineers, beginner or expert, there are several key skills and areas of knowledge that we should focus on to succeed in our field. We should have a solid understanding of database design principles, including data modeling, normalization, indexing, and query optimization to design efficient, scalable, and well-organized databases. Understanding of data warehousing and ETL (Extract, Transform, Load) tools is essential and use these tools to efficiently move data from source systems to target systems.
Big data technologies such as Hadoop, Spark, and NoSQL databases are becoming increasingly important in the world of data engineering. We should be familiar with these technologies and understand how they can be used to store, process, and analyze large volumes of data. Data architects and engineers often need to write code to automate processes or manipulate data. This is a Data Engineer's journey from Zero to Hero. Ignore Imperfections!
Hi, I'm a Tableau Consultant with 9 years of experience, could you please suggest a cloud service which gives me value to my profile? I didn't have any cloud technology in my bucket so far.
Very Helpful video. Requesting a video regarding importance of Group Surrogate key & Surrogate Key. How to decide where shall we apply it while warehouse implementation. Facing Lot of confusion while integration of similar schema from various source systems.
@@dataanalyticstalks Example : Customer ID can be a surrogate key where Orders can have multiple items, and each item can belong to a specific product group. To uniquely identify each item in each order, we might need a composite key consisting of Order ID and Product ID also called a Group Surrogate key.
@@Eldo_thomasbaby In OLAP there is no question of master details, all we do is convert the transaction to dimensions and facts. Of course, in fact table we can use surrogate key to identify each record as unique. Again, this is a non-business key most of the cases.
Please create a comprehensive series for certification based on DMBOK2. Organize it chapter by chapter to assist those aiming to pass the exam and enhance their understanding.
Thanks for the video. Sir, if possible please make a complete data engineering project video of ETL from data injection, to ETL and if possible orchestration. With some real practical example.
Thanks for the video i have seen no one talking abouut these important topic on you tube., if they do i don't know them. I learned about these data management authority in a book "Fundamental of Data Engineering"
Sir i have seen your channel you have guided about AZURE cloud. I am learning about AWS. One thing i want to ask you can you make a guided video playlist about how we can move our on-premises data e.g. database to AWS.
I have not much experience with AWS, moving data to any of the cloud services should not be an issue these days. There will be native migration tools available. Azure Data Factory comes with SSIS runtime feature which can easily move on-premises data to cloud.