This channel has Big data technologies related videos for beginner to expert levels. Topics Range from cloud computing and BigData related topics such as AWS, Talend, SSIS, NoSql, Cassandra, MongoDB, Sql server, Snowflake etc to ETL and data ware housing topics. All videos will have some hands on step by step guide related to one or multiple technologies related to bigdata.
Hi Sanjay, Nice demonstration. I am facing issue, files are getting load when I am manually refreshing pipe. Seems auto ingest not working. Please suggest
Thank you for the video. If I have an on prem oracle database as the source and the frequency with which the data changes is quite high (insert, update and delete) and the current requirement from the client is near real time data insights. so do you recommend using snowpipe to ingest data from azure(adls in this case) into snowflake tables. but what should be the load mechanism or tool or technology used for data ingestion into azure from the source database? Thanks in advance :)
I am from NON IT background .. planning to move to IT INDUSTRY. which is the best technology to be choosed.. 1) Snowflakes, SQL 2) power bi, power apps, power automate,SQL 3) POWER BI, (SSIS/ ADF),SQL I am confused to those.. please guide me...
Hi Sanjay, I have a small question, I am trying to build a dwh pipeline, I got 2 json source files in on an s3 bucket, I created a pipe from s3 to my staging schema but I am not sure how do I continue from here, is there a way to create a pipline between two schemas\ tables or should I create a task + procedure to refresh every few minutes to load the data into prod? Thanks Alex
i have a scenario where two accounts are in same region but diff provider .if we are sharing tables and in consumer account we want to do dml on tables after creating database on share would it be possible?
wouldn't it be faster and more efficient to cut the lambda in the middle, and stream directly from api gateway to kinesis? or what about using a lambda to directly put record onto firehose?
Where are the previous videos? I am new to Talend and this is too advance for me. I can't find this "code" thing in the component. More than this I need to incorporate in the email's text the content of some store variable (or the results of running sql server stored procedures). I'd like a video that addresses this. Thanks.
@Sanjay Kattimani Bro Awesome explanation... 👌 Using this explanation I have created project on my resume using my current ETL project business with additional transformation mapping rules... I can now start marketing my resume for snowflake positions... Thank you so much...