This is my RU-vid channel where I explain various topics on Data analytics,Data Engineer, Azure Data engineering ,AWS data engineering , aws glue, aws athena, aws s3, aws emr, Cloud Platform ,AWS, Big data ,ETL, machine learning, deep learning, devops, Linux, GCP,big data engineer, data engineer, and AI with many real-world problem scenarios. My main aim is to make everyone familiar by showing them practical example with real case scenario .Please subscribe and support the channel. As i love new technology, all these videos are free and I promise to make more interesting content as we go ahead.
it is the best video I have seen so far, thanks a lot. I have a question please, when you created the stored procedure linked service you used the sql server name, could you please mention from where can i get it?
Hello Sir, As I have zero knowledge about pyspark,in this series what ever you are teaching,is it enough or should I study your other py spark module videos. Please suggest
Employe_department use case 1)upload emp and dept csv files in adls 2)check if the files are empty or not. If it is not empty process as below. Otherwise, display it as empty file and stop the pipeline. 3)Get all employee details from research and operations department and save the result in azure sql db. 4)get the details of managers belonging to sales and increment the salary of those employees you 5% and update it. Save the result as JSON file in ADLS Please help me to solve this Thanks in Advance
Please Help me Out through this ERROR Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'MANASAGADDI\SQLEXPRESS01', Database: 'sales', User: 'pizza'. Check the linked service configuration is correct, and make sure the SQL Database firewall allows the integration runtime to access. A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.) The certificate chain was issued by an authority that is not trusted
i think always inferschema is not a good way to do in production because sometime some of the column will be wrong so better we can create our own schema .while creating the dataframe set our schema in schema parameter
Please make a video how to connect from SQL server configuration (permission, certificate authentication, reaming )and adf with linked service configuration because all are getting errors while test the connection in linked service
I am getting error while sql server to adf to build a linked service in test connection check """"Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'MADHU-VALLEPU\madhu', Database: 'AdventureWorksLT2017', User: 'mrk'. Check the linked service configuration is correct, and make sure the SQL Database firewall allows the integration runtime to access. A connection was successfully established with the server, but then an error occurred during the login process. (provider: SSL Provider, error: 0 - The certificate chain was issued by an authority that is not trusted.) The certificate chain was issued by an authority that is not trusted""" Please make a video on this or give solution for this it's helpful for all
Thank you so much for your great effort, very helpful contents,,,Just one question, in really time scenario normally we are automating data loading activities so in that case how we can get help from parametrized table selection. In that case single pipeline will work or we need to create multiple pipelines for loading multiple tables?
bhai aapko idea hai kya macbook mei hum sql server docker ki help se chalate right, then sql server ko azure data factory se kaise connect karenge .. coz woh chl e docker ke through hai
Hi bro, Thanks for the video. I created df using simple json which you mentioned in the video, it is working. When I tried to create df with complex nested jason file, I am not getting expected output. Can you please create a video for complex json df example
Hi Bro when I tried to df, I used both df.show() and display(df). I can see data for df.show() but for display(df) i cant not see data, it showing datatype of each column DataFrame[id: string, zipcode: string, type: string, city: string, state: string, population: string]