Thank you so much for your kind words! 🙏 I'm happy to hear that you find the videos easy to understand. Enjoy exploring the rest of the series, and feel free to share any questions or feedback you might have. Happy learning! 😊📊👨🏫 Mastering Microsoft Fabric 40+ Videos: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-p-v0I5S-ybs.html
Thanks for sharing. By loading the data through DFG2, the table names are appearing in proper case but if we do the same process through Apache Spark, the table names will always be in lower case? Any reference point ....
You have create tables in fabric and use those as folders. Like we created a table with name measure with one column and no data and used that as measure folder.
I have no data on cloud,what ever i am using that is on sql server ,and have created gateway connection for this,so when i am creating dataflow gen 2 through sql server ,using that on premise gateway getting this error " The selected connection has an on-premises data gateway that is different than the project's gateway. Committing will apply to all queries in this mashup" what to do next?
Please check this video in the series. ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-oEF-jHVmvdo.html Always use latest on-premise gateway. If Dataflow uses already existing connection and not working, Then use new connection and update the details again.
Hello Amit, I am a new member to your channel as I was following the same steps of loading csv file through WEB API options but its fails stating "Files contains corrupted data" can you please update the file. Thanks
I tried this file again, and it worked for me. However, some users in the past have also complained that the Git file is not accessible, so giving file at Drobox too: github.com/amitchandakpbi/powerbi/raw/main/Sales%20Data%20for%20Fabric.xlsx Please download this file from Dropbox, and you can upload it to Fabric Lakehouse or OneDrive to use it. www.dropbox.com/s/z9d6qfah6451ct3/retailData_fabric.csv?dl=0 www.dropbox.com/scl/fi/ru5jusnq4c89bsbhganm0/Sales-Data-for-Fabric.xlsx?rlkey=e62al8qyleagpfm4ryggtwkvx&dl=0
Yes, We can do an end-to-end flow on the Warehouse too. Dataflow Gen2, Data Pipeline sqn SQL can write to Warehouse. As of now, notebooks do not write directly to Warehouse. I have covered the flow in the series. Initially, the Warehouse destination was not available to I used an alternative. But now you can use warehouse destination Mastering Microsoft Fabric 35+ Videos: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-p-v0I5S-ybs.html
Thanks for the video I used dataflow to add data to a data lake from web api, which worked fine, but then I wanted to switch from "New table" to "Existing table" to add data every day by refreshing the dataflow, but when I go to edit or recrate destination - it doesnt see the table in the data lake that was created by initial refresh :(
Please refer Microsoft Fabric: Various Components| Microsoft Fabric Create Workspace, Lakehouse, Warehouse- ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pVvvb-MzxKw.html Microsoft Fabric-What it is, Why Use, Who Should, When and How to use ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-sbWNJMaspeA.html
I was following this video till 23:03. Created a lake house but accidentally clicked on "Publish Later". Its been over 12 hours and I do not see the tables I added to the Lakehouse. These are the tables from my On Prem SQL server and are fairly large in size. Now I have a couple of questions: 1. What advantage I will have of keep these on the Lakehouse v Keeping them on SQL Server VM? 2. Can I also add Views from SQL server to Lake house and they will perform process data from their respective tables in the Lakehouse? 3. To me its seems like Fabric provides an alternative to everything I used to do with SSIS, SQL server, SQL job agents etc. So if my current system is working why do I need to shift to fabric?
On the three dots near to Dataflow gen2 name click and check Refresh history , if refresh is not running of failed. manually run it 1. What advantage I will have of keeping these on the Lakehouse v Keeping them on the SQL Server VM? Lakehouse is on the cloud and saves data in Delta parquet format, also it Read only SQL end point, you can use to query data in SQL. Power BI dataset do not copy data, so only one copy 2. Can I also add Views from SQL server to Lake house and they will perform process data from their respective tables in the Lakehouse? If you want to import yes. If you want a shortcut, no as of now 3. To me its seems like Fabric provides an alternative to everything I used to do with SSIS, SQL server, SQL job agents etc. So if my current system is working why do I need to shift to fabric It is a complete analytics system. It can replace many existing systems like SSIS, SQL server, ADLS gen2 etc
Create an empty table. And move you measures there and hide the column of the table. For Lakehouse use Spark SQL for Warehouse you can Use SQL endpoint(Warehouse) Create table Measure (column1 varchar(5))
In power Bi fabric I am able to create a measures, but not able to create new column as we create in powerbi desktop. Please explain how to create a column in power bi fabric
Thanks, nice video. How to create a scheduled refresh? When I create a scheduled refresh on Dataflow gen 2, it creates new table every time it refreshs. Basically what I want is, import data via Dataflow Gen2 into Lakehouse, transform it (on SQL endpoint, notebook python or Power Query as you done) and then use this data set on PBI. Everything is okay but can't manage the refresh data. When I refresh dataflow gen2 it creates new table which power bi and transform process is not connected to this new table.
Refer if these two incremental approaches, if these can help Microsoft Fabric Part 15: Incremental Data Load(ETL) for Warehouse using Dataflow Gen 2, SQL Procedure, Data Pipeline: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-mpFRnZBXsvQ.html How about incremental without Dataflow, Only using pipeline and SQL procedure for Microsoft Fabric Warehouse Microsoft Fabric Part 16: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-qsOIfTzjCSQ.html
@@AmitChandak Thanks for super quick replay, But I'm a bit confused. What I want to do is refresh table. Why should I go and deal with SQL? Is it not possible just refresh table in Data Flow Gen 2? When I run Dataflow Gen 2, it creates new table because Dataflow Gen 2 that we created is getting data from a file. I think we need to update data via Dataflow Gen2, not get data. Can we not do this without SQL?
hello, I have added data into lakehouse through Gen2 from sharepoint and then want to model the tables, however for two tables, I need many to many relationship, but there is no option for this, how can i achieve this in fabric? Also if I connect the model to Power Bi desktop and want to change the relationship to many to many it would be converted to import locale mode, however i want it in direct/live connection, your inputs please thanks
Please create a custom semantic model; you can have it there. But be careful, as many-to-many relationships sometimes do not work as expected. I have already shared this feedback with support team.
@@AmitChandak thanks for reply actually we are migrating old semantic models to fabric, yes we will try to be careful with many to many relationship as do we in past. Thanks amit
Thank you Sir for making this video. However, I am getting error while at video 32:50 when I add implicit and explicit measure like Gross, Discount in visual in Power BI Web. It is showing error like "Can't display the visual - couldn't load the data for this visual. Please try again later or contact support" Could you please advise on this?
Nice video. Is it possible to write data PARTITIONED through Dataflow gen2 into a Lakehouse? I know I can do it with a notebook after data is ingested from source. But what if I want to read data from source through DF Gen2 and write it into the Lakehouse partitioned at the same time?
Amit, really appreciate your way of teaching... I am not able to find perfect video on Git integration in power bi, version control and multiple users working on same power bi report, how to merge and manage conflicts can you please make video on it.
Warehouse I will load tomorrow. For difference refer Microsoft Fabric: Various Components| Microsoft Fabric Create Workspace, Lakehouse, Warehouse- ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-pVvvb-MzxKw.html Microsoft Fabric-What it is, Why Use, Who Should, When and How to use ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-sbWNJMaspeA.html
Hi Amit. Great Job. I have one doubt I am getting below error, can you help me with this? Error Details: Error: Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Microsoft SQL: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server) Details: DataSourceKind = Lakehouse;DataSourcePath = Lakehouse;Message = A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server);ErrorCode = -2146232060;Number = 53;Class = 20 GatewayObjectId: 6309202d-8f81-4235-9651-68924de52acf. Request ID: 490256f4-86ec-4c9b-a71f-83515aa6f59f. (Request ID: c5b3731b-2a50-4ee7-a227-03045b44d390).
Is it Azure SQL or SQL server? If this on-premise SQL server Microsoft Fabric Part 17: Load local SQL server data in Lakehouse using on-premise Gateway and Dataflow Gen2: ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-oEF-jHVmvdo.html For last few days this one was not working (Using on-premise gateways)
@@AmitChandak Thank you so much, I am using an on premise gateway, and as you said its not working for last few days. I ll go through the video, Please do make more videos.
