In a small team, business analysts are also data engineer, architect, etc etc. Wear all the hats. Reuse is huge for me and for supporting my team as all our data comes from soloed systems via manual exports. This can take me out of the loop/future proof!
Again, thank you very much for bringing light and to dare with this poliedric issue. You can study it or draw conclusions from very different angles... As far as me I believe i will use it for our employee table. Many people try to ETL (to cook) a new employee table every month for their KPIs and for further reports in Power BI. With Datflows this table can be cooked only once and all the people who build reports from employees have a common place to take this file. Before dataflows each one had to cook the file more or less in the same way... and besides, if the boss decides to cook it in a different way then it will be inmediatelly available for the rest of the people. The advantage is just to cook just once in a higher level of the tree and not many times on the leaves!
Thanks for sharing your point of view and with a specific example, it helps me get a better understanding of use cases for dataflows and I am sure will help others! Awesome and thank you! /Ruth
First time i actually went through a white paper (directly/indirectly), thanks for this, you made an interesting video out of a white paper. you did quite an ETL :)
Nice video Ruth! Agree with your comment on Business Analyst........the BA's that I know don't want to touch anything remotely technical!!! For some reason a lot of BI tools (Power BI, Qlik) are now calling developers as business analyst....not sure why though!
Funny you say that as I have been wondering the same thing. I dislike the term developer as it somehow implies that you need to have an it background to do it ... I wonder why that is too.. /Ruth
Excelente Ruth , ya me baje tu paper de apoyo, ya lo traduje y muy buena explicacion , ahora juntar todos esos tipo de datos y hacer magia como dicen los amigos de PowerPivotPro , aparte de aprender todo este mundo facinante de excel, dax ,pq y power bi. Tambien aprendo ingles contigo , ya que aprendo palabras que no se , ejemplo tu video tuvo 4538 palabras , solo 37 no sabia , osea tengo menos del 1% de palabras que desconocia, esa es mi manera de aprender ingles ,ver videos y anotar las que no se y voy mejorando .Siempre me preguntan donde aprendiste tanto ingles escrito y es solo perseverancia con ganas gracias Ruth.
Thank you for breaking it down for us, Ruth! I WOULD like to see a video demo of setting up and consuming dataflows entities. The process appears to be pretty much identical to PBI/Excel Power Query process, but I have not been successful connecting to any data sources. I consider myself more of an analyst, but having no one else to rely on to create/manage usable datasets (i.e. data warehouses), I see a HUGE potential in data flows for my quest for a wider adoption of Power BI by my employer (who is shockingly primitive despite its size). I think it can make things less intimidating (and more self-service-y) if users don't have to feel like they must learn M or DAX. I will just have to learn how it works first. lol
I will, it is on my to-do list. I will try to put it for tomorrow’s video, but as I am travelling today, I might not make it. But you will see one for sure :) And thanks for adding your point of view, I have a hard time to see dataflows as a self-service tool for business users or analysts, but then again, I don’t think I have seen a need for it on the .org I have worked with so hearing your perspective is useful for me and others. Thanks! /Ruth
@@CurbalENmadam, It's possible to create report directly from dataflow not in power BI desktop in power BI service?? if possible please explain the steps?
Fantastic video...and yes I do not see the average Joe or Jane self-service Power Bi user evolving to PQ M (propriety) language rather than evolving to either Data factory v2 or a Databricks notebook running in SQL, Python or R. I agree 100% with you let’s see some movement on Data Catalogue rather than this slightly weird investment. 🙃
Ruth, Excellent Video - Very helpful learning on how the Dataflow is connected to its CDM Folder tables with the M expression. Also, is there a way we can restore a dataflow (with the history) in case of a disaster?
Great video Ruth, im still trying to learn about dataflows and how it can help me in my deployments. The point at 9:50 about data locked in Power BI, I thought it meant that the data you clean and analyse in Power BI can't be used as a data source for other apps like Power Apps, Azure MI, etc. NOT about the capacity of data. What about using dataflows as a way to standardise data source structures? For example Microsoft has a CDM for different sectors and I was hoping to use the standard data structure in there to create a "plug-and-play" system, where I create a standard template for a report, and then connect my clients data to these reports. Is this industry-standard? I'm not quite sure if it is.
When I try to do a "FuzzyNestedJoin" I get this error in Data Flows: "The name 'Table.FuzzyNestedJoin' wasn't recognized. Make sure it's spelled correctly" Here is what I am doing......... Table.FuzzyNestedJoin(#"Expanded Exclude", {"Account_Name"}, HPG_Roster, {"Name2"}, "HPG_Roster", JoinKind.LeftOuter, [IgnoreCase = true, IgnoreSpace = true, NumberOfMatches = 1])................ Any help would be amazing!
Hola Ruth. Espero que estés muy bien!!! ¿Es normal que al conectarme a un Flujo de Datos, no se vean datos en las Tablas en el Desktop? He creado el flujo de datos correctamente en el Servicio Power BI. He abierto Power BI desktop; lo conecto correctamente con el flujo de datos. Pero... No se ven datos en las tablas de Power BI. En Power Query (desktop) tampoco se ven datos. Al actualizar, pone: "La Tabla está vacía". Sin embargo, en el Flujo de Datos del Servicio Power BI sí puedo ver los datos... Es muy extraño... ¿no? ¡Muchas gracias!!!
Hi Ruth. Great video. Thanks a lot Trying to summarize about costs, if I understood correctly, it is something like: - using Data flows “seriously “ needs Premium license ($$$) - Data flows with Pro license ($) doesn’t make too much sense since it is limited to 10 GB and almost all new features need Premium - PowerQuery is (by now) free and it is almost same thing as Data Flows with Pro license but running locally instead of web based Am I right?
Yes, dataflows are Power query online with the difference that dataflows store the data in a datalake so it can be reused in other systems while power query stores it on power bi or Powerpivot. The only remark is that the cost is not for dataflows but for the data lake storage, otherwise bullseye 🎯, as I understand it at least! /Ruth
Hi Ruth.. thankyou for providing such beautiful explanations. One issue that i found using Data flows is that the time for refresh is huge compared to datasources directly connected to pbix file. I guess the underlying reason is compression (vertipaq vs csv files). Is there any way to compress the data in dataflow itself? Thanks in advance!
Hello Ruth ! Do you know any workaround to use excel get data from a dataflow ? I know the feature is supposed to be release this year but nothing yet and it is a really needed feature. Please help !
So if a user wanted to keep the ETL and the data in the cloud why not use Data Factory and Azure SQL/Data Warehouse? Can't see how this adds anything. Unless this is meant to be ETL light? Friendly and less intimidating for Data Analysts (not DataWarehouse/BI Developers)? What about failure logging, restarts, success and failure notifications that you would need with ETL?
I am just hesitant managing Data Flows, the referencing can build layers of complexities, also validating the data can be challenging. Bottom line, it may take away from the advantage of the 'Single Source of Truth' for developers.
Hi Ruth, great video. Thank you so much. What hardware and software did you use for the notes? Would love to use it for our Internal Training Videos...
One quick question - If I use dataflow in Power BI Desktop, will it store data or data will be available on cloud? Will the size of Power BI Desktop workbook increase?
@@CurbalEN actually, we are having crazy discussion on what to use for report to improve performance from Dataflow or Dataset. And end end with Dataflow. What you think?
Can try out Premium features using Power BI Embedded "A" SKUs: christopherfinlan.com/2018/11/18/how-to-use-a-skus-to-try-out-paginated-reports-in-power-bi-without-upfront-cost-or-long-term-commitment/comment-page-1/ Also, to handle the Dataflows version of Power Query being limited to GUI features, many people having success copying and pasting M code from Power BI into Dataflows, and having them run okay.
There are for sure advantages and disadvantages with dataflows, but it will come in handy for some users. Let’s see how it develops, it is at an early stage at the moment. /Ruth