Тёмный

Power Automate - Fast Data Aggregation - Group By, Sum, Count  

DamoBird365
Подписаться 12 тыс.
Просмотров 6 тыс.
50% 1

In this video, I will show you how to group by, count and sum (add large arrays of numbers) values from a data source, such as SharePoint, using the most efficient method possible. I will use select to get distinct values, then convert the JSON data into XML. I will use select again to sum and count the values with XPath expressions. I will also demonstrate how to use xpather.com, a free online tool that helps you write and test XPath expressions. You will learn how to build a complete solution that can handle complex data aggregation scenarios in Power Automate Cloud Flow. No apply to each actions were harmed during the recording of this video. Only 7 actions and therefore 7 api calls are used in this flow to do all of the above.
[00:00:00] The purpose and benefits of data aggregation
Helps with writing, researching, and organizing data
Reduces the amount of API calls and avoids throttling
Enhances the performance and functionality of flows
[00:01:02] The example scenario and data source
A SharePoint list of employee expenses with 50 items
The goal is to get the total sum, the group by employee, and the sum and count per employee
A comparison with the built-in group by and sum features in SharePoint
[00:02:14] The steps of building the flow
Using a manual trigger and a get items action
Using a select action to get an array of employee IDs
Using a compose action with a union expression to get the distinct employee IDs
Using another compose action with an object to create a root for XML conversion
[00:05:02] The techniques of using XML and XPath
Using another compose action with an XML expression to convert the JSON into XML
Using another compose action with an XPath expression to sum up the total cost of all expenses
Using a select action with a concat expression to create dynamic XPath expressions for each employee
Using XPath expressions to get the name, email, total cost, and count of transactions for each employee
[00:08:14] The final output and verification
Using a compose action to display the final output as an array of objects
Comparing the output with the SharePoint list and the group by view
Explaining the advantages and limitations of this approach
[00:10:12] The setup and structure of the flow
The flow has a manual trigger and a get items action
The flow uses select, compose, and xpath actions to manipulate data
The flow outputs the total sum, the group by, and the individual sums and counts
[00:14:02] The use of select and compose to get distinct values
Select allows to select values from an array
Compose allows to create an object with a root and values
Union and sort expressions can be used to get distinct and sorted values
[00:17:15] The use of compose and xpath to convert JSON to XML
XML allows to use xpath expressions to query data
Compose allows to convert JSON to XML using the xml function
XML can be tested and validated using the Xpaather tool
[00:20:10] The use of xpath to perform data aggregation
Xpath allows to use expressions such as sum, count, and text
Xpath can be used to filter data by employee number
Xpath can be used to get the name, email, cost, and count for each employee
[00:24:22] The final output and review of the flow
The flow runs fast and returns the expected results
The flow uses only seven API calls and no apply to each
The flow can be modified and improved as needed
Check out my efficiency playlist for more great videos • Power Automate Efficiency
#PowerAutomate #Tutorial #demo
Please buy me a coffee www.buymeacoffee.com/DamoBird365 ☕

Наука

Опубликовано:

 

4 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 56   
@jamesclark2020
@jamesclark2020 4 месяца назад
I applied this technique today at work. What a thrill it was when it did exactly what it was supposed to do! Thank you again Damien ❤
@DamoBird365
@DamoBird365 4 месяца назад
Cheers James 😍
@ukm365
@ukm365 5 месяцев назад
It's good that you've split it out like this, Damo. I often did this with solutions in the community. Whilst we know that we can smoosh a lot of these into one expression, it doesn't provide the easy path to learn that.
@neilthackeray181
@neilthackeray181 Месяц назад
Simply AMAZING! very easy to follow what could've otherwise been a complicated process. Thank You Damo
@DamoBird365
@DamoBird365 Месяц назад
Glad you enjoyed it!
@om0729
@om0729 5 месяцев назад
I really love 💖 your videos. please keep sharing the knowledge. this is another great example where we can perform group by with in the Power Automate.👏👏👏
@cvkealey
@cvkealey 5 месяцев назад
Thank you for the reminder that I REALLY need to sit down and learn/get comfortable with xpath. I've been working with more flows that need to "post-process" data pulled in from SharePoint or Excel and the "filter array" action doesn't always do what I want.
@jamesclark2020
@jamesclark2020 5 месяцев назад
Wow. Excellent video, Damien! You made xpath seem more approachable
@emmanuelmaceda2475
@emmanuelmaceda2475 5 месяцев назад
I was just reading your old post from 2021 on this topic with a hack using length function. Excellent vid as usual. Thank you for sharing!
@wendylumsden2224
@wendylumsden2224 2 месяца назад
Thank you so much! I have learnt heaps and will apply to many of my flows which create custom reports from Dataverse tables. Greatly appreciate your generously sharing your knowledge to assist others
@DamoBird365
@DamoBird365 2 месяца назад
Thanks Wendy 👍
@thanura.m
@thanura.m 4 месяца назад
thank you for the demo. i constructed a similar usecase using a loads of loops. i should use this from now on!
@temidayolonge4566
@temidayolonge4566 5 месяцев назад
Great Video as always.
5 месяцев назад
"Old friend classic designer" FTW :) Great demo Damo again. 👍💡🥳
@DamoBird365
@DamoBird365 5 месяцев назад
Ah yes. I’m working hard to feedback my findings with PG. It will get there.
@DamoBird365
@DamoBird365 5 месяцев назад
😂😂😂
@dessyducasse3003
@dessyducasse3003 2 месяца назад
Thanks this method is what I was looking
@filipwinski4219
@filipwinski4219 5 месяцев назад
Awesome!!!
@mrZerg00s
@mrZerg00s 5 месяцев назад
Thank you! I had no idea xpath was so powerful. 😲
@DamoBird365
@DamoBird365 5 месяцев назад
Glad it was helpful!
@wjameswilliams
@wjameswilliams 5 месяцев назад
I really feel for newcomers to Power Automate who will have no idea about the classic designer but also that will have no chance at figuring their way around all the bugs in the new editor. It's difficult enough to determine the issue when you know what you're doing! That said, this video was excellent. Avoiding unnecessary loops is so important, not least because of consumption but also because it's so much faster! Scheduling overheads can add minutes to a flow because of a loop before any actions even take place, which can even be exacerbated by other Microsoft customers' use of the multi-tenant environment 😞
@DamoBird365
@DamoBird365 5 месяцев назад
It’s definitely not an easy time to jump into Power Automate with both classic and new designer on the go and new having so many quirky bugs - agreed. I’m not a personal fan of Copilot in Power Automate either with so much of the automation incomplete but it will improve no doubt. I’ll keep sharing ideas on the new designer and hopefully folk can return to these videos. The concept won’t change and hopefully the video stands the test of time. I am planning on some newbie content 🤞 Just got to find the time as this is really just a hobby, albeit one I enjoy.
@arturmusialak8704
@arturmusialak8704 5 месяцев назад
Great video! Thanks a lot 👍
@DamoBird365
@DamoBird365 5 месяцев назад
Cheers Artur 👍
@chritoni
@chritoni 4 месяца назад
Thanks! This provided a great way to handle large invoicing and accounting materials with efficiency. I´d really like to see how you would handle something that is large enough to require paging.
@DamoBird365
@DamoBird365 4 месяца назад
Paging from the data source? Or something else? I’ve got graph api paging here ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-4IphRdADJBc.htmlsi=jTbBHRjpUY68veTW
@chritoni
@chritoni 4 месяца назад
@@DamoBird365 Yes, I suppose I meant paging a source and in general I´m especially interested in handling a large number of rows in just about any setting. Did not find paging behind that link, but if you have such a video, I´ll find it eventually 🙈 Thanks for you blog and videos 👍👍👍
@DamoBird365
@DamoBird365 4 месяца назад
Oooops, wrong link apologies 👉 Boost Your Power Automate Skills with Complex Arrays, Select, XML, and Join Techniques ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-afqvGAb20Dw.html
@aaronalbers7280
@aaronalbers7280 16 дней назад
Does xpath in this context support string lookups instead of integers? For example, lets say you wanted to to use John Smith instead. "//values[Name="John Smith"]/Expense[1]/text()"
@DamoBird365
@DamoBird365 10 дней назад
It does indeed. 👍
@sankhlayashraj
@sankhlayashraj 4 месяца назад
Thank you for the lovely video Damien. Leart a lot. I had one question for you on the xpath expression - The name in the map of is the name that you have given; however in the expression, are those the internal names of the columns. Also want to know would that be same if I am trying to get the count of data from a choice column in SP.
@DamoBird365
@DamoBird365 4 месяца назад
The easiest way to find the name is to look at the output history. SharePoint might have field1,2,3 etc if you’ve created a list from an import. Good luck 👍
@sankhlayashraj
@sankhlayashraj 4 месяца назад
Thank you so much. I was able to create a flow which is working perfectly fine. Appreciate your help.
@ukm365
@ukm365 5 месяцев назад
OOOOOH ... xpather is LUSH.
@Xwodsarecool
@Xwodsarecool 3 месяца назад
I have a similar list, with a column called County and Status and I want to 1.) go through the list and count the number of items grouped by the county name and be provided totals and then 2.) I have another column called status that’s a choice column and then I want to count the number of Active, Closed, and Pending statuses also by the county and be given totals. Would your demonstration work with what I am looking to do as well?
@DamoBird365
@DamoBird365 3 месяца назад
It would certainly help you with most of your requirements 👍
@abbeyesval
@abbeyesval 4 месяца назад
thanks for this excellent video, I just have a question, how can I add number format on the xpath function in order to summarize the costs? I tried with formatNumber(Item()?['AmountLocalCurrency'],'N2') on the initial table, however the sum result that I receive is NAN, hope you can help me
@DamoBird365
@DamoBird365 4 месяца назад
I wonder if your currency is a string? You could try float() around the item()?[‘currency’]. The history will let you see if number is in quotes (a string) or not (a number).
@humayonsalim1877
@humayonsalim1877 Месяц назад
This is very useful. Can I ask how you would build the XPath expression with the sum() inside this xpath(outputs('Compose_XML), concat('(//values[EmployeeNumber=',item(),']/Email)[1]/text()'))). ?
@ma.teresatabaco5398
@ma.teresatabaco5398 Месяц назад
Hi Damo, thank you for sharing this great video. This is exactly what I'm looking for. However, I want to ask how about the calculated columns? Is it possible to use this also? I'm sorry I am new to Power Automate.
@DamoBird365
@DamoBird365 Месяц назад
In theory yes, but what you need to look at is the history of get items. Run the flow, check the history, can you see the calculated column value in output history. If yes, the logic is the same.
@ma.teresatabaco5398
@ma.teresatabaco5398 Месяц назад
@@DamoBird365 Hi Damo, yes, I can see the calculated columns now. For many days I did a lot of searching online for this. I'm happy I found your channel.
@ma.teresatabaco5398
@ma.teresatabaco5398 Месяц назад
By the way, can you help me on how to call this flow from another Sharepoint List? I have second SP List with a column for the sum (by employee) of the calculated column from first SP list. I am not sure if you have video regarding this, but I'm trying to watch all your videos one by one. If you have, please share with me here the link... Thank you so much.
@klebermedeiros8742
@klebermedeiros8742 2 месяца назад
Very good. Is it possible to get a max value, using a expression like sum?
@DamoBird365
@DamoBird365 2 месяца назад
There is a max expression learn.microsoft.com/en-us/azure/logic-apps/workflow-definition-language-functions-reference#max
@mk15minut64
@mk15minut64 5 месяцев назад
Can you make the same video in classic ui? Or maybe provide a zip copy of the flow?
@DamoBird365
@DamoBird365 5 месяцев назад
I’ve not got plans to do videos in both designer. But the video should give you everything you need.
@McIlravyInc
@McIlravyInc 5 месяцев назад
I use power query and VBA to do this, but i can see that this might be useful in conjunction with power apps to return some data for the user. Any other use cases where this technique is better than power query/bi/vba ? (VBA is no bigger lift than flows or apps IMO).
@DamoBird365
@DamoBird365 5 месяцев назад
There are definitely other products / platforms that can do this but some folk work in the low code space and have this requirement. An apply to each and a variable is not an efficient way of doing this. Office Scripts could achieve this easily I bet. But some orgs don’t have Office Scripts. Good to have options.
@McIlravyInc
@McIlravyInc 5 месяцев назад
Yes this technique has a lot of gems in it for other processes too. If they have power platform they have Excel also, and i find greater trust and acceptance with excel solutions in my area (especially because they can edit the email before sending, decide not to send one or more, etc) than just having emails come automagically (as one friend calls it). But I can see a use in a power app I'm thinking of for sure!
@DamoBird365
@DamoBird365 5 месяцев назад
@@McIlravyInc nice 👍 let me know how you get on.
@klebermedeiros8742
@klebermedeiros8742 2 месяца назад
My flow gives an error when the dynamic item has letters and numbers together. What can it be?
@klebermedeiros8742
@klebermedeiros8742 2 месяца назад
It worked when I inserting " before '.
@DamoBird365
@DamoBird365 2 месяца назад
Glad you worked it out ok 👍
@joao0790
@joao0790 Месяц назад
@@klebermedeiros8742 Hi Kleber, what do you mean by inserting "before"? I am having the same issue. The field I am using to find the xml value is a string (very long, indeed), and I noticed using the xpather, it cannot find elements. Do you have any clue?
Далее
🤘РОК или ПОП?💖
3:20:26
Просмотров 1,7 млн
Лайфхак с колой не рабочий
00:16
Просмотров 361 тыс.
КОРОЧЕ ГОВОРЯ, ШКОЛА БУДУЩЕГО
10:40
Group By in Power Apps
9:27
Просмотров 6 тыс.
Adding Agentic Layers to RAG
19:40
Просмотров 16 тыс.
ПОКУПКА ТЕЛЕФОНА С АВИТО?🤭
1:00