Тёмный

Faster Firestore via Data Aggregation 

Fireship
Подписаться 3,2 млн
Просмотров 61 тыс.
50% 1

Retrieve data from Firestore in a way that is faster and more cost effective with data aggregation. In this episode, we use Firebase Cloud Functions to read data from a sub-collection and write it to its parent document. angularfirebase.com/lessons/f...
Firestore: firebase.google.com/docs/fire...
NoSQL Aggregation: www.thoughtworks.com/insights...

Наука

Опубликовано:

 

15 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 68   
@davidheisnam743
@davidheisnam743 5 лет назад
So if there are 1000 comments and a new comment is added, the cloud function is going to query all 1000 comments. That is 'disturbing' for me. I wish they had a built-in collection size property.
@zuzelstein
@zuzelstein 5 лет назад
This should be top comment. Absolutely not production ready app. This is not cost effective to query all comments each time.
@RobertKing
@RobertKing 5 лет назад
I suggest putting a "limit 100" on your query. that way it's production ready. Then in the user interface, if you get back 100, just show "100+". If you're getting more than 100 comments, you have other issues such as spam detection you need to take care of.
@b5a5m5
@b5a5m5 5 лет назад
I may be mistaken for I haven't gone in depth in the docs but, is the "querySnapshot" not a reference to the documents that match the query from the indexes? Meaning the query isn't actually incurring a charge for reading the documents until forEach is executed?
@ScrubsFrance
@ScrubsFrance 5 лет назад
BAM5 size contains the number of documents returned firebase.google.com/docs/reference/android/com/google/firebase/firestore/QuerySnapshot.html#size() But it contains all the documents even before the foreach :( Have to limit the number of documents expected directly in the query request ;)
@otheraw5659
@otheraw5659 5 лет назад
You can have the trick in the firebase channel. I forgot which videos, but I think it is one of the newest video about pricing. Basically you can just use math, save savedCommentsCount + 1 or something like that, u don't have to re-read all the doc in the collection
@pandarzzz
@pandarzzz 6 лет назад
Thank you for sharing this informative video! 🐋🖐🏾 I find this video very useful. I hope you are well.
@johanchouquet2872
@johanchouquet2872 5 лет назад
Thanks a lot Jeff! Could you prepare a video where you have to do Transaction in Cloud Functions for aggregation of a counter for example in order to prevent contention on multiple accesses on data ? It's a really common use case, and there's few docs about this on the web!
@niyongaboeric
@niyongaboeric 6 лет назад
Special Angular Men Thank you.
@Fireship
@Fireship 6 лет назад
Thank you!
@POSITIVO509
@POSITIVO509 6 лет назад
Amazing
@artursponchiado7265
@artursponchiado7265 5 лет назад
Good video! But I was thinking, does the firestore don't have a limit of 1 transaction/update per second? Even if I wrap all that code in a transaction, it will be limited by this 1sec update limit, right?
@drantunes
@drantunes 6 лет назад
Use a limit(5) query in comments collection seems more simple, or no?
@saadabbasi2063
@saadabbasi2063 5 лет назад
Hey Diago, Good approach, but correct me if i am wrong. Till this point that "we need to isolate reviews/comments in to a separate model" we are on the same channel, right? If we query by limit on othher document then n of population queries will be made to reviews/comments collection (here n is number of posts) and reffing in mongodb is not thhat performant as Joins in SQL. Is that still a good approach, if you could share your thoughts
@tntg5
@tntg5 4 года назад
I have a similar feature but I don't nest comments under the parent collection. Instead, I have a separate collection at the root that is called comments, and have a property called parentId. That is because a comment can be attached to a post or event or some other object. Then when I call a post, I also make another call to comments with the post id, order desc, and limit to 5. But if I needed to display the number of comments it would be different
@TimeoutMegagameplays
@TimeoutMegagameplays 5 лет назад
1:48 What would be ideal for something that would get as many reads as writes?
@RageBasterd
@RageBasterd 6 лет назад
excellent content as always my feed is full of firebase today, with all this dev summit and waht not. to bad most of it is for native mobile
@Fireship
@Fireship 6 лет назад
Thank you! Bunch of great content right now, hard to keep up.
@basdfgwe
@basdfgwe 5 лет назад
Instead of reading all documents each time can you simply have a cloud function that increments a counter, and decrements based on a deletion.
@im.thatoneguy
@im.thatoneguy 5 лет назад
There is a danger of a race condition. If you have a user add and then a user delete smultaneously you could have a condition where they both start from the same document count and attempt to modify at once. Say we start at 100 posts -1 Delete. 99 posts. Now the Add function is also running in the background and it read the 100 posts before updating and your post count is now "101" instead of 100. It's still possible for the number to be off with parallel operators but it'll never be more off than the current number of active functions. If you have 9 people all try to delete simultaneously you might go from 100 -1 (9x times) and end up at 99 instead of 91 comments. Now when someone adds a document even if it was 99, it'll still recount and end back up at 92.
@basdfgwe
@basdfgwe 5 лет назад
@@im.thatoneguy hi Gavin, what if you did it with a transaction ?
@eric-id6bk
@eric-id6bk 3 года назад
@@basdfgwe This is a bit late, however, this wouldn't work. Transactions are not cross-user. You'd be best using a queue for the actions (increment, decrement) which invokes a cloud function and performs that action on that document. That way you'd only be writing to that content one at a time.
@dionricky
@dionricky 3 года назад
For anyone looking for answer to this, maybe you can just increment/decrement the counter based on the action without even reading all the comments doc. But for certain interval, update the counter to match the size of comments in the firestore. This way is more "cheap" but requires you to set up another service for scheduling the counter "cache" refresh job. Edit: race condition on post-comment case is totally fine. But if you need something more reliable, for ticket reservation for example, probably use SQL instead of NoSQL db.
@basdfgwe
@basdfgwe 3 года назад
@@dionricky hi Don, I didn't think there was a way of getting a count like what we have in SQL.
@JeffrinJ
@JeffrinJ 6 лет назад
What Operation system are you using?
@boon_keng
@boon_keng 5 лет назад
Firestore support createdAt ordering?
@KosratDAhmad-ti4qy
@KosratDAhmad-ti4qy 6 лет назад
event.params doesn't exist from DocumentSnapshot??
@murthy900
@murthy900 6 лет назад
how to add new subcollection from angular ?
@niyongaboeric
@niyongaboeric 6 лет назад
Hi Jeff, I may you advise me the level of typescript I must have to start developing SPA. Thanks
@primeraposicion
@primeraposicion 6 лет назад
Thank you { while (true) { n * 'so' } } much!! :D
@krisnaldoHD
@krisnaldoHD 5 лет назад
Is this not as insufficient as reading all of the comments and then counting the number of objects? Or is it different when you're doing this on the server side?
@sulaiman2606
@sulaiman2606 6 лет назад
Great work😍 Can you please provide us with the structure of this application
@Fireship
@Fireship 6 лет назад
Thank you! Follow the link in the description to get the full code on github.
@erperejildo
@erperejildo 5 лет назад
but where is that index.js located?
@or.o.s.t8190
@or.o.s.t8190 3 года назад
How does he write in batches (not letter by letter)? I assume it's done in editing but could there be an extension for vscode/some other magic in play?
@crowdozer3592
@crowdozer3592 3 года назад
I'm not 100% sure how he's doing it, maybe editing, but I also vaguely recall there being an extension that does something similar to this for demonstrative purposes. I remember a coworker saying it'd be funny to troll someone by pretending to be doing something super complex super fast with it
@crowdozer3592
@crowdozer3592 3 года назад
Given the mouse movements I'm more inclined to think it is actually an extension
@harpymaslow
@harpymaslow 6 лет назад
You rock. Gotta take the pro stuff. Don't want to pay every month though. Is the 40% offer still up for the one-time purchase ? Thx man
@Fireship
@Fireship 6 лет назад
Thank you! The one-time membership is not currently discounted, but the monthly/quarterly is. You can cancel anytime. But with the lifetime you also get a free t-shirt :)
@harpymaslow
@harpymaslow 6 лет назад
Angular Firebase Ah the t-shirt. See I need the lifetime plan ^^ The discount was only during Google I/O ?
@Fireship
@Fireship 6 лет назад
Yes, but I will likely discount it again in the future. Not definite timeline tho.
@harpymaslow
@harpymaslow 6 лет назад
Angular Firebase Ok. I'll wait for it then. Great job on the videos. You're helping a LOT
@jajasaria
@jajasaria 5 лет назад
i know mysql but why nosql look so hard to me. hope someone could help me.
@saffanalvy
@saffanalvy 2 года назад
Hi. Can you please make this process in Flutter for Android section. I'll highly appreciate it. Thanks.
@whatthefunction9140
@whatthefunction9140 3 года назад
If I have 1 million documents and want to delete 500k Oldest is that one delete or 500k deletes?
@zacturner7894
@zacturner7894 6 лет назад
Is this code still valid? I get a log error on the server that 'commentId' and 'postId' are trying to call on undefined ie ( 'event.params'). Any ideas?
@Fireship
@Fireship 6 лет назад
For the most part yes, but there were some changes in v1.0 that is causing your error firebase.google.com/docs/functions/beta-v1-diff
@zacturner7894
@zacturner7894 6 лет назад
@@Fireship Thanks mate, got it sorted.
@mozartzerna7024
@mozartzerna7024 6 лет назад
i have this error Cannot read property 'commentId' of undefined ? need help?
@ngrund1
@ngrund1 5 лет назад
You are using a newer version of Firebase SDK for Cloud Functions than he is in this video. Check out the changes needed: firebase.google.com/docs/functions/beta-v1-diff. Here is how I changed the code he provided to get this working: const functions = require('firebase-functions'); const admin = require('firebase-admin'); admin.initializeApp(); exports.aggregateComments = functions.firestore .document('posts/{postId}/comments/{commentId}') .onWrite((change, context) => { const commentId = context.params.commentId; const postId = context.params.postId; Pretty much the same from there on out.
@game-o-zone8170
@game-o-zone8170 6 лет назад
Everything you teach is too good, only thing is you have to slow down a bit, and explain it in depth. rest everything seems gr8 :)
@ashb9254
@ashb9254 6 лет назад
he thinks he is teaching to experts
@ImperiumLibertas
@ImperiumLibertas 6 лет назад
Most of these videos are directed towards those who are "experts." If you want to learn angular, he has a beginners tutorial series.
@Nek208
@Nek208 5 лет назад
How would you solve relationship? Example: see userName or userPhoto on every comment?
@nickstylezmedia
@nickstylezmedia 5 лет назад
ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-jm66TSlVtcc.html :)
@yeilmusic
@yeilmusic 4 года назад
FAVOURITES, likes ? how to fetch which video like user.
@hirenpatelcr
@hirenpatelcr 6 лет назад
Super Fast.... cant understand.
@RafaelTorquato
@RafaelTorquato 2 года назад
This solution works well with 1s limit on the updates of a document?
Далее
Firestore Data Modeling - Five Cool Techniques
11:44
Просмотров 253 тыс.
Minecraft Pizza Mods
00:18
Просмотров 1,1 млн
Firestore Security Rules - How to Hack a Firebase App
10:05
I tried 5 Firebase alternatives
10:31
Просмотров 766 тыс.
Advanced offline caching techniques in Cloud Firestore
12:34
7 Database Paradigms
9:53
Просмотров 1,6 млн
Understanding Firestore queries - Firebase v9
13:21
Просмотров 6 тыс.
Solving one of PostgreSQL's biggest weaknesses.
17:12
Просмотров 178 тыс.
This Is Why Python Data Classes Are Awesome
22:19
Просмотров 795 тыс.
Здесь упор в процессор
18:02
Просмотров 332 тыс.
Acer Predator Тараканьи Бега!
1:00
Просмотров 466 тыс.