The Firestore videos series is awesome. Its nice to have a cool dude make you laugh while you learn about underlying technology. Great content and very helpful. Keep sharing please :D ;)
Remember that incremental (limit) pagination breaks if you have persistence on, queries might trigger from old cached values. Say you query for 10 elements, you might get 2 element from older cache.
Some people tend to say awesome for just 'ok' stuff. But this playlist is really... awesome :) The content, the presentation, the animations. Everything is top quality. Keep up the good work!
I think if you want to make this work you need to read the whole collection with attached listener so when anything changes only the change will count as new read
If I increase the limit from 20 to 40 to 60, do I really fetch also the first 20 documents all the time? I thought that the Firestore client SDKs cache these documents and send me those documents only if it's needed. Or is it just that I get billed for those, but Firestore tells the client SDK that it already has the most recent version of the first 20 documents?
This is something i struggle with as there are just hints in the document of how caching works and when you will benefit from it. I can do tests and sometimes I do to try to figure if a query is expensive, but it would be easier to remember if I could get a lesson on it. Please make this a future topic for an episode or add a section devoted to it in the documentation.
@zztop3000 Yeah I know that Todd says that you are re-requesting the old data, but this confuses me. Why should the documents be fetched again? They are already present in the cache of the app. Or do I just get billed for those documents even though they don't get sent to the client?
@@ToddKerpelmanCorp One follow up question. If with the offset you are basically just querying and charging for every record up to the offset, then what is the point of having that parameter at all? You could just request all the records with limit and that would be the same. I always thought of it like pagination and avoiding requesting and charging money for these records. This is actually a huge thing that should be in the docs.
Hey Todd, are you available to do the instructional videos for our app🤣? Seriously, another great video. I haven't got up to thinking about pagination yet but I will soon and this explains pretty much everything I need to know.
Thanks. Would be great to have a new video focusing more in pagination for realtime cases, like chat apps, for instance. It's kind of complicated to find help online for a consistent way to implement pagination with real-time updates with Firestore
Thanks for this! I would love a video that suggests a schema for commenting on user's posts. It's a little tricky to set it up so it's safe permission wise, but also logical query-wise.
Thx for the simple show of options and associated tradeoffs . PS - where on Earth did you find such a enormous gummy bear? Must have doubled this episode's production costs...
So this means that conventional pagination (which has a nav containing 1 - total page count) is impossible out of the box with Firebase - the only option is to infinitely scroll pagination as described. I cant believe this is no way to do this easily with Firebase. Think ill bite the bullet and move over to AWS
@Todd There is a nice idea for you! What if we can extend range of values to listen to while paginating. For example, I do this query for first 20 docs and listen to it -- [limit: 20]. Then when I reach end while scrolling, I do this another query for next 20 docs [startAfter: , limit: 20]. Now it would be awesome if I could just change the limit of what I was listening to in the 1st batch to 40 or maybe change the "end doc" to listen for changes upto. Not send the request, just change what I am listening to.That way, I will only request for 20 docs (20 reads) in 2nd batch and I would be listening to changes in 1st 40 docs. Problem of realtime updating solved! The problem will also be solved if there is a way to just listen to changes in a range of docs without retrieving (requesting for) the docs in the specified range.
This wouldn't work because as he mentioned, the database won't keep the pagination state, so the range [0,40] for a specific query is not something it can do without re-doing the entire query.
@@ToddKerpelmanCorp Thanks for writing back!! You are GOOGLE! There's no such thing for you guys that's tough. If you put your will to it.. I guess you'll get it done. Try using softwares like Video Scribe. They are fast and easy to make yet crisp and useful for educational purposes.
for the 3rd option, why not check for changes before pulling everything. if there are no updates on the old data its should just pull the new one and merge it with the one it has locally on cache
@@ToddKerpelmanCorp Gotcha, I think I'll end up using a cloud function to periodically update the random fields. Not a perfect solution, but cheaper than other options that I can think of
you missed one thing, to paginate without infinite scroll we need to know the amount of records the search has, so we can know the amount of pages, how we can do that?
Firestore is crap, specially the web SDK. Any supposed gain in productivity vanishes when you reach such simple and common use case. In a SQL database, it's terribly simple.
Thanks, I really like your videos. Very clear. One question though: let’s say i page in 20 documents per page and i decide to get the top 60 documents at my 3rd page request. That means the first 40 documents are probably in my local cach alrery ?righ?. If that is the case, the server would only sent me the last missing 20 and not bill me for the 40 that i already have. Is thatvright? And would the system do that automatically our do i need to configure this local cach somehow? Thank, Rik
@@nivaldolemos5280 I cannot believe this isn't a capability. Oh my god. What the h*ll... I might need to migrate to a completely new database platform...
great video; 2:31; does firestore support query on one field and order by another field now? just wondering if it now supports this what I love to hear or is it just a makeup example to simplify the video? thanks #askfirebase
Very nice. But how about jump pagination, imagine i'm on page 3, and want to go to page 7, how do i implement that? there is a function to "jump x docs"?
You can cache previous results in your store or what have you. But yeah, if there's a possibility that the user can request say page 20 right off the bat, like say in a forum board app, you'd probably have to work around this. Maybe adding an index to the document so that you can specifically query for document with index > 20 for example.
This would be expensive because the offset query for 20 records (limit: 20 offset: 40) that you would need for true oldschool pagination would cost 60 reads. Wait, let's say you have 5 pages with 10 records each. In your first query, you must retrieve all first and last records of each page. Then you can navigate with 'start before' and 'start after'. Hmm..
I have a question. If my documents update index real time, What happen if I separate my collection to 2 collections ? First collection keep data and second keep only index but same key. Then I query first 500 keys from second collection and keep in array call "keys_array". Then I query data from first collection that same key in array 1 by 1 every 20 time when user scroll to the bottom of page. So I can manage pagination by myself and documents will not swap order in user's screen. And if this work, it can make numerical paginations. Because only index that's a number is more low cost than full data of document.
after watching this, im just not sure wether to paginate at all.. if you orderBy: date will it still return weird order? can you at least show some best practice code examples?
This tutorial is misleading. There is no pagination. As after wasting half a day, there is still a very hard thing to do to just get the items with index 20 to 40 as they are in a document.
Well, if the reads from database is scalling to the loaded documents+new documents instead of just new documents (already loaded documents not need load again) then is lower than the original recyclerView and RecyclerView FirebaseUI... this is very bad idea (you will lost realtime updates and your reads from database increases exponentially) .-.
A downloadable Xcode sample project would have been nice.. You spoke on how to use it BUT didn't show the specifics concerning snapshot/listener code implementation... Disappointed.
offset is bad for security. Giving your users the power to read a million documents with a slim query is a leverage point for attacks. And this has nothing to do with Firebase. Limit/skip pagination for arbitrary queries must read all previous documents, and cursor-based ones don't. The only thing is that if you manage your own database this isn't a billing problem, but Firestore is billed per-read.
Hi! I have approximately 4.000 records on my firestore and I am paginating it by 20.But what if I am at the beginning and I want to search for an item that hasn't been loaded yet?How would I handle this situation?Thank you!
How could I implement this idea in a chat according to what it mentions from the minute: 6:40, I am working in a chat and it is ready, but if I limit the number of messages in the first load and through a scroll I want to access data Previous, I miss the realtime, if someone has a very helpful example, thank you.
An earlier video in this series explains how security rules cannot be used to limit the amount retrieved. If they are in place and a query is made that would be limited by a security rule, the query will fail.
so you are charging on every single document read, that's so bad when I just want to have a total number of my documents and you charge me like I am doing 1000 read just because I have 1000 documents
keep a counter that you increment decrement on every document add/delete respectively. Then you will have the count in one read and you can do as many as you want
You can build a Cloud Function which cleverly counts the number of documents and saves it somewhere in the database. Keep in mind to deduplicate possible duplicate function executions.
I tried this with streambuilder and I don't recommend, If you just can load the documents, add in a list and show, It's better, that's how I doing on my app and it's working fine.
Awesome but I would prefer either limit star, end or after row_id. That way I can see from code what's going on and save on execution of extra code. May be I am too old fashioned :) Awesome, I had no clue that offset would charge me for all reads I am not actually getting. Another warning that developer should read documentation to get the best results for lowest price :)
This video has opened the way for me, I am doing geospatial queries with geofirex, github.com/codediodeio/geofirex, but it is not possible to page and at the moment it does not have a limit method, I could see in its explanation that the way, although it is expensive would be to increase the limit in each query ... This library does have the ability to limit queries https: //github.com/geofirestore/geofirestore-js ... There is a better alternative for geospatial queries for javascript?
