Adding a cache is not as simple as it may seem... 

Can you share your neovim distro?

@@user-jb4pt2or2j you can find it on GitHub at elliottminns/dotfiles

That's a cool idea. I imagine it makes things a little more simple and can work for more advanced aggregations! I like it!

lol just left long winded comment about this

​@@xorlop I'm glad you did!

I would do the same 😊

Good idea. Thank you..

And off by one errors 😉

And cache invalidation

​@@hansenchriswand scope bloat

Concurrency and floating bugs: "Am I a joke for you?"

@@Rundik and cache invalidation

Thank you for the great suggestions!

This is a great idea! Deleting does make a lot of sense when it comes to fast accessed data. I think the time you'd want to use write through would be when the data itself takes a long time to populate. But even then, you'd likely use some sort of expiration/deletion based resync. Another approach would be to not extend the expiration which I believe you can do with another redis SET option.

yes, the principle is delete cache on every operation except get requests, that's where we store the cache!

Yep, 100%. In a distributed system, this is even more challenging as you'd likely need to lock the key in the cache as well, which is even more complexity.

@@dreamsofcode Eventual consistency is both a blessing and a curse. Just like how async colours your functions, eventual consistency colours your whole system.

Debezium is pretty great. I wanted to showcase it in this video but it blew the scope out way too much! CDC caching is dope.

Thank you! I shall do one then! 😁

Yep, you're correct! Apologies if I didn't make that clear in the video, I didn't want to complicate the caching implementation itself so went for a simple query

I think this is right choice. Session tokens are a good use of caching.

honestly I almost never use an external cache for anything I've written, because it helps me considerably more to consider how and WHY I want to cache information for each particular type of data. Some of it never needs a cache at all, and some of it only needs to cache a few bits of data, and it also gives you a hint on how to tell when your cache is stale since you know exactly what you're caching and when, rather than caching all data all the time

also when data is relational caching just triplicates the effort to keep it synced properly

I don't exactly understand you store the userId as the key and the token as the value ? Why is this better than using cookie ?

@@anthonycavagne4880 I use cookies and in the cookie I store the session id. Then in redis I have an hashmap with the user id and session id, so I can get the session data quickly and can also invalidate all sessions if needed

yes

Thank you! They definitely have their use cases, but they're not that simple to implement and there's a lot to consider, more so that I even do in the video!

external IDP with SAML2 & OIDC (PKCE) is my biggest weakness

My advice is squeeze as much perf as you can out of your primary data store before you add a caching layer! Your RDBMS can take you a lot further than some people realize.

That's correct! This issue becomes even more problematic in a distributed system as well if we horizontally scale our app.

100% agree. Most use cases it's not needed. And the complexity can often outweigh the benefit. I don't know if I would consider UNLOGGED tables a viable alternative. I did some benchmarking on them for another video I had planned and they're no where near as fast. There's also a lot of caveats with using them and if you're unaware it's unlogged, mistakes can happen.

I think Redis is more perfomant than Postgres' unlogged tables, just because Redis a specific tool optimized for in-memory store. Postgres, OTOH, has layers of abstraction for a simple store and retrieve functionality. Use correct tools for the correct uses.

@@dreamsofcode *The complexity often does not outweigh the benefit. It's extremely simple and will make everything 100x faster. For example, say you have a bunch of base64 encoded images stored in pg, you can make it so the image (likely ~10 kb each or so), gets stored in memory as it's result format, allowing you to make anything involving images significantly faster. It can make things super fast if you use it right, eg. big queries for a product's info or whatever.

@@peppybocan Yeah, the difference is that Redis is very lightweight. If you're looking to throw more RAM & CPU at a DB problem, then Redis starts to make sense. Because 8GB Redis + 8GB Postgres is going to outperform 32GB Postgres in many cases.

not necessarily, depends on the workload. If you have a transaction-heavy processing, there is no way around it. E.g. if you are a paypal and you need strong ACID guarantees you may find yourself in a pickle. Storing payment information in-memory is fine as long as you have resiliency built into the application.@@OneShore

This is much simpler! Thank you.

Awesome write up. I was about to write a similar comment. Now I write this reply to push this one. 👌

I still consider myself lacking experience with Rust, but somehow I knew about these "ok" methods and was about to point it out after watching the vid ^^'.

Your correct, that's one drawback of concurrency as it can introduce a race condition. The only way to solve it would be to lock the cache and pass that lock through to the concurrent task. The other approach is to do write through caching with the cache being the first target, although this can lead to some weird state if the database operation fails. Either way adds complexity!

I was thinking something similar, like the DB or the cache fails, you need a way to sync the state again. I think the easiest solution is to spawn 2 tasks, one for the DB and one for the cache and await the results of both and handle those cases. However, the response time is the slowest of the two

@@tiagocerqueira9459Even spawning two tasks can be complicated, however, especially if one fails and the other doesn't. You need to then reconcile afterwards.

@@dreamsofcode yes, but I guess you always need to wait for the result of both in the handler when you mutate data, you can't just "set and forget" as an optimization

I think one solution workflow would be: lock cache key > update database > (update failed > unlock cache) update success > delete cache and let a cache miss happen in the future.

It's a great question. My implementation in the video is naive, mainly due to simplifying the code as much as possible. The requests are still asynchronous but you're correct the lock would prevent concurrent requests both accessing the shared state. An improved implementation would be to use either a RW lock, or have a better abstraction of the state to only lock when needed (rather than at the start of the request)

thanks kinda new to rust and async

I'll be redoing the neovim content soon! I recommend staying with NVChad 2.0 for the mean time!

Youre correct.There absolutely is a chance. A race condition is introduced by making the update concurrent from the response. If you want to ensure 100% consistency then performing the update synchronously would be preferable!

@@dreamsofcode Cool, thanks for replying!

@@Avanta1Thanks for asking the question!

Read replication is a decent solution in many use cases, especially read heavy as you mentioned. Just like caching, it's a tradeoff so it does depending on what your data model / system looks like.

You're correct. However, even with it being serial, there's no guarantee in a distributed / horizontally scaled system of a race condition not occuring. With a cache, it's almost impossible to guarantee consistency without locking the actual cache itself. In a distributed system, that's going to be even more complex.

I think for that size, you're likely not going to need caching. Caching is more for when you have slower queries, such as aggregations or hitting an API that has poor performance. Adding in a cache adds in complexity and it's probably not worth the performance gain you might receive.

Cant you load them at the start of the request and use some singleton or put them in some "global" variable so you wont have to request the unless needed. Even if thats not the case the db usually has caching mechanisms for repeating queries so for such small size of records its probably unnecessary.

Colorscheme looks like catpuccin

+1, though If-Modified-Since is a bit simpler and usually sufficient

@CesarLP96 search for HTTP conditional requests

developer pages from mozilla would be one recommendation from me, also known as mdn

@CesarLP96 mozilla developer pages would be one page, just search for etag

fyi, i answered multiple times now but yt refuses to show it for some strange reason. not sure you will see this comment as well. one example would be the mozilla developer network

I think that's optimistic mutation, if I'm not wrong.

100% Profiling your queries and using well placed indexes is always a better option. Sometimes it's not possible (such as hitting a remote AP)I, but if you have control of the database then it's always the better option

Even then, do I really need a cache? Why not just index what needs to be indexed?

yeah absolutely! Those limits are external, and that's when it matters. I think Redis is a viable option in that case. even things like session authentication can be done with a silly in-memory LRU cache and it will get you 90-95% to the goal. But people tend to be very quick to stuff the project with a billion dependencies. @@dreamsofcode

@@parkourbee2If you don't have access to the database? I'm thinking more like Remote API's etc where you don't control the data at all. for example, we had an API that hit NIST for CVE's and was incredibly slow, in that case, caching was a good solution.

Postgres has built-in caching which is already crazy performant

LRU comes with a small overhead since you have to somehow store/maintain a "list" of which items were last accessed. In many "normal" cases, it is likely that an item that was recently accessed will be accessed again, so keeping the newer/most frequently used ones in cache is worth the overhead. If your access patterns on the other hand are mostly random, keeping track of usage patterns isn't really worth it, so you can just delete any random entry. You might still want to use a cache even in random use cases when the occasional random cache hit might still give a big enough boost/save you money in bandwidth/storage access cost to make the added complexity of a cache worth it.

This is a great explanation. As for specific use cases, it's hard to really describe any that would fall into this. But any data / queries that have no discernable pattern, or in a system where the likelihood of needing a key is the same across your data set.

You mean until the TTL?

yeh, but if you do sync, update redis after successful dB write then you sacrifice speed. I guess you can implement some kind of rollback if either fails?

Correct, and in a distributed system, this is even more difficult!

😅 It's not known for it's beauty

You won't need caching in a simple project, your database will be able to handle the load fine.

Absolutely! I'll likely do it on my other channel which is more focused on Linux and FOSS. I've been playing with NixOS more on there

Can you tell me which channel it is?

Absolutely! That's the problem in the case of inserts at first. It improved the performance of the reads on a cache hit, but caused the timings to increase by 66% on a cache miss.

You're correct, this is known as the thundering herd problem. You can solve this by using something such like a single flight mechanism or connection pooling, again it's more complexity though.

I don't know if it's optimal, but what I usually do is never let the key expire, and instead just create a cronjob (or something similar) that periodically refreshes the key with updated data.

@@mind.journey This works depending on the goals yes. If it's data that you know will be highly saught after by your code, it can definitely work. However, you are now burdened with the task of guessing which data would benefit from it. It can also lead to you having a lot of data in the cache that you may only need once under the full-moon, thus wasting resources in fetching and keeping it cached.

Yep, I agree with you (I believe I stated something similar at the end). There are certain use cases where caching applies, but in general, optimizing your database queries is the correct approach. There is a case for horizontal scaling over vertical still, especially wrt availability. But even then you can use read replication to improve that.

If you want a really fast cache, it is a good idea to scale the cache together with your application and have no http request to the cache. Most of the time spent to get the data is probably http overhead. I think cache as micro service is - most of the time - a bad idea.

Yeah, that's fair. I wanted to keep the interface as simple as possible so as not to distract from the caching itself. My original setup was doing a string search across 10m rows, but then it added more complexity to the examples (and at that point and index is still a better solution).

This is a good one! The thundering herd problem. There are ways to solve it, using something such as single flight, although again it adds more complexity. Much harder to solve across in a distributed system. I'll probably do a video on it as a few people have mentioned it!

@@dreamsofcode Last time I had that issue, I solved it by forcibly fetching and caching with cron. A bit of a hacky and antipattern way of solving it, but it works really well.

+1, engineers often over complicate things. I think it was Dijkstra who said premature optimization is the root of all evil.

Bad timing! Although tbf, this can apply to any caching solution. + I get to review all of the forks that are coming

@@dreamsofcode Honestly the timing is perfect! Redis is hot in the news cycle right now and you're right - this video isn't really about 'Redis' persay. But it's actually a great introduction to people who are uninformed about the software and want to get up to speed on all that is happening with it right now. I hope you get hella views from this bud! :)

I mean it is only closed source if you are a big company

@@HUEHUEUHEPony no? The new licensing doesn't fall under the definition of open source, and isn't accepted by the open source initiative.

🤣🤣🤣😭😭😭

@@dreamsofcodeit's just way too verbose and not easily readable 😂🙀 I hope this comment doesn't turn into a flamewar!!!

Would like to join you in this

LETS GOO

Ok

Is this your life achievement ?

Congratulations and/or I'm sorry to hear that

@itsme3217 yeah my mom is proud of me 😁

I'm with ya. I think I'm gonna use Go more for demonstrating anything non language specific in the future!

I don’t think that’s necessarily the best decision, if you don’t cache you will incur performance issues under certain circumstances. The example illustrated in this video is a very simple one which could have been solved by efficiently indexing your database, but as you scale or encounter more complex problems you may want to consider caching for latency sensitive functionality. At work we have a workflow that requires our operators to pick orders in a warehouse, fetching the pick list (all the instructions to carry out the picking of an order) takes around 500ms to generate. The pick list reflects the entire state of the current pick journey and uses a cache write through strategy to update the cached pick list after every scan in the warehouse. Without a cache, the front end would need to rebuild the list from the database every time it retrieved the next instruction, 500ms after completing a stop and fetching the next stop would suck, fetching from cache and having a result in 30ms is far better. The tradeoff here is maintaining the complexity of the cache in order to achieve the performance SLO (service level objective) we promised to our consumer (warehouse staff). For simple applications you may be able to keep away from caching but I’d definitely learn and keep it as a tool in your toolbox, I’m sure sometime in your career it’ll be useful :) hope that helps!

I use caching (through postgres, I should really switch to redis) currently to make values live after restart of the application, as requesting the data takes a lot of time and consumes rate limit tokens.

Thank you! Yeah, I will be doing a video on my process this year I hope. I need to find the time to do so! Will drop a community post when I do :)

@@dreamsofcodeawesome, thanks! Would be interesting to know as well!