Тёмный

Amazon/Flipkart Ecommerce Design Deep Dive with Google SWE! | Systems Design Interview Question 18 

Jordan has no life
Подписаться 40 тыс.
Просмотров 11 тыс.
50% 1

Pro Tip: You better hope the shared cart isn't working when you're throwing your tissues and lotion in there
00:00 Introduction
01:02 Functional Requirements
02:01 Capacity Estimates
03:15 API Design
04:24 Database Schema
05:50 Architectural Overview

Наука

Опубликовано:

 

9 июл 2024

Поделиться:

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист
Посмотреть позже
Комментарии : 61   
@cambriandot3665
@cambriandot3665 Год назад
6:08 Search index - partitioning issues 8:22 Adding items to carts: Real time updates, Version vectors, CRDTs 13:42 Placing an order: Writeback Cache 16:35 Streaming/batch stock updates 17:55 Architecture
@dearozero3631
@dearozero3631 10 месяцев назад
I liked just for the intro.
@edmondlebeau
@edmondlebeau Год назад
Great content! It's been a productive summer for you Jordan Love the jokes too
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Appreciate it Edmond, thanks for watching! Always helps that no one gets anything done at Google, who knows what would be happening if I were at Amazon haha
@KishoreShashank
@KishoreShashank 4 месяца назад
Jordan, your content is super authentic and your style of presenting is very practical without any bullshit. Keep up the great work!!!
@jordanhasnolife5163
@jordanhasnolife5163 4 месяца назад
Thanks Kishore!
@olabanji
@olabanji Год назад
I dig your vibe throughout this video bro! Looking forward to more of this.
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Thanks man! Stay tuned for 5pm EST today in that case! Glad to have you as part of the community.
@samanwayamaurya2838
@samanwayamaurya2838 9 месяцев назад
Hi jordan , in the product table (mongoDB) you talked about , are you sharding the product table itself or you are sharding the search cluster built over this product table? Also is this doable to shard the cluster indexes in mongodb.
@jordanhasnolife5163
@jordanhasnolife5163 9 месяцев назад
I think that both would be required, and yes.
@curtissitruc6894
@curtissitruc6894 Год назад
After this intro I neeeeed the AWS system Design Video! Time to sub and wait Edit: Also am a big fan of the the references to other videos
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Haha if I do that it'll have about 50 parts
@ninad11081991
@ninad11081991 7 месяцев назад
Would a cache be a decent option for the management of carts? A) it’s transient data B) updates and fetching would be quick 3) Cost is not going to be a huge problem considering that we’re storing pretty less data on the cache
@jordanhasnolife5163
@jordanhasnolife5163 7 месяцев назад
Seems very reasonable to me!
@franklinyao3833
@franklinyao3833 Год назад
Can I mention DB locking to deal with concurrent editing on the shopping carts? Although lower throughput.
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Definitely a solution, but let me remind you that it's still the case that you'd need something like a websocket to propagate the changes, unless a special database set structure was used. Agreed on the lower throughput part.
@metalalive2006
@metalalive2006 Год назад
Great content. I am learning the design and have a question : Whenever the order service in the diagram (since 17:54) receives a order request, does it require another (relational) database for saving the order first, and then push the same order to Kafka queue for inventory function (which includes another set of databases) ? so in the diagram there would be another link from `order service block` to `SQL DB icon` with `Kafkaqueue icon` side by side, does that make sense ? thanks
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
I don't think it requires it, but you certainly could do that. In the end you can give the order an ID before placing it in the Kafka queue so that you can reassemble the order again later from the individual items.
@metalalive2006
@metalalive2006 Год назад
yea, it may be good to assemble the same order again for reasons like status update or invoice (maybe integration with other accounting software if required) ... etc
@metalalive2006
@metalalive2006 8 месяцев назад
Hi, Jordan, at 13:48 , you described that atomic operations and locks in database might slow down the application when a lot of customers are placing new orders concurrently, one of feasible option for the improvement is write-back cache. Can you provide links to relevant online articles or discussions describing similar write-back-cache idea ? (I watched the ticket-master design video from this channel, just wonder any other discussion which describes similar idea) , thanks again.
@blackcatcaptain2022
@blackcatcaptain2022 9 месяцев назад
version technique update is called optimistic locking
@c_hlee
@c_hlee 3 месяца назад
Hey Jordan, love your channel! This is my first time ever doing system design and your vids help so much. One question -- The way that the diagram has the LB is a little confusing, wouldn't an API gateway direct traffic to the correct service, and then LB placed in front of each service to handle distributing traffic between the replicated server instances? Or can a load balancer alone accomplish traffic directing too?
@jordanhasnolife5163
@jordanhasnolife5163 3 месяца назад
Yeah I tend to lump them together, but you're correct!
@c_hlee
@c_hlee 3 месяца назад
@@jordanhasnolife5163 thanks for the response and helping me to become the best giga chad I can be 😂 lol
@soumik76
@soumik76 5 месяцев назад
Hi Jordan, when would you prefer CRDT vs version vectors for merging data between nodes?
@jordanhasnolife5163
@jordanhasnolife5163 5 месяцев назад
Well version vectors don't allow us to do any merging at the database level, they do allow us to detect conflicts though so that we can store siblings and do merging on the user level. I'd use a CRDT if they can handle merging the data, but if it's too complicated, version vectors and user merging may be better.
@soumik76
@soumik76 5 месяцев назад
@@jordanhasnolife5163 thank you
@mann8737
@mann8737 Год назад
Great video Jordan! Keep going! I have some questions tho. Why did you use MYSQL to store the orders? How about something like Mongo? Also, if SQL is the choice, how would the schema look like since we are partitioning the Kafka queue by product id? Thank you 🙂
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Yeah I think that you could use another DB than MySQL since we don't really need transactions for that part. As for your second question, I think maybe something just like orderId productId quantity would work for schema.
@user-rr9nz6lk5z
@user-rr9nz6lk5z 9 месяцев назад
If someone made order then I think we should reduce amount of orders in product DB but how to do that?
@jordanhasnolife5163
@jordanhasnolife5163 9 месяцев назад
I was proposing we do it in microbatches - have a stream processing framework collect them and every 10 or so minutes update the count
@anupamdey4893
@anupamdey4893 Год назад
When is the AWS System Design video coming ? We could have a series on AWS where you do an in-depth coverage of various AWS services.
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Once I start smoking crack (shouldn't be too long at this rate)
@rajrsa
@rajrsa Год назад
Hey Jordan, Thanks for all these videos! I had a question regarding the choice of technologies in the architecture. At my job, I have used things like MongoDB, Dynamo, MySQL etc. but the applications that I have worked on usually had less than 1k users per day so we didn't really run into many concurrency or scaling issues. Now that I know about stuff like Riak, Flink etc. it makes sense to use them but in practice I have no idea how to implement them. In this case, do you think it would be a good idea to even talk about them in the interview? What if the interviewer asks me to double click on one of these technologies? I probably wouldn't know the answer... Wondering if it will be a good idea to stick with what I know and mention stuff like "btw, I heard some technologies that support X, I would love to explore them." etc. during the interview.
@rajrsa
@rajrsa Год назад
Also, are you really this funny in real life or do you think about this content before making these vids? lol I bet you pull like nobody's business because you're pretty funny!
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
1) I'd probably say: even though I haven't used these as much in my professional career, I think that theoretically for this type of problem we want want to use x 2) Still no bitches 😭
@kendochen6226
@kendochen6226 Год назад
like your attitude to those asking to design AWS. 😄
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Their wives are mine
@yiweizhang286
@yiweizhang286 8 месяцев назад
Hey Jordan, I'm reviewing this design and have a question about the usage of message broker, specifically in the product update/search part. The product service updates the mongoDB table which then sends change-data to a Kafka queue to update the search index. Why do we use a message queue there between MongoDB and Search Index, but we don't use it between the product service and MongoDB? I'm trying to understand when to use a message queue and how to justify the usage. Again thanks for the great content, this video is specially clear and I learned a lot.
@jordanhasnolife5163
@jordanhasnolife5163 8 месяцев назад
Hey Yiwei! Thanks! I think the message queue was used here for between the db and search index because afaik change data in mongo has to be sent to a queue :) Could be wrong though, you're right in that it's not inherently necessary in this case
@yiweizhang286
@yiweizhang286 8 месяцев назад
@@jordanhasnolife5163 thanks Jordan, my question is more about why we choose NOT to use a MQ between the product service and the MongoDB?
@jordanhasnolife5163
@jordanhasnolife5163 8 месяцев назад
@@yiweizhang286 I'd say when you can avoid having more components you probably should, it just adds complexity
@asdasdaa7063
@asdasdaa7063 3 месяца назад
Hey jordan can you do a system design for something like shopify?
@jordanhasnolife5163
@jordanhasnolife5163 3 месяца назад
Can you try and elaborate on which part of Shopify you think is tough to design?
@asdasdaa7063
@asdasdaa7063 3 месяца назад
⁠@@jordanhasnolife5163the database and the multi tenant aspect of it. Im trying to improve my database design skills. Would you say nosql would work better for something like shopify?
@Jashan77114
@Jashan77114 Год назад
Can you recommend other resources for system design?
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Well let me answer your question with a question? Is there anything else you want me to cover? If not and you just want something more formal, designing data intensive applications
@Jashan77114
@Jashan77114 Год назад
@@jordanhasnolife5163 maybe videos on low level and high level designs both
@xavierzhao3460
@xavierzhao3460 Год назад
New potential topic to cover: eBay (Auction System)?
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Will look into this as well!
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Debating whether it's fundamentally any different than something like a chat application - as far as I can think of for now it seems somewhat similar, with the exception of not allowing illegal bids, meaning that you'd probably have to use atomics on a database if you want synchronous responses, or if stream processing is okay you could use that as well.
@barzakovich
@barzakovich Год назад
I understand shit about system design, here only to enjoy jerk jokes
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
I'm not joking 😈
@grabarzowaty
@grabarzowaty Год назад
Keep the jokes. Don'e be a chicken.
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Alright but if I get fired you're gonna have to pimp me out
@grabarzowaty
@grabarzowaty Год назад
@@jordanhasnolife5163 Best fire is HR violation fire. In the worst case you will monetize om yt your personal life dramas.
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
@@grabarzowaty Haha I think you may be slightly overestimating how lucrative youtube is compared to software engineering, but perhaps one day I can live the sigma male dream lol
@franklinyao3833
@franklinyao3833 Год назад
I am the first viewer. So what are the to-make videos? Options: - Payment like digit wallet or Paypal as an example of using transactions. - Real-time gaming leaderboard ...
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
Thanks for the suggestions, I'll think about those and if I feel the solutions are unique enough I'll probably go ahead and add them! Thanks Franklin!
@2tce
@2tce Год назад
@@jordanhasnolife5163 Yes. Leaderboard would be nice to explore. Then I'll like to know about an online chess game (or any online board games). I believe CRDTs will come in handy here.
@jordanhasnolife5163
@jordanhasnolife5163 Год назад
@@2tce Will try to make a gaming video at some point, but do note that there is a leaderboard video already up (#19)
Далее
ХЕРЕЙД БОИТСЯ МОЕЙ СОБАКИ!
37:08
Просмотров 614 тыс.