Google system design interview: Design Spotify (with ex-Google EM) 

Get 1-1 system design interview coaching with FAANG ex-interviewers: app.igotanoffer.com/coaching/tech/

This video is a perfect example of how things should be explained. The way Mark has explained entire design is commendable. Kudos to the guy interviewing for being so patient and polite.

The elegance with which Mark explained it 🤌🤌. Exquisite!!!

Just the fact that the interviewer could shut up and listen to the answer makes this interview great. There is nothing otherworldly about design interviews, not much has changed or invented in the recent decades the only issue in my experience is that people can't just sit and listen, they'll be constantly asking questions, breaking up the train of thought, I'd say its a tutorial for the interviewer and not the other way around)

Great content, thank you! This channel should have more subs.

That was a nice one. I like how Mark evolved the design.

Im still a junior but I remember some classes that featured system design and watching this interview brings up a lot of memories. What I also love about that is the "doing things from scratch" part. When you're dealing with system design, it usually means you're creating something new, a new app, a new service etc and that's always an exciting endeavour

This was the most realistic System Design interview video I've watched.

great video to watch before an interview for any position in computer science fields...

What a pleasure it's to listen to this kind of people and the way they design solutions, they make it look easy but it takes years of experience to abstract like that

Mark does a great job of explaining the different aspects of design in a clear and concise way.I really enjoyed this video,keep going on man 🤟

This is pure gold, explains almost everything when you need to learn what a system is and how it functions...very very useful !!! Thanks man !

This is a great session. This format works when interviewer is a good listener and allows Mark to finish what he has to say, put a logical end before transitioning to next stage of SD or asking questions. and that is great. Can you do a session where the interviewer is constantly interrupting? you neatly define stages of the SD interview and its a flow we (interviewees) would like to get into. but more often than not, interviewer doesn't wait for you to finish a topic.(usually non FAANG companies) they just want to get into details of a component. or more often than not ask "why". I personally find it hard to transition from "answering their question"(which could take easily 2-5mins) to getting back into original format I had planned for the session. and because I'm unable to logically complete i fail the interview.

This was super interesting! Thanks for bringing this content to us!

Great job! Enjoyed watching really much

Great value please create more content like this

Really like how Mark communicates so effectively, and designs iteratively.

it's always best to start the interview answer and define the "Functional requirements"(FR) and the "Non-Functional Requirements" (NFR) that are needed for the design. NFR could look like this for this design: 1. low latency 2. high availability 3. secured connection etc.... this helps to flush out point of failures, and bottlenecks early in the design.

Thank you for making the video! it's so useful for me to know something that needs to prepare once I want to look for new opportunities 😀

I knew every single measure and strategy which Mark presented here. But I dont think I would have been able to present it the way he did with a gradual continuous increase of complexity. Awesome answer Mark. I wish I could get interviews to be able to deliver these answers, Im good at that.

Very informative, thank you! Start with simpler design and get buy in in order to avoid going on a tangent into details.

He is just genius!! The way he is explaining is REMARKABLE!

That's interesting to watch. The design looks very similar to the one I produced during an Amazon Interview with the Load Balancing, Cache and Server Geo Localization. I was feeling good as well about my interview however I have failed it. The most dramatic part about failing the interview is that we do not have any feedback on our mistake to improve on. The only mistake I see is that at the last test, I did not write down one of the requirements and when I finished coding the interviewer told me he said the opposite about this particular requirement and I had nothing to back up / verify who was right. So if you are about to go through an interview, lay down on paper all the requirements, validate them and then proceed to the coding part. Good luck out there.

Like many others I was designing my own version alongside Mark's and I think the area he was a little weaker in (which he himself admits) was load balancing. My background is systems administration so I may have a different perspective on this. I think going back a few steps, chunking the data also serves an important role in the load balancing process. I would have songs chunked, from every retrieval source, so that as soon as the user presses Play, the song begins playing, and playback should always be an instantaneous process unless the servers are over capacity, which can occur because some song or album has gone viral. I would structure the web server load balancing so that client apps attempt to contact the server geographically closest to them first and utilize GSLB (global site load balancing) which combines layer 4 and layer 7 load balancing, as I/O capacity or concurrent connections (the two metrics I would prioritize) reach a threshold. Again, when talking about load balancing, it's important to determine what happens when maximum capacity on all servers is exceeded. When this happens in my design, the system will issue "tickets" for data chunks, served in the order they arrive in. This is where song chunking comes into play. Because we are chunking the MP3 data, we can still serve the first chunk of the song from the nearest available server as soon as that I/O is available, further ensuring near-instantaneous playback upon pressing the Play button. The rest of the song then has some time to download and cache to the client device, reducing the number of interruptions and pauses in playback due to bandwidth and concurrent connection overages.

Recently had an interview with the same, hadn't come across this video then. I wish my design was as neat as it is here. The simplicity does help explain the data flow a lot better.

Amazingly explained. Thank you Mark.

usually I get bored by tech teaching videos but this is the first one that I am still watching.

Thanks mark! Very helpful to basically see how to communicate effectively calmly and enhance the design step by step. I would've added couple of more things here though 1. Separate the application servers for Querying the songs vs playing the songs (As you mentioned the load can be very different and the servers which are playing the songs will have high network bandwidth usage) 2. Add cache to the metadata server also (Songs metadata to maybe cache the songs which are recently, from some famous genre etc)

this interview is much more useful than my 3 months university course 😜

It was great. Thanks for doing this. I would like to see more of system design interviews.

this was great . maybe it didn't add a lot to me in terms of technical aspects but the way mark was connecting the dots was really interesting that's exactly what you expect from an top notch engineering manager

thank you very much for the video! I was looking for something like this. I am not the best solution architect ever but I would design Spotify by very similar way but I am grateful for design ideas

Great format. Thank you very much :)

A very intuitive tutorial interview ❤❤❤

A few things I did along side while understanding Mark's POV: 1. I would usually introduce DNS geo routing earlier in the stage to route to the nearest LB 2. Also worth to have a Metrics collector that can always keep track of HITs of top 100 (or emerging hits depending on BI) per region basis in some form of a max heap and then have a scheduler to periodically walk through them to ensure that nearest CDNs are hot loaded / prewarmed with them. Reading from S3 is very slow and I would usually find other alternatives instead of chunk reading in an instance memory. Packet roundtrips can be costly especially in use case of streaming. 3. I also split the durable storage into two - user data storage (less frequently used in comparison) and songs metadata storage - this way DBs can be fine tuned for workloads. 4. If I told S3, I would also mention cross region replication just to touch it a bit and indicate that I was thinking of a DC going down entirely.

Love the video! Before playing the whole video I played around with a design of my own and ended up with pretty much the same design with some variations that I'll add bellow. I think Mark and the Interviewer missed on digging a bit deeper into one of the main requirements Finding Music. Mark talks about performing the search operation directly from RDS. Taking into consideration the scale of the system, that would have been a terrible decision. With millions of users, the search function would hit the DB constantly and generating read queries in the RDS instance that stores its data on disk. Resulting in overuse of the DB and high latency. In my design, I went for a dedicated search service that is powered by a Search Engine such as ElasticSearch. This service is populated in the background asynchronously by a Consolidator service. Essentially, each time a data is added to the RDS (new songs, etc) an event or message is sent to a queue, the Consolidator Service would get the new data and push it to ES. Then the users can search very fast for songs using a highly optimised Search Engine.

A great video to explain how solution architects work and what knowledge they need. Actually if you need to stream it would be difficult through cdn that will send the whole file. If you have own servers close to users I would just make some large cache and a small streaming server from local file system. As RAM is not that expensive now I would even suggest RAM disk for songs. So when a user needs a song it is read from some cdn(just to minimize hops for geo regions of own local servers), then file reader marks access and starts streaming just using file read and write to socket or the file is passed to the end user. Such a simple streamer/reader will be able to handle tens of thousands of connections on a single server. At end of day or some percent of disk full a job should just delete files ordered by last access. Some small local rdb can help for the marking as you will not have 1 billion songs on the local disk. This may even be better than commercial cdn as it is your own one and price is lower.

15:00 it’s important to mention that the problem with the storage separation is not about the data im/mutability (actually you can update data even in the blob storage). Primarily it is about how inefficient it would be to store 5MB blobs in any general kind of OLTP database that will cut each blob into pieces of 2KB sizes, build a separate table (toast) for it with index over each piece. And only then you would want have more efficient streaming and completely different types of local and global caching. So separation makes lots of sense just because one data is in small pieces and another is in big.

It was nice that the interviewer just listened, and the interviewee presented a simple design. However, in real FANNG interviews, especially for Senior roles, you're expected to go into more detail, and the interviewer usually challenges your decisions.

wow, love this mock interview ! I think a lot of this is covered in AWS Cloud Certifications !!!

To be able to watch this for free is just amazing. Thanks so much

Really nice video. Another point is to dress up during the interview. Mark looks like a CTO-level person. That first impression is really important when leveling.

What this interview really shows is that you don't really need to know every detail of the future solution(spotify i much more complex than this), but those solutions that you choose to invent - you should be capable of explaining why they are needed in the most understandable way.

honest interview...just missed some chunking idea for songs at my opinion...btw great interviewer....always acknowledging with positive body expression 😊

This is very good! Also the separate meta database could make possible to search for multiple language content in the user language. I mean; the songs are available in every language to every user where the user doesn't need to worry about knowing foreign languages. Only the developer needs to make the data available to the local language, including the meta data while the streaming media could be gobal regardless language.

Thanks both of you!

This session is really good. Thanks for the video :)

Great clip, thanks!

Pleasantly surprised he came up w the example of european punkrock, as I’ve been playing in european punkrock bands for a while 😊 nice choice!! (And it really is a bit of a niche)

For load balancing, you'd also want to think about having certain webservers marked for specific tasks. Though I suppose that would be more like having 2 services - your lookup service and your streaming service. That way you don't have to worry about the weight/priority of IO based balancing vs CPU. Then your lookup services are CPU based and your streaming services are IO based.

This was very interesting to watch. I am currently a Senior Software Engineer, and will probably end my career at this level as I'm quickly approaching retirement age. I've always loved getting my hands dirty writing code, and have never had any aspirations to advance to the level of an Engineering Manager (or Development Director, etc.). But while watching this interview, I found that my thinking was in lock-step with Mark's, and I found myself answering the interview questions with essentially the same responses. I even blurted out several of the same responses _before_ Mark answered in the same way.

This was so amazing. Thanks for sharing

This was so damn cool, as a rookie CTO this is a great transfer application of SD concepts to learn from. Definitely coming back for more!

Redis for heatmap. Timescale db for stats and playback history. Asynnc Via a message queue

24:00 the clinet app should be able to get the chunk of the mp3 directly with some sort of token (expires in, for example,15 mins, per Id or user, etc.), the web server should not fetch the mp3 chunk data for client app but generates access token only

This is such an amazing and informative video. Keep it up guys

Super helpful. Thanks for the walkthrough!

The audio streaming would better work with 30 sec. chunks of audio, instead of loading the full track, which can vary in length, from am minute long track to 20 minutes long. Also, ordering the artists and songs based on both relevance to the search terms and popularity and user's personal listening habits and preferences should make sense. Artist, song, and user metadata are all connected with relations on multiple vectors, like genre, mood, country of origin, and lots of unknown relations (aspects) that come up from machine learning, etc.

this video is mind blowing, it teach me a lot, thanks you.

Very helpful. Thanks for sharing.

Very useful! Thank you.

ex google cause he used AWS instead of Google Cloud

It's funny that the interviewer is trying so hard to nitpick everything that the more experienced guy is saying. "He should have said up front why he was splitting the databases into two." There's so much going on and you're working through a problem you were just given 10 minutes ago, no interviewer is going to care about if he addresses it up front or if they have to ask for clarification. It's all part of the process

Great video! Thank you!

Google engineer and still uses as reference AWS lol, poor GCP. Nice vid BTW.

It is interesting to see that even a 13 year Google Engineering lead (guy's a BIG shot) has to think about an approach. Makes my own work so much more relatable. I like the fact that he was not given the question beforehand

I feel that with a humungous list of 100 million songs, we can implement a separate search server for the search functionality like a Solr search. It will reduce the searching time by a huge margin.

Thank you Pastor Rob

Great and interesting interview! AWS Cloudfront with S3 backend automatically pulls a file from S3 if it is not cached already so the webserver could return the mp3_link at the Cloudfront distribution endpoint and Cloudfront would take care of everything else.

Thank you, useful.

Elastic search might work really well for the metadata db. It should cover the storage as well as the search functionality.

This is gold

great video mate

23:40 I think downloading the songs to play will need TCP, so not sure what he means by reading directly from DB into the server as an optimization. 33:10 he flows from one idea to another very smoothly, but the cache talk feels like he's just listing whatever is top of mind as well and a bit rambling so structure would be good

I think you don't need a websocket connection for chunk loading. Both HLS and MPEG-DASH are working though HTTP protocol for this purpose

I would have gone deeper on API specs (some endpoints, how would they work?), the searching algorithm (roughly, db indexes? some middle caches?), and audio service (streaming, shared cached besides CDN, loading all in mem takes time where the user hears nothing, and is costly in RAM, discuss alternatives). A way to deal w/ metrics (data pipeline, no need for too many details). Also, mention CAP, what would u choose and why. Normally, you will forget to mention things, and the interviewer will ask accordingly; but as mentioned, it is usually better to have your key points exposed w/o the interviewer needing to question you.

Great interview style! Q. Which tool is Mark using for drawing diagrams/texts?

I think concurrency and fault tolerance is a big design consideration. If a web server goes down will it take down N users . I’d probably look at adopting something with Erlang. Great content and appreciate the input . Joined as a sub

also, definetly wouldnt go with streaming audio from the webservers - for both scalability and separation of concerns. a finelly tuned CDN (having price constraints in mind) would do the job.

The fact that he's bringing up specific kpop groups makes my day.

You could queries and can use O(1) operations partitioning in S3. Just wanted to clarify that.

This is very good!

Great video!!!!

Wild that he didn't think about bandwidth. This is the make or break number for any streaming service

This is probably for some junior engineers. It has very basic concepts. The questions were not too technical to kinda push the interview towards high decision making skills. This is just list all technologies in any saas , and connect them,

I would have loved to see how albums, playlist, recommendations and song radio world be stored and served.

thanks, was really usefull

As an IT veteran, I wasn't that impressed that. Thought it was a bit rough. No API for example, no multiple levels of load balancing (geo dns load balancing in first tier, at least second tier across databases. No multiple database, for example, per country, or per letter, like users a* database, the b* database for stability. Web app would be the website and the Web player. Api would a tier below that, serving apps and the Web tier. Would also need a app tier, for data crunching for playlist generation, integration to 3rd parties services like lyrics, user auth and subscription billing and what about ato scaling, monitoring, alerting, emailing and maintenance. CDN setup for geo awareness for localised streaming. No mention on third party auth or app integration. Was okay, but not great. Sorry.

It would have been great if he could explain about what compute options to use. Some like gke gce or app engine etc

this is so easy, I don't know why some make it sound like a big deal. I am not even a native english speaker and I can understand this fully and can do same thing with any other system design requested in an interview. The only problem is getting the interview lmao

What I noticed missing was TTL or time to live or file expiration. That should be part of the API call as we dont want to indefinitely store songs in our CDN or in Cache. And really any reference to APIs or tracking of session state to be able to continhe where a user left off.

Great job done. I only have one question to understand. Did you miss talking about the security (authorization & authentication ) of data (music)? Or it is out of scope for this interview?

I would prefer a location based load balancer as a primary way and spread web-servers proportional to users geographically considering that spotify is used by almost all countries.

My first thought was "the level of confidence to question a senior ex-Googler". Then, I remembered that Google has put out some less than stellar solutions. All in all, Mark explained it beautifully and it must have been a joy to work with him.

I hope mark will teach a about making systems or apps in the futrure

awesome content

Great interview! Love from Greece!😀😀

The way he talks about CDN is only about cache, but you also don't want to provide a direct link to the source that can be hijacked or abused in any other way

In real world you will be bombarded with a lot of questions.

I would argue with assumption that reading whole 5MB file would be good. Assuming that there are like 1M users listening at one moment I would split that file into smaller chunks, like 1MB to reduce RAM usage in one particular moment.