Recorded live on twitch, GET IN Perhaps the greatest meme master: • Microservices Author: / @krazam / theprimeagen MY MAIN YT CHANNEL: Has well edited engineering videos / theprimeagen Discord / discord
This is a text book example of something "cool" that someone in management, marketing, UX, etc comes with That makes engineering a suffers for a while for little or nothing.
loool, is that what they ended on? Just pointing you to a different season? I assumed they were switching the metadata to make S1E1 a different video, like they did with Love Death and Robots
4 months and multiple services for something most people considered to be a bug. I now look at my boring job in a whole different light, thanks for that. 😀
Wow I remember trying to introduce someone to Black Mirror after S4 but i had already watched it, and i remember how S4 would play first and it was really confusing. Can't believe that was you lmao
@@ThePrimeTimeagen galactus at it again, predicted time and space for the episodes that must be watched before the viewer knows what they want to watch
I was showing my younger brother this video and before you got around to your story. Black Mirror was mentioned and he said 'Oh, the show where they screwed up the order of the seasons.'
Leseon learned: Dont use "smart" names for your services. Racoon, bingo, etc. It gets easier to communicate later to other devs if every service has a boring name related to what it is supposed to do.
@@zyzlol Yep, exactly. I've worked with both. I find the "smart" names, or random names, easier. Cause then you're saying real words rather than just spewing letters that are ultimately meaningless. The real thing to avoid is making services with illogical responsibilities. If something is a bit of logic, put it in a library. If something stores some data, make it a service. And if you're going to make a service, THINK about it. But in large organizations people are looking for something to do to prove themselves, because the organization over-hired. Then everyone wants to make a service, and you get nightmare fuel.
OMG, I was like "Man, what does Black Mirror specifically have to do with microservices?". Hold onto your monoliths tight peeps. As long as you can, never let go.
When you have 3600 hammers connected with unique cog sizes in a 4D map with multiple space time continuum flaws, everything starts to look like a nail to you.
Nope, MS should never be overcomplicated, they should be simple individual jobs of.execution. this.type.of overcomplification is just showing "rush.to.complite the job", not ever take a second to think and design proper solution. Basically is.showing difference between Developer and Engineer. Prime is a developer, very good.and experienced developer, but Engineer would solve this quite differently
I'm currently working at a place that does a hybrid microservice /monolith style of architecture. The decision to structure it this way came about from specific needs (think content creation vs content moderation) and many code refractors (services being deprecated quickly so it's hard to rely on one). The monolith single responsibility app consistently performs better than the microservices based apps. I think we'll see the industry start to realize sometimes it's better to have a tiny monolith single org responsibility style repo vs microservices hell.
I right now work on a fe that support multiple themes ( think of same app for different country or slightly different ui ). Project manager who is also main be dev made a decision to split one int o a separate repo ( which dev ops need to set up and they don't seem to have enough time as it is ). I could not figure it out how it will make anything easier for us since you don't need to share code if it is the same repo what difference is there if a directory is in the same repo or not? Then I got a little story how they have 20 repos on the back end and ever since then I don't bother to even try sway anything anymore. Few months back we got 2 user tokens so 1 user could change and see data of the other user. But you have to send the token from the correct user to see what is going on.
If I remember correctly, Netflix used to have a monolithic architecture. They eventually switched to a microservice architecture because they realized the cost of developing and maintaining a microservice architecture is lower than the loss of revenue caused by more frequent crashes from a monolithic architecture.
Prime... I would've just hardcoded the Black Mirror ID and sort that out hardcoded on the front-end. Yes, do I that crap when they want things for yesterday, I do it in a 1~2 days and then I would take the 4~6 months to do it properly and remove the Hardcoded stuff, I call this: "Tech Debt Quantitative Easing" as Manager most likely would never want me to pay that debt.
I hate that I can relate to this. We have feature flags, but there's no backend bandwidth so let's just hardcode org IDs in the UI application and we'll clean it up never.
Some years ago, I watched that video and did not feel much because I was a student. Now as someone who has recently done a new feature with many microservices involved, I totally feel the pain. Sometimes I wonder, are microservices actually a better way than monoliths to do things.
The problem with software engineering is that there is no such thing as a senior or we got to point where because there are so few senior software engineers that the blind started leading the blind. Back in the day, we were building this beautiful machine that is sending us bunch of data from space. Me being a young engineer that is full of ideas brought up the idea of using micro services to process the data and it sounds like this micro service idea help a lot with scalability. A engineer on the team that having been coding since the 80s. Went home and did a bunch of research on micro services. What he found out was shocking. It turns out that micro service are services with the world micro attach to it. After studying SOAs in depth. I came to the same conclusion also..
I always wondered why the episode numbering for Black Mirror was broken. Figured someone dropped a bollock in the database, never occurred to me it might be deliberate... 😆
Learned a lot today love Content Metadata Service uh, but we'll talk next week about adding a new reverse button player. Awesome sounds good, take care. 😂😂👌💪
So much energy and network bandwidth used for something so simple. Feels like rather than writing code for business logic, we're calling services and composing their results to build the logic up. Then again, I'm just a student and have no idea about the kinds of compromises that happen in production code.
I am also a student but I guess this much complexity is because of Scalability. Simple things are complex to do but large scale stuff become easier I guess.
@@reralt I feel this is because they're using the same components that are used to solve bigger issues, to solve small issues. It seems like you send a condition to a service that evaluates it to true and false returns the result to you - kinda thing.
@@VivekYadav-ds8oz I agree. Basically all they do is build abstractions based on some assumptions that are invalidated by a business requirement later on. So they have to do all of this patching and jumping around just so that their existing abstractions can be reused. Modifying your existing abstractions is a no-go because it can break more things than you can imagine. I find it funny to be honest.
back in the days, when you're not a "student" microservice isn't a thing. and suddenly gets trendy and becomes the "norm" as with any innovation that comes in the industry. and then goes to your curriculum in school. until we learn something in the industry.
I have so many feelings from this because currently this is my life at work. I nearly wrote a giant paragraph about how this made me feel then needed to delete it sigh.
Oh man. I’ve been on the internet long enough I basically delete anything past a certain number of lines. All but guaranteed to be a toxic vent on unsuspecting strangers if you go on that long, and you feel like shit afterward. I feel you.
But as some defense to my tribe, for all the complexity of game dev, we have one problem they don't. Let's forget multiplayer for a second coz I promise their backend is just as webby tangled as ours'. Single player is the core and there the task is just making the best game that can get the most out of one computer. We server side engineers are trying to make a massive brain out of thousands of networked computers with the assumption that they're never all working. It has to be able to take some wrench strikes to the head on a bad day and keep limping on. That means your system has to be architected a lot less like a single expert airplane pilot and more like a committee of various flaky halfwits who each know how to operate one control or are middle management. I mean that's the microservices way. You could do it the Google monorepo way but you need their insanely advanced internal development platform for it to stand a chance.
True. But convinced also of the opposite. Ask game devs to realize an enterprise web platform, and you'll endup with 10s responses and unmaintainable stuff.
Was the craziness of all this explained to the person(s) who wanted that feature? Maybe after seeing the cost of developing all this would have made them change their mind
I've tried explaining to the affected that the solution they were asking for, even if it seems simple, would complicate our current architecture because it was essentially a departure from the initial spec of the project, but it's hard to make someone who only thinks in features instead of maintainability actually see any value in forgoing a feature for a healthier stack.
now I'm just an itty bitty teeny tiny fullstack web dev, so my understanding is limited, but would you say that backend was over-engineered? or was every services' existence justifiable?
It's over engineered. The system starts simple but as you add more features, adding another service becomes the fastest way to solve business problems. The problem is simplifying the system is hard to justify and even harder to quantify. Deprecating and migrating services can take a looong time. All these factors make deprecation projects risky. And so... things just tend to get increasingly complicated over time.
@@videoguy640i feel the problem here is mostly the restrictive access to data for a service when you have multiple disconnected datasources which are all associated to their micro service.
I LOLd at VMS. Maybe I'm slowly getting the microservice scepticism. I'm definitely doubling down on 1.) a minimum number of layers in an architecture and 2.) well defined layers and terminology.
recently finished helping modernise a monolith. Treasure your microservice architecture. you think that takes a while to get stuff done. 25 years of code piled on top. took me months to migrate just the db out
Reverse season order is such garbage. When it first came out, it was easy to accidentally start watching from a later season rather than S01E01. And it's such horrible UX to scroll through the episodes and them not be in order. You want to promote a certain season? Sure, fine. But when I'm viewing the full episode list, show them in gogdarn order!
I knew there was something fucked up when I went to talk about the episodes with friends and I realized the order of the episodes everyone watched was different from mine.
As the team leader for the BE team on the project I'm working on, Primes story made feel better on two fronts. First of all I feel better about our microservice architecture after seeing that diagram that prime drew. But the best part is when he said how the front end guys complained about the naming of the flag... like yesss thanks for sharing this, now i feel like this is something that is just happening to me! P.S. Love you Prime but the way you write the letter s should be illegal xD
well, i feel that the architecture could be better. Microservices are fine but what do you think about just using a distributed database like yugabytedb where each microservice can read the data it needs. I mean the goal of these services is to provide data, isn't it. So why not let the database handle the scalability issues when you want to provide data to all the users. Even if you call other microservices to access data in the end you will create a request to a database, you added just a lot of network requests
@@hfspace That sounds super cool to me, i would be down to try something like that. But sometimes you are constrained with the tech you can use. For example if because of budget constraints you can only use the cheap AWS serverless services like lambdas with DynamoDB and do as best as you can with that. But what you said sounds super interesting to me :)
@@blackfrog1534 well, at my client, we also don't do it like that. I would also like to try it out, but i did not have the opportunity, yet. In my mental model micro services are the ones handling the application logic, but the data access would be managed by such a distributed database, which makes sense imo as you want to protect data at the source.
@@hfspace What happens if the data is one thing when one microservice reads it but changed before another microservice reads it, etc? Then they have different data from the same place. How is this handled? Who knows what is the correct data and sync it?
@@dealloc one thing: you define responsibilities. Only the microservice who owns it, writes to the table. That would at least be my suggestion. The other microservices can only read which will be needed for aggregation of data.And you can set up roles for that in the database. For reading before writing, i think that problem can happen all the time even in the architecture as described here in the video
I'm still not quite sure why you need to do all of this to reverse the season number. Why can't the website team put some special cases in JS code for Black Mirror? Is it just for the sake of decoupling frontend and backend?
It is funny to me looking back - micro-services done well are identical to what we already had established and working in the 1990's (even 80's) with SOA. :)
I remember when I first got into black mirror it started playing s4e1 and I was like wait, why is it playing from the start of season 4, and I remember wondering why they did that. It's cool that I now know why that was. lol
Man I must be extremely dumb because I came up with the same question as in the end and I thought wait I need to re-watch, and now I feel like the interior of my skull is a gigantic ocean of nothing because I'm still wondering why you needed all those microservices to reverse a fucking list of videos
I still dont understand why they couldnt order the list on FE for when it is necessary , yes it is messy but so is the pipeline that is being shown for it.Would appreciate if anyone can help me understand
Probably something along the lines of: Netflix is supported on a wide range of devices, some of which may be eol and not receiving updates. Solid chance trying to do this on the front end would result in patches for hundreds or thousands of different types of devices.
@@lvlinty No, but they did something with the hidden flag names, and even had to switch them, so they had to do updates. Maybe the FE team didn't want to bloat the bundle size. IMO, the FE doesn't look good in this picture, maybe it's their fault. The issue with the flag names hints at an obsession with procedure. "This is not the right way" kind of mentality.
@@DF-ss5ep Doesn't make sense, since most of the lolomo things are server-side rendered. Only things like interaction, streaming and loading additional content after first load is fetched on client. I'm sure they have a sort of BFF that could do the mapping.
I still don't get why you couldn't get all seasons as a list of lists, and then move the fourth list (a fourth season) to the beginning of the list without reordering anything else. Feature flag it and make the front-end reorder them.
My theory is they wanted it to be done all in the backend because then it would work across every device Netflix is on without having to ship new versions for all of them.
Pure genius to migrate the “Ooops, something went wrong.” microservice to the Netflix “Whoops, something went wrong.” microservice. On par with Under Construction animated GIF.
And that's why we are so slow, Omega Star don't support ISO timestamps like they said they will A MONTH AGO! We are constantly blocked! My story is the same but with old IBM ecommerce software.
I was feeling every second of that video skit. It all was too relatable. About The PF episode, I watched the show from the first season. I love the show and listened to the explaination, but it sounds like several services and rewiring with a v2 for what was a single use case. If it is broadly used for more purposes now, it is fine, but if it is just for PF, I'd say some hardcoding and to put to bed those PFs nightmares and never forbid speaking of it again 🙊.
I am a RF hardware designer who sometimes codes too. If the system so inflexible to make it “impossible” to add feature and so difficult, isn’t that a sign of bad software design to begin with? I apologize in advance for potentially asking stupid question.
Have a look at Uncle Bob vs Casey Muratori. Bob (a Clean Code book selling guru) argues for these kinds of highly separated architecture so it's easier to change. Casey (a game dev) argues for just telling the computer what you want it to do, because either way it's not going to be easy to change and you might as well write less code and when you inevitably have to rewrite it later, have less code to rip out.
The pain will always be present. Sure we'd love to have simple straightforward services that do exactly what we think they should, but we also want performant systems that will last a long time as it turns out product doesn't want the user to get hit with a 5 second query at request time so we build caches and indexes to solve that problem. Turns out you from 5 years ago did not possess a third eye that can see into the future and anticipate what product wants today so you don't have direct access to the data you want that is guaranteed to be up to date and valid for a transaction and now you have to make elaborate changes to accommodate that feature. Ultimately this is why any sufficiently long lived codebase starts to look like a hoarder's house as you build seemingly redundant features that meet varying use cases.
This is where companies need to be willing to bite the bullet and refactor. The initial architecture is always going to be a mess because you don't know what you don't know. But the lessons learned from it allow you to create a much better successor.
