What I Wish I Had Known Before Scaling Uber to 1000 Services • Matt Ranney • GOTO 2016

Подписаться 990 тыс.

Просмотров 326 тыс.

50% 1

This presentation was recorded at GOTO Chicago 2016. #gotocon #gotochgo
gotochgo.com
Matt Ranney - Chief Systems Architect at Uber, Co-founder of Voxer
ABSTRACT
To Keep up with Uber's growth, we've embraced microservices in a big way. This has led to an explosion of new services, crossing over 1,000 production services in early March 2016. Along the way we've learned a lot, and if we had to do it all over again [...]
Download slides and read the full abstract here:
gotocon.com/chicago-2016/pres...
RECOMMENDED BOOKS
Sam Newman • Monolith to Microservices • amzn.to/2Nml96E
Sam Newman • Building Microservices • amzn.to/3dMPbOs
Ronnie Mitra & Irakli Nadareishvili • Microservices: Up and Running• amzn.to/3c4HmmL Mitra, Nadareishvili, McLarty & Amundsen • Microservice Architecture • amzn.to/3fVNAb0
Chris Richardson • Microservices Patterns • amzn.to/2SOnQ7h
Adam Bellemare • Building Event-Driven Microservices • amzn.to/3yoa7TZ
Dave Farley • Continuous Delivery Pipelines • amzn.to/3hjiE51
/ gotochgo
/ gotoconference
#Microservices #ScalingSoftware #scaling
Looking for a unique learning experience?
Attend the next GOTO Conference near you! Get your ticket at gotocon.com
SUBSCRIBE TO OUR CHANNEL - new videos posted almost daily.
ru-vid.com...

Наука

Опубликовано:

9 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 123

@OgnyanDimitrov 4 года назад

Plus one for admitting a lot of hard truths in such a diplomatic and polite tone. Scaling from 200 to 2000 for a year sounds like a monstrosity to me! - Whoa! Great job! I want to greet the one who was in charge of this situation!

@RobertWierzbowski 7 лет назад

On rationalizing "using your baby" vs switching to a better supported service, I like the quote on writing by William Faulkner: "You must kill your darlings". We get attached to complex, clever, difficult code, but the best code is dull, simple, unclever, or non-existant.

@SenorQuichotte 6 лет назад

you've probably never wrote a killer perl one-liner

@joshpetit Год назад

@@SenorQuichotte ahaha no one would doubt my 200 character awk one liners are a piece of work.

@dac514 7 лет назад

Step 1) Hire two thousand developers. Step 2) ??? Step 3) Profit.

@_stokyo_ 2 года назад

The takeaways for me were: Own your uptime Use a common profiling shape that works for all used programming languages Configure distributed tracing Use a common metric dashboard Consistent logging Performance IS important even if it's a small subset of a call chain Stop logging everything

@cupofkoa 3 года назад

Honesty is incredibly valuable.

@MargaretLeber 3 года назад

"Immutability changes everything."

@leozilla 7 лет назад

one of the best presentations about the danger of (micro)services

@Mike-rt2vp 5 лет назад

@@qm3ster ((((:-))))

@csvegso 3 года назад

I fully agree with all the statements. We faced all of these issues. One by one. In the painful ways.

@PratyushAgarwal 3 года назад

One of the key takeaways (that I didn’t see in other comments) was the point regarding tradeoffs. Everything that is being decided (small or big) has tradeoffs. Thus, decisions need to be taken deliberately instead of just accepting it as the way things are headed.

@richardhp77 3 года назад

Yeah that was cool. Make your trade offs tailored to your organisation so the bad stuff doesn't have such a big impact

@globetrotter373 Год назад

Architecture 101

@webmastergrumpy 7 лет назад

Great story, great presentation.

@ChuckJHardy 7 лет назад

Great talk, well done.

@jenesuispluslameme 7 лет назад

Great talk from experience with microservices - thanks

@dardanbekteshi3177 2 года назад

Just realized we have the exact same problems like Uber, only thing that differs is we don't have Uber as a product 😂

@revmusical 6 лет назад

Great content , pretty much captures every pitfall of using the microservices

@vanhoangtran1270 5 лет назад

thanks for sharing

@_stokyo_ 2 года назад

Can someone explain what cross language context propagation means? He talks about passing context down to intermediary calls but I don't grasp what the alternative is?

@web3tel 7 лет назад

Great presentation

@AlexKavanagh29x 7 лет назад

Politics - succinctly - is the process of deciding who gets what, when and how. (Harold Lasswell) .. whenever there are scarce resources (including people), and people fight over them, politics is what happens.

@thunderloong 7 лет назад

Politics is what happens when people agree not to kill each other over resources. Scarcity has no validity. Distribution has no validity.

@richardwalker3760 7 лет назад

Strongly disagree. *Economics* is the process of deciding who gets what, when and how. Politics only matters when the government tries to control the economy.

@anis.lounis 7 лет назад

Well said!

@MattMcCormick1 7 лет назад

This is a much better definition. I've heard the Company > Team > Self before and always wonder a) where is the end customer in that equation because they are what really matters and b) there are plenty of cases where the self's decision could go against the company but it would be the right one (ie. illegal behaviour at the company)

@fnjordy 7 лет назад

Multiple languages for services are advantageous as it encourages a better separation of the transaction. The challenge is an environment with bias on one language that leads to 2nd tier broken support for the other languages as no one wishes to fix the dominant platform when it errs. Much like RFCs vs. an implementation of a protocol being the reference.

@ArchimedesTrajano 5 лет назад

43:29 I go with the notion that politics specifically "playing politics" is the notion of utilizing people to achieve a goal but making it so that the people that are important to you are satisfied. Bad politics reduces the amount of people being satisfied, good politics conversely increase the number of people satisfied.

@MarshallSorenson 7 лет назад

tldr; like, we thought micro-services meant nano-services, so like, that's bad.

6 лет назад

ge looks so happy!

@HoldOffHunger 7 лет назад

"Don't bring out the sticks. Carrots are the only way to go." Nailed it.

@Skorps1811 4 года назад

That's a COVID-like growth of microservices there

@transparentpolitics 7 лет назад

Good talk. Politics is an "obvious" (to me) of teams owning their *own* uptime -- not uber's uptime as a whole. Suggestion: I have this same problem : you were not looking at the audience and engaging with your eyes. Lift up your head and not down at the monitor all the time.

@KangoV 2 года назад

Very nice talk. Eclipse allows cross cutting changes. IntelliJ does not. I really miss workspaces.

@TheBookDoctor 7 лет назад

Great talk. Interesting how many of the problems you discuss really do boil down to the "politics" category of human problems: emotional sunk-costs arguments for sticking with the known-but-broken vs. taking the cost of switching to the new-but-not-broken. Or as Douglas Adams said, "People are a problem."

@rickardoberg 7 лет назад

I think a majority of the issues brought up could be minimized by using EventSourcing rather than REST as the internal distributed system pattern. A service is responsible for writes, and letting other services consume those through e.g. feeds (using Atom, this would still be RESTful btw). Those dependent services can then project those events into whatever database they need locally, and do not depend on uptime or scalability of the other service. This also minimizes need and impact of migrations, as all databases are just projections of events, and can be rebuilt by simply deleting them (we do this in production all the time, in a rolling fashion). Blue/green deployments is also an option (and VASTLY simpler using EventSourcing) to constantly do major upgrades to services without downtime.

@MartinMortensen 5 лет назад

I don't even know where to start. But eventaorucing and microservices is by no means less complexity.

@richardhp77 3 года назад

@@MartinMortensen event sourcing brings it own demons, but in this particular use-case, based on what uber are trying to achieve with rapid development of decouple features, you could make the argument that event sourcing would be a better trade off than distributed micro services

@baptistepattyn7555 2 года назад

@@richardhp77 Event sourcing does not exclude microservices.... on the contrary

@vincentmax4571 5 лет назад

Cool stuff

@pkeshava 7 лет назад

Pardon my ignorance. What is WIWIK?

@dn5426 7 лет назад

"What I Wish I Knew"

@bigdlamz 3 года назад

The is what happens when you embrace microservices with all its buzzwordy glory. Some decisions need to be centralized eg decomposition & language to use.

@eugeneleroux6140 6 лет назад

Lots to think about, thanks.

@jabberl. 4 года назад

node, go, java, python - "Wow, that's like a lot of languages!" LMAO

@reespozzi4334 2 года назад

It is. Less the better

@mennovanlavieren3885 4 года назад

29:37 Ah, ORM induced performance problems.

@tauruscolvin 5 лет назад

Insightful talk -- thank you. Regarding Matt's question: 'How can we safely insert the "ride with a puppy" step into the Uber architecture?'. Perhaps a dab of functional programming? That is, we assert that each microservice should be a composition of (pure) functions -- aka "pipe". These "micro-pipes", would give us the *synchronous* composability we want while preserving *asynchronicity* in the space between the services achieved through event sourcing. Google seems to call this pattern "functional microservices".

@omarmoataz 4 года назад

I watched 21 minutes of this talk and I still can't figure out if I learned anything.

@RayBrooks0 7 лет назад

Matt, such a great presentation! Also, you have tape on your face.

@TheNiters 5 лет назад

10x growth in engineers in 1 year is probably too fast... but might be hard to avoid

@tsingylu 4 года назад

I can feel his frustration

@Silmarieni1 7 лет назад

Isn"t a bunch of criticisms of microservices in fact a failure to choose the right size for a microservice ?

@waleedjubeh5955 Год назад

When I hear "Buying new computers (scaling) is cheaper than hiring software engineers." I have shocked about this fact. Bom!

@Divinii 4 года назад

10:55 Wow! *"By having multiple languages, it can fragment the culture."*

@rentsy3444 4 года назад

don't do it

@Divinii 4 года назад

@@rentsy3444 Thanks!

@HaithamElGhareeb 7 лет назад

One of the most interesting Microservices sessions I ever saw at all. Thanks a lot.

@rftluong 7 лет назад

Great talk ... the number of ads in it are ridiculous though.

@tux1968 7 лет назад

@saidimuapale 7 лет назад

Subtext of this talk: the Uber Engineering mega-team is fairly dysfunctional.

@JulienCROUZET 7 лет назад

Plot twist : After reading a comment on RU-vid from a random guy with no real experience of high traffic they decided to change everything because I thinked that they're "fairly dysfunctional"

@jerrygreenest 3 года назад

Double Plot Twist: After they noticed another guy used the word “thinked” in his arguments, they reconsidered what the else random guy is saying, and indeed fired half of people

@tscan89 7 лет назад

Seems like they need a new microservice to coordinate all their microservices. :)

@sahild6584 7 лет назад

Tim Scanlin nice sarcasm

@Calphool222 6 лет назад

Actually, Netflix has a presentation where they talk about that. Even when you're doing a microservice based architecture, you end up having some services that are far more critical than others. Aspects of monolithic systems begin to re-emerge. In monoliths we addressed these "critical components" with various high availability tricks. Netflix took a slightly different approach. They built Hystrix to provide systemic fallback and short-circuiting of cascade failures, which are an emergent property of a microservice based architecture.

@hariveturi4193 4 года назад

@@Calphool222 I happened to stumble onto this talk after the one Netflix one :) I know it has been a long time since this comment but still. Really loved the netflix talk. Wish they would've done QA later.

@Beebop121 7 лет назад

I really dodged a bullet staying away from Silicon Valley.

@-Jason-L Год назад

Approaching microservices like RPC is why you will have so many issues. It's a mistake made by folks rooted in older distributed system approaches.

@minyakonga8897 3 года назад

everything is a tradeoff, everything is a choice

@jake8217 Год назад

All of these sounds like Uber problems than microservices problems.

@n8style 6 лет назад

"It's so hard to get a count of how many services are running because it's changing so rapidly" Wait what, you don't have an automatic count you do it manually?!?!

@charlessharpe2121 7 лет назад

Why not use an event-driven, non-blocking I/O to speed up REPO synchronization, aka nodejs? Is the company focus dependent upon centralized or decentralized philosophies?

@christopherstamp9716 7 лет назад

Charles B. Sharpe it's not that easy. They obviously use node js but node js is single threaded and can run into similar scaling problems as any language. The complication is concurrency/caching and cpu metrics/resource management. Very complicated environment

@boltonkim7090 4 года назад

My thoughts after watching this video: 1. Are engineering teams at Uber comprised only of junior engineers and fresh grads? or 2. Maybe Uber doesn't have much of an engineering management culture? or 3. Maybe Uber's product organization wants things moving so fast that the engineering organization has to throw common sense away?

@nickbarton3191 4 года назад

I'm surprised that he's not a jibbering wreck of shredded nerves.

@DodaGarcia Год назад

He kind of is though

@Pjblabla2 2 года назад

Very informative It was unusual to see someone with the chief systems architect role in modern IT organisation It'll be good to get an understanding of different roles within Uber's IT organisation and how it is structured E.g. are these just his observations in retrospect or does he have the authority to influence future decisions based on these insights

@dotanoob466 4 года назад

1. break laws 2. have a toxic culture 3. get a-holes to invest in an app made by ppl who end up doing all the work but getting no credit.

@beedeeuniko 4 года назад

the frequency of him saying "like", "kinda", "sort of", "totally" and "you know" is - like - staggering, you know?

@Skorps1811 4 года назад

is like kinda sort of totally staggering, you know?

@szym1 7 лет назад

like like like like like like I can't bear it over 13:25

@dieterplaetinck4431 7 лет назад

me too. i gave up on this presentation. I couldn't stand the "street" talk. like like you know like

@electroman7396 4 года назад

Uber does no profit and just pumped up and hyped. Why is everyone taking it as an example?

@minyakonga8897 3 года назад

best have so many defs, like best libs, best cuz i know it, or best ....

@georgesmith3022 4 года назад

So somebody discovered distributed systems again, called them microservices, but nobody seems to know how to make them. Is it another fad that will go away?

@ahmedbadia1485 4 года назад

this guy talks like Richard from silcion valley, guess the show was accurate after all

@chashdeveloper 7 лет назад

He is an excellent presenter. I have attended one of his talk in 2015

@olegzandrvondenmanoresoftw596 5 лет назад

How many "likes" can you say in a minute lulz

@billmoney1 7 лет назад

sup

@kobac8207 6 лет назад

Thanks for this presentation and for a completely unexpected reason. I thought that guys doing architecture at Uber actually know to do architecture. Who the fuck scales to 1k microservices in a year??? Do they even have a clue how and when to separate a microservice? Reason for a new microservice is a new bounded context. How in the hell did you manage to find 1k bounded context in the app that routes the vehicles for taxi rides? Uber is a distributed ball of mud. This guy is an antipattern himself.

@mensaswede4028 7 лет назад

So Uber is a great business idea, but as usual it was poorly implemented. But it succeed in spite of its poor implementation because it was a great business idea. The world is not a meritocracy.

@m424t 7 лет назад

This was a very disappointing video. Unfortunately it seems like Uber architecture was designed without much thought in a way junior devs would approach it. Given the very basic nature of problems they're having, I am surprised that Matt hasn't mentioned problems with transaction support and REST API versioning.

@Teslawaverunner 3 года назад

Interesting how people have one personality during structured presentations and another when asked questions: the way the presenter dealt with the questions at the end was rude and condescending and probably gives an insight into the culture present at Uber

@nelsonc5339 6 лет назад

Wow...! Are the *investors* watching...?! this is _like_ a *Technical Debt Time Bomb* of a "stack"! Uber is going to *implode* so spectacularly... 😧

@anujpandey8753 3 года назад

too high level and too abstract. Touching so many things at a time and not diving into details. This presentation could have been better.

@DavinStewart 5 лет назад

Like, please stop saying like, like all the time.

@cantwaittowatch 4 года назад

rambling so many things..yikes!

@hoopssheaffer 7 лет назад

This guy doesn't seem like a very good engineer.

@stoneshou 7 лет назад

previous CTO of Voxer

@Calphool222 6 лет назад

Huh? He's pulling back the curtain and describing the difficult problems Uber is facing, most of which were emergent properties of being engineering purists. The ignorance of Conway's Law and a hundred other systems thinking phenomena in engineering circles is epidemic. Code slinging is an important baseline skill that everyone needs to have, but it's *not* what it takes to lead armies of technical people, and having intentionally ignorant HR managers is *not* a solution.

@tj8870 6 лет назад

It's the complete opposite. This is how a great engineer thinks. Seeing the trade-off between all the different types of methodology, technology and architectures. It is impossible to design a fully working system at a large scale. Every layer you add increase the complexity of the project exponential not matter what architecture you use or how great your starting UML was. Trade-offs are constantly added to the ones you were already facing. On top of that you are building off the previous architecture. Any flaw in the system or outdated methods or technologies only get propagated with growth. Small scale architecture is ridiculously simple relative to large scale architecture. Personally, I often find that maintaining an MVC project with 1 or 2 teammates is more time consuming and produces worse results then just coding it by myself and they are handling 2000 coders... it is crazy...

@MrHics 7 лет назад

regular ads? downvoted

@suckerformimi 4 года назад

This conference is crap

@SC1240 3 года назад

This dude can't present well at all. Decent info though.

@ralfoeldi 6 лет назад

After 15 minutes... why isn't this guy sooo embarrassed that he crawls under a rock and hides? THIS guy is responsible for anything? Sounds like a junior engineer after having completed his first project (which might be close to the mark).