Monorepos - How the Pros Scale Huge Software Projects // Turborepo vs Nx

Подписаться 3,1 млн

Просмотров 675 тыс.

50% 1

Big companies, like Google & Facebook, store all their code in a single monolithic repository or monorepo... but why? Learn how to use tools like NPM or Yarn workspaces, Learna, Nx, and Turborepo to scale your codebase fireship.io/pro
#software #js #compsci
🔗 Resources
Turborepo turborepo.org/
Nx nx.dev/
Lerna lerna.js.org/
Google's Monorepo cacm.acm.org/magazines/2016/7...
📚 Chapters
00:00 The World’s Largest Codebase
00:49 Benefits of Monorepos
01:50 The Problem with Monorepos
02:18 Yarn & NPM Workspaces
02:48 Lerna
03:30 PNPM
03:48 Nx vs Turborepo
06:10 Turborepo Tutorial
🔥 Get More Content - Upgrade to PRO
Upgrade to Fireship PRO at fireship.io/pro
Use code lORhwXd2 for 25% off your first payment.
🎨 My Editor Settings
- Atom One Dark
- vscode-icons
- Fira Code Font
🔖 Topics Covered
- Monorepo Pros and Cons
- What is a monorepo?
- Who uses Monorepos?
- Should I use a monorepo?
- Nx compared to Turborepo
- How Google's Monorepo works.

Наука

Опубликовано:

1 июл 2024

Ссылка:

Скачать:

Готовим ссылку...

Добавить в:

Мой плейлист

Посмотреть позже

Комментарии : 566

@AzureFlash 2 года назад

Finally, I can have a giga-scalable monorepo and build tools for my 100-line Discord bot written in Python

@gh05tparkourfreerunning31 2 года назад

I was actually thinking of using this for my Discord bot lol. It has a REST API to manage a roster of members in a DB and I wanted to make a website to interact with the database directly as well. Except in this case, the Discord bot would be a tool for the entire application

@donquixoteupinhere 2 года назад

Hahaha. Yeah context is VERY important as to whether this is the recommended route someone chooses, for highly practical reasons.

@Mkrabs 2 года назад

***written in a single Python file

@andrey2001v 2 года назад

And 40 of those lines are imports. This is literally most of my uni coding experience

@arctic.wizard 2 года назад

This is the way

@brennanfife 2 года назад

Software teams then: ‘This codebase is too coupled together… let’s break it up’ Software teams now: ‘This codebase is too broken up… let’s couple it together’

@overloader7900 2 года назад

When the moores law died (as in single core SISD performance, cuz its too hard to make parallel code), code performance became important again

@ronanharris8216 2 года назад

That one guy on the team : "... this goes against everything that I know to be right and true ..."

@muizzy 2 года назад

At the risk of sounding like I don't get the joke; a little context for the uninitiated: We're still decoupling as much as possible, we're just doing it in a smarter way which allows us to make simpler use of a single source of truth. (1 monorepo, but still many micro-services)

@MiniKodjo 2 года назад

do then undo is work

@dorktales254 2 года назад

Google's codebase: moaaaaai

@patrik5123 2 года назад

This is by far the best channel to get enough knowledge in order to to make it sound like you know what you're talking about.

@patrik5123 2 года назад

@@mikkelens Yea I've spent enough time in front of online courses during the various lockdowns to know how demanding it is. I've spent countless hours on React and I still feel like I don't _actually_ 'get it'.

@reed6514 2 года назад

I like these, more for entertainment than learning. My preferred learning style mainly involves diving in, writing code, and looking at documentation.

@noskillzdad5504 2 года назад

Fake it till you make it!

@reed6514 2 года назад

@@noskillzdad5504 i do this when I'm manic lol. Sometimes it's good. Sometimes not

@heroe1486 Год назад

@@reed6514 It's good to discover things and know if you want to dive deep into them, like if a video showcases a new framework which has some particularities you can't live with, if you were able to spot them in the video it may save you hours of reading the docs. Same for this one, most people probably went to the conclusion that Monorepos weren't yet a fit for their projects. These videos are handy in a field with so much (sometimes useless) novelties

@KeithGraves 2 года назад

What people call best practices in software engineering amuse me. Backend: Everything is a microservice Frontend: Use micro frontends Git: Put everything in a giant monorepo

@matheusvictor9629 2 года назад

The thing is about performance and not everything going down at once, where you store the codebase doesn't matter

@TheBillionDollarSaaS 2 года назад

Infrastructure and code have different requirements

@webentwicklungmitrobinspan6935 Год назад

no! its all about decoupling. microservices to enforce a api and maybe consider an integration layer. and you split your frontend because you dont want to compile for minutes at a time and have a 100gb node module installed for fixing a a hardcoded link in a selfservice tool which no one seems to use anymore but people manage to complain about somehow

@dadudeme 3 месяца назад

@@matheusvictor9629as linus torvalds said: Only wimps use tape backup: real men just upload their important stuff on ftp, and let the rest of the world mirror it

@oussamasethoum1665 2 года назад

You explained TurboRepo better than the creator himself, huge respect to Fireship 🎩

@andresramos7965 2 года назад

At this point, all software should feature a Fireship video in their mainpage

@oussamasethoum1665 2 года назад

@@andresramos7965 I couldn't agree more, they should pay this guy to make more understandable explanations.

@hydra4370 2 года назад

@@oussamasethoum1665 I already pay him to explain stuff to me 😭

@magnusmarkling 2 года назад

@@oussamasethoum1665 Maybe you mean WE should pay him

@forrestmorrisey 2 года назад

You're comparing apples to oranges here. Engineers are nutoriously bad at explaining their own work. Fireship is a seasoned technology communicator.

@krishgarg2806 2 года назад

Thank you for covering this. Recently vercel acquired turbo repo and I was confused what was it as I use vercel a lot.

@ultimatum97 2 года назад

As somone who works at a certain large software company with proprietary source code manager, repository and build tool I can tell you it is no fun at all. It is a nightmare dealing with dependencies when you don't have Maven or Gradle.

@JChen7 2 года назад

Sounds like someone has a case of the NIH blues.

@bobobo1673 2 года назад

"Maven or Gradle" Look with Gradle I can't comment but with Maven? are you serious? Do you know how complicated it is to simply add a project and the problems it gives looking for its dependencies? Sometimes you have to solve the problems yourself by hand (looking at the dependency tree, and that I don't even talk about projects with Wildfly is a horror) and it makes me lose time from work when all I wanted was to install a library

@not_nerp 2 года назад

would this proprietary build tool happen to share a name with a certain south american country

@ultimatum97 2 года назад

@@bobobo1673 Be thankful that atleast it downloads the versions you mention in pom. The build system I work with needs a kind of dummy project just to deal with dependencies. It then does some black magic, pulls the wrong jars in a secret location and then shoves it into the cryptic buildpath directly during environment level global build 🤣🤣

@ultimatum97 2 года назад

@@not_nerp Maybe....not

@Flackon 2 года назад

I've actually started using monorepos for a couple of solo projects, due to the ease of packaging and dependency sharing

@vinaymama 2 года назад

Just few days back , i saw the Fireship comment in the TURBOREPO launch video. Now we have a complete tutorial on it... Thank You dude

@akillersquirrel5880 2 года назад

Having recently left a company that was built around a monorepo, here's my takes: Pros: * Visibility - it's easy to see where things are used and discover stuff. * Uniformity - no need to manage versions between things used internally. Cons: * Upgradability - you can only upgrade something once *all* of its users are upgraded. * Maintenance - if you don't invest enough talent into setting up and maintaining your monorepo tools, it's easy to get to a point where they are no longer usably fast. * Speed - if a simple `git status` command takes multiple seconds to complete, nothing in your project will feel snappy.

@bakedbeings 2 года назад

Another fantastic video, just one nit pick: it wasn't super clear (to me) early on that this was about repositories for web tech specifically. When you started out with 2 billion lines of Google code I had their C++, Go etc code in mind - a major driver of Go's toolchain design was long C/C++ build times on huge code bases - but Turborepo is js/ts only.

@omarkarim9298 2 года назад

Using build tools like maven/gradle and scripting I found is the best way to go for multiple projects of different programming languages

@tmthyrd 2 года назад

Thanks for pointing this out, I completely missed this point. It's somewhat ironic that a tool written in Go to manage repos doesn't actually support Go. Does NX or Lerna support a larger set of languages?

@IxMeTutorials 2 года назад

@@tmthyrd Lerna is based on npm, so probably not.

@nathanhedglin931 2 года назад

There is only JavaScript 😂

@Zokiio 2 года назад

But Bazel that google use does actually handle multiple codebases like Java, C++, Go, Android, iOS

@VarunGupta3009 Год назад

I scouted the entire internet a few months ago to get the exact same monorepo build setup as I'm used to at Google, but here you are, who explained it all so beautifully in a video under 10 minutes. Bless you.

@webentwicklungmitrobinspan6935 Год назад

how long have u beeing coding before getting into google? is it still worth it?

@KerimWillem Год назад

You have no idea how helpful this video is and how much time it saved me. THANK YOU.

@a.c.vermillion 2 года назад

I was waiting for so long to finally see this uploaded. Thank you!

@badreddine8727 2 года назад

Thank you for the video I was confused when vercel acquired turbo repo and what it was, this cleared up a lot of things

@dmitrii_zolotuhin 2 года назад

Man your channel is pure gold.

@isaac4386 2 года назад

I feel like your tracking my Google search wow. Been researching monorepo since the weekend and now you post this 🤣 . Love it

@clementbowe7594 2 года назад

I'm probably really late on this but congratulations on reaching 1m subs!! I watch daily and always learn a new thing or two. I hope that you keep making amazing content!

@cornheadahh 2 года назад

I'm loving the frequent uploads

@akashkumar121213 2 года назад

I have been setting up a typescript monorepo for one of my personal projects ... And lets just say there are some pains you are gonna feel that you wont forget.

@webentwicklungmitrobinspan6935 Год назад

.. cant wait to get through these pains in my next vacation.

@osagiee9553 2 года назад

ive been feeling the need for a monorepo for quite some time. this is helpful.

@yodkwtf 2 года назад

So much information in one video. Huge respect. ❤️

@devbyemil5201 2 года назад

You are really doing some great explanations! How about a video about how to plan software and what tools to use? Would love to see it👍

@pablopoggio4899 2 года назад

That’d be pretty cool actually

@code_react 2 года назад

you saved my life by explaining how monorepos works. Thank you so much.

@ShawnCZek 2 года назад

Another issue with monorepos is the access management. Especially with the public available tools, you cannot easily define what directories one should have access to.

@devnol 2 года назад

Doesn't github's codeowners do that?

@qwerty-or1yg 2 года назад

@@devnol I know that they can limit access to specific repos, but not directories inside a repo. Don't think they can limit your access to specific directories inside a repo (unless I missed and not aware of something)

@mudscuffer 2 года назад

@@devnol Codeowners can limit write / merge / PR rights. But not read access.

@muizzy 2 года назад

Aside from fringe cases (like fraud rules), why would you want to manage read access?

@mudscuffer 2 года назад

@@muizzy For example if you use freelancers or outsourced development for the frontend or for integrations, but don't want them to see your proprietary backend algorithms.

@PatricioHondagneuRoig 2 года назад

Your videos are my non-guilty pleasure, I always leave having learned cool stuff and enjoying it, it's like having celery for dessert.

@yurayurec6071 2 года назад

Top quality as always 👌 waiting for your video on webpack module federation, another hyped tool on the market

@IsaacAlcocer 2 года назад

You just give me an idea for a personal project, kudos... this is a nice video to watch.

@alexIVMKD 2 года назад

Your videos are spectacular, congratulations in advance for 1M

@kettenbach 2 года назад

Frikin love you man! It's been 84 years. That was frikin hilarious bro. 💪❤️👍

@abh1yan 2 года назад

Great video Jeff, really wanted to see this.

@erkinkurt6799 2 года назад

I was searching for this to decide if I should use this approach or not. OMG either you can hear me or you have amazing skills to access my search history

@Balance-8 2 года назад

you literally read my mind on the video I wanted to see. Incredible!

@abhinavrobinson2310 2 года назад

Amazing,

@0xedb 2 года назад

turborepo is great. I just want a demo where everything is done from scratch instead of the boilerplate to better explain it.

@AndersonSousa33 2 года назад

me too

@kevyyar 2 года назад

That is my only problem with Fireship. Everything else is great.

@floffah264 2 года назад

their docs have a good step by step process of this and their examples help a lot

@sakthisanthosh0103 2 года назад

You're videos are great. Congratulations for your 1M subscriber mark. 🎉🎉🎉

@GagandeepSingh1984 Год назад

So much to understand in this build flows .. thanks for sharing it..

@thepianist6425 2 года назад

Would love to see a video covering Micro Frontends. Having said that, always a big fan of the quality of videos. 10/10 college degree material right here. I should start adding fireship io under the education section of my resume now.

@DominicanRepublicInvestment 2 года назад

Second this!

@muizzy 2 года назад

I work with a micro frontend every day (though admittedly on the backend). Do you have any specific questions? I may be able to answer them.

@ThiagoVieira91 2 года назад

@@muizzy Do you think micro frontends improved your development experience in you project? How did micro frontends improved your team workflow? Can you talk about some trade offs in comparison to currently traditional frontend architectures?

@JChen7 2 года назад

Watched "Java in 100 seconds". - Proficient in Java

@essayemyoung4009 2 года назад

Would check out Jack Herrington’s videos on this (e.g., ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-lKKsjpH09dU.html where he goes through a React example in detail)

@williamchamberlain2263 9 месяцев назад

0:30 I like that the codebase is elvoving blindfolded _and_ being pulled side to side by different teams _and_ will irrevocably topple over with crushing force after just one pull too hard or in the wrong direction. Great metaphor.

@jonasbroms 2 года назад

THANKS! Great video! I have to test this now on my monorepos. :D

@dlysele 2 года назад

Quality video that everyone should watch

@AlexEscalante 2 года назад

I think I am trying Turborepo soon! Thanks for your videos!

@nathanbrown19 2 года назад

Worked in both mono and traditional repos. Monorepo removes so many dependency and versioning headaches.

@archmad 2 года назад

It can go to an opposite way. 1 app only supported old version while the other uses the new

@conororeilly5492 2 года назад

I've got 3 apps, possibly a 4th coming if i finish the current project soon all along the same theme, i might try creating a monorepo for them to share some of the common functionality. It's most likely overkill at this stage but each app was to learn something, so i may as well do it and see what i learn

@OptrixTV 2 года назад

i knew this was coming when i saw you put "nice" on Vercel's stream on Turborepo :D

@andyhall7032 Год назад

I like how the narrator sounds like text-to-speech software...it's one of the many appealing aspects of this channel.

@Filaxsan 2 года назад

Our man Jeff becoming funnier and funnier (and informative!) every video. Way to go Jeff, love you

@amsterdamislove_123 2 года назад

Amazing & concise explanation.

@Viviko 2 года назад

I usually only use Monorepo on things like Microservice based systems. I kind of feel like having one repository for ALL your products kinda is overkill. But I guess not, after watching this.

@AhmedKachkach 2 года назад

What's overkill is going through the effort of maintaining different repos for a small project. If your web app, your mobile app and your data processing jobs all use the same data representation, why not store everything in the same repo and keep everything in sync?

@FilledStacks 2 года назад

Awesome vid. Good idea to build something like this for flutter as well

@chinmaykabi 2 года назад

Melos?

@FilledStacks 2 года назад

@@chinmaykabi We use melos but it's not a build tool, it's more accurately defined as a "script coordinator". You're bundling commands for multiple packages under a single command. No caching, no smart rebuilds, etc. It will always execute all commands that you defined.

@saurabhjainwal4604 2 года назад

We use nx at our firm, it’s really cool and definitely helps maintain our mono repo. 😀

@middlefloor 2 года назад

🙌

@skyhappy 2 года назад

@@middlefloor Why are you raising your hands? Are you catching a basketball?

@middlefloor 2 года назад

@@skyhappy yes!

@skyhappy 2 года назад

@@middlefloor well this is a tech channel not a basketball one

@johngc2010 2 года назад

@@skyhappy Jeffrey Cross is one of the maintainers of Nx...

@sessionswithtemitope 2 года назад

Thank you for this. It's what's I need right now

@anubhavbansal8922 4 месяца назад

too much of information in just 10mins that's why I love you

@krishgarg2806 2 года назад

Time to use turbo repo for my todo app.

@DiegoBM 2 года назад

So two questions mainly. How do those tools help solve the massive git stack of changes and vscode to keep up with them, and do nx and turborepo override lerna? Or is lerna supposed to work in tandem with any of those? From what I can read in turborepo's site it seems like it can do all that lerna does?

@ThaChillz 2 года назад

Thanks Jeff, this video was very cache

@jedcal 2 года назад

i love this channel

@AbdulbakiDursun 2 года назад

HOW. DO. YOU. KNOW. ALL. THIS.?

@toilet_man_G 2 года назад

Just need this！ Thanks so much for the explanation

@officialismailshah 2 года назад

Sir you are just awesome pro pro level knowledge.....

@JonathanAdami 2 года назад

"You might know him with packages like Formik" Telling me that after 3 weeks of trying to solve issues with fomik and eventually having to refactor my entire code base to use react-hook-forms (which is MUUUUCH better) makes me wanna ditch Turborepo even if I know nothing about it hahaha Cheers for the video!

@jztsaurabh8111 2 года назад

Formik has its own issue but it created the whole form library ecosytem

@codevev 2 года назад

That's literally what we've been doing in the last couple weeks haha

@albert21994 2 года назад

Get the Linux DX with remote containers ;) you can even clone the repository to the docker instance to get the speeds

@nanotichorizon9644 2 года назад

Thank you so much for this content!

@lixou 2 года назад

Me finishing after 3 days my MonoRepo setup. 2 hours later, this video got uploaded….

@vikrantshah1633 2 года назад

Thanks for fulfilling the request

@nmanikiran 2 года назад

Please build a course around it (mono repo) with all the tools you have mentioned. take 1 big project and architect the same in all tools like nx, turborepo, lerna, yarn, blaze, webpack module federation etc

@davidzavala9414 9 месяцев назад

Hi, great content!!! Do you have any video where you explain the differences between monorepo and microfrontend? and if it is possible to use both?

@wheytomuchforher 2 года назад

Serious question. Why would I use this instead of NX's built in CLI tooling to run the "affected" aspects of my code base. I use NX for both of my side projects

@syamjulio3685 2 года назад

I'm interested in this as well.

@middlefloor 2 года назад

Turbo would probably be used instead of Nx, not alongside it. If you're already using Nx, turbo won't do anything for you that Nx doesn't do.

@felixoghina553 Год назад

Setting up a monorepo for all my unfinished projects

@rayanez 2 года назад

Awesome video, thanks for the info. Do you know if there's a similar tool for Java codebases? or, I don't know if it makes sense, heterogeneous code bases?

@middlefloor 2 года назад

Gradle's probably the closest Java-first tool. Most of the tools in the video can be used for multi-language codebases, but Bazel and Nx have plugins that make it easier to work with different technologies/languages.

@wxrld1 2 года назад

perfect, now I just need to start a big project. maybe next weekend. yeah.

@KevinVandyTech 2 года назад

I just converted most of my company's front end code to Nx. It's been a game changer

@middlefloor 2 года назад

Awesome to hear!

@mylifegamer1 2 года назад

Hey, I used that in creating a blockchain 😃 Thanks

@BlackdestinyXX 2 года назад

Really interesting, thanks

@uwuLegacy 2 года назад

YES finally monorepos are getting more coverage

@MrMeszaros 9 месяцев назад

Cool - like Gradle! Except that build caching sharing - that sounds like fun

@michaeldausmann6066 18 дней назад

Just finished refactoring my big messy Nuxt + background processes app into a clean and efficient monorepo with turborepo. Should have done this from the start, so much better

@midas6659 2 года назад

Best tech RU-vidr on the planet yo

@nitroflap 2 года назад

Funniest thing is that today I was thinking exactly about monorepos & multi repos. Fireship reads our minds.

@falxie_ 2 года назад

Had my eye on Turhorepo, didn't know they went open source!

@lambrohan 2 года назад

I knew it coming, since you starred the repo 2 days ago!

@NezzyLawd 2 года назад

Been watching for ages, but not subscribed. I don't know how!

@AnimusAgent 2 года назад

One thing I felt it went missed on this video was about the easy integration with microfrontends and testing tools with Nx.

@theatypicaldeveloper 2 года назад

you should always think big and leave an extra space in your app (if possible) so it's growth-friendly great vid, as always!

@andreb.1352 2 года назад

you should always think small, and iterate it to the better.. otherwise you end up in perfection and you need a lot of time instead of just shipping a lot of small parts

@zyriab5797 9 месяцев назад

Keep it lean and modular

@Ultimacho 2 года назад

I wonder what are the main advantages of using Turborepo against the standard yarn workspaces? We are using the latter, but we will move to a new tool if we see obvious benefits. We only run one app at a time during development, so the process doesn't seem to differ much from the one described in the video.

@agentsamumu 2 года назад

You talked about microservices in that, can you do a video on it? That sounds really interesting

@junzhengca 5 месяцев назад

Monorepo coupled with almost perfect IaC is a god send, it gives amazing visibility to everything. Yes it is complex, and the on-boarding learning curve will be steep, but once you are familiar with it, you don’t even need to leave your IDE to find anything.

@racecarjonny8460 2 года назад

I have worked with monorepos built with lerna. OMG are they difficult to maintain!

@besm_a Год назад

Thank you for the video :)

@bruceaxelrod8291 2 года назад

Monorepos are great but how to control visibility of the code & access to the code if everything is in one repository. Like some departments are not allowed to see specific directories/code & things like that

@homeofmotivation2339 2 года назад

Yes, I was also thinking about that. Everyone can see everything

@reed6514 2 года назад

i'm a php dev with about 10 foss packages, many depending on eachother, and composer (php's dependency manager) works great for me. & I think gitlab has a built-in feature for mono-repos, though idk how feature rich it is.

@reed6514 2 года назад

Okay, i have quite a few more foss packages than 10 ... but i estimate only about 10 of them are useful ... i just open source most stuff i write.

@wie5493 2 года назад

You’re amazing! How can you put out such high quality videos so quickly?

@skyhappy 2 года назад

His amazing intelligence and work ethic

@andresramos7965 2 года назад

My man fireship has a brain so dense, that rumors says in the center there is a blackhole

@MikePatterson8831 2 года назад

The weirdestthing for me is seeing big companies modify distributed version control systems so that they operate more like centralized version control systems. All the problems with git scaling is from git's own setup. Really glad I no longer work at one of the big companies mentioned here. Central version control systems manage monorepos so much better in terms of speed and out of the box access management. And this particular company's proprietary build tool had such poor documentation that it was almost tribal knowledge when you needed someone to setup the build pipeline or fix a bug. Great video!

@titzko987 2 года назад

expected something about an opensource java logging library today

@alegian7934 2 года назад

great vid! is there some similar tool for much smaller monorepos? for example if I wanted to merge my frontend and backend repos (one of which isnt a node app), what's the best option?

@middlefloor 2 года назад

Nx, Lerna, and Turborepo can all be used in a lightweight fashion, even for a repo with just a few projects.

@maxencedc 2 года назад

Google dev : changes one line of code Piper : here, deal with those 15131 merge conflicts

@haxney 2 года назад

You only deal with merge conflicts in those specific files you are modifying. There is (almost) no branching, so you don't have to deal with merge conflicts from other people's commits. If someone changed the API of a library you depend on between when you start writing a change and when you submit it, then you'd have to deal with that, but that's a reason to make frequent, small changes. Ideally, each commit to head should take under a week to write and be only a few hundred lines of code at most. Obviously, that's not always possible, but it's a good starting point. If I make a change to a common library which would break other people, then it's my responsibility to provide a migration path to the new behavior. Since it's a monorepo, I can easily find all of the callers of my library and either fix them myself or ask the owners of those files to fix them.

@maxencedc 2 года назад

@@haxney Very interesting, thanks