Explanations like "yes, this is good practice but it does not apply to my project because we have ..." are my favorite :) 10 years ago I heard it when people were saying that we cannot write unit tests because we are writing UI, now I hear that we cannot do continuous delivery because our domain is too complex
The domain might be too complex. However, it is probably too complex because no one is willing to refactor it. Years of adding layers on top of layers and with the goal to deliver something fast tend to produce unnecessary complexity. Refactoring is one of the most important activities which managers usually do not want to spend time on.
Under these definitions: 1) Only the insane would do continuous deployment. Unless the only changes you ever make are pure refactoring with no (intended) side-effects visible to users, databases or other external systems then you're in a situation where you're either a) the only one testing your own code before it hits production, b) deploying completely untested code while you wait for someone else to "continuously" add/update tests or c) both. I guess if you also have an automated way to detect errors in the wild and rollback quickly this might be somewhat mitigated, but that's an extremely complex ask. 2) Only the insane would do "continuous" anything. That implies that every task can be broken down into
Many companies deploy multiple times a day, and i doubt that they are insane. I strongly suggest reading about TDD and feature toggles. Those will answer some of your questions.
@@DevOpsToolkit > Many companies deploy multiple times a day With how many people, and across how many projects? Is this just a measure of dev team size? If individual developers are all managing to deploy separate pieces of code without turning into an absolutely nightmare of spaghetti, I'd be more interested. But if turning a 2 day deploy cycle into a "continuous" one day deploy cycle amounts to little more than doubling the team size, that's not really helpful in my opinion. > TDD Is helpful, but it only inverts the problem. Now your tests are merged first, which sets the pipeline on fire because the code they're supposed to be testing doesn't exist (or is incomplete). By your description that means everyone now has to drop everything they're doing to complete that one single feature before the day is out, I guess? I'm sure you can work around that by having the mainline developer and the test developer working extremely close and ensuring that only a small number of tests are pushed at a time so that the mainline dev can complete the code and put out the fire before EOD rinse and repeat, but that's heading into the "massive overhead" territory. Basically a continuous stream of half-baked code getting pushed for no purpose beyond retaining a label. > feature toggles Just.. ugh. Now you have a stack of flags that you have to maintain, have to be sure you're checking all over the place when you implement a new feature, and then have to go back and delete again later (assuming you don't want your code to become an unmaintainable pile of garbage). The end result of which is pushing something that is explicitly not being included in the product (assuming your checked your flags right) for the sole purpose of saying you pushed something. If you don't want it in the product yet you could just.. not push it until its ready. Certainly words are just words and you can define things however you want, I just don't see this particular definition as being terribly useful. It adds a load of overhead and risk with very little apparent benefit. I like the automated everything. Pipelines should absolutely be strong enough that a dev can push a change and have it go all the way (I prefer having the final business decision button, but maybe that's because I operate mostly in B2B world where updates often require customer sign-off and the like.. probably less important for B2C sites where there isn't a potential contract breach should things go down for 20 minutes). Its just the arbitrary "every single day" cadence I disagree with. I just can't wrap my head around the idea of intentionally pushing broken code and just crossing your fingers that you were careful enough not to expose it until you can get around to finishing it at some future time.
Great video, especially like the steps to verify if we really run the real CD :) As a side note, maybe it is not obvious, but being able to deploy to production any of the last 10 green builds, means that we also need to have automated database rollbacks, as maybe the chosen build is older than the one already in the production!
Rolling back DB is often impossible since that might mean loss of data. You could, for example, add a new column, and rolling back might mean removing that column and the data in it. Even worse, it might be a completely new table. The only "real" option is to NOT roll back DBs, at least not in production. That means that changes to the DB need to be backward compatible so that they work not only with the new release of the app, but also with older releases in case they are rolled back while the DB stays intact.
Great video, but actually whether or not something gets pushed to the "main line" everyday highly depends on the team size. I'd not go as far and say this needs to happen every single day in order for it to be called CI.
In the past, I tended to use words like f*ck, sh*t, and other a lot. I had to stop using those so it ended up being "silly" instead (without planning to use it).
I have to say something. Continuous Delivery does not imply that *every* step is automated. While most of the checks are automated, there should still be room for manual testing within the pipeline, especially if it's a user facing product. CD is about workflow automation (as in SDLC process), not to defer all thinking to computers.
I would say that CD is about delivering continuously. If manual testing or any other operations do not prevent you from making it continuous (end frequent) it is OK. In most of the cases it is fully automated simply because the required speed is hard to accomplish manually. Now, if you can, let's say, deliver daily with manual testing you are indeed doing CD.
@@DevOpsToolkit True. Maybe a better way to say it would be: "CD is a practice where we are codifying our knowledge and current understanding by leveraging automation in all things; this enables us to continuously deliver working and safe software with minimal lead times."
I would go as far as to say that there is no need for the first part of that definition. The point is that we are continuously delivering production-ready releases. If it can be done without automation, it's still valid. It's not likely but, if it can be done like that, it is still cd, as long as if it happening continuously and frequently.
Soooo.... If I have full automation from merge to deployment, but I am only doing releases once a week, what am I supposed to call it? Fully Automated Deployment?
I don't believe that DevOps is about tech so I never explored such a subject.vi think it is first and foremost a cultural change and mostly focused on creating self-sufficient teams that, among other things, can operate their own apps. From that perspective, I do not think there are direct strategies. What I do think is that platform engineering complements DevOps very well and for that there are indeed principles and strategies involved. You might want to watch ru-vid.com/video/%D0%B2%D0%B8%D0%B4%D0%B5%D0%BE-j5i00z3QXyU.html.
I mean at individual level. A day of work deserves to be pushed to the mainline if that work is done in a way that it is safe to use it. Now, keep in mind that might be the goal, but not necessarily something that everyone can do right away.
So if you lets say have Continues Delivery to Blue/Green deployment in which you reroute traffic. Do you consider that as Continues Delivery since you need to reroute traffic?
Viktor, please can you adopt me and take me to Spain?! I cannot live in this IT wilderness any longer. The clients, they don’t listen. And the devs tell me they are doing Continuous Integration because they use bamboo to build their long running dev branch off a feature branch off a…..!
Hi, Great videos, i'm going to watch all. Can you please do a video on gitflow + devops/pipelines and (why not how that would work with argo) from dev perspective? Like how would someone test their feature/bug branch, how are changes reaching dev/qa/staging/prod envs if they are on different branches and must go via different pipelines? How/where would a manual QA test a feature (in terms of devops/env process)? Update: If everything goes to the main line, does this mean that there is one pipeline which deliver first to dev, then qa, then staging then prod? If yes, why do we need dev/release/main branches?
@@DevOpsToolkit Thanks. Also, how to handle versioning/tags, how to automate tagging/versioning, when and which version goes where in pipleines? And what do you use for dynamic environments, what do you use to create such?
@@n4870s For versioning/tagging, I tend to create releases on PRs and, by default, increase the patch version (SemVer). If the PR contains a label that a minor or major version should be increased, then the patch is reset to zero, and a release with major or minor increased is created. Further on, that same release is gradually promoted to all the envs. For dynamic environments, I use either Shipa/Ketch or, when doing GitOps, just add/remove that app from a dedicated repo and let Argo CD or Flux take care of the rest.
Do you have any tips for CI/CD (or as close as we can get to it) on a very large and somewhat coupled separated monolith? Thanks, love you videos, they are super helpful!
The logic and the process is, more or less, the same, no matter the size of the application. That being said, large monoliths tend to have additional complexity due to years of accumulation of "random stuff". What I'm trying to say is that most CI/CD pipelines are doing the same thing on the high-level, but that the details might vary greatly. If you can give me more info about your app or tell me what is the challenge you're facing, I can probably come up with a more useful suggestion.
@@DevOpsToolkit We have a requirement where we have multiple environments (dev, sandbox, qa, pre-prod, prod) We are using git branches to segregate those environments, and its getting messy. How we are doing it now, is.. we develop and merge stuff to the develop branch, that deploys the pipeline onto the dev environment, then we test manually on dev (QA resources test the features), raise a PR from develop branch to the sandbox branch which deploys it to the sandbox environment. The entire process is manual and no where near what CICD defines What would you suggest for our use-case?
@@chrisjaimon9137 One of the things I would suggest is to stop mixing releases with code. Normally, we create a release once, and then we deploy it to multiple environments (dev, QA, staging, sandbox, etc.). If the same release is deployed to multiple environments, there is no need for all those environments to have separate branches (unless you're applying GitOps in which case there would be separate repos for those envs and unrelated to the code).
@@DevOpsToolkit So what you're suggesting is to promote artifacts (build artifacts) instead of a commit or code And have a single release and single branch. And in case of hot-fixes (lets say on production), we create a branch from the master branch, add our fix and create a release and build artifacts for the same. Then we merge the hotfix back to master as and when suited. Am i right?
@@chrisjaimon9137 Correct. Create branches from the mainline, do whatever you need to do, and merge them back to mainline when it's ready for production. Create intermediary releases from those branches and deploy the same release to as many environments as you need. P.S. Branching strategies might be a good subject for one of the upcoming videos.
