Day-4 | Terraform State DeepDive | Remote Backend | State Locking  

Hi Bro. Thanks anna for the free courses , i got a good job in one of the MNC company as senior Cloud infrastructure as service role. If every person in IT dedicates some free time for others instead focusing on scraping money from others for time, no one would be jobless. Your initiative is helping others including their dependent families, no other employee would do same as you. Thanks anna.

🎯 Key Takeaways for quick navigation: 01:42 📋 *State file in Terraform is where Terraform records the information of the infrastructure it has created.* 03:29 📋 *State file is important because it helps Terraform update and manage existing infrastructure instead of creating new instances or resources.* 07:36 📋 *Advantages of using the state file include updating existing infrastructure and destroying infrastructure in a controlled manner.* 08:05 📋 *One drawback of the state file is that it records sensitive information, such as passwords, by default.* 09:17 📋 *Storing the state file on a personal laptop or any vulnerable device can compromise the security of sensitive information in the state file.* 10:23 🛡️ *Real-world scenarios involve multiple DevOps engineers collaborating on a Terraform project.* 12:12 🛡️ *Storing the entire Terraform logic, including the state file, in a Version Control System can compromise sensitive information.* 14:31 🛡️ *If the state file is not pushed to the Version Control System, Terraform may not know about updates and can delete the corresponding infrastructure.* 16:35 🛡️ *Using a remote backend solves the drawbacks of storing the state file in a Version Control System.* 19:04 🛡️ *With a remote backend, the state file is automatically updated in the external resource (e.g., S3 bucket) when applying Terraform configurations.* 20:41 🛡️ *Storing the state file in a separate remote backend, such as an S3 bucket, enhances security and prevents sensitive information from being exposed in a Version Control System.* 21:09 🛡️ *Multiple remote backends can be used, such as S3 or Azure storage, depending on the cloud provider being used.* 23:13 🛡️ *With a remote backend workflow, DevOps engineers can clone the GitHub repository, make changes locally, verify them with `terraform apply`, and raise a pull request to update the remote state file stored in the S3 bucket.* 27:30 📂 *Before running `terraform init`, there will be no state file available. After running `terraform apply`, the state file will be created and populated with resource information.* 29:03 📂 *The populated state file contains information about the created resources, such as the instance ID, public and private IP addresses.* 30:46 🛡️ *Storing the state file in a Version Control System is not recommended due to security risks and the potential for forgetting to push updates to the state file.* 31:16 🛡️ *Terraform can show the state file using the `terraform show` command.* 31:29 📂 *Deleting the state file can cause Terraform to lose track of the created resources.* 32:10 🛡️ *Using a remote backend like S3 requires creating a `backend.tf` file with the necessary backend configuration.* 33:24 🛡️ *Copying and pasting backend configuration from the HashiCorp documentation or using extensions can simplify the process.* 35:16 🛡️ *S3 bucket creation can be done manually or with Terraform, ensuring unique bucket names.* 37:02 📂 *The S3 bucket name and region are required in the backend configuration.* 38:38 🛡️ *Initializing the project with the backend configuration will create the S3 bucket.* 40:29 📂 *The state file will be created and stored in the S3 bucket instead of locally.* 41:37 📂 *The state file can be stored in an S3 bucket instead of locally.* 42:09 📂 *Using a remote backend allows storing the state file in S3 and accessing it from there.* 42:38 📏 *Modifying the code no longer requires handling the state file manually.* 44:13 🛡️ *Terraform uses locking mechanism to prevent conflicts when multiple people try to update the same project simultaneously.* 45:22 🛡️ *DynamoDB can be used as a locking mechanism for Terraform projects.* 48:15 🛡️ *Adding the DynamoDB lock details to the backend configuration enables Terraform to handle locking.* Made with HARPA AI

Exactly you explain very well like to become zero to hero ☺👍 thanks a lot for such a great inputs and making useful resources... Amazing👍

Hi! Thanks for free but very valueble information. I have had many technical interviews. Trust me, almost in every interview they asked about terraform state file. " What is the best managent policy for statefile?" "How to keep secure terraform state file?' etc.... Greatings from US.

Hi Abhishek, At the end of this series, please let us know about the common issues/ frequent issues faced in terraform and interview questions too.

Hi @ Abhishek sir, Great explanation, Actually im azure guy and i watched all the playlists and i implemented same in Azure Cloud , i got expected Results without any issues while implementing things on azure using terraform backends and modules and with using variables , by saying Thank's is very small word for the wonderful content you delivered 😍✌ 🎉🎉 I'm very happy while implementing same things on Azure by getting expected Results with out Any Errors 🤩🤩🤩,,, Thanks a lot sir 🥳🥳🥳

Gr8 explanation about statefiles and remote backend from end to end.Thanks.

Thanks a lot bro i spend lot of money but no one explained the way u did..You are a gem..may god bless you bro ❤

Guys do not forget to always hit the like button. And subscribe. Hi bro @AbhishekVeeramalla greetings from Africa Cameroon

after seen this video i am first time understood the workflow of statefile and his mechanism.

Bro, you made this look so easy and very cool for interview preparation. Thank you so much

Great session Bayya 👌..Previously I had listened terraform from two resources but still lot of confusion on these state and remote backends ...Now everything is crystal clear ...Thanks Bayya

Thankyou so much #Abhishek for this valuable tutorial for free it's very informative and easy to understand for me, very helpful 👏👍

thanks GURU-JI for teaching these valuable skills

Great Work @Abhishek. really a refresher course for me. Hope you will share with us the case of using Terraform cloud as remote backend

Thanks Abhishek!! Able to implement this locking with dynamo db and s3 as remote state backend, It was good hands on practical.

Thank you Abisheek, For spending your valuable time with us. It will really help a lot of people around the globe

Thank you Abisheek, For spending your valuable time for us. It will really help a lot of peoples in globe

@Abhishek.Veeramalla - Super very clear..

Great sir.. neat and clean explanation.. everyone can understand... Thanks 🙏

Great session ,Thanks Abhishek for your great work

good explaination

Hats-off Abhishek for wonderful explaination and practicals.

Hurray, I was waiting for this video and completed with good understanding Sir..thank you so much for your time.🎉

Thanks for the fantastic video-your clear explanations made everything so easy to understand! 👏

Thanks for the consistency and efforts ❤

Really ah wonderful explanation Abhishek , Thankyou much!

Hi Abhishek Bhaiya.. At the end of this series please make one video on end to end project on terraform like creation of multiple AWS services...

Hey Abhishek, Please consider creating one Production kind of eks cluster setup using terraform and using all the concepts we are learning in this series.. That would be really helpful to get idea on where to use what i believe.

00:06 Terraform state file and backend 02:43 Using the State file in Terraform 07:36 Advantages and disadvantages of using the state file in Terraform. 10:07 Sensitive information in state files and potential risks 14:59 Using remote backend in terraform to fix the drawbacks of state files 17:19 Store terraform state file in S3 bucket for security and easy access 21:37 Explaining the workflow of updating AWS resources using Terraform in interviews 23:55 Configuring Terraform and AWS in Code Spaces environment 28:12 Executing the terraform apply command creates a state file and updates it with the resources being created. 30:19 Creating a Version Control System-supported project in Terraform 34:35 To create a bucket in AWS S3, you can use Terraform or create it manually. 36:45 Create a unique AWS S3 bucket for Terraform 41:37 Terraform project modified to store state file on S3 bucket 43:48 To avoid conflicts in the terraform project, locking mechanism is important 47:58 Implement backend mechanism with DynamoDB locking. 50:05 This is a simple code that can be executed with the help of documentation and the visual studio extension.

Hello Abhishek thanks for the Zero to Hero Terraform Series. Could you please include a few real-life challenges which could be very beneficial for all people who are learning from you.

Best man for devops🙌

Day 4, lesss gooo! 🙌🔥

Beautifully explained 👏

Hi sir, one request... By the end of series... Plz make full project of how to create real-time used vpc ,subnet in Aws infra using tf.. Thanks in advance

I am going to master in Terraform by you sir. 😇

hi annaya !!! actually i paid for courses but ur courses are more than those courses .i watched terraform in one stretch and understood the concepts!!! thanks anna for ur help!!!

Hi @abhishek, The way you explaining the content is really good and must appreciate. at the END of this terraform course, are you going to provide video to set-up production ready EKS cluster in AWS for our understandings ? Thank you, Mahendra.

Hi abhishek Thanks for your all effort, in real time how the resources will create , whether applying infrastructure using apply command or they use pipeline for infra creation.... If yes then which case we need to create pipeline ams

Thank you so much Abhishek ❤😊

youre a great teacher

thank uu bro for this seesion very neat explanation and pratical too

Amazing content as always...forget ppl who r preparing for interviews even working devops ppl are also learning from ur videos that much of relevance is there in ur teaching...i have one question...what is the default behaviour of terraform when we dont create dynamodb table and state lock? how the situation is different before applying dynamodb state lock configuration and aftermath of it?

Hi Abhishek. Initially you have created S3 bucket and later we deleted statefile. then again we tried to create S3 bucket with same name and configuration with out state file. and we used remote backend to store state file in S3. But if we use same name and configuration which already used we'll get error right. stating Resource already exists. Can you explain on this.

Hi Abhishek sir, is it possible to use another database resource for the locking system instead of DynamoDB? Or is DynamoDB the best option for the locking system?

Hi Anna...today are you posting the day-5 vedio?

Hi Abhishek, Thanks for the amazing videos of terraform. I am learning a lot with that. at the time stamp of 19:50, the solution for the second issue with local state file I got that, but can you explain a little bit more what exactly the problem is. I got little confused with that issue.

Hi Abhiske, I hvae few doubts on terraform 1. how can we give the IAM policy to the S3 and instances 2. Can you please explain the how can we create the VPC, Subnets with CIDR . CAN YOU MAKE A VIDEO .

Very good explanation Abhi

As the other engineers make new changes, statefile will keep on updating. We can enable s3 versioning and can revert back if someone has made some incorrect configurations with the infra. Am I right @Abhishek.Veeramalla??

@ 42:43 As you mentioned no one can access the statefile but when a user execute the command terraform show, it will show the statefile details so anyway the user can see the statefile details right? Got this question, am I missing anything?

మన వరంగల్ అబ్బాయి యూట్యూబ్ లో క్లౌడ్, devops పైన ఉచితంగా వీడియోలు అందిస్తున్నాడు అని చెప్పా కదా., Topmate అనే సంస్థ ఇతని కంటెంట్ మరియు కష్టాన్ని గుర్తించి న్యూయార్క్ నగరంలో ఉండే టైం స్క్వేర్ బిల్డింగ్ పైన అతని ఫోటో ఇలా present చేసింది.,

Thanks Abhishek

Hi Abhishek, I request you to make an in-depth video on the Kubernetes cluster step-up on the Ubuntu ec2 Instance there is no proper video on RU-vid.

Hi, Great video and explanation, one quick question is it good to have aws s3 configuration and resource information in same repo ?? beacause while destroying it was giving error and it should not delete s3 bucket as it consist state file.

I wish you would have chosen Azure cloud instead. :) Thanks heaps anyway!!

@abhishek, for eg, we created multiple resources using terraform like multiple ec2, vc, sg. I we can delete a specific resource using terraform destroy command? if we run terraform destroy will it destroy everything which was created previously ? Please help me to understand

Hi @ Abhishek sir At end of the series plz give some interview questions and answers.

Hi Abhishek, your content and method of teaching is good and i personally like that style, so thank you. I have one questions on Remote Backend, what if there are two Devops person access the same file at the same time and during update there could be a chance that second person could miss the update or over ride first persons changes. Is that possible and if YES then what is the solution?

❤ love it

Excellent 😇

Hi Abhishek, in timestamp@42:00, when you say, no one will have access to state file/backend.tf, but when devOps is creating a resource, wouldn't the state file required and visible at the root of the repository? Terraform must know the presence of the state file to avoid duplication of the resource. Correct?

Super sir

bro do one video on postgresql state locking

Question - How does the ".tfstate" file in the S3 bucket get updated in these cases: 1) Another developer raised and got his PR merged without doing 'terraform apply'. 2) Another developer did 'apply' the code, then ran 'terraform destroy', and then got his PR merged.

very good series Abhishek sir❤ if we are following this to stored state file in s3 and locked it with dynamodb, if new member of terraform team wants to access state file , then how he access it.. please explain this sir.. Thank you🙏

Did hands-on on this. And at last tried deleting all resources. But it is showing as destroy complete. But I can see ec2 instance and S3 bucket still running. May I know why

🎉🎉🎉wow

well explained💯

Hello abhishek ! As we are deleting the state file on the first apply then how we can destroy the created s3 and dynamodb without statefile? and the second doubt is there are two instances are creating rather than one ...can you please explain ?😔

Hi Abhishek please make interview questions on all devops tools and AWS cloud

How do we remove lock from remote state file from dynamodb lock table?

❤❤❤

Even if we store in git hub repo and forget to push statefile.. cant we always fire terraform refresh command once before starting any changes.. then there wont be any negative side of using statefile in git? Am i right?

Hi bro, Thanks it was a wonderful section, I have tried implementing the lock and backend state, but I could see a file called terraform.tfstate.backup after using the backend as s3. As it contains the same information, Is there is any to block that, Am I missing something!!

When we moved state file to S3 and linked locked file to Dyanamo DB , my assumption was .terraform.lock.hcl shouldn't be created in local as locking is taken care by Dynamodb (Like the way i dont see state file ln local ) . However , I do see lock file in local . I created new folder and tried doing same , still lock file is getting created in local . -- Can you help understand this ?Im i missing something?

As suggested by terraform, can we go with option suggested ? will solve if we miss configured ? Initializing the backend... Error: Backend configuration changed A change in the backend configuration has been detected, which may require migrating existing state. 1. If you wish to attempt automatic migration of the state, use "terraform init -migrate -state" 2. If you wish to store the current configuration with no changes to the state, use "terraform init -reconfigure".

Also anyone could suggest about the s3 bucket and dynamo DB table deletion steps as well

awesome explainnation 😊 . can we store state file in database?

what to do if we have to download the Huge size State file that is present in S3 or Azure Storage Account ? apart from downloading manually from bucket ? Do we have any other options?

Hi Abhishek, The lock file is pushed to remote n not on local only so it will be able to use lock mechanism if simultaneously t-apply is done? Also in our project we use terragrunt and the explanation is given by the architect on his LinkedIn post i wish you code read n explain it to me in simple words..i cant paste screenshot here but he has written about backend cant be modified without init and that terragrunt allows module level statefile

One doubt, We are using S3 to store state file and dynamo db to maintain locking, Does state file also will be stored in Dynamo db or we are using dynamo db only for locking. Just we clarification on dynamo db some more.

First to comment. Thanks sir

When i used terraform with bitbucket, when i pushed my branch to remote repo then i didnt need to separately edit and push the statefile…. Am i missing something?

is there any mechanism to execute the resources and files order wise, instead of deleting the state file multiple times

Hello Sir, Can we configure NFS mount as a remote backend?

Hi Abhishek, I have doubt at 20:54 , What if DevOps engineer pushes the Terraform code with out executing it.? He updated the infrastructure but didn't execute "Terraform apply" and pushed it to repo. what happens in that case? Does it lies in the hands of PR reviewer to check ?

Hi, As the state file contains sensitive data and it will be in s3. What about the file providers.tf which contains aws_secret and aws_key ? How can I secure it ? Can I create it and after the first terraform apply command, delete it ?

Hi Abhishek, first of all thanks for amazing series. I have few doubts in this video. 1) I don’t understand the flow how statefile is being stored in S3 bucket. You said when we run terraform apply then statefile is generated in s3 bucket. But what if I run terraform apply and then my PR got rejected? Basically I wanted to know how this remote backend mechanism will work in case of feature branches? 2) What if I just wanted to try out some things and run terraform apply several times. How statefile is stored in that particular scenario?

1)Can we create a backend remote as module and module it as in the outer amin.tf file 2)why is the terraform destroy is not happening for the backend s3 bucket if we manually destroy it then after sometime if your try to apply it it is showing error 404 why this sir??

Hai Abhishek small question For ami id and subnet id we have see aws UI only, without opening can't we do ??

sir my terraform project create resources on AWS cloud but couldn't delete or wipe out it by running the command "terraform destroy". please guide me sir.

abhishek sir can you please make a video on mostly interview asked questions like how to explain about the projects that you worked on please sir

Thanks for ur video bro. It helps me a lot to understand about terraform state file. Actually I've one doubt in backend.tf file. When I configure backend in terraform, state file successfully moved to s3 bucket but in terraform file terraform.tfstate.backup file is now created which holds all information about resource. How do I remove this file? Plz provide some tips.

In real time, are we using terraform for complete setup in AWS? or only for major resources creation only?

Hi Abhishek, Kind request. Please answer the below common question which i encountered in most of interviews. We have created the AWS resource for ex an ec2 instance through terraform, and then the state file gets deleted. If we reapply the same terraform code for recreating the ec2 instance. what will happen, will the ec2 instance will be created or not or there will be any errors.

Hi abhi, If we are required to work with multiple people will work the same project, then what is the option.

Thank you ^^

Hi Abhishek Anna. What happened in remote backend and Locking mechanism. If some one pull the terraform config file and made some changes. And without terraform apply he was push the updated terraform config file to Github then how tfstate file will track the changes.

Hi Abhishek - If we are using the remote back-end also the terraform show command reveals the contents of the state file . How can we secure it in this scenario