Terraform State in Managing Infrastructure for DevOps Engineers

📝Introduction

Today, in this blog post, we will cover the basic concepts of Terraform Managing Infrastructure, focusing on the importance of state and how to store state files and remote state management options such as AWS S3. This is part of #Day 4 of the #TerraWeek challenge initiated by Shubham Londhe.

📝Terraform State

Terraform must store state about your managed infrastructure and configuration.

This state is used by Terraform to map real world resources to your configuration, keep track of metadata, and to improve performance for large infrastructures.

This state is stored by default in a local file named terraform.tfstate, but we recommend storing it in a remote local to version, encrypt, and securely share it with your team.

Terraform uses state to determine which changes to make to your infrastructure. Prior to any operation, Terraform does a refresh to update the state with the real infrastructure.

These are the benefits of using State files:

Idempotence - Whenever a Terraform configuration is applied, Terraform checks if there is an actual change made. Only the resources that are changed will be updated.
Deducing dependencies - Terraform maintains a list of dependencies in the state file so that it can properly deal with dependencies that no longer exist in the current configuration.
Performance - Terraform can be told to skip the refresh even when a configuration change is made. Only a particular resource can be refreshed without triggering a full refresh of the state, hence imporving performance.
Collaboration - State keeps track of the version of an applied configuration, and it's stored in a remote, shared location. So collaboration is easily done without overwriting.
Auditing - Invalid access can be identified by enabling logging.
Safer storage - Storing state on the remote server helps prevent sensitive information.

📝Terraform State Files: Store Local vs. Remote

local state : Refers to Terraform state stored on the local filesystem, i.e. on your laptop or whatever system you are running the terraform command from.

By default, Terraform stores your state in terraform.tfstate file on the local machine.

remote state : A remote backend is a remote state that can be shared, providing helpful capabilities such as preventing conflicts and inconsistencies.

i.e.

terraform {
  backend "local" {
    path = "relative/path/to/terraform.tfstate"
  }
}

The use of remote state is a best practice in an Enterprise, or when collaborating across a team. The main reason for using remote state is to have a single source of truth. This protects against ‌divergent state files or multiple copies of state files.

When we are working on IaC as a team, having state stored remotely ensures consistency, so that any developer will know what changes have already been applied and not inadvertently overwrite or revert another's changes. It helps to avoid merge conflicts.

Terraform offers several state storage options, to support the remote backend, as such S3 Bucket, Terraform Cloud, or HashiCorp Consul.

i.e.

backend "s3" {
    bucket = "<s3-bucket-name>"
    key    = "<s3/path/to/terraform.tfstate>"
    region = "eu-west-3"
  }
}

AWS S3 is a popular object storage service that can be used to store Terraform state files.

To use it as a remote backend, first of all, you need to configure the AWS provider in your Terraform configuration and set up an S3 bucket to store your state files.

Create an S3 bucket to store your state files and add the following backend block to your Terraform configuration:

#required providers
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.1.0"
    }
  }

  #backend remote to store Terraform state files
backend "s3" {
    bucket = "<s3-bucket-name>"
    key    = "<s3/path/to/terraform.tfstate>"
    region = "eu-west-3"
  }
}

#dynamodb table to store state lock
resource "aws_dynamodb_table" "<your-terraform_state_lock>" {
  name           = "<your-terraform-state-lock>"
  hash_key       = "LockID"
  read_capacity  = 5
  write_capacity = 5

  attribute {
    name = "LockID"
    type = "S"
  }
}