Terraform: chicken/egg problem

4 min readJan 2, 2020

So you’re starting to work with Terraform and would like to store the state in S3 bucket. Sounds good but how are you going to create the S3 bucket? You don’t want to use AWS Console to do that — you decided to have IaC and having first step breaking the rule is awkward at least. You can find a few articles and ideas how to deal with that and here I would like to present approach I tried.

What are our goals?

No manual operations through AWS Console
All resources managed by Terraform (includes state file in S3 bucket)

Step 1 — configure Terraform

Let’s start with our main.tf file to configure provider and backend. The problem here is that for terraform block you cannot use resources or even variables.

So let’s try another approach. We can define our Terraform backend as s3 and do not provide required variables — we will do that later. Additionally we have standard definition of provider and two modules:

backend — it will define our infrastructure for storing Terraform’s state.
infrastructure — it will define infrastructure required by our service.

Here is our main.tf file:

terraform {
  backend "s3" {
  }
}provider "aws" {
  version = "~> 2.31.0"
}module "infrastructure" {
  source = "./modules/v1.0.0"
}module "backend" {
  source     = "./modules/backend"
  s3_tfstate = var.s3_tfstate
}

We have one variable to define configuration of our bucket defined in variables.tf like this:

variable "s3_tfstate" {
  type = object({
    bucket = string
  })
}

Now let’s take a look on our backend module — modules/backend/main.tf

resource "aws_s3_bucket" "tfstate_bucket" {
  bucket = var.s3_tfstate.bucket
  versioning {enabled = true}
}output "TFSTATE_BUCKET_NAME" {
  value = aws_s3_bucket.tfstate_bucket.bucket
}

Nothing special — we’ve got our AWS resource and we would like to turn on versioning (as HashiCorp advise).

Step 2 — try & solve issues

Let’s try to run terraform init and provide required variables (bucket and key). As you can expect it fails:

Error: Failed to get existing workspaces: S3 bucket does not exist.The referenced S3 bucket must have been previously created. If the S3 bucket
was created within the last minute, please wait for a minute or two and try
again.Error: NoSuchBucket: The specified bucket does not exist
        status code: 404, request id: 0F85F99A913ED2C1, host id: 7Ng65i4TX6qcEkbgffbs/6XT5BY6uDeUHeaz78QtF3Nz4Ct86T+rLkQka2248OZOgxo8DgbcxoA=

Bucket doesn’t exist. Let’s fix it and create bucket using AWS CLI: aws s3api create-bucket --bucket ${BUCKET} --region ${AWS_REGION} --create-bucket-configuration LocationConstraint=${AWS_REGION} (don’t forget to export AWS credentials). So bucket exists, let’s try to init our configuration once — we can provide backend configuration in command line (we have new environment variable SERVICE_NAME which defines the filename in bucket to store state):

terraform init -reconfigure \
    -backend-config="bucket="${BUCKET}"" \
    -backend-config="key="${SERVICE_NAME}""

And…

Initializing modules...Initializing the backend...Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.Initializing provider plugins...
- Checking for available provider plugins...
- Downloading plugin for provider "aws" (hashicorp/aws) 2.31.0...Terraform has been successfully initialized!You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Great! But Terraform doesn’t know about our backend S3 bucket. So for example it won’t enable versioning for our bucket (and we specified that in our modules/backend/main.tf file). How we can solve that? By importing resource: terraform import module.backend.aws_s3_bucket.tfstate_bucket ${BUCKET}. Now you should see nice message: Import successful!

Step 3 — verify

Let’s do our terraform plan :

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.module.backend.aws_s3_bucket.tfstate_bucket: Refreshing state... [id=mm-tfstate]------------------------------------------------------------------------An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-placeTerraform will perform the following actions:# module.backend.aws_s3_bucket.tfstate_bucket will be updated in-place
  ~ resource "aws_s3_bucket" "tfstate_bucket" {
      + acl                         = "private"
        arn                         = "arn:aws:s3:::mm-tfstate"
        bucket                      = "mm-tfstate"
        bucket_domain_name          = "mm-tfstate.s3.amazonaws.com"
        bucket_regional_domain_name = "mm-tfstate.s3.eu-west-2.amazonaws.com"
      + force_destroy               = false
        hosted_zone_id              = "Z3GKZC51ZF0DB4"
        id                          = "mm-tfstate"
        region                      = "eu-west-2"
        request_payer               = "BucketOwner"
        tags                        = {}~ versioning {
          ~ enabled    = false -> true
            mfa_delete = false
        }
    }Plan: 0 to add, 1 to change, 0 to destroy.

We can see that Terraform wants to modify our resource to enable versioning. Everything looks as expected.

Summary

With that approach we have all resources managed by Terraform. We can set appropriate permissions etc. And what’s even more important we will avoid any manual configuration or complex scripts. This a few steps we can wrap into nice script (you can make it easily nicer and more verbose — please share in comments your version :) ):

#!/bin/bashaws s3api create-bucket --bucket ${BUCKET} --region ${AWS_REGION} --create-bucket-configuration LocationConstraint=${AWS_REGION}cat > terraform.tfvars << EOL
s3_tfstate = {
  bucket = "${BUCKET}"
}EOLterraform init -reconfigure \
    -backend-config="bucket="${BUCKET}"" \
    -backend-config="key="${SERVICE_NAME}""terraform import module.backend.aws_s3_bucket.tfstate_bucket ${BUCKET}

The whole repository can be found here: https://github.com/mmatecki/tf-s3-state

It’s my first article on Medium.com comments are more than welcome!