Hello, fellow cloud enthusiasts. The legend says that even now, up there in the cloud, there is a conflict between a Terraform pipeline and AWS CodeDeploy. Let us try to bring peace and joy into the blue-green land. From an infrastructure perspective, we will go from top to bottom in the images below.
Let us start by answering a few questions for our not-so-technical readers.
WHAT IS TERRAFORM, AND HOW DO WE USE IT?
Terraform is like a big cookbook with many excellent recipes. When we start making the apple pie (our infrastructure), we must respect the recipe from the book; otherwise, the pie will not be good, and our guests (users) will question our cooking ability. More on what Terraform is and how to install it here.
WHAT IS A TERRAFORM PIPELINE, AND HOW DO WE USE IT?
Imagine that the pipeline is a kitchen robot that will cook the pie for us, but it needs clear instructions on what ingredients should be put in.
In the process of making (building) the pie, the robot will more or less copy the recipe from the book onto a separate piece of paper. That piece of paper is called a Terraform state file. For each ingredient we will add, the robot will also add it to the state file. This ensures the robot will not forget how our pie (infrastructure) should look.
Each time we turn the robot on (trigger the pipeline), it will check if our pie (infrastructure) has every ingredient from the state file. If the pie has a missing ingredient (one that is present in the state file), the robot will add that to the pie. If we manually remove one ingredient from the pie, the robot, when it runs, will see that, and it will ‘fix’ it (will add the ingredient).
WHAT IS CODEDEPLOY, AND HOW DO WE USE IT?
Continuing on the same note, imagine that CodeDeploy is an ingredient from our recipe (infrastructure), but this ingredient could actually be a chef on its own. It has the ability to manage other ingredients (other parts of the infrastructure) on its own. CodeDeploy could ‘operate’ on two different levels: in-place and blue-green.
In-place means that if we update some pieces, it will go one by one and start upgrading them.
Blue-green means that it will create new pieces (green) that are identical to the existing pieces (blue). It will update the green ones first, making sure that these look good, and then remove the blue ones. A blue-green deployment strategy means having two identical but separate environments. The blue environment runs the current version of the application, and the green environment runs the new version of the application.
Why is blue-green deployment important? It minimizes downtime of applications during deployment.
Because in every happy story there will be a not so good part, on our use case with blue-green deployment on AWS, here it is: One of the main cons of using blue-green deployment is its higher cost. Since you need a replica of the running environment, you will require double the resources. For an already costly environment, this deployment strategy may not be the best.
WHAT IS ASG (AutoScalingGroup)?
ASG is a collection of components (computers) that share similar characteristics and are treated as a logical grouping for fleet management and dynamic scaling. This can expand or retract according to needs.
LET'S START BRINGING PEACE BETWEEN TERRAFORM AND CODEDEPLOY:
Now, after this not-so-short introduction, let’s dive into the ‘battle’. We will start here:
And from this Terraform configuration of CodeDeploy:
The above means that CodeDeploy will update our collection of computers one at a time, causing potential downtime.
If we want to achieve zero downtime when we update our application, CodeDeploy allows us to have a blue-green deployment type instead of in-place.
With blue-green deployment, CodeDeploy creates a new set of instances while keeping the old ones, then switches traffic to the new instances, ensuring zero downtime. And because money plays a vital role in everyday life, in the cloud, it is the same; we want to ensure we don’t spend more than we need to. For this, we can let CodeDeploy manage the creation and termination of ASGs. That means that CodeDeploy will do all the job for us, and we only need to take care of our application and not worry about infrastructure.
We will switch to this (comparing to the previous configuration):
If we apply this configuration, we will let CodeDeploy manage our ASGs. It will create a new ASG due to the parameter "COPY_AUTO_SCALING_GROUP" and terminate the old one after everything is successful due to the parameter "TERMINATE".
But let’s think for a second. If we do this, CodeDeploy will create new resources on its own (new ASG) and terminate resources on its own (old ASG) when the Terraform pipeline runs. It will see those ‘differences’ thanks to the state file, and it will recreate everything as it was in the first place. This means that our ASG will be recreated, and our previous version of the application will be again online for our users. This is not good.
HOW CAN WE KEEP OUR TERRAFORM PIPELINE HAPPY WHILE LETTING CODEDEPLOY MANAGE THE DEPLOYMENT (ASG) PART?
The solution is something called ‘lifecycle’. The ‘lifecycle’ meta-argument can be used to specify how Terraform should handle the creation, modification, and destruction of resources.
Meta-arguments are arguments used in resource blocks. Based on this: https://developer.hashicorp.com/terraform/language/meta-arguments/lifecycle, we have multiple options, but we will focus only on the "ignore_changes" one.
The terraform 'lifecycle' "ignore_changes" meta-argument is intended to be used when a resource is created with references to data that may change in the future but should not affect it after its creation. In some rare cases, settings of a remote object are modified by processes outside of Terraform, which Terraform would then attempt to 'fix' on the next run. To make Terraform share the management responsibilities of a single object with a separate process, the "ignore_changes" meta-argument specifies resource attributes that Terraform should ignore when planning updates to the associated remote object.
TO SOLVE THIS CONFLICT, WE HAVE TWO OPTIONS:
🟧 OPTION 1: 'SIGNING THE PEACE TREATY' IN ONE STEP
Step 1: Will add 'lifecycle' into the CodeDeploy configuration.
This ensures that Terraform will ignore the ASG parameter from the CodeDeploy configuration. This step is necessary as CodeDeploy manages the creation and termination of ASGs. When running the Terraform pipeline, it will try to revert to the previous ASG (which is not what we want).
🟧 OPTION 2: WE COULD 'SIGN THE PEACE TREATY' IN TWO STEPS
Step 1: Same as before, but with a slight difference in our CodeDeploy configuration:
Now, instead of "TERMINATE", we have "KEEP_ALIVE". That means that after our deployment is finished, the old ASG will not be terminated, but this also means that the old ASG will produce cost for us, and we don’t need that, so the additional step 2 is now necessary.
Step 2: Will add 'lifecycle' into the ASG configuration.
This is required because, after we see that our deployment was successful and everything is perfect, we will manually go into the AWS Management Console and set these three arguments ("desired_capacity","min_size, "max_size") to 0. By doing so, we prevent the old ASG from producing costs.
We should use the second approach only if we want better transparency in our infrastructure.
Potential scenario: Let us assume we apply the 1st option, and a new specialist joins the team to review the Terraform code. At first, the specialist will not understand why the ASG is absent, and if they're unfamiliar with CodeDeploy blue-green, they might not understand how that works. However, suppose we apply the 2nd option, and they see the ASG present in AWS Management Console (but scaled at zero) and the same ASG in Terraform but with a ‘lifecycle’. In that case, they can easily understand what is happening.
CONCLUSION
In the grand tale of cloud infrastructure, peace between Terraform and CodeDeploy is not just a dream—it’s a reality you can achieve with the correct configuration. Using Terraform's ‘lifecycle’ blocks to ignore specific changes enables seamless blue-green deployments with CodeDeploy, ensuring a smooth and reliable application deployment process.
So, let there be peace in the blue-green fields! May your deployments be smooth, your downtime be zero, and your users always happy. Happy deploying, fellow cloud enthusiasts!
View the article in our blog: Let there be peace! Solve the ancient ‘battle’ between Terraform and CodeDeploy in the blue-green fields of AWS - SoftServe (softserveinc.com)
*AUTHOR: GEORGE DOBRIȘAN | SOFTSERVE SENIOR AWS DEVOPS ENGINEER
