Infrastructure as Code (IaC) is an essential practice for modern DevOps and Agile teams to manage cloud infrastructure consistently, efficiently, and with increased resilience. Terraform has emerged as the leading tool for IaC, enabling teams to provision cloud infrastructure across multiple providers regardless of organization size. With Terraform, DevOps engineers can quickly and easily manage cloud infrastructure with code, speeding up the deployment process and ensuring consistency.
In addition to Terraform, Gitlab has become a popular choice for CI/CD management among developers and DevOps engineers. Gitlab’s vast integration with different tools allows for better management of the deployment process, making it an essential tool for organizations looking to streamline their DevOps workflow. By leveraging both Terraform and Gitlab, organizations can manage their cloud infrastructure and deployment processes efficiently and effectively, improving their overall DevOps Process.
Let’s look at some of the advantages that Terraform and Gitlab together provide along with a walkthrough of how to Integrate Gitlab and Terraform for the management of cloud infrastructure.
For the purpose of demonstration, we have published terraform code to the public Gitlab repository here Terraform-Gitlab-Pipeline
Terraform states are like a database for your infrastructure deployment. It keeps track of all the cloud resources deployed and managed by Terraform.
With GitLab, you can:
Gitlab offers CI/CD pipelines which are defined with the help of gitlab-ci.yml files in the project repository’s root directory. The pipelines feature was initially designed for application code deployments but is now widely used to manage infrastructure deployments also.
A typical pipeline includes Gitlab’s standard Merge request based workflow for infrastructure deployment
For any infrastructure change required, a feature branch is created from the mainline branch. Once the required changes are done, a Merge Request is raised to integrate the changes in the mainline branch, say main
As per this workflow, a new branch is created for an infrastructure change request, from the main branch ( e.g. branch name — CR1/demo_change_request_for_vpc ). The branch is then worked upon, for required changes, and a Merge Request is raised in GitLab for review.
As soon as an MR is raised, the pipeline gets triggered to perform certain terraform tasks related to the first level check of the recent changes. This is stage 1, which includes –
The generated plan[file] is then saved as a pipeline artifact, to ensure the exact planned changes are applied when this Merge Request gets approved
Upon review and approval of the Merge request, the Change is merged into the main branch, leading to stage 2 of the pipeline, this time, taking the pre-generated plan and applying the changes to cloud deployment. The steps are
This entire workflow is defined herein gitlab-ci.yml file
With the concepts fully explained and the code prepared, the next step is to set up the deployment pipeline for AWS Cloud Resource Provisioning. We will go through a step-by-step guide on how to build this pipeline and expected output results.
The credentials for your AWS account can be configured under the variables section <mention the path, e.g. Settings -> CI/CD-> Variables ( The sensitive tokens need to be masked)
Gitlab is configured as a remote state storage backend in terraform’s backend.tf file
The Gitlab project-specific configuration for backend configuration is defined in the Variables section of the .gitlab-ci.yml file
In order to simulate the behavior explained above, in the core concepts section,
The last stage in the gitlab-ci.yml file consists of the destroy or the clean up action. When manually approved, it executes the terraform destroy command.
Let’s summarize the entire deployment process and how it’s helpful :