Freefuse Cloud Journey on AWS

Introduction

FreeFuse Inc. offers a platform that transforms videos into interactive learning tools. The Engineering team sought to enhance their infrastructure management with Terraform and streamline the deployment process through the blue-green deployment model for application releases. They also aimed to migrate their Python AI video editor component from IBM Cloud to AWS and enable multiple parallel processes for the AI video editor.

To achieve these goals, FreeFuse Inc. engaged SquareOps as their Service Partner, drawn by SquareOps’s demonstrated experience with Terraform, CI/CD and AWS Advanced consulting partner badge.

Customer Challenge

During the evaluation phase, the SquareOps team discovered five primary challenges:

1. Terraform coverage: One of the significant challenges was to import the existing resources like S3 buckets, policies etc into the Terraform code and state without having to recreate the entire deployment. This required a deep understanding of Terraform and the existing infrastructure.

2. Blue-green deployments: This mechanism creates new Autoscaling groups on every deployment, which posed a challenge in keeping the state of Terraform up to date.

3. Migration of Python AI video editor component: The component needed to be migrated from IBM Cloud to AWS for streamlined operations and billing, which was a significant task.

4. Serial processing of videos: The Video AI editor was only capable of processing one video at a time, leading to delays in the availability of edited videos to users. This needed to be improved to enable parallel processing.

5. Compliance: The company aimed to achieve SOC-2 compliance in the near future. Therefore, all the resources managed using Terraform needed to comply with SOC-2 controls from the beginning to avoid rework later.This added another layer of complexity to the project.

Solution Implemented

With careful planning of the solution design and successful POCs , SquareOps came up with a solution to achieve the desired goals

The solution involved:

1. Infrastructure as a code : Using Terraformer, infrastructure code for existing resources like VPC, S3, policies, etc., was generated. This code was then extended using a mix of Terraform modules provided by AWS and our own. Our modules were developed to comply with CIS benchmarks and were tested with Prowler scans before being published.

2. Modifying AWS EC2 Autoscaling Group Module: The official Terraform module for AWS EC2 Autoscaling groups was modified to handle the challenges posed by blue-green deployments. This included:
   • Adding a lifecycle policy to the resource autoscaling_group. This policy allowed the creation of a new Autoscaling Group (ASG) before destroying the old one and ignored changes to the name tag.
   • Disabling the use_name_prefix while calling the autoscaling group module. This prevented the Terraform ASG module from creating an autoscaling group with random names.
   • Creating a unique tag id for the autoscaling group’s name, controlled by the Terraform format(“%s-%s”, local.unique, “asg”).
   • Fetching the name of the new autoscaling group created by CodeDeploy using Terraform’s data block data.aws_autoscaling_groups.frontend.names.

3. CodeBuild Pipeline Development: A CodeBuild pipeline was developed to deploy and manage Terraform changes. A script was also used in the pipeline to refresh the Terraform state before planning Terraform changes.

4. Parallel Video Processing: We integrated AWS Simple Queue Service (SQS) into the architecture. The AI video editor’s processing requests were placed in an SQS queue. This allowed multiple videos to be processed simultaneously, significantly reducing the time taken to make edited videos available to users. We also configured the Autoscaling group instances to be provisioned based on the number of pending videos to be processed in the SQS queue. This ensured that the processing capacity of the AI video editor scaled up and down according to demand, optimizing resource usage and cost.

5. Compliance with SOC-2 Controls: tfsec was used in the Terraform pipelines to ensure compliance with SOC-2 controls. AWS account level checks were also configured to comply with SOC-2 controls and remediate any non-compliant resources/services.

Results and Benefits

1. Automation : The implementation of Terraform led to better infrastructure change management, governance, and reproducibility, making the infrastructure management more efficient.

2. Blue-Green Deployment : The use of blue-green deployments allowed us to test new releases in a separate environment before making them live to users, ensuring seamless and reliable updates.

3. Compliance : The readiness for SOC-2 compliance enabled faster approval of the SOC-2 Type 1 application, streamlining the compliance process.

4. Speed : The parallel processing of videos improved the user experience by reducing delays in the availability of edited videos.

5. Centralised Management : By migrating services from IBM to AWS, we achieved unified billing and a single mechanism for operation, simplifying the operational process.

Conclusion

In our transformative collaboration with FreeFuse Inc., we deployed a suite of cutting-edge strategies and digital tools, resulting in a marked improvement in the robustness and efficiency of their system.

We prioritized creating a secure and reliable framework for their software updates by implementing stringent testing mechanisms. This ensures that any update is thoroughly vetted for stability and performance, thus reinforcing user trust and satisfaction.

One of the most significant aspects of our engagement was preparing FreeFuse Inc. for SOC-2 compliance. In an age where data security is paramount, achieving this certification is not just about ticking a regulatory box; it represents a commitment to ensuring customer data is managed with the utmost security and confidentiality. This not only raises their security standards but also positions FreeFuse Inc. as a trusted player in the market.

We further simplified their operational processes by migrating their services from IBM to AWS. This move reduced complexity while increasing operational flexibility and scalability, an essential aspect of their business’s continuous growth.

Moreover, we revolutionized their user experience by making notable improvements to their video processing system. Our technological enhancements have resulted in smoother video processing and have significantly impacted their users, making their interactions more engaging and efficient.