DevSecOps in Action: A Complete Guide to Secure CI/CD Workflows

Network Security

Networking is our first defense against any kind of attack and to prevent attacks on our application we should harden our network.

• Create a separate private network for the workload (eg. App and DB) and only allow internet access from NAT.
• Set fine-grained access on inbound and outbound network rules. Also, we can use Cloud custodian to set the security compliance which automatically removes any unwanted network traffic.
• Always configure Network ACL (NACL) for subnets in AWS. The best practice would be to block all outbound traffic and then allow the required rules.
• Use Web Application Firewall (WAF).
• Enable DDOS protection.
• Nmap and Wireshark, tcpdump tools can be used to scan networks and packets.
• Use VPN or Bastion host for connecting to infrastructure networks.

Web Application Firewall (WAF)

WAF is a layer7 firewall that protects our web applications against common web exploits (like XSS and SQL injection) and bots that may affect availability, compromise security, or consume excessive resources. Most cloud service providers provide WAF and with a few clicks, we can easily integrate it with our application. Curiefense is an open source cloud native self-managed WAF tool that can be used to protect all forms of web traffic, services, DDoS, and APIs. We can also use WAF as a service from Cloudflare and Imperva.

Identity Access Management (IAM)

IAM is a centrally defined policy to control access to data, applications, and other network assets. Below are a few methods that help prevent unauthorized access.

• Have centralized user management using Active Directory or LDAP.
• Use RBAC access management.
• Have fine-grained access Policy for AWS IAM role.
• Rotate user’s access and secret keys periodically.
• Use Teleport for centralized connectivity, authentication, authorization, and audit.
• Store secrets into vaults and ensure that it is only accessible to authorized users.
• Implement Zero trust within our services.

Cloud, server, and application hardening

We can use CIS benchmark to harden the cloud, operating system, and application. It is always a good practice to use a hardened OS as it reduces the attack surface of the server. Most of the cloud providers provide a hardened image or we can create our own custom hardened image.

Nowadays, most applications run inside containers. We need to harden our applications as well as containers by doing static analysis and container image scanning. 

To protect against viruses, trojans, malware, and other malicious threats we can install Antivirus like Falcon, SentinelOne, or Clamav.

Server patching

The most common attack vector exploits vulnerabilities in the OS or applications running on servers. Running regular vulnerability scans against the environments and updating regular packages reduces the risk of vulnerability. 

We can create an automation pipeline to patch the server using Foreman or Red Hat Satellite and for scanning, we can use OpenVAS or Nessus to get the list of vulnerabilities.

Securing Kubernetes

Kubernetes has become the backbone of modern infrastructure and to make sure we are running it securely we can use the below tools:

• Use the correct security-context in Kubernetes YAML file.
• Use Network Policy to block all the traffic by default and allow only required traffic.
• Use Service Mesh (Linkerd, Istio) to have mTLS communication between microservices and implement Authorization to have fine-grained access.
• Implement kube-bench for a CIS benchmark report for the Kubernetes cluster. We can run this scan daily in our Kubernetes cluster and fix any reported vulnerabilities.
• Use tools like Kube-hunter, Popeye and Kubescape for security weaknesses and misconfigurations in kubernetes clusters and visibility of security issues.
• Use Checkov, KubeLinter, and Terrascan for scanning Kubernetes YAML, Helm charts with best practices and vulnerabilities.
• Implementing pre-deployment policy checks like Kyverno, Kubewarden, and Gatekeeper can block non-standard Deployment.
• Use a hardened image for the worker server. All cloud providers provide CIS benchmark hardened images. We can also build our own custom hardened image using amazon-eks-ami.
• Store Kubernetes secret in an encrypted format or use an external secret manager like Vault.
• Use IAM roles for service accounts to assign AWS roles directly to Kubernetes service accounts.
• Implement Chaos Mesh and Litmus chaos engineering framework to understand the behavior and stability of application in real-world use cases.
• Follow best practice to secure Kubernetes 
• Use tools like Falco, Tracee to monitor runtime privileged and unwanted system calls.

Containers

Containers are the smallest level of abstraction for running any workload in modern infrastructure. Below are a few methods to secure our containers and we have also seen above how to integrate them into our CI/CD pipeline.

• Scan the Container image and Dockerfile.
• Reduce the Docker image size with a multi-stage build and using a distroless image to reduce the attack surface.
• Don’t use the root user and privileged containers.
• Have Gvisor and Kata containers for kernel isolation.
• Use container image signing and verification.
• Set up a list of known good containers registry.
• Implement the Container security best practices.