Introduction to Kubernetes Environment Cloning

Kubernetes environment cloning is the practice of creating complete replicas of existing Kubernetes environments including all deployments, services, configurations, secrets, persistent volumes, and networking setup to establish identical staging environments for testing, ephemeral preview environments for feature branches, disaster recovery environments in different regions or clouds, or development environments that mirror production exactly. Manual environment cloning traditionally requires hours of work copying dozens of YAML manifests, updating namespace references, adjusting resource allocations, recreating secrets, provisioning storage volumes, configuring ingress rules, and verifying that all service dependencies are correctly replicated — a tedious, error-prone process that often results in subtle configuration drift where cloned environments differ from source in ways that cause bugs to manifest differently across environments.

The benefits of effective environment cloning are transformative for development velocity and quality: developers can test changes in production-like environments without risking actual production, QA teams can spin up isolated test environments per feature branch eliminating test interference, debugging production issues becomes easier when you can reproduce the exact production state in a safe environment, disaster recovery is simplified since you can pre-clone production to standby regions ready for instant failover, and compliance testing can occur in dedicated environments that are exact production replicas without touching actual customer data or production systems.

This comprehensive guide teaches you everything about Kubernetes environment cloning from manual approaches through automated solutions, covering: what environment cloning means and why it's valuable beyond simple namespace creation, manual cloning using kubectl and kustomize with complete step-by-step procedures, automated cloning with Helm and templating strategies, handling secrets and sensitive data during cloning without exposing credentials, managing persistent data — whether to clone volume snapshots or start with fresh volumes, implementing ephemeral preview environments for pull requests using tools like Argo CD Pull Request Generator or Flux, scaling cloning to manage dozens of dynamic environments without overwhelming cluster capacity, and cost optimization for cloned environments to avoid budget explosion from running many replicas of production. It closes with environment cloning in Atmosly, which duplicates a complete environment — services, config, databases — without re-wiring anything.

What is Environment Cloning? Beyond Simple Namespace Copy

Understanding Kubernetes Environments

A Kubernetes "environment" typically consists of:

  • Namespace: Logical isolation boundary (production, staging, dev, feature-xyz)
  • Workloads: Deployments, StatefulSets, DaemonSets, Jobs, CronJobs
  • Services: ClusterIP, LoadBalancer, NodePort services for network access
  • Configuration: ConfigMaps, Secrets, environment variables
  • Storage: PersistentVolumeClaims, StorageClasses
  • Networking: Ingress rules, NetworkPolicies, Service entries
  • RBAC: ServiceAccounts, Roles, RoleBindings specific to environment
  • Resource Quotas: CPU/memory limits for namespace

Cloning an environment means replicating ALL of these components, not just creating an empty namespace.

Why Clone Environments?

Use Case 1: Staging Environment from Production

Create production-identical staging for testing changes safely:

  • Same microservices architecture (if production has 12 services, staging has same 12)
  • Same service versions (testing upgrade from v1.2 to v1.3 in environment matching production)
  • Same configuration structure (different values like database hosts, but same keys)
  • Same ingress routing and load balancer setup
  • But with reduced resources (staging 1 replica vs production 10)

Use Case 2: Preview Environments for Feature Branches

Ephemeral environments per pull request:

  • Developer creates PR for new feature
  • Automation clones production environment
  • Deploys PR's code in cloned environment
  • QA tests feature in isolation (doesn't affect other testing)
  • Environment auto-deleted when PR merged or closed

Use Case 3: Disaster Recovery

Pre-cloned production in different region or cloud:

  • Primary production in us-east-1
  • DR environment cloned to eu-west-1
  • Same services, ingress, configuration
  • Data replication via database replication or volume snapshots
  • DNS failover switches traffic if primary fails

Manual Environment Cloning: Step-by-Step Guide

Step 1: Export Source Environment

# Export all resources from source namespace
kubectl get all,ingress,configmap,secret,pvc -n production -o yaml > production-export.yaml

# This exports: Deployments, Services, Ingresses, ConfigMaps, Secrets, PVCs
# But includes cluster-specific fields that must be cleaned

Step 2: Clean Exported YAML

Remove cluster-specific metadata:

# Remove these fields (they're auto-generated):
# - metadata.uid
# - metadata.resourceVersion  
# - metadata.creationTimestamp
# - metadata.selfLink
# - status sections
# - ownerReferences

# Use yq or manual editing
yq eval 'del(.metadata.uid, .metadata.resourceVersion, .metadata.creationTimestamp, .status)' \
  production-export.yaml > production-clean.yaml

Step 3: Transform for Target Environment

Update namespace references:

# Replace namespace: production with namespace: staging
sed 's/namespace: production/namespace: staging/g' production-clean.yaml > staging.yaml

# Update resource allocations (staging uses less)
# Change replicas: 10 to replicas: 2
# Change CPU: 2000m to CPU: 500m
# Requires manual YAML editing or scripts

Step 4: Handle Secrets Separately

Never copy production secrets to staging!

# Extract secret structure (keys, not values)
kubectl get secrets -n production -o yaml | \
  yq eval '.items[] | .data = (.data | to_entries | map({(.key): "PLACEHOLDER"}))' > secret-templates.yaml

# Manually fill in staging-appropriate values
# Or use different secret source (staging database password, not production)

Step 5: Create Target Namespace and Apply

# Create staging namespace
kubectl create namespace staging

# Apply all resources
kubectl apply -f staging.yaml -n staging

# Verify all pods running
kubectl get pods -n staging

# Typical issues:
# - ImagePullSecrets missing (copy from production namespace)
# - PVCs pending (different StorageClass needed)
# - Ingress conflicts (different hostnames required)

Manual cloning problems:

  • Time-consuming: 1-3 hours for complex environments
  • Error-prone: Easy to miss resources or misconfigure
  • Not repeatable: Each clone requires manual work
  • Drift: Clones diverge from source over time

Automated Cloning with Kustomize

Repository Structure for Multiple Environments

k8s-manifests/
+-- base/  # Shared configuration
|   +-- deployment.yaml
|   +-- service.yaml
|   +-- ingress.yaml
|   +-- kustomization.yaml
+-- overlays/
    +-- production/
    |   +-- kustomization.yaml  # 10 replicas, high resources
    |   +-- ingress-patch.yaml  # app.example.com
    |   +-- resources-patch.yaml
    +-- staging/
    |   +-- kustomization.yaml  # 2 replicas, medium resources
    |   +-- ingress-patch.yaml  # staging.example.com
    |   +-- resources-patch.yaml
    +-- preview-pr-123/  # Ephemeral preview environment
        +-- kustomization.yaml  # 1 replica, low resources
        +-- ingress-patch.yaml  # pr-123.preview.example.com

Base kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- deployment.yaml
- service.yaml
- ingress.yaml

labels:
- pairs:
    app: myapp
  includeSelectors: true

Staging overlay (overlays/staging/kustomization.yaml):

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: staging  # Deploy to staging namespace

resources:
- ../../base

patches:
- path: resources-patch.yaml  # Reduce resources
- path: ingress-patch.yaml    # Different hostname

replicas:
- name: myapp
  count: 2  # Staging uses 2 replicas vs prod 10

Deploy staging environment:

# Build and apply staging configuration
kubectl apply -k overlays/staging

# This creates complete staging environment from base templates
# With staging-specific overrides (replicas, resources, ingress)

Benefits:

  • DRY (Don't Repeat Yourself): Base config shared
  • Consistent: All environments use same base
  • Versioned: Overlays in Git track environment differences
  • Repeatable: kubectl apply -k creates environment consistently

Ephemeral Preview Environments

The Preview Environment Pattern

Create temporary environment per pull request for testing:

  1. Developer creates PR with new feature
  2. CI/CD detects PR creation
  3. Automation clones environment with PR's code
  4. Unique URL generated (pr-456.preview.example.com)
  5. QA/stakeholders test feature in isolation
  6. Environment destroyed when PR merged/closed

Implementation with ArgoCD ApplicationSet:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: preview-environments
spec:
  generators:
  - pullRequest:
      github:
        owner: myorg
        repo: myapp
        labels:
        - preview  # Only PRs with "preview" label
  
  template:
    metadata:
      name: 'pr-{{number}}'
    spec:
      source:
        repoURL: https://github.com/myorg/myapp
        targetRevision: '{{head_sha}}'
        path: k8s/overlays/preview
      destination:
        server: https://kubernetes.default.svc
        namespace: 'preview-{{number}}'
      syncPolicy:
        automated:
          prune: true  # Auto-delete when PR closed

# ArgoCD automatically:
# - Creates namespace per PR
# - Deploys PR's code
# - Deletes namespace when PR merged/closed

Managing Secrets During Cloning

Challenge: Secrets Cannot Be Copied Directly

Production secrets (database passwords, API keys) should NEVER be used in staging/dev.

Solution 1: Separate Secrets Per Environment

# Production (real customer database)
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: production
data:
  password: cHJvZHVjdGlvblBhc3N3b3Jk  # base64: productionPassword

# Staging (test database)
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
  namespace: staging  
data:
  password: c3RhZ2luZ1Bhc3N3b3Jk  # base64: stagingPassword

# Same secret name, different values
# Applications work in both environments without code changes

Solution 2: External Secrets Operator

# Store secrets externally (AWS Secrets Manager, Vault)
# Reference different paths per environment

# Production ExternalSecret
spec:
  data:
  - secretKey: password
    remoteRef:
      key: production/database/password  # Prod secret

# Staging ExternalSecret (same structure, different path)
spec:
  data:
  - secretKey: password
    remoteRef:
      key: staging/database/password  # Staging secret

# Cloning creates ExternalSecret with correct path for target environment

Persistent Data Handling in Cloned Environments

Option 1: Fresh Volumes (Most Common)

Cloned environment gets new empty volumes:

  • Databases start empty or with seed data
  • Faster cloning (no data copy)
  • Lower cost (no duplicate storage)
  • Safe (no production data exposure in staging)

Option 2: Snapshot and Restore (For Testing with Real Data)

# 1. Snapshot the production volume (needs a CSI driver with snapshot support)
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: prod-db-snapshot
  namespace: production
spec:
  volumeSnapshotClassName: csi-snapclass
  source:
    persistentVolumeClaimName: postgres-data
---
# 2. Restore it into a new PVC. A PVC's dataSource must name a VolumeSnapshot
#    in the PVC's own namespace, so to restore into staging, first create a
#    VolumeSnapshot in staging bound to the same snapshot (a pre-provisioned
#    VolumeSnapshotContent that points at the snapshot handle).
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: postgres-data
  namespace: staging
spec:
  dataSource:
    name: prod-db-snapshot
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi

# Staging database starts with production data snapshot
# Remember to scrub sensitive data for compliance!

Option 3: Logical Data Cloning

For databases, use database-specific tools:

# PostgreSQL: pg_dump from prod, pg_restore to staging
kubectl exec -n production postgres-0 -- \
  pg_dump -U postgres mydb > prod-backup.sql

# Sanitize sensitive data
sed 's/real-email@example.com/test-email@example.com/g' prod-backup.sql > sanitized.sql

# Restore to staging
kubectl exec -n staging postgres-0 -- \
  psql -U postgres mydb < sanitized.sql

Cost Optimization for Cloned Environments

Challenge: Cloning Multiplies Costs

If production costs $10,000/month, having 10 clones (staging, 5 developer environments, 3 preview environments) could cost $100,000/month — unsustainable.

Solution: Right-Size Clones

Resource Reduction Strategy:

  • Production: 10 replicas, 2 CPU, 4Gi RAM per pod = 20 CPU, 40Gi total
  • Staging: 2 replicas, 1 CPU, 2Gi RAM per pod = 2 CPU, 4Gi total (90% cost reduction)
  • Dev: 1 replica, 500m CPU, 1Gi RAM = 500m CPU, 1Gi total (95% reduction)
  • Preview: 1 replica, 250m CPU, 512Mi RAM = minimal cost

Automated scaling in Kustomize:

# overlays/dev/kustomization.yaml
replicas:
- name: frontend
  count: 1
- name: backend  
  count: 1

patches:
- target:
    kind: Deployment
    name: frontend
  patch: |-
    - op: replace
      path: /spec/template/spec/containers/0/resources/requests/cpu
      value: 250m
    - op: replace
      path: /spec/template/spec/containers/0/resources/requests/memory
      value: 512Mi

Schedule-Based Scaling

Shut down non-production environments nights and weekends:

# Scale down staging at 6 PM (CronJob)
apiVersion: batch/v1
kind: CronJob
metadata:
  name: scale-down-staging
spec:
  schedule: "0 18 * * *"  # 6 PM daily
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: kubectl
            image: bitnami/kubectl
            command:
            - /bin/sh
            - -c
            - kubectl scale deployment --all --replicas=0 -n staging

# Scale up at 8 AM
apiVersion: batch/v1
kind: CronJob
metadata:
  name: scale-up-staging
spec:
  schedule: "0 8 * * 1-5"  # 8 AM weekdays
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: kubectl
            image: bitnami/kubectl
            command:
            - /bin/sh  
            - -c
            - kubectl scale deployment --all --replicas=2 -n staging

# Savings: 16 hours/day × 5 days + 48 hours weekend = 128/168 hours off = 76% cost reduction

Smaller overlays and scheduled shutdowns are manual levers, and requests drift back up over time. To find over-requested pods across every namespace automatically, SpendZero's Kubernetes workload rightsizing compares each container's requests with its real usage, flags idle namespaces such as forgotten preview environments, and applies fixes in one click with an audit trail.

Environment Cloning in Atmosly

The kubectl, Kustomize and ArgoCD approaches above all work, but each one is something your team builds and maintains. Atmosly, the internal developer platform from SquareOps, includes environment cloning as a self-service feature: duplicate a complete environment — services, config, databases — for a new tenant, region or load test, without re-wiring anything.

Cloning sits next to the platform's other self-service features: a preview environment for every pull request, with seed data, RBAC, network policies and resource defaults inherited from production and automatic expiry on merge, and a Helm marketplace of curated charts with your policies and resource defaults pre-applied. The practices in this guide still apply to every clone: keep secrets environment-specific and size non-production copies down.

Best Practices for Environment Cloning

1. Automate Cloning (Don't Do Manually)

Use Kustomize overlays, Helm values, or platforms like Atmosly for repeatable cloning.

2. Never Clone Production Secrets

Always use environment-specific secrets. Production secrets in staging/dev is security violation.

3. Right-Size Cloned Environments

Staging doesn't need 10 replicas if production has 10. Use 1-2 replicas saving 80-90% cost.

4. Implement Auto-Cleanup

Preview environments should auto-delete after PR merged. Set TTL (time-to-live) for dev environments.

5. Monitor Clone Costs

Track spending on non-production environments. Often surprises: "We have 15 forgotten preview environments costing $2,000/month"

Conclusion: Environment Cloning for Agile Development

Kubernetes environment cloning enables rapid iteration, safe testing, and better collaboration. Whether using manual kubectl and kustomize, automated tools like ArgoCD ApplicationSets, or platform features like Atmosly's environment cloning, the key is making environment creation fast, repeatable, and safe.

Key Takeaways:

  • Environment cloning replicates entire stack (workloads, config, networking, storage)
  • Manual cloning with kubectl: Time-consuming but possible (use kustomize for efficiency)
  • Automated cloning with ArgoCD/Flux: Scales to many environments
  • Always use separate secrets per environment (never clone production secrets)
  • Right-size clones (staging 2 replicas vs prod 10 saves 80% cost)
  • Implement auto-cleanup for ephemeral environments (prevent cost accumulation).
  • Atmosly environment cloning: duplicates a complete environment (services, config, databases) without re-wiring anything

Ready to clone Kubernetes environments effortlessly? Start with Atmosly's environment cloning to duplicate staging, development, or test environments without re-wiring anything.