Cicd

Introduction CI/CD (Continuous Integration/Continuous Deployment) automates the software delivery process from code commit to production deployment. This automation reduces manual errors, speeds up releases, and improves software quality. This guide visualizes the complete CI/CD pipeline: Code Commit: Developer pushes code Continuous Integration: Automated testing and building Continuous Deployment: Automated deployment to production Quality Gates: Checkpoints ensuring code quality Rollback Mechanisms: Handling deployment failures Part 1: Complete CI/CD Pipeline Overview End-to-End Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% flowchart TD Start([Developer writes codecommits changes]) --> Push[git push origin main] Push --> Webhook[Git Provider WebhookTriggers CI/CD pipeline] Webhook --> Checkout[Stage 1: CheckoutClone repositoryFetch dependencies] Checkout --> Lint[Stage 2: LintCheck code styleESLint, Prettier, golangci-lint] Lint --> LintResult{Lintingpassed?} LintResult -->|No| LintFail[❌ Pipeline FailedNotify developerFix linting errors] LintResult -->|Yes| UnitTest[Stage 3: Unit TestsRun test suiteGenerate coverage report] UnitTest --> TestResult{Testspassed?} TestResult -->|No| TestFail[❌ Pipeline FailedSome tests failedCoverage too low] TestResult -->|Yes| Build[Stage 4: BuildCompile applicationBuild Docker image] Build --> BuildResult{Buildsuccessful?} BuildResult -->|No| BuildFail[❌ Pipeline FailedBuild errorsDependency issues] BuildResult -->|Yes| IntegTest[Stage 5: Integration TestsTest with real dependenciesDatabase, APIs, etc.] IntegTest --> IntegResult{Integrationtests passed?} IntegResult -->|No| IntegFail[❌ Pipeline FailedIntegration issuesService communication errors] IntegResult -->|Yes| Security[Stage 6: Security ScanScan for vulnerabilitiesOWASP, Snyk, Trivy] Security --> SecResult{Securitychecks passed?} SecResult -->|No| SecFail[❌ Pipeline FailedSecurity vulnerabilities foundFix before deploying] SecResult -->|Yes| Push2Registry[Stage 7: Push ImageTag: myapp:abc123Push to container registry] Push2Registry --> DeployStaging[Stage 8: Deploy to Stagingkubectl apply -f staging/Run smoke tests] DeployStaging --> SmokeTest[Stage 9: Smoke TestsTest critical pathsHealth checksBasic functionality] SmokeTest --> SmokeResult{Smoke testspassed?} SmokeResult -->|No| StagingFail[❌ Pipeline FailedStaging deployment issuesRollback staging] SmokeResult -->|Yes| Approval{Manualapprovalrequired?} Approval -->|Yes| WaitApproval[⏸️ Waiting for ApprovalNotify team leadReview changes] WaitApproval --> ApprovalDecision{Approved?} ApprovalDecision -->|No| Rejected[❌ Deployment RejectedNot ready for production] ApprovalDecision -->|Yes| DeployProd Approval -->|No| DeployProd[Stage 10: Deploy to ProductionRolling updateOr blue-green deployment] DeployProd --> ProdHealth{Productionhealthy?} ProdHealth -->|No| AutoRollback[❌ Auto-RollbackRevert to previous versionAlert on-call team] ProdHealth -->|Yes| Success[✅ Deployment Successful!Monitor metricsNotify teamUpdate status] style LintFail fill:#7f1d1d,stroke:#ef4444 style TestFail fill:#7f1d1d,stroke:#ef4444 style BuildFail fill:#7f1d1d,stroke:#ef4444 style IntegFail fill:#7f1d1d,stroke:#ef4444 style SecFail fill:#7f1d1d,stroke:#ef4444 style StagingFail fill:#7f1d1d,stroke:#ef4444 style AutoRollback fill:#7f1d1d,stroke:#ef4444 style Success fill:#064e3b,stroke:#10b981 style WaitApproval fill:#78350f,stroke:#f59e0b Part 2: Continuous Integration (CI) Stages CI Pipeline Detailed Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% sequenceDiagram participant Dev as Developer participant Git as Git Repository participant CI as CI Server participant Docker as Docker Registry participant Notify as Slack/Email Dev->>Git: git push origin feature/new-api Note over Git: Webhook triggeredon push event Git->>CI: Trigger pipeline:Branch: feature/new-apiCommit: abc123Author: [email protected] CI->>CI: Create build environmentUbuntu 22.04 container CI->>Git: git clone --depth 1Checkout abc123 Note over CI: Stage 1: Setup CI->>CI: Install dependenciesnpm installgo mod download Note over CI: Stage 2: Code Quality CI->>CI: Run lintereslint src/golangci-lint run alt Linting Failed CI->>Notify: ❌ Linting failed26 issues foundFix formatting CI-->>Dev: Pipeline failed end Note over CI: Stage 3: Unit Testing CI->>CI: Run unit testsnpm testgo test ./... CI->>CI: Generate coverage reportCoverage: 87% alt Tests Failed or Low Coverage CI->>Notify: ❌ Tests failed5 tests failingCoverage: 72% < 80% CI-->>Dev: Pipeline failed end Note over CI: Stage 4: Build CI->>CI: Build applicationnpm run buildgo build -o app CI->>CI: Build Docker imagedocker build -t myapp:abc123 alt Build Failed CI->>Notify: ❌ Build failedCompilation errors CI-->>Dev: Pipeline failed end Note over CI: Stage 5: Integration Tests CI->>CI: Start test dependenciesdocker-compose up -dpostgres, redis CI->>CI: Run integration testsTest database connectionsTest API endpoints CI->>CI: Stop test servicesdocker-compose down alt Integration Tests Failed CI->>Notify: ❌ Integration tests failedDatabase connection timeout CI-->>Dev: Pipeline failed end Note over CI: Stage 6: Security Scanning CI->>CI: Scan dependenciesnpm auditsnyk test CI->>CI: Scan Docker imagetrivy image myapp:abc123 alt Security Issues Found CI->>Notify: ⚠️ Security issues3 high severity CVEsUpdate dependencies CI-->>Dev: Pipeline failed end Note over CI: All checks passed! ✓ CI->>Docker: docker push myapp:abc123Tag: myapp:latest Docker-->>CI: Image pushed successfully CI->>Notify: ✅ Build successful!Image: myapp:abc123Ready for deployment CI-->>Dev: Pipeline succeededDuration: 8m 32s GitHub Actions CI Configuration # .github/workflows/ci.yml name: CI Pipeline on: push: branches: [ main, develop ] pull_request: branches: [ main ] env: REGISTRY: ghcr.io IMAGE_NAME: ${{ github.repository }} jobs: # Job 1: Code Quality Checks lint: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Node.js uses: actions/setup-node@v3 with: node-version: '18' cache: 'npm' - name: Install dependencies run: npm ci - name: Run ESLint run: npm run lint - name: Run Prettier run: npm run format:check # Job 2: Unit Tests test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Setup Node.js uses: actions/setup-node@v3 with: node-version: '18' cache: 'npm' - name: Install dependencies run: npm ci - name: Run tests run: npm test -- --coverage - name: Check coverage threshold run: | COVERAGE=$(cat coverage/coverage-summary.json | jq '.total.lines.pct') if (( $(echo "$COVERAGE < 80" | bc -l) )); then echo "Coverage $COVERAGE% is below 80%" exit 1 fi - name: Upload coverage to Codecov uses: codecov/codecov-action@v3 # Job 3: Build build: runs-on: ubuntu-latest needs: [lint, test] # Wait for lint and test to pass steps: - uses: actions/checkout@v3 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v2 - name: Log in to GitHub Container Registry uses: docker/login-action@v2 with: registry: ${{ env.REGISTRY }} username: ${{ github.actor }} password: ${{ secrets.GITHUB_TOKEN }} - name: Extract metadata id: meta uses: docker/metadata-action@v4 with: images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} tags: | type=sha,prefix={{branch}}- type=ref,event=branch type=ref,event=pr - name: Build and push Docker image uses: docker/build-push-action@v4 with: context: . push: true tags: ${{ steps.meta.outputs.tags }} cache-from: type=gha cache-to: type=gha,mode=max # Job 4: Integration Tests integration-test: runs-on: ubuntu-latest needs: build services: postgres: image: postgres:15 env: POSTGRES_PASSWORD: postgres options: >- --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5 redis: image: redis:7 options: >- --health-cmd "redis-cli ping" --health-interval 10s --health-timeout 5s --health-retries 5 steps: - uses: actions/checkout@v3 - name: Setup Node.js uses: actions/setup-node@v3 with: node-version: '18' cache: 'npm' - name: Install dependencies run: npm ci - name: Run integration tests run: npm run test:integration env: DATABASE_URL: postgresql://postgres:postgres@localhost:5432/test REDIS_URL: redis://localhost:6379 # Job 5: Security Scan security: runs-on: ubuntu-latest needs: build steps: - uses: actions/checkout@v3 - name: Run npm audit run: npm audit --audit-level=high - name: Run Snyk security scan uses: snyk/actions/node@master env: SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }} - name: Scan Docker image with Trivy uses: aquasecurity/trivy-action@master with: image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} format: 'sarif' output: 'trivy-results.sarif' - name: Upload Trivy results to GitHub Security uses: github/codeql-action/upload-sarif@v2 with: sarif_file: 'trivy-results.sarif' Part 3: Continuous Deployment (CD) Stages Deployment Pipeline Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% flowchart TD Start([CI Pipeline PassedImage ready: myapp:abc123]) --> DeployDecision{Whichbranch?} DeployDecision -->|feature/*| SkipDeploy[Skip deploymentCI only forfeature branches] DeployDecision -->|develop| DeployDev[Deploy to Dev EnvironmentNamespace: devAuto-deploy on push] DeployDecision -->|main| DeployStaging[Deploy to StagingNamespace: stagingAuto-deploy on push] DeployDev --> DevSmoke[Run smoke testsBasic health checks] DevSmoke --> DevDone[✅ Dev deployment complete] DeployStaging --> UpdateManifest[Update Kubernetes manifestsimage: myapp:abc123Apply configuration] UpdateManifest --> ApplyStaging[kubectl apply -f k8s/staging/Create/Update resourcesWait for rollout] ApplyStaging --> WaitReady{All podsready?} WaitReady -->|No timeout| CheckHealth[Check pod statuskubectl get pods -n staging] CheckHealth --> HealthStatus{Healthy?} HealthStatus -->|No| RollbackStaging[❌ Rollback stagingkubectl rollout undodeployment myapp -n staging] RollbackStaging --> NotifyFail[Notify team:Staging deployment failedCheck logs and fix] HealthStatus -->|Yes| StagingSmoke[Run staging smoke tests- Health endpoint- Critical API endpoints- Database connectivity] StagingSmoke --> SmokePass{Smoke testspassed?} SmokePass -->|No| RollbackStaging SmokePass -->|Yes| StagingReady[✅ Staging ReadyAll tests passedReady for production] StagingReady --> ApprovalGate{Require manualapproval?} ApprovalGate -->|Yes| WaitApproval[⏸️ Wait for approvalCreate deployment requestNotify reviewers] WaitApproval --> ReviewDecision{Approvedby reviewer?} ReviewDecision -->|No| Rejected[❌ Deployment rejectedFeedback providedMake changes] ReviewDecision -->|Yes| DeployProd ApprovalGate -->|No| DeployProd[Deploy to ProductionNamespace: productionStrategy: Rolling update] DeployProd --> BackupProd[Create backup:- Current deployment state- Database snapshot- Config backup] BackupProd --> ApplyProd[kubectl apply -f k8s/prod/Rolling update:maxSurge: 1maxUnavailable: 0] ApplyProd --> MonitorRollout[Monitor rollout statuskubectl rollout statusdeployment myapp -n production] MonitorRollout --> ProdHealth{All new podshealthy?} ProdHealth -->|No| AutoRollback[🚨 Auto-rollback triggeredkubectl rollout undoRestore previous version] AutoRollback --> AlertTeam[Alert on-call teamPagerDuty notificationProduction incident] ProdHealth -->|Yes| ProdMonitor[Monitor production metrics- Error rates- Latency- Business KPIs] ProdMonitor --> MetricsOK{Metricshealthy for10 minutes?} MetricsOK -->|No| AutoRollback MetricsOK -->|Yes| Complete[✅ Deployment Complete!Production healthyNew version liveUpdate status page] Complete --> CleanupOld[Cleanup old resourcesRemove old replica setsPrune old images] style SkipDeploy fill:#1e3a8a,stroke:#3b82f6 style WaitApproval fill:#78350f,stroke:#f59e0b style RollbackStaging fill:#7f1d1d,stroke:#ef4444 style AutoRollback fill:#7f1d1d,stroke:#ef4444 style Complete fill:#064e3b,stroke:#10b981 style DevDone fill:#064e3b,stroke:#10b981 Part 4: Quality Gates Quality Gate Decision Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% flowchart TD Start([Code ready to deploy]) --> Gate1{Quality Gate 1:Code Quality} Gate1 --> CheckLint[Check LintingESLint, Prettier] Gate1 --> CheckComplexity[Check ComplexityCyclomatic complexity< 10 per function] Gate1 --> CheckDuplication[Check DuplicationCode duplication < 3%] CheckLint --> LintScore{Pass?} CheckComplexity --> ComplexScore{Pass?} CheckDuplication --> DupScore{Pass?} LintScore -->|No| Fail1[❌ Gate 1 Failed] ComplexScore -->|No| Fail1 DupScore -->|No| Fail1 LintScore -->|Yes| Gate2{Quality Gate 2:Testing} ComplexScore -->|Yes| Gate2 DupScore -->|Yes| Gate2 Gate2 --> CheckCoverage[Check CoverageLine coverage >= 80%Branch coverage >= 75%] Gate2 --> CheckTests[All Tests PassUnit + Integration] Gate2 --> CheckPerf[Performance TestsResponse time < baseline] CheckCoverage --> CovScore{Pass?} CheckTests --> TestScore{Pass?} CheckPerf --> PerfScore{Pass?} CovScore -->|No| Fail2[❌ Gate 2 Failed] TestScore -->|No| Fail2 PerfScore -->|No| Fail2 CovScore -->|Yes| Gate3{Quality Gate 3:Security} TestScore -->|Yes| Gate3 PerfScore -->|Yes| Gate3 Gate3 --> CheckVuln[Scan VulnerabilitiesNo high/critical CVEs] Gate3 --> CheckSecrets[Check for SecretsNo hardcoded credentials] Gate3 --> CheckDeps[Dependency CheckAll deps up-to-date] CheckVuln --> VulnScore{Pass?} CheckSecrets --> SecretScore{Pass?} CheckDeps --> DepScore{Pass?} VulnScore -->|No| Fail3[❌ Gate 3 Failed] SecretScore -->|No| Fail3 DepScore -->|No| Fail3 VulnScore -->|Yes| Gate4{Quality Gate 4:Production Readiness} SecretScore -->|Yes| Gate4 DepScore -->|Yes| Gate4 Gate4 --> CheckHealth[Health ChecksLiveness + Readiness] Gate4 --> CheckResources[Resource LimitsCPU + Memory defined] Gate4 --> CheckDocs[DocumentationREADME + API docs] CheckHealth --> HealthScore{Pass?} CheckResources --> ResScore{Pass?} CheckDocs --> DocScore{Pass?} HealthScore -->|No| Fail4[❌ Gate 4 Failed] ResScore -->|No| Fail4 DocScore -->|No| Fail4 HealthScore -->|Yes| AllGates[✅ All Quality Gates Passed!Ready for deployment] ResScore -->|Yes| AllGates DocScore -->|Yes| AllGates Fail1 --> Block[Block deploymentFix issues first] Fail2 --> Block Fail3 --> Block Fail4 --> Block style Fail1 fill:#7f1d1d,stroke:#ef4444 style Fail2 fill:#7f1d1d,stroke:#ef4444 style Fail3 fill:#7f1d1d,stroke:#ef4444 style Fail4 fill:#7f1d1d,stroke:#ef4444 style AllGates fill:#064e3b,stroke:#10b981 Part 5: GitLab CI/CD Example .gitlab-ci.yml Configuration # .gitlab-ci.yml stages: - lint - test - build - security - deploy-staging - deploy-production variables: DOCKER_DRIVER: overlay2 DOCKER_TLS_CERTDIR: "/certs" IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA # Template for Docker jobs .docker-login: &docker-login before_script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY # Stage 1: Linting lint:code: stage: lint image: node:18 script: - npm ci - npm run lint - npm run format:check cache: paths: - node_modules/ # Stage 2: Testing test:unit: stage: test image: node:18 script: - npm ci - npm test -- --coverage - | COVERAGE=$(cat coverage/coverage-summary.json | jq '.total.lines.pct') if (( $(echo "$COVERAGE < 80" | bc -l) )); then echo "Coverage $COVERAGE% is below threshold" exit 1 fi coverage: '/Lines\s*:\s*(\d+\.\d+)%/' artifacts: reports: coverage_report: coverage_format: cobertura path: coverage/cobertura-coverage.xml test:integration: stage: test image: node:18 services: - name: postgres:15 alias: postgres - name: redis:7 alias: redis variables: DATABASE_URL: postgresql://postgres:postgres@postgres:5432/test REDIS_URL: redis://redis:6379 script: - npm ci - npm run test:integration # Stage 3: Build build:image: stage: build image: docker:24 services: - docker:24-dind <<: *docker-login script: - docker build -t $IMAGE_TAG . - docker push $IMAGE_TAG - docker tag $IMAGE_TAG $CI_REGISTRY_IMAGE:latest - docker push $CI_REGISTRY_IMAGE:latest only: - main - develop # Stage 4: Security Scanning security:scan: stage: security image: aquasec/trivy:latest script: - trivy image --severity HIGH,CRITICAL --exit-code 1 $IMAGE_TAG allow_failure: true security:sast: stage: security image: node:18 script: - npm audit --audit-level=high - npx snyk test --severity-threshold=high allow_failure: true # Stage 5: Deploy to Staging deploy:staging: stage: deploy-staging image: bitnami/kubectl:latest script: - kubectl config set-cluster k8s --server="$K8S_SERVER" - kubectl config set-credentials admin --token="$K8S_TOKEN" - kubectl config set-context default --cluster=k8s --user=admin - kubectl config use-context default - | kubectl set image deployment/myapp \ myapp=$IMAGE_TAG \ -n staging - kubectl rollout status deployment/myapp -n staging --timeout=5m - kubectl get pods -n staging environment: name: staging url: https://staging.example.com only: - main # Stage 6: Deploy to Production deploy:production: stage: deploy-production image: bitnami/kubectl:latest script: - kubectl config set-cluster k8s --server="$K8S_SERVER" - kubectl config set-credentials admin --token="$K8S_TOKEN" - kubectl config set-context default --cluster=k8s --user=admin - kubectl config use-context default - | kubectl set image deployment/myapp \ myapp=$IMAGE_TAG \ -n production - kubectl rollout status deployment/myapp -n production --timeout=10m - | # Check pod health READY=$(kubectl get deployment myapp -n production -o jsonpath='{.status.readyReplicas}') DESIRED=$(kubectl get deployment myapp -n production -o jsonpath='{.spec.replicas}') if [ "$READY" != "$DESIRED" ]; then echo "Deployment unhealthy: $READY/$DESIRED pods ready" kubectl rollout undo deployment/myapp -n production exit 1 fi environment: name: production url: https://example.com when: manual # Require manual approval only: - main Part 6: Pipeline Best Practices Pipeline Optimization Fast Feedback Loop: ...

Introduction Multi-environment pipelines enable safe, progressive deployment of code changes through isolated environments. Each environment serves a specific purpose in validating changes before they reach production users. This guide visualizes the multi-environment deployment flow: Environment Hierarchy: Dev → Staging → Production Environment Isolation: Separate configs, databases, resources Progressive Promotion: Automated testing at each stage Approval Gates: Manual checkpoints for production Configuration Management: Environment-specific settings Part 1: Multi-Environment Architecture Complete Environment Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% flowchart TD Dev([👨‍💻 Developer]) --> LocalDev[Local DevelopmentLaptop/Docker DesktopFast iteration] LocalDev --> Push[git push origin feature/new-api] Push --> CI[CI Pipeline TriggeredBuild + Test + Lint] CI --> CIPass{CIPassed?} CIPass -->|No| FixLocal[❌ Fix locallyCheck logsRun tests] FixLocal -.-> LocalDev CIPass -->|Yes| FeatureBranch{Branchtype?} FeatureBranch -->|feature/*| DevEnv[🔧 Dev EnvironmentNamespace: devAuto-deploy on push] FeatureBranch -->|main| StagingEnv[🎯 Staging EnvironmentNamespace: stagingAuto-deploy on merge] subgraph DevEnvironment[Development Environment] DevEnv --> DevConfig[Configuration:- Debug mode ON- Verbose logging- Mock external APIs- Dev database- Minimal replicas: 1] DevConfig --> DevTest[Basic Tests:- Smoke tests- Health checks- Manual QA] DevTest --> DevDone[✅ Dev validatedReady for staging] end DevDone --> MergePR[Merge Pull Requestto main branch] MergePR --> StagingEnv subgraph StagingEnvironment[Staging Environment] StagingEnv --> StagingConfig[Configuration:- Production-like setup- Staging database- Real external APIs test- Replicas: 2-3- Resource limits] StagingConfig --> StagingTest[Comprehensive Tests:- Integration tests- E2E tests- Performance tests- Security scans] StagingTest --> StagingResult{All testspassed?} StagingResult -->|No| StagingFail[❌ Staging failedRollback stagingFix issues] StagingFail -.-> FixLocal StagingResult -->|Yes| StagingMonitor[Monitor staging:- Error rates- Performance metrics- User acceptance testing] StagingMonitor --> StagingReady[✅ Staging validatedReady for production] end StagingReady --> ApprovalGate{ManualApprovalRequired} ApprovalGate --> ReviewTeam[Team Lead Review:- Code changes- Test results- Risk assessment- Deployment timing] ReviewTeam --> Approved{Approved?} Approved -->|No| Rejected[❌ RejectedMore testing neededor wrong timing] Approved -->|Yes| ProdEnv[🚀 Production EnvironmentNamespace: productionManual trigger only] subgraph ProductionEnvironment[Production Environment] ProdEnv --> ProdConfig[Configuration:- Production settings- Production database- High availability- Replicas: 5-10- Strict resource limits- Auto-scaling enabled] ProdConfig --> ProdDeploy[Deployment Strategy:- Blue-green or- Canary or- Rolling update] ProdDeploy --> ProdHealth{Productionhealthy?} ProdHealth -->|No| AutoRollback[🚨 Auto-rollbackRevert to previousAlert on-call team] ProdHealth -->|Yes| ProdMonitor[Monitor Production:- Real user metrics- Error rates- Business KPIs- SLO compliance] ProdMonitor --> ProdStable{Stable for15 minutes?} ProdStable -->|No| AutoRollback ProdStable -->|Yes| Success[✅ Deployment Complete!New version liveMonitor continues] end style DevEnv fill:#064e3b,stroke:#10b981 style StagingEnv fill:#78350f,stroke:#f59e0b style ProdEnv fill:#1e3a8a,stroke:#3b82f6 style Success fill:#064e3b,stroke:#10b981 style StagingFail fill:#7f1d1d,stroke:#ef4444 style AutoRollback fill:#7f1d1d,stroke:#ef4444 style Rejected fill:#7f1d1d,stroke:#ef4444 Part 2: Environment Comparison Environment Characteristics %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% graph TB subgraph Local[🏠 Local Development] LocalProps[Properties:✓ Fast iteration✓ Developer's laptop✓ Docker Compose✓ Mock services✓ Hot reload enabled] LocalData[Data:- SQLite or local DB- Seed data- No real user data- Quick reset] LocalAccess[Access:- localhost only- No authentication- Debug tools enabled] end subgraph Dev[🔧 Development Environment] DevProps[Properties:✓ Shared team env✓ Kubernetes cluster✓ Continuous deployment✓ Latest features✓ Can be unstable] DevData[Data:- Dev database- Synthetic test data- Reset weekly- No PII] DevAccess[Access:- VPN required- Basic auth- All developers- Debug mode ON] end subgraph Staging[🎯 Staging Environment] StagingProps[Properties:✓ Production mirror✓ Same infrastructure✓ Pre-production testing✓ Stable builds only✓ Performance testing] StagingData[Data:- Staging database- Anonymized prod data- Or realistic test data- Refreshed monthly] StagingAccess[Access:- VPN required- OAuth/SSO- Developers + QA- Debug mode OFF] end subgraph Prod[🚀 Production Environment] ProdProps[Properties:✓ Live customer traffic✓ High availability✓ Auto-scaling✓ Disaster recovery✓ Maximum stability] ProdData[Data:- Production database- Real user data- Encrypted at rest- Regular backups] ProdAccess[Access:- Public internet- Full authentication- Limited admin access- Audit logging enabled] end Local --> |git push feature/*| Dev Dev --> |Merge to main| Staging Staging --> |Manual approval| Prod style Local fill:#064e3b,stroke:#10b981 style Dev fill:#064e3b,stroke:#10b981 style Staging fill:#78350f,stroke:#f59e0b style Prod fill:#1e3a8a,stroke:#3b82f6 Environment Configuration Matrix Aspect Local Dev Staging Production Purpose Development Feature testing Pre-production validation Live users Deployment Manual Auto on push Auto on merge Manual approval Replicas 1 1-2 2-3 5-10+ Database Local SQLite Shared dev DB Staging DB (prod-like) Production DB Resources Minimal Low Medium (prod-like) High Monitoring None Basic Full Full + Alerts Debug Mode Yes Yes No No Logging Level DEBUG DEBUG INFO WARN/ERROR External APIs Mocked Test endpoints Test endpoints Production endpoints Data Seed data Synthetic Anonymized Real user data Access localhost VPN + Basic auth VPN + SSO Public + Full auth Uptime SLA N/A None None 99.9%+ Part 3: Progressive Promotion Pipeline Promotion Flow with Quality Gates %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% flowchart LR subgraph LocalStage[Local Stage] L1[Write Code] L2[Run Unit Tests] L3[Manual Testing] L1 --> L2 --> L3 end subgraph DevStage[Dev Stage] D1[Auto Deploy] D2[Smoke Tests] D3{TestsPass?} D4[Dev Validated ✓] D1 --> D2 --> D3 D3 -->|Yes| D4 D3 -->|No| D5[❌ Fix] D5 -.-> L1 end subgraph StagingStage[Staging Stage] S1[Auto Deploy] S2[Integration Tests] S3[E2E Tests] S4[Performance Tests] S5{All Pass?} S6[Staging Validated ✓] S1 --> S2 --> S3 --> S4 --> S5 S5 -->|Yes| S6 S5 -->|No| S7[❌ Fix] S7 -.-> L1 end subgraph ApprovalStage[Approval Gate] A1[Create Release] A2[Code Review] A3[Change Advisory] A4{Approved?} A1 --> A2 --> A3 --> A4 A4 -->|No| A5[❌ Rejected] A5 -.-> L1 end subgraph ProdStage[Production Stage] P1[Manual Deploy] P2[Canary 10%] P3{Healthy?} P4[Increase to 50%] P5{Healthy?} P6[Complete 100%] P7[Monitor] P8[Success ✓] P1 --> P2 --> P3 P3 -->|Yes| P4 --> P5 P5 -->|Yes| P6 --> P7 --> P8 P3 -->|No| P9[🚨 Rollback] P5 -->|No| P9 end L3 --> |git push| D1 D4 --> |Merge PR| S1 S6 --> A1 A4 -->|Yes| P1 style L3 fill:#064e3b,stroke:#10b981 style D4 fill:#064e3b,stroke:#10b981 style S6 fill:#064e3b,stroke:#10b981 style P8 fill:#064e3b,stroke:#10b981 style D5 fill:#7f1d1d,stroke:#ef4444 style S7 fill:#7f1d1d,stroke:#ef4444 style P9 fill:#7f1d1d,stroke:#ef4444 Part 4: Environment-Specific Configuration Configuration Management Strategy %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% flowchart TD Start([Application needs config]) --> Method{ConfigMethod?} Method --> EnvVars[Environment Variables] Method --> ConfigMaps[Kubernetes ConfigMaps] Method --> Secrets[Kubernetes Secrets] EnvVars --> EnvExample[Examples:- NODE_ENV=production- LOG_LEVEL=info- FEATURE_FLAGS=true] ConfigMaps --> CMExample[Examples:- app-config.yaml- nginx.conf- application.properties] Secrets --> SecretExample[Examples:- DATABASE_PASSWORD- API_KEYS- TLS certificates] EnvExample --> Override{Override perenvironment?} CMExample --> Override SecretExample --> Override Override --> DevOverride[Dev Environment:DEBUG=trueDB_HOST=dev-dbREPLICAS=1CACHE_TTL=60s] Override --> StagingOverride[Staging Environment:DEBUG=falseDB_HOST=staging-dbREPLICAS=3CACHE_TTL=300s] Override --> ProdOverride[Production Environment:DEBUG=falseDB_HOST=prod-dbREPLICAS=10CACHE_TTL=600s] DevOverride --> Inject[Inject at deployment:kubectl apply -f k8s/dev/- deployment.yaml- configmap.yaml- secrets.yaml] StagingOverride --> Inject ProdOverride --> Inject style EnvVars fill:#1e3a8a,stroke:#3b82f6 style ConfigMaps fill:#1e3a8a,stroke:#3b82f6 style Secrets fill:#7f1d1d,stroke:#ef4444 Kubernetes Configuration Example # k8s/base/deployment.yaml (Common base) apiVersion: apps/v1 kind: Deployment metadata: name: myapp spec: selector: matchLabels: app: myapp template: metadata: labels: app: myapp spec: containers: - name: myapp image: myapp:latest # Overridden per environment ports: - containerPort: 8080 envFrom: - configMapRef: name: myapp-config - secretRef: name: myapp-secrets resources: # Overridden per environment requests: memory: "128Mi" cpu: "100m" limits: memory: "256Mi" cpu: "200m" --- # k8s/dev/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: myapp-config namespace: dev data: NODE_ENV: "development" LOG_LEVEL: "debug" DATABASE_HOST: "postgres.dev.svc.cluster.local" REDIS_HOST: "redis.dev.svc.cluster.local" FEATURE_NEW_UI: "true" FEATURE_BETA_API: "true" --- # k8s/staging/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: myapp-config namespace: staging data: NODE_ENV: "staging" LOG_LEVEL: "info" DATABASE_HOST: "postgres.staging.svc.cluster.local" REDIS_HOST: "redis.staging.svc.cluster.local" FEATURE_NEW_UI: "true" FEATURE_BETA_API: "false" --- # k8s/production/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: myapp-config namespace: production data: NODE_ENV: "production" LOG_LEVEL: "warn" DATABASE_HOST: "postgres.production.svc.cluster.local" REDIS_HOST: "redis.production.svc.cluster.local" FEATURE_NEW_UI: "false" # Gradual rollout FEATURE_BETA_API: "false" --- # k8s/dev/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: dev resources: - ../base/deployment.yaml - configmap.yaml - secrets.yaml images: - name: myapp newTag: dev-abc123 replicas: - name: myapp count: 1 patches: - patch: |- - op: replace path: /spec/template/spec/containers/0/resources/requests/memory value: 128Mi - op: replace path: /spec/template/spec/containers/0/resources/limits/memory value: 256Mi target: kind: Deployment name: myapp --- # k8s/production/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: production resources: - ../base/deployment.yaml - configmap.yaml - secrets.yaml images: - name: myapp newTag: v1.2.3 replicas: - name: myapp count: 10 patches: - patch: |- - op: replace path: /spec/template/spec/containers/0/resources/requests/memory value: 512Mi - op: replace path: /spec/template/spec/containers/0/resources/limits/memory value: 1Gi - op: replace path: /spec/template/spec/containers/0/resources/requests/cpu value: 500m - op: replace path: /spec/template/spec/containers/0/resources/limits/cpu value: 1000m target: kind: Deployment name: myapp Part 5: Database Migration Strategy Multi-Environment Database Flow %%{init: {'theme':'dark', 'themeVariables': {'primaryTextColor':'#e5e7eb','secondaryTextColor':'#e5e7eb','tertiaryTextColor':'#e5e7eb','textColor':'#e5e7eb','nodeTextColor':'#e5e7eb','edgeLabelText':'#e5e7eb','clusterTextColor':'#e5e7eb','actorTextColor':'#e5e7eb'}}}%% sequenceDiagram participant Dev as Developer participant DevDB as Dev Database participant StagingDB as Staging Database participant ProdDB as Production Database participant Migration as Migration Tool Note over Dev: Write migration:001_add_users_table.sql Dev->>DevDB: Run migration locallyCREATE TABLE users... DevDB-->>Dev: Migration applied ✓ Dev->>Dev: Test applicationwith new schema Dev->>Dev: git push feature/add-users Note over DevDB: CI/CD Pipeline triggered Dev->>DevDB: Auto-run migrationsin dev environment DevDB-->>Dev: Dev DB updated ✓ Note over Dev: Create Pull RequestMerge to main Dev->>StagingDB: Trigger staging deployment Note over Migration,StagingDB: Pre-deployment hook Migration->>StagingDB: Backup databasepg_dump > backup.sql Migration->>StagingDB: Run migrations001_add_users_table.sql StagingDB-->>Migration: Migration applied ✓ Note over StagingDB: Deploy applicationTest with new schema alt Migration Failed Migration->>StagingDB: Rollback migrationRestore from backup StagingDB-->>Migration: Rolled back end Note over Dev: Manual approvalfor production Dev->>ProdDB: Trigger production deployment Note over Migration,ProdDB: Pre-deployment steps Migration->>ProdDB: Full database backupSnapshot created Migration->>ProdDB: Check migration statusSELECT version FROM schema_migrations ProdDB-->>Migration: Current version: 000 Migration->>ProdDB: Run migrationsin transaction Note over Migration,ProdDB: BEGIN;CREATE TABLE users;INSERT INTO schema_migrationsVALUES ('001');COMMIT; ProdDB-->>Migration: Migration successful ✓ Note over ProdDB: Deploy new applicationversion alt Production Issues Migration->>ProdDB: Rollback migrationRun down migration:DROP TABLE users; Note over ProdDB: Deploy previousapplication version end Migration->>ProdDB: Verify data integrityCheck constraints ProdDB-->>Migration: All checks passed ✓ Note over Dev,ProdDB: Production updated successfully Part 6: Multi-Environment CI/CD Pipeline Complete Pipeline Configuration # .github/workflows/multi-env-deploy.yml name: Multi-Environment Deployment on: push: branches: - main - develop pull_request: branches: - main env: REGISTRY: ghcr.io IMAGE_NAME: ${{ github.repository }} jobs: # CI - Same for all environments build-and-test: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Run linting run: npm run lint - name: Run unit tests run: npm test - name: Build Docker image run: docker build -t $IMAGE_NAME:${{ github.sha }} . - name: Run integration tests run: docker-compose -f docker-compose.test.yml up --abort-on-container-exit - name: Push image run: | echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin docker push $IMAGE_NAME:${{ github.sha }} # Deploy to Dev - Auto on feature branches deploy-dev: needs: build-and-test if: github.ref != 'refs/heads/main' runs-on: ubuntu-latest environment: name: development url: https://dev.example.com steps: - uses: actions/checkout@v3 - name: Deploy to Dev run: | kubectl config set-cluster dev --server="${{ secrets.DEV_K8S_SERVER }}" kubectl config set-credentials admin --token="${{ secrets.DEV_K8S_TOKEN }}" kubectl set image deployment/myapp myapp=$IMAGE_NAME:${{ github.sha }} -n dev kubectl rollout status deployment/myapp -n dev - name: Run smoke tests run: | curl https://dev.example.com/health npm run test:smoke -- --env=dev # Deploy to Staging - Auto on main branch deploy-staging: needs: build-and-test if: github.ref == 'refs/heads/main' runs-on: ubuntu-latest environment: name: staging url: https://staging.example.com steps: - uses: actions/checkout@v3 - name: Run database migrations run: | kubectl exec -n staging deployment/postgres -- \ psql -U postgres -d app -f /migrations/migrate.sql - name: Deploy to Staging run: | kubectl config set-cluster staging --server="${{ secrets.STAGING_K8S_SERVER }}" kubectl config set-credentials admin --token="${{ secrets.STAGING_K8S_TOKEN }}" kubectl apply -k k8s/staging/ kubectl rollout status deployment/myapp -n staging --timeout=5m - name: Run E2E tests run: npm run test:e2e -- --env=staging - name: Run performance tests run: | k6 run --vus 10 --duration 30s tests/performance.js - name: Check staging health run: | curl https://staging.example.com/health | jq '.status' | grep -q "healthy" # Deploy to Production - Manual approval required deploy-production: needs: deploy-staging runs-on: ubuntu-latest environment: name: production url: https://example.com steps: - uses: actions/checkout@v3 - name: Backup production database run: | kubectl exec -n production deployment/postgres -- \ pg_dump -U postgres app > backup-$(date +%Y%m%d-%H%M%S).sql - name: Run database migrations run: | kubectl exec -n production deployment/postgres -- \ psql -U postgres -d app -f /migrations/migrate.sql - name: Deploy to Production (Blue-Green) run: | kubectl config set-cluster prod --server="${{ secrets.PROD_K8S_SERVER }}" kubectl config set-credentials admin --token="${{ secrets.PROD_K8S_TOKEN }}" # Deploy green version kubectl apply -k k8s/production/ kubectl rollout status deployment/myapp-green -n production --timeout=10m # Switch traffic to green kubectl patch service myapp -n production -p '{"spec":{"selector":{"version":"green"}}}' - name: Monitor production metrics run: | sleep 300 # Wait 5 minutes ERROR_RATE=$(curl -s prometheus.example.com/api/v1/query?query=rate5m) if [ "$ERROR_RATE" -gt "0.01" ]; then echo "Error rate too high, rolling back" kubectl patch service myapp -n production -p '{"spec":{"selector":{"version":"blue"}}}' exit 1 fi - name: Notify team if: success() uses: slackapi/slack-github-action@v1 with: payload: | { "text": "✅ Production deployment successful!", "version": "${{ github.sha }}", "deployed_by": "${{ github.actor }}" } Part 7: Best Practices Environment Management Checklist ✅ DO: ...