CI/CD Pipeline Guide

CI/CD Pipeline Guide

This document describes the comprehensive CI/CD pipeline implemented for the Bartendie application.

Overview

The CI/CD pipeline ensures code quality, runs comprehensive tests, and automates deployment. It consists of multiple workflows that run different checks based on the trigger event.

Workflows

1. Main CI/CD Pipeline (.github/workflows/deploy.yml)

Triggers:

• Push to main branch
• Pull requests to main branch
• Manual workflow dispatch

Jobs:

Backend Tests & Quality (backend-tests)

• Runtime: Ubuntu Latest with PostgreSQL 15
• Dependencies: Elixir 1.16, OTP 26
• Steps:
  • Install Elixir dependencies with caching
  • Check code formatting (mix format --check-formatted)
  • Run static analysis (mix credo --strict)
  • Compile with warnings as errors
  • Run full test suite with coverage (mix test --cover)
  • Upload test results and coverage reports

Frontend Unit Tests (frontend-tests)

• Runtime: Ubuntu Latest
• Dependencies: Node.js 18, pnpm
• Steps:
  • Install frontend dependencies with caching
  • Run ESLint (pnpm lint)
  • Run Vitest unit tests (pnpm test:run)
  • Generate test coverage (pnpm test:coverage)
  • Build frontend (pnpm build)
  • Upload test results and build artifacts

End-to-End Tests (e2e-tests)

• Runtime: Ubuntu Latest with PostgreSQL 15
• Dependencies: Elixir 1.16, Node.js 18, Playwright
• Steps:
  • Set up both backend and frontend environments
  • Install Playwright browsers
  • Set up test database
  • Build frontend for testing
  • Run Playwright E2E tests across multiple browsers
  • Upload test results, screenshots, and videos

Build Verification (build-verification)

• Runtime: Ubuntu Latest with PostgreSQL 15
• Dependencies: Full stack setup
• Steps:
  • Compile backend for production
  • Build frontend for production
  • Verify backend can start and respond
  • Upload production build artifacts

Deploy Documentation (deploy-docs)

• Condition: Only on main branch pushes
• Dependencies: All test jobs must pass
• Steps:
  • Build Docusaurus documentation
  • Deploy to Digital Ocean via SSH/rsync

Deploy Application (deploy-app)

• Condition: Only on main branch pushes
• Dependencies: All test jobs must pass
• Status: Placeholder for future implementation

Test Results Summary (test-summary)

• Condition: Always runs (even if tests fail)
• Dependencies: All test jobs
• Steps:
  • Download all test artifacts
  • Generate comprehensive test summary
  • Create GitHub Step Summary with results
  • Fail pipeline if any critical tests failed

2. PR Quality Checks (.github/workflows/pr-checks.yml)

Triggers:

• Pull request opened, synchronized, or reopened

Purpose: Fast quality checks for pull requests without running the full test suite.

Steps:

• Code formatting checks
• Static analysis (Credo)
• Compilation warnings check
• Frontend linting
• Build verification
• Automated PR comment with results

Environment Variables

Global Environment

ELIXIR_VERSION: '1.16'
OTP_VERSION: '26'
NODE_VERSION: '18'
MIX_ENV: test
DATABASE_URL: postgres://postgres:postgres@localhost:5432/bartendie_test
EVENTSTORE_URL: postgres://postgres:postgres@localhost:5432/bartendie_eventstore_test

Required Secrets

• SSH_PRIVATE_KEY: SSH private key for deployment
• SSH_KNOWN_HOSTS: Known hosts for SSH connection
• SSH_USER: SSH username for deployment
• SSH_HOST: SSH hostname for deployment

Database Configuration

PostgreSQL Services

Each job that needs database access includes a PostgreSQL 15 service:

services:
  postgres:
    image: postgres:15
    env:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: bartendie_test
    ports:
      - 5432:5432
    options: >-
      --health-cmd pg_isready
      --health-interval 10s
      --health-timeout 5s
      --health-retries 5

Database Setup

• Test databases are automatically created and migrated
• Event store databases are configured for event sourcing tests
• Health checks ensure database readiness before tests run

Caching Strategy

Elixir Dependencies

- uses: actions/cache@v4
  with:
    path: |
      deps
      _build
    key: ${{ runner.os }}-mix-${{ hashFiles('**/mix.lock') }}

Frontend Dependencies

- uses: actions/setup-node@v4
  with:
    node-version: ${{ env.NODE_VERSION }}
    cache: 'pnpm'
    cache-dependency-path: frontend/pnpm-lock.yaml

Test Artifacts

Collected Artifacts

• Backend: Test coverage reports, compiled artifacts
• Frontend: Test coverage, build output, lint results
• E2E: Playwright test results, screenshots, videos, traces
• Build: Production-ready artifacts for deployment

Retention

• All test artifacts are retained for 7 days
• Build artifacts are available for deployment jobs

Parallel Execution

Job Dependencies

backend-tests ──┐
frontend-tests ─┼── build-verification ──┐
e2e-tests ──────┘                        ├── deploy-docs
                                         ├── deploy-app
                                         └── test-summary

Optimization

• Backend and frontend tests run in parallel
• E2E tests run independently (can be parallelized further)
• Build verification waits for unit tests but not E2E
• Deployment only happens after all tests pass

Error Handling

Fail-Fast Strategy

• Compilation errors fail immediately
• Critical test failures stop dependent jobs
• Quality checks must pass for deployment

Comprehensive Reporting

• Test summary always runs (even on failures)
• All artifacts are collected regardless of test outcomes
• GitHub Step Summary provides detailed results

Status Badges

Add to your README.md:

[![CI/CD Pipeline](https://github.com/r26D/bartendie/actions/workflows/deploy.yml/badge.svg)](https://github.com/r26D/bartendie/actions/workflows/deploy.yml)
[![PR Checks](https://github.com/r26D/bartendie/actions/workflows/pr-checks.yml/badge.svg)](https://github.com/r26D/bartendie/actions/workflows/pr-checks.yml)

Local Development

Running Tests Locally

# Backend tests
mix test

# Frontend tests
cd frontend && pnpm test:run

# E2E tests
cd frontend && pnpm e2e

# Quality checks
mix quality.ci
cd frontend && pnpm lint

Debugging CI Issues

1. Check job logs in GitHub Actions
2. Download test artifacts for detailed analysis
3. Run tests locally with same environment
4. Use act tool to run GitHub Actions locally

Future Enhancements

Planned Improvements

1. Performance Testing: Add load testing with Artillery or k6
2. Security Scanning: Integrate SAST/DAST tools
3. Dependency Scanning: Automated vulnerability checks
4. Multi-environment Deployment: Staging → Production pipeline
5. Blue-Green Deployment: Zero-downtime deployments
6. Monitoring Integration: Post-deployment health checks

Deployment Targets

The current pipeline includes a placeholder for application deployment. Future implementations could target:

• Heroku: Simple PaaS deployment
• AWS: ECS, Lambda, or EC2 deployment
• Digital Ocean: App Platform or Droplet deployment
• Docker: Containerized deployment to any platform

Troubleshooting

Common Issues

1. Database Connection: Check PostgreSQL service health
2. Cache Issues: Clear cache and retry
3. Dependency Conflicts: Update lock files
4. Test Timeouts: Increase timeout values in config
5. Artifact Upload Failures: Check artifact paths and permissions

Debug Commands

# Check workflow syntax
act --list

# Validate workflow files
yamllint .github/workflows/

# Test specific job locally
act -j backend-tests