runState/docs/devops/phase-1-infra.md

# Infrastructure & Containerization (Phase 1 → Phase 2)

This document describes the infrastructure setup and containerization strategy implemented before introducing CI pipelines. It covers Dockerization, service orchestration, externalized services, and environment management.

---

# Phase 1 — Local Containerization

---

## 1. Go Backend Containerization

### Backend Location

- `apps/api-go`

### Dockerfile Strategy

A multi-stage Dockerfile is implemented.

#### Stage 1 — Builder

- Base Image: `golang:1.22-alpine`
- Installs build dependencies
- Copies `go.mod` and `go.sum`
- Downloads modules
- Compiles the Go binary

Purpose:
- Reduce final image size
- Ensure reproducible builds

#### Stage 2 — Runtime

- Minimal base image (Alpine)
- Copies compiled binary from builder stage
- Exposes required port
- Sets entrypoint

Outcome:
- Lightweight production-ready container
- No build tools included in runtime image

---

## 2. Docker Compose Orchestration

Docker Compose is used to orchestrate multi-service local environments.

Two modes are maintained:

---

### 2.1 Local Infrastructure Mode

File:
- `docker-compose.yml`

Services:

- `postgres`
- `redis`
- `api`
- `worker-monitoring`
- `worker-status-change`
- `worker-notification`

Characteristics:

- Postgres runs inside container
- Redis runs inside container
- All services communicate via internal Docker network
- Ports exposed for local development

Command:

docker compose up -d --build

Purpose:

- Simplify local development
- Ensure full system runs with one command
- Enable end-to-end testing

---

### 2.2 Internal Networking Design

- All services share Docker network
- Containers reference each other by service name
  - Example:
    - `postgres:5432`
    - `redis:6379`

Benefits:

- No hardcoded localhost dependencies
- Production-like internal service resolution

---

# Phase 2 — Externalized Services

Phase 2 replaces containerized infrastructure with managed cloud services.

---

## 3. External Database (Neon)

Instead of running Postgres locally:

- Uses managed Postgres (Neon)
- Connection string passed via:

DATABASE_URL

Example format:

postgresql://user:password@host/database?sslmode=require

Benefits:

- Production-like database
- No local container dependency
- Enables real-world SSL configurations

---

## 4. External Redis (Upstash)

Instead of local Redis container:

- Uses managed Redis instance (Upstash)

Configuration options:

- `REDIS_URL` (recommended)
- `REDIS_ADDR`

Connection logic:

- If `REDIS_URL` exists → parse URL
- Else → fallback to `REDIS_ADDR`

Benefits:

- Serverless Redis
- Production-grade latency testing
- No local Redis dependency

---

## 5. External Infrastructure Compose Mode

File:
- `docker-compose.external.yml`

Differences from local mode:

- No Postgres container
- No Redis container
- API and workers connect to cloud services

Command:

docker compose -f docker-compose.external.yml up -d --build

Purpose:

- Simulate production environment locally
- Validate cloud connectivity
- Test TLS configurations

---

## 6. Environment Configuration Strategy

Environment variables are separated per mode.

Files:

- `.env.local` → Local infra mode
- `.env` → Cloud infra mode

Common variables:

- DATABASE_URL
- REDIS_URL
- REDIS_ADDR
- Worker configuration values

This enables:

- Environment isolation
- Clean separation between dev and production configs
- Safe switching between modes

---

## 7. Service Isolation & Worker Architecture

Workers are separated into independent containers.

Examples:

- monitoring worker
- status-change worker
- notification worker

Each worker:

- Connects to Redis streams
- Processes events independently
- Has isolated lifecycle
- Can be horizontally scaled

Benefits:

- Fault isolation
- Production-like architecture
- Scalable worker model

---

## 8. Health Checks & Startup Dependencies

Compose configuration includes:

- Postgres healthcheck using `pg_isready`
- Services wait for dependencies before starting

Purpose:

- Prevent race conditions during startup
- Ensure DB readiness before API connects

---

## 9. Development Workflow Summary

### Local Mode

docker compose up -d --build

- All services run in Docker
- Full stack simulation

### External Mode

docker compose -f docker-compose.external.yml up -d --build

- Uses managed services
- Tests real-world connectivity

---

# Phase 1 & 2 Outcome

After completing Phase 1 and Phase 2:

- The Go backend is fully containerized.
- Services are isolated and orchestrated via Docker Compose.
- Infrastructure can run in two modes:
  - Fully local
  - Production-like cloud-managed
- Environment variables cleanly separate configurations.
- Workers are independently scalable.
- The system architecture mirrors production service separation.

This establishes a stable infrastructure foundation.

Next Phase:
- Continuous Integration (CI)
- Image publishing
- Branch protection
- Container registry workflows

runState/docs/devops/phase-3-ci.md

# DevOps Workflow (Phase 3)

This document describes the CI/CD and DevOps workflow implemented so far: Pull Request checks, Docker image build & push to GHCR, branch protection, caching, and where to find published images.

---

## 1. CI Workflows

All GitHub Actions workflows are stored under:

- `.github/workflows/`

Currently implemented:

- `pr-check-api-go.yml` → Pull Request checks (quality gate)
- `deploy-api-go.yml` → Build & push Docker image to GHCR on `main`

---

## 2. Pull Request CI (PR Check)

### Workflow File

- `.github/workflows/pr-check-api-go.yml`

### Trigger

- Runs on: `pull_request`

### Goal

- Prevent unformatted or broken Go code from being merged into `main`.

### Steps Executed

#### 2.1 Checkout Repository
- Checks out source code using `actions/checkout`.

#### 2.2 Setup Go Toolchain
- Installs Go `1.22` using `actions/setup-go`.
- Enables Go module caching using:
  - `cache: true`
  - `cache-dependency-path: apps/api-go/go.sum`

#### 2.3 gofmt Check (Formatting Gate)
- Runs `gofmt -l .` inside `apps/api-go`.
- If any file is returned by `gofmt -l`, the workflow fails and prints file names.

#### 2.4 go vet (Static Analysis)
- Runs `go vet ./...` inside `apps/api-go`.
- Detects common Go issues and suspicious patterns.

#### 2.5 go build (Compilation Gate)
- Runs `go build ./...` inside `apps/api-go`.
- Ensures code compiles successfully.

---

## 3. Main Branch CI (Build & Push Image)

### Workflow File

- `.github/workflows/deploy-api-go.yml`

### Trigger

- Runs on:
  - `push` to `main`
  - `workflow_dispatch` (manual run from GitHub UI)

### Goal

- Build a production Docker image for the Go backend and publish it to GHCR.

### Steps Executed

#### 3.1 Checkout Repository
- Pulls the repository source code in the GitHub Actions runner.

#### 3.2 Docker Hub Login (Base Image Pull)
- Logs into Docker Hub using repository secrets.
- Purpose:
  - Avoid auth/rate-limit issues when pulling base images like `golang:*` / `alpine:*`.

#### 3.3 Setup Docker Buildx
- Enables BuildKit / Buildx features (better caching, efficient builds).

#### 3.4 Login to GHCR (Push Permission)
- Logs into GitHub Container Registry using:
  - `username: ${{ github.actor }}`
  - `password: ${{ secrets.GITHUB_TOKEN }}`
- Requires workflow permissions:
  - `contents: read`
  - `packages: write`

#### 3.5 Docker Metadata (Tags + Labels)
- Uses `docker/metadata-action` to generate tags and labels.
- Image name:
  - `ghcr.io/<repo_owner>/runstate-api-go`

#### 3.6 Build and Push
- Uses `docker/build-push-action` to:
  - build from `apps/api-go/Dockerfile`
  - push to GHCR
  - apply tags generated by metadata action

---

## 4. Image Tagging Strategy

Each successful merge to `main` produces **two tags**:

- `latest`
- `sha-<commit>`

Example:

- `ghcr.io/<username>/runstate-api-go:latest`
- `ghcr.io/<username>/runstate-api-go:sha-abcdef1`

### Why Both Tags Exist

- `latest`:
  - Convenient for local pulls and “latest main build”.
  - Moves forward with every merge.

- `sha-*`:
  - Immutable.
  - Reproducible deployments.
  - Enables rollbacks by pinning to a specific commit build.

---

## 5. Docker Layer Caching (Speed Optimisation)

The build workflow uses GitHub Actions cache backend:

- `cache-from: type=gha`
- `cache-to: type=gha,mode=max`

### Behavior

- Restores previously stored build layers before building.
- Saves new layers after successful build.
- Reduces CI build time significantly when Dockerfile layers remain unchanged.

---

## 6. Branch Protection (Main Branch Governance)

`main` branch is protected via GitHub settings.

### Rules Enforced

- Pull Request required before merge
- Required status check must pass:
  - PR Check workflow (Go checks)
- Direct pushes to `main` blocked
- (Optional) Require branch to be up-to-date before merging

### Outcome

- No broken code reaches `main`.
- CI is mandatory for merges.

---

## 7. Pre-Commit Hook (Local Formatting Guard)

A pre-commit hook auto-runs `gofmt` to avoid CI failures.

### Hook Location

- `.githooks/pre-commit`

### Hook Logic

- Formats Go code:
  - `gofmt -w .`
- Stages updated files:
  - `git add .`

### Activation

Run once on your machine:

- `git config core.hooksPath .githooks`
- `chmod +x .githooks/pre-commit`

---

## 8. Where to Find Published Images (GHCR)

### Location in GitHub UI

- Repository page → **Packages** section
- Or GitHub profile → **Packages**

### Image Path

- `ghcr.io/<github-username>/runstate-api-go`

Example:

- `ghcr.io/RitikaxG/runstate-api-go`

### Pull Locally

- `docker pull ghcr.io/<username>/runstate-api-go:latest`
- `docker pull ghcr.io/<username>/runstate-api-go:sha-<commit>`

---

## 9. Local Development Modes

### 9.1 Local Infra Mode (Local Postgres + Redis)

From repo root:

- `docker compose up -d --build`

### 9.2 External Infra Mode (Neon + Upstash)

From repo root:

- `docker compose -f docker-compose.external.yml up -d --build`

---

## 10. Phase 3 Outcome

After completing Phase 3:

- PR CI prevents unformatted/unbuildable code from merging.
- `main` is protected and requires checks.
- Docker images are automatically built and pushed to GHCR.
- Image tags support both convenience (`latest`) and reproducibility (`sha-*`).
- CI builds are faster with Docker layer caching.
- Local commits are guarded by pre-commit gofmt.

This completes the CI foundation. The next phase is deployment (CD) to a VPS or Kubernetes.