---
title: "Devops Deployment"
description: "Use when setting up CI/CD pipelines, containerizing applications, deploying to Kubernetes, or writing infrastructure as code. DevOps & Deployment covers GitHub Actions, Docker, Helm, and Terraform patterns."
canonical: "https://orchestkit.yonyon.ai/docs/reference/skills/devops-deployment"
---

# Devops Deployment

Use when setting up CI/CD pipelines, containerizing applications, deploying to Kubernetes, or writing infrastructure as code. DevOps & Deployment covers GitHub Actions, Docker, Helm, and Terraform patterns.

<span className="badge badge-gray">Reference</span> <span className="badge badge-yellow">medium</span>

> **Auto-activated** — this skill loads automatically when Claude detects matching context.

<ContextualSkillSidebar slug="devops-deployment" />

> **Devops Deployment** Use when setting up CI/CD pipelines, containerizing applications, deploying to Kubernetes, or writing infrastructure as code. DevOps & Deployment covers GitHub Actions, Docker, Helm, and Terraform patterns.


# DevOps & Deployment Skill

Comprehensive frameworks for CI/CD pipelines, containerization, deployment strategies, and infrastructure automation.

> **Note:** If `disableSkillShellExecution` is enabled (CC 2.1.91), the Docker install check won't run. Verify Docker is available for container operations: `docker --version`.

## Overview

- Setting up CI/CD pipelines
- Containerizing applications
- Deploying to Kubernetes or cloud platforms
- Implementing GitOps workflows
- Managing infrastructure as code
- Planning release strategies

## Pipeline Architecture

```
┌─────────────┐   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐
│    Code     │──>│    Build    │──>│    Test     │──>│   Deploy    │
│   Commit    │   │   & Lint    │   │   & Scan    │   │  & Release  │
└─────────────┘   └─────────────┘   └─────────────┘   └─────────────┘
       │                 │                 │                 │
       v                 v                 v                 v
   Triggers         Artifacts          Reports          Monitoring
```

## Key Concepts

### CI/CD Pipeline Stages

1. **Lint & Type Check** - Code quality gates
2. **Unit Tests** - Test coverage with reporting
3. **Security Scan** - npm audit + Trivy vulnerability scanner
4. **Build & Push** - Docker image to container registry
5. **Deploy Staging** - Environment-gated deployment
6. **Deploy Production** - Manual approval or automated

### Container Best Practices

**Multi-stage builds** minimize image size:
- Stage 1: Install production dependencies only
- Stage 2: Build application with dev dependencies
- Stage 3: Production runtime with minimal footprint

**Security hardening**:
- Non-root user (uid 1001)
- Read-only filesystem where possible
- Health checks for orchestrator integration

### Kubernetes Deployment

**Essential manifests**:
- Deployment with rolling update strategy
- Service for internal routing
- Ingress for external access with TLS
- HorizontalPodAutoscaler for scaling

**Security context**:
- `runAsNonRoot: true`
- `allowPrivilegeEscalation: false`
- `readOnlyRootFilesystem: true`
- Drop all capabilities

### Deployment Strategies

| Strategy | Use Case | Risk |
|----------|----------|------|
| **Rolling** | Default, gradual replacement | Low - automatic rollback |
| **Blue-Green** | Instant switch, easy rollback | Medium - double resources |
| **Canary** | Progressive traffic shift | Low - gradual exposure |

**Rolling Update** (Kubernetes default):
```yaml
strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 25%
    maxUnavailable: 0  # Zero downtime
```

### Secrets Management

Use External Secrets Operator to sync from cloud providers:
- AWS Secrets Manager
- HashiCorp Vault
- Azure Key Vault
- GCP Secret Manager

---

## References

### Docker Patterns
**Load: `Read("$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/docker-patterns.md")`**

Key topics covered:
- Multi-stage build examples with 78% size reduction
- Layer caching optimization
- Security hardening (non-root, health checks)
- Trivy vulnerability scanning
- Docker Compose development setup

### OrchestKit Delta (house rules)
**Load: `Read("$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/ork-delta.md")`**

Key topics covered:
- CI concurrency groups, the sub-5-minute feedback budget, path filtering
- Service-container health gating, SHA-pinned deploys, CDN invalidation order
- Kubernetes probe budget, request/limit baseline, PodDisruptionBudget floor
- External Secrets refresh interval, ArgoCD prune plus selfHeal, Terraform state locking
- Alert thresholds with dwell windows, request-id log binding, rollback rehearsal

### Railway Deployment
**Load: `Read("$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/rules/railway-deployment.md")`**

Key topics covered:
- railway.json configuration, Nixpacks builds
- Environment variable management, database provisioning
- Multi-service setups, Railway CLI workflows
- References: `$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/railway-json-config.md`, `$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/nixpacks-customization.md`, `$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/multi-service-setup.md`

### Deployment Strategies
**Load: `Read("$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/deployment-strategies.md")`**

Key topics covered:
- Rolling deployment (`maxUnavailable` / `maxSurge`)
- Blue-green deployment (service-selector switch and rollback)
- Canary releases (replica-ratio traffic split)

---

## Upstream coverage (do not restate)

This skill wraps third-party products. Vendor mechanics are not restated here; fetch them from the
source below. Where a row says "house subset stays in X", that file keeps only OrchestKit's
threshold, config or ordering decision, not the vendor tutorial.

| Topic | Fetch from |
|-------|-----------|
| GitHub Actions workflow syntax, matrix builds, artifact upload/download, cache mechanics | https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax (house cache-key subset stays in `rules/devops-ci-caching.md`) |
| Trigger filters (`on.push.paths`, schedules, `workflow_dispatch`) | https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow |
| Service containers for integration tests | https://docs.github.com/en/actions/tutorials/use-containerized-services/use-docker-service-containers |
| Deployment environments and approval gates | https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments |
| Protected branches and required status checks | https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches (house approval counts stay in `rules/devops-branch-protection.md`) |
| Kubernetes probe semantics and every probe field | https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/ (house probe numbers stay in `references/ork-delta.md`) |
| Requests, limits, quotas and QoS classes | https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/ (house baseline stays in `references/ork-delta.md`) |
| PodDisruptionBudget semantics and eviction API | https://kubernetes.io/docs/tasks/run-application/configure-pdb/ (house `minAvailable` floor stays in `references/ork-delta.md`) |
| StatefulSets, ordinal identity, volumeClaimTemplates | https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/ |
| Helm chart authoring, templating and values files | https://helm.sh/docs/topics/charts/ (the chart directory layout we use stays in `references/checklists-and-templates.md`) |
| External Secrets Operator CRD fields and backends | https://external-secrets.io/latest/api/externalsecret/ (house refresh/creation policy stays in `references/ork-delta.md`) |
| ArgoCD automated sync, prune and self-heal | https://argo-cd.readthedocs.io/en/stable/user-guide/auto_sync/ (house retry/backoff stays in `references/ork-delta.md`) |
| Terraform S3 backend and state locking | https://developer.hashicorp.com/terraform/language/backend/s3 (house backend config stays in `references/ork-delta.md`) |
| Terraform module composition and variable files | https://developer.hashicorp.com/terraform/language/modules |
| Alembic revision, upgrade and downgrade CLI | https://alembic.sqlalchemy.org/en/latest/tutorial.html (the zero-downtime migration ordering stays in `rules/devops-db-migrations.md`) |
| Prometheus client instrumentation (Counter, Histogram, exposition) | https://prometheus.github.io/client_python/ |
| PromQL functions (`rate`, `histogram_quantile`) for dashboards | https://prometheus.io/docs/prometheus/latest/querying/functions/ |
| Prometheus alerting rule syntax | https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/ (house thresholds and dwell windows stay in `references/ork-delta.md`) |
| OpenTelemetry FastAPI auto-instrumentation and manual spans | https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/fastapi/fastapi.html |
| structlog context binding | https://www.structlog.org/en/stable/contextvars.html (the bind-then-call ordering stays in `references/ork-delta.md`) |
| Trivy severity filtering and scan configuration | https://trivy.dev/latest/docs/configuration/filtering/ (the house image-scan CI wiring stays in `references/docker-patterns.md`, the weekly-scan schedule and severity floor in `references/ork-delta.md`) |
| CloudFront invalidation semantics and cost | https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html (the sync-then-invalidate order stays in `references/ork-delta.md`) |
| Pre-launch security, load-testing and monitoring checklists | `ork:security-patterns`, `ork:testing-perf`, `ork:monitoring-observability` (the deploy-day checklist stays in `references/checklists-and-templates.md`) |

---

## Deployment Checklist & Templates

Load: `Read("$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/checklists-and-templates.md")` for pre/during/post-deployment checklists, Helm chart structure, template reference table, and extended thinking triggers.

---

## Related Skills

- `ork:security-patterns` - Security scanning and hardening patterns for CI/CD pipelines
- `ork:monitoring-observability` - Prometheus, Grafana and alerting for deployed applications
- `ork:database-patterns` - Python/Alembic migration workflow for backend deployments
- `portless` (upstream) - Named `.localhost` URLs for multi-service local dev (`portless alias api 8080`)

## Key Decisions

| Decision | Choice | Rationale |
|----------|--------|-----------|
| Container user | Non-root (uid 1001) | Security best practice, required by many orchestrators |
| Deployment strategy | Rolling update (default) | Zero downtime, automatic rollback, resource efficient |
| Secrets management | External Secrets Operator | Syncs from cloud providers, GitOps compatible |
| Health checks | Separate startup/liveness/readiness | Prevents premature traffic, enables graceful shutdown |

## Capability Details

Load: `Read("$\{CLAUDE_PLUGIN_ROOT\}/skills/devops-deployment/references/capability-details.md")` for full keyword index and problem-solution mapping across all 6 capabilities (ci-cd, docker, kubernetes, infrastructure-as-code, deployment-strategies, observability).

---

## Rules (6)

### Protect CI/CD branches from direct pushes to enforce code review and audit trails — HIGH


## CI/CD: Branch Protection

Configure branch protection rules to enforce code review, passing CI checks, and linear history on critical branches. This prevents untested or unreviewed code from reaching production.

**Incorrect:**
```bash
# Direct push to main — no review, no CI checks
git checkout main
git commit -m "quick fix"
git push origin main

# Force push overwrites history
git push --force origin main
```

```yaml
# CI workflow with no branch restrictions
on:
  push:
    branches: ['*']
```

**Correct:**
```
Branch strategy with protection rules:

main (production) ─────●────────●──────>
                       |        |
dev (staging)  ─────●──●────●──●──────>
                    |        |
feature/*  ─────────●────────┘
                    ^
                    └─ PR required, CI checks, code review
```

**GitHub branch protection settings:**
```
main branch:
  - Require pull request before merging
  - Required approving reviews: 2
  - Require status checks to pass (lint, test, security)
  - Require branches to be up to date before merging
  - Do not allow force pushes
  - Do not allow deletions

dev branch:
  - Require pull request before merging
  - Required approving reviews: 1
  - Require status checks to pass (lint, test)
```

**Key rules:**
- `main` requires PR + 2 approvals + all status checks passing before merge
- `dev` requires PR + 1 approval + all status checks passing
- Never allow direct commits or force pushes to `main` or `dev`
- Feature branches must be created from `dev` and merged back via PR
- Require branches to be up-to-date before merging to prevent integration gaps
- Enable "Require linear history" to keep the commit graph clean and auditable

Reference: https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches for the full ruleset surface.


### Cache CI/CD pipeline dependencies to avoid re-downloading and save minutes per run — HIGH


## CI/CD: Pipeline Caching

Cache dependencies in CI pipelines using lockfile-based cache keys. Proper caching reduces dependency installation from 2-3 minutes to 10-20 seconds (~85% time savings).

**Incorrect:**
```yaml
# No caching: re-downloads everything on every run
steps:
  - uses: actions/checkout@v3
  - run: npm install
  - run: npm test
```

```yaml
# Bad cache key: no lockfile hash, stale deps served indefinitely
- uses: actions/cache@v3
  with:
    path: node_modules
    key: ${{ runner.os }}-modules
```

**Correct:**
```yaml
- name: Cache Dependencies
  uses: actions/cache@v3
  with:
    path: |
      ~/.npm
      node_modules
      backend/.venv
    key: ${{ runner.os }}-deps-${{ hashFiles('**/package-lock.json', '**/poetry.lock') }}
    restore-keys: |
      ${{ runner.os }}-deps-

- name: Install Dependencies
  run: npm ci

- name: Run Tests
  run: npm test
```

```yaml
# Python example with Poetry
- name: Cache Poetry Dependencies
  uses: actions/cache@v3
  with:
    path: ~/.cache/pypoetry
    key: ${{ runner.os }}-poetry-${{ hashFiles('backend/poetry.lock') }}

- name: Install Dependencies
  run: poetry install
```

**Key rules:**
- Always include `hashFiles()` of the lockfile in the cache key so caches invalidate when dependencies change
- Use `restore-keys` as a fallback prefix to get a partial cache hit when the exact key misses
- Cache the package manager cache directory (`~/.npm`, `~/.cache/pypoetry`), not just `node_modules`
- Use `npm ci` (not `npm install`) after cache restore for reproducible installs
- Cache multiple dependency directories in a single step when possible (npm + pip + venv)
- Set artifact retention policies (`retention-days: 7`) to prevent storage bloat

Reference: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows for `actions/cache` mechanics; `references/ork-delta.md` for the house CI feedback budget.


### Run database migrations safely during deployment to prevent downtime and data loss — CRITICAL


## DevOps: Database Migrations

All schema changes must be backward-compatible with the currently running application version. Destructive changes require a multi-phase migration to achieve zero-downtime deployments.

**Incorrect:**
```sql
-- Destructive: renames column while old code still references 'name'
ALTER TABLE users RENAME COLUMN name TO full_name;

-- Destructive: adds NOT NULL column, old inserts fail immediately
ALTER TABLE users ADD COLUMN email VARCHAR(255) NOT NULL;

-- Destructive: drops column while old code still reads it
ALTER TABLE users DROP COLUMN legacy_field;
```

**Correct (3-phase zero-downtime migration):**
```sql
-- Phase 1: Add nullable column (safe with old code running)
ALTER TABLE users ADD COLUMN email VARCHAR(255);
```

```python
# Phase 2: Deploy new code that writes to both + backfill
def create_user(name: str, email: str):
    db.execute(
        "INSERT INTO users (name, email) VALUES (%s, %s)",
        (name, email),
    )

async def backfill_emails():
    users = await db.fetch("SELECT id FROM users WHERE email IS NULL")
    for user in users:
        email = generate_email(user.id)
        await db.execute(
            "UPDATE users SET email = %s WHERE id = %s",
            (email, user.id),
        )
```

```sql
-- Phase 3: Add constraint after backfill is verified complete
ALTER TABLE users ALTER COLUMN email SET NOT NULL;
```

**Backward-compatible changes (safe to deploy directly):**
- Add nullable column
- Add new table
- Add index
- Rename column with a view alias

**Backward-incompatible changes (require 3-phase migration):**
- Remove column
- Rename column without alias
- Add NOT NULL column
- Change column type

**Deploy order:** migrate (phase 1) --> deploy new code (phase 2) --> migrate (phase 3)

**Key rules:**
- Always deploy migrations before the application code that depends on them
- Never add a NOT NULL column in a single step — use the 3-phase pattern (add nullable, backfill, add constraint)
- Always write a `downgrade()` function so migrations can be rolled back (`alembic downgrade -1`)
- Always review auto-generated migrations before applying (`alembic revision --autogenerate`)
- Test rollback procedures regularly — do not assume `downgrade()` works without verification
- Column renames require a view alias to maintain backward compatibility during rollout

Reference: https://alembic.sqlalchemy.org/en/latest/tutorial.html for revision/upgrade/downgrade mechanics; `ork:database-patterns` for the Python migration workflow.


### Secure Docker layers by running as non-root and excluding secrets from image builds — CRITICAL


## Docker: Layer Security

Every Docker image layer is immutable and inspectable. Running as root or embedding secrets in layers creates critical security vulnerabilities that persist even if later layers attempt to remove them.

**Incorrect:**
```dockerfile
FROM node:24
WORKDIR /app

# BAD: Copies .env, .git, node_modules, and everything else
COPY . .
RUN npm install

# BAD: Secret baked into image layer (visible via docker history)
ARG DATABASE_URL
ENV DATABASE_URL=$DATABASE_URL

# BAD: Running as root (default)
EXPOSE 3000
CMD ["node", "dist/main.js"]
```

**Correct:**
```dockerfile
FROM node:24-alpine AS runner
WORKDIR /app

# GOOD: Create and use non-root user (uid 1001)
RUN addgroup -g 1001 -S nodejs && adduser -S nodejs -u 1001

# GOOD: Copy only what's needed with explicit ownership
COPY --from=deps --chown=nodejs:nodejs /app/node_modules ./node_modules
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
COPY --chown=nodejs:nodejs package*.json ./

# GOOD: Run as non-root
USER nodejs

# GOOD: Secrets injected at runtime, never in image
# Use: docker run -e DATABASE_URL=... or Kubernetes secrets
ENV NODE_ENV=production
EXPOSE 3000

HEALTHCHECK --interval=30s --timeout=3s CMD node healthcheck.js || exit 1
CMD ["node", "dist/main.js"]
```

**Required `.dockerignore`:**
```
.git
.env
.env.*
node_modules
*.md
tests/
.vscode/
```

**Key rules:**
- Never run containers as root — always create a non-root user with `USER` directive
- Never pass secrets via `ARG` or `ENV` in the Dockerfile — they are visible in `docker history`
- Always use a `.dockerignore` to exclude `.env`, `.git`, `node_modules`, and test files
- Use `COPY --chown` to set file ownership without a separate `chown` layer
- Prefer minimal base images (`-alpine`) to reduce the CVE surface area
- Enable read-only root filesystem in Kubernetes (`readOnlyRootFilesystem: true`)
- Add health checks so orchestrators can detect and restart unhealthy containers

Reference: `references/docker-patterns.md` (lines 52-85)


### Use Docker multi-stage builds to exclude dev dependencies and reduce image size by 4-5x — HIGH


## Docker: Multi-Stage Builds

Separate build-time concerns from runtime to produce minimal, secure production images. A well-structured multi-stage build can reduce image size by 78% or more.

**Incorrect:**
```dockerfile
# Single-stage: build tools and dev deps ship to production
FROM node:24
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build
EXPOSE 3000
CMD ["node", "dist/main.js"]
# Result: ~850 MB image with dev dependencies, source files, build tools
```

**Correct:**
```dockerfile
# Stage 1: Install production dependencies only
FROM node:24-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production && npm cache clean --force

# Stage 2: Build with dev dependencies
FROM node:24-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm run test

# Stage 3: Minimal production runtime
FROM node:24-alpine AS runner
WORKDIR /app
RUN addgroup -g 1001 -S nodejs && adduser -S nodejs -u 1001
COPY --from=deps --chown=nodejs:nodejs /app/node_modules ./node_modules
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
COPY --chown=nodejs:nodejs package*.json ./
USER nodejs
EXPOSE 3000
ENV NODE_ENV=production
HEALTHCHECK --interval=30s --timeout=3s CMD node healthcheck.js || exit 1
CMD ["node", "dist/main.js"]
# Result: ~180 MB image with only production runtime
```

**Key rules:**
- Use separate stages for dependency installation, building, and runtime
- Copy only production `node_modules` and compiled artifacts into the final stage
- Use `-alpine` base images to minimize base layer size
- Run `npm ci` (not `npm install`) for reproducible, lockfile-exact installs
- Clean caches (`npm cache clean --force`) in the same layer as install to avoid bloating layers
- Always include a `HEALTHCHECK` in the production stage for orchestrator integration
- Run tests in the builder stage so test failures prevent image creation

Reference: `references/docker-patterns.md` (lines 7-50)


### Configure Railway PaaS deployment with correct Nixpacks, environment, and railway.json settings — HIGH


# Railway Deployment Patterns

## railway.json Configuration

```json
{
  "$schema": "https://railway.com/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "buildCommand": "npm ci && npm run build"
  },
  "deploy": {
    "startCommand": "npm start",
    "healthcheckPath": "/health",
    "healthcheckTimeout": 30,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 3
  }
}
```

## Nixpacks vs Dockerfile

| Factor | Nixpacks (default) | Dockerfile |
|--------|-------------------|-----------|
| Setup | Zero config, auto-detect | Manual, full control |
| Build time | Fast (Nix cache) | Depends on layers |
| Customization | nixpacks.toml | Unlimited |
| Use when | Standard apps | Custom runtimes, multi-stage |

## Environment Variables

- Use Railway's **shared variables** for cross-service config (DATABASE_URL, REDIS_URL)
- **Service-specific** variables override shared ones
- Reference other vars: `$\{\{shared.DATABASE_URL\}\}`
- Never hardcode secrets — use Railway's encrypted env vars

## Database Provisioning

Railway provisions managed databases with one click:
- PostgreSQL, MySQL, Redis, MongoDB
- Connection string auto-injected as env var
- Backups included on paid plans

## Multi-Service Setup

- Use **monorepo** config: set `rootDirectory` per service
- Internal networking: services communicate via `$\{\{service.RAILWAY_PRIVATE_DOMAIN\}\}:port`
- Shared env groups for common config

## Railway CLI

```bash
railway login              # Authenticate
railway link               # Connect to project
railway up                 # Deploy from local
railway logs               # View deployment logs
railway variables          # List env vars
railway shell              # Open shell in service
```

## Anti-Patterns

**Incorrect:**
- Running `railway up` from CI without `railway link` — deploys to wrong project
- Using Dockerfile when Nixpacks handles the stack — unnecessary complexity
- Storing secrets in railway.json — use env vars
- Skipping healthcheck config — Railway can't detect failed deploys

**Correct:**
- Configure healthcheckPath for all web services
- Use shared variables for cross-service config
- Set restart policy for resilience
- Use Nixpacks unless you need custom runtime

## References

- `references/railway-json-config.md` — Full railway.json schema and examples
- `references/nixpacks-customization.md` — Custom build configs, environment detection
- `references/multi-service-setup.md` — Monorepo deploy, service networking



---

## References (8)

### Capability Details

# DevOps Deployment - Capability Details

> Vendor mechanics for these capabilities are not restated in this skill. See the
> "Upstream coverage (do not restate)" table in `SKILL.md` for where to fetch them, and
> `references/ork-delta.md` for the house thresholds, configs and ordering constraints.

### ci-cd
**Keywords:** ci, cd, pipeline, github actions, gitlab ci, jenkins, workflow
**Solves:**
- What is our CI feedback budget and how do we hold it? (`references/ork-delta.md`)
- Cache-key discipline for dependency caching (`rules/devops-ci-caching.md`)
- Which branch protections are required on main and dev (`rules/devops-branch-protection.md`)
- Workflow syntax and matrix mechanics: fetch upstream (see `SKILL.md`)

### docker
**Keywords:** docker, dockerfile, container, image, build, compose, multi-stage
**Solves:**
- How do I containerize my app?
- Multi-stage Dockerfile best practices
- Docker Compose development setup
- Container security hardening

### kubernetes
**Keywords:** kubernetes, k8s, deployment, service, ingress, helm, statefulset, pdb
**Solves:**
- What probe timings, request/limit baseline and PDB floor do we ship? (`references/ork-delta.md`)
- Which manifests a service needs and the chart layout (`references/checklists-and-templates.md`)
- Ready-to-copy manifests (`scripts/k8s-manifests.yaml`, `scripts/helm-values.yaml`)
- Probe, quota, StatefulSet and Helm semantics: fetch upstream (see `SKILL.md`)

### infrastructure-as-code
**Keywords:** terraform, pulumi, iac, infrastructure, provision, gitops, argocd
**Solves:**
- What backend, locking and version floor does our Terraform use? (`references/ork-delta.md`)
- Which ArgoCD sync policy and retry budget we run (`references/ork-delta.md`)
- Ready-to-copy IaC (`scripts/terraform-aws.tf`, `scripts/argocd-application.yaml`, `scripts/external-secrets.yaml`)
- Terraform, ArgoCD and External Secrets reference docs: fetch upstream (see `SKILL.md`)

### deployment-strategies
**Keywords:** blue green, canary, rolling, deployment strategy, rollback, zero downtime
**Solves:**
- Which deployment strategy should I use?
- Zero-downtime database migrations
- Blue-green deployment setup
- Canary release with traffic splitting

### observability
**Keywords:** prometheus, grafana, metrics, alerting, monitoring, health check
**Solves:**
- Which alert thresholds and dwell windows do we page on? (`references/ork-delta.md`)
- How request ids get bound into log context and echoed back (`references/ork-delta.md`)
- Full monitoring and alerting design: `ork:monitoring-observability`
- Prometheus client, PromQL and OpenTelemetry docs: fetch upstream (see `SKILL.md`)


### Checklists And Templates

# Deployment Checklists and Templates

## Deployment Checklist

### Pre-Deployment
- [ ] All tests passing in CI
- [ ] Security scans clean
- [ ] Database migrations ready
- [ ] Rollback plan documented

### During Deployment
- [ ] Monitor deployment progress
- [ ] Watch error rates
- [ ] Verify health checks passing

### Post-Deployment
- [ ] Verify metrics normal
- [ ] Check logs for errors
- [ ] Update status page

---

## Helm Chart Structure

```
charts/app/
├── Chart.yaml
├── values.yaml
├── scripts/
│   ├── deployment.yaml
│   ├── service.yaml
│   ├── ingress.yaml
│   ├── configmap.yaml
│   ├── secret.yaml
│   ├── hpa.yaml
│   └── _helpers.tpl
└── values/
    ├── staging.yaml
    └── production.yaml
```

---

## Templates Reference

| Template | Purpose |
|----------|---------|
| `github-actions-pipeline.yml` | Full CI/CD workflow with 6 stages |
| `Dockerfile` | Multi-stage Node.js build |
| `docker-compose.yml` | Development environment |
| `k8s-manifests.yaml` | Deployment, Service, Ingress |
| `helm-values.yaml` | Helm chart values |
| `terraform-aws.tf` | VPC, EKS, RDS infrastructure |
| `argocd-application.yaml` | GitOps application |
| `external-secrets.yaml` | Secrets Manager integration |

---

## Extended Thinking Triggers

Use adaptive thinking for:
- **Architecture decisions** - Kubernetes vs serverless, multi-region setup
- **Migration planning** - Moving between cloud providers
- **Incident response** - Complex deployment failures
- **Security design** - Zero-trust architecture


### Deployment Strategies

# Deployment Strategies

Blue-green, canary, and rolling deployment patterns.

## Rolling Deployment (Default)

Update pods gradually:

```yaml
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 1
```

- **Pros**: No downtime, gradual rollout
- **Cons**: Mixed versions running simultaneously

## Blue-Green Deployment

Two identical environments, switch traffic:

```bash
# Deploy to green (inactive)
kubectl apply -f green-deployment.yaml

# Test green
curl https://green.example.com/health

# Switch traffic (update service selector)
kubectl patch service myapp -p '{"spec":{"selector":{"version":"green"}}}'

# Rollback if needed
kubectl patch service myapp -p '{"spec":{"selector":{"version":"blue"}}}'
```

- **Pros**: Instant rollback, no mixed versions
- **Cons**: 2x resources, database migrations tricky

## Canary Deployment

Gradually shift traffic:

```yaml
# 90% to stable
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-stable
spec:
  replicas: 9

# 10% to canary
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-canary
spec:
  replicas: 1
```

- **Pros**: Limit blast radius, test with real traffic
- **Cons**: Complex traffic management

See `scripts/argocd-application.yaml` for GitOps patterns.


### Docker Patterns

# Docker Patterns

Best practices for Dockerfile optimization, multi-stage builds, and container security.

## Multi-Stage Build Example

```dockerfile
# ============================================================
# Stage 1: Dependencies (builder)
# ============================================================
FROM node:24-alpine AS deps
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production && npm cache clean --force

# ============================================================
# Stage 2: Build (with dev dependencies)
# ============================================================
FROM node:24-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci  # Include dev dependencies
COPY . .
RUN npm run build && npm run test

# ============================================================
# Stage 3: Production runtime (minimal)
# ============================================================
FROM node:24-alpine AS runner
WORKDIR /app

# Security: Non-root user
RUN addgroup -g 1001 -S nodejs && adduser -S nodejs -u 1001

# Copy only production dependencies and built artifacts
COPY --from=deps --chown=nodejs:nodejs /app/node_modules ./node_modules
COPY --from=builder --chown=nodejs:nodejs /app/dist ./dist
COPY --chown=nodejs:nodejs package*.json ./

USER nodejs
EXPOSE 3000
ENV NODE_ENV=production
HEALTHCHECK --interval=30s --timeout=3s CMD node healthcheck.js || exit 1
CMD ["node", "dist/main.js"]
```

**Image size comparison:**
- Single-stage: **850 MB** (includes dev dependencies, source files)
- Multi-stage: **180 MB** (only runtime + production deps)
- **78% reduction**

## Layer Caching Optimization

**Order matters for cache efficiency:**

```dockerfile
# BAD: Invalidates cache on any code change
COPY . .
RUN npm install

# GOOD: Cache package.json layer separately
COPY package*.json ./
RUN npm ci  # Cached unless package.json changes
COPY . .    # Source changes don't invalidate npm install
```

## Security Hardening

**Non-root user (uid 1001):**
```dockerfile
RUN addgroup -g 1001 -S nodejs && adduser -S nodejs -u 1001
USER nodejs
```

**Read-only filesystem where possible:**
```dockerfile
# In K8s deployment
securityContext:
  readOnlyRootFilesystem: true
```

**Health checks for orchestrator integration:**
```dockerfile
HEALTHCHECK --interval=30s --timeout=3s CMD node healthcheck.js || exit 1
```

## Security Scanning with Trivy

```yaml
- name: Build Docker Image
  run: docker build -t myapp:${{ github.sha }} .

- name: Scan for Vulnerabilities
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: 'myapp:${{ github.sha }}'
    format: 'sarif'
    output: 'trivy-results.sarif'
    severity: 'CRITICAL,HIGH'

- name: Upload Scan Results
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: 'trivy-results.sarif'

- name: Fail on Critical Vulnerabilities
  run: |
    trivy image --severity CRITICAL --exit-code 1 myapp:${{ github.sha }}
```

## Docker Compose Development Setup

```yaml
version: '3.8'
services:
  postgres:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_USER: orchestkit
      POSTGRES_PASSWORD: dev_password
      POSTGRES_DB: orchestkit_dev
    ports:
      - "5437:5432"  # Avoid conflict with host postgres
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U orchestkit"]
      interval: 5s
      timeout: 3s
      retries: 5

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    command: redis-server --appendonly yes --maxmemory 512mb --maxmemory-policy allkeys-lru
    volumes:
      - redisdata:/data

  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile.dev
    ports:
      - "8500:8500"
    environment:
      DATABASE_URL: postgresql://orchestkit:dev_password@postgres:5432/orchestkit_dev
      REDIS_URL: redis://redis:6379
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_started
    volumes:
      - ./backend:/app  # Hot reload

  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile.dev
    ports:
      - "5173:5173"
    environment:
      VITE_API_URL: http://localhost:8500
    volumes:
      - ./frontend:/app
      - /app/node_modules  # Avoid overwriting node_modules

volumes:
  pgdata:
  redisdata:
```

**Key patterns:**
- Port mapping to avoid host conflicts (5437:5432)
- Health checks before dependent services start
- Volume mounts for hot reload during development
- Named volumes for data persistence

See `scripts/Dockerfile` and `scripts/docker-compose.yml` for complete examples.

### Multi Service Setup

# Multi-Service Setup on Railway

Deploy multiple services in one Railway project for monorepos, microservices, or web + worker architectures.

## Monorepo Configuration

Each service in a Railway project can point to a different root directory:

```
my-monorepo/
├── apps/
│   ├── api/           ← Service 1 (root: apps/api)
│   │   ├── package.json
│   │   └── railway.json
│   ├── web/           ← Service 2 (root: apps/web)
│   │   ├── package.json
│   │   └── railway.json
│   └── worker/        ← Service 3 (root: apps/worker)
│       ├── package.json
│       └── railway.json
├── packages/          ← Shared packages
└── package.json       ← Root workspace
```

Set each service's root directory in the Railway dashboard under Settings > Source.

## Private Networking

Services within the same project communicate over Railway's private network:

```
# From the web service, call the API service:
http://${{api.RAILWAY_PRIVATE_DOMAIN}}:${{api.PORT}}/endpoint

# In environment variables (set on web service):
API_URL=http://${{api.RAILWAY_PRIVATE_DOMAIN}}:${{api.PORT}}
```

**Key points:**
- Private networking uses internal DNS, no public internet
- Zero egress costs between services
- Always use the `PORT` variable — not hardcoded ports
- Services must listen on `0.0.0.0` (not `localhost`)

## Common Architectures

### Web + API + Worker

| Service | Role | Public? |
|---------|------|---------|
| `web` | Frontend (Next.js, Vite) | Yes |
| `api` | Backend API | Yes (or private if only web calls it) |
| `worker` | Background jobs (BullMQ, Celery) | No |
| `postgres` | Database | No (private only) |
| `redis` | Cache / queue broker | No (private only) |

### Shared Environment Variables

Use Railway's shared variables (project-level) for values needed by all services:
- `NODE_ENV=production`
- `LOG_LEVEL=info`

Use reference variables for cross-service connections:
- `DATABASE_URL=$\{\{Postgres.DATABASE_URL\}\}`
- `REDIS_URL=$\{\{Redis.REDIS_URL\}\}`
- `API_URL=http://$\{\{api.RAILWAY_PRIVATE_DOMAIN\}\}:$\{\{api.PORT\}\}`

## Deploy Order

Railway deploys services in parallel by default. If you need ordering (e.g., run migrations before starting web):
1. Put migrations in the API service's `startCommand`
2. Use healthchecks — dependent services will retry connections until the API is healthy
3. For strict ordering, use separate deploy triggers via Railway CLI


### Nixpacks Customization

# Nixpacks Customization

Railway uses Nixpacks to auto-detect your stack and generate a build plan. Customize when auto-detection falls short.

## Auto-Detection

Nixpacks detects your language by looking for:

| Language | Detection File |
|----------|---------------|
| Node.js | `package.json` |
| Python | `requirements.txt`, `pyproject.toml`, `Pipfile` |
| Go | `go.mod` |
| Rust | `Cargo.toml` |
| Ruby | `Gemfile` |
| Java | `pom.xml`, `build.gradle` |
| PHP | `composer.json` |

## nixpacks.toml

Place at project root (or set `nixpacksConfigPath` in `railway.json` for monorepos).

### Adding System Packages

```toml
[phases.setup]
nixPkgs = ["...", "ffmpeg", "imagemagick", "poppler_utils"]
aptPkgs = ["libvips-dev"]
```

### Custom Build Phases

```toml
[phases.install]
cmds = ["npm ci --production=false"]

[phases.build]
cmds = [
  "npx prisma generate",
  "npm run build"
]
dependsOn = ["install"]

[start]
cmd = "npm run start:prod"
```

### Environment Variables in Build

```toml
[variables]
NODE_ENV = "production"
NEXT_TELEMETRY_DISABLED = "1"
```

## Monorepo Root Path

For monorepos, set the root directory per service in the Railway dashboard or via `railway.json`:

```json
{
  "build": {
    "builder": "NIXPACKS",
    "nixpacksConfigPath": "apps/api/nixpacks.toml"
  }
}
```

Each service points to its own directory and `nixpacks.toml`.

## When to Switch to Dockerfile

Use Dockerfile instead of Nixpacks when:
- Multi-stage builds are needed to reduce image size
- Build requires conditional logic (e.g., `ARG`-based feature flags)
- Precise control over base image (e.g., distroless, Alpine variants)
- Nixpacks doesn't support a required system dependency

Set in `railway.json`:

```json
{
  "build": {
    "builder": "DOCKERFILE",
    "dockerfilePath": "Dockerfile.production"
  }
}
```


### Ork Delta

# OrchestKit DevOps Delta

House rules that survive after the vendor tutorials were routed to their first-party sources.
Everything here is a threshold, a working config, or an ordering constraint we actually rely on.
Vendor mechanics live upstream; see the "Upstream coverage" table in `SKILL.md`.

A "retired `&lt;path&gt;`" in a Why line is a historical citation of a file that was deleted in this
change, not a live pointer. Do not try to open those paths.

## Give every CI workflow a concurrency group keyed on workflow and ref

Why: distilled from the retired `examples/github-actions-cicd.md`; no traced incident. The house shape is `group: $\{\{ github.workflow \}\}-$\{\{ github.ref \}\}` with `cancel-in-progress: true`, so a force-push to a PR branch cannot leave two runs racing toward the same deploy target.
Upstream: https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax

## Keep CI feedback under 5 minutes by path-filtering per service

Why: distilled from the retired `references/ci-cd-pipelines.md`; the house budget was written as "tests complete in &lt; 5 min" and was met by scoping `on.push.paths` and `on.pull_request.paths` to `backend/**` and `frontend/**`, so a frontend-only PR never pays for the Python matrix.
Upstream: https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow

## Gate integration tests behind a service-container health check before the first query

Why: distilled from the retired `examples/github-actions-cicd.md`; the Postgres service carried `--health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5` so pytest never opened a connection to a socket that was not listening yet. This is an ordering constraint, not decoration.
Upstream: https://docs.github.com/en/actions/tutorials/use-containerized-services/use-docker-service-containers

## Deploy the exact image SHA that CI tested, never a rebuild

Why: house rule. The retired `examples/github-actions-cicd.md` tagged images `$\{\{ github.sha \}\}` at build time but then deployed with `aws ecs update-service --force-new-deployment`, which pins nothing and re-pulls whatever the task definition points at. That gap is the reason this rule is written down rather than assumed: tag with the SHA AND reference that tag in the deploy step, so "the artifact we tested" and "the artifact we shipped" are byte-for-byte identical. Production deploys additionally sit behind `environment: production` for the approval gate.
Upstream: https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments

## Invalidate the CDN after the object sync, never before

Why: distilled from the retired `examples/github-actions-cicd.md`; the static frontend deploy runs `aws s3 sync dist/ --delete` and only then `cloudfront create-invalidation --paths "/*"`. Invalidating first re-caches the old bundle straight back from a not-yet-updated origin.
Upstream: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html

## Run the dependency and image scan weekly at HIGH,CRITICAL

Why: distilled from the retired `examples/github-actions-cicd.md`; the house schedule is `cron: "0 0 * * 0"` plus `workflow_dispatch`, with `npm audit --audit-level=high`, `pip-audit`, and Trivy pinned to `severity: HIGH,CRITICAL`. The severity floor is what keeps the weekly gate actionable rather than advisory.
Upstream: https://trivy.dev/latest/docs/configuration/filtering/

## Size the three Kubernetes probes as one budget, not as copies of each other

Why: distilled from the retired `references/kubernetes-basics.md`; the house numbers are startup `periodSeconds: 5` with `failureThreshold: 30` (a 150s boot ceiling), liveness `initialDelaySeconds: 60`, `periodSeconds: 10`, `failureThreshold: 3`, readiness `initialDelaySeconds: 10`, `periodSeconds: 5`, `failureThreshold: 2`. Readiness must fail fastest because pulling a pod out of the load balancer is cheap, and liveness slowest because a restart is not: liveness polls at half the readiness rate for exactly that reason.
Upstream: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

## Set both requests and limits on every container, limits at twice requests

Why: distilled from the retired `references/kubernetes-basics.md`; the house baseline is requests `128Mi` / `100m` and limits `256Mi` / `200m`. Requests drive scheduling and limits drive throttling, so omitting requests makes the scheduler blind to the pod and omitting limits lets one pod starve the node.
Upstream: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

## Ship a PodDisruptionBudget with minAvailable 2 for anything that takes traffic

Why: distilled from the retired `references/kubernetes-basics.md`; without a PDB a cluster upgrade node drain or an autoscaler downscale can evict every replica at once, which is a self-inflicted outage during planned maintenance. The house floor of 2 keeps a serving quorum through voluntary disruptions.
Upstream: https://kubernetes.io/docs/tasks/run-application/configure-pdb/

## Pull secrets through External Secrets Operator on a 1h refresh, never into Git

Why: `SKILL.md` Key Decisions picks External Secrets Operator so the cloud secret store stays the source of truth and the in-cluster Secret is derived. The house config is `refreshInterval: 1h` with `creationPolicy: Owner`, so a hand-edited Kubernetes Secret is reconciled away instead of silently persisting as undocumented drift.
Upstream: https://external-secrets.io/latest/api/externalsecret/

## Enable ArgoCD prune and selfHeal together, with a bounded retry

Why: distilled from the retired `references/environment-management.md`; `prune: true` without `selfHeal: true` leaves manual cluster edits standing and makes Git only a partial truth. The house retry is `limit: 5` with backoff from `5s` to a `3m` cap, so a transient API-server blip does not park the application in Degraded.
Upstream: https://argo-cd.readthedocs.io/en/stable/user-guide/auto_sync/

## Keep Terraform state remote with a lock table, and pin the CLI floor

Why: distilled from the retired `references/environment-management.md`; the house backend is S3 with `dynamodb_table = "terraform-locks"` and `required_version = ">= 1.5"`. Two concurrent applies without the lock table corrupt state, and corrupt state is not recoverable the way a failed apply is.
Upstream: https://developer.hashicorp.com/terraform/language/backend/s3

## Rehearse the rollback, do not just document it

Why: distilled from the retired `references/environment-management.md` and `checklists/production-readiness.md`; both carried the same standing instruction, that a rollback path is only real once it has been executed. The three house handles are `helm rollback &lt;release&gt; &lt;revision&gt;`, `kubectl rollout undo deployment/&lt;name&gt;`, and `alembic downgrade -1`, and the same rehearsal rule applies to backup restore, not only to backup creation.
Upstream: https://helm.sh/docs/helm/helm_rollback/

## Alert on the house thresholds, and always require a dwell window

Why: distilled from the retired `references/observability.md`; the numbers are error rate above 5% critical, p95 latency above 2s warning, CPU above 80% sustained, and memory above 85% sustained. Every rule carries `for:` (5m for rate and latency, 10m for crash-loop) so a single bad scrape does not page anyone.

Pod restarts carry TWO numbers in the retired file and they are not interchangeable. The operative alerting rule is `increase(kube_pod_container_status_restarts_total[1h]) > 5` with `for: 10m`, which is what actually pages. The dashboard summary table lists `> 3 in 1 hour` as the review threshold, the point at which a human should look without being woken. Use 5 for the alert and 3 for review; collapsing them into one number either pages on ordinary rollout churn or misses a real crash loop.
Upstream: https://prometheus.io/docs/prometheus/latest/configuration/alerting_rules/

## Bind the request id into log context before the handler runs, and echo it on the response

Why: distilled from the retired `references/observability.md`; the logging middleware opens `structlog.contextvars.bound_contextvars(request_id=...)` around `await call_next(request)` and sets `X-Request-ID` on the way out. Bind after the handler and every log line emitted inside it is unattributable; drop the header and a user-reported failure cannot be walked back to a log line.
Upstream: https://www.structlog.org/en/stable/contextvars.html

## Hold the house ResourceQuota baseline per namespace

Why: distilled from the retired `references/kubernetes-basics.md`. The numbers are
`requests.cpu: "10"`, `requests.memory: 20Gi`, `limits.cpu: "20"`, `limits.memory: 40Gi`,
`pods: "50"`. Kubernetes documents what a ResourceQuota is and how it is enforced but
cannot supply a house baseline, and a namespace shipped without one lets a single
runaway Deployment starve every neighbour on the node pool. Treat these as the starting
allocation and raise them deliberately, per namespace, rather than omitting the object.
Upstream: https://kubernetes.io/docs/concepts/policy/resource-quotas/

## Pin the CI toolchain to Python 3.12 and Node 20, and matrix-test Node 20, 22 and 24

Why: distilled from the retired `examples/github-actions-cicd.md` and
`references/ci-cd-pipelines.md`. The single-version jobs pin `python-version: "3.12"` and
`node-version: "20"`; the compatibility matrix runs `node-version: [20, 22, 24]` across
`os: [ubuntu-latest, windows-latest]`. These are house choices, not upstream defaults:
an unpinned `setup-node` silently follows the runner image and turns an unrelated image
bump into a red build with no code change behind it.
Upstream: https://github.com/actions/setup-node and https://github.com/actions/setup-python

## Gate release on 80 percent unit coverage, and name the tool for every pre-launch check

Why: distilled from the retired `checklists/production-readiness.md`. The house floor is
`>80%` unit coverage before release. The same checklist named the tool for each remaining
gate, and the names are the useful part: k6 or Locust for the load test, OWASP ZAP for
the security scan. It also carried reliability items that no upstream row can supply:
graceful shutdown handling, circuit breakers and timeouts on every external call,
point-in-time recovery enabled, and multi-AZ deployment. Circuit breakers, retries and
timeouts are owned by the in-repo `distributed-systems` skill; the rest are checks this
skill still expects before a release goes out.
Upstream: the in-repo `ork:testing-perf`, `ork:security-patterns` and
`ork:distributed-systems` skills


### Railway Json Config

# railway.json Configuration

Complete reference for `railway.json` schema — the primary way to configure build and deploy behavior on Railway.

## Full Schema

```json
{
  "$schema": "https://railway.com/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "buildCommand": "npm ci && npm run build",
    "watchPatterns": ["src/**", "package.json"],
    "nixpacksConfigPath": "nixpacks.toml",
    "dockerfilePath": "Dockerfile"
  },
  "deploy": {
    "startCommand": "node dist/server.js",
    "healthcheckPath": "/health",
    "healthcheckTimeout": 30,
    "restartPolicyType": "ON_FAILURE",
    "restartPolicyMaxRetries": 3,
    "numReplicas": 1,
    "sleepApplication": false,
    "region": "us-west1",
    "cronSchedule": "0 */6 * * *"
  }
}
```

## Builder Options

| Builder | When to Use |
|---------|-------------|
| `NIXPACKS` | Default — auto-detects language and builds (Node, Python, Go, Rust, etc.) |
| `DOCKERFILE` | Complex builds, multi-stage images, custom system deps |
| `PAKETO` | Cloud Native Buildpacks alternative |

## Deploy Settings

| Field | Default | Description |
|-------|---------|-------------|
| `startCommand` | Auto-detected | Overrides default start command |
| `healthcheckPath` | None | HTTP path to check for 200 response |
| `healthcheckTimeout` | 30 | Seconds before healthcheck is considered failed |
| `restartPolicyType` | `ON_FAILURE` | `ON_FAILURE`, `ALWAYS`, or `NEVER` |
| `restartPolicyMaxRetries` | 3 | Max restart attempts before marking deploy failed |
| `numReplicas` | 1 | Number of instances (horizontal scaling) |
| `sleepApplication` | false | Sleep service when no traffic (saves credits) |
| `cronSchedule` | None | Cron expression for scheduled services |

## Examples

### Node.js API with migrations

```json
{
  "$schema": "https://railway.com/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "buildCommand": "npm ci && npx prisma generate && npm run build"
  },
  "deploy": {
    "startCommand": "npx prisma migrate deploy && node dist/server.js",
    "healthcheckPath": "/api/health",
    "healthcheckTimeout": 60
  }
}
```

### Python FastAPI

```json
{
  "$schema": "https://railway.com/railway.schema.json",
  "build": {
    "builder": "NIXPACKS",
    "buildCommand": "pip install -r requirements.txt"
  },
  "deploy": {
    "startCommand": "uvicorn main:app --host 0.0.0.0 --port $PORT",
    "healthcheckPath": "/health"
  }
}
```

### Cron Worker (no web traffic)

```json
{
  "$schema": "https://railway.com/railway.schema.json",
  "deploy": {
    "startCommand": "node dist/worker.js",
    "cronSchedule": "*/15 * * * *",
    "restartPolicyType": "NEVER"
  }
}
```