Deployment & Infrastructure Specification

[5] 배포 & 인프라 설계

Project: Soli (OMR Choir Practicing App)/Author: Yoonjae Song/v1.0.0

01

INFRASTRUCTURE ARCHITECTURE (High-efficiency Bootstrap Strategy)

Cloud Provider Selection

  • - Primary Compute: Hetzner Cloud (VPS) - CX series or AX dedicated for extreme CPU/RAM per dollar. Audiveris OMR runs on a JVM (Java 17+) and requires sustained CPU for multi-page sheet music transcription; Hetzner offers 6-8 dedicated vCPUs at roughly 1/3 the cost of equivalent AWS/Azure instances.
  • - Object Storage: Cloudflare R2 - zero egress fees, S3-compatible API. Stores user-uploaded sheet music PDFs, OMR output (MusicXML/MIDI), and WAL-G database backups.
  • - Database: Supabase Postgres (managed, 8GB RAM tier) or self-hosted Postgres 16 on the same Hetzner VPS (with automated WAL-G backups to R2).
  • - CDN & SSL Termination: Cloudflare proxy (orange-clouded DNS) - DDoS protection, HTTP/2 termination, edge caching of static assets.

Architecture Decision Records

DecisionChoiceRationale
ComputeHetzner VPSBest perf/dollar for JVM workloads
Object StoreCloudflare R2Zero egress, S3 compat
DBSupabase / Self-hosted PGSQL, row-level security
QueueRedis (on-VPS)Low latency, zero infra cost
CDNCloudflareFree tier, global POPs

System Diagram

+-CLIENT----------- CLOUD -----------+--HETZNER VPS-----------+
|  HTTPS   -->  Cloudflare   -->  nginx  -->  FastAPI/Gunicorn  |
|  Browser      (Proxy/CDN)       (:443)       (Uvicorn)        |
|              SSL Term                  |                      |
|                                        +--> Supabase PG       |
|                   WAL-G --> R2         |    (port 5432)       |
|                   (hourly backups)     |                      |
|                                        +--> Redis             |
|                                        |    (port 6379)       |
|                                        |                      |
|                                        +--> OMR Worker        |
|                                             (Audiveris 5.3)   |
|                                             (Celery consumer) |
+----------------------------------------+----------------------+

Cost Projection (Monthly)

ItemTierEst. Cost
Hetzner VPS (CX42, 4 vCPU, 16GB)Standard$15-25
Cloudflare R2 (50GB, minimal egress)Pay-as-you-go~$1
Supabase Pro (8GB PG, 10GB disk)Pro$25
Domain (soli.app or similar)Standard~$15/yr
Total~$45/mo
02

DOCKER-COMPOSE MULTI-CONTAINER SETUP

File: docker-compose.yml

version: "3.9"

services:
  soli-api:
    image: ghcr.io/jaytherampage/soli-api:latest
    build:
      context: .
      dockerfile: Dockerfile.api
    container_name: soli-api
    restart: unless-stopped
    ports:
      - "8000:8000"
    expose:
      - "8000"
    env_file:
      - .env.production
    environment:
      - DATABASE_URL=postgresql://soli:${DB_PASSWORD}@soli-db:5432/soli
      - REDIS_URL=redis://soli-redis:6379/0
      - OMR_BROKER_URL=redis://soli-redis:6379/1
      - CORS_ORIGINS=https://soli.app
    volumes:
      - soli_media:/app/media
    depends_on:
      soli-redis:
        condition: service_healthy
      soli-db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  soli-omr-worker:
    image: ghcr.io/jaytherampage/soli-omr-worker:latest
    build:
      context: .
      dockerfile: Dockerfile.omr
    container_name: soli-omr-worker
    restart: unless-stopped
    env_file:
      - .env.production
    environment:
      - REDIS_URL=redis://soli-redis:6379/0
      - OMR_BROKER_URL=redis://soli-redis:6379/1
      - JAVA_OPTS=-Xms1g -Xmx4g
    volumes:
      - soli_media:/app/media
    depends_on:
      soli-redis:
        condition: service_healthy
      soli-db:
        condition: service_healthy
    deploy:
      resources:
        limits:
          cpus: "4"
          memory: "6g"
        reservations:
          cpus: "1"
          memory: "1g"
    healthcheck:
      test: ["CMD-SHELL", "celery -A soli_omr inspect ping || exit 1"]
      interval: 60s
      timeout: 15s
      retries: 3
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

  soli-redis:
    image: redis:7-alpine
    container_name: soli-redis
    restart: unless-stopped
    expose:
      - "6379"
    volumes:
      - redis_data:/data
    command: >
      redis-server
      --appendonly yes
      --appendfsync everysec
      --maxmemory 256mb
      --maxmemory-policy allkeys-lru
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5

  soli-db:
    image: postgres:16-alpine
    container_name: soli-db
    restart: unless-stopped
    expose:
      - "5432"
    env_file:
      - .env.production
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./backups:/backups
    environment:
      - POSTGRES_DB=soli
      - POSTGRES_USER=soli
      - POSTGRES_PASSWORD=${DB_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U soli"]
      interval: 10s
      timeout: 5s
      retries: 5
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

volumes:
  soli_media:
  redis_data:
  postgres_data:

Service Topology

soli-api ---http--> soli-redis
   |                (broker: DB 1)
   |                       |
   |              <--- soli-omr-worker
   |                (Celery consumer)
   |
   +---tcp--> soli-db
              (Postgres 16)

Deployment Commands

# First deploy
docker compose -f docker-compose.yml up -d

# Update services
docker compose pull soli-api soli-omr-worker && \
  docker compose up -d --no-deps soli-api soli-omr-worker

# View logs
docker compose logs -f soli-api

# Restart worker after OOM
docker compose restart soli-omr-worker

# Full stop and prune
docker compose down -v
03

CI/CD PIPELINE (GitHub Actions)

File: .github/workflows/deploy.yml

name: Deploy Soli

on:
  push:
    branches: [main]
    paths-ignore:
      - "**.md"
      - ".gitignore"
      - "LICENSE"

env:
  REGISTRY: ghcr.io
  IMAGE_API: ghcr.io/jaytherampage/soli-api
  IMAGE_OMR: ghcr.io/jaytherampage/soli-omr-worker
  HETZNER_HOST: ${{ secrets.HETZNER_HOST }}
  HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}

jobs:
  build-and-deploy:
    runs-on: ubuntu-22.04
    permissions:
      contents: read
      packages: write

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to GHCR
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      # ---- Build cache keys ----
      - name: Cache Docker layers (API)
        uses: actions/cache@v4
        with:
          path: /tmp/.buildx-cache-api
          key: ${{ runner.os }}-buildx-api-${{ hashFiles('requirements*.txt', 'pyproject.toml') }}
          restore-keys: |
            ${{ runner.os }}-buildx-api-

      - name: Cache Docker layers (OMR)
        uses: actions/cache@v4
        with:
          path: /tmp/.buildx-cache-omr
          key: ${{ runner.os }}-buildx-omr-${{ hashFiles('omr-requirements*.txt') }}
          restore-keys: |
            ${{ runner.os }}-buildx-omr-

      # ---- Build API image ----
      - name: Build and push soli-api
        uses: docker/build-push-action@v6
        with:
          context: .
          file: Dockerfile.api
          push: true
          tags: ${{ env.IMAGE_API }}:${{ github.sha }}, ${{ env.IMAGE_API }}:latest
          cache-from: type=local,src=/tmp/.buildx-cache-api
          cache-to: type=local,dest=/tmp/.buildx-cache-api-new,mode=max

      # ---- Build OMR worker image ----
      - name: Build and push soli-omr-worker
        uses: docker/build-push-action@v6
        with:
          context: .
          file: Dockerfile.omr
          push: true
          tags: ${{ env.IMAGE_OMR }}:${{ github.sha }}, ${{ env.IMAGE_OMR }}:latest
          cache-from: type=local,src=/tmp/.buildx-cache-omr
          cache-to: type=local,dest=/tmp/.buildx-cache-omr-new,mode=max

      # ---- Move caches (prune old) ----
      - name: Move API cache
        run: |
          rm -rf /tmp/.buildx-cache-api
          mv /tmp/.buildx-cache-api-new /tmp/.buildx-cache-api

      - name: Move OMR cache
        run: |
          rm -rf /tmp/.buildx-cache-omr
          mv /tmp/.buildx-cache-omr-new /tmp/.buildx-cache-omr

      # ---- Deploy to Hetzner ----
      - name: Deploy via SSH
        uses: appleboy/ssh-action@v1.0.3
        with:
          host: ${{ env.HETZNER_HOST }}
          username: deploy
          key: ${{ env.HETZNER_SSH_KEY }}
          script: |
            set -e

            cd /home/deploy/soli

            docker compose pull soli-api soli-omr-worker

            docker compose up -d --no-deps \
              --pull never soli-api soli-omr-worker

            # Prune old images
            docker image prune -af --filter "until=24h"

            echo "Deploy complete: $(date)"

Build Cache Strategy

  • - Layer caching: Docker layers are cached by hash of dependency files (requirements.txt, pyproject.toml). Only changed layers rebuild.
  • - GHCR tagging: Each deploy tags :latest AND :sha for quick rollbacks (docker compose pull soli-api:<previous-sha>).
  • - SSH deploy safety: The deploy script uses --no-deps to avoid restarting Redis/Postgres on every deploy.

Rollback Procedure

ssh deploy@hetzner-vps
cd /home/deploy/soli

# Rollback API to specific version
sed -i 's|soli-api:latest|soli-api:abc123def|' docker-compose.yml
docker compose up -d soli-api

# Or pull the previous tag directly
docker compose pull soli-api:abc123def
docker compose up -d --no-deps soli-api
04

AUDIVERIS DOCKER (v5.3) SYSTEM TUNING

File: Dockerfile.omr

FROM audiveris/audiveris:5.3-jdk17 AS base

RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 \
    python3-pip \
    netcat-traditional \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /app

# Copy Celery worker + OMR bridge
COPY omr_worker/requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt

COPY omr_worker/ .

# JVM tuning for OMR engine
ENV JAVA_OPTS="-Xms1g -Xmx4g \
  -XX:+UseG1GC \
  -XX:MaxGCPauseMillis=200 \
  -XX:ParallelGCThreads=2 \
  -XX:ConcGCThreads=1 \
  -XX:G1HeapRegionSize=4m \
  -XX:+UseStringDeduplication \
  -XX:+ExitOnOutOfMemoryError \
  -Dorg.audiveris.omr.memory.max=4096"

ENV OMR_WORKERS=2
ENV OMR_TEMP_DIR=/tmp/omr

RUN mkdir -p $OMR_TEMP_DIR

HEALTHCHECK --interval=30s --timeout=15s --start-period=60s --retries=3 \
  CMD celery -A soli_omr inspect ping || exit 1

ENTRYPOINT ["celery", "-A", "soli_omr", "worker", "--loglevel=info", \
  "--concurrency=1", "--max-tasks-per-child=1"]

JVM Flags Explanation

FlagValuePurpose
-Xms1gInitial heap allocation (fast startup)
-Xmx4gMax heap - prevents OOM on dense scores
-XX:+UseG1GCenabledLow-pause GC for interactive-ish OMR
-XX:MaxGCPauseMillis200Balance throughput vs latency
-XX:ParallelGCThreads2Cap GC parallelism (leave CPU for JIT)
-XX:G1HeapRegionSize4mFine-grained region sizing for large heaps
-XX:+UseStringDeduplicationenabledDedup repeated music symbol names
-XX:+ExitOnOutOfMemoryErrorenabledDie fast on OOM, Docker restarts

Docker Compose Resource Constraints

# In docker-compose.yml, under soli-omr-worker:
deploy:
  resources:
    limits:
      cpus: "4"
      memory: "6g"
    reservations:
      cpus: "1"
      memory: "1g"
  • - Memory limit 6GB: JVM heap (4GB) + JVM overhead (~500MB) + Python/Celery runtime (~300MB) + Audiveris native memory (~1GB for OpenCV, Tesseract) = ~5.8GB peak. The 6GB hard limit catches OOM before Docker kills the host.
  • - CPU limit 4 cores: Prevents a single OMR task from saturating the VPS and starving the FastAPI API container, which needs ~0.5-1 core for request serving.
  • - --concurrency=1 Celery flag: One OMR task at a time per container. Scale horizontally by adding replicas (docker compose up -d --scale soli-omr-worker=2) rather than increasing concurrency, since Audiveris is single-process CPU bound.

Monitoring OMR Worker Health

# Check JVM heap usage inside container
docker exec soli-omr-worker jcmd 1 VM.native_memory summary

# Check Celery queue depth
redis-cli -h localhost -p 6379 LLEN omr_queue

# Worker stats via Celery flower
docker run -d --rm -p 5555:5555 \
  --network soli_default \
  mher/flower \
  --broker=redis://soli-redis:6379/1
05

REVENUECAT WEBHOOK HANDLER FLOW

Endpoint: POST /api/webhooks/revenuecat

Security & Validation Flow

Incoming HTTPS POST from RevenueCat
              |
              v
  +-----------------------------+
  | 1. Extract Authorization    |
  |    header: "Bearer <token>" |
  +-------------+--------------+
                |
                v
  +-----------------------------+
  | 2. Validate HMAC signature  |
  |    HMAC-SHA256(             |
  |      body,                  |
  |      RC_WEBHOOK_SECRET      |
  |    ) == X-Hub-Signature     |
  |                             |
  |    If mismatch -> 401       |
  +-------------+--------------+
                |
                v
  +-----------------------------+
  | 3. Parse JSON body          |
  |    event.type               |
  |    event.app_user_id        |
  |    event.product_id         |
  |    event.expiration_at_ms   |
  |    event.period_type        |
  |    event.store              |
  +-------------+--------------+
                |
                v
  +-----------------------------+
  | 4. Route by event.type      |
  |                             |
  | INITIAL_PURCHASE -----+     |
  | RENEWAL --------------+     |
  |                       |     |
  |                       v     |
  |    UPDATE users SET         |
  |      subscription_status =  |
  |        'active',            |
  |      subscription_expires_at|
  |        = event.expiration,  |
  |      updated_at = NOW()     |
  |      WHERE id = app_user_id |
  |                             |
  | CANCELLATION ---------+     |
  | BILLING_ISSUE --------+     |
  |                       v     |
  |    UPDATE users SET         |
  |      subscription_status =  |
  |        'grace_period',      |
  |      updated_at = NOW()     |
  |      WHERE id = app_user_id |
  |      AND                    |
  |      subscription_status    |
  |      != 'expired'           |
  |                             |
  | EXPIRATION -----------+     |
  |                       v     |
  |    UPDATE users SET         |
  |      subscription_status =  |
  |        'expired',           |
  |      updated_at = NOW()     |
  |      WHERE id = app_user_id |
  +-------------+--------------+
                |
                v
  +-----------------------------+
  | 5. Return 200 OK            |
  |    (RevenueCat expects      |
  |     2xx within 10s)         |
  +-----------------------------+

FastAPI Implementation (Pseudocode)

# app/api/webhooks/revenuecat.py

import hashlib
import hmac
import json
import logging
from datetime import datetime, timezone
from typing import Literal

from fastapi import APIRouter, Depends, HTTPException, Request
from sqlalchemy import update
from sqlalchemy.ext.asyncio import AsyncSession

from app.core.config import settings
from app.core.database import get_db
from app.models.user import User, SubscriptionStatus

logger = logging.getLogger(__name__)
router = APIRouter(prefix="/webhooks/revenuecat", tags=["webhooks"])

RC_EVENT_TYPES = Literal[
    "INITIAL_PURCHASE",
    "RENEWAL",
    "CANCELLATION",
    "BILLING_ISSUE",
    "EXPIRATION",
    "NON_RENEWING_PURCHASE",
    "SUBSCRIBER_ALIAS",
]


def verify_webhook_signature(
    payload: bytes,
    signature_header: str,
    secret: str,
) -> bool:
    expected = hmac.new(
        secret.encode("utf-8"),
        payload,
        hashlib.sha256,
    ).hexdigest()
    return hmac.compare_digest(expected, signature_header)


@router.post("")
async def handle_revenuecat_webhook(
    request: Request,
    db: AsyncSession = Depends(get_db),
):
    # 1. Read raw body (must be bytes for HMAC)
    body = await request.body()
    content_type = request.headers.get("content-type", "")

    # 2. Validate signature
    signature = request.headers.get("x-hub-signature", "")
    if not signature:
        logger.warning("Missing x-hub-signature header")
        raise HTTPException(status_code=401, detail="Missing signature")

    if not verify_webhook_signature(
        body, signature, settings.RC_WEBHOOK_SECRET
    ):
        logger.warning("Invalid webhook signature")
        raise HTTPException(status_code=401, detail="Invalid signature")

    # 3. Parse event
    try:
        event = json.loads(body)
    except json.JSONDecodeError:
        raise HTTPException(status_code=400, detail="Invalid JSON")

    event_type: str = event.get("event", {}).get("type", "")
    app_user_id: str = event.get("event", {}).get("app_user_id", "")
    expiration_ms: int = (
        event.get("event", {}).get("expiration_at_ms", 0) or 0
    )

    if not app_user_id:
        logger.warning("Webhook missing app_user_id")
        return {"status": "skipped", "reason": "no_user_id"}

    # 4. Route event type
    expiration_dt = (
        datetime.fromtimestamp(expiration_ms / 1000, tz=timezone.utc)
        if expiration_ms
        else None
    )

    if event_type in ("INITIAL_PURCHASE", "RENEWAL"):
        stmt = (
            update(User)
            .where(User.id == app_user_id)
            .values(
                subscription_status=SubscriptionStatus.ACTIVE,
                subscription_expires_at=expiration_dt,
                updated_at=datetime.now(timezone.utc),
            )
        )
        await db.execute(stmt)
        await db.commit()
        logger.info(
            "Subscription activated for user=%s expires=%s",
            app_user_id,
            expiration_dt,
        )

    elif event_type in ("CANCELLATION", "BILLING_ISSUE"):
        stmt = (
            update(User)
            .where(User.id == app_user_id)
            .where(User.subscription_status != SubscriptionStatus.EXPIRED)
            .values(
                subscription_status=SubscriptionStatus.GRACE_PERIOD,
                updated_at=datetime.now(timezone.utc),
            )
        )
        await db.execute(stmt)
        await db.commit()
        logger.info(
            "Subscription grace period for user=%s (event=%s)",
            app_user_id,
            event_type,
        )

    elif event_type == "EXPIRATION":
        stmt = (
            update(User)
            .where(User.id == app_user_id)
            .values(
                subscription_status=SubscriptionStatus.EXPIRED,
                subscription_expires_at=expiration_dt,
                updated_at=datetime.now(timezone.utc),
            )
        )
        await db.execute(stmt)
        await db.commit()
        logger.info("Subscription expired for user=%s", app_user_id)

    else:
        logger.info("Unhandled event type=%s user=%s", event_type, app_user_id)

    # 5. Always return 2xx within 10 seconds
    return {"status": "received", "event_type": event_type}

RevenueCat User Mapping Schema

-- Users table extension for IAP subscription tracking
ALTER TABLE users ADD COLUMN IF NOT EXISTS
  subscription_status VARCHAR(20)
  NOT NULL DEFAULT 'free'
  CHECK (subscription_status IN (
    'free', 'active', 'grace_period', 'expired'
  ));

ALTER TABLE users ADD COLUMN IF NOT EXISTS
  subscription_expires_at TIMESTAMPTZ;

ALTER TABLE users ADD COLUMN IF NOT EXISTS
  rc_app_user_id VARCHAR(255);

CREATE INDEX IF NOT EXISTS idx_users_subscription_status
  ON users (subscription_status);

CREATE INDEX IF NOT EXISTS idx_users_rc_app_user_id
  ON users (rc_app_user_id);

Testing Webhooks Locally

# Use ngrok or SSH tunnel for RevenueCat to reach localhost
ngrok http 8000

# Then configure RevenueCat dashboard:
# Webhook URL: https://<ngrok-id>.ngrok.dev/api/webhooks/revenuecat
# Authorization header: Bearer <your-secret>

# Simulate a test event with curl:
curl -X POST http://localhost:8000/api/webhooks/revenuecat \
  -H "Content-Type: application/json" \
  -H "X-Hub-Signature: $(echo -n '{"event":{"type":"INITIAL_PURCHASE","app_user_id":"test_user_1","expiration_at_ms":1893456000000}}' | openssl dgst -sha256 -hmac 'test-secret' | cut -d' ' -f2)" \
  -d '{"event":{"type":"INITIAL_PURCHASE","app_user_id":"test_user_1","expiration_at_ms":1893456000000}}'

DEPLOYMENT CHECKLIST

[ ]Hetzner VPS provisioned with Docker + Docker Compose
[ ]Cloudflare DNS proxied (orange cloud)
[ ]Supabase project created or Postgres container running
[ ]Cloudflare R2 bucket created for backups
[ ]GitHub Actions secrets configured (SSH key, Hetzner host)
[ ]RevenueCat webhook configured pointing to /api/webhooks/revenuecat
[ ]WAL-G cron scheduled for hourly DB backups to R2
[ ]Monitoring: Uptime Kuma or Grafana health checks
[ ]SSL via Cloudflare (Full/Strict mode)