ai-stack-deployer/docs/DEPLOYMENT_NOTES.md

Deployment Notes - AI Stack Deployer

Automated Deployment Documentation

Date: 2026-01-09 Operator: Claude Code Target: Dokploy (10.100.0.20:3000) Domain: portal.ai.flexinit.nl (or TBD)

---

Phase 1: Pre-Deployment Verification

Step 1.1: Environment Variables Check

Purpose: Verify all required credentials are available

Commands:

# Check if .env file exists
test -f .env && echo "✓ .env exists" || echo "✗ .env missing"

# Verify required variables are set (without exposing values)
grep -q "DOKPLOY_API_TOKEN=" .env && echo "✓ DOKPLOY_API_TOKEN set" || echo "✗ DOKPLOY_API_TOKEN missing"
grep -q "DOKPLOY_URL=" .env && echo "✓ DOKPLOY_URL set" || echo "✗ DOKPLOY_URL missing"

Automation Notes:

• Script must check for .env file existence
• Validate required variables: DOKPLOY_API_TOKEN, DOKPLOY_URL
• Exit with error if missing critical variables

---

Step 1.2: Dokploy API Connectivity Test

Purpose: Ensure we can reach Dokploy API before attempting deployment

Commands:

# Test API connectivity (masked token in logs)
curl -s -o /dev/null -w "%{http_code}" \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  "${DOKPLOY_URL}/api/project.all"

Expected Result: HTTP 200 On Failure: Check network access to 10.100.0.20:3000

Automation Notes:

• Test API before proceeding
• Log HTTP status code
• Abort if not 200

---

Step 1.3: Docker Environment Check

Purpose: Verify Docker is available for building

Commands:

# Check Docker installation
docker --version

# Check Docker daemon is running
docker ps > /dev/null 2>&1 && echo "✓ Docker running" || echo "✗ Docker not running"

# Check available disk space (need ~500MB)
df -h . | awk 'NR==2 {print "Available:", $4}'

Automation Notes:

• Verify Docker installed and running
• Check minimum 500MB free space
• Fail fast if Docker unavailable

---

Phase 2: Docker Image Build

Step 2.1: Build Docker Image

Purpose: Create production Docker image

Commands:

# Build with timestamp tag
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
IMAGE_TAG="ai-stack-deployer:${TIMESTAMP}"
IMAGE_TAG_LATEST="ai-stack-deployer:latest"

docker build \
  -t "${IMAGE_TAG}" \
  -t "${IMAGE_TAG_LATEST}" \
  --progress=plain \
  .

Expected Duration: 2-3 minutes Expected Size: ~150-200MB

Automation Notes:

• Use timestamp tags for traceability
• Always tag as :latest as well
• Stream build logs for debugging
• Check exit code (0 = success)

---

Step 2.2: Verify Build Success

Purpose: Confirm image was created successfully

Commands:

# List the newly created image
docker images ai-stack-deployer:latest

# Get image ID and size
IMAGE_ID=$(docker images -q ai-stack-deployer:latest)
echo "Image ID: ${IMAGE_ID}"

# Inspect image metadata
docker inspect "${IMAGE_ID}" --format='{{.Config.ExposedPorts}}'
docker inspect "${IMAGE_ID}" --format='{{.Config.Healthcheck.Test}}'

Automation Notes:

• Verify image exists with correct name
• Log image ID and size
• Confirm healthcheck is configured

---

Phase 3: Local Container Testing

Step 3.1: Start Test Container

Purpose: Verify container runs before deploying to production

Commands:

# Start container in detached mode
docker run -d \
  --name ai-stack-deployer-test \
  -p 3001:3000 \
  --env-file .env \
  ai-stack-deployer:latest

# Wait for container to be ready (max 30 seconds)
timeout 30 bash -c 'until docker exec ai-stack-deployer-test curl -f http://localhost:3000/health 2>/dev/null; do sleep 1; done'

Expected Result: Container starts and responds to health check

Automation Notes:

• Use non-conflicting port (3001) for testing
• Wait for health check before proceeding
• Timeout after 30 seconds if unhealthy

---

Step 3.2: Health Check Verification

Purpose: Verify application is running correctly

Commands:

# Test health endpoint from host
curl -s http://localhost:3001/health | jq .

# Check container logs for errors
docker logs ai-stack-deployer-test 2>&1 | tail -20

# Verify no crashes
docker ps -f name=ai-stack-deployer-test --format "{{.Status}}"

Expected Response:

{
  "status": "healthy",
  "timestamp": "...",
  "version": "0.1.0",
  "service": "ai-stack-deployer",
  "activeDeployments": 0
}

Automation Notes:

• Parse JSON response and verify status="healthy"
• Check for ERROR/FATAL in logs
• Confirm container is "Up" status

---

Step 3.3: Cleanup Test Container

Purpose: Remove test container after verification

Commands:

# Stop and remove test container
docker stop ai-stack-deployer-test
docker rm ai-stack-deployer-test

echo "✓ Test container cleaned up"

Automation Notes:

• Always cleanup test resources
• Use --force flags if automation needs to be idempotent

---

Phase 4: Image Registry Push (Optional)

Step 4.1: Tag for Registry

Purpose: Prepare image for remote registry (if not using local Dokploy)

Commands:

# Example for custom registry
REGISTRY="git.app.flexinit.nl"
docker tag ai-stack-deployer:latest "${REGISTRY}/ai-stack-deployer:latest"
docker tag ai-stack-deployer:latest "${REGISTRY}/ai-stack-deployer:${TIMESTAMP}"

Automation Notes:

• Skip if Dokploy can access local Docker daemon
• Required if Dokploy is on separate server

---

Step 4.2: Push to Registry

Purpose: Upload image to registry

Commands:

# Login to registry (if required)
echo "${REGISTRY_PASSWORD}" | docker login "${REGISTRY}" -u "${REGISTRY_USER}" --password-stdin

# Push images
docker push "${REGISTRY}/ai-stack-deployer:latest"
docker push "${REGISTRY}/ai-stack-deployer:${TIMESTAMP}"

Automation Notes:

• Store registry credentials securely
• Verify push succeeded (check exit code)
• Log image digest for traceability

---

Phase 5: Dokploy Deployment

Step 5.1: Check for Existing Project

Purpose: Determine if this is a new deployment or update

Commands:

# Search for existing project
curl -s \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  "${DOKPLOY_URL}/api/project.all" | \
  jq -r '.projects[] | select(.name=="ai-stack-deployer-portal") | .projectId'

Automation Notes:

• If project exists: update existing
• If not found: create new project
• Store project ID for subsequent API calls

---

Step 5.2: Create Dokploy Project (if new)

Purpose: Create project container in Dokploy

Commands:

# Create project via API
PROJECT_RESPONSE=$(curl -s -X POST \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  -H "Content-Type: application/json" \
  "${DOKPLOY_URL}/api/project.create" \
  -d '{
    "name": "ai-stack-deployer-portal",
    "description": "Self-service portal for deploying AI stacks"
  }')

# Extract project ID
PROJECT_ID=$(echo "${PROJECT_RESPONSE}" | jq -r '.projectId')
echo "Created project: ${PROJECT_ID}"

Automation Notes:

• Parse response for projectId
• Handle error if project name conflicts
• Store PROJECT_ID for next steps

---

Step 5.3: Create Application

Purpose: Create application within project

Commands:

# Create application
APP_RESPONSE=$(curl -s -X POST \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  -H "Content-Type: application/json" \
  "${DOKPLOY_URL}/api/application.create" \
  -d "{
    \"name\": \"ai-stack-deployer-web\",
    \"projectId\": \"${PROJECT_ID}\",
    \"dockerImage\": \"ai-stack-deployer:latest\",
    \"env\": \"DOKPLOY_URL=${DOKPLOY_URL}\\nDOKPLOY_API_TOKEN=${DOKPLOY_API_TOKEN}\\nPORT=3000\\nHOST=0.0.0.0\"
  }")

# Extract application ID
APP_ID=$(echo "${APP_RESPONSE}" | jq -r '.applicationId')
echo "Created application: ${APP_ID}"

Automation Notes:

• Set all required environment variables
• Use escaped newlines for env variables
• Store APP_ID for domain and deployment

---

Step 5.4: Configure Domain

Purpose: Set up domain routing through Traefik

Commands:

# Determine domain name (use portal.ai.flexinit.nl or ask user)
DOMAIN="portal.ai.flexinit.nl"

# Create domain mapping
curl -s -X POST \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  -H "Content-Type: application/json" \
  "${DOKPLOY_URL}/api/domain.create" \
  -d "{
    \"domain\": \"${DOMAIN}\",
    \"applicationId\": \"${APP_ID}\",
    \"https\": true,
    \"port\": 3000
  }"

echo "Configured domain: https://${DOMAIN}"

Automation Notes:

• Domain must match wildcard DNS pattern
• Enable HTTPS (Traefik handles SSL)
• Port 3000 matches container expose

---

Step 5.5: Deploy Application

Purpose: Trigger deployment on Dokploy

Commands:

# Trigger deployment
DEPLOY_RESPONSE=$(curl -s -X POST \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  -H "Content-Type: application/json" \
  "${DOKPLOY_URL}/api/application.deploy" \
  -d "{
    \"applicationId\": \"${APP_ID}\"
  }")

# Extract deployment ID
DEPLOY_ID=$(echo "${DEPLOY_RESPONSE}" | jq -r '.deploymentId // "unknown"')
echo "Deployment started: ${DEPLOY_ID}"
echo "Monitor at: ${DOKPLOY_URL}/project/${PROJECT_ID}"

Automation Notes:

• Deployment is asynchronous
• Need to poll for completion
• Typical deployment: 1-3 minutes

---

Phase 6: Deployment Verification

Step 6.1: Wait for Deployment

Purpose: Monitor deployment until complete

Commands:

# Poll deployment status (example - adjust based on Dokploy API)
MAX_WAIT=300  # 5 minutes
ELAPSED=0
INTERVAL=10

while [ $ELAPSED -lt $MAX_WAIT ]; do
  # Check if application is running
  STATUS=$(curl -s \
    -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
    "${DOKPLOY_URL}/api/application.status?id=${APP_ID}" | \
    jq -r '.status // "unknown"')

  echo "Status: ${STATUS} (${ELAPSED}s elapsed)"

  if [ "${STATUS}" = "running" ]; then
    echo "✓ Deployment completed successfully"
    break
  fi

  sleep ${INTERVAL}
  ELAPSED=$((ELAPSED + INTERVAL))
done

if [ $ELAPSED -ge $MAX_WAIT ]; then
  echo "✗ Deployment timeout after ${MAX_WAIT}s"
  exit 1
fi

Automation Notes:

• Poll with exponential backoff
• Timeout after reasonable duration
• Log status changes

---

Step 6.2: Health Check via Domain

Purpose: Verify application is accessible via public URL

Commands:

# Test public endpoint
echo "Testing: https://${DOMAIN}/health"

# Allow time for DNS/SSL propagation
sleep 10

# Verify health endpoint
HEALTH_RESPONSE=$(curl -s "https://${DOMAIN}/health")
HEALTH_STATUS=$(echo "${HEALTH_RESPONSE}" | jq -r '.status // "error"')

if [ "${HEALTH_STATUS}" = "healthy" ]; then
  echo "✓ Application is healthy"
  echo "${HEALTH_RESPONSE}" | jq .
else
  echo "✗ Application health check failed"
  echo "${HEALTH_RESPONSE}"
  exit 1
fi

Expected Response:

{
  "status": "healthy",
  "timestamp": "2026-01-09T...",
  "version": "0.1.0",
  "service": "ai-stack-deployer",
  "activeDeployments": 0
}

Automation Notes:

• Test via HTTPS (validate SSL works)
• Retry on first failure (DNS propagation)
• Verify JSON structure and status field

---

Step 6.3: Frontend Accessibility Test

Purpose: Confirm frontend loads correctly

Commands:

# Test root endpoint returns HTML
curl -s "https://${DOMAIN}/" | head -20

# Check for expected HTML content
if curl -s "https://${DOMAIN}/" | grep -q "AI Stack Deployer"; then
  echo "✓ Frontend is accessible"
else
  echo "✗ Frontend not loading correctly"
  exit 1
fi

Automation Notes:

• Verify HTML contains expected title
• Check for 200 status code
• Test at least one static asset (CSS/JS)

---

Step 6.4: API Endpoint Test

Purpose: Verify API endpoints respond correctly

Commands:

# Test name availability check
TEST_RESPONSE=$(curl -s "https://${DOMAIN}/api/check/test-deployment-123")
echo "API Test Response:"
echo "${TEST_RESPONSE}" | jq .

# Verify response structure
if echo "${TEST_RESPONSE}" | jq -e '.valid' > /dev/null; then
  echo "✓ API endpoints functional"
else
  echo "✗ API response malformed"
  exit 1
fi

Automation Notes:

• Test each critical endpoint
• Verify JSON responses parse correctly
• Log any API errors for debugging

---

Phase 7: Post-Deployment

Step 7.1: Document Deployment Details

Purpose: Record deployment information for reference

Commands:

# Create deployment record
cat > deployment-record-${TIMESTAMP}.txt << EOF
Deployment Completed: $(date -Iseconds)
Project ID: ${PROJECT_ID}
Application ID: ${APP_ID}
Deployment ID: ${DEPLOY_ID}
Image: ai-stack-deployer:${TIMESTAMP}
Domain: https://${DOMAIN}
Health Check: https://${DOMAIN}/health
Dokploy Console: ${DOKPLOY_URL}/project/${PROJECT_ID}

Status: SUCCESS
EOF

echo "Deployment record saved: deployment-record-${TIMESTAMP}.txt"

Automation Notes:

• Save deployment metadata
• Include rollback information
• Log all IDs for future operations

---

Step 7.2: Cleanup Build Artifacts

Purpose: Remove temporary files and images

Commands:

# Keep latest, remove older images
docker images ai-stack-deployer --format "{{.Tag}}" | \
  grep -v latest | \
  xargs -r -I {} docker rmi ai-stack-deployer:{} 2>/dev/null || true

# Clean up build cache if needed
# docker builder prune -f

echo "✓ Cleanup completed"

Automation Notes:

• Keep :latest tag
• Optional: clean build cache
• Don't fail script if no images to remove

---

Automation Script Skeleton

#!/usr/bin/env bash
set -euo pipefail

# Configuration
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="${SCRIPT_DIR}/.."
TIMESTAMP=$(date +%Y%m%d-%H%M%S)

# Load environment
source "${PROJECT_ROOT}/.env"

# Functions
log_info() { echo "[INFO] $*"; }
log_error() { echo "[ERROR] $*" >&2; }
check_prerequisites() { ... }
build_image() { ... }
test_locally() { ... }
deploy_to_dokploy() { ... }
verify_deployment() { ... }

# Main execution
main() {
  log_info "Starting deployment at ${TIMESTAMP}"

  check_prerequisites
  build_image
  test_locally
  deploy_to_dokploy
  verify_deployment

  log_info "Deployment completed successfully!"
  log_info "Access: https://${DOMAIN}"
}

main "$@"

---

Rollback Procedure

If deployment fails:

# Get previous deployment
PREV_DEPLOY=$(curl -s \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  "${DOKPLOY_URL}/api/deployment.list?applicationId=${APP_ID}" | \
  jq -r '.deployments[1].deploymentId')

# Rollback
curl -X POST \
  -H "x-api-key: ${DOKPLOY_API_TOKEN}" \
  "${DOKPLOY_URL}/api/deployment.rollback" \
  -d "{\"deploymentId\": \"${PREV_DEPLOY}\"}"

---

Notes for Future Automation

1. Error Handling: Add || exit 1 to critical steps
2. Logging: Redirect all output to log file: 2>&1 | tee deployment.log
3. Notifications: Add Slack/email notifications on success/failure
4. Parallel Testing: Run multiple verification tests concurrently
5. Metrics: Collect deployment duration, image size, startup time
6. CI/CD Integration: Trigger on git push with GitHub Actions/GitLab CI

---

End of Deployment Notes

---

Graphiti Memory Search Results

Dokploy Infrastructure Details:

• Location: 10.100.0.20:3000 (shares VM with Grafana/Loki)
• UI: https://deploy.intra.flexinit.nl (requires login)
• Config Location: /etc/dokploy/compose/
• API Token Format: app_deployment{random}
• Token Generation: Via Dokploy UI → Settings → Profile → API Tokens
• Token Storage: BWS secret 6b3618fc-ba02-49bc-bdc8-b3c9004087bc

Previous Known Issues:

• 401 Unauthorized errors occurred (token might need regeneration)
• Credentials stored in Bitwarden at pass.cloud.flexinit.nl

Registry Information:

• Docker image referenced: git.app.flexinit.nl/oussamadouhou/oh-my-opencode-free:latest
• This suggests git.app.flexinit.nl may have a Docker registry