ai-stack-deployer/docs/SHARED_PROJECT_DEPLOYMENT.md

# Shared Project Deployment Architecture

## Overview

The AI Stack Deployer portal deploys **all user AI stacks to a single shared Dokploy project** instead of creating a new project for each user.

---

## Architecture Diagram

```
┌─────────────────────────────────────────────────────────────────┐
│  Dokploy: ai-stack-portal (Shared Project)                     │
│  ID: 2y2Glhz5Wy0dBNf6BOR_-                                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  📦 Portal Application: ai-stack-deployer-prod                 │
│     ├─ Domain: portal.ai.flexinit.nl                           │
│     ├─ Image: git.app.flexinit.nl/.../ai-stack-deployer:latest│
│     └─ Env: SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}  │
│                                                                 │
│  ───────────────────────────────────────────────────────────── │
│                                                                 │
│  📦 User Stack: john-dev                                        │
│     ├─ Domain: john-dev.ai.flexinit.nl                         │
│     ├─ Image: git.app.flexinit.nl/.../agent-stack:latest      │
│     └─ Deployed by: Portal                                     │
│                                                                 │
│  📦 User Stack: jane-prod                                       │
│     ├─ Domain: jane-prod.ai.flexinit.nl                        │
│     ├─ Image: git.app.flexinit.nl/.../agent-stack:latest      │
│     └─ Deployed by: Portal                                     │
│                                                                 │
│  📦 User Stack: alice-test                                      │
│     ├─ Domain: alice-test.ai.flexinit.nl                       │
│     ├─ Image: git.app.flexinit.nl/.../agent-stack:latest      │
│     └─ Deployed by: Portal                                     │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘
```

---

## How It Works

### Step 1: Portal Reads Configuration

When a user submits a stack name (e.g., "john-dev"), the portal:

1. **Reads environment variables**:
   ```javascript
   const sharedProjectId = process.env.SHARED_PROJECT_ID;
   const sharedEnvironmentId = process.env.SHARED_ENVIRONMENT_ID;
   ```

2. **These are set via Dokploy's project-level variables**:
   ```yaml
   environment:
     - SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
     - SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
   ```

### Step 2: Portal Deploys to Shared Project

Instead of creating a new project, the portal:

```javascript
// OLD BEHAVIOR (legacy):
// createProject(`ai-stack-${username}`)  ❌ Creates new project per user

// NEW BEHAVIOR (current):
// Uses existing shared project ID  ✅
const projectId = sharedProjectId;  // From environment variable
const environmentId = sharedEnvironmentId;

// Creates application IN the shared project
createApplication({
  projectId: projectId,
  environmentId: environmentId,
  name: `${username}-stack`,
  image: 'git.app.flexinit.nl/.../agent-stack:latest',
  domain: `${username}.ai.flexinit.nl`
});
```

### Step 3: User Accesses Their Stack

User visits `https://john-dev.ai.flexinit.nl` → Traefik routes to their application inside the shared project.

---

## Configuration Steps

### 1. Create Shared Project in Dokploy

1. In Dokploy UI, create project:
   - **Name**: `ai-stack-portal`
   - **Description**: "Shared project for all user AI stacks"

2. Get the **Project ID**:
   ```bash
   # Via API
   curl -s "http://10.100.0.20:3000/api/project.all" \
     -H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
     jq -r '.[] | select(.name=="ai-stack-portal") | .id'
   
   # Output: 2y2Glhz5Wy0dBNf6BOR_-
   ```

3. Get the **Environment ID**:
   ```bash
   curl -s "http://10.100.0.20:3000/api/project.one?projectId=2y2Glhz5Wy0dBNf6BOR_-" \
     -H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
     jq -r '.environments[0].id'
   
   # Output: RqE9OFMdLwkzN7pif1xN8
   ```

### 2. Set Project-Level Variables

In the shared project (`ai-stack-portal`), add these **project-level environment variables**:

| Variable | Value | Example |
|----------|-------|---------|
| `SHARED_PROJECT_ID` | Your project ID | `2y2Glhz5Wy0dBNf6BOR_-` |
| `SHARED_ENVIRONMENT_ID` | Your environment ID | `RqE9OFMdLwkzN7pif1xN8` |

**How to set in Dokploy UI**:
- Go to Project → Settings → Environment Variables
- Add variables at **project level** (not application level)

### 3. Deploy Portal Application

Deploy the portal **inside the same shared project**:

1. **Application Details**:
   - Name: `ai-stack-deployer-prod`
   - Type: Docker Compose
   - Compose File: `docker-compose.prod.yml`
   - Branch: `main`

2. **The docker-compose file automatically references project variables**:
   ```yaml
   environment:
     - SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}  # ← Magic happens here
     - SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
   ```

3. **Dokploy resolves `${{project.VAR}}`** to the actual value from project-level variables.

---

## Benefits

### ✅ Centralized Management
All user stacks in one place:
- Easy to list all active stacks
- Shared monitoring dashboard
- Centralized logging

### ✅ Resource Efficiency
- No overhead of separate projects per user
- Shared network and resources
- Easier to manage quotas

### ✅ Simplified Configuration
- Project-level environment variables shared by all stacks
- Single source of truth for common configs
- Easy to update STACK_IMAGE for all users

### ✅ Better Organization
```
Projects in Dokploy:
├── ai-stack-portal (500 user applications)  ✅ Clean
└── NOT:
    ├── ai-stack-john
    ├── ai-stack-jane
    ├── ai-stack-alice
    └── ... (500 separate projects)  ❌ Messy
```

---

## Fallback Behavior

If `SHARED_PROJECT_ID` and `SHARED_ENVIRONMENT_ID` are **not set**, the portal falls back to **legacy behavior**:

```javascript
// Code in src/orchestrator/production-deployer.ts (lines 187-196)
const sharedProjectId = config.sharedProjectId || process.env.SHARED_PROJECT_ID;
const sharedEnvironmentId = config.sharedEnvironmentId || process.env.SHARED_ENVIRONMENT_ID;

if (sharedProjectId && sharedEnvironmentId) {
  // Use shared project ✅
  state.resources.projectId = sharedProjectId;
  state.resources.environmentId = sharedEnvironmentId;
  return;
}

// Fallback: Create separate project per user ⚠️
const projectName = `ai-stack-${config.stackName}`;
const existingProject = await this.client.findProjectByName(projectName);
// ...
```

**This ensures backwards compatibility** but is not recommended.

---

## Troubleshooting

### Portal Creates Separate Projects Instead of Using Shared Project

**Cause**: `SHARED_PROJECT_ID` or `SHARED_ENVIRONMENT_ID` not set.

**Solution**:
1. Check project-level variables in Dokploy:
   ```bash
   curl -s "http://10.100.0.20:3000/api/project.one?projectId=YOUR_PROJECT_ID" \
     -H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
     jq '.environmentVariables'
   ```

2. Ensure the portal application's docker-compose references them:
   ```yaml
   environment:
     - SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
     - SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
   ```

3. Redeploy the portal application.

### Variable Reference Not Working

**Symptom**: Portal logs show `undefined` for `SHARED_PROJECT_ID`.

**Cause**: Using wrong syntax.

**Correct syntax**:
```yaml
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}  ✅
```

**Wrong syntax**:
```yaml
- SHARED_PROJECT_ID=${SHARED_PROJECT_ID}  ❌ (shell substitution, not Dokploy)
- SHARED_PROJECT_ID={{project.SHARED_PROJECT_ID}}  ❌ (missing $)
```

### How to Verify Configuration

Check portal container environment:
```bash
# SSH into Dokploy host
ssh user@10.100.0.20

# Inspect portal container
docker exec ai-stack-deployer env | grep SHARED

# Should show:
SHARED_PROJECT_ID=2y2Glhz5Wy0dBNf6BOR_-
SHARED_ENVIRONMENT_ID=RqE9OFMdLwkzN7pif1xN8
```

---

## Environment-Specific Shared Projects

You can have **separate shared projects for dev/staging/prod**:

| Portal Environment | Shared Project | Purpose |
|--------------------|----------------|---------|
| Dev | `ai-stack-portal-dev` | Development user stacks |
| Staging | `ai-stack-portal-staging` | Staging user stacks |
| Prod | `ai-stack-portal` | Production user stacks |

Each portal deployment references its own shared project:
- `portal-dev.ai.flexinit.nl` → `ai-stack-portal-dev`
- `portal-staging.ai.flexinit.nl` → `ai-stack-portal-staging`
- `portal.ai.flexinit.nl` → `ai-stack-portal`

---

## Migration from Legacy

If you're currently using the legacy behavior (separate projects per user):

### Option 1: Gradual Migration
- New deployments use shared project
- Old deployments remain in separate projects
- Migrate old stacks manually over time

### Option 2: Full Migration
1. Create shared project
2. Set project-level variables
3. Redeploy all user stacks to shared project
4. Delete old separate projects

**Note**: Migration requires downtime for each stack being moved.

---

## Reference

- **Environment Variable Syntax**: See Dokploy docs on project-level variables
- **Code Location**: `src/orchestrator/production-deployer.ts` (lines 178-200)
- **Example IDs**: `.env.example` (lines 25-27)

---

**Questions?** Check the main deployment guide: `DOKPLOY_DEPLOYMENT.md`