feat: add shared project deployment with Dokploy project-level variables

- Add SHARED_PROJECT_ID and SHARED_ENVIRONMENT_ID to all docker-compose files
- Use Dokploy's project-level variable syntax: ${{project.VARIABLE}}
- Deploy all user AI stacks to a single shared Dokploy project
- Update DOKPLOY_DEPLOYMENT.md with shared project configuration guide
- Add comprehensive SHARED_PROJECT_DEPLOYMENT.md architecture documentation

Benefits:
- Centralized management (all stacks in one project)
- Resource efficiency (no per-user project overhead)
- Simplified configuration (project-level shared vars)
- Better organization (500 apps in 1 project vs 500 projects)

How it works:
1. Portal reads SHARED_PROJECT_ID from environment
2. Docker-compose uses ${{project.SHARED_PROJECT_ID}} to reference project-level vars
3. Dokploy resolves these at runtime
4. Portal deploys user stacks as applications within the shared project

Fallback: If variables not set, falls back to legacy behavior (separate project per user)
This commit is contained in:
Oussama Douhou
2026-01-13 12:01:59 +01:00
parent 10ed0e46d8
commit 9a593b8b7c
5 changed files with 402 additions and 0 deletions

View File

@@ -15,6 +15,8 @@ services:
- STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl} - STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl}
- STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest} - STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest}
- RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal} - RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal}
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
env_file: env_file:
- .env - .env
restart: unless-stopped restart: unless-stopped

View File

@@ -15,6 +15,8 @@ services:
- STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl} - STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl}
- STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest} - STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest}
- RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal} - RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal}
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
env_file: env_file:
- .env - .env
restart: unless-stopped restart: unless-stopped

View File

@@ -15,6 +15,8 @@ services:
- STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl} - STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl}
- STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest} - STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest}
- RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal} - RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal}
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
env_file: env_file:
- .env - .env
restart: unless-stopped restart: unless-stopped

View File

@@ -91,6 +91,75 @@ main branch → git.app.flexinit.nl/oussamadouhou/ai-stack-deployer:latest
--- ---
## Shared Project Configuration (IMPORTANT)
### What is Shared Project Deployment?
The portal deploys **all user AI stacks as applications within a single shared Dokploy project**, instead of creating a new project for each user. This provides:
- ✅ Better organization (all stacks in one place)
- ✅ Shared environment variables
- ✅ Centralized monitoring
- ✅ Easier management
### How It Works
```
Dokploy Project: ai-stack-portal
├── Environment: deployments
│ ├── Application: john-dev
│ ├── Application: jane-prod
│ └── Application: alice-test
```
### Setting Up the Shared Project
**Step 1: Create the Shared Project in Dokploy**
1. In Dokploy UI, create a new project:
- Name: `ai-stack-portal` (or any name you prefer)
- Description: "Shared project for all user AI stacks"
2. Note the **Project ID** (visible in URL or API response)
- Example: `2y2Glhz5Wy0dBNf6BOR_-`
3. Get the **Environment ID**:
```bash
curl -s "http://10.100.0.20:3000/api/project.one?projectId=2y2Glhz5Wy0dBNf6BOR_-" \
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | jq -r '.environments[0].id'
```
- Example: `RqE9OFMdLwkzN7pif1xN8`
**Step 2: Configure Project-Level Variables**
In the shared project (`ai-stack-portal`), add these **project-level environment variables**:
| Variable Name | Value | Purpose |
|---------------|-------|---------|
| `SHARED_PROJECT_ID` | `2y2Glhz5Wy0dBNf6BOR_-` | The project where user stacks deploy |
| `SHARED_ENVIRONMENT_ID` | `RqE9OFMdLwkzN7pif1xN8` | The environment within that project |
**Step 3: Reference Variables in Portal Applications**
The portal's docker-compose files use Dokploy's variable syntax to reference these:
```yaml
environment:
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
```
**This syntax `${{project.VARIABLE}}` tells Dokploy**: "Get this value from the project-level environment variables"
### Important Notes
- ⚠️ **Both variables MUST be set** in the shared project for deployment to work
- ⚠️ If not set, portal will fall back to creating separate projects per user (legacy behavior)
- ✅ You can have different shared projects for dev/staging/prod environments
- ✅ All 3 portal deployments (dev/staging/prod) should point to their respective shared projects
---
## Setting Up Dokploy ## Setting Up Dokploy
### Step 1: Create Dev Application ### Step 1: Create Dev Application
@@ -107,12 +176,28 @@ main branch → git.app.flexinit.nl/oussamadouhou/ai-stack-deployer:latest
- Enable SSL (via Traefik wildcard cert) - Enable SSL (via Traefik wildcard cert)
3. **Set Environment Variables**: 3. **Set Environment Variables**:
**Important**: The portal application should be deployed **inside the shared project** (e.g., `ai-stack-portal-dev`).
Then set these **project-level variables** in that shared project:
```env
SHARED_PROJECT_ID=<your-shared-project-id>
SHARED_ENVIRONMENT_ID=<your-shared-environment-id>
```
And these **application-level variables** in the portal app:
```env ```env
DOKPLOY_URL=http://10.100.0.20:3000 DOKPLOY_URL=http://10.100.0.20:3000
DOKPLOY_API_TOKEN=<your-token> DOKPLOY_API_TOKEN=<your-token>
STACK_DOMAIN_SUFFIX=ai.flexinit.nl STACK_DOMAIN_SUFFIX=ai.flexinit.nl
STACK_IMAGE=git.app.flexinit.nl/flexinit/agent-stack:latest STACK_IMAGE=git.app.flexinit.nl/flexinit/agent-stack:latest
``` ```
The docker-compose file will automatically reference the project-level variables using:
```yaml
SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
```
4. **Configure Webhook**: 4. **Configure Webhook**:
- Event: **Push** - Event: **Push**

View File

@@ -0,0 +1,311 @@
# Shared Project Deployment Architecture
## Overview
The AI Stack Deployer portal deploys **all user AI stacks to a single shared Dokploy project** instead of creating a new project for each user.
---
## Architecture Diagram
```
┌─────────────────────────────────────────────────────────────────┐
│ Dokploy: ai-stack-portal (Shared Project) │
│ ID: 2y2Glhz5Wy0dBNf6BOR_- │
├─────────────────────────────────────────────────────────────────┤
│ │
│ 📦 Portal Application: ai-stack-deployer-prod │
│ ├─ Domain: portal.ai.flexinit.nl │
│ ├─ Image: git.app.flexinit.nl/.../ai-stack-deployer:latest│
│ └─ Env: SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}} │
│ │
│ ───────────────────────────────────────────────────────────── │
│ │
│ 📦 User Stack: john-dev │
│ ├─ Domain: john-dev.ai.flexinit.nl │
│ ├─ Image: git.app.flexinit.nl/.../agent-stack:latest │
│ └─ Deployed by: Portal │
│ │
│ 📦 User Stack: jane-prod │
│ ├─ Domain: jane-prod.ai.flexinit.nl │
│ ├─ Image: git.app.flexinit.nl/.../agent-stack:latest │
│ └─ Deployed by: Portal │
│ │
│ 📦 User Stack: alice-test │
│ ├─ Domain: alice-test.ai.flexinit.nl │
│ ├─ Image: git.app.flexinit.nl/.../agent-stack:latest │
│ └─ Deployed by: Portal │
│ │
└─────────────────────────────────────────────────────────────────┘
```
---
## How It Works
### Step 1: Portal Reads Configuration
When a user submits a stack name (e.g., "john-dev"), the portal:
1. **Reads environment variables**:
```javascript
const sharedProjectId = process.env.SHARED_PROJECT_ID;
const sharedEnvironmentId = process.env.SHARED_ENVIRONMENT_ID;
```
2. **These are set via Dokploy's project-level variables**:
```yaml
environment:
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
```
### Step 2: Portal Deploys to Shared Project
Instead of creating a new project, the portal:
```javascript
// OLD BEHAVIOR (legacy):
// createProject(`ai-stack-${username}`) ❌ Creates new project per user
// NEW BEHAVIOR (current):
// Uses existing shared project ID ✅
const projectId = sharedProjectId; // From environment variable
const environmentId = sharedEnvironmentId;
// Creates application IN the shared project
createApplication({
projectId: projectId,
environmentId: environmentId,
name: `${username}-stack`,
image: 'git.app.flexinit.nl/.../agent-stack:latest',
domain: `${username}.ai.flexinit.nl`
});
```
### Step 3: User Accesses Their Stack
User visits `https://john-dev.ai.flexinit.nl` → Traefik routes to their application inside the shared project.
---
## Configuration Steps
### 1. Create Shared Project in Dokploy
1. In Dokploy UI, create project:
- **Name**: `ai-stack-portal`
- **Description**: "Shared project for all user AI stacks"
2. Get the **Project ID**:
```bash
# Via API
curl -s "http://10.100.0.20:3000/api/project.all" \
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
jq -r '.[] | select(.name=="ai-stack-portal") | .id'
# Output: 2y2Glhz5Wy0dBNf6BOR_-
```
3. Get the **Environment ID**:
```bash
curl -s "http://10.100.0.20:3000/api/project.one?projectId=2y2Glhz5Wy0dBNf6BOR_-" \
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
jq -r '.environments[0].id'
# Output: RqE9OFMdLwkzN7pif1xN8
```
### 2. Set Project-Level Variables
In the shared project (`ai-stack-portal`), add these **project-level environment variables**:
| Variable | Value | Example |
|----------|-------|---------|
| `SHARED_PROJECT_ID` | Your project ID | `2y2Glhz5Wy0dBNf6BOR_-` |
| `SHARED_ENVIRONMENT_ID` | Your environment ID | `RqE9OFMdLwkzN7pif1xN8` |
**How to set in Dokploy UI**:
- Go to Project → Settings → Environment Variables
- Add variables at **project level** (not application level)
### 3. Deploy Portal Application
Deploy the portal **inside the same shared project**:
1. **Application Details**:
- Name: `ai-stack-deployer-prod`
- Type: Docker Compose
- Compose File: `docker-compose.prod.yml`
- Branch: `main`
2. **The docker-compose file automatically references project variables**:
```yaml
environment:
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}} # ← Magic happens here
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
```
3. **Dokploy resolves `${{project.VAR}}`** to the actual value from project-level variables.
---
## Benefits
### ✅ Centralized Management
All user stacks in one place:
- Easy to list all active stacks
- Shared monitoring dashboard
- Centralized logging
### ✅ Resource Efficiency
- No overhead of separate projects per user
- Shared network and resources
- Easier to manage quotas
### ✅ Simplified Configuration
- Project-level environment variables shared by all stacks
- Single source of truth for common configs
- Easy to update STACK_IMAGE for all users
### ✅ Better Organization
```
Projects in Dokploy:
├── ai-stack-portal (500 user applications) ✅ Clean
└── NOT:
├── ai-stack-john
├── ai-stack-jane
├── ai-stack-alice
└── ... (500 separate projects) ❌ Messy
```
---
## Fallback Behavior
If `SHARED_PROJECT_ID` and `SHARED_ENVIRONMENT_ID` are **not set**, the portal falls back to **legacy behavior**:
```javascript
// Code in src/orchestrator/production-deployer.ts (lines 187-196)
const sharedProjectId = config.sharedProjectId || process.env.SHARED_PROJECT_ID;
const sharedEnvironmentId = config.sharedEnvironmentId || process.env.SHARED_ENVIRONMENT_ID;
if (sharedProjectId && sharedEnvironmentId) {
// Use shared project ✅
state.resources.projectId = sharedProjectId;
state.resources.environmentId = sharedEnvironmentId;
return;
}
// Fallback: Create separate project per user ⚠️
const projectName = `ai-stack-${config.stackName}`;
const existingProject = await this.client.findProjectByName(projectName);
// ...
```
**This ensures backwards compatibility** but is not recommended.
---
## Troubleshooting
### Portal Creates Separate Projects Instead of Using Shared Project
**Cause**: `SHARED_PROJECT_ID` or `SHARED_ENVIRONMENT_ID` not set.
**Solution**:
1. Check project-level variables in Dokploy:
```bash
curl -s "http://10.100.0.20:3000/api/project.one?projectId=YOUR_PROJECT_ID" \
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
jq '.environmentVariables'
```
2. Ensure the portal application's docker-compose references them:
```yaml
environment:
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
```
3. Redeploy the portal application.
### Variable Reference Not Working
**Symptom**: Portal logs show `undefined` for `SHARED_PROJECT_ID`.
**Cause**: Using wrong syntax.
**Correct syntax**:
```yaml
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}} ✅
```
**Wrong syntax**:
```yaml
- SHARED_PROJECT_ID=${SHARED_PROJECT_ID} ❌ (shell substitution, not Dokploy)
- SHARED_PROJECT_ID={{project.SHARED_PROJECT_ID}} ❌ (missing $)
```
### How to Verify Configuration
Check portal container environment:
```bash
# SSH into Dokploy host
ssh user@10.100.0.20
# Inspect portal container
docker exec ai-stack-deployer env | grep SHARED
# Should show:
SHARED_PROJECT_ID=2y2Glhz5Wy0dBNf6BOR_-
SHARED_ENVIRONMENT_ID=RqE9OFMdLwkzN7pif1xN8
```
---
## Environment-Specific Shared Projects
You can have **separate shared projects for dev/staging/prod**:
| Portal Environment | Shared Project | Purpose |
|--------------------|----------------|---------|
| Dev | `ai-stack-portal-dev` | Development user stacks |
| Staging | `ai-stack-portal-staging` | Staging user stacks |
| Prod | `ai-stack-portal` | Production user stacks |
Each portal deployment references its own shared project:
- `portal-dev.ai.flexinit.nl` → `ai-stack-portal-dev`
- `portal-staging.ai.flexinit.nl` → `ai-stack-portal-staging`
- `portal.ai.flexinit.nl` → `ai-stack-portal`
---
## Migration from Legacy
If you're currently using the legacy behavior (separate projects per user):
### Option 1: Gradual Migration
- New deployments use shared project
- Old deployments remain in separate projects
- Migrate old stacks manually over time
### Option 2: Full Migration
1. Create shared project
2. Set project-level variables
3. Redeploy all user stacks to shared project
4. Delete old separate projects
**Note**: Migration requires downtime for each stack being moved.
---
## Reference
- **Environment Variable Syntax**: See Dokploy docs on project-level variables
- **Code Location**: `src/orchestrator/production-deployer.ts` (lines 178-200)
- **Example IDs**: `.env.example` (lines 25-27)
---
**Questions?** Check the main deployment guide: `DOKPLOY_DEPLOYMENT.md`