feat: add shared project deployment with Dokploy project-level variables
- Add SHARED_PROJECT_ID and SHARED_ENVIRONMENT_ID to all docker-compose files
- Use Dokploy's project-level variable syntax: ${{project.VARIABLE}}
- Deploy all user AI stacks to a single shared Dokploy project
- Update DOKPLOY_DEPLOYMENT.md with shared project configuration guide
- Add comprehensive SHARED_PROJECT_DEPLOYMENT.md architecture documentation
Benefits:
- Centralized management (all stacks in one project)
- Resource efficiency (no per-user project overhead)
- Simplified configuration (project-level shared vars)
- Better organization (500 apps in 1 project vs 500 projects)
How it works:
1. Portal reads SHARED_PROJECT_ID from environment
2. Docker-compose uses ${{project.SHARED_PROJECT_ID}} to reference project-level vars
3. Dokploy resolves these at runtime
4. Portal deploys user stacks as applications within the shared project
Fallback: If variables not set, falls back to legacy behavior (separate project per user)
This commit is contained in:
Oussama Douhou
@@ -15,6 +15,8 @@ services:
|
||||
- STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl}
|
||||
- STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest}
|
||||
- RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal}
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
env_file:
|
||||
- .env
|
||||
restart: unless-stopped
|
||||
|
||||
@@ -15,6 +15,8 @@ services:
|
||||
- STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl}
|
||||
- STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest}
|
||||
- RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal}
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
env_file:
|
||||
- .env
|
||||
restart: unless-stopped
|
||||
|
||||
@@ -15,6 +15,8 @@ services:
|
||||
- STACK_DOMAIN_SUFFIX=${STACK_DOMAIN_SUFFIX:-ai.flexinit.nl}
|
||||
- STACK_IMAGE=${STACK_IMAGE:-git.app.flexinit.nl/flexinit/agent-stack:latest}
|
||||
- RESERVED_NAMES=${RESERVED_NAMES:-admin,api,www,root,system,test,demo,portal}
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
env_file:
|
||||
- .env
|
||||
restart: unless-stopped
|
||||
|
||||
@@ -91,6 +91,75 @@ main branch → git.app.flexinit.nl/oussamadouhou/ai-stack-deployer:latest
|
||||
|
||||
---
|
||||
|
||||
## Shared Project Configuration (IMPORTANT)
|
||||
|
||||
### What is Shared Project Deployment?
|
||||
|
||||
The portal deploys **all user AI stacks as applications within a single shared Dokploy project**, instead of creating a new project for each user. This provides:
|
||||
|
||||
- ✅ Better organization (all stacks in one place)
|
||||
- ✅ Shared environment variables
|
||||
- ✅ Centralized monitoring
|
||||
- ✅ Easier management
|
||||
|
||||
### How It Works
|
||||
|
||||
```
|
||||
Dokploy Project: ai-stack-portal
|
||||
├── Environment: deployments
|
||||
│ ├── Application: john-dev
|
||||
│ ├── Application: jane-prod
|
||||
│ └── Application: alice-test
|
||||
```
|
||||
|
||||
### Setting Up the Shared Project
|
||||
|
||||
**Step 1: Create the Shared Project in Dokploy**
|
||||
|
||||
1. In Dokploy UI, create a new project:
|
||||
- Name: `ai-stack-portal` (or any name you prefer)
|
||||
- Description: "Shared project for all user AI stacks"
|
||||
|
||||
2. Note the **Project ID** (visible in URL or API response)
|
||||
- Example: `2y2Glhz5Wy0dBNf6BOR_-`
|
||||
|
||||
3. Get the **Environment ID**:
|
||||
```bash
|
||||
curl -s "http://10.100.0.20:3000/api/project.one?projectId=2y2Glhz5Wy0dBNf6BOR_-" \
|
||||
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | jq -r '.environments[0].id'
|
||||
```
|
||||
- Example: `RqE9OFMdLwkzN7pif1xN8`
|
||||
|
||||
**Step 2: Configure Project-Level Variables**
|
||||
|
||||
In the shared project (`ai-stack-portal`), add these **project-level environment variables**:
|
||||
|
||||
| Variable Name | Value | Purpose |
|
||||
|---------------|-------|---------|
|
||||
| `SHARED_PROJECT_ID` | `2y2Glhz5Wy0dBNf6BOR_-` | The project where user stacks deploy |
|
||||
| `SHARED_ENVIRONMENT_ID` | `RqE9OFMdLwkzN7pif1xN8` | The environment within that project |
|
||||
|
||||
**Step 3: Reference Variables in Portal Applications**
|
||||
|
||||
The portal's docker-compose files use Dokploy's variable syntax to reference these:
|
||||
|
||||
```yaml
|
||||
environment:
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
```
|
||||
|
||||
**This syntax `${{project.VARIABLE}}` tells Dokploy**: "Get this value from the project-level environment variables"
|
||||
|
||||
### Important Notes
|
||||
|
||||
- ⚠️ **Both variables MUST be set** in the shared project for deployment to work
|
||||
- ⚠️ If not set, portal will fall back to creating separate projects per user (legacy behavior)
|
||||
- ✅ You can have different shared projects for dev/staging/prod environments
|
||||
- ✅ All 3 portal deployments (dev/staging/prod) should point to their respective shared projects
|
||||
|
||||
---
|
||||
|
||||
## Setting Up Dokploy
|
||||
|
||||
### Step 1: Create Dev Application
|
||||
@@ -107,12 +176,28 @@ main branch → git.app.flexinit.nl/oussamadouhou/ai-stack-deployer:latest
|
||||
- Enable SSL (via Traefik wildcard cert)
|
||||
|
||||
3. **Set Environment Variables**:
|
||||
|
||||
**Important**: The portal application should be deployed **inside the shared project** (e.g., `ai-stack-portal-dev`).
|
||||
|
||||
Then set these **project-level variables** in that shared project:
|
||||
```env
|
||||
SHARED_PROJECT_ID=<your-shared-project-id>
|
||||
SHARED_ENVIRONMENT_ID=<your-shared-environment-id>
|
||||
```
|
||||
|
||||
And these **application-level variables** in the portal app:
|
||||
```env
|
||||
DOKPLOY_URL=http://10.100.0.20:3000
|
||||
DOKPLOY_API_TOKEN=<your-token>
|
||||
STACK_DOMAIN_SUFFIX=ai.flexinit.nl
|
||||
STACK_IMAGE=git.app.flexinit.nl/flexinit/agent-stack:latest
|
||||
```
|
||||
|
||||
The docker-compose file will automatically reference the project-level variables using:
|
||||
```yaml
|
||||
SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
```
|
||||
|
||||
4. **Configure Webhook**:
|
||||
- Event: **Push**
|
||||
|
||||
311
docs/SHARED_PROJECT_DEPLOYMENT.md
Normal file
311
docs/SHARED_PROJECT_DEPLOYMENT.md
Normal file
@@ -0,0 +1,311 @@
|
||||
# Shared Project Deployment Architecture
|
||||
|
||||
## Overview
|
||||
|
||||
The AI Stack Deployer portal deploys **all user AI stacks to a single shared Dokploy project** instead of creating a new project for each user.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ Dokploy: ai-stack-portal (Shared Project) │
|
||||
│ ID: 2y2Glhz5Wy0dBNf6BOR_- │
|
||||
├─────────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ 📦 Portal Application: ai-stack-deployer-prod │
|
||||
│ ├─ Domain: portal.ai.flexinit.nl │
|
||||
│ ├─ Image: git.app.flexinit.nl/.../ai-stack-deployer:latest│
|
||||
│ └─ Env: SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}} │
|
||||
│ │
|
||||
│ ───────────────────────────────────────────────────────────── │
|
||||
│ │
|
||||
│ 📦 User Stack: john-dev │
|
||||
│ ├─ Domain: john-dev.ai.flexinit.nl │
|
||||
│ ├─ Image: git.app.flexinit.nl/.../agent-stack:latest │
|
||||
│ └─ Deployed by: Portal │
|
||||
│ │
|
||||
│ 📦 User Stack: jane-prod │
|
||||
│ ├─ Domain: jane-prod.ai.flexinit.nl │
|
||||
│ ├─ Image: git.app.flexinit.nl/.../agent-stack:latest │
|
||||
│ └─ Deployed by: Portal │
|
||||
│ │
|
||||
│ 📦 User Stack: alice-test │
|
||||
│ ├─ Domain: alice-test.ai.flexinit.nl │
|
||||
│ ├─ Image: git.app.flexinit.nl/.../agent-stack:latest │
|
||||
│ └─ Deployed by: Portal │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## How It Works
|
||||
|
||||
### Step 1: Portal Reads Configuration
|
||||
|
||||
When a user submits a stack name (e.g., "john-dev"), the portal:
|
||||
|
||||
1. **Reads environment variables**:
|
||||
```javascript
|
||||
const sharedProjectId = process.env.SHARED_PROJECT_ID;
|
||||
const sharedEnvironmentId = process.env.SHARED_ENVIRONMENT_ID;
|
||||
```
|
||||
|
||||
2. **These are set via Dokploy's project-level variables**:
|
||||
```yaml
|
||||
environment:
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
```
|
||||
|
||||
### Step 2: Portal Deploys to Shared Project
|
||||
|
||||
Instead of creating a new project, the portal:
|
||||
|
||||
```javascript
|
||||
// OLD BEHAVIOR (legacy):
|
||||
// createProject(`ai-stack-${username}`) ❌ Creates new project per user
|
||||
|
||||
// NEW BEHAVIOR (current):
|
||||
// Uses existing shared project ID ✅
|
||||
const projectId = sharedProjectId; // From environment variable
|
||||
const environmentId = sharedEnvironmentId;
|
||||
|
||||
// Creates application IN the shared project
|
||||
createApplication({
|
||||
projectId: projectId,
|
||||
environmentId: environmentId,
|
||||
name: `${username}-stack`,
|
||||
image: 'git.app.flexinit.nl/.../agent-stack:latest',
|
||||
domain: `${username}.ai.flexinit.nl`
|
||||
});
|
||||
```
|
||||
|
||||
### Step 3: User Accesses Their Stack
|
||||
|
||||
User visits `https://john-dev.ai.flexinit.nl` → Traefik routes to their application inside the shared project.
|
||||
|
||||
---
|
||||
|
||||
## Configuration Steps
|
||||
|
||||
### 1. Create Shared Project in Dokploy
|
||||
|
||||
1. In Dokploy UI, create project:
|
||||
- **Name**: `ai-stack-portal`
|
||||
- **Description**: "Shared project for all user AI stacks"
|
||||
|
||||
2. Get the **Project ID**:
|
||||
```bash
|
||||
# Via API
|
||||
curl -s "http://10.100.0.20:3000/api/project.all" \
|
||||
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
|
||||
jq -r '.[] | select(.name=="ai-stack-portal") | .id'
|
||||
|
||||
# Output: 2y2Glhz5Wy0dBNf6BOR_-
|
||||
```
|
||||
|
||||
3. Get the **Environment ID**:
|
||||
```bash
|
||||
curl -s "http://10.100.0.20:3000/api/project.one?projectId=2y2Glhz5Wy0dBNf6BOR_-" \
|
||||
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
|
||||
jq -r '.environments[0].id'
|
||||
|
||||
# Output: RqE9OFMdLwkzN7pif1xN8
|
||||
```
|
||||
|
||||
### 2. Set Project-Level Variables
|
||||
|
||||
In the shared project (`ai-stack-portal`), add these **project-level environment variables**:
|
||||
|
||||
| Variable | Value | Example |
|
||||
|----------|-------|---------|
|
||||
| `SHARED_PROJECT_ID` | Your project ID | `2y2Glhz5Wy0dBNf6BOR_-` |
|
||||
| `SHARED_ENVIRONMENT_ID` | Your environment ID | `RqE9OFMdLwkzN7pif1xN8` |
|
||||
|
||||
**How to set in Dokploy UI**:
|
||||
- Go to Project → Settings → Environment Variables
|
||||
- Add variables at **project level** (not application level)
|
||||
|
||||
### 3. Deploy Portal Application
|
||||
|
||||
Deploy the portal **inside the same shared project**:
|
||||
|
||||
1. **Application Details**:
|
||||
- Name: `ai-stack-deployer-prod`
|
||||
- Type: Docker Compose
|
||||
- Compose File: `docker-compose.prod.yml`
|
||||
- Branch: `main`
|
||||
|
||||
2. **The docker-compose file automatically references project variables**:
|
||||
```yaml
|
||||
environment:
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}} # ← Magic happens here
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
```
|
||||
|
||||
3. **Dokploy resolves `${{project.VAR}}`** to the actual value from project-level variables.
|
||||
|
||||
---
|
||||
|
||||
## Benefits
|
||||
|
||||
### ✅ Centralized Management
|
||||
All user stacks in one place:
|
||||
- Easy to list all active stacks
|
||||
- Shared monitoring dashboard
|
||||
- Centralized logging
|
||||
|
||||
### ✅ Resource Efficiency
|
||||
- No overhead of separate projects per user
|
||||
- Shared network and resources
|
||||
- Easier to manage quotas
|
||||
|
||||
### ✅ Simplified Configuration
|
||||
- Project-level environment variables shared by all stacks
|
||||
- Single source of truth for common configs
|
||||
- Easy to update STACK_IMAGE for all users
|
||||
|
||||
### ✅ Better Organization
|
||||
```
|
||||
Projects in Dokploy:
|
||||
├── ai-stack-portal (500 user applications) ✅ Clean
|
||||
└── NOT:
|
||||
├── ai-stack-john
|
||||
├── ai-stack-jane
|
||||
├── ai-stack-alice
|
||||
└── ... (500 separate projects) ❌ Messy
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Fallback Behavior
|
||||
|
||||
If `SHARED_PROJECT_ID` and `SHARED_ENVIRONMENT_ID` are **not set**, the portal falls back to **legacy behavior**:
|
||||
|
||||
```javascript
|
||||
// Code in src/orchestrator/production-deployer.ts (lines 187-196)
|
||||
const sharedProjectId = config.sharedProjectId || process.env.SHARED_PROJECT_ID;
|
||||
const sharedEnvironmentId = config.sharedEnvironmentId || process.env.SHARED_ENVIRONMENT_ID;
|
||||
|
||||
if (sharedProjectId && sharedEnvironmentId) {
|
||||
// Use shared project ✅
|
||||
state.resources.projectId = sharedProjectId;
|
||||
state.resources.environmentId = sharedEnvironmentId;
|
||||
return;
|
||||
}
|
||||
|
||||
// Fallback: Create separate project per user ⚠️
|
||||
const projectName = `ai-stack-${config.stackName}`;
|
||||
const existingProject = await this.client.findProjectByName(projectName);
|
||||
// ...
|
||||
```
|
||||
|
||||
**This ensures backwards compatibility** but is not recommended.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Portal Creates Separate Projects Instead of Using Shared Project
|
||||
|
||||
**Cause**: `SHARED_PROJECT_ID` or `SHARED_ENVIRONMENT_ID` not set.
|
||||
|
||||
**Solution**:
|
||||
1. Check project-level variables in Dokploy:
|
||||
```bash
|
||||
curl -s "http://10.100.0.20:3000/api/project.one?projectId=YOUR_PROJECT_ID" \
|
||||
-H "Authorization: Bearer $DOKPLOY_API_TOKEN" | \
|
||||
jq '.environmentVariables'
|
||||
```
|
||||
|
||||
2. Ensure the portal application's docker-compose references them:
|
||||
```yaml
|
||||
environment:
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}}
|
||||
- SHARED_ENVIRONMENT_ID=${{project.SHARED_ENVIRONMENT_ID}}
|
||||
```
|
||||
|
||||
3. Redeploy the portal application.
|
||||
|
||||
### Variable Reference Not Working
|
||||
|
||||
**Symptom**: Portal logs show `undefined` for `SHARED_PROJECT_ID`.
|
||||
|
||||
**Cause**: Using wrong syntax.
|
||||
|
||||
**Correct syntax**:
|
||||
```yaml
|
||||
- SHARED_PROJECT_ID=${{project.SHARED_PROJECT_ID}} ✅
|
||||
```
|
||||
|
||||
**Wrong syntax**:
|
||||
```yaml
|
||||
- SHARED_PROJECT_ID=${SHARED_PROJECT_ID} ❌ (shell substitution, not Dokploy)
|
||||
- SHARED_PROJECT_ID={{project.SHARED_PROJECT_ID}} ❌ (missing $)
|
||||
```
|
||||
|
||||
### How to Verify Configuration
|
||||
|
||||
Check portal container environment:
|
||||
```bash
|
||||
# SSH into Dokploy host
|
||||
ssh user@10.100.0.20
|
||||
|
||||
# Inspect portal container
|
||||
docker exec ai-stack-deployer env | grep SHARED
|
||||
|
||||
# Should show:
|
||||
SHARED_PROJECT_ID=2y2Glhz5Wy0dBNf6BOR_-
|
||||
SHARED_ENVIRONMENT_ID=RqE9OFMdLwkzN7pif1xN8
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment-Specific Shared Projects
|
||||
|
||||
You can have **separate shared projects for dev/staging/prod**:
|
||||
|
||||
| Portal Environment | Shared Project | Purpose |
|
||||
|--------------------|----------------|---------|
|
||||
| Dev | `ai-stack-portal-dev` | Development user stacks |
|
||||
| Staging | `ai-stack-portal-staging` | Staging user stacks |
|
||||
| Prod | `ai-stack-portal` | Production user stacks |
|
||||
|
||||
Each portal deployment references its own shared project:
|
||||
- `portal-dev.ai.flexinit.nl` → `ai-stack-portal-dev`
|
||||
- `portal-staging.ai.flexinit.nl` → `ai-stack-portal-staging`
|
||||
- `portal.ai.flexinit.nl` → `ai-stack-portal`
|
||||
|
||||
---
|
||||
|
||||
## Migration from Legacy
|
||||
|
||||
If you're currently using the legacy behavior (separate projects per user):
|
||||
|
||||
### Option 1: Gradual Migration
|
||||
- New deployments use shared project
|
||||
- Old deployments remain in separate projects
|
||||
- Migrate old stacks manually over time
|
||||
|
||||
### Option 2: Full Migration
|
||||
1. Create shared project
|
||||
2. Set project-level variables
|
||||
3. Redeploy all user stacks to shared project
|
||||
4. Delete old separate projects
|
||||
|
||||
**Note**: Migration requires downtime for each stack being moved.
|
||||
|
||||
---
|
||||
|
||||
## Reference
|
||||
|
||||
- **Environment Variable Syntax**: See Dokploy docs on project-level variables
|
||||
- **Code Location**: `src/orchestrator/production-deployer.ts` (lines 178-200)
|
||||
- **Example IDs**: `.env.example` (lines 25-27)
|
||||
|
||||
---
|
||||
|
||||
**Questions?** Check the main deployment guide: `DOKPLOY_DEPLOYMENT.md`
|
||||
Reference in New Issue
Block a user