Custom AI Stack Deployment

Powered by Prayog.io - Your Ultimate AI Development Platform

This comprehensive guide provides detailed step-by-step instructions for custom deployment and configuration of your complete AI development environment.

Architecture Overview

                           AI STACK ARCHITECTURE

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   OpenWebUI     │    │    Grafana      │    │    Langfuse     │
│   (Port 3000)   │    │   (Port 4000)   │    │   (Port 3001)   │
│  AI Interface   │    │  Infrastructure │    │ LLM Analytics   │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         │              ┌─────────────────┐              │
         │              │      N8N        │              │
         │              │   (Port 5678)   │              │
         │              │ Workflow Engine │              │
         │              └─────────────────┘              │
         │                       │                       │
         └───────────────────────┼───────────────────────┘
                                 │
                    ┌─────────────────────────┐
                    │      PostgreSQL         │
                    │     (Port 5433)         │
                    │   Shared Database       │
                    └─────────────────────────┘
                                 │
         ┌───────────────────────┼───────────────────────┐
         │                       │                       │
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Pipelines     │    │     Qdrant      │    │  OpenTelemetry  │
│   (Port 9099)   │    │   (Port 6333)   │    │    (OTLP)       │
│ AI Processing   │    │ Vector Database │    │     Traces      │
└─────────────────┘    └─────────────────┘    └─────────────────┘

Prerequisites

Before you begin, ensure you have the following set up:

1. Docker and Docker Compose installed - Download from the official Docker website
2. Git installed - For cloning the repository
3. 8GB+ RAM recommended - For optimal performance of all services
4. Available ports - Ensure ports 3000, 3001, 4000, 5433, 5678, 6333, and 9099 are available

Step 1: Clone and Setup the Repository

First, clone the AI stack repository to your local machine:

git clone https://github.com/prayog-io/prayog-ai-stack.git
cd prayog-ai-stack

Step 2: Configure Your Environment

Copy the example environment file and customize it for your needs:

# Copy the environment template
cp env.example .env

# Edit the configuration (optional)
nano .env

Key Configuration Options

AI Configuration

# For real OpenAI models (optional)
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_API_BASE_URL=https://api.openai.com/v1

# Pipeline API (for custom processing)
PIPELINES_API_BASE_URL=http://host.docker.internal:9099
PIPELINES_API_KEY=0p3n-w3bu!

Monitoring Configuration

# Grafana
GRAFANA_PORT=4000
GRAFANA_ADMIN_PASSWORD=admin123

# Langfuse
LANGFUSE_PORT=3001
LANGFUSE_SECRET=change-in-production

Database Configuration

POSTGRES_USER=admin
POSTGRES_PASSWORD=adminpass
POSTGRES_PORT=5433

Step 3: Start Your AI Stack

Option A: Quick Start (Recommended)

./quick-start.sh

Option B: Manual Startup

# Start all services
docker-compose up -d

# Check service status
docker ps

Option C: Step-by-Step Startup

# 1. Configure domains
./configure-domains.sh

# 2. Start pipeline service
./start-pipeline-service.sh

# 3. Start main services
./start-openwebui.sh

Step 4: Verify Your Installation

Check that all services are running properly:

# Check service status
./status.sh

# Test main endpoints
curl http://localhost:3000  # OpenWebUI
curl http://localhost:4000  # Grafana
curl http://localhost:3001  # Langfuse
curl http://localhost:9099  # Pipelines

Step 5: Access Your Services

Once everything is running, access your services:

Service	URL	Purpose	Default Login
OpenWebUI	http://localhost:3000	AI Chat Interface	Sign up on first visit
Grafana	http://localhost:4000	Infrastructure Monitoring	admin / admin123
Langfuse	http://localhost:3001	LLM Observability	Create account on first visit
Pipelines	http://localhost:9099	AI Pipeline API	API Key: 0p3n-w3bu!
N8N	http://localhost:5678	Workflow Automation	Setup on first visit
Qdrant	http://localhost:6333	Vector Database API	API Key: difyai123456

Configuring AI Models

Using OpenAI Models

To connect to OpenAI’s models, update your .env file:

OPENAI_API_KEY=your_openai_api_key_here
OPENAI_API_BASE_URL=https://api.openai.com/v1

Using Local Models with Ollama

If you want to use local models, install Ollama:

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama3.1:8b

# Start Ollama server
ollama serve

Then update your OpenWebUI configuration to point to the local Ollama instance.

Setting Up Monitoring and Observability

Your AI stack includes comprehensive monitoring with Grafana and Langfuse:

Grafana Monitoring Setup

1. Access Grafana: Open http://localhost:4000
2. Login: Use admin / admin123
3. View Dashboards: Pre-configured dashboards for:
   • Infrastructure metrics (CPU, memory, disk)
   • Application metrics (N8N workflows, PostgreSQL performance)
   • OpenTelemetry traces (request tracing across services)

Langfuse Analytics Setup

1. Access Langfuse: Open http://localhost:3001
2. Create Account: Sign up on first visit
3. Get API Keys: Navigate to Settings → API Keys
4. Configure OpenWebUI Integration:
   • Go to OpenWebUI Settings
   • Add Langfuse API keys
   • Enable conversation tracking

Pipeline Configuration

Your AI stack includes powerful pipeline processing capabilities:

Available Pipelines

N8N Integration Pipeline

Connects OpenWebUI with N8N workflows for external processing:

{
  "n8n_url": "http://localhost:5678/webhook-test/your-webhook-id",
  "n8n_bearer_token": "your-bearer-token",
  "input_field": "chatInput",
  "response_field": "output",
  "emit_interval": 2.0,
  "enable_status_indicator": true
}

Langfuse Monitoring Pipeline

Provides comprehensive LLM monitoring and analytics:

{
  "langfuse_secret_key": "your-secret-key",
  "langfuse_public_key": "your-public-key",
  "langfuse_host": "http://langfuse:3000/",
  "insert_tags": true,
  "use_model_name_instead_of_id": true,
  "debug": true
}

Testing Pipelines

cd pipelines
python3 test_pipelines.py

Management Commands

Your AI stack includes convenient management scripts:

Command	Description
./quick-start.sh	Start everything (one command)
./status.sh	Check service health
./logs.sh	View service logs
./stop.sh	Stop all services
./configure-domains.sh	Configure domains

Advanced Configuration

Production Deployment

For production deployment, update your configuration:

# Update domains
WEBUI_DOMAIN=your-app.com
LANGFUSE_DOMAIN=https://langfuse.your-app.com
N8N_DOMAIN=https://n8n.your-app.com

# Update security settings
WEBUI_SECRET_KEY=$(openssl rand -base64 32)
WEBUI_JWT_SECRET_KEY=$(openssl rand -base64 32)
NEXTAUTH_SECRET=$(openssl rand -base64 32)

Scaling Configuration

For high-load scenarios, adjust resource limits:

docker-compose.yml

services:
  openwebui:
    deploy:
      resources:
        limits:
          memory: 2G
          cpus: '1.0'

Custom Domain Configuration

Local Development

# .env file
WEBUI_DOMAIN=localhost
WEBUI_PORT=3000
LANGFUSE_DOMAIN=http://langfuse:3001
LANGFUSE_PORT=3001

Production with Custom Domain

# .env file
WEBUI_DOMAIN=your-app.com
WEBUI_PORT=3000
LANGFUSE_DOMAIN=https://langfuse.your-app.com
LANGFUSE_PORT=3001

SSL/TLS Configuration

For production deployments with SSL:

docker-compose.yml

services:
  openwebui:
    environment:
      - WEBUI_URL=https://your-app.com
      - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY}
    volumes:
      - ./ssl:/app/ssl:ro

Troubleshooting

Common Issues

Port Conflicts

# Check what's using the port
lsof -i :3000

# Change port in .env
WEBUI_PORT=3001

Service Connection Issues

# Restart all services
docker-compose down
docker-compose up -d

# Check service logs
docker-compose logs servicename

Pipeline Loading Issues

# Test pipelines
cd pipelines
python3 test_pipelines.py

# Check pipeline logs
docker logs pipeline-service

Docker Network Issues

# Recreate network
docker network prune
docker-compose down
docker-compose up -d

Database Connection Issues

# Check PostgreSQL logs
docker-compose logs postgres

# Test database connection
docker-compose exec postgres psql -U admin -d aistack

Debug Mode

Enable debug logging:

# .env file
WEBUI_LOG_LEVEL=DEBUG
LANGFUSE_LOG_LEVEL=DEBUG
N8N_LOG_LEVEL=debug

Performance Optimization

Memory Optimization

docker-compose.yml

services:
  openwebui:
    environment:
      - WEBUI_WORKERS=2
      - WEBUI_WORKER_CONNECTIONS=1000

Database Optimization

# PostgreSQL performance tuning
POSTGRES_SHARED_PRELOAD_LIBRARIES=pg_stat_statements
POSTGRES_MAX_CONNECTIONS=200
POSTGRES_SHARED_BUFFERS=256MB

Security Considerations

Change Default Secrets

Always update default credentials:

# Generate secure secrets
openssl rand -base64 32  # For various secret keys

Network Security

The stack uses isolated Docker networks for security:

networks:
  ai-stack:
    driver: bridge
    internal: false

API Key Management

Store API keys securely:

# Use Docker secrets for production
echo "your-api-key" | docker secret create openai_api_key -

Firewall Configuration

Configure firewall rules for production:

# Allow only necessary ports
ufw allow 3000  # OpenWebUI
ufw allow 4000  # Grafana
ufw allow 3001  # Langfuse
ufw deny 5433   # PostgreSQL (internal only)

Backup and Recovery

Database Backup

# Backup PostgreSQL
docker-compose exec postgres pg_dump -U admin aistack > backup.sql

# Restore PostgreSQL
docker-compose exec -T postgres psql -U admin aistack < backup.sql

Configuration Backup

# Backup configuration files
tar -czf ai-stack-config-$(date +%Y%m%d).tar.gz .env docker-compose.yml pipelines/

Volume Backup

# Backup Docker volumes
docker run --rm -v ai-stack_data:/data -v $(pwd):/backup alpine tar czf /backup/volumes-backup.tar.gz /data

API Reference

OpenWebUI API

• Base URL: http://localhost:3000
• Health Check: GET /health
• Authentication: Bearer token required

Langfuse API

• Base URL: http://localhost:3001
• Health Check: GET /api/public/health
• Authentication: API key required

Qdrant API

• Base URL: http://localhost:6333
• Collections: GET /collections
• Authentication: API key required

N8N API

• Base URL: http://localhost:5678
• Webhooks: POST /webhook-test/{webhook-id}
• Authentication: Bearer token required

Monitoring and Alerting

Grafana Alerts

Configure alerts for system health:

grafana/alerts.yml

groups:
  - name: ai-stack-alerts
    rules:
      - alert: HighMemoryUsage
        expr: (node_memory_used / node_memory_total) > 0.9
        for: 5m

Log Aggregation

Centralize logs using Docker logging drivers:

docker-compose.yml

services:
  openwebui:
    logging:
      driver: "json-file"
      options:
        max-size: "10m"
        max-file: "3"

That’s your complete custom deployment guide! You now have detailed instructions for every aspect of deploying and configuring your AI development environment.