Skip to main content

​
Overview

The Xyne Portable Deployment system provides a one-click deployment solution that can be easily transferred and deployed on any system. This advanced deployment method separates infrastructure and application components for efficient updates and maintenance.
This deployment method is ideal for production environments, development teams, and situations where you need to frequently update the application without affecting the database or search infrastructure.

​
Software Requirements

​
Minimum System Requirements

  • OS: Linux (Ubuntu 20.04+ recommended), macOS, or Windows with WSL2
  • RAM: 8GB minimum, 16GB+ recommended
  • Storage: 50GB+ available disk space
  • CPU: 4+ cores recommended

​
Required Software

Docker Engine

Version: 20.10+Installation: Docker Official GuideEnsure Docker daemon is running and accessible without sudo (add user to docker group)

Docker Compose

Version: 2.0+Installation: Usually included with Docker Desktop or install separately via Compose docsVerify with: docker-compose --version

​
GPU Support (Optional)

For optimal search performance with GPU-accelerated embeddings:

NVIDIA Container Toolkit

Required for: GPU-accelerated Vespa search engineInstallation:
# Ubuntu/Debian
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo systemctl restart docker
Verify: docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

​
Quick Start

​
Method 1: Import Pre-built Package

If you received a pre-built Xyne package:
1

Extract Package

# Extract the portable package
tar -xzf xyne-portable-YYYYMMDD_HHMMSS.tar.gz
cd xyne-portable-YYYYMMDD_HHMMSS/
2

Import Docker Images

# Import pre-built images and pull dependencies
./import.sh
This will:
  • Load the main Xyne application image
  • Load the GPU-enabled Vespa image
  • Pull supporting images (PostgreSQL, Prometheus, Grafana, etc.)
3

Configure Environment

# Copy and customize environment settings
cp .env.example .env
nano .env  # Optional: customize settings
4

Deploy

# Start all services
./deploy.sh start
5

Access Application

Navigate to http://localhost:3000 to access XyneAdditional Services:
  • Grafana Dashboard: http://localhost:3002
  • Prometheus Metrics: http://localhost:9090
  • Loki Logs: http://localhost:3100

​
Method 2: Build from Source

If deploying directly from the Xyne repository:
1

Clone Repository

git clone https://github.com/xynehq/xyne.git
cd xyne/deployment/portable/
2

Build and Export

# Build images and create portable package
./quick-export.sh

# Or build without export for same-machine deployment
./quick-export.sh --no-export
3

Deploy

# Start all services
./deploy.sh start

​
Deployment Architecture

The portable deployment uses a modular architecture with separated concerns:

Split Compose Structure

deployment/portable/
├── docker-compose.yml               # Base configuration
├── docker-compose.infrastructure.yml # Database, Vespa, Monitoring  
├── docker-compose.app.yml          # Xyne application only
└── deploy.sh                       # Deployment orchestration
Benefits:
  • Fast app updates: Update application without touching database/search
  • Selective control: Manage infrastructure and application independently
  • Zero-downtime: Infrastructure remains running during app updates

​
Service Components

Infrastructure Services

Always-On Components:
  • PostgreSQL Database
  • Vespa Search Engine (GPU-enabled)
  • Prometheus (Metrics)
  • Grafana (Dashboards)
  • Loki + Promtail (Logging)
Update Frequency: Rare (stable foundation)

Application Service

Frequently Updated:
  • Xyne Node.js Application
  • Frontend React Build
  • API Endpoints
  • Business Logic
Update Frequency: Regular (feature releases)

​
Deployment Commands

The deploy.sh script provides comprehensive deployment management:

​
Basic Operations

./deploy.sh start
# Starts infrastructure first, then application
# Includes health checks and dependency waiting

​
Efficient Updates

./deploy.sh update-app
# ⚡ Fast: ~30 seconds
# Updates only the Xyne application
# Database and Vespa remain running
# Zero downtime for search and data

​
Database Management

./deploy.sh db-generate
# Create migration files after schema changes
# Migrations saved to ./data/app-migrations/
# Persists across container updates

​
Monitoring & Debugging

./deploy.sh logs
# Real-time logs from all services
# Press Ctrl+C to exit

​
Environment Configuration

​
Essential Environment Variables

Create or modify the .env file in your deployment directory: 
# Database Configuration
DATABASE_HOST=xyne-db
DATABASE_PORT=5432
DATABASE_USER=xyne
DATABASE_PASSWORD=xyne
DATABASE_NAME=xyne

# Application Settings
NODE_ENV=production
HOST=http://localhost
PORT=3000

# Search Engine
VESPA_HOST=vespa
VESPA_PORT=8080
EMBEDDING_MODEL=bge-small-en-v1.5

​
Advanced Configuration

# Embedding Model Options
EMBEDDING_MODEL=bge-small-en-v1.5    # Default: 384 dimensions, fast
EMBEDDING_MODEL=bge-base-en-v1.5     # 768 dimensions, balanced  
EMBEDDING_MODEL=bge-large-en-v1.5    # 1024 dimensions, best quality

# Vespa JVM Settings (adjust based on available RAM)
VESPA_CONFIGSERVER_JVMARGS="-Xms1g -Xmx16g -XX:+UseG1GC"
VESPA_CONFIGPROXY_JVMARGS="-Xms512m -Xmx8g -XX:+UseG1GC"

# Prometheus Retention
PROMETHEUS_RETENTION_TIME=15d
PROMETHEUS_RETENTION_SIZE=10GB

# Grafana Settings
GF_SECURITY_ADMIN_PASSWORD=your_grafana_password
GF_INSTALL_PLUGINS=grafana-clock-panel,grafana-simple-json-datasource

# Log Levels
VESPA_LOG_LEVEL=info
NODE_LOG_LEVEL=info

​
Data Persistence

All application data is stored in the ./data/ directory:

Data Directory Structure

data/
├── postgres-data/      # Database files
├── vespa-data/        # Search indices and models  
├── app-uploads/       # User uploaded files
├── app-logs/         # Application logs
├── app-assets/       # Static assets
├── app-migrations/    # Drizzle ORM migration files (persisted)
├── grafana-storage/  # Grafana dashboards
├── loki-data/       # Log storage
├── promtail-data/   # Log collector data
└── prometheus-data/ # Metrics storage
Backup Strategy:
  • Regular backup of postgres-data/ for database
  • Backup app-migrations/ for schema history
  • Backup vespa-data/ for search indices (can be rebuilt)
  • Include app-uploads/ for user content
Migration Persistence:
  • Drizzle migrations are stored in app-migrations/ and persist across updates
  • Generated migrations remain available after container rebuilds
  • Schema changes are tracked and versioned automatically

​
Performance Tuning

​
Resource Allocation

Memory Optimization

Small Deployment (8GB RAM):
# Reduce Vespa memory allocation
VESPA_CONFIGSERVER_JVMARGS="-Xms512m -Xmx4g"
VESPA_CONFIGPROXY_JVMARGS="-Xms256m -Xmx2g"
Large Deployment (32GB+ RAM):
# Increase for better performance
VESPA_CONFIGSERVER_JVMARGS="-Xms2g -Xmx24g"
VESPA_CONFIGPROXY_JVMARGS="-Xms1g -Xmx12g"

Storage Optimization

SSD Recommended for:
  • PostgreSQL data (postgres-data/)
  • Vespa indices (vespa-data/)
HDD Acceptable for:
  • Logs (*-logs/, loki-data/)
  • Backups and archives
Cleanup Automation:
# Automated via built-in log rotation
# Vespa cleanup runs daily automatically

​
Network Configuration

# Modify docker-compose.app.yml to change application port
ports:
  - "8080:80"  # Change first number for external access

# Modify infrastructure compose for monitoring ports
# Grafana: "3002:3000"
# Prometheus: "9090:9090" 
# Loki: "3100:3100"

​
Troubleshooting

​
Common Issues

Symptoms: Container startup failures, file permission errorsSolution:
# Fix data directory permissions
sudo chown -R 1000:1000 ./data/

# Or use the automated permission setup
./deploy.sh start  # Handles permissions automatically
Symptoms: Vespa falls back to CPU-only modeDiagnosis:
# Check GPU availability
docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

# Check Docker GPU runtime
docker info | grep nvidia
Solution: Install NVIDIA Container Toolkit (see requirements section)
Symptoms: App container keeps restartingDiagnosis:
# Check app logs
./deploy.sh logs app

# Check service dependencies
./deploy.sh status
Common Fixes:
  • Ensure database is healthy before app starts
  • Verify environment variables in .env
  • Check available disk space and memory
Symptoms: Services stop, write errors in logsSolution:
# Quick cleanup
./deploy.sh cleanup

# Manual cleanup
docker system prune -a -f
docker volume prune -f

# Check large files
du -sh ./data/*

​
Health Checks

Monitor system health using built-in endpoints: 
curl http://localhost:3000/api/v1/me
# Should return user/auth information

​
Migration Guide

​
From Simple Docker Compose

If migrating from the basic quickstart deployment:
1

Backup Current Data

# Stop current deployment
docker-compose down

# Backup data (adjust paths as needed)
cp -r ./server/xyne-data ./xyne-data-backup
cp -r ./server/vespa-data ./vespa-data-backup
2

Deploy Portable System

cd deployment/portable/
./deploy.sh start
3

Migrate Data

# Stop new deployment
./deploy.sh stop

# Copy old data to new structure
cp -r ../../xyne-data-backup/* ./data/postgres-data/
cp -r ../../vespa-data-backup/* ./data/vespa-data/

# Restart with migrated data
./deploy.sh start

​
To Production Environment

Always test the portable deployment in a staging environment before production use.
1

Production Environment Setup

  • Ensure adequate resources (see requirements)
  • Configure firewall for required ports
  • Set up SSL/TLS termination (reverse proxy)
  • Configure backup procedures
2

Security Hardening

  • Change default passwords in .env
  • Restrict network access to management ports
  • Configure log rotation and monitoring
  • Set up automated security updates
3

Monitoring Setup

  • Configure Grafana alerts
  • Set up log aggregation
  • Monitor disk space and performance
  • Test backup and recovery procedures

​
Support

Documentation

Complete setup guides and API documentation

Community Support

Join our Slack community for help and discussions

GitHub Issues

Report bugs and request features

Direct Support

Contact the team for enterprise support
⌘I