docs refactor
All checks were successful
Datadog Software Composition Analysis / Datadog SBOM Generation and Upload (push) Successful in 52s
Datadog Secrets Scanning / Datadog Static Analyzer (push) Successful in 1m1s
Datadog Static Analysis / Datadog Static Analyzer (push) Successful in 5m50s

This commit is contained in:
2025-10-09 18:21:23 -04:00
parent c9a3a21f07
commit ad81d7f3db
39 changed files with 12551 additions and 1037 deletions

View File

@ -0,0 +1,810 @@
---
version: "1.0"
date: "2025-01-15"
author: "DWS Foldsite Team"
title: "Production Deployment"
description: "Deploying Foldsite for production use"
summary: "Complete guide to deploying Foldsite in production - from VPS servers to static hosting, with security, performance, and reliability best practices."
quick_tips:
- "Use Gunicorn with multiple workers for dynamic deployments"
- "Static export is fastest and cheapest for sites that don't change frequently"
- "Always use HTTPS in production with proper SSL certificates"
---
# Production Deployment
Deploy Foldsite to serve real traffic with reliability, security, and performance.
## Deployment Options
### Option 1: Dynamic Server (Python)
**Best for:**
- Frequently updated content
- Sites needing template helpers in real-time
- Admin file browser interface
- Dynamic content generation
**Characteristics:**
- Runs Python/Gunicorn server
- Content updates appear immediately
- Template helpers work dynamically
- Requires server with Python
**Hosting:** VPS, dedicated server, PaaS platforms
### Option 2: Static Export
**Best for:**
- Infrequently updated sites
- Maximum performance
- Minimal cost
- CDN delivery
**Characteristics:**
- Pre-rendered HTML files
- Blazing fast delivery
- Can host anywhere (GitHub Pages, Netlify, S3)
- Rebuild required for updates
**Hosting:** Static hosts, CDN, object storage
## Dynamic Server Deployment
### Prerequisites
- Linux server (Ubuntu 20.04+ recommended)
- Python 3.10+
- Domain name
- SSH access
### Step 1: Server Setup
**Update system:**
```bash
sudo apt update
sudo apt upgrade -y
```
**Install dependencies:**
```bash
# Python and pip
sudo apt install -y python3.10 python3-pip python3-venv
# Build tools
sudo apt install -y build-essential python3-dev
# Nginx
sudo apt install -y nginx
# Certbot for SSL
sudo apt install -y certbot python3-certbot-nginx
```
### Step 2: Deploy Foldsite
**Create user:**
```bash
sudo useradd -m -s /bin/bash foldsite
sudo su - foldsite
```
**Clone and setup:**
```bash
# Clone repository
git clone https://github.com/DWSresearch/foldsite.git
cd foldsite
# Create virtual environment
python3 -m venv venv
source venv/bin/activate
# Install dependencies
pip install -r requirements.txt
pip install gunicorn # Production WSGI server
```
**Setup your content:**
```bash
# Create site structure
mkdir -p ~/site/content
mkdir -p ~/site/templates
mkdir -p ~/site/styles
# Copy your content
# (upload via SCP, rsync, or git)
```
**Create production config:**
```bash
vim ~/foldsite/production-config.toml
```
```toml
[paths]
content_dir = "/home/foldsite/site/content"
templates_dir = "/home/foldsite/site/templates"
styles_dir = "/home/foldsite/site/styles"
[server]
listen_address = "127.0.0.1" # Only accept local connections
listen_port = 8081
admin_browser = false # Disable for security
max_threads = 8 # Adjust based on server
debug = false # Never enable in production
access_log = true
```
### Step 3: Systemd Service
Create service file:
```bash
sudo vim /etc/systemd/system/foldsite.service
```
```ini
[Unit]
Description=Foldsite Web Server
After=network.target
[Service]
Type=simple
User=foldsite
Group=foldsite
WorkingDirectory=/home/foldsite/foldsite
Environment="PATH=/home/foldsite/foldsite/venv/bin"
ExecStart=/home/foldsite/foldsite/venv/bin/gunicorn \
--workers 4 \
--bind 127.0.0.1:8081 \
--access-logfile /var/log/foldsite/access.log \
--error-logfile /var/log/foldsite/error.log \
--config /home/foldsite/foldsite/production-config.toml \
main:app
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
```
**Create log directory:**
```bash
sudo mkdir -p /var/log/foldsite
sudo chown foldsite:foldsite /var/log/foldsite
```
**Enable and start service:**
```bash
sudo systemctl daemon-reload
sudo systemctl enable foldsite
sudo systemctl start foldsite
# Check status
sudo systemctl status foldsite
```
### Step 4: Nginx Reverse Proxy
**Create Nginx config:**
```bash
sudo vim /etc/nginx/sites-available/foldsite
```
```nginx
upstream foldsite {
server 127.0.0.1:8081;
keepalive 32;
}
server {
listen 80;
server_name your-domain.com www.your-domain.com;
# Security headers
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
# Logs
access_log /var/log/nginx/foldsite-access.log;
error_log /var/log/nginx/foldsite-error.log;
# Max upload size
client_max_body_size 10M;
# Proxy to Foldsite
location / {
proxy_pass http://foldsite;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Cache static assets
location ~* \.(jpg|jpeg|png|gif|ico|css|js|svg|woff|woff2|ttf|eot)$ {
proxy_pass http://foldsite;
expires 30d;
add_header Cache-Control "public, immutable";
access_log off;
}
# Security: deny access to hidden files
location ~ /\. {
deny all;
access_log off;
log_not_found off;
}
}
```
**Enable site:**
```bash
sudo ln -s /etc/nginx/sites-available/foldsite /etc/nginx/sites-enabled/
sudo nginx -t # Test configuration
sudo systemctl reload nginx
```
### Step 5: SSL Certificate
**Get certificate with Certbot:**
```bash
sudo certbot --nginx -d your-domain.com -d www.your-domain.com
```
Follow prompts. Certbot will:
- Obtain certificate from Let's Encrypt
- Modify Nginx config for HTTPS
- Setup auto-renewal
**Verify auto-renewal:**
```bash
sudo certbot renew --dry-run
```
**Final Nginx config (with SSL):**
Certbot updates your config to include:
```nginx
server {
listen 443 ssl http2;
server_name your-domain.com www.your-domain.com;
ssl_certificate /etc/letsencrypt/live/your-domain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/your-domain.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
# ... rest of config
}
# Redirect HTTP to HTTPS
server {
listen 80;
server_name your-domain.com www.your-domain.com;
return 301 https://$server_name$request_uri;
}
```
### Step 6: Firewall
**Configure UFW:**
```bash
sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable
sudo ufw status
```
### Verification
Visit your domain:
```
https://your-domain.com
```
Should see your Foldsite with valid SSL!
## Static Export Deployment
### Generate Static Files
*Note: Static export functionality may need to be implemented or use a static site generator mode*
**Conceptual approach:**
```python
# export.py
import os
from pathlib import Path
from src.rendering.renderer import render_page
from src.config.config import Configuration
def export_static(config_path, output_dir):
"""Export all pages to static HTML"""
config = Configuration(config_path)
config.load_config()
output = Path(output_dir)
output.mkdir(parents=True, exist_ok=True)
# Walk content directory
for content_file in config.content_dir.rglob('*'):
if content_file.is_file() and not content_file.name.startswith('___'):
# Generate output path
rel_path = content_file.relative_to(config.content_dir)
out_path = output / rel_path.with_suffix('.html')
out_path.parent.mkdir(parents=True, exist_ok=True)
# Render and save
html = render_page(
content_file,
base_path=config.content_dir,
template_path=config.templates_dir,
style_path=config.styles_dir
)
with open(out_path, 'w') as f:
f.write(html)
# Copy styles
import shutil
shutil.copytree(config.styles_dir, output / 'styles')
if __name__ == '__main__':
import sys
export_static(sys.argv[1], sys.argv[2])
```
**Usage:**
```bash
python export.py config.toml ./dist
```
### Deploy Static Files
#### GitHub Pages
```bash
# Export to docs/
python export.py config.toml ./docs
# Commit and push
git add docs/
git commit -m "Update site"
git push
# Enable Pages in GitHub repo settings
# Source: docs/ folder
```
#### Netlify
```bash
# Export
python export.py config.toml ./dist
# Install Netlify CLI
npm install -g netlify-cli
# Deploy
netlify deploy --prod --dir=dist
```
**Or use continuous deployment:**
```yaml
# netlify.toml
[build]
command = "pip install -r requirements.txt && python export.py config.toml dist"
publish = "dist"
```
#### AWS S3 + CloudFront
```bash
# Export
python export.py config.toml ./dist
# Sync to S3
aws s3 sync ./dist s3://your-bucket-name --delete
# Invalidate CloudFront cache
aws cloudfront create-invalidation --distribution-id YOUR_DIST_ID --paths "/*"
```
#### Vercel
```bash
# Export
python export.py config.toml ./dist
# Deploy
vercel --prod ./dist
```
## Performance Optimization
### Nginx Caching
Add to Nginx config:
```nginx
# Define cache path
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=foldsite_cache:10m max_size=1g inactive=60m use_temp_path=off;
server {
# ...
location / {
proxy_cache foldsite_cache;
proxy_cache_valid 200 10m;
proxy_cache_valid 404 1m;
proxy_cache_bypass $cookie_session;
proxy_no_cache $cookie_session;
add_header X-Cache-Status $upstream_cache_status;
proxy_pass http://foldsite;
}
}
```
### Gzip Compression
```nginx
# In /etc/nginx/nginx.conf
gzip on;
gzip_vary on;
gzip_min_length 1024;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml+rss application/json;
```
### Image Optimization
Foldsite automatically generates thumbnails, but optimize source images:
```bash
# Install optimization tools
sudo apt install jpegoptim optipng
# Optimize JPEGs
find content/ -name "*.jpg" -exec jpegoptim --strip-all {} \;
# Optimize PNGs
find content/ -name "*.png" -exec optipng -o2 {} \;
```
### CDN Integration
Use CDN for static assets:
```nginx
# Separate static assets
location /styles/ {
alias /home/foldsite/site/styles/;
expires 1y;
add_header Cache-Control "public, immutable";
}
location /download/ {
# Proxy to Foldsite for thumbnails
proxy_pass http://foldsite;
expires 30d;
}
```
Then point DNS for `static.yourdomain.com` to CDN origin.
## Monitoring & Logging
### Application Logs
**View logs:**
```bash
# Systemd logs
sudo journalctl -u foldsite -f
# Application logs
tail -f /var/log/foldsite/error.log
tail -f /var/log/foldsite/access.log
```
### Nginx Logs
```bash
tail -f /var/log/nginx/foldsite-access.log
tail -f /var/log/nginx/foldsite-error.log
```
### Log Rotation
Create `/etc/logrotate.d/foldsite`:
```
/var/log/foldsite/*.log {
daily
missingok
rotate 14
compress
delaycompress
notifempty
create 0640 foldsite foldsite
sharedscripts
postrotate
systemctl reload foldsite
endscript
}
```
### Monitoring Tools
**Install basic monitoring:**
```bash
# Netdata (system monitoring)
bash <(curl -Ss https://my-netdata.io/kickstart.sh)
# Access at http://your-server:19999
```
**Check application health:**
```bash
#!/bin/bash
# health-check.sh
response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:8081/)
if [ $response -eq 200 ]; then
echo "Foldsite is healthy"
exit 0
else
echo "Foldsite is down (HTTP $response)"
systemctl restart foldsite
exit 1
fi
```
Run via cron:
```bash
*/5 * * * * /usr/local/bin/health-check.sh >> /var/log/foldsite/health.log 2>&1
```
## Backup Strategy
### Content Backup
```bash
#!/bin/bash
# backup.sh
BACKUP_DIR="/backups/foldsite"
DATE=$(date +%Y%m%d_%H%M%S)
mkdir -p $BACKUP_DIR
# Backup content, templates, styles
tar czf $BACKUP_DIR/site-$DATE.tar.gz \
/home/foldsite/site/
# Keep only last 30 days
find $BACKUP_DIR -name "site-*.tar.gz" -mtime +30 -delete
echo "Backup completed: site-$DATE.tar.gz"
```
**Run daily via cron:**
```bash
0 2 * * * /usr/local/bin/backup.sh
```
### Remote Backup
```bash
# Sync to remote server
rsync -avz /home/foldsite/site/ user@backup-server:/backups/foldsite/
# Or sync to S3
aws s3 sync /home/foldsite/site/ s3://your-backup-bucket/foldsite/
```
## Updating Foldsite
### Update Process
```bash
# As foldsite user
cd ~/foldsite
# Pull latest code
git pull
# Activate venv
source venv/bin/activate
# Update dependencies
pip install -r requirements.txt
# Restart service
sudo systemctl restart foldsite
# Check logs
sudo journalctl -u foldsite -n 50
```
### Zero-Downtime Updates
Use multiple Gunicorn workers and graceful reloading:
```bash
# Graceful reload (workers restart one by one)
sudo systemctl reload foldsite
# Or send HUP signal to Gunicorn
sudo pkill -HUP gunicorn
```
## Security Hardening
### Disable Directory Listing
Nginx automatically prevents this, but verify:
```nginx
autoindex off;
```
### Rate Limiting
Add to Nginx config:
```nginx
limit_req_zone $binary_remote_addr zone=foldsite_limit:10m rate=10r/s;
server {
location / {
limit_req zone=foldsite_limit burst=20 nodelay;
# ... rest of config
}
}
```
### Fail2ban
Protect against brute force:
```bash
sudo apt install fail2ban
# Create /etc/fail2ban/jail.local
[nginx-foldsite]
enabled = true
port = http,https
filter = nginx-foldsite
logpath = /var/log/nginx/foldsite-access.log
maxretry = 5
bantime = 3600
```
### Security Headers
Already in Nginx config, but verify:
```nginx
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src 'self' https:; script-src 'self' 'unsafe-inline' https:; style-src 'self' 'unsafe-inline' https:;" always;
```
### File Permissions
Ensure proper permissions:
```bash
# Content should be readable by foldsite user
chmod -R 755 ~/site/content
chmod -R 755 ~/site/templates
chmod -R 755 ~/site/styles
# Application should not be writable
chmod -R 555 ~/foldsite/src
```
## Troubleshooting
### Service Won't Start
```bash
# Check logs
sudo journalctl -u foldsite -xe
# Common issues:
# - Wrong Python path
# - Missing dependencies
# - Port already in use
# - Permission errors
```
### 502 Bad Gateway
Nginx can't reach Foldsite:
```bash
# Check if Foldsite is running
sudo systemctl status foldsite
# Check if port is listening
sudo netstat -tulpn | grep 8081
# Check Nginx error log
sudo tail /var/log/nginx/error.log
```
### High Memory Usage
```bash
# Check process memory
ps aux | grep gunicorn
# Reduce workers or add swap
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
```
### Slow Response Times
```bash
# Check Nginx access logs for slow requests
sudo tail -f /var/log/nginx/foldsite-access.log
# Enable query logging in Foldsite
# Check for expensive template helpers
# Consider caching with Redis/Memcached
```
## Platform-Specific Guides
### DigitalOcean
```bash
# Create droplet (Ubuntu 22.04)
# Follow server setup steps above
# Use DigitalOcean firewall for security
# Enable backups in control panel
```
### AWS EC2
```bash
# Launch Ubuntu instance
# Setup security groups (ports 22, 80, 443)
# Use Elastic IP for static IP
# Consider RDS for database if needed
```
### Hetzner Cloud
```bash
# Create CX11 or larger instance
# Follow server setup
# Use Hetzner firewall
# Consider Hetzner volumes for storage
```
## Next Steps
- **[Local Development](local-development.md)** - Development workflow
- **[Docker Deployment](docker.md)** - Container deployment
- **[Support](../support.md)** - Get help
Your Foldsite is now production-ready! Monitor it regularly, keep it updated, and enjoy your self-hosted corner of the internet.