Supervision - AI Ops Helpdesk

An AI-powered system log analysis and incident management tool for homelabs and small infrastructure teams.

Features

• Automatic Log Monitoring - Collects logs from Docker containers and systemd/journald via Loki/Promtail
• AI-Powered Analysis - Uses local Ollama LLM for intelligent log analysis, pattern detection, and recommendations
• Incident Ticketing - Automatically creates tickets from detected issues with deduplication
• Multi-Provider AI Chat - Chat with AI assistants to troubleshoot issues:
  • Ollama (local) - Fast, free, runs on your hardware
  • Claude Code (subscription) - Smarter analysis via Claude Code CLI
  • Gemini API (optional) - Google's AI as an alternative
• Configurable Technical Level - Adjust AI explanations from beginner-friendly to expert-level
• Usage Tracking - Monitor token usage and costs across providers
• Action Logging - Full audit trail of all troubleshooting steps taken
• Notifications - Alerts via ntfy for critical issues

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│ Docker/journald │────▶│    Promtail     │────▶│      Loki       │
└─────────────────┘     └─────────────────┘     └────────┬────────┘
                                                         │
                                                         ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Supervision (Python)                          │
│  ┌──────────┐  ┌────────────┐  ┌──────────┐  ┌──────────────┐   │
│  │ Fetcher  │  │Preprocessor│  │ Analyzer │  │   Notifier   │   │
│  │(Loki API)│─▶│(templatize)│─▶│ (Ollama) │─▶│   (ntfy)     │   │
│  └──────────┘  └────────────┘  └──────────┘  └──────────────┘   │
│                                                                  │
│  ┌──────────┐  ┌────────────┐  ┌──────────────────────────────┐ │
│  │Detector  │  │ Providers  │  │     Web UI (FastAPI)         │ │
│  │(tickets) │  │(Ollama/CLI)│  │  Dashboard/Incidents/Usage   │ │
│  └──────────┘  └────────────┘  └──────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘

Quick Start

Prerequisites

• Docker and Docker Compose
• Ollama running on your network (for local AI analysis)
• (Optional) Claude Code CLI installed for smarter AI chat
• (Optional) ntfy server for notifications

1. Clone and Configure

git clone https://git.chitekmedia.club/scott/supervisor.git
cd supervisor

# Edit configuration
cp config/supervision.yml config/supervision.yml.local
nano config/supervision.yml

2. Update Configuration

Edit config/supervision.yml:

# Point to your Ollama instance
providers:
  ollama:
    enabled: true
    host: "http://your-ollama-server:11434"
    model: "qwen3:14b"  # or llama3.2, mistral, etc.

  claude:
    enabled: true  # Enable Claude Code CLI

# Point to your ntfy server (optional)
notifications:
  ntfy:
    enabled: true
    server: "https://your-ntfy-server"
    topic: "supervision-alerts"

3. Start the Stack

docker compose up -d

This starts:

• Loki - Log storage (port 3100)
• Promtail - Log collector
• Supervision - Web UI (port 7860)

4. Access the Web UI

Open http://localhost:7860

• Dashboard - Overview of open incidents and stats
• Incidents - List and manage tickets
• LLM Usage - Monitor AI token usage and costs

Using Claude Code for AI Chat

Supervision supports Claude Code CLI for "tech support" chat. This uses your Claude Code subscription (flat rate with limits that reset every 5 hours) instead of per-token API billing.

Setup Claude Code

1. Install Claude Code CLI:

   # Follow instructions at https://claude.ai/code
   

2. Authenticate:

   claude auth
   

3. Enable in config:

   providers:
     claude:
       enabled: true
   

4. In the incident chat, select "Claude" from the provider dropdown

How It Works

• Ollama handles initial log analysis and ticket creation (fast, local)
• Claude Code is available for interactive troubleshooting chat (smarter, subscription-based)
• All conversations are logged in the incident timeline
• Token usage is tracked in the Usage dashboard

Configuration Reference

config/supervision.yml

# AI behavior settings
ai:
  # Technical level for AI explanations (1-5)
  # 1 = beginner (plain English, no jargon)
  # 3 = intermediate (standard IT terminology)
  # 5 = expert (dense technical output)
  technical_level: 3
  default_provider: "ollama"

# AI Providers
providers:
  ollama:
    enabled: true
    host: "http://localhost:11434"
    model: "qwen3:14b"
    timeout: 120

  claude:
    enabled: false  # Uses Claude Code CLI (subscription)

  gemini:
    enabled: false
    api_key_env: "GEMINI_API_KEY"  # Set env var
    model: "gemini-1.5-pro"
    cost_per_1k_input: 0.00125
    cost_per_1k_output: 0.005

# Loki log storage
loki:
  host: "http://loki:3100"

# Background analysis
analysis:
  interval_minutes: 15  # How often to scan logs
  lookback_minutes: 30  # How far back to analyze
  batch_size: 100       # Max logs per analysis

# Log sources to monitor
sources:
  - name: "docker"
    query: '{container=~".+"}'
    priority: high
  - name: "systemd"
    query: '{job="journal"}'
    priority: medium

# Notifications
notifications:
  ntfy:
    enabled: true
    server: "https://ntfy.sh"
    topic: "supervision-alerts"
    priority: "default"

# Alert keywords
alerts:
  critical_keywords:
    - "OOMKilled"
    - "fatal"
    - "panic"
  warning_keywords:
    - "error"
    - "failed"
  rate_limit_minutes: 5

# Web UI
web:
  host: "0.0.0.0"
  port: 7860

Technical Level Examples

Level	Setting	AI Response Style
1	beginner	"The container ran out of memory and was stopped. Think of it like a glass overflowing - we need to either use a bigger glass or pour less water."
3	intermediate	"Container was OOMKilled due to memory exhaustion. Check memory limits in docker-compose.yml and review the application's memory usage patterns."
5	expert	"OOMKilled (cgroup memory.max exceeded). Check /sys/fs/cgroup memory.current, review container memory limits, and profile heap allocation. Consider memory.high for soft limits."

Web UI Pages

Dashboard (/)

• Open incident count
• Critical incidents
• New/resolved today
• Recent incidents table
• Manual scan trigger

Incidents (/incidents)

• Filterable incident list
• Status, priority, source filters
• Click to view details

Incident Detail (/incidents/{id})

• AI summary and suggestions
• Chat with AI assistants
• Resolution steps tracking
• Alert log samples
• Status management

LLM Usage (/usage)

• Token usage today/this month
• Cost breakdown by provider
• Daily usage chart
• Top incidents by usage

API Endpoints

Endpoint	Method	Description
/api/stats	GET	Dashboard statistics
/api/incidents	GET	List incidents
/api/incidents/{id}	DELETE	Delete incident
/api/providers	GET	Available AI providers
/api/usage/summary	GET	Usage cost summary
/api/usage/daily	GET	Daily usage breakdown
/api/usage/providers	GET	Usage by provider

Development

Local Setup

# Create virtual environment
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install -e ".[dev]"

# Run locally (requires Loki and Ollama)
python -m supervision.web

Project Structure

src/supervision/
├── web.py              # FastAPI web application
├── detector.py         # Incident detection engine
├── analyzer.py         # Ollama LLM integration
├── fetcher.py          # Loki log queries
├── preprocessor.py     # Log templatization
├── notifier.py         # ntfy notifications
├── database.py         # SQLAlchemy models
├── config.py           # Configuration loading
├── usage_tracker.py    # LLM usage tracking
├── action_logger.py    # Resolution audit trail
├── providers/          # AI provider implementations
│   ├── base.py         # Base provider class
│   ├── ollama.py       # Ollama provider
│   ├── claude.py       # Claude Code CLI provider
│   └── gemini.py       # Gemini API provider
└── templates/          # Jinja2 HTML templates

Troubleshooting

Ollama not connecting

• Check that Ollama is running: curl http://your-ollama:11434/api/tags
• Verify the host in config matches your Ollama server
• Ensure the model is pulled: ollama pull qwen3:14b

Claude Code not working

• Verify CLI is installed: claude --version
• Check authentication: claude auth
• Ensure providers.claude.enabled: true in config

No incidents being created

• Check Loki is receiving logs: curl http://loki:3100/ready
• Verify Promtail is running and configured
• Check the log queries in config match your log labels
• Manually trigger a scan from the Dashboard

Database issues

• Database is stored at /app/data/supervision.db (in Docker)
• To reset: docker compose down -v and restart

License

MIT

Contributing

See ROADMAP.md for planned features and progress.