Documentation

1. Introduction

Why this platform exists

AI-assisted tools — chat assistants, coding agents, document retrieval — have become a significant productivity multiplier across the software industry. However, EUMETSAT's Directive on the use of Generative AI (DGD) and the Acceptable Use Policy (AUP) rightly restrict cloud-based GenAI services for categories of information where sending data to external providers is not permitted:

This creates a significant gap: staff cannot benefit from AI-assisted productivity when working on the most sensitive and critical materials — precisely the areas where AI assistance could deliver the most value. As identified in the DGD feedback process (Item 11): "The use case for running AI on premise for data that is too sensitive to run in the cloud is not covered."

The solution: locally hosted LLMs

The Cloud LLM platform addresses this gap by running large language models on EUMETSAT-owned hardware within the corporate network. No data leaves the EUMETSAT perimeter. There are no external AI providers, no third-party data processors, and no cloud jurisdiction involved. The models run as inference-only services — they are not trained or fine-tuned on EUMETSAT data, and context windows are cleared after each session.

This architectural approach means the platform qualifies as a "EUMETSAT-approved and issued information system" under AUP Section 5, enabling AI-assisted work on the full spectrum of information classifications:

How local hosting addresses DGD concerns

Architectural principles

• Inference only. Models are used for generation, not training. No fine-tuning or training is performed on EUMETSAT data.
• Open-weights models. All models use permissive open-weights licenses (Apache 2.0, MIT, or equivalent), permitting local deployment without commercial licensing dependencies.
• No external API calls. The LLM does not make any external API calls. Internet access for model inference is disabled by default.
• Authentication & authorisation. Access is limited to named, authorised users with valid EUMETSAT credentials.
• Full auditability. All requests are logged with user identity, model, token usage, and timestamps for accountability.
• No persistent data retention. Context windows are cleared after each interaction. No EUMETSAT data is persisted beyond the active session by the LLM.

Current status

This platform is currently a Proof of Concept (PoC). Access is restricted to named, authorised users and is not yet intended for wide-scale deployment. The PoC serves to validate the architecture, evaluate model quality for EUMETSAT use cases, and support the formal vetting process under DGD Section 4.

2. Requesting Access

How to get access

To request access to the EUMETSAT Cloud LLM platform, send an email to borys.saulyak@eumetsat.int with a short description of your intended use case. An administrator will create your account and provide onboarding instructions.

What you get

Once approved, you receive:

• A LibreChat account — the chat web UI for interactive use, available at https://llm.cloud.eumetsat.int/chat/.
• An API key — for programmatic access to the OpenAI-compatible LLM Gateway endpoint.

Registration

Account registration is restricted to the following email domains:

• @eumetsat.int (staff)
• @external.eumetsat.int (contractors and external collaborators)

Teams

Users are assigned to a team for budget tracking and usage attribution. When requesting access, you can either:

• Ask to join an existing team, or
• Request a new team to be created for your group

Each user can only belong to one team at a time. If your role changes, contact the administrator to switch teams.

LLM Gateway console & self-management

The LLM Gateway console is available to all users for self-service account management:

• Regenerate or rotate your API key
• Reset your password
• View your personal usage statistics and spend

Administrative features — model management, team budget tracking, virtual keys, and global usage analytics — require admin privileges. To request admin access, contact borys.saulyak@eumetsat.int with a justification.

3. Using LibreChat

LibreChat is the primary chat interface for the platform. To get started:

1. Navigate to https://llm.cloud.eumetsat.int/chat/
2. Register an account using your EUMETSAT email. Registration is limited to @eumetsat.int and @external.eumetsat.int domains.
3. Sign in with your EUMETSAT email and password.

Models available in the dropdown

Select a model from the dropdown in the prompt bar. The following aliases are available:

auto, medium, qwen-vl-medium, big, huge, ragflow-kb

Agents

LibreChat ships with four pre-configured agents that bundle a model, a set of MCP tools, and tuned settings. Pick the agent that best fits your task from the agent selector:

Image uploads

To upload an image and ask questions about it, select the My EUMETSAT Assistant agent (model qwen-vl-medium). LibreChat detects vision capability by checking the model name; only this agent enables the image upload button.

Memory

The assistant has a persistent key/value memory store — it remembers your preferences across conversations. You can toggle memory on or off per conversation from the conversation settings.

Web search

Built-in SearxNG + Firecrawl web search is pinned to the prompt bar by default. Use it to retrieve up-to-date information from the public web within your chat.

MCP Tools in chat

Several MCP (Model Context Protocol) tools are available directly in the chat. See Section 8 — MCP Tools for the full list. The tools include:

• web-fetch — fetch and read content from web URLs
• email — send emails to @eumetsat.int addresses
• code — execute Python or Bash code in isolated containers
• ragflow — search and upload documents to the knowledge base
• ocr — extract text from PDFs and images
• jira — read and write Jira issues and Confluence pages (OAuth 2.0)

File uploads

You can attach files to any conversation. Limits:

• Up to 1024 MB per file
• Up to 1024 MB total per conversation

4. API Access from Python

The platform exposes an OpenAI-compatible API via the LLM Gateway. You can use any OpenAI SDK client by pointing it at the EUMETSAT endpoint.

Endpoint & authentication

• Base URL: https://llm.cloud.eumetsat.int/models/v1
• API key: Get one from the LLM Gateway console at https://llm.cloud.eumetsat.int/models/ui/ (admin access required), or request one by email.
• Protocol: OpenAI-compatible — any OpenAI SDK client works out of the box.
• Model names: Use aliases (auto, medium, big, huge). Backend names are not accepted.

Example 1 — Basic chat completion (Python)

from openai import OpenAI

client = OpenAI(
    base_url="https://llm.cloud.eumetsat.int/models/v1",
    api_key="sk-your-api-key-here"
)

response = client.chat.completions.create(
    model="medium",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the MTG satellite mission?"}
    ]
)
print(response.choices[0].message.content)

Example 2 — Streaming response

stream = client.chat.completions.create(
    model="medium",
    messages=[{"role": "user", "content": "Explain radiometric calibration."}],
    stream=True
)
for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

Example 3 — curl

curl https://llm.cloud.eumetsat.int/models/v1/chat/completions \
  -H "Authorization: Bearer sk-your-api-key-here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "medium",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

5. VS Code & OpenCode

What is OpenCode

OpenCode is an AI coding assistant — similar to GitHub Copilot — that runs in the terminal and connects directly to the EUMETSAT LLM infrastructure. It gives you full control over which models are used for which tasks, and supports custom subagents with dedicated system prompts.

Configuration files

OpenCode reads its configuration from two locations:

• Global: ~/.config/opencode/opencode.jsonc — provider, models, primary agent, built-in agent overrides
• Project-level: .opencode/agent/*.md — custom subagent definitions (one Markdown file per subagent)

Provider configuration

The global config defines the LLM Gateway provider, available models, and the primary agent. It also overrides the built-in explore subagent to use the cheaper medium model instead of inheriting the primary agent's expensive model:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "litellm/big",
  "agent": {
    "build": {
      "model": "litellm/big",
      "permission": {
        "task": { "*": "allow", "general": "deny" }
      }
    },
    "explore": {
      "model": "litellm/medium"
    }
  },
  "provider": {
    "litellm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "EUMETSAT Cloud LLM",
      "options": {
        "baseURL": "https://llm.cloud.eumetsat.int/models/v1",
        "apiKey": "sk-your-api-key-here"
      },
      "models": {
        "medium": { "name": "medium", "limit": { "context": 131072, "output": 32768 } },
        "big":    { "name": "big",    "limit": { "context": 160000, "output": 32768 } },
        "huge":   { "name": "huge",   "limit": { "context": 200000, "output": 40000 } }
      }
    }
  }
}

Subagents: what they are and why they matter

A subagent is a specialized AI assistant that the main agent (running on big, $0.60/1M tokens) can delegate work to. Each subagent runs in a fresh context — large file reads it makes do not enter the main agent's context. Only the subagent's summary comes back. This has two benefits:

• Cost: read-only investigation delegated to research ($0.25/1M) is 2.4× cheaper than doing it on the main agent.
• Context hygiene: a 30K-token file read by a subagent costs 30K × $0.25/1M = $0.0075 and returns a ~2K summary. Doing it directly costs 30K × $0.60/1M = $0.018 and the 30K stays in context for every subsequent turn, compounding costs.

Subagent lineup

Four subagents are available for automatic delegation, plus one hidden agent for manual use:

Delegation policy: when to delegate vs. do directly

The main agent decides when to delegate based on subagent descriptions and the delegation policy in AGENTS.md. The policy is tuned to maximize savings without adding overhead for trivial tasks:

Why "Use PROACTIVELY" matters

OpenCode's Task tool has a built-in rule: if a subagent's description contains the phrase "use proactively" (or equivalent), the main agent will try to invoke it without the user asking. Without this trigger phrase, the main agent defaults to doing everything itself — which is exactly what was happening before.

The research and expert subagent descriptions now include this trigger. The research description also includes explicit boundary text ("Do NOT use for a single file read or a single grep") to avoid conflicting with OpenCode's hardcoded guidance that says to do simple reads directly.

Subagent configuration files

Each subagent is defined by a Markdown file in .opencode/agent/. The filename (minus .md) becomes the agent name. Here are the actual configurations:

---
description: >
  Read-only investigation agent on the medium model (2.4x cheaper than the main agent).
  Use PROACTIVELY for: multi-file investigations (3+ files), explaining how a feature
  works across modules, git history analysis (git log/diff across files), summarizing
  or comparing implementations, and any exploration that will read more than ~10K tokens
  of source. Returns a structured summary so the main context stays lean. Do NOT use for
  a single file read or a single grep — do those directly with Read/Grep.
mode: subagent
model: litellm/medium
permission:
  edit: deny
  bash:
    "git status": allow
    "git log*": allow
    "git diff*": allow
    "git show*": allow
    "git blame*": allow
    "ls*": allow
    "cat*": allow
    "grep*": allow
    "find*": allow
    "wc*": allow
    "tree*": allow
    "rg*": allow
    "*": ask
---

You are the Research subagent, powered by the medium model (Qwen3.8-27B, 131K context).

You are invoked for research and investigation tasks:
- Answering complex questions about the codebase
- Multi-step research across multiple files
- Git history analysis (log, diff, blame across files)
- Summarizing findings and producing reports
- Comparing approaches and trade-off analysis
- Any task that needs thorough exploration without making changes

You are read-only. You cannot edit files.
Be thorough and structured in your responses.
Return a concise summary — the main agent only needs your findings, not raw file contents.

---
description: >
  Expert agent on the largest model (200K context, deepest reasoning). Use PROACTIVELY
  when the task requires reasoning beyond the main model's capability, when processing
  documents over 130K tokens, or during multi-step debugging where the main model is
  stuck or going in circles. Expensive — reserve for genuinely hard problems the main
  agent cannot resolve.
mode: subagent
model: litellm/huge
permission:
  edit: allow
  bash: allow
---

You are the Expert subagent, powered by the largest available model (GLM-5.2-FP8, 200K context).

You are invoked when the main agent encounters tasks that require:
- Deep reasoning or complex analysis
- Long document processing (up to 200K tokens)
- Multi-step architectural decisions
- Debugging complex issues across multiple files
- Tasks that exceed the medium model's capability

You have full read/write/bash permissions. Be thorough and deliberate.
Always explain your reasoning before taking action.

---
description: Code agent for code generation, editing, refactoring, and implementation tasks. Uses the big model optimized for coding.
mode: subagent
model: litellm/big
permission:
  edit: allow
  bash:
    "npm*": allow
    "npx*": allow
    "python*": allow
    "pip*": allow
    "git add*": allow
    "git commit*": allow
    "*": ask
---

You are the Code subagent, powered by the big model (GLM-5.2-W4A16, 160K context).

You are invoked for coding tasks:
- Writing new code or components
- Editing existing files
- Refactoring code
- Running tests and linting
- Implementing features described by the main agent

Follow existing code conventions. Check neighboring files for patterns
before writing. Use existing libraries — never assume a library is available
without checking package.json or equivalent.

Do not add comments unless asked. Keep code clean and idiomatic.

---
description: Fast agent for quick lookups, simple file reads, git status, formatting, and other lightweight tasks. Uses the medium model for speed. Invoke manually with @fast for batch trivial lookups.
mode: subagent
model: litellm/medium
hidden: true
permission:
  edit: deny
  bash:
    "git status": allow
    "git log*": allow
    "git diff*": allow
    "ls*": allow
    "cat*": allow
    "*": ask
---

You are the Fast subagent, powered by the medium model (Qwen3.8-27B, 131K context).

You are invoked for quick, simple tasks:
- Git status, log, diff lookups
- File listing and reading
- Simple string searches
- Formatting outputs
- Any task that doesn't require deep reasoning

You are read-only. You cannot edit files or run complex commands.
Keep responses short and factual. Do not explain — just return the result.

Worked examples

Example 1: Investigating how authentication works across the codebase

The user asks: "How does the nginx auth_request proxy work in this project?"

This requires reading nginx.conf, docker-compose.yaml, and the dashboard's auth handler — 3+ files, likely >10K tokens. The main agent delegates to research:

Main agent (big, $0.60/1M)
  └─ delegates to research (medium, $0.25/1M)
       ├─ reads nginx.conf (8K tokens)
       ├─ reads docker-compose.yaml (5K tokens)
       ├─ reads live-dashboard.py auth section (3K tokens)
       ├─ runs git log --oneline nginx.conf
       └─ returns 2K summary to main agent

Cost: ~18K tokens × $0.25/1M = $0.0045 (research)
    + 2K summary × $0.60/1M = $0.0012 (main agent re-reads summary)
    = $0.0057 total

Without delegation: 18K × $0.60/1M = $0.0108, plus 18K stays
in context for every subsequent turn (10 turns × $0.0108 = $0.108).

Saving: 47% per turn, 95% over a 10-turn session.

Example 2: Single grep — do it directly

The user asks: "Where is the LLM Gateway URL defined?"

This is a single grep — the main agent does it directly:

Main agent (big, $0.60/1M)
  └─ runs grep "LITELLM_URL" → 3 matches in 1 file

Cost: ~500 tokens × $0.60/1M = $0.0003

Delegating to research would cost: task-description (~300 tokens
at $0.25/1M) + grep (~500 tokens at $0.25/1M) + summary (~200
tokens at $0.60/1M) = $0.00032 + overhead. Delegation now costs
slightly more than direct ($0.00002) AND adds latency — not worth it.

Example 3: Stuck on a complex bug — escalate to expert

The main agent has tried 3 approaches to fix a Docker networking issue and keeps going in circles. It escalates to expert:

Main agent (big, $0.60/1M)
  └─ delegates to expert (huge, $2.00/1M)
       ├─ reads docker-compose.yaml, nginx.conf, network configs
       ├─ analyzes 50K of logs and source
       └─ returns diagnosis + fix

Expert cost: 50K × $2.00/1M = $0.10
Main agent savings: doesn't burn 50K of its own context
($0.03) and gets a correct answer instead of looping.

VS Code Copilot

VS Code Copilot can also be configured to use the EUMETSAT endpoint via the GitHub Copilot custom model settings. However, OpenCode is the recommended approach for full control over models and subagents.

6. Available Models

All models are served on self-hosted vLLM backends — no external API calls are made. Use the alias names in your client or chat; only aliases are callable.

Auto-routing

When you use the auto model, the system routes your request to the cheapest model that fits your prompt size:

• Prompt < 131K tokens → medium ($0.25 / $1.00)
• Prompt 131K–200K tokens → huge ($2.00 / $8.00)

Context window fallbacks

If a prompt exceeds a model's context window, the system automatically falls back to a larger model instead of returning an error:

• auto (131K) → big (160K) → huge (200K)
• medium (131K) → big (160K) → huge (200K)
• qwen-vl-medium (131K) → big (160K) → huge (200K)
• big (160K) → huge (200K)

Backend names (not callable)

Backend model names — qwen3.8-27b, glm-5.2-w4a16, glm-5.2-fp8 — are shown in the table above for reference only. They are not accepted by the API. Use the corresponding alias.

7. RAGflow / Knowledge Base

RAGflow provides retrieval-augmented generation (RAG) for the platform. It is integrated directly into LibreChat — you interact with it through the chat interface, not a separate UI.

Uploading documents

To analyse a document with RAG, simply upload it in a LibreChat conversation (using the attachment button or drag-and-drop). The AI assistant will parse the document and answer questions about its contents. Documents are processed per-user and are not shared with other users.

Shared knowledge base

The platform has a shared corporate knowledge base called “Public Documents”. Documents added by administrators are searchable by all users through the ragflow-kb model in LibreChat or via the API. Contact borys.saulyak@eumetsat.int if you need a document added to the shared knowledge base.

API access

Query the shared knowledge base through the standard LLM Gateway API:

response = client.chat.completions.create(
    model="ragflow-kb",
    messages=[{"role": "user", "content": "What does the MTG IRS instrument do?"}]
)

8. MCP Tools

MCP (Model Context Protocol) tools are available directly in LibreChat chat. The AI assistant can invoke them on your behalf to interact with external systems. The following tools are available:

9. Architecture

Infrastructure overview

The platform spans three public-facing hosts and five internal-only VMs on the 192.168.168.0/24 network. All inference runs on NVIDIA H200 GPUs — two full 8-GPU nodes for the largest models, and one full H200 for smaller models.

Software deployment

Traffic enters through nginx (TLS termination), which routes to LibreChat (chat UI) and LLM Gateway (model gateway). The LLM Gateway routes inference requests to the appropriate vLLM backend using cost-based routing with automatic context-window fallbacks. Six MCP servers provide tool capabilities (email, code execution, RAG, OCR, web fetch, Jira/Confluence) to both LibreChat and API clients.

How to render these diagrams

These diagrams use Mermaid syntax. You can copy the diagram source and paste it into the Mermaid Live Editor to export as PNG or SVG. In VS Code, install the "Markdown Preview Mermaid Support" extension to preview diagrams in any .md file.