Technical Blueprint — Index

This folder is the technical blueprint for the Prime Products Ltd on-prem AI platform: what we build, on what hardware, with which stack, and how it is secured and operated. It is opinionated by design — one recommended stack per layer, alternatives listed with reasons rejected.

Reading order

1. architecture-overview.md — goals, principles, the big-picture diagram, environments, deployment model, roles, ops calendar. Start here.
2. hardware-and-sizing.md — GPU server options (A/B/C), sizing math, power/cooling, network, backup hardware, procurement.
3. llm-serving-stack.md — vLLM serving, model gateway, Open WebUI, model matrix (Greek+English), quantization, model lifecycle, orchestration (n8n + LlamaIndex).
4. rag-and-data-pipeline.md — Qdrant + PostgreSQL, ingestion pipeline, sources & sync cadence, retrieval design, data governance, feedback loop.
5. integrations-m365-softone.md — Entra ID SSO, Graph API ingestion, Teams surface, SoftOne read-first semantic layer, e-shop.
6. security-and-operations.md — security model, monitoring, backup/DR, runbook pointers, patching cadence.

Related: ../08-ai-portfolio/_index.md (what runs on this platform), ../06-knowledge-base/obsidian-vault-blueprint.md (the curated RAG core), ../04-departments/it.md, ../../02-Requirements/_index.md, ../../03-Infrastructure/_index.md.

Recommended stack — one screen

Layer	Choice	Why (one line)
Compute	1× GPU server, 4× NVIDIA L40S 48 GB (Option B)	Best price/perf for a 30–70B quantized model at 10–15 concurrent users; air-cooled, standard rack.
Virtualization	Proxmox VE	Free, battle-tested, GPU passthrough, built-in backup — right-sized for a 2–4 person IT team.
Container runtime	Docker Compose per VM	Boring, debuggable, no cluster to babysit; k3s is the documented growth path, not the start.
LLM serving	vLLM (OpenAI-compatible)	Continuous batching handles concurrency; the de-facto standard for on-prem serving.
Model gateway	LiteLLM proxy	One API key surface, model routing, per-team quotas, audit logs, provider abstraction.
Chat UI	Open WebUI	Mature self-hosted UI with OIDC SSO, RBAC, RAG hooks — beats building custom.
Primary model	Llama-3.3-70B-Instruct (AWQ 4-bit)	Strongest open generalist with acceptable Greek; fits 4× 48 GB with headroom.
Greek specialist	ILSP Llama-Krikri-8B-Instruct	Athena RC model trained on Greek — drafting/tone tasks in Greek.
Embeddings	BGE-M3	Multilingual (incl. Greek), dense+sparse+multi-vector in one model.
Vector DB	Qdrant	Single binary, rich payload filtering (ACLs!), snapshots — lowest ops burden.
Relational DB	PostgreSQL 16	Backbone for n8n, Open WebUI, semantic layer, audit — one database engine to operate.
Orchestration	n8n (+ LlamaIndex for RAG code)	Self-hosted low-code with mature M365 connectors; IT can own flows without a dev team.
Identity	Microsoft Entra ID (OIDC)	Already the company IdP; SSO + MFA everywhere, no second user directory.
Monitoring	Prometheus + Grafana + Loki + DCGM + Langfuse	Standard observability plus LLM-level tracing/cost per request.
Backup	Proxmox Backup Server + restic offsite (3-2-1)	Immutable, deduplicated, drill-able restores.