Hardware & Sizing

All € figures are rough placeholder ranges for budgeting only — actual quotes required. SKUs current as of mid-2026; re-verify at RFQ time (GPU market moves quarterly).

Sizing analysis

Workload: ~50 daily users, 10–15 concurrent at peak, primary model in the 30–70B class (quantized), plus an 8B Greek model, embeddings, reranker, and Whisper.

Memory (the binding constraint):

• Llama-3.3-70B-Instruct AWQ 4-bit weights: ~40 GB.
• KV cache: at FP8 KV, roughly 160 KB/token for the 70B → a 16k-token context ≈ 2.6 GB per full-context sequence; 15 concurrent sequences averaging 8–16k context ≈ 20–40 GB KV.
• Greek penalty: Llama/Qwen tokenizers spend ~2–3× more tokens per Greek word than English. A “2-page” Greek document costs 2–3× the context and KV of its English twin. Size KV for the Greek-heavy case, not the English brochure case.
• Krikri-8B (FP8/AWQ): ~8–10 GB incl. cache. Embeddings + reranker + Whisper: ~8–12 GB, ideally on a separate GPU so ingestion never steals serving KV.
• Total comfortable envelope: ~100–140 GB VRAM → 4× 48 GB or 2× 96 GB class.

Throughput: 4× L40S with tensor parallelism on a 70B AWQ model delivers on the order of 400–700 aggregate tok/s under continuous batching — at 15 concurrent users that is ~25–45 tok/s per active stream, above the ~15 tok/s “feels responsive” floor. Comfortable for chat + RAG; batch jobs (ingestion, summarization) run off-peak.

Server options

	Option A — “Lean start”	Option B — “Recommended”	Option C — “Headroom / HA”
GPUs	2× NVIDIA L40S 48 GB (or 2× RTX 6000 Ada)	4× NVIDIA L40S 48 GB	8× L40S in one chassis, or 2× servers à 4× L40S (HA), or 4× RTX PRO 6000 Blackwell 96 GB
VRAM total	96 GB	192 GB	384 GB
Primary model	32B class (Qwen3-32B AWQ) only; 70B does not fit with real concurrency	70B AWQ + Krikri-8B + embeddings, with KV headroom	70B at FP8/16 or two model generations side by side; N+1 failover
Concurrency @16k ctx	~5–8	10–15+	25+ or HA
Chassis	2U, 2× GPU (e.g. Dell R760xa / Supermicro / Lenovo SR670-class)	2U–4U, 4× GPU, air-cooled	4U 8-GPU or 2× Option-B chassis
CPU / RAM	1× 32c EPYC/Xeon, 256 GB	2× 32c, 512 GB	2× 48c+, 768 GB–1 TB per node
Indicative € (placeholder)	~€35–55k	~€70–100k	~€140–220k
Power draw (GPU max)	~0.7 kW GPUs, ~1.2 kW system	~1.4 kW GPUs, ~2.2 kW system	~3–4.5 kW

Recommendation: Option B. The 70B class is where Greek quality becomes acceptable for customer-facing drafts; Option A locks us to 32B and to painful mid-life GPU additions. L40S over H100 NVL: H100 NVL (~94 GB, ~2× the €/GPU) buys HBM bandwidth we don’t need at 15 concurrent users, and L40S is air-cooled, widely stocked in EU channels, and runs in a standard rack. Check at RFQ time: RTX PRO 6000 Blackwell Server Edition (96 GB) — if price/availability has settled, 2× of these can replace 4× L40S with better perf and simpler tensor-parallel config; treat as a drop-in variant of Option B, not a redesign. Option C is the documented growth path (second server ⇒ HA ⇒ k3s trigger per architecture-overview.md).

Power, cooling, room

Item	Requirement (Option B)	Note
Power	~2.2 kW sustained, 2× redundant PSUs, dedicated 16A circuits ×2	Validate Piraeus HQ server-room supply — likely built for a few 1U boxes, not GPU load
UPS	Online double-conversion, ≥5 kVA, 10–15 min runtime + clean-shutdown integration (NUT)	Existing UPS almost certainly undersized — budget ~€3–6k placeholder
Cooling	~7,500 BTU/h added heat; room must hold <27 °C at Piraeus summer ambient	Split-unit audit required; add redundant cooling or the server throttles in August
Rack	42U with proper airflow (blanking panels), locked	Physical access control — this box holds tender data
Fire/env	Smoke + temp + leak sensors alerting to Teams	Cheap; do it

If the room audit fails, fallback is a colocation cage in an Athens-area datacenter — still “on-prem” in the confidentiality sense (our hardware, our keys) but weakens the air-gap story; decision for steering if triggered.

Storage

Tier	What	Size	Media
Hot	Model weights, KV offload, active indexes	2× 3.84 TB NVMe (RAID1)	Enterprise NVMe (mixed-use)
Warm	PostgreSQL, Qdrant, documents, n8n	2× 7.68 TB NVMe (RAID1)	Read-intensive NVMe
Backup (local)	PBS datastore, snapshots	≥24 TB usable	NAS with HDD RAID6 (separate box)
Offsite	Immutable copies	≥24 TB	Second NAS at branch office or EU S3-compatible with object lock

Model zoo grows fast (each 70B quant ~40 GB, keep N−1 versions) — the hot tier is not a place to economize.

Networking

• 25 GbE dual-port on the GPU server (10 GbE minimum), LACP to a 10/25 GbE switch; ingestion and backup traffic must not fight chat latency.
• Dedicated AI VLAN, firewalled from the office LAN; only the reverse proxy (Open WebUI/Teams webhook endpoints) exposed to the user VLAN. Egress default-deny — see security-and-operations.md.
• Out-of-band management (iDRAC/IPMI) on a management-only VLAN.

Backup strategy (3-2-1)

• 3 copies, 2 media, 1 offsite/immutable: Proxmox Backup Server to local NAS (nightly VM + hourly DB dumps), replicated via restic/PBS sync to offsite immutable target (branch-office NAS or EU object storage with object lock).
• Targets: RPO ≤ 24 h for the platform, ≤ 1 h for PostgreSQL (WAL archiving); RTO ≤ 1 business day full platform, ≤ 4 h for chat+RAG core. Full table and drill schedule in security-and-operations.md.
• Model weights are re-downloadable — back up the model registry manifest (hashes, sources), not 400 GB of weights.

Procurement notes

• Lead times: GPU servers 6–10 weeks EU-configured; order the server, UPS, and network gear in the same wave. Room remediation (power/cooling) runs in parallel — it is usually the long pole.
• Vendors: buy through a Greek/EU enterprise integrator (Dell/Lenovo/HPE/Supermicro channel) with NBD on-site support, 3–5 yr, GPUs explicitly on the support contract. Placeholder ~8–12% of hardware cost for 3-yr support.
• Get two competing quotes; specify “4× L40S or current-equivalent NVIDIA 48–96 GB server SKU” so the integrator can propose newer silicon at equal-or-better €/VRAM.
• Dependency and risk entries: ../02-program/risk-register.md (lead time, room remediation, GPU price volatility).

Growth path

1. Now (Option B): 4× L40S, single server.
2. +12–18 mo if adoption holds: second identical server → HA pair, staging gets real GPUs, k3s migration trigger.
3. Model growth: next-gen 96 GB+ cards slot into the same chassis class; 70B → 100B+ class becomes feasible without platform redesign.