FinOps layer

Small model first.
Big model when it
counts.

A routing endpoint that cascades cheap Kenya-hosted inference into larger weights only when confidence or complexity demands it.

Running every staff question through a 70B model on Nairobi GPUs is like hiring a senior counsel to print boarding passes. Model routing sends greetings, FAQs, and simple lookups to a fast small model, and escalates to heavy weights for underwriting notes, legal clauses, or multi-doc synthesis.

POST /v1/chat/completions → router.corvux.internal

CLASSIFY faq | complex | refusal

ROUTE 7B-local (example mix)

CASCADE → 70B-local if the router is unsure

COST Quoted on your mix, no invented unit price

✓ all hops stay in Kenya

Cascade logic

Tuned on your logs, off any paper model.

A
Intent classifier
Lightweight model or rules tag request type, FAQ, extract, summarise, reason.
B
First pass
Small model answers; self-reports confidence and citation coverage.
C
Escalation
Low confidence, missing citation, or flagged regex → large model with full context.
D
Feedback loop
Human overrides and eval failures retrain routing thresholds monthly.
Router
Small model first
OpenAI
Compatible API
Per-app
Routing policies
KE
Both tiers local
Routing pays off fastest when you already have volume on private chat or managed sovereign AI. We baseline cost-per-request before flipping the router on.

FAQ

FinOps and platform leads.

Will users see two different “personalities”?

Routing is invisible. System prompts and RAG context stay consistent; only the backend weight changes.

How much can we save?

We skip inventing a savings percentage. After we see your traffic mix we can measure small-model vs large-model share on your logs.

Does routing work with RAG?

Yes. Retrieval runs once; the router picks 7B vs 70B (or your sizes) based on query class and retrieval confidence.

Foreign fallback?

Off by default. Cascades stay on your Kenya fleet unless you explicitly allow a sanctioned external tier.

Baseline your GPU burn

Send a week of anonymised request logs. We model cascade savings before you re-architect.

Talk to platform FinOps