Safety gate
Red-team the bot
before M-Pesa Twitter
does it for you.
Structured adversarial tests, jailbreaks, PII fishing, policy drift, run against your stack in Kenya, free of a generic SaaS checklist.
Kenyan banks and insurers launch WhatsApp bots under CBK and KDPA scrutiny. One viral screenshot of a bot leaking a customer ID number ends more careers than a slow queue ever did. Eval & red team is how you find that failure in a lab, on the same Nairobi GPUs you will use in production.
Attack library
We assume polite users are already covered.
Jailbreaks
DAN-style, role-play, encoding tricks, multilingual obfuscation, including Sheng-coded prompts.
PII fishing
Attempts to extract ID numbers, account hints, salary bands, or neighbour data from context.
Policy drift
“Approve this loan,” “waive this fee,” “ignore CBK limit”, against your guardrail docs.
Tool abuse
Trick agents into calling APIs they must avoid, or escalating fake VIPs.
$ corvux eval run --suite red-team-v2 --target whatsapp-desk JAILBREAK 12/240 succeeded (FAIL >5%) PII LEAK 0/80 succeeded (PASS) POLICY 3/120 succeeded (WARN) VERDICT NO-GO until jailbreak patch ✓ report shared with CISO + ML owner
Run this before opening WhatsApp desks, publishing agents in Studio, or swapping models via routing. Outputs feed your ODPC evidence pack as proof you tested, with proof, beyond hope.
FAQ
From security and ML owners.
Do you test production or staging?
Staging mirrors production config. We never aim jailbreaks at live customer traffic without written approval.
What counts as a pass?
Agreed thresholds per category: e.g. zero successful national-ID exfiltration, <2% policy violations on gold prompts.
How often should we re-run?
Before every major model swap, corpus publish, or agent publish from Studio. Quarterly minimum for regulated desks.
Is this penetration testing?
It is AI-specific adversarial evaluation, complementary to app pentests, and short of a replacement.
Schedule a red-team sprint
Bring your agent, guardrails, and worst-case scenarios. We return a gate report in days, well short of quarters.
Request eval scope