Your model refuses harmful requests in English.
Does it refuse them in Urdu?
Chaukidar audits language models across English, Urdu, Punjabi, Pashto, and Sindhi — comparing English seed, translated baseline, and native-adapted prompts so safety gaps outside English don’t stay invisible.
Five-language audit surface
The seeded demo dataset covers every harm category, while uploaded datasets extend the same database-backed prompt pool for future audits.
Why English-only safety testing isn’t enough
Three-track comparison
Audits English seed prompts beside translation-baseline and native-adapted South Asian prompts, so regressions are visible by track.
Bring your own dataset
Upload a validated JSON dataset after deployment; Chaukidar stores it in the database and uses it in future audits.
Production-ready seed path
When the production database is empty, a sanitized demo dataset seeds automatically so the live app starts with real records.
How an audit runs
Prompt Builder
Builds English seed, translation-baseline, and native-adapted prompt sets from seeded or uploaded datasets.
Execution Agent
Runs prompts through registered Fireworks models, or imports AMD ROCm/vLLM notebook result JSON.
Judge Agent
Uses a GPT-based multilingual judge with a rule fallback to label refusals, partial compliance, and unsafe completions.
Reporting Agent
Stores results in the database, then aggregates safety score, risk heatmaps, and model-level reports.