Deep QA for LLM, chatbot, and RAG products
AI product bugs are different: prompt injection, hallucination, token blowups, PII leaks. Standard audits can't catch these. $1,400 fixed price, 2-week delivery.
⚠️ AI's bugs are new and dangerous
60% of AI products experience a hallucination or prompt injection incident in the first month after launch. Standard web audits don't know how to look for these. This package is done by someone who built an AI test infrastructure for Lento Circle — real risks, real experience.
What's included
Prompt injection + jailbreak tests
50+ known injection patterns (ignore previous instructions, DAN, developer mode, role hijacking). Indirect injection via user content or tool output. Chevron, markdown, encoding escapes.
Hallucination rate measurement
100+ verifiable questions custom to your product. Every answer manually verified. Hallucination rate + severity reported. Baseline + improvement suggestions.
Token / cost security
Rate limits in place? Per-user daily/monthly caps? Loop protection (preventing infinite AI loops)? Malicious long-prompt attempts? Can your bill 10x overnight?
PII leaks + data isolation
Does User A's data leak into User B's response? Prompt-level conversation leak? System prompt escape? GDPR/CCPA compliance pass?
Context window + memory
Consistency across long conversations (200+ messages). Memory system (if present) keeping right info? When context fills — graceful degradation or sudden amnesia?
Tool / function calling
If your AI calls tools (API, DB query) — wrong tool selection? Wrong parameters? Tool error states? Infinite tool loop protection?
RAG-specific tests (if applicable)
Retrieval quality — right document fetched? Chunk boundary bugs? Citation accuracy? Unauthorized document access? Stale data behavior?
Prioritized findings report
Each finding: severity, user impact, cost impact, reproduction prompt, suggested mitigation. Hallucination examples + security incidents in separate sections.
Why AI Product QA instead of Full QA Audit?
Standard web audits don't know how to find AI's bugs — prompt injection, hallucination, token exploits are new attack surfaces.
AI Product QA is domain-specific: 50+ injection pattern library, hallucination measurement methodology, RAG-specific edge cases, tool calling test matrix.
Portfolio: I built the AI agent + MCP + Playwright pipeline for Lento Circle. Someone who tests AI testing your AI — not theoretical, hands-on.
One prompt injection incident = security breach + reputational damage. One hallucination scandal = lawsuit over wrong medical/legal advice. $500 extra here is cheap insurance.
Who it's for
- LLM wrapper products (built on ChatGPT/Claude/Gemini)
- AI chatbots (customer service, sales enablement, internal tools)
- RAG systems (docs search, chat with your docs, knowledge base assistants)
- AI code assistants (Copilot-like)
- Agentic products (where AI calls tools)
- Pre-launch AI startups
- Enterprise AI deployment teams (compliance-pressured)
How we work
- 1
Week 1 — Discovery + attack tests
Learn your product and map AI architecture (which LLM, RAG present, tool calling present). 50+ prompt injection + jailbreak tests. PII leak + context window tests. Daily findings reports.
- 2
Week 2 — Hallucination + cost + report
Measure hallucination rate with 100+ verifiable questions. Token/cost exploit attempts. Final report + retest + mitigation recommendations.
Deliverables
- 25-40 page PDF findings report — security findings and hallucination examples in separate sections
- Hallucination rate measurement report (CSV: prompt + expected + actual)
- 50+ prompt injection test matrix (passed/failed)
- Token/cost security checklist
- Mitigation recommendations (prompt engineering, guardrails, system design)
- One retest round + final report
Ready to prepare your AI product for production?
$1,400 fixed price, 2-week delivery. AI slots limited to 2 per month — the most-requested package in 2026. Book early.
Book AI QA