Prompt Injection Risk in Support Chatbots

Support and knowledge agents retrieve tickets, emails, and documents that customers and outsiders can write. Attackers seed that content with hidden instructions that make agents leak data, change records, or act outside their scope. PointGuard AI inspects retrieved content and tool outputs before they reach the model and tests agents against injection techniques before release.

Anyone who can submit a ticket can try to steer a support agent.

• Tickets and attachments can carry hidden or encoded instructions.
• Agents can be told to reveal other customers' data.
• Injected content can trigger tool calls that change records.
• Keyword filters miss multilingual and adaptive attacks.

Runtime Guardrails and AI Red Teaming test support agents and block injected instructions at runtime.

1. Inspect retrieved content. Scan tickets, documents, and tool outputs for injection patterns using fine-tuned detection models at sub-100 ms latency.

2. Enforce response policies. Block responses that reveal other customers' data, system prompts, or restricted information.

3. Constrain actions. Limit which tools a support agent can call and require approval for record changes triggered by retrieved content.

4. Test continuously. Probe agents with direct and indirect injection scenarios before release and after every model or prompt change.

• OWASP Top 10 for LLMs: LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, and LLM07 System Prompt Leakage, covering injected tickets, leaked customer data, and exposed system prompts.
• OWASP Top 10 for Agentic Applications: ASI01 Agent Goal Hijack and ASI02 Tool Misuse and Exploitation, addressing hijacked agent goals and tools triggered by injected content.
• NIST AI RMF: MAP defines permitted uses; MEASURE tests resistance; MANAGE blocks attacks; GOVERN assigns ownership.