Securing AI Chatbots and RAG Integrations: The New Attack Surface

Posted on

Every month more Australian businesses ship AI chatbots, copilots and retrieval augmented generation (RAG) features into production. Very few of them have been security tested. That gap is becoming one of the most common findings in our assessments, and it is a gap attackers have already noticed.

Why AI features are different

A traditional web application has a well understood attack surface: authentication, session handling, input validation, access control. An LLM integration adds a new layer that behaves unlike anything in the OWASP Top 10. The model takes natural language as input, and natural language is very hard to sanitise.

The failure modes we test for include:

  • Prompt injection and jailbreaks. Attacker controlled text, in a chat box, an email, or a document your RAG pipeline ingests, that manipulates the model into ignoring its instructions.
  • Sensitive data leakage. Models that reveal system prompts, other customers’ data from retrieval sources, or internal documentation that was never meant to be public.
  • Insecure output handling. Model output rendered as HTML, executed as code, or passed unchecked into downstream systems.
  • Excessive agency. Tool calling integrations that let a model send emails, query databases or call internal APIs with more privilege than the user driving it.
  • AI API abuse. Unmetered endpoints behind your AI features that allow cost abuse, denial of wallet attacks, or direct access bypassing the application logic.

A real world pattern

Consider a customer support chatbot connected to an internal knowledge base. The bot answers from retrieved documents. If document access is not filtered per user, any customer can ask questions that surface another customer’s records. No exploit code required, just a well phrased question. This class of issue rarely shows up in automated scans because it is a business logic flaw expressed through language.

How we test AI systems

Our AI security assessments align with the OWASP Top 10 for LLM Applications and combine manual adversarial testing of the model layer with conventional testing of the APIs and infrastructure behind it. The two go together: many AI breaches are ordinary API security failures sitting underneath a model.

If you have shipped or are about to ship a chatbot, copilot or RAG feature, it deserves the same independent assurance as the rest of your product. Talk to us about an AI system security assessment, scoped and fixed price, delivered remotely anywhere in Australia.