Prompt Injection: Risks for AI in a Brokerage

Prompt injection is an attack that uses text an AI assistant can mistake for an instruction. A broker might ask an assistant to summarise a bank statement and get a misleading answer because the assistant followed text hidden in the PDF. If the assistant can send email or edit files, the same mistake can expose client information or change records.

What is Prompt Injection?

Prompt injection happens when someone supplies text designed to steer an AI assistant away from its authorised task. In a direct injection, a user puts the instruction into the chat. In an indirect injection, it arrives in material the assistant reads, such as a document or web page.

Jailbreaking is an attempt to bypass the model’s safety restrictions. Prompt injection describes how an instruction reaches and misdirects the assistant. The two can overlap, but an injection can cause harm using ordinary features. An assistant may be allowed to send email, for example, even when a particular message has no business being sent.

How a Prompt Injection Attack Reaches a Brokerage

A brokerage can encounter indirect prompt injection during routine document review, email triage or online research. These are hypothetical examples.

  • A broker asks for figures from a PDF bank statement. Hidden text in the PDF leads the assistant to omit a transaction from the summary. Compare the summary with the original statement before relying on it.
  • A shared inbox receives an email that directs the assistant to forward attachments. The assistant treats text from the sender as an authorised task. Review any proposed forward against the original request and check each recipient and attachment before approval.
  • An assistant summarises a lender’s policy from the web. Instructions in a page skew the summary so it leaves out a condition. Open the lender’s own policy and confirm the wording before using it in a file.

Why AI Agents Raise the Stakes

An AI agent can use connected tools to act on the assistant’s decisions. With the right permissions, it can open files, visit websites or send email. Each connection gives a misleading instruction another way to affect the brokerage.

A chat with no connected tools can still give a wrong answer that a broker later acts on. An agent with access to a client folder and a mailbox can expose a file through a send action before anyone checks the reply. If an assistant only needs to draft replies, leave sending to a staff member.

Protect Client Files From Injected Instructions

Give the assistant access only to the information and actions it needs for one approved task. The Australian Signals Directorate recommends restricting access and keeping people in charge of sensitive actions. See its guidance on adopting AI agents.

  • Use read-only access for summaries. Give the assistant a designated folder or approved document copy instead of the brokerage’s entire drive.
  • Require a person to approve sending or sharing. The reviewer needs to see the destination and the exact material that will leave the brokerage.
  • Keep public web research and untrusted documents away from assistants with access to client records. Use a separate session or tool with the client-data connections disabled.
  • Check the output against the original source before it goes into a client file or informs advice. Check the relevant wording or figure yourself.

Detection tools can flag some suspicious content. An instruction can still get through, so keep access restrictions and approvals in place when a scan shows no warning. Use the brokerage’s AI risk assessment to record what each assistant can reach and who can approve its actions.

What to Do After a Suspected Injection

Stop the assistant’s access as soon as you suspect it followed an instruction from an untrusted source.

  1. Pause the workflow and have your administrator revoke the assistant’s access to connected mailboxes and file stores. Preserve the original content and available activity logs.
  2. Check what the assistant sent or changed. Review sent items, sharing activity and file version history. Record recipients, affected files and the period at issue.
  3. Establish whether anyone gained unauthorised access to client information or received it. Restrict unintended sharing and escalate to the person responsible for the brokerage’s data breach response.
  4. Follow the brokerage’s breach process even if the extent of disclosure is still unclear. Use the Office of the Australian Information Commissioner’s breach response guide to assess notification obligations where they apply.

Before reconnecting the assistant, confirm which access the job needs and check that a staff member must approve any send or share.

Track My Trail

Trail analysis, simplified

Less time checking your trail, more time growing your book.

See how it works

7 days free · No credit card

Track My Trail Team

Read more

We develop software to simplify trail book management for mortgage brokers. Our tools provide fast and practical insights to help brokers get the most out of their trail books.

Track My Trail
See how it works

7 days free · No credit card