Shadow AI is the use of AI tools for work without your brokerage’s approval. Find it by asking staff what they use and reviewing work-device apps and account records. Replace personal tools with approved accounts and teach staff which information they can share.
If an assistant pastes a client file into a personal chatbot, first establish what was shared. Stop further uploads while the person responsible for privacy assesses the exposure.
What Shadow AI is
Shadow AI includes any AI tool used for brokerage work outside your approved approach. The National AI Centre’s guidance on unapproved AI use describes the same approval gap. The tool can be useful and still be unapproved.
Examples in a brokerage include a personal ChatGPT account used to summarise client emails, a browser extension that reads application pages and a phone app that transcribes client calls. An AI feature added to an existing app can also need review before staff use it with client information.
A free ChatGPT account counts as shadow AI when someone uses it for work without approval. Cost does not decide the issue. A paid personal account can create the same problem if the business has not approved its use.
An approved tool has an agreed purpose and data rules. The business knows which account staff use and has assessed the relevant settings and terms. A business account provides a way to manage permitted use, but buying a subscription alone does not establish approval.
Consider this hypothetical example: an assistant copies a borrower’s email and bank-statement extracts into a personal chatbot to draft a lender explanation. The owner sees the polished draft but has no record of the upload. The issue starts with the information leaving the approved workflow, before anyone checks whether the draft is accurate.
Why Shadow AI Puts Client Files at Risk
Shadow AI can send client information to a provider the brokerage has never assessed. The risks depend on the tool’s terms and what the staff member shared. An unapproved upload warrants assessment, even when there is no evidence that an attacker accessed it.
| Risk | What can happen | What the brokerage needs to establish |
|---|---|---|
| Retained uploads | A vendor can keep prompts or uploaded documents under its retention terms. | Who can access the information and how long copies remain. Confidentiality duties continue after the task ends. |
| Training on inputs | Some products allow customer inputs to be used for model training. | Whether this further use is permitted and whether settings prevent it for the approved account. |
| Overseas handling | A tool or its service providers can process information overseas. | Whether the arrangement is an overseas disclosure and which privacy obligations apply. An Australian user account does not establish Australian storage. |
| Missing records | The owner cannot reconstruct which client details were pasted or which account received them. | The scope of a possible exposure and the clients who might be affected. A lack of records makes that assessment harder. |
The OAIC’s guidance on commercial AI products recommends keeping personal information out of publicly available generative AI tools. It also explains that some products allow training on inputs. A staff member removing a client’s name can still leave enough detail to identify them.
For brokerages covered by the Privacy Act, Australian Privacy Principle 6 governs permitted uses and disclosures. Client consent to prepare a loan application does not automatically authorise every further use of that information.
Australian Privacy Principle 8 covers overseas disclosures, with exceptions and conditions. Australian Privacy Principle 11 requires reasonable steps to protect personal information. Consider lender, aggregator and client confidentiality terms alongside these privacy duties.
How to Detect Shadow AI
Start with staff conversations, then check the work systems you already control. You can build a useful inventory without buying detection software. Explain the review to staff and use authorised access to work systems. Ask about personal accounts instead of searching their private contents.
- Ask each person which AI tools they use for work. Ask about the task they wanted to finish and the categories of information they entered. Include casual use on phones.
- Review browser extensions and installed apps with each staff member on work devices. Look at permissions, especially access to page content, email or files. Record tools that need assessment.
- Review available work-account sign-in and connected-app records. Look for unfamiliar AI services or permissions. These records will not reveal every personal account or show the content of every prompt.
- Check expense claims and business-card subscriptions for AI services. Ask about free tools too, because they leave no subscription charge.
- Review work-mailbox forwarding rules and automated workflows for client emails sent to outside addresses or services. Establish the destination and purpose before changing a legitimate workflow.
- Make one inventory of the tools found. Record the account type, work task and data categories, then mark each tool approved, awaiting assessment or stopped.
For each unapproved tool, ask whether client information has already been shared. Preserve the information needed for an incident assessment before asking staff to delete histories. A missing expense entry or sign-in record cannot prove there has been no AI use.
Replace It With Approved Tools
Give staff an approved account that can handle the work they were trying to finish. A permitted route for drafting generic follow-ups or improving public copy removes a reason to reach for a personal account.
For example, official OpenAI documentation says Business workspace data is not used to train models by default. That protection is one part of an assessment. Check retention and connected-app permissions separately, then decide which tasks and information the brokerage permits.
Use named staff accounts with appropriate access controls. Confirm the team can distinguish the approved workspace from a personal one. Test a permitted task using invented client information before rolling it out.
For a small brokerage, a practical changeover can work like this:
- Give staff a short disclosure amnesty, such as one working week, to identify previous use. Make clear that possible client exposure must be reported immediately.
- Explain that the amnesty encourages disclosure of past use. Staff must stop new unapproved uploads while the review takes place.
- Publish a short approved-tools list with the exact account or workspace and permitted tasks. Include the data restrictions staff need at the point of use.
- Give staff one way to request a new tool, such as a message to the owner containing its name and intended task. Require approval before use.
- Revisit the inventory after the changeover. Ask staff to demonstrate a permitted task in the approved account and confirm that unapproved work use has stopped.
Keep any privacy incident assessment separate from the amnesty. An internal promise about how staff can disclose use cannot remove duties to assess or report an exposure.
Train Staff to Use AI Safely
AI training for employees must show what they can enter, how to check the result and who to tell when something goes wrong. Use examples from the brokerage’s actual work, with invented client details.
Set a clear rule for general AI assistants: keep real client identifiers and source documents out of prompts and uploads. This covers identity documents, bank statements, tax returns and payslips. Include account numbers and contact details, alongside health or hardship information. Passwords, authentication codes and portal credentials must never be shared with an AI tool.
If the brokerage approves a separate document-processing workflow, train staff on its exact limits. Approval of a general business chatbot does not automatically authorise that workflow.
Teach staff to check a draft against the original source before using it. For a lender email, confirm every figure and the borrower’s circumstances. For a policy answer, open the lender’s authoritative source. A fluent answer can still contain an invented fact. Keep a person responsible for the final communication or decision.
Practise reporting an accidental upload. The staff member stops further sharing and contacts the owner or designated privacy contact immediately. They record the tool, account and categories of information involved without circulating another copy of the client file. The incident process then determines containment and any notification duties.
For a small team, use this suggested training schedule. These intervals are an operating choice, not a legal training minimum.
| When | Practical exercise | Record to keep |
|---|---|---|
| Before first access | Identify information that must stay out of a prompt and find the approved account. | Staff attendance and exercise result. |
| Monthly team meeting | Spend 10 minutes checking one AI draft against its source or practising mistake reporting. | Topic, attendees and follow-up action. |
| Quarterly | Review the approved-tools list and a sample of permitted work. | Review outcome and any retraining assigned. |
| When a tool or rule changes | Demonstrate the changed setting or data restriction before staff use it. | Who completed the briefing and who still needs it. |
Use one shared attendance record and arrange catch-up training for anyone absent. Finish by asking each person to show where they request a new tool and how they report an accidental upload.
