An AI risk assessment checks what an artificial intelligence tool can access, what could go wrong and which controls must work before you use it. For a mortgage brokerage, assess the particular task and client data involved, then record the remaining risk and approval conditions. Test with artificial data before sending a real client file.
AI Privacy Risks for Client Data
AI privacy risks start with what happens to the information you put into the tool. A bank statement can expose account details and reveal personal circumstances through transaction descriptions. Removing the client’s name alone does not establish that the remaining information is anonymous.
Check whether the vendor retains uploads, prompts or outputs. Establish whether it uses them for model training and whether staff or other suppliers can access them. A vendor breach can expose retained files. Shared chat links or broad team permissions can also disclose a conversation to the wrong person.
The Office of the Australian Information Commissioner’s guidance on commercially available AI products calls for due diligence on the intended use and data flows. A setting that stops model training answers only one part of the assessment. It does not, by itself, establish zero retention or restricted support access.
Overseas processing needs its own check. Identify storage locations and who can access the information from elsewhere. Where Australian Privacy Principle 8 applies, the brokerage generally must take reasonable steps before disclosure to an overseas recipient. The OAIC’s cross-border disclosure guidance explains the rules and exceptions. A server’s country alone does not settle how the information is handled.
Generative AI security risks include prompt injection, where instructions inside a document or message try to redirect the AI’s behaviour. An agent connected to a mailbox or client relationship management system (CRM) can then act on those instructions. The Australian Signals Directorate’s agentic AI security guidance recommends controlling access and permitted actions. A prompt telling an agent to behave safely cannot replace those controls.
Accuracy creates a separate risk. A statement reader can misread an amount or omit a transaction even when the data stays private. The broker still needs to verify figures before relying on them.
Run the Assessment Step by Step
Assess each proposed use before approval, including a new use of a tool your brokerage already has. The National AI Centre’s AI adoption guidance treats risk as dependent on how a system is used.
- Describe the job and its limits. Write a specific purpose, such as extracting transactions for a broker to check. State whether the tool can only return a draft or can also send messages, edit records or make decisions.
- List the information involved. Include uploads, generated summaries and connected records. Trace where each goes, who can see it and how long copies remain. Include service logs and backups where they hold client information.
- Check the exact product terms and account settings. Record the plan, processing terms and applicable settings. Check training use, retention and deletion. Confirm that the brokerage can lawfully use or disclose the information for this purpose and provide required client notices.
- List each failure and its consequence. Cover disclosure, unauthorised actions and inaccurate outputs. Consider whose information could be exposed and what a wrong result would cause in a loan file.
- Rate each risk before controls. Consider both likelihood and impact using your brokerage’s existing risk method. Record the reason for the rating. An unknown retention period is unresolved risk, not evidence of a low rating.
- Assign a control and an owner to each risk. Set the evidence needed to confirm the control works. Then rate the residual risk, meaning the risk left after those controls operate.
- Test the approved boundaries with artificial files. Check extraction against known values, try an unauthorised action and test access restrictions. Record expected and actual results before seeking approval for client data.
A free chatbot used to reword public text needs a shorter assessment than a tool that reads an entire mailbox. Price does not determine the risk. Client information, connected permissions and the consequences of errors determine the depth required.
For a connected tool, list each permission separately. Start with the minimum read access needed and require human approval before consequential actions. Confirm that disconnecting the tool revokes its access. A failed boundary test means changing the configuration or stopping that use before real files enter it.
When to Do a Privacy Impact Assessment
A privacy impact assessment (PIA) examines how a project affects people’s privacy and recommends ways to reduce those effects. The OAIC’s PIA guide recommends an initial threshold assessment for projects involving personal information. Its process can be adapted to the project’s size and risk.
For a brokerage, a statement reader handling identifiable financial records warrants a PIA before client use. Broader CRM access or a new overseas disclosure also warrants closer privacy analysis. Start early enough to remove unnecessary data or change the proposed workflow.
In a small practice, put the PIA inside the AI risk assessment. Describe the data flows, identify privacy impacts and assess applicable privacy obligations. Record consultation, chosen controls and any unresolved privacy risks. A short document can cover a simple project, but brevity must not conceal a material issue.
An AI impact assessment has wider scope. It can examine fairness, accuracy and effects on people’s decisions as well as privacy. The Digital Transformation Agency’s AI impact assessment tool illustrates this broader approach for government teams. A brokerage can use that distinction without treating government-specific requirements as its own.
Check the Vendor’s Data Processing Agreement
A data processing agreement (DPA) sets contractual terms for how a supplier processes personal information for its customer. It can be a separate agreement or an addendum to the service contract. The document’s title does not prove it covers your account or intended use.
Find the processing terms through the vendor’s legal pages, service agreement or security documentation. Follow the terms linked to the specific business plan or developer service you intend to use. If the agreement requires acceptance or signature, complete that step and keep the executed version. General website privacy wording alone does not establish the contract between the brokerage and supplier.
Read the clauses that control what happens to client information:
- Limit processing to agreed purposes and establish whether training or product development is permitted.
- Specify retention periods for files and outputs, including any exceptions for logs or backups.
- Identify sub-processors, meaning other suppliers handling the data, and how changes are notified.
- Establish processing locations and controls over overseas access.
- Set security responsibilities and breach-notification arrangements, including who receives a notice and what information it contains.
- Explain deletion or return when the service ends and how the brokerage can obtain confirmation.
Compare the agreement with the settings you tested. A training opt-out does not resolve a contract that permits another unwanted use. Where the terms do not support the proposed handling of client information, restrict the use to artificial data or choose a different arrangement.
A Worked Assessment for a Statement Reader
This hypothetical assessment concerns a tool that extracts bank transactions into a draft table. The brokerage proposes a standalone tool with manual file uploads and permission only to return a draft. A broker will compare the output with the original statement before using any figure.
The example uses High, Medium and Low ratings. High means a material exposure or harmful action needs further treatment. Medium means controls reduce the risk but ongoing checks remain necessary. Low means the permitted use has limited consequences under the stated controls. These are illustrative judgements, not vendor ratings or a regulatory scoring system.
Hypothetical Risk and Mitigation Record
| Risk | Before controls | Chosen control | Residual risk |
|---|---|---|---|
| Client records retained or used for training | High | Require terms excluding training use and a defined retention period. Verify the relevant account settings. | Medium once confirmed, because processing still exposes data to the service |
| Files or chats accessed by the wrong person | High | Restrict account access, disable public sharing and test that another user cannot open the upload. | Medium once access tests pass |
| Overseas recipients and sub-processors unidentified | High | Obtain the processing chain and assess the proposed disclosures before client uploads. | High while recipients remain unresolved |
| Instructions inside a statement cause an unwanted action | High | Keep the tool disconnected from business systems and prevent external actions. Treat document text as untrusted input. | Low for external actions within this restricted setup |
| A transaction or amount is extracted incorrectly | High | Reconcile the draft against the source and require broker approval before use. | Medium because review can still miss an error |
| Uploaded records remain after deletion | High | Establish deletion terms and test the user-facing deletion process with artificial files. Obtain evidence for retained-copy handling. | Medium only when deletion arrangements are established |
The decision is an artificial-data pilot only. Real client uploads remain unapproved while the processing recipients are unknown. Even if extraction works well, that unresolved privacy risk prevents approval of the proposed client-data use.
Before reconsidering, the assessor must supply evidence for the outstanding processing chain and complete the privacy analysis. The broker must also record whether the pilot’s output and permission checks meet their expected results. A lower residual rating depends on completed controls, not a promise to implement them later.
Record, Approve and Review
Save the assessment with the exact use assessed and its permitted data. Keep the terms and settings evidence with the risk ratings. Record test results, unresolved issues and the approval conditions, including any prohibition on client files.
The person authorised by the brokerage’s approval process signs off the residual risks and permitted use. Give staff the resulting limits before they start. Keep the broader role assignments and tool register in the brokerage’s AI governance process.
Reassess when the intended use changes, the tool gains a new connection or its permissions expand. Changed training terms, retention arrangements or sub-processors also trigger review. A security incident or repeated extraction error can invalidate the earlier decision.
Before the first client file is uploaded, confirm that every approval condition is met in the account staff will actually use. If a condition remains unmet, keep that use restricted until the assessment supports it.
