What Happens to Your Customer Data When You Use AI Tools in Australia

You're drafting a follow-up email to a client. You paste their name, contact details and a few notes about their situation into ChatGPT to speed things up. The email writes itself in seconds. Job done. Except it probably wasn't, because the moment that customer's personal information left your system and entered a free AI tool, you most likely triggered obligations under the Privacy Act 1988 that you haven't thought about and possibly can't reverse.

Thousands of Australian small businesses are in exactly this position right now. AI tools customer data risks in Australia are real and immediate, and most business owners aren't aware the regulator has already issued explicit guidance on exactly this scenario.

How AI tools process your customer data

The problem isn't that AI tools are dangerous. Most of them work fine for generic tasks. The issue is what happens to data on the other side of that paste.

When you input personal information into a publicly available AI tool (ChatGPT, Grammarly, Google Gemini or Microsoft Copilot), that information is typically transmitted to servers operated by the AI provider. Depending on the platform's default settings, that data may be used to train or improve the model. It may be retained for extended periods, made accessible to third parties, or both. The OAIC has confirmed it is very difficult to track or control how that information is subsequently used, and in some cases removal is impossible.

Under the Privacy Act and the Australian Privacy Principles (APPs), personal information must only be used or disclosed for the primary purpose for which it was collected. If your customer gave you their name, email address and business details so you could manage their account, they didn't consent to that information being processed by a US-based AI company's servers. That secondary use is potentially a breach of APP 6.

The OAIC published comprehensive guidance on this in October 2024, updated in January 2025, with a clear recommendation: organisations should not enter personal information, and particularly sensitive information, into publicly available generative AI tools.

A real-world scenario that shows how quickly this unravels

The OAIC's December 2025 blog included a case study based on an actual data breach notification. A car insurance company allowed staff to use individual ChatGPT accounts for routine tasks like summarising documents. One employee uploaded a customer's financial hardship application, including health and family details, to generate a summary faster. The AI's summary minimised key aspects of the customer's situation. The company rejected the application. The customer suffered significant financial and emotional harm.

The company had a policy in place. The employee ignored it. The company still had to deal with a notifiable data breach, regulatory scrutiny and real damage to a real person.

For a small business without a formal privacy policy, dedicated compliance staff or any internal training on AI use, the exposure is considerably higher.

Who the Privacy Act actually covers

Many small businesses assume they're safe because of the Privacy Act's turnover threshold. Businesses with an annual turnover below $3 million are generally exempt from the Act's main obligations. That exemption is real, but it's narrower than most owners think.

If your business provides health services, handles tax file numbers, trades in personal information, or is a contractor to a government agency, the exemption doesn't apply. The OAIC's guidance also extends to any business that chooses to engage responsibly with customer data, regardless of size. Privacy Act reform has been under active discussion for several years, with proposals to remove or narrow the small business exemption entirely.

Even setting aside the legal question, there's a commercial one. Clients increasingly ask about data handling practices. A data breach involving customer details processed through a free AI tool would be difficult to explain, and potentially damaging to business relationships in ways that have nothing to do with regulatory fines.

Three decisions that fix most of the problem

You don't need a lawyer or a dedicated compliance officer to reduce your exposure significantly. Three practical decisions cover the majority of the risk.

The first is to separate what you put into AI tools. Generic tasks (drafting blog posts, rewriting headings, summarising non-client content, generating marketing copy from a brief) are low risk. Anything containing a customer's name, contact details, financial information, health details or other identifying information belongs in a different category. The working rule is simple: if you'd need the customer's consent to share that information with a third party, don't paste it into a free tool.

The second decision is to check the settings on any AI tools you use regularly. Most commercial platforms, including the paid tiers of ChatGPT and others, offer settings that turn off data use for model training. Grammarly's business plan provides contractual data processing terms. Google Workspace versions of Gemini operate under Google's enterprise data protection commitments. The default settings on free, consumer-facing versions of these tools frequently permit broader data use. It takes about ten minutes to find and update these settings, and it materially changes your risk profile.

The third decision is to write down your AI use policy, even a short one. The OAIC's guidance specifically flags that policies and procedures governing AI use are an expectation for any business using these tools with personal information. A one-page internal document covering what staff can and cannot enter into AI tools gives you a defensible position and reduces the risk of an employee making a well-intentioned but damaging decision.

The regulator is watching and the guidance is getting sharper

The OAIC has published dedicated guidance on AI and privacy twice in the past 18 months. The December 2025 post on GenAI tools in the workplace was explicit that businesses face reputational damage, harm to individuals and regulatory consequences when AI use goes wrong. Australia's privacy reform agenda is also moving, with changes to the Privacy Act likely to strengthen obligations and extend coverage in the coming years.

None of this requires businesses to stop using AI tools. Searchline works with marketing clients across Sydney and the wider Australian market, and AI-assisted content creation is part of the workflow for many of them. The practical difference between a compliant approach and a risky one comes down to what data goes in, not which tool you're using.

For businesses wanting to understand the current regulatory position in plain terms, the OAIC's guidance on commercially available AI products is the most practical starting point. It includes a checklist specifically designed for businesses selecting and using AI tools.

The Privacy Act isn't going to chase down every small business that pasted a client's name into ChatGPT last Tuesday. The businesses that get ahead of this now, with a clean separation of data types, updated tool settings and a written policy, will be in a stronger position as both the technology and the regulatory environment continue to develop. Start with those three decisions. Everything else follows from there.