Every week, an Australian business asks some version of the same question: if my team pastes a client list, a draft contract or next quarter's pricing into ChatGPT, is that information still confidential? The short answer is that it depends, and it depends mostly on decisions your business controls: which ChatGPT product you use, how it is configured, and what you have agreed with the provider, your clients and your staff.
Confidentiality is not a fixed property of the tool. It is the result of a chain of arrangements, and each link in the chain can be tightened or left loose. This article walks through how confidentiality actually operates when a business uses ChatGPT: who holds the data at each step, what separates a safe prompt from an exposed one, when Australian law steps in, and where confidentiality most often breaks down in practice.
Who holds your data when your team uses ChatGPT
Confidentiality is easier to reason about once you can see the parties involved and what each one does:
- Your business: decides what goes into prompts, which ChatGPT product staff may use, and what your policies and contracts allow.
- OpenAI: operates ChatGPT. Its systems receive, process and store your prompts and outputs, and its terms and settings determine what happens to that content after someone presses enter.
- Your clients, customers and staff: their personal information and confidential information can be swept into a prompt, which is what turns a routine question into a legal issue.
- Regulators: the Office of the Australian Information Commissioner (OAIC) oversees the Privacy Act 1988 (Cth) and the Notifiable Data Breaches scheme, and the Australian Competition and Consumer Commission (ACCC) enforces the Australian Consumer Law where AI output is used to mislead customers.
- A court: if a dispute arises, a court decides whether information was confidential in the legal sense, and whether its disclosure or use was authorised.
The chain matters because each party's rights and obligations come from different instruments: OpenAI's terms, your contracts with clients and suppliers, your employment arrangements, and the legislation that applies to your sector.
What decides whether ChatGPT is confidential
The first thing to understand is that there is no single answer that holds across all of ChatGPT. OpenAI draws a sharp line between its personal and business products, and the line is set out in its enterprise privacy commitments:
-
Personal plans (Free, Plus and Pro): By default, OpenAI may use conversations from personal accounts to improve its models. You can turn this off in your data controls, but unless you do, prompts and outputs from a personal account can be used for training, and content may also be reviewed by human trainers and abuse reviewers. That is a poor fit for anything confidential.
-
Business plans (ChatGPT Business, Enterprise, Edu) and the API platform: OpenAI's commitments state that it does not train its models on your inputs and outputs by default, and that you own your inputs and outputs. Data sent through the API is typically retained for up to 30 days before deletion, unless you have a zero-data-retention arrangement. These products also add central administration: workspace controls, sign-on management, audit logs and usage analytics that personal accounts simply do not have.
-
"No training" is not "no processing": Even on a business plan, every prompt travels to OpenAI's systems, which are largely based in the United States. Content can be logged, reviewed for safety and abuse, and handled by subprocessors under OpenAI's arrangements. "No training" means the provider promises not to feed your content into its models. It does not mean your content stays inside your own environment, or that nobody at the provider end can ever see it.
-
Shared links are a common leak: ChatGPT conversations can be shared by link, and a link shared with the wrong person, or pasted into a public space, exposes the whole conversation. Workspace admins on business plans can restrict who can share, but on personal accounts the control sits with each individual user.
The practical default, then, is this: assume a prompt is visible to the provider and its subprocessors, and possibly to other people, unless your plan, your settings and your contract say otherwise. That assumption should drive everything else in this article.
When the law gets involved
Once a prompt contains personal information, or information that belongs to somebody else, Australian law starts to matter regardless of which plan you are on.
Personal information in a prompt engages the Privacy Act
The APPs apply to "APP entities": most businesses with annual turnover above $3 million, plus certain organisations regardless of size. Under s 6D of the Privacy Act 1988 (Cth), a business is a small business if its annual turnover for the previous financial year was $3 million or less, and small businesses are generally exempt from the APPs. There are exceptions: health service providers, and businesses that buy or sell personal information, are covered whatever their turnover, and the small business exemption itself is under review as part of ongoing privacy reform.
If the Privacy Act applies to you, pasting personal information into an AI prompt engages several of the Australian Privacy Principles at once. APP 6 limits use and disclosure to the purpose for which the information was collected, or a related purpose the person would reasonably expect. APP 8 governs overseas disclosure: OpenAI is a US company, so personal information in a prompt is likely to be transferred offshore, and you must take reasonable steps to ensure the recipient handles it consistently with the APPs, while remaining accountable for breaches. APP 11 requires you to take reasonable steps to protect personal information from misuse, interference, loss and unauthorised access, modification or disclosure. Feeding identifiable customer details into a tool you have not assessed sits uncomfortably with each of these.
Client information can be protected by the equitable duty of confidence
Independently of privacy law, information you hold for clients, suppliers and partners can be protected by the equitable doctrine of breach of confidence. Under the classic statement in Coco v A N Clark (Engineers) Ltd [1969] RPC 41, adopted in Australia by the High Court in Smith Kline & French Laboratories (Aust) Ltd v Secretary, Department of Community Services and Health (1991), three things must hold: the information must have the necessary quality of confidence, it must have been imparted in circumstances importing an obligation of confidence, and there must be unauthorised use (or threatened use) of it to the detriment of the person who communicated it.
Pasting client materials into a third-party tool can be exactly the kind of "disclosure" that breaches these obligations, unless your contract with the client permits it or you have consent. Confidentiality clauses in your client and supplier agreements add a contractual layer on top of the equitable one, and breaching them can expose you to damages and injunctions even where no regulator is involved.
A leak can trigger the Notifiable Data Breaches scheme
If personal information held by a covered entity is lost, or subjected to unauthorised access or disclosure, and a reasonable person would conclude there is a likely risk of serious harm, the breach is an "eligible data breach" under Part IIIC of the Privacy Act 1988 (Cth) (ss 26WE and 26WG). The entity must then notify affected individuals and the OAIC as soon as practicable (s 26WL), with recommendations about the steps individuals should take. The OAIC's guidance is straightforward about the trigger: notification is required when a data breach is likely to result in serious harm.
AI makes this concrete. A staff member who prompts ChatGPT with a spreadsheet of customer contact details, and then shares the conversation link by mistake, has handed personal information to a third party. If that information is sensitive, or the exposure is broad, the business may have an eligible data breach on its hands and a legal obligation to notify.
AI output can create consumer law exposure
The risk does not stop at inputs. Publishing or acting on AI-generated content without review can breach the Australian Consumer Law, which is Schedule 2 of the Competition and Consumer Act 2010 (Cth). Section 18 prohibits misleading or deceptive conduct in trade or commerce, and breaches can be met with damages, injunctions and other court orders. Where output becomes a false or misleading representation to customers (for example, in marketing copy), s 29 is a penalty provision: under s 224 the maximum penalty for a company is the greater of $100 million, three times the benefit obtained, or 30% of adjusted turnover, and $2.5 million for an individual.
AI models can be wrong, outdated or overconfident, so the practical rule is human review of anything client-facing, particularly claims that a customer might rely on.
A new statutory tort for serious invasions of privacy
The Privacy and Other Legislation Amendment Act 2024 (Cth) added a further layer. From 10 June 2025, individuals can sue directly for a serious invasion of their privacy, under a statutory tort that does not require them to prove a breach of the Privacy Act. The OAIC explains that the tort sits alongside the existing privacy framework. For a business, the practical effect is that mishandling someone's personal information, including through an AI tool, can now produce a direct claim from the affected individual as well as regulatory action. The same Act also increased the maximum penalties for serious interferences with privacy.
Where confidentiality most often breaks down
The mechanism is clear in theory, but in practice confidentiality fails at a handful of predictable points:
- Personal accounts used for work: Staff sign into ChatGPT with their own free or paid accounts, where training is on by default and the employer has no admin controls, no audit trail and no ability to restrict sharing.
- Reading "no training" as "confidential": A business plan stops model training; it does not stop processing, logging, human review for abuse, or subprocessors. Treating the enterprise tier as a vault is a mistake.
- Partial redaction: Names and phone numbers get stripped, but the prompt still contains a client's industry, pricing structure or identifiable project details, which is enough to breach confidence.
- Shared links left open: Conversations shared by link remain accessible, and the link can outlive the project it was created for.
- Outputs that echo the input: AI can reproduce or recombine the content it was given, so a sanitised prompt does not guarantee a sanitised output. Outputs need their own review.
- The exempt business that assumes privacy law never applies: A business under the $3 million turnover threshold may be outside the APPs, but the equitable duty of confidence and contractual confidentiality clauses do not care about turnover, and the small business exemption is under active review.
The controls that change the outcome
Confidentiality is a design decision, and the levers are all within reach of a small business:
- Choose the right product and configure it centrally: If business information will go into AI tools at all, use a business or enterprise plan, or the API, where training is off by default. Configure the workspace settings centrally, disable training and retention where offered, restrict sharing, and remove the ability for individual staff to change the settings.
- Classify before you prompt: Give the team three buckets: content that is fine to use as an example, internal content that is safe once de-identified, and confidential or personal content that must not enter an AI prompt without specific approval and controls.
- Redact as a habit: Strip names, contact details, dollar figures and client identifiers before prompting, and check outputs for echoes of the input before they are used.
- Bake in human review: Anything client-facing or compliance-related gets checked: facts verified, claims substantiated, tone appropriate, and no confidential detail slipped in.
- Put the contracts in place: A vendor agreement with the AI provider that covers confidentiality, security and data use where one is available; a data processing agreement if a provider handles personal information on your behalf; non-disclosure agreements with collaborators; and client terms that say whether and how AI may be used in delivering your services.
- Write the internal rules down: A short AI use policy, a privacy policy that reflects actual practice, and an information security policy covering access and incident response turn good intentions into something enforceable.
When a lawyer should be involved
A lawyer's role here is to close the gaps between what the provider promises, what your contracts allow, and what the law requires. That means reviewing OpenAI's terms and your workspace settings against your obligations, deciding whether a data processing agreement or updated client terms are needed, drafting the AI use policy, and assessing whether particular uses of client information amount to an unauthorised disclosure under confidence law. If something has already gone wrong, a lawyer can help assess whether a data breach is notifiable, coordinate the OAIC notification, and advise on exposure to the new statutory tort and to claims from clients. Getting the assessment done before a leak occurs is far cheaper than responding to one.
Decide which data can enter a prompt before anyone does
The single point where risk concentrates is the moment a prompt is submitted. Before that moment, everything is controllable: the product can be changed, settings tightened, contracts signed and staff trained. After it, the information is outside your environment, and no setting you change afterwards can pull it back. The decisions that matter most are made in advance, and they reduce to one: decide now which categories of information may enter an AI prompt at all, and match your plan, settings and contracts to that decision. A business that makes that call deliberately, and writes it down, gets the speed benefit of AI without betting its clients' trust on a default setting.