Someone Just Pasted a Client File Into ChatGPT

It’s 4:40 on a Thursday.

A senior associate has a client review at nine tomorrow. She’s got eleven pages of file notes she hasn’t read, and she’s been sitting in other people’s meetings since ten.

She opens a browser tab, ChatGPT, the same one her nephew uses for his assignments. Select all, copy, paste.

Then she types: “Summarise the key issues and flag anything that needs a decision.”

Four seconds later there’s a clean one-page summary in front of her. It’s good. She reads it twice, feels the whole afternoon lift off her shoulders, and goes home at a reasonable hour for once.

Nothing broke. No alarm sounded. Nobody did anything they thought was wrong…

That’s the entire problem, and it’s why this one’s harder than the threats you already pay somebody to worry about.

Is ChatGPT confidential? It’s a convention, not a control

You know how “off the record” works. It isn’t a control. It’s a convention, honoured by a professional you’ve decided to trust, and the moment you say the words out loud they exist in someone else’s head whether the convention holds or not.

Everyone in your firm instinctively understands journalists. Almost nobody applies it to a text box.

So when a partner asks me whether ChatGPT is confidential, there’s no yes-or-no answer at the end of it. What there is instead is somebody else’s settings, somebody else’s terms and somebody else’s commercial incentives, none of which you control and all of which can change on a Tuesday.

A public AI tool is a party you’ve never met, operating under terms you haven’t read, in a jurisdiction you didn’t choose. The privacy toggle is a convention too, and we’ve gone through what the free tools actually do with what you type before. Your client’s financial position, health circumstances, family arrangements or settlement position are now sitting on the other side of it.

IT Companies have been selling AI to firms like yours for two years and has spent about ten minutes explaining where the words go. We’re included in that. So here’s the ten minutes, done properly.

The privacy regulator puts the consequence about as plainly as a regulator can. In its guidance on generative AI in the workplace, the OAIC says that once personal information has been put into these systems, depending on the privacy settings, “it will be very difficult to track or control how it is used, and potentially impossible to remove.”

Not difficult to recover. Impossible to remove.

Beware: Every other risk on your register can be contained after the fact. This one can’t be walked back.

The next four weeks, as they actually unfold

Nothing here looks like an incident.

4:40pm. The paste. No error, no warning, no record. If you went looking tomorrow you’d find nothing, because the logs that would have caught it mostly watch your own systems, and this left through a browser.

4:44pm. A useful summary. People skip past this part, and it’s the part that matters: the tool worked. She isn’t careless. She’s under pressure and she’s good at her job, and it just gave her back ninety minutes. She’ll do it again on Monday, and she’ll mention it to a colleague, because that’s what capable people do with something that works.

Friday, 9:00am. The client review goes well. The summary was accurate. Everyone’s pleased.

Three weeks later. Two more people are doing it. One of them has started pasting draft advice, because the tool’s good at tightening prose. Nobody’s told you. Nobody thinks of it as a thing that would need telling.

The following quarter. Your professional indemnity renewal asks whether client information has been disclosed to any third party. Someone in your firm answers no. That answer’s given in good faith, and it’s wrong.

Eventually. A client asks, in the way clients now do, whether your firm uses AI, and what happens to their information when you do. That’s the moment that decides everything, and you’ll answer it with whatever you actually know…

There’s no breach anywhere in that story. No attacker, no ransom note, nothing to report under the notifiable scheme in most versions of it. What happens instead is slower and much quieter. The one thing your firm is really selling stops being true, and nobody in the building can tell you when.

Your clients have already decided

This is the part I’d put in front of your partners rather than your IT provider.

The OAIC surveyed 1,504 Australians in March this year for its Australian Community Attitudes to Privacy Survey. 93% said it’s not fair and reasonable for an organisation to use the personal information it collected to provide a service to train AI models. Trust in AI companies specifically sat at 4%.

Read that one carefully, because it isn’t quite our Thursday afternoon. The survey asked about organisations training AI on personal information, not about an employee pasting a file into a chatbot. But that gap is much smaller in a client’s mind than it is in ours. What she hears is that her information went into an AI system, and 93 out of 100 Australians have already told a government surveyor how they feel about that.

Financial institutions, for what it’s worth, sit third on the trust rankings in that survey at 59%, behind health providers and government. That’s a mandate, not a licence.

The OAIC’s guidance on commercially available AI products, published in October 2024, is unambiguous about best practice: organisations shouldn’t enter personal information, and particularly sensitive information, into publicly available generative AI tools. It also flags the trap most firms miss. Under APP 6, information collected to give someone financial advice can only be used for that purpose, and a client wouldn’t reasonably expect it to be handed to a third-party AI service on the way. That’s the ordinary privacy law your firm already lives under, not a new AI rule waiting to be written.

What actually stops it

Three things, in the order they’re worth doing.

Give people a tool that doesn’t need the workaround. Nobody in this story wanted to break a rule. They wanted eleven pages summarised. A sanctioned option inside your own environment removes the motive entirely, and it’s the only thing on this list people will thank you for.

Stop the data at the door, not in the policy. Microsoft Purview data loss prevention can classify client information and block it from being pasted somewhere it shouldn’t go. Microsoft’s also building the ability to inspect AI interactions at the network layer, which would catch browser use directly. That one’s still in preview as I write this, and the clean licensing path runs through Microsoft 365 E7, or E5 plus Entra Internet Access. Well above where most firms your size sit. So do the controls on the laptops and in the browser now, and put the network version on next year’s list.

Write down what people may do. A paragraph, not a PDF. A rule a second-year can repeat from memory beats a policy document nobody opens. Ours fits on half a page, and it answers the question people are actually asking, which is whether ChatGPT or Claude is confidential enough for this particular document, right now, at 4:40.

Tip: Ask your team, with no blame attached, what they’ve pasted this year. You aren’t running a disciplinary process. You’re establishing a baseline, and you won’t get a truthful answer twice if the first one costs someone their standing.

Everything after that, the notification questions, the client communications, the sequencing, only matters once this has already happened, and we’ve written about that separately. The point of the three things above is never needing it.

One last thing

If you’ve read this and felt a flicker of dread about what your firm has already pasted, that reaction isn’t a sign you’ve been careless. It’s a sign you understand exactly what your firm sells.

You’ve spent thirty years building something whose entire value is that clients tell you things they tell nobody else. Nobody warned you the technology would arrive inside the browser, unlogged and helpful. Nobody warned you it’d be picked up first by your best people, because they’re the ones carrying the most.

You couldn’t have known. You know now, and this is a fixable problem with a boring answer, which in my experience is the best kind to have.

If you want to find out what’s already left the building, we can look, and you’ll get it in writing rather than as a reassurance. Written down is the version your insurer and your board will care about.