Shadow AI in Customer Support: What Happens When Staff Paste Chats into Personal Tools?

Shadow AI in customer service means an operator uses an AI tool outside the approved set: a personal chatbot account, a browser extension or an app on their phone. The trouble is that the conversation travels along with the prompt: the customer's name, an order number, the details of a complaint. Your company then has no idea where that data is stored, who can access it or whether the provider may use it for its own purposes.

This guide is about your own staff, not customers. For customers who share data in chat that they should not, see the article on sensitive data in chat. This is not legal advice; agree the details with whoever is responsible for data protection in your company.

Shadow AI in support: where it comes from and what you risk

Shadow AI rarely starts with bad intent. It usually follows queue pressure, the lack of an approved tool or a vague rule along the lines of "be careful with AI". A ban without an alternative pushes that use somewhere nobody can see it, but it does not make it go away.

The risk has several layers. The first is data: a customer conversation almost always contains information about a specific person, even without a surname. The second is the terms of service: a personal account is governed by terms the employee accepted, not by your company's contract. The third is quality: an answer from a tool that does not know your offer can sound confident and still be wrong. The fourth is the missing trail: when a customer disputes a reply, nobody knows where the contested sentence came from.

In its checklist for LLM-based chatbots from November 2023, the Hamburg data protection authority recommends organisation-owned accounts instead of personal ones, written usage rules and no personal data input when the provider may process it for its own purposes. It is guidance from a German regulator, not law in every country, and the tool settings it describes may have changed since.

Take stock: ask your team instead of watching them

Before writing rules, find out what people actually use and for what. The simplest way is a short survey or a team meeting conversation, making clear that the answers are for choosing tools, not for judging employees. Do not install monitoring software on personal phones and do not look into private accounts.

Ask about tasks, not brands. "What did you last use AI for when replying to a customer?" tells you more than "Do you use ChatGPT?". The example below is for a fictional online shop with a five-person support team.

↔ Scroll the table sideways to see all columns.

Every row marked "on hold" means the team has a real need without a good answer. If summaries are needed every day, a ban alone will not work; you have to offer a way of working that delivers a similar benefit.

One-page rules: what is allowed, what is not and whom to ask

Rules have to be usable in the middle of a conversation, without digging through documents. The Hamburg checklist advises giving specific examples of permitted and prohibited uses, because a general "use it sensibly" does not help an operator five minutes before the end of a shift.

• Tools approved for conversation content, used via a company account, not a personal one.
• No pasting customer data, order numbers, attachments or whole threads into any other tool.
• An example of permitted use without data: "Suggest a polite sentence declining a return after the deadline".
• An example of prohibited use: "Reply to Anna from order 4821, who writes the following...".
• A duty to check facts, prices and dates before sending, whichever tool produced the draft.
• A channel for reporting mistakes and a contact person for data protection questions.

The same boundary applies to material the team wants to add to the assistant's knowledge. Documents with data about specific customers belong neither in a personal chatbot nor in a shared knowledge base; the guide on what data belongs in an AI chatbot knowledge base explains how to choose sources.

A safe procedure for drafting replies

Rules work better when a ban comes with a simple, approved path. The procedure below assumes your company has an approved tool with a work account and knowledge based on your own sources.

1. Name the purpose of the reply.State in one sentence what the message needs to resolve. That reduces the temptation to paste the whole conversation "for context".
2. Use the approved tool in your inbox.The draft is created where the conversation happens, under company permissions and the contract with the provider.
3. Check facts and promises.Compare prices, dates and completed actions with the source or system before you leave them in the text.
4. Edit and send manually.The operator is responsible for the whole message, including sentences added after the draft was inserted.
5. Log the knowledge gap.When the tool lacked information, pass the topic to the source owner instead of looking for an answer in a personal app.

In Meet AI, Copilot prepares a private suggestion for the operator who is handling the conversation and has permission to reply. The suggestion bases its facts on the website's active knowledge and never sends itself: the operator inserts it into the editor, edits it and sends it manually. During installation an administrator accepts information about the model provider and the transfer of conversation content, so the decision is made by the company, not an individual employee. That does not replace your own compliance assessment. The guide on approving AI Copilot replies covers the daily review of such drafts. What a business should check before accepting a model provider is covered in our GDPR checklist for AI in customer service.

When a conversation has already reached a personal tool

Reporting has to be simple and must not automatically mean punishment. Employees should know that their job is to inform the named person quickly, not to decide whether a breach occurred. Assessing the impact and any obligations towards the supervisory authority is the task of whoever is responsible for data protection.

The report describes the type of data, not its content. Do not paste the customer's confidential message again into an email, the team messenger or a note, because that creates yet another copy. If the reply from the personal tool contained an error, prepare the correction for the customer using the guide on correcting a wrong answer.

Training instead of a one-off email

Rules read once, on the day a contract is signed, are quickly forgotten. A short conversation about concrete examples works better: show two drafts, one with customer data and one without, and ask which may be pasted into which tool.

According to the European Commission's questions and answers on AI literacy, Article 4 of the AI Act has applied since 2 February 2025, does not require a certificate and also covers employees who use tools such as ChatGPT at work. It does not set a training length or decide whether a particular company complies.

Include the topic in a new colleague's first week, alongside the rules on customer data. The guide to onboarding a new operator describes such a start. Keep a simple record of who went through the rules and when; the Commission notes that an internal record of training can take the place of a certificate.

Questions about shadow AI in a support team

Is it enough to ban personal AI tools? A ban on its own usually moves the use out of sight. The Hamburg checklist warns that without written rules you have to assume employees will use such tools in an unauthorised and uncontrolled way. A ban works better combined with an approved alternative, examples and a simple reporting channel.

Is removing the name enough before pasting a conversation? Not always. The same document points out that removing names and addresses is not enough, because a person can be identified from context. In support, that is often an order description, a place and a time. If an excerpt describes a specific customer's case, treat it as personal data and work with it only in the approved tool.