This article accompanies Hour 6: Privacy, AI and Cloud Services in our full-day CPD programme on XpertAcademy. Completion of the full one-hour session, including the related learning materials, contributes to the one-hour CPD certificate issued for that session. You can access the course here: CPD Event B: Full-Day AI, Technical Privacy & Emerging Technology Training.
A customer support team starts using a vendor's AI feature to summarise complaint histories and suggest response text. A user reports that one generated summary appears to include another customer's information. Security cannot immediately tell whether the issue came from the source ticketing system, the AI retrieval layer, the vendor model, a logging defect, a permission problem or a hallucinated output. The vendor says it is investigating.
The organisation now needs answers quickly. Was personal data exposed? Which customers were affected? Can the issue be contained? Who has the logs? Is this a personal data breach? When did the controller become aware? What must the processor tell the controller? What evidence should be preserved? Should the feature be paused?
These questions are hard to answer during an incident if nobody agreed the ownership model before go-live.
This article is general guidance, not legal advice on any specific incident, supplier or notification decision. Breach assessment is fact-specific and should consider the nature of the data, affected individuals, likelihood and severity of risk, contractual terms, regulator expectations and local reporting duties.
Cloud AI incident planning should start before the incident, with clear ownership for detection, log access, notification, containment and review.
Why cloud AI incidents are not always obvious
Traditional cloud incidents often involve familiar patterns: unauthorised access, misdirected email, lost device, compromised account, exposed database, failed deletion or vendor breach. Cloud AI can involve those same issues, but it can also create less familiar incident patterns.
An AI feature may expose unintended personal data through an output, retrieve information from a restricted source, retain prompts in logs, send data to an unexpected model provider, use support content for quality review, misapply permissions, surface metadata from a document the user should not see, or produce a confident but inaccurate summary that affects a person.
Not every poor AI output is a personal data breach. Not every hallucination is a security incident. But privacy teams need a route for distinguishing model behaviour, source-data issues, access-control failures, excessive logging, vendor incidents and actual unauthorised disclosure.
The DPC's cloud guidance highlights security, control, transparency, location, written contracts and breach procedures for cloud providers. For AI-enabled cloud services, those points need to be translated into operational incident questions: who detects, who investigates, who accesses logs, who decides containment, who notifies whom, and who updates the assessment after lessons learned?
Ownership cannot wait for the first incident
Incident ownership is often assumed until it is tested. The business owner assumes security will investigate. Security assumes the vendor will have logs. The vendor assumes the customer can identify affected records. Legal assumes privacy will make the notification decision. Privacy assumes product or operations can pause the feature. Nobody has the whole chain.
For cloud AI, the ownership model should be agreed during due diligence and kept with the vendor evidence pack. It should identify:
- the internal business owner for the AI use case;
- the technical owner for integrations and configuration;
- the security owner for detection and containment;
- the privacy owner for breach assessment and regulator or individual notification advice;
- the vendor owner for contractual escalation;
- the vendor contact route for urgent incident investigation;
- the log owner for source systems, AI platform logs and vendor logs;
- the decision-maker for pausing, restricting or disabling the feature.
The point is not to create bureaucracy. It is to avoid losing the first 24 hours to a search for the person who can answer basic questions.
Logs are evidence, not decoration
AI logging and monitoring need careful design because logs can both reveal and create privacy risk.
Logs may be needed to investigate what happened: user identity, prompt, source document, retrieved passage, output, model call, timestamp, configuration state, permission result, admin action, support access, export event or deletion action. Without those logs, it may be impossible to identify affected individuals, determine whether data was disclosed, or decide whether notification is required.
At the same time, logs can contain personal data, confidential content, credentials, special category data or sensitive complaint details. Excessive logging may increase the impact of an incident. Long retention may create unnecessary exposure. Poorly controlled access to logs may create a separate risk.
The privacy review should therefore ask for a logging position before approval. What is logged? Can prompts and outputs be logged? Are source snippets captured? Are logs encrypted? Who can access them? Are admin and support actions logged? How long are logs retained? Can logs be exported for investigation? Are logs stored in the same region as service data? Do subprocessors process them? Can the customer restrict log content or retention?
This is where cloud, AI and transfer governance meet. A service may store customer records in one region but send logs or telemetry elsewhere. For higher-risk deployments, the transfer evidence should cover logs and monitoring data, not only primary storage.
Controller and processor notification routes
The EDPB controller and processor guidelines remain relevant during incidents because notification duties depend on roles and facts. A processor must notify the controller without undue delay after becoming aware of a personal data breach. The controller then needs enough information to assess risk, decide whether supervisory authority notification is required, and decide whether affected individuals must be informed.
In cloud AI services, role mapping may be more complicated. A vendor may be a processor for the customer's AI workspace, a controller for separate security monitoring purposes, and a processor or controller in relation to support logs depending on the facts. The incident playbook should not wait until an incident to discover that the contract and operational reality are unclear.
The vendor contract should set out incident notification channels, timescales, minimum information, update cadence, evidence preservation, cooperation duties, subprocessor escalation, log access, root cause analysis and remediation commitments. It should also distinguish between general service incidents, security incidents, personal data breaches, AI safety events and product defects where the terminology matters.
Worked example: mapping the incident route
The organisation uses a vendor AI feature inside a customer support platform. The AI summarises ticket histories and suggests draft responses. It is configured to retrieve information only from the customer's own tickets and approved knowledge base articles. Customer data includes contact details, complaint details, order history and occasional sensitive disclosures.
The incident starts with one agent reporting that an AI-generated summary appears to include another customer's account note. The first facts are narrow: one output, one user, one ticket, one timestamp and one suspected cross-customer reference. The vendor has not confirmed a platform issue.
The unknowns are material. The team does not know whether the output was generated from an incorrectly linked source record, a permissions defect, a retrieval bug, a cached embedding, a model hallucination, a prompt containing another customer's text, a support action, or an issue in the vendor's logging and monitoring layer. It does not know whether other outputs were affected.
The decision question is:
Can the organisation identify what happened, contain further risk, determine whether personal data was exposed, meet controller/processor notification duties and preserve enough evidence for lessons learned?
The incident route starts with detection and triage. The support team preserves the output, ticket ID, user ID, timestamp and any prompt or source material visible to the user. Security checks internal access logs, ticket history, integration logs and permission changes. The privacy lead opens a breach assessment record but does not assume the conclusion. The vendor owner raises the contractual incident route and requests preservation of relevant vendor logs.
The log review is structured. The team asks what prompt was sent, what documents or records were retrieved, what model endpoint was used, what output was returned, whether any safety or error event was recorded, whether support staff accessed the workspace, whether the relevant logs include other customer identifiers, and whether similar events appear in the same time window.
Containment is proportionate. If the issue appears isolated and the source is a mislinked ticket, the fix may be source-data correction and monitoring. If the retrieval layer ignored permissions, the AI feature may be paused for the affected workspace. If logs show cross-tenant retrieval or vendor platform failure, escalation is urgent and broader containment is needed. If the output is a hallucination with no actual retrieval or disclosure of another customer's data, the issue may still require quality control and user guidance, but the breach analysis may differ.
The controller/processor notification route is recorded. If the vendor is the first party aware of a personal data breach in its processor role, it must notify the customer without undue delay under the contract and GDPR processor obligations. If the customer is first aware of a suspected issue, the customer still needs vendor cooperation quickly enough to make its own notification decision. The record should show when the organisation became aware, what facts were known then, what further information was sought, and when the risk assessment was updated.
The evidence created includes the initial report, preserved output, log request, vendor updates, internal log extracts, containment decision, breach assessment, regulator or individual notification decision if applicable, action log, root cause summary and lessons-learned record. The DPIA or AI assessment is updated if the incident reveals a new risk or weak control.
Escalation would be triggered by evidence of cross-customer exposure, sensitive data, repeated events, inability to access logs, vendor delay, support access from unapproved locations, transfer uncertainty, unclear affected individuals, or any disagreement over who owns notification.
Monitoring should be tied to real risks
Monitoring should not mean collecting every possible event forever. It should be tied to the risks created by the AI use case.
For an AI support assistant, useful monitoring may include unusual retrieval events, access to restricted sources, high-volume exports, permission errors, support access, admin configuration changes, unusual prompt patterns, repeated outputs flagged by users, model provider errors, subprocessor incidents and changes to log retention or location.
For an AI HR tool, monitoring may need to focus on access to employee relations records, special category data, fairness flags, unauthorised managers, output use and escalated decision workflows.
For an AI document assistant connected to a knowledge repository, monitoring may need to test whether permission changes are reflected quickly, deleted files are removed from indexes, embeddings are refreshed and restricted documents are not surfaced through summaries or snippets.
The monitoring design should be documented with the DPIA or assessment record. If monitoring itself processes employee or customer data, privacy teams should also consider transparency, minimisation, access controls and retention.
What the DPO or privacy team should check
During due diligence or pre-go-live review, privacy teams should test the incident route as carefully as the procurement route.
Check the use case and affected individuals. A low-risk drafting aid does not need the same incident model as a customer, employee, health, financial or child-facing AI system.
Check the data categories and sensitivity. Free text, attachments, transcripts and repository content often contain data the business did not expect the AI tool to process.
Check role mapping. Identify when the vendor is processor, when it may be controller, and how that affects notification and cooperation.
Check logging and monitoring. Ask what is logged, where logs are stored, who can access them, how long they are retained, whether they contain personal data and how they can be exported for investigation.
Check incident terms. The contract should cover notification timing, content, updates, evidence preservation, subprocessor escalation, root cause analysis, remediation and customer support.
Check transfer evidence. Logs, telemetry, support access and incident investigation data may involve different locations from primary hosting.
Check containment powers. The organisation should know who can pause an AI feature, revoke an integration, restrict a workspace, disable training or retrieval, or switch off support access.
Check review triggers. Incidents, near misses, user reports, model changes, new subprocessors, support-location changes, permission defects and logging changes should bring the assessment back for review.
Evidence that should exist before and after an incident
Before go-live, the organisation should have an incident ownership matrix, vendor incident contact route, log and monitoring description, breach notification terms, containment playbook, DPIA or screening note, transfer evidence covering logs and support, and a review trigger list.
After an incident or near miss, the record should show the report, timeline, facts known at each stage, preserved evidence, internal and vendor log review, affected data and individuals, containment decision, breach assessment, notification decision, vendor cooperation, root cause, remediation and lessons learned.
The lessons-learned stage matters. If the incident reveals that logs are unavailable, too broad, retained too long, stored in unexpected locations or inaccessible to the customer, that is not only an incident issue. It is a vendor governance issue. The vendor evidence pack, contract schedule, DPIA, training materials and review calendar may all need updating.
What this means for CPD
For Hour 6, the practical learning is that cloud AI governance does not end at approval. DPOs and privacy teams need to know what will happen when the system behaves unexpectedly, exposes unintended data, logs too much, retrieves the wrong source or changes under the organisation's feet.
After reading this article and completing the course hour, a privacy lead should be better able to ask incident-focused due diligence questions, challenge vague logging answers, map notification responsibilities, and create a record that can survive a real incident rather than only a procurement meeting.
XpertDPO supports this through Vendor / Third-Party Privacy Governance, AI Governance and DPIA Lifecycle Support and practical support connecting cloud AI vendor records, DPIAs, transfer evidence, incident playbooks and board-level assurance.
The aim is not to predict every AI failure. It is to make sure the organisation knows who will act, what evidence will be available, and how the privacy decision will be made when something goes wrong.
This article is intended to support the learning covered in Hour 6 of our XpertAcademy CPD programme. The relevant CPD certificate is issued for completion of the full one-hour session on XpertAcademy, rather than for reading this article on its own. You can return to the course here: CPD Event B: Full-Day AI, Technical Privacy & Emerging Technology Training.