Can Sensitive Enterprise Data Be Made Safe for AI Use?

1. Start with purpose and minimisation

Before choosing a privacy technique, define what the AI task needs. “We might use this for AI” is not specific enough to decide which fields are necessary.

For example, a model that classifies equipment faults may need the equipment type, fault description, relevant sensor readings and verified repair outcome. It may not need a customer’s name, personal email address or billing account number. Removing or excluding such fields can reduce exposure without weakening the task.

The UK GDPR data-minimisation principle requires personal data to be adequate, relevant and limited to what is necessary for the stated purpose. The ICO’s AI guidance notes that this obligation still applies when developing AI systems; organisations should assess what information is needed rather than defaulting to “use everything.” ICO guidance on data minimisation in AI ICO data-minimisation principle.

Purpose-to-data map

Exhibit 1. Purpose-to-data minimisation map

This is a scoping exercise, not a universal field-removal rule. A location or timestamp might be essential to a particular safety analysis; in another task it may create unnecessary identification risk. Document why retained fields are needed and reassess them for each use.

A purpose specification needs to go beyond 'AI research'. The parties should identify whether the recipient will fine-tune a model, run a protected evaluation, retrieve records at inference time or analyse aggregate operational behaviour. Each changes who sees what, for how long and whether examples may enter outputs, model artefacts or downstream systems. A fault classifier may require equipment type, error code and independently verified repair outcome; it may not need contact details. A model of customer communications might genuinely need language features but still have no reason to retain bank-account numbers or hidden credentials.

For personal data, lawful basis, fairness, transparency, purpose limitation and security are related but separate questions. The controller must examine whether the new use is compatible with the original purpose or requires a distinct basis, and whether additional processing conditions apply to special-category material. A contract saying 'licensing permitted' is not a substitute for these assessments. The ICO's data-sharing agreement guidance describes governance practices, but states that an agreement does not itself grant immunity or make the processing lawful. Businesses should also check whether commercial confidentiality and source agreements restrict use after personal data has been removed.

2. Distinguish anonymisation, pseudonymisation and other risk controls

These terms are not interchangeable.

Exhibit 2. Anonymisation, pseudonymisation and associated controls

The ICO’s current anonymisation guidance emphasises that identifiability depends on the data, context, recipient, other information that may be available and the controls used. It describes a “motivated intruder” assessment: consider practical means a determined person could reasonably use to identify someone. It also states that simply removing direct identifiers is insufficient if records can still be linked to a person. ICO anonymisation guidance ICO on assessing anonymisation effectiveness.

Example: the identifier is gone, but the person may not be

Suppose a service record contains:

customer_name
case_id
event_date
small_town
rare_equipment_model
unusual_failure_description
resolution

Removing customer_name does not necessarily prevent identification. A local news report, public post or someone’s prior knowledge could connect a rare failure on a specific date in a small town to a particular person or organisation. The combination of date, location and unusual event can act as quasi-identifiers, even when no single field is a direct identifier.

Possible mitigations could include generalising the date or location, suppressing rare combinations, removing unnecessary narrative detail, separating the linkage key, or restricting access to a controlled environment. Each can reduce risk, but may also remove context needed for the task. The provider needs to assess both sides rather than claim that one transformation guarantees safety.

Identification risk changes with recipient and context

Anonymity cannot be inferred from a universal number of removed columns. An internal employee may recognise a rare maintenance incident through knowledge of the asset, date and location; an external partner may combine the same fields with online material. A controlled recipient with strict query restrictions creates different practical opportunities for identification from an unrestricted published dataset. This means the identity-risk assessment should name realistic adversaries, external reference sources, data subjects, likely harms and the limits of available testing. Anonymisation should be treated as an assessed result for an environment and use, not a file format.

The ICO distinguishes anonymous from pseudonymised personal data, with pseudonymised records generally remaining within data-protection law. NIST SP 800-188 similarly frames de-identification in terms of techniques and governance. Neither source provides a universal guarantee that a tokenisation technique, minimum group size or synthetic generation method prevents disclosure. Risk changes with additional data, new linkage capabilities and shifting access rights; material sharing arrangements should therefore include re-assessment triggers and a stop or remediation procedure.

3. Treat free text, media and model outputs as part of the risk surface

Structured columns are comparatively easy to scan for known identifiers. Sensitive information can also appear in:

• customer emails and support transcripts;
• technician notes and incident narratives;
• scanned forms and attached documents;
• call recordings, screenshots, photographs and video;
• logs containing tokens, URLs or account references;
• model inputs, outputs, embeddings or other derived artefacts.

A free-text note might say, “I visited Jane’s flat on Tuesday after the account holder called from the number ending 4821.” Removing a name field elsewhere does not remove those details. Automated detection can help find likely personal information, but unusual names, contextual clues, quoted text and image/audio content may require sampling and human review.

A practical review should ask:

1. Where can sensitive details occur? Include attachments, metadata and logs—not only the main table.
2. What is needed for the task? Preserve useful signal, but remove unrelated fields and details.
3. What transformations will be applied? Record redaction, generalisation, token replacement and other changes.
4. How will transformation quality be checked? Test samples, including edge cases and rare records.
5. What remains accessible? Restrict the source data, linkage keys, transformed copy and derived outputs separately.

Privacy risk can also extend beyond the delivered source records. The ICO cautions that some AI-related derived information—such as gradients in federated learning—can still reveal personal information, so a different technical representation should not be presumed risk-free. ICO guidance on security and minimisation in AI.

Examine how data travels through model development

A privacy review that examines only the training file can miss other disclosure routes. Reviewers may see unredacted exception queues; feature stores or search indexes may hold recoverable content; observability logs may capture prompts and answers; external evaluation vendors may receive reference solutions; backups and cached results may persist after nominal deletion. If records are used for model adaptation, assess the possibility of memorisation or unwanted reproduction as a risk requiring testing, not as proof that every model will expose its source data.

Derived material does not necessarily fall outside legal or security control because it has a new name. Embeddings may leak attributes in some settings, synthetic examples may be too similar to rare originals, and a summary can repeat a confidential detail. The provider should define whether each artefact can be retained, exported, audited or reused in other products. Controls should follow actual movement of the information rather than the simplistic distinction between 'raw' and 'processed' files.

4. Choose an access model that fits the residual risk

Sometimes the data can be transformed sufficiently for a buyer to receive a copy. In other cases, the useful details are difficult to remove without destroying the task, or the consequences of disclosure are high. The choice is not limited to “send the full dataset” or “do nothing.”

Exhibit 3. Comparison of data-sharing and access models

NIST recommends choosing a data-sharing model as part of de-identification governance. Options it discusses include de-identified data, synthetic data, query interfaces and non-public protected enclaves; it also stresses risk assessment and measurable controls. These are useful design options, not a substitute for jurisdiction-specific legal analysis. NIST SP 800-188.

The controls need to be operational

A contract clause saying “use securely” is not a complete control plan. Depending on the data and use, the parties may need to specify:

• named users, roles and access approval;
• encryption in transit and at rest;
• restrictions on downloads, onward sharing and re-identification;
• logging and review of access or exports;
• retention periods, backup handling and deletion evidence;
• incident notification and investigation;
• permitted model development, evaluation and deployment purposes;
• how corrections, objections or withdrawal requests are handled;
• audit evidence and responsibility for remediation.

Controls should match the actual workflow. If a buyer can export raw data to an uncontrolled environment, a secure workspace elsewhere in the process may not meaningfully limit exposure.

Quantify the eligible population and the cost of protection

Consider a wholly hypothetical 20,000-record archive. A purpose-and-rights screen finds 60% eligible for a restricted evaluation. Of that population, 75% pass a first privacy and quality screen; of those, 90% pass independent validation. The resulting 20,000 × 0.60 × 0.75 × 0.90 = 8,100 candidate records is a preparation forecast, not an estimate of legal compliance or anonymous status. The other records may be omitted, restricted to an alternative task or retained internally; none should automatically be transferred simply to raise volume.

If the 9,000 records remaining after the purpose-and-rights and first privacy/quality screens require £1.80 each in manual review, plus £14,000 in fixed engineering and controlled access, the specified costs total (9,000 × £1.80) + £14,000 = £30,200. Against 8,100 candidates after the final validation step, this is approximately £3.73 per final candidate. If final-validation yield alone falls to 45% while all 9,000 screened records still incur review costs, there would be 4,050 candidates at approximately £7.46 each. This sensitivity holds review workload and fixed costs constant; if the screening or review workload changed, costs would need to be recalculated. That does not prove the dataset is worth £3.73 per record, and the model excludes external counsel, security assurance and incident contingency. It illustrates why additional masking is not always an economic solution. Sometimes a limited evaluation in a protected environment is safer and more practical than repeatedly transforming an unrestricted copy.

5. Make a documented decision—not a blanket promise

A provider should avoid assurances such as “fully anonymous,” “risk-free” or “safe for any AI use” unless those claims have been rigorously assessed and are appropriate to the specific context. Anonymisation is not a magic processing step; its effectiveness can depend on who receives the data and what other information they can reasonably access. The ICO recommends assessing identifiability from the relevant parties’ perspectives and reviewing controls and access over time. ICO anonymisation effectiveness guidance.

Sensitivity decision path

1. Define the task and recipient — What capability is being developed, evaluated or supplied? Who will access the information, including contractors and affiliates?

↓

2. Minimise before transformation — Remove fields and records not needed for that task. Record why any sensitive information remains.

↓

3. Examine identifiability and other harms — Consider direct identifiers, quasi-identifiers, free text, rare events, linkage opportunities and consequences of disclosure.

↓

4. Select the least-exposure workable model — Use a transformed extract, controlled environment, bounded query, synthetic output—or no transfer if residual risk is unacceptable.

↓

5. Test and govern — Validate redaction and transformations; document the assessment, access controls, retention, monitoring and response plan.

Executive review checklist

• Is the purpose specific enough to identify what data is necessary?
• Can the task work with fewer fields, fewer records or a safer representation?
• Have free text, attachments, audio, images and metadata been considered?
• Could records be re-linked using dates, rare events, location or outside information?
• Is the data anonymous in the relevant recipient’s hands, or merely pseudonymised?
• Has someone tested realistic identification and disclosure scenarios?
• Are model outputs and derived artefacts included in the risk analysis?
• Can the buyer use a secure environment or bounded query instead of receiving a copy?
• Are access, purpose, retention, deletion, audit and incident terms workable?
• Who is accountable for reviewing the risk when the dataset, buyer or use changes?

Executive takeaway: The safest useful dataset is often the smallest one that preserves the required signal. When removing or generalising sensitive details would destroy that signal, consider controlled access or a different use model rather than pretending the risk has disappeared.

Make a risk–utility decision that can be challenged

A defensible decision file should record the proposed recipient, precise use, lawful basis where required, source rights, the necessity of sensitive fields, technical safeguards, test results and unresolved risks. It should also document whether the dataset remains useful: does it preserve the diagnostic symptom, decision-time context and outcome, or has generalisation removed the signal? The European Data Protection Board's Opinion 28/2024 addresses assessment of anonymisation and lawful bases in AI-model contexts, emphasising case-specific analysis rather than generic assertions that all trained models are anonymous.

Executive accountability does not end with signed documents. Assign an owner for ongoing access review, complaint and rights requests, data corrections and security incidents. Do not make unqualified promises about recall, deletion from model weights or immunity from re-identification. If the risks cannot be reduced to an acceptable level without destroying the intended evidence, the correct decision may be controlled internal analysis or no external transaction. This is a commercial outcome, not a failure of monetisation strategy.