Data anonymization: techniques and tools

·TL;DR

• Data anonymization transforms personal data so an individual cannot be re-identified, irreversibly. Done right, the result is no longer personal data under GDPR.
• Pseudonymization is not anonymization. It swaps identifiers for reversible tokens, so the data stays personal data and stays in legal scope.
• The core anonymization techniques: masking, generalization, suppression, perturbation (noise), k-anonymity, differential privacy, pseudonymization, and tokenization.
• The tradeoff is fixed: anonymization destroys detail to remove identifiability; pseudonymization and tokenization keep detail but keep the data personal.
• Tools landscape: Microsoft Presidio (Python, PII detection), ARX (Java, statistical de-identification), and Anonde for the LLM case: tokenize before the model, reveal inside your trust boundary.

01What is data anonymization?

Data anonymization is the process of altering personal data so that the people in it can no longer be identified, directly or indirectly, and the change cannot realistically be undone. The test is re-identification risk. If someone with reasonable effort and available data can still single out an individual, the data is not anonymized.

This matters legally. Under GDPR, truly anonymized data is out of scope: it is no longer personal data, so the regulation does not apply to it. That is the prize, and also why the bar is high. Recital 26 of GDPR makes clear that data is only anonymous when re-identification is no longer reasonably likely, accounting for the cost, time, and technology available.

One line of contrast: anonymization is irreversible, while pseudonymization is reversible. Pseudonymization replaces an identifier with a token but keeps a key that can restore it, so the data stays personal data. We cover that split in depth in anonymisation vs pseudonymisation.

02The main anonymization techniques

There is no single anonymization tool that fits every case. There is a toolbox of anonymization techniques, and the right choice depends on what you need to keep useful. Here are the main ones, with concrete before/after.

Masking. Hide or replace characters in a value. The shape stays; the content goes.

4716 8923 0011 4567 becomes **** **** **** 4567

Generalization. Lower the precision of a value so it covers a range or category instead of a point.

age: 34 becomes age: 30-40; ZIP: 94107 becomes ZIP: 941**

Suppression. Remove the field or record entirely. Blunt, but the safest when a value is too rare to keep.

diagnosis: rare_condition becomes diagnosis: [REMOVED]

Perturbation (noise). Add small random changes so individual values shift but aggregate statistics stay close to true.

salary: 92,000 becomes salary: 93,150 (noise added per record)

k-anonymity. A property, not a transform. A dataset is k-anonymous if every record is indistinguishable from at least k-1 others on its quasi-identifiers (age, ZIP, sex, and similar). You usually reach it with generalization and suppression. With k=5, any combination of quasi-identifiers maps to at least 5 people, so no single row stands alone. Its known weakness: if all k people share the same sensitive value, you learn it anyway. Extensions like l-diversity and t-closeness exist to patch that.

Differential privacy. A formal guarantee, introduced by Dwork and colleagues in 2006. You add calibrated noise to query results so that whether any one person is in the dataset barely changes the output. The strength is tunable through a privacy budget (epsilon): smaller epsilon means more noise and more privacy. It is the strongest definition here because it bounds risk mathematically rather than by inspection. It is also the hardest to apply without hurting accuracy on small datasets.

Pseudonymization. Replace direct identifiers with tokens, keeping a mapping so you can reverse it. "Jane Roe" becomes [PERSON_1]. Reversible, so it is not anonymization, but it shrinks exposure and is explicitly encouraged by GDPR as a safeguard.

Tokenization. A specific form of pseudonymization. Swap a value for a token and store the token-to-value pair in a separate vault. jane@acme.com becomes [EMAIL_2], with the real address living only in the vault. The output carries no recoverable secret; reversal is a lookup, not a decryption.

03Techniques compared

The split runs down the "Reversible?" column. Masking through differential privacy are irreversible anonymization. Pseudonymization and tokenization are reversible, so the output is still personal data.

04How to choose

Start with one question: do you ever need the real value back? If no, use irreversible anonymization and pick by what you must keep useful. If yes, you want pseudonymization or tokenization, and you accept that the data stays in legal scope.

• Publishing a dataset to outsiders. Anonymize. Use k-anonymity for tabular data, or differential privacy if you are releasing statistics and want a formal bound.
• Analytics inside your team. Generalization plus perturbation usually keeps the numbers useful while cutting identifiability.
• Showing data on a screen or in logs. Masking is enough; you only need to hide the value, not analyze it.
• Sending data to a system that returns a result you must merge back. Tokenization. This includes calling an LLM, a payment processor, or a third-party API.

Name the tradeoff honestly. Anonymization buys you out of legal scope but destroys detail, and aggressive generalization can make a dataset useless. Pseudonymization and tokenization keep every detail and let you reverse the change, but the data remains personal data and must be protected accordingly. There is no option that is both fully reversible and fully out of scope.

05Data anonymization tools

Three open tools cover most of the ground, and they aim at different problems.

Microsoft Presidio. An MIT-licensed Python SDK for detecting and anonymizing PII in text, images, and structured data. An analyzer finds entities with recognizers (regex, rules, checksums, NER); an anonymizer transforms them with operators like replace, redact, mask, hash, and encrypt. It is the reference design for PII work in the Python ecosystem. For a deeper look, see Anonde vs Presidio.

ARX. A Java tool and library focused on statistical de-identification of datasets. It implements k-anonymity, l-diversity, t-closeness, and differential privacy, with a desktop UI for tuning the privacy-utility tradeoff. If your problem is releasing a tabular dataset with a measurable risk model, ARX is built for exactly that.

anonde. An Apache 2.0, Go tool built for the LLM case. It tokenizes PII and secrets in text, JSON, PDFs, and logs before they reach a model, then reveals the real values only inside your trust boundary using a vault that maps each token to its original. Detection uses pattern and recognizer matching plus NER, with GLiNER bundled. In Anonde's own public benchmark it has the lowest leak rate on 29 of 29 corpora: 10.1% rolled up across all corpora, versus 41.5% for Presidio, where leak rate is the share of gold PII spans missed (lower is better).

These are not competitors so much as different shelves of the same toolbox. Presidio for broad PII detection in Python, ARX for statistical dataset release, Anonde for guarding the LLM boundary.

06The LLM-specific angle

LLMs break the usual assumptions. You cannot generalize or mask a prompt the way you would a dataset, because the model needs coherent text to do its job, and you need the real answer afterward. A support ticket with [PERSON_1] and a city of 30-40 would be both useless to the model and impossible to merge back.

Tokenization fits this shape. Replace each value with a stable placeholder before the call, send the tokens, then reveal on return inside your boundary:

1. Tokenize before the model. "Email Jane Roe at jane@acme.com" becomes "Email [PERSON_1] at [EMAIL_2]".
2. Send tokens to the provider. The model only ever sees placeholders, never the real personal data.
3. Reveal on return. Inside your trust boundary, the tokens in the response map back to the originals via the vault.

The model gets coherent, structured text and the right answer; the provider gets no personal data. This is reversible by design, which is exactly what an LLM workflow needs and exactly why static anonymization alone does not solve it. For the full pattern, see how to redact PII before the LLM.

07FAQ

What is data anonymization?

The process of transforming personal data so no individual can be identified from it, with no realistic way to reverse the result. Done correctly, anonymized data falls outside GDPR because it is no longer personal data.

What are the main data anonymization techniques?

Masking, generalization, suppression, perturbation (noise), k-anonymity, differential privacy, pseudonymization, and tokenization. The first six are irreversible; the last two are reversible and keep the data in legal scope.

What is the difference between anonymization and pseudonymization?

Anonymization is irreversible and removes the link to the individual, so the data is no longer personal data. Pseudonymization is reversible: a key or vault can restore the originals, so the data stays personal data and in scope for GDPR.

What are the best data anonymization tools?

Microsoft Presidio for PII detection in Python, ARX for statistical dataset de-identification (k-anonymity, differential privacy), and Anonde for the LLM case: tokenize before the model, reveal inside your trust boundary.

·Try Anonde

See how it works, try the live demo, or read the quickstart to run Anonde on your own infrastructure.

·Sources

• GDPR Recital 26: not applicable to anonymous data
• Dwork et al., differential privacy (2006)
• Sweeney, k-anonymity
• ARX data anonymization tool
• Microsoft Presidio on GitHub
• Anonde on GitHub