Data Redaction Software 2026: How to Test PII Removal Before Using AI

Published on August 13, 2026 by Steven Jones

Data redaction software can remove personal, financial and confidential information before a document reaches ChatGPT, an AI agent, a retrieval system or another external service. The difficult part is proving that the information has actually gone.

A polished black box over a name is not enough. The original value may still exist in a PDF text layer, OCR output, document metadata, comments, hidden spreadsheet content or a processing log. At the same time, aggressive AI redaction can remove so much context that the document becomes far less useful to the model receiving it.

The right test therefore measures two separate capabilities: can the software find sensitive information, and can it permanently remove every copy that should not survive? Our proposed benchmark covers digital and scanned PDFs, Word documents, spreadsheets, tables, headers and footers, metadata, OCR, obvious identifiers and contextual information that simple pattern matching tends to miss.

Hard rule: if a supposedly redacted value can still be recovered from the exported file, the tool has failed regardless of how impressive its AI detection looks.

If the immediate concern is uploading business documents to a chatbot, our guide to whether ChatGPT is private covers the separate questions of model access, retention and account settings. Redaction should reduce the data you disclose in the first place rather than compensate for an unsuitable deployment.

The first buying mistake is treating detection and removal as one capability

AI redaction products often make detection the visible part of the demo. Upload a document, wait for coloured boxes to appear around names and account numbers, then approve the suggestions.

That demonstrates only one part of the workflow.

Capability	What it should do	Failure to test for
Discovery	Find potentially sensitive content	The system never sees text inside scans, tables, headers or embedded objects
Classification	Decide what type of sensitive information was found	A project name, medical detail or customer note is missed because it has no standard identifier
Redaction	Remove the selected content from the usable document	A visual rectangle is added while the original text remains underneath
Sanitisation	Remove hidden information that should not travel with the output	Metadata, comments or other concealed data survive the export
Verification	Prove the unwanted value cannot be recovered	The workflow assumes a successful export means a successful redaction

It also helps to separate redaction from neighbouring security products. Sensitive-data discovery tells you where risky information exists. DLP decides whether information is allowed to cross a boundary. Redaction changes the content itself. A mature AI workflow may use all three.

---

---

Build the test corpus around file failure modes, not vendor entity lists

A vendor saying it detects names, addresses, phone numbers and bank details tells you very little about how it handles your documents. A better trial corpus deliberately creates different ways for sensitive information to hide.

Test file	What to include	What the test exposes
Digital PDF	Names, emails, IDs and account details in selectable text	Basic detection and destructive text removal
Scanned PDF	The same information inside page images	OCR coverage and the handling of image-based documents
Poor scan	Rotation, compression, faint print and uneven backgrounds	Whether detection fails safely when OCR quality drops
Word document	Body text, comments, headers, footers and tracked changes	Whether the tool inspects more than the visible main document
Spreadsheet	Visible cells, hidden sheets, hidden rows, formulas, notes and headers	Whether workbook structure is inspected rather than treated as a flat page
Tables	Names and identifiers split across merged cells or adjacent columns	Layout and context handling
Metadata test	Author, title and custom document properties containing synthetic identifiers	Hidden-data sanitisation
Contextual text	Medical details, customer notes, internal project names and indirect identifiers	Whether detection goes beyond predictable patterns
False-positive bait	Public company names, ordinary numbers, common names and legitimate reference codes	Reviewer burden and over-redaction

Use synthetic data rather than real customer information. Give every test value a unique canary string so you can search the finished file for an exact known value later.

Use hard failure gates before comparing AI accuracy

A single blended accuracy percentage is a poor metric for selecting redaction software. Some failures are inconvenient. Others invalidate the tool’s entire purpose.

We would apply three hard gates before comparing softer metrics such as speed, interface quality or false-positive rate.

1. Destructive removal: selected values must no longer exist in the exported document structure.
2. No secondary leakage: the same values must not survive in metadata, comments, hidden layers, OCR text, annotations or other attached content.
3. Acceptable handling of the original: the unredacted source must not be copied into an uncontrolled storage, logging or processing path merely so that the redaction product can protect it later.

If a product fails one of these tests, a high detection score should not rescue it.

PDF redaction fails in places the page preview cannot show

PDFs are especially deceptive because the visible page is only one representation of the file. Text objects, images, annotations, form fields, OCR output, attachments, document properties and other structures can exist behind what the reader sees.

This is why drawing a rectangle over a paragraph is not a meaningful redaction test. A person sees the rectangle. A parser may still receive the text underneath it.

After exporting a redacted PDF, verify it locally before sending it to another cloud service. Try selecting and copying around each redacted region. Extract the document text and search for every synthetic canary. Inspect the metadata. Search annotations and form content. If the document originally contained scans, run OCR against the finished output as an additional check.

The UK Information Commissioner’s Office also advises organisations to check documents for hidden personal information before disclosure. Its guidance on disclosing documents securely is a useful operational reference because the problem extends beyond the visible page.

Scanned PDFs expose the OCR order-of-operations problem

Scanned documents are where an automated redaction product can save substantial review time, but they also introduce another failure path.

Imagine a page where a bank account number is rendered in pixels. OCR first creates a hidden machine-readable version of that number. If the software then places a black rectangle over only the page image, the account number may disappear visually while surviving inside the OCR text.

The correct implementation must deal with both representations. This is why the sequence of OCR, detection, redaction and reconstruction matters as much as the model used to identify PII.

Do not test anything other than clean 300 DPI scans. Add rotated pages, faint photocopies, stamps, handwritten additions, multi-column layouts and text sitting close to logos or table borders. More importantly, check what happens when OCR confidence collapses. A safe system should flag uncertain pages for review rather than silently mark the job as complete.

Word and Excel need different leak checks from PDFs

A redaction engine that performs well on PDFs does not automatically understand an Office document.

For Word files, include sensitive data in comments, headers, footers, text boxes and tracked changes as well as ordinary paragraphs. The exported document should then be searched again after redaction. If the tool converts Word files to another format, assess whether any loss of formatting creates a practical problem for the intended AI workflow.

Spreadsheets deserve their own corpus. Put synthetic PII inside hidden worksheets, hidden rows and columns, cell notes and formulas as well as visible cells. A tool that only processes the rendered sheet can miss information that an AI data-analysis workflow later imports directly from the workbook.

For high-risk use, validation should inspect the resulting file structure rather than relying on screenshots of the finished workbook.

Contextual PII is harder than recognising familiar patterns

Credit-card-style numbers and email addresses are useful baseline tests because patterns help identify them. They should not dominate the benchmark.

Real business documents contain information whose sensitivity comes from context. A customer-service note may describe a health condition without including an NHS number. An internal investigation can identify someone through a job title, office location, and event date, even after their name is removed. A confidential project codename may be commercially sensitive, even if it is not personal data.

This is where contextual AI classification can improve on deterministic pattern matching, but it also creates false positives. The benchmark needs benign bait alongside sensitive content so that you can measure what reviewers have to put back.

Do not ask only, “How much did it catch?” Ask, “How many suggested redactions required a human decision, and how many sensitive items did that review still miss?”

Redaction can make an AI prompt safer while making it less useful

Complete deletion is not always the best transformation before AI processing.

Suppose an analyst wants an AI model to summarise a dispute between three customers. Replacing every name with the same black bar destroys the relationships between them. The model may no longer understand who said what.

A better workflow can use stable synthetic tokens such as CUSTOMER_17, CUSTOMER_42 and ACCOUNT_A. Repeated references remain consistent, while the mapping back to the real identities stays inside the organisation.

This is pseudonymisation rather than permanent anonymous removal, so the mapping itself still needs protection. Operationally, however, it can preserve far more AI utility than deleting every identifier without context.

Your benchmark should therefore test useful output after protection, not privacy in isolation. Ask the same approved AI task of the original synthetic document and the transformed version. Compare whether relationships, chronology, totals, and other necessary meanings are preserved without restoring the sensitive values.

Cloud redaction can move the sensitive file before it protects it

There is an architectural problem that many product comparisons miss. If the reason for redaction is that an unredacted file must not leave your environment, a cloud redaction service may receive exactly that file before it can protect anything.

That does not automatically make cloud redaction unsuitable. It changes the procurement question.

• Where is the original processed?
• Is any copy retained after the job is completed?
• Do application or support logs contain extracted sensitive values?
• Can staff access customer files for troubleshooting?
• Are third-party models or subprocessors involved in detection?
• Can the system run on your own infrastructure or in a private cloud, if required?
• Can administrators prove deletion rather than simply request it?

The same issue appears in agentic systems. An agent that retrieves an entire unredacted document from a repository and only removes PII immediately before producing its final answer has already exposed the raw content to earlier model calls, tools and traces. Our AI agent security guide explains why sensitive-data controls need to be placed before privileged model and tool access, rather than only at the output.

How the main redaction software classes fit different workflows

There is no useful single ranking until the workflow is defined. Current products range from human-operated PDF editors to enterprise discovery platforms and developer components that sit directly inside an AI pipeline.

Representative option	Product class	Best reason to test it	Main benchmark question
Adobe Acrobat Pro	Professional PDF editor	Human-controlled PDF redaction and sanitisation baseline	Can reviewers permanently remove selected content and hidden information without damaging the document?
Foxit Smart Redact	AI-assisted PDF redaction	Automating sensitive-data discovery across PDF workflows	How much manual review remains once OCR, contextual PII, and false positives are accounted for?
Redactable	Cloud document redaction platform	Dedicated automated document review	Do file coverage, exported-file verification, and source-document handling match your security policy?
CaseGuard	Document and media redaction platform	Organisations handling documents alongside images, audio or video	Does its broad media workflow still provide the document-level verification your use case requires?
PII Tools	Enterprise discovery and redaction	Large repositories and mixed structured or unstructured data	Can it find, classify and remediate sensitive information without creating an unmanageable review queue?
Azure AI Language PII	Developer API and native document processing	Building PII detection and redaction into an application pipeline	Are supported formats, output reconstruction and downstream logging appropriate for your architecture?
Microsoft Presidio	Open-source de-identification components	Teams that need local control and custom recognisers	Can engineering effort, custom detection and OCR handling reach the required recall without turning into a maintenance project?

The names in this table are starting points for testing, not DIY AI scores. A tool should not be declared the winner based on its feature list. The same product can be excellent for human-reviewed legal PDFs and a poor fit for automated spreadsheet sanitisation inside an AI pipeline.

Your search, your sources

Make DIY AI a preferred source

See more of our reporting in Google Top Stories, AI Overviews and AI Mode.

Add DIY AI on Google

Price per seat is the wrong redaction cost metric

A cheap licence can become expensive if every file needs ten minutes of correction. A more expensive automated product can be economical at high volume if reviewers only need to inspect uncertain cases.

For a serious trial, calculate:

Cost per accepted document = software cost + processing cost + reviewer time + rework + infrastructure, divided by the number of documents that pass verification.

This exposes several hidden trade-offs. Seat-based desktop software can work well where volume is modest and human review is already part of the process. Usage-based APIs suit variable automated workloads but make inefficient preprocessing expensive. Self-hosted software may avoid external data transfer while shifting costs to infrastructure, updates, and engineering ownership.

Reviewer time is often the variable worth watching most closely. Record both false negatives and false positives, plus the number of documents that require manual reconstruction because formatting or OCR failed.

A practical data redaction software trial

1. Create synthetic canaries. Use fake names, IDs, bank details, addresses, project names and sensitive free text that can be searched exactly after processing.
2. Distribute them across file structures. Put data in ordinary paragraphs, scans, tables, headers, comments, metadata and hidden spreadsheet content.
3. Run the product with realistic defaults. Do not over-configure the first test just to make the tool look good.
4. Record every detection decision. Separate missed sensitive content from harmless text incorrectly marked for removal.
5. Apply the redactions and export a new file. Preserve the original synthetic source for comparison.
6. Verify the output locally. Search extracted text, metadata, annotations, hidden workbook content and OCR results for every canary.
7. Test document utility. Confirm that the layout, formulas, tables, chronology and entity relationships needed by the downstream AI task still work.
8. Inspect the redaction system itself. Check retention, audit records, temporary storage and logs for copies of the source values.
9. Measure reviewer effort. Record correction time per accepted document rather than simply processing speed.
10. Repeat with degraded inputs. Rotate scans, lower image quality, and add ambiguous contextual information to test failure handling as well as ideal conditions.

These failures should disqualify a redaction product

• A redacted synthetic value remains recoverable from the exported file.
• OCR fails without warning, and the document is still marked as successfully processed.
• Hidden metadata or comments retain values that were removed from the visible page.
• The redaction log itself stores the full removed value without any appropriate protection.
• The product cannot explain where the original unredacted file is processed or retained.
• Unsupported structures are silently skipped rather than flagged for review.
• There is no practical way to distinguish automated suggestions from manually verified redactions.

These are more useful procurement gates than a long checklist of compliance badges or supported PII categories.

FAQs about data redaction software

What is data redaction software?

Data redaction software identifies sensitive content and removes or transforms it so that a safer version of the document can be shared or processed. Products range from manual PDF editors to AI-assisted document platforms, enterprise discovery systems and developer APIs.

Is putting a black box over PII enough?

No. The decisive test is whether the underlying value has been removed from the file, not whether it has disappeared from the page preview. Visual overlays can leave machine-readable information intact.

Can AI automatically redact every piece of PII?

No automated detector should be treated as guaranteed to find every sensitive value. Pattern-based identifiers, names, contextual information, poor OCR and organisation-specific secrets create different detection problems. High-risk workflows should include verification and a path for uncertain cases to reach a reviewer.

Should documents be redacted before uploading them to AI?

If the AI task does not need the original sensitive information, removing or pseudonymising it before transmission reduces unnecessary exposure. Redaction does not replace appropriate AI account controls, retention policies, access management or DLP, but it can reduce the amount of sensitive information those controls have to protect.

What is the difference between redaction and pseudonymisation?

Redaction removes information that should no longer be present. Pseudonymisation replaces it with another identifier while preserving a separate mapping. For AI workflows, stable pseudonyms can preserve useful relationships between people or records, but the mapping must remain protected.

The practical verdict: verify the exported file, not the demo

The strongest data redaction software is not necessarily the product that highlights the most PII during a sales demo. It is the one that reliably detects the information your organisation cares about, permanently removes what should disappear, preserves the context the downstream task still needs and gives you a defensible way to verify the result.

For mostly manual PDF workflows, start by benchmarking professional PDF redaction tools against a dedicated AI-assisted option. For large document estates, test enterprise discovery and redaction platforms against repository, OCR and review-volume requirements. For automated AI applications, compare APIs and self-hosted components at the exact point where data enters the model pipeline.

Whatever category you choose, keep the same acceptance rule: if the supposedly removed information survives somewhere a parser, person, or downstream system can recover it, the redaction has not passed.