What It Means to Mask PII Before HoverBot Answers

“Send the return label to shopper@example.com.” That is the sort of message a visitor can type before a support bot has asked for an address. In a local test of HoverBot's masking path on November 2, Presidio changed it to “Send the return label to <EMAIL_1>.” before the reply step. The address is synthetic and uses the reserved example.com domain. It identifies no shopper. This is a concrete way to mask PII before a chatbot replies.

How do we mask PII, or personally identifiable information, in a chatbot? The practical answer is to inspect the message after detection and replacement, at the boundary where the reply model receives it. A policy that merely tells the model to avoid repeating private details comes too late: the original details have already entered the model request. We use a self-hosted Presidio analyzer and anonymizer for that earlier step. The test below shows where that design worked and where it did not.

Before: Send the return label to shopper@example.com.

After: Send the return label to <EMAIL_1>.

Before: I pasted test card 4111 1111 1111 1111 into chat.

After: I pasted test card <CREDIT_CARD_1> into chat.

Mask PII before a chatbot reply, not after it

The first operation asks the detector which spans look sensitive. The second swaps those spans for labels that retain enough shape for the conversation to make sense. “Send the return label to <EMAIL_1>” still tells the bot that the shopper wants a label. It does not supply the email address to the reply model. The original message may still exist in the application's own conversation record for authorized support work; masking a model request is not the same as erasing all stored data.

Presidio's analyzer documentation explains that recognizers combine patterns, language models and context. Its anonymizer replaces the spans those recognizers report. We chose this separation because the detector can run in our own deployment before the request goes to an external reply model, while the reply can keep the surrounding question. The choice has a trade-off: the detector only protects the types we ask it to find, at the confidence we choose, and it can still miss one.

What six synthetic messages showed

We sent six separate, fictional messages through the local HoverBot masking service using the same selected categories and threshold as the reply path. Two target strings were replaced: the email and the test card number. A phone-shaped string was not replaced. Three other messages, carrying a name, street address and order number, remained unchanged. Those last three are not a surprise result: this reply path does not ask Presidio to detect names, locations or order numbers, to avoid turning ordinary product and order context into meaningless labels. The phone miss matters differently because phone numbers are among the selected types.

These are six probes, not a protection rate. They are too few to estimate performance across languages, countries, spelling styles or real conversations. The exercise ran against a local service and inspected the text after masking. It did not inspect a production model request or a real customer's data. The useful finding is more modest: the exact boundary can be tested, and a green detector service does not mean each intended field was caught.

The ordering also matters to the visitor. If the bot needs an order number to answer “Where is my order?”, replacing every number would break the task. If it needs the customer's name or address for a human handoff, the support workflow must still handle that information with appropriate access and retention controls. Masking is one boundary in a larger design, not a substitute for deciding what the bot should collect in the first place.

A miss is a product decision to investigate

Presidio's own overview warns that automated detection cannot guarantee it finds every sensitive detail. The missed phone probe is why we show the negative case as well as the successful email. A team should decide whether a missed type makes the answer unsafe, whether the bot should ask the visitor to use a secure form instead, and whether the message should go to a person. The right answer depends on what the chat is allowed to do, not on the appeal of a clean masking screenshot.

OWASP's sensitive-information guidance treats data that reaches a language-model application as part of its disclosure risk. We agree with the direction: minimize what reaches the model, limit what can come back, and give operators a way to review failures. We would not call any chatbot “PII safe” because it masked an email in a demo. We would ask for the before-and-after evidence across the information people actually type.

Run the same check on your bot this week

1. Write five to ten synthetic messages that resemble your own support questions, including a contact detail, a payment detail and an order question. Use reserved domains and test numbers, never customer records.
2. Capture the text immediately before it goes to the reply model. Compare it with the original message, and mark each intended sensitive span caught or missed.
3. Include cases you want to preserve, such as a product code or order reference. Record false replacements as well as missed sensitive data.
4. Repeat the check after changing recognizers or thresholds. Have a person review any type whose miss could create harm, and route those conversations accordingly.

If you want to see a chatbot answer from a fictional business's approved knowledge, try the HoverBot demo hub. Test your own data boundary separately: the visible answer cannot tell you everything the model was sent.