"We Don't Train on Your Conversations": What That Should Actually Mean

June 17, 20265 min read

Short answer

"We do not train on your conversations" is useful only if the product explains the full data path.

Training is one use of data. It is not the only one. Messages may still be stored, summarized into memory, processed by vendors, used for abuse detection, reviewed after safety flags, or retained for legal and security reasons.

A strong privacy claim should be specific enough that a user can understand what happens after they send a message.

Training is not the whole question

People often hear "not used for training" as "private." That is not always what it means.

A company may avoid using your messages to improve a model while still storing those messages to run the product. It may keep logs, process content through infrastructure providers, or review some content when safety systems flag it.

Those practices may be legitimate. A product has to operate, secure accounts, prevent abuse, and handle support requests. The problem is vague wording.

If the product invites personal conversation, the user should not have to guess which parts of the conversation are temporary, which parts are saved, and which parts may affect future replies.

What training actually means

Training usually means using data to improve or create a model so future users may benefit from patterns learned across many examples.

An app can choose not to use your conversations for that purpose and still use your conversations in other ways. It may send your message to a model provider so the model can respond. It may store the chat so you can reopen it. It may create a summary so the companion can remember context. It may save safety logs when something serious happens.

This is why the phrase needs follow-up. "Not trained on" answers one question. It does not answer the whole privacy question.

The five questions a clear policy should answer

A clear policy should answer five plain questions.

First: what data is collected? This includes messages, profile details, device information, payment data, logs, and generated memory.

Second: what is each type of data used for? Response generation, personalization, safety, support, analytics, billing, and security are different uses.

Third: who can access it? The answer should cover employees, contractors, infrastructure providers, AI model providers, and automated safety systems.

Fourth: how long is it retained? A chat history, account log, backup, and safety record may not have the same retention period.

Fifth: what can the user delete or export? A delete button should explain what disappears immediately and what may remain for a limited operational reason.

Memory needs its own explanation

AI companion apps need an extra layer of clarity because memory is part of the product.

If the app remembers personal details, users should be able to inspect and correct what is remembered. A memory may be inaccurate, outdated, or too sensitive to keep.

The app should also explain whether memory is created automatically, manually saved by the user, or inferred from repeated conversations.

A strong memory system does not only say, "We remember you." It says what is remembered, why, where it appears, and how to remove it.

Third-party processing should be visible

Many AI apps do not run every part of the system themselves.

A message may pass through model providers, hosting providers, analytics tools, payment processors, email systems, moderation tools, or customer support software. That does not automatically make the product unsafe. It does mean the privacy claim should name the categories clearly.

A user should be able to tell whether a vendor processes message content, account metadata, payment records, or only technical logs.

The more sensitive the product, the less room there is for vague phrases like "trusted partners" without explanation.

Watch the wording

Some privacy language is technically true but still incomplete.

"We do not sell your data" does not say whether data is shared with processors. "We do not train our models" does not say whether third-party models process the message. "You can delete your account" does not say what happens to backups, logs, or saved memory.

"Encrypted" is also not the whole answer. Encryption matters, but users still need to know who can access the data in normal product operation.

The goal is not paranoia. The goal is clarity before intimacy.

A better standard

For emotionally sensitive AI products, privacy should be easy to understand before the first personal conversation.

Users should not need legal training to know whether their words may be stored, reviewed, remembered, exported, deleted, or used to improve systems.

A good product can say: here is what we collect, here is what we do not use it for, here is who processes it, here is what memory means, here is how to delete it, and here is what may remain for security or legal reasons.

If a product invites trust, it should make data control obvious.

What a clearer promise sounds like

A stronger policy does not hide behind one reassuring sentence. It separates training, memory, support access, safety review, analytics, vendor processing, and deletion into plain categories.

For example: your chats are not used to train general models; selected memories are stored only to personalize your account; you can view and delete those memories; limited logs may be processed by infrastructure providers; and deletion requests remove account-level data subject to legal and backup limits.

That kind of wording is less glossy, but it is more useful. It tells the user what the company is promising and where the promise has boundaries.

In companion products, boring clarity is a virtue. The user should not have to become a privacy lawyer to understand what happens after they open up.

Sources worth reading

OpenAI Platform data controls

FTC: Bringing dark patterns to light