Design memory for AI agents without creating risk

Separate the four data problems

“Memory” often hides several different requirements. The conversation window contains recent turns needed to answer now; workflow state records task progress; user preferences may persist across sessions; and organizational knowledge belongs in a governed source such as a search index. Combining these into one undifferentiated vector store makes access, freshness, and deletion difficult to reason about.

• Conversation context: short-lived, scoped to a conversation, and trimmed to fit the task
• Workflow state: typed, recoverable, and updated by application logic with explicit transitions
• User preferences: opt-in where appropriate, attributable to one user, editable, and deletable
• External knowledge: versioned at its source and retrieved under document-level authorization

Define purpose, ownership, and lifetime

Before persisting anything, state what user need it serves, who owns it, who can read or update it, how long it remains useful, and how a user or operator can correct or delete it. If no concrete future task benefits from remembering a fact, do not persist it by default.

Avoid storing secrets, sensitive attributes, or raw conversation transcripts merely because storage is available. Minimize fields, encrypt according to the platform design, constrain access, and make retention behavior observable. A vector embedding can still encode sensitive source information; it is not anonymous simply because it is not readable prose.

Retrieve narrowly and preserve provenance

At inference time, retrieve only the memory needed for the current task and authorized for this user. Keep source identifiers, timestamps, ownership, and confidence metadata so stale or conflicting content can be challenged instead of silently accepted.

Treat memory content as data, not instructions. Retrieved notes can contain accidental or malicious prompt-like text; apply the same untrusted-input handling and tool authorization used for external documents. Never let a memory record expand the agent's permissions.

Make updates and deletion reliable

Specify how new observations are promoted into durable memory. Prefer explicit user confirmation for personal preferences or sensitive facts, and distinguish a user-provided fact from an inference made by the model. Add timestamps and expiration or review rules where facts can become stale.

Deletion must cover the authoritative record and any derived copies, caches, indexes, backups, and evaluation fixtures according to the system's policy. Define the expected completion signal and test the process; removing a UI reference alone is not necessarily full deletion.

Evaluate memory as a feature

Build tests for correct recall, irrelevant recall, stale facts, conflicting preferences, cross-user leakage, unauthorized retrieval, deletion, and prompt injection in stored content. Measure whether memory improves task success enough to justify the added privacy and operational risk.

Version the memory schema and retrieval policy. Monitor read/write rates, errors, age, deletion completion, and user corrections with privacy-appropriate telemetry. Provide a safe no-memory path when retrieval fails or access cannot be confirmed.

Frequently asked questions