Article series · 5 parts · 34 min
Privacy in personal AI
Assistants now remember what we tell them. This series explains what that means for privacy, which technologies can protect the most intimate data we have ever handed to software, and how to build or choose a personal AI you can actually trust.
Short answer
A personal AI is only as useful as what it knows about you, so it will end up holding more intimate data than any service before it. Protecting that data takes three things: encryption at rest and in transit, processing that the provider cannot see (on-device or in confidential computing), and real user ownership, meaning you can export, move and delete what the AI knows.
The parts
- Part 1 · Why it mattersWhy privacy matters more in personal AI than anywhere elsePersonal AI now remembers what you tell it. Why that makes privacy the foundation of trust, and the four ways intimate AI data leaks.
- Part 2 · EncryptionEncryption at rest, in transit and in use: the three states of dataData is either stored, moving or being processed. Encryption at rest and in transit are solved problems. Encryption in use is where your privacy is decided.
- Part 3 · Encryption in useEncryption in use: five technologies that protect data while AI works on itHomomorphic encryption, on-device AI, multi-party computation, confidential computing and zero-knowledge proofs: how they work and which are ready for AI.
- Part 4 · Confidential AIConfidential AI in practice: how Apple, Google and Meta process AI requests privatelyHow Apple Private Cloud Compute, Google Private AI Compute and WhatsApp Private Processing keep AI requests private, and where they fall short.
- Part 5 · Architecture and checklistHow to build or choose a private personal AIA reference architecture for a private personal AI, based on what we built at Kin, and a ten-question checklist for choosing one.
Why an update, and what changed
I wrote the first version of this series for mykin.ai while building Kin, a personal AI designed around privacy. It argued that everyone would soon have a personal AI, that this AI would hold years of intimate data, and that encryption in use was the unsolved problem.
Since then the first prediction has largely come true. ChatGPT, Gemini and Claude all remember users across conversations, and the largest platforms have shipped confidential cloud systems for AI. At the same time, researchers have shown that the hardware enclaves those systems rely on can be broken with physical access, courts have ordered AI chat logs preserved, and new privacy policies have turned conversations into training data by default. The new version keeps the original argument, reorganises it into five parts and replaces what has aged with what holds today.
Frequently asked questions
What is privacy in personal AI?
Privacy in personal AI means that the assistant which remembers your conversations, habits and relationships keeps that data under your control: encrypted at rest and in transit, processed where the provider cannot read it, never used for training or advertising without consent, and possible to export or delete.
Why is a personal AI a bigger privacy risk than a search engine or a social network?
Because it is built to know you over years. People tell assistants things they would never post or search for, and memory features keep that context. A personal AI therefore concentrates more intimate data in one place than any service before it, which makes a breach, a subpoena or a policy change far more damaging.
Can an AI process my data without the provider seeing it?
Partly, today. On-device models never send data anywhere, and confidential computing (as in Apple Private Cloud Compute, Google Private AI Compute and WhatsApp Private Processing) runs cloud models inside hardware enclaves the operator cannot inspect. Fully homomorphic encryption would remove the need to trust hardware, but it is still too slow for large language models.
Is this an update of the Kin article series?
Yes. It updates and expands the four-part series I wrote for mykin.ai between December 2024 and March 2025. The structure is now five parts, and every part has been rewritten with what has changed in products, research and regulation since.
Working on AI with sensitive data?
Let's find the right architecture.
I help companies decide what should run locally, what can run in a confidential cloud and what should not be built at all.