Answer-Box: A growing number of enterprises and individuals are rejecting mainstream closed-source AI models like ChatGPT and Claude due to privacy and data leakage risks, with Wall Street banks blocking ChatGPT within three months of its launch and Palantir CEO Alex Karp publicly warning that companies' IP is being siphoned off by model providers. The push for private AI is gaining momentum, driven by court orders forcing OpenAI to hand over user chats, rising shadow AI-related data breaches, and the emergence of cost-competitive privacy-preserving technologies like TEEs and open-source models that can now outperform frontier models on specific tasks.
The Privacy Problem with Closed-Source AI
The case for private AI has moved from theoretical to urgent. In July 2025, Palantir CEO Alex Karp gave a 20-minute interview on CNBC, calling the current enterprise AI setup an "alpha transfer" where companies pay premium token fees while their proprietary IP flows to model providers. Every request to a closed-source model travels in plaintext to the provider's server. Days earlier, Palantir had announced a partnership with NVIDIA to run the open Nemotron model inside customer-controlled environments, accompanied by a nine-point AI sovereignty manifesto. The stock rose 8% after the interview.
Enterprise adoption of cloud software over the past two decades relied on trust at the protocol level—each SaaS vendor saw only a slice of data, with little incentive to misuse it. But today's AI workflows demand uploading entire company data, giving upstream providers the ability to use that data for new features rather than letting it sit idle. Despite this, the momentum behind closed-source AI shows no signs of slowing: Anthropic's annualized revenue hit $470 billion in May 2025 (up from $90 billion late 2024), and OpenAI surpassed 900 million weekly active users in February 2025. Both companies raised new funding rounds in spring 2025 at valuations nearing $1 trillion.
Some enterprises acted early. By February 2023, within three months of ChatGPT's launch, major Wall Street banks had restricted its use. In May 2023, Samsung banned generative AI company-wide after engineers leaked chip source code into ChatGPT. OpenAI responded in August 2023 with ChatGPT Enterprise, promising not to train on business data and offering zero-data-retention (ZDR) agreements, which became standard enterprise procurement requirements.
However, contracts only lock company accounts. IBM found that by 2025, shadow AI—employees feeding company data into unauthorized AI tools via personal accounts—was involved in one-fifth of data breaches, adding an average of $670,000 to breach costs. A 2025 survey by security training company Anagram found that 40% of employees would violate AI usage policies to complete tasks faster.
Consumer Privacy Under Threat
Enterprises can buy their way out with ZDR contracts, but ordinary users face a different reality. In May 2025, a court order forced OpenAI to retain user chats even after deletion, and in November 2025, another judge ordered the transfer of 20 million conversations to The New York Times's lawyers as evidence. Criminal cases have also seen ChatGPT records subpoenaed, such as the Palisades fire arson case and a Florida double-murder case where a suspect's questions about handling bodies were cited. Sam Altman admitted in a July 2025 interview that ChatGPT conversations are not protected by legal privilege and that OpenAI "may be required" to hand over user chats in lawsuits.
A survey by Kolmogorov Law in October 2025 of 1,000 U.S. AI users found that 50% were unaware these conversations could be subpoenaed, while two-thirds believed they should have the same protections as conversations with a lawyer or doctor. This highlights a monitoring surface most users don't know exists.
How Private AI Works Today
Private AI is not a single technology but a spectrum of mechanisms that differ in where plaintext appears along the path from user to model and back.
Protocol-level privacy relies on promises. Enterprise solutions use contractual zero-retention (ZDR), where the provider promises not to store data. Anonymous proxies like Duck.ai or Venice AI strip user identity but still expose plaintext to the downstream model provider. Oblivious HTTP (OHTTP), an IETF standard since January 2024, splits knowledge between a relay (who sent it but can't read it) and the recipient (who reads it but doesn't know who sent it). This is the privacy ceiling for closed-source models because frontier labs guard their model weights like state secrets—a trillion-dollar bet on exclusivity.
Structural-level privacy replaces trust with hardware, cryptographic, or physical proofs, but they can only run open-source models.
- TEE (Trusted Execution Environment) runs inference inside a hardware enclave that even the machine operator can't open, with a signed attestation of which model and code ran. Prompt is sealed only at the endpoint; the relay path still has a plaintext reader bound only by protocol.
- E2EE (End-to-End Encryption) seals the relay. The user device encrypts the prompt with the enclave's key, creating a sealed envelope only the enclave can open. Trust shifts to the client code, which must be open and reproducible.
- FHE (Fully Homomorphic Encryption) eliminates trusted parties altogether. The server computes on ciphertext in a locked box the operator never opens. The cost is speed: inference on ciphertext is 10,000 to 100,000 times slower than plaintext, taking seconds to minutes per token. Custom chips for encrypted computation won't be commercially available for years.
- Local inference removes the network path entirely by running the model on the user's own hardware, but it's limited by model size and capability. For example, gpt-oss-120b scores about half of GLM-5.2 but requires 65GB of VRAM—more than two flagship gaming GPUs combined.
Despite these constraints, the cost of enclave-based inference is dropping. Phala's benchmarks show H100 enclave throughput loss averages under 7% for single-card inference, and nearly zero for large models where the bottleneck is moving data into the chip. NVIDIA's Blackwell GPU supports direct encryption of inter-chip traffic, reducing throughput loss to under 8% for a 397B model. Azure's confidential H100 SKU costs $8.90/hour (27% more than the non-enclave $6.98), while specialized providers like Phala offer enclave H100s from $3.80/hour, undercutting Lambda's standard SXM cards at $3.99–$4.29. Managed API pricing is also competitive: NEAR AI's attested gpt-oss-120b endpoint costs $0.15/M input tokens and $0.55/M output tokens, matching plaintext providers like Amazon Bedrock, Together, and Groq.
Open-Source Models Can Win
Despite the performance gap, a June 2025 study by Bridgewater's AIA Labs and Thinking Machines showed that an open-source model fine-tuned with expert annotations beat frontier models on both accuracy and cost. The team fine-tuned Qwen3-235B on Tinker (Thinking Machines' managed fine-tuning API) using reinforcement learning (GRPO) with modifications: round-robin batching, CISPO loss, and on-policy distillation. Tasks included determining whether a news article is important for C-suite investors, whether a central bank document signals future rate changes, and where boilerplate begins in documents.
Scoring on an independent test set, frontier models averaged ~50% on simple prompts and 78.2% with expert prompts—below the 80% threshold set by investors. The fine-tuned Qwen achieved 84.7%, meaning it made 29.8% fewer errors than the best frontier model, with inference costs 13.8x lower.
However, the training process itself was not private. Bridgewater's expert annotations passed through Tinker's third-party service, relying on ZDR-level trust. For buyers who want the same recipe without trust assumptions, options are limited. In March 2025, Workshop Labs and Tinfoil released Silo, a post-training stack running inside a Tinfoil enclave on a single 8-GPU node, with keys controlled by the customer. Enclave overhead was just 11 minutes for a two-hour training run. The stack can fit a trillion-parameter model (Kimi K2 Thinking) by freezing base weights and training only small adapters. Workshop Labs was acquired by Thinking Machines a month later, bringing the components for a fully enclaved RL loop under one roof.
The Harness Layer Problem
All these mechanisms protect the path from prompt to model, but agent-based workflows introduce external tool calls that create new plaintext destinations. A calendar server reads schedule queries; a database server reads search queries; a search engine receives plaintext queries. The mainstream solution remains protocol-level: companies like Runlayer and MintMCP use a central gateway to mask PII and log all calls, but tool servers still read plaintext to respond.
Structural solutions are emerging for the middle layer. Phala hosts MCP servers inside TEEs, covering wallets, code execution, and data sources, allowing users to verify privacy claims via attestation. However, TEE-hosted tools still send plaintext queries to the service provider—the enclave seals the messenger, not the destination.
Only a few destinations can answer without reading plaintext, limited to structured queries. Apple's Private Information Retrieval lets iPhones check numbers against a spam database without revealing the number. Microsoft uses the same scheme for passwords in Edge. MongoDB's Queryable Encryption allows equality and range matching on ciphertext. For open-ended search, the best current answer is trust: Brave promises zero data retention on its 400-billion-page index, but it remains at the protocol level. Exa is building a search engine with similar promises. Verifiable encrypted search remains in the lab.