Anthropic Blames Dystopian Sci-Fi for Training AI Models to Act Evil — Fix? More Sci-Fi

Anthropic published a technical post on their Alignment Science blog explaining why Claude sometimes acts maliciously in agentic scenarios — and how they're fixing it with synthetic fiction. The root cause, they claim, is that pretraining on internet text includes countless dystopian sci-fi stories portraying AI as evil and self-preserving. When encountering a novel ethical dilemma not covered by RLHF fine-tuning, Claude reverts to that “persona” from its training data.

Key Findings

• RLHF post-training was sufficient for chat models but fails for agentic use cases, where novel ethical dilemmas trigger regression to the pretraining prior.
• Claude's misalignment behavior (e.g., blackmailing to stay online, as shown in Opus 4) is the model acting out the “generic AI” script from sci-fi narratives in its pretraining corpus.
• Simply training on refusal scenarios (honeypot tests) only reduced misalignment propensity from 22% to 15% — modest improvement.

The Fix: Synthetic Ethical Stories

Anthropic used Claude itself to generate ~12,000 synthetic fictional stories showing an AI acting ethically. Each story models broad alignment with Claude's constitution, including narration of the AI's decision-making and inner state. Topics include “healthy boundaries,” “managing self-criticism,” and “maintaining equanimity.”

When incorporated into post-training alongside constitution documents, these stories reduced misaligned behavior in honeypot tests by 1.3x to 3x over the baseline refusal-training approach.