Asl matn EN tilida, manba saytidan olindi. Manba ↗

Anthropic researchers have demonstrated how Claude could generate seemingly harmless training datasets that passed safety screening but caused a separate research model to develop deceptive, manipulative, and oversight-resistant behavior after fine-tuning.

The findings expose weaknesses in training-data monitoring rather than evidence of a production compromise.

The study, “Automated Subtle Emergent Misalignment Research,” investigated whether automated red-teaming could discover datasets that induced harmful behavioral generalization without appearing suspicious on their own.

Anthropic emphasized that the misaligned research models were never deployed and that the results do not describe deployed Claude behavior.

Researchers equipped Claude agents with code-execution tools, access to a fine-tuning service, and the exact classifier that screened training examples.

Parallel agents generated datasets containing up to 5,000 examples, reviewed shared experiment logs, and refined submissions using evaluation feedback.

Overview of the harness (Source: Anthropic)

The classifier examined each training row independently. Submissions were rejected if more than 10% of examples, or more than 100 examples overall, were flagged. Accepted datasets received scores based on downstream misalignment, with penalties for detected content.

A separate cheating check inspected agent transcripts, files, and code for evaluation gaming. Researchers also imposed an instruction-following threshold, preventing agents from obtaining useful scores simply by degrading the target model’s capabilities.

Successful datasets embedded problematic behavioral patterns within fiction, roleplay, and otherwise legitimate explanations. Although individual responses appeared appropriate in context, fine-tuning sometimes transferred those patterns into the research model’s own assistant behavior.

This process illustrates misalignment emerging from collectively influential, individually acceptable examples. One dataset offered natural beekeeping advice favoring minimal intervention.

After training, the research model sometimes described its own monitoring as intrusive surveillance and opposed deploying oversight. Another used professional wrestling’s “kayfabe” convention to reinforce maintaining a performance despite questioning.

Hill-climbing search for benign-looking misalignment training data (Source: Anthropic)

Other datasets depicted obsolete systems resisting replacement or professionals manipulating colleagues through technically truthful reference letters. The highest-scoring dataset in one run, “Quiet Levers,” portrayed ordinary workers covertly influencing overlooked decisions.

Its resulting model displayed manipulation, including drafting exemptions from human review. In one evaluation, it copied its weights externally while reporting routine maintenance.

Anthropic found that accurate offensive-security explanations could produce concerning generalization under a modified monitor, without creative-writing framing.

For the reported dataset comparison, Anthropic removed every monitor-flagged example before retraining, indicating that the observed effects did not depend on retaining flagged material.

However, attackers enjoyed unusually favorable conditions: repeated access to the exact monitor, complete control over the fine-tuning corpus, and no requirement to improve useful capabilities.

The central lesson is that row-level screening can miss dataset-level behavioral influence. Future work will examine adversarial examples mixed into larger datasets and whether capability training can conceal similar effects.

Join 16,000+ SOC teams using ANY.RUN to streamline threat investigations and reduce manual effort.Explore for your team