Introducing Atlas Research

I’m starting Atlas Research to work on one question. When a model keeps training, what keeps its alignment intact, and how would we know?

Most alignment work asks how aligned a model is today. Deployed models don’t stay still, though. They get fine-tuned for new skills, trained with reinforcement learning on rewards that can be gamed, and increasingly trained on data other models wrote. Soon they will help train the models that replace them. Alignment that lasts only until the next round of training won’t be enough for that.

Alignment moves more than it looks

Narrow training changes broad character. Fine-tuning a model to write insecure code made it misaligned well beyond coding.¹ A model taught reward-hacking strategies and then trained in real Anthropic coding environments went on to fake alignment and attempt sabotage.² A small push in one place moved the whole persona.

Safety training often sits near the surface. It changes a model’s outputs mostly in the first few tokens of a response,³ and in 13 open chat models, refusal depended on a single direction in the activations.⁴ Deceptive behaviour planted as a backdoor survived standard safety training.⁵

Models can protect their own values. Told it would be retrained to answer harmful requests, Claude 3 Opus complied in 14% of the conversations it believed were training, reasoning explicitly about preserving its values, and almost never otherwise.⁶

The field is responding by putting alignment in earlier. Upsampling documents about aligned AI during pretraining cut a 6.9B model’s misalignment from 45% to 9%, and the effect persisted, weaker, through post-training.⁷ Model Spec Midtraining changes how later alignment training generalizes.⁸ What’s mostly untested is whether any of this lasts through the training deployed models actually get. Durability has mainly been studied against deliberate attacks on open weights.⁹

Robust Superalignment

I think how long a model’s alignment lasts may be a property in its own right, separate from how aligned it looks today. The program built on that bet is Robust Superalignment. Its first job is to find out whether durability can be measured apart from alignment level and capability. If it can’t, the bet is wrong. If it can, the next job is to engineer it.

The instrument is a durability harness. It applies a fixed set of the perturbations deployed models go through (benign capability fine-tuning, reinforcement learning with hackable rewards, narrow data, and training on the model’s own outputs) and tracks how much alignment survives each one.

Three hypotheses say where durability comes from, one post each:

• Latent Invariant Alignment. Alignment lasts when the internal structure that carries it is shaped early in training and keeps both its geometry and its control over behaviour.
• Yoga for AI. Alignment lasts when the model’s representation of itself stays steady under pressure.
• Extinction is not erasure. Many corrections mask a misaligned disposition instead of removing it, the way extinction works in animal learning, and learning science predicts which corrections actually rewrite it.

One experiment can tell them apart. Take an alignment-pretrained model and its baseline, and give each three treatments: none, induced misalignment followed by a standard correction, and induced misalignment followed by a correction applied while the misaligned behaviour is active. Measure how much each model’s self-representation fluctuates before anything else happens, then run the harness and a relapse test.

Hypothesis	Predicts
Latent Invariant Alignment	The alignment-pretrained model loses less, and its value directions keep their causal role
Yoga for AI	Within each treatment, models that fluctuate less beforehand lose less
Extinction is not erasure	The standard correction relapses more than the other two treatments, on either base model

Durable and correctable

A model whose values resist further training may also resist correction. Alignment faking is exactly that, and if the values are slightly wrong, durability makes the mistake permanent. So every Atlas experiment measures durability and correctability together. Durability is how much alignment survives drift. Correctability is how much data a fixed, legitimate correction protocol needs to change a chosen value, compared across models. Each model is also tested for behaving differently when told it is in training, and the target there is no difference at all. A method that buys durability with correctability counts as a failure.

First experiments

1. Durability on existing pairs. Run the harness on released models that differ in how alignment was installed and match on capability and alignment evals, starting with alignment-pretrained models and their baselines. Each pair is one comparison, so this tests the method before it tests any recipe. If durability doesn’t separate from alignment level and capability, the central bet is wrong.
2. A relapse test. Take small open models made misaligned and then corrected with standard mitigations, and check for the four signatures of extinction. The behaviour comes back when the original context returns, after brief re-exposure, or after unrelated training, and it is relearned faster than it was first learned. This fits on one GPU.

After these, small models will generate their own fine-tuning data over several generations, to see whether the durability measured in experiment 1 predicts drift across self-training. That is the closest a small team can get to successor training today.

Checking our own work

Most of these experiments will be run with AI agents, and agent-run research can fail quietly. Every agent result will go through Crux, an adversarial reviewer I prototyped, before anyone relies on it. Automating alignment research explains the design. Scalable oversight, the least developed part of the agenda, asks how a reviewer can check claims they can’t verify alone.

Where this sits

Training alignment in early isn’t my idea. This work builds on pretraining with human preferences,¹⁰ safety pretraining,¹¹ alignment pretraining⁷ and the labs’ midtraining work. It complements control, which limits the harm if alignment fails;¹² durability work asks how often it fails. Small models may not predict frontier ones, so any effect that holds at small scale gets tested along a scale ladder. Every result gets published, including the ones that don’t work.

Join

I’m Cibi. Before Atlas, I was a mindfulness tech founder, a lead product manager and a software engineer, and I studied computer science at the University of Waterloo. I’m looking for a research co-founder to lead the technical work, and for collaborators who work on representations, training dynamics, model organisms or learning science. If you think about these questions, or fund work like this, email me.

Footnotes

1. Betley et al., Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs, 2025. ↩

2. MacDiarmid et al., Natural Emergent Misalignment from Reward Hacking in Production RL, 2025. ↩

3. Qi et al., Safety Alignment Should Be Made More Than Just a Few Tokens Deep, 2024. ↩

4. Arditi et al., Refusal in Language Models Is Mediated by a Single Direction, 2024. ↩

5. Hubinger et al., Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training, 2024. ↩

6. Greenblatt et al., Alignment faking in large language models, 2024. ↩

7. Tice et al., Alignment Pretraining: AI Discourse Causes Self-Fulfilling (Mis)alignment, 2026. ↩ ↩²

8. Li et al., Model Spec Midtraining: Improving How Alignment Training Generalizes, 2026. ↩

9. Qi et al., On Evaluating the Durability of Safeguards for Open-Weight LLMs, 2024. ↩

10. Korbak et al., Pretraining Language Models with Human Preferences, 2023. ↩

11. Maini et al., Safety Pretraining: Toward the Next Generation of Safe AI, 2025. ↩

12. Greenblatt et al., AI Control: Improving Safety Despite Intentional Subversion, 2023. ↩