The Atlas research agenda

Atlas Research is an early-stage ASI alignment organization. The question behind it is how to align AI systems that may become far more capable than us, including systems that can improve themselves.

Why this agenda

Capabilities keep climbing, and there is still no high-confidence method for aligning systems smarter than the people overseeing them. Most safety work today, like interpretability, control and evals, is built for current and near-term models. That work matters. My worry is that much of it may stop holding once models can substantially improve themselves, and that relatively few teams are working on alignment meant to survive that point.

I come at this partly from yoga, which works on suffering at its root by stilling the mind, before suffering arises. I want alignment to work the same way: deal with misalignment where it would form, early in what a model learns, instead of catching it in behavior afterward. I sometimes call this yoga for AI. It’s also why I’m drawn to under-explored methods from outside ML, like ideas from neuroscience and from how people come to hold values in the first place.

The agenda starts there and works outward, in order of priority.

1. Robust Superalignment

Robust Superalignment is my name for an umbrella of under-explored methods that try to make alignment an intrinsic, invariant property of a model, one that holds as capabilities scale and resists being undone by recursive self-improvement. Most current methods shape behavior after pretraining. A smaller body of work intervenes earlier, like pretraining with human preferences¹ and tamper-resistant training.² Robust Superalignment groups and extends that direction.

The first hypothesis I want to test is Latent Invariant Alignment. Some behaviors in language models are mediated by surprisingly low-dimensional structure in their activations; refusal, for example, can be controlled through a single direction.³ Representation engineering reads and steers concepts like honesty through linear directions in activation space.⁴ Latent Invariant Alignment asks whether the structure that carries a model’s goals and values can be identified, shaped early in training, and kept invariant, meaning stable under further fine-tuning, capability gains and self-modification.

If it works, some kinds of misalignment would be less likely to arise, and alignment would be much harder to train away. It may well not work, and the obvious objection is strong. The refusal direction is easy to find, and that also makes it easy to cut out: ablating it removes refusal cheaply. Tamper-resistant training exists, but its robustness has broken under modest changes to the attack.⁵ The bet is that structure shaped during training, and entangled with the model’s capabilities, would be much costlier to remove than a safety behavior added afterward. That’s a hypothesis to test, and the first step is a scoped experiment on open models: can such a structure be found, shaped during training, and kept stable under further training aimed at undoing it?

2. Automated alignment research

Labs are starting to hand alignment research to AI agents. That could speed up everything above, as long as the results can be trusted. Crux is my first concept here: a second model that tries to break an agent’s research results before a researcher relies on them. It also serves the first goal directly, since Atlas will use AI to run its own experiments.

3. Scalable oversight

Scalable oversight asks how people can evaluate the work of systems that are better than them at the task, through methods like debate, weak-to-strong generalization and process supervision. It’s the least developed part of this agenda. I’m still reading the literature, and I’ll narrow it once the first two areas are underway.

How I want to do the work

Experiments should run on current open models, connect to concrete threat models like fine-tuning that strips safeguards from open weights or a model modifying itself, and be written up so other researchers can check and build on them. Where possible, they should explain why a method works, and not only show that it seems to.

What’s next

I’m looking for a research co-founder to lead the technical work, starting with the first Latent Invariant Alignment experiment, and for advisors and collaborators who work on representations, training dynamics or tamper resistance. If that’s you, email me.