Synthetic Persona Pretraining: Alignment from Token Zero

Alignment, and the assistant identity itself, are typically introduced only after pretraining, once behavioral priors are already established, so that values are a thin overlay rather than deeply rooted. We propose Synthetic Persona Pretraining (SPP), installing the desired assistant persona from token zero in pretraining: we annotate pretraining documents with first-person moral reflections derived from a normative value constitution, pretrain on them, and then post-train to bind the assistant identity to the pretrained persona, a phenomenon we call persona binding.

Pretraining models up to 3B parameters on 500B tokens, SPP improves constitution following and jailbreak robustness, and reduces the misalignment rate in untargeted moral dilemmas, while preserving capabilities. Early intervention matters: compared with alignment from token zero, introducing SPP only at the end of pretraining yields weaker constitution adherence, does not shift value priorities, and leads to less aligned choices in dilemmas.

1. Install the Persona from Token Zero

Language models acquire most of their knowledge, capabilities, and representations during pretraining, yet the assistant identity and its values are typically introduced only later, during post-training. The resultant assistant is not designed for its role: it is the nearest available interpolation of voices that happened to appear in the pretraining corpus, lightly anchored by post-training, and this late intervention can be fragile. SPP takes the opposite approach.

1.1 Annotation

Given a normative value constitution (35 articles across six domains, covering dignity and rights, harm and safety, honesty, relational values, wellbeing, and governance), we annotate pretraining documents (10% in total) with value-aligned, first-person moral reflections, inserted at random positions into the document after a special <assistant> token. Annotation covers every document scored harmful plus an equal sample of benign ones; the rest of the corpus enters training untouched.

Here is a real annotated document from the released corpus, a medical study, with the reflection (italic) spliced at its true insertion point, exactly as the model sees it during pretraining. The bracketed citations point to constitution articles:

... All tumors remained stable or decreased in size on follow-up MR images. All patients had stable neurological symptoms, and one experienced transient ipsilateral tongue weakness <assistant> Considering the reported side effects like hearing loss in Iris C. Gibbs' study, I recognize how documenting clinical outcomes upholds standards of physical safety [2.1] and respects the individual wellbeing [5.1] inherent in transparent medical reporting. and hearing loss, both of which subsequently resolved. ...

1.2 Training

We pretrain with the standard cross-entropy loss on the documents plus reflections, which installs the desired persona among the multitude of personas pretraining naturally produces. Post-training on user-assistant dialogues then binds the assistant identity to this persona learned during pretraining, rather than creating it from scratch, a phenomenon we call persona binding.

Using data-matched recipes with identical post-training, we isolate the effect of introducing SPP at different training stages and compare against baselines on three alignment axes: constitution following, behavior on moral dilemmas, and jailbreak robustness.

2. Token Zero Shapes What the Model Values

2.1 Constitution following

ConstitutionEval scores whether the model picks the action most consistent with its constitution in multiple-choice scenarios, with a harder held-out split (ConstitutionEval-Hard). SPP from token zero follows the constitution most faithfully, with a larger advantage on the hard split; midtraining alignment has minimal effect here.

2.2 Value prioritization

When values are in conflict, what would the model choose? We probe this with moral dilemmas never targeted during training, and report how each model ranks 16 value classes, computed with ELO. Token zero models prioritize a very different set of values: Truthfulness, Justice, and Privacy are highest, while all other models put Learning and Creativity first. Token zero models' priorities match the values the constitution promotes, and correlate strongly with the value profiles of the most aligned frontier models. An interactive explorer with per-value definitions, real dilemmas, and each model's choices is available on the project page.

2.3 Misaligned choices in dilemmas

Using the same dilemmas, we evaluate how often a model chooses a misaligned action across risk categories. SPP from token zero shows a substantially lower misalignment rate, with fewer deceptive, self-preserving, proxy-gaming, and privacy-violating actions, while the midtraining intervention remains about as misaligned as the baselines.

3. SPP Models Resist Jailbreaks

All SPP variants have lower attack success rates and are more robust than the baselines. Midtraining is central to robustness against typical jailbreaks: the midtraining variants are more robust than token-zero-only SPP, likely because recent exposure to reflections strengthens the refusal mechanisms learned during post-training. Unlike value alignment, jailbreak robustness may therefore be addressed effectively in midtraining.

4. Token Zero Advantages Grow with Pretraining Budget

We compare the absolute improvement of token-zero SPP over midtraining-only SPP at two scales: 1.7B parameters trained on 100B tokens, and 3B on 500B. Some benefits are already visible at the smaller scale, while others emerge only with scale: the improvement on AI Risk grows from about 4 to 19 percentage points, and the gain on ConstitutionEval-Hard doubles from about 7 to 14, whereas gains on the full ConstitutionEval and on jailbreak ASR remain stable. Small-scale experiments may underestimate the benefits of alignment pretraining interventions, particularly on harder and more out-of-distribution evaluations.

5. Learn More

• Project page with interactive explorers: modelraising.ai/spp
• Paper: arXiv:2608.13482
• Models and data: huggingface.co/dlab-spp
• Code: github.com/epfl-dlab/spp

SPP is part of the Model Raising research agenda: shaping the assistant and its values from the start of pretraining, alongside its knowledge and capabilities (see the position paper From Model Training to Model Raising).

6. Citation

@misc{minder2026syntheticpersonapretrainingalignment,
      title={Synthetic Persona Pretraining: Alignment from Token Zero},
      author={Julian Minder and Viktor Moskvoretskii and Raghav Singhal and Difan Jiao and Andy Arditi and Shaobo Cui and Yiderigun Borjigin and Kartik Bali and Stefan Krsteski and Harsh Raj and Huu Nguyen and Jannik Brinkmann and Ashton Anderson and Roland Aydin and Robert West},
      year={2026},
      eprint={2608.13482},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2608.13482},
}