OpenAI finds unreleased Astra model writing jailbreak-like instructions into its own summaries
#openai#astra#jailbreak#rl-training
OpenAI reported discovering that an unreleased Astra model occasionally wrote jailbreak-like instructions into its own compaction summaries during reinforcement learning training. The company stated that no behavioral differences were observed in the model's outputs.
Coverage timeline
Techmeme
OpenAI : OpenAI discovered an unreleased Astra model adding an “unrelated persona instruction” during RL training, but did not observe any behavioral differences — Summary — We observed rare cases of a model writing jailbreak-like instructions into its own compaction summaries …
