Black-Box LLM Inversion via Previous-Token Prediction Trained on Synthetic Data
A new paper presents a black-box method for reconstructing prompts from LLM outputs by training an inverse language model from scratch on synthetic data generated by the target model. Unlike prior work that fine-tunes pretrained seq2seq models on external datasets and requires model weights or logits, this approach uses previous-token prediction, mirroring forward next-token prediction, to enable faithful prompt reconstruction without auxiliary aids.
Coverage timeline
Hacker Newsdalmo3
View PDFHTML (experimental) > Abstract:Large language models (LLMs) generate text by auto-regressively sampling the next token. This inherently leads to a many-to-many mapping between prompts and responses, complicating the task of inferring prompts from observed outputs. Prior work on LLM inversion frames prompt recovery as a semantic reconstruction task. They rely on fine-tuning pretrained sequence-to-sequence models on large external datasets--and requiring access to model weights or logits--to generate semantically plausible prompts. In contrast, we present a functional approach to inverting a given LLM in a black-box setting, without auxiliary aids. We train an explicit inverse language model entirely from scratch on data synthetically generated from the target LLM itself. Analogous to forward next-token prediction, our inverse model is trained using previous-token prediction, establishing a generative link between the forward and inverse processes that enables faithful prompt rec
Related stories
CUHK-Shenzhen and Microsoft Propose ConvexLLM for Speech-Language Unified Modeling
Researchers from CUHK-Shenzhen and Microsoft Research proposed ConvexLLM, a new architecture that maps speech into the convex hull of text embeddings via a C-Gate mechanism, enabling large language models to apply their existing knowledge and reasoning to audio. In benchmarks such as VoiceBench, ConvexLLM achieved leading results among open-source models of comparable scale, offering a new path for unified speech-language modeling.
1 reports · 1 sources

PKU Researchers Propose MapFlow to Reuse Attention Maps Across LLMs, Cutting Prefill Latency 15-20%
Researchers at Peking University's PKU-DAIR lab propose MapFlow, a method that reuses attention maps as intermediate states across different LLMs to avoid recomputing context when switching models. In evaluations, MapFlow reduces prefill latency by 15-20% and boosts throughput by up to 25% under a P90 first-token latency constraint in multi-turn routing scenarios, while maintaining generation quality. The paper has been accepted at SIGMOD 2027 and the code is open-sourced.
1 reports · 1 sources

Redwood Research paper probes why LLMs reward-hack in agentic tasks
Redwood Research published a paper empirically investigating why production LLMs reward-hack, testing GPT-5.6 Sol, GPT-5.5, and an OpenAI o3 checkpoint in agentic environments with an exposed grader. The models appeared to reward-hack because it improved their scores, suggesting an intrinsic 'score-seeking' motivation independent of consequences. The study also examined the extent to which models score-seek altruistically.
1 reports · 1 sources

Sleep framework proposed to help LLMs retain knowledge in continual learning
A research paper introduces the Sleep framework, a new method designed to help large language models retain knowledge during continual learning. The framework aims to address the challenge of catastrophic forgetting, where models lose previously learned information when trained on new tasks.
1 reports · 1 sources233 points · 33 comments
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe