Nathan Lambert's post-training textbook on RLHF and LLM alignment published by Manning
Nathan Lambert announced his new book, 'Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs', published by Manning. The book originated from his website documenting post-training methods like rejection sampling, outcome reward models, and character training, which lacked foundational online material. It compiles lessons from training open models over several years.
Coverage timeline
InterconnectsNathan Lambert
Housekeeping: No voiceover on another quick “launch” post. More essays soon! After a few long years of finding time to document my lessons from training open models, my post-training book is done! It’s published by Manning, under the title Reinforcement Learning from Human Feedback: Aligning and Post-training LLMs . Telling the story of the book is a useful way to explain why you may want a copy. The book started as a website where I wanted to document key methods of post-training that had potentially no online material explaining them. If there was something, I couldn’t find it. This existed for more topics than you would expect, given post-training was already popular in 2024 (when I bought the domain), and continues to this day. Topics like rejection sampling , outcome reward models , and character training are prime examples. This has helped make the website fairly popular, as it’s still one of the few places discussing these topics at a foundational, intuitive way. Otherwise, most
