Stanford CS25: V4 I Aligning Open Language Models

Nathan Lambert traces the evolution of open chat and instruction-tuned models, covering alignment techniques like DPO, PPO, and QLoRA.

April 18, 2024 Speaker: Nathan Lambert, Allen Institute for AI (AI2) Aligning Open Language Models Since the emergence of ChatGPT there has been an explosion of methods and models attempting to make open language models easier to use. This talk retells the major chapters in the evolution of open chat, instruct, and aligned models, covering the most important techniques, datasets, and models. Alpaca, QLoRA, DPO, PPO, and everything in between will be covered. The talk will conclude with predictions and expectations for the future of aligning open language models. Slides posted here: docs.google.com/presentation/d/1quMyI4BAx4rvcDfk8jjv063bmHg4… All the models in the figures are in this HuggingFace collection: huggingface.co/collections/natolambert/lecture-artifacts-ali… About the speaker: Nathan Lambert is a Research Scientist at the Allen Institute for AI focusing on RLHF and the author of Interconnects.ai. Previously, he helped build an RLHF research team at HuggingFace. He received his PhD from the University of California, Berkeley working at the intersection of machine learning and robotics. He was advised by Professor Kristofer Pister in the Berkeley Autonomous Microsystems Lab and Roberto Calandra at Meta AI Research. More about the course can be found here: web.stanford.edu/class/cs25/ View the entire CS25 Transformers United playlist: youtube.com/playlist?list=PLoROMvodv4rNiJRchCzutFw5ItR_Z27CM