Skip to content
Hallway
0:00 / 0:00

#gpt#rlhf#promptengineering

State of GPT | BRK216HFS

Microsoft Developer

700K subscribers • 17 videos on Hallway

YouTube
  • Recommended by 6 people: 6 on Hacker News.
  • Presented by Andrej Karpathy

Intermediate. Assumes working knowledge.

785K views3 years ago6 recommendations

AI and machine learningTransformers and LLMsNeural networks and deep learningAI agents and tool use

Andrej Karpathy explains the training pipeline of GPT models, covering pretraining, supervised finetuning, RLHF, and practical prompting strategies.

Learn about the training pipeline of GPT assistants like ChatGPT, from tokenization to pretraining, supervised finetuning, and Reinforcement Learning from Human Feedback (RLHF). Dive deeper into practical techniques and mental models for the effective use of these models, including prompting strategies, finetuning, the rapidly growing ecosystem of tools, and their future extensions. *Speakers:* * Andrej Karpathy *Session Information:* This video is one of many sessions delivered for the Microsoft Build 2023 event. View the full session schedule and learn more about Microsoft Build at build.microsoft.com BRK216HFS | English (US) | AI #MSBuild

Why it's here

6 people recommend this

What people say. Highly recommended as the clearest and most comprehensive explanation of how LLMs are trained and how to prompt them effectively.

sagarpatilon Hacker News • Recommends

On “GPT Best Practices”

I would highly recommend watching - State of GPT video by Karpathy at Microsoft Build to get an idea of how LLM’s work. There are some really good prompting ideas. It changed the way I use GPT-4. Video: link to this video
View on Hacker News

mcinton Hacker News • Recommends

On “It’s infuriatingly hard to understand how closed models train on their input”

…cleaning data. A non-answer for you, but for curious others, State of GPT 10 days ago provides a through introduction to the process used to train, Karpathy speaking at a Microsoft event providing a deep summary review of concepts, training phases, and techniques proving useful in the world of Generative Pretrained Transformers. State of GPT: link to this video…
View on Hacker News

valgazeon Hacker News • Recommends

On “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT”

…ieve reasonably consistent output from current-state LLMs. As models improve concerns about context window limitations will be reduced and it will be easier to discern user intent. These are good straight-to-the-point guides: - Prompt Engineering by BrexHQ: link - OpenAI guidance: link - link - (great examples): link - (~hr) Karpathy talk: link to this video tl;dr: - Begin with the best and most forgiving model available, optimizing it later if necessary. - Write your prompts in a clear, specific, and detailed manner - Be mindful, however, of the tradeoff between including more detail and increasing latency or cost. - If you're doing any complex reasoning, ask the model to show its work, help "…
View on Hacker News

jsighton Hacker News • Recommends

On “The future of AI according to thousands of forecasters”

Karpathy explains it far better than I did in his "State of GPT" talk about a month ago: link to this video Yes, language can do that, but books and other texts, on average, leave out a lot of steps that humans have learned to do naturally. They can be prompted to do them, and their answers improve, indicates that there is definitely some level of "reasoning" there. But it has weird limitations in its current forms. I expect that to improve pretty quickly, tbh.
View on Hacker News

tikkunon Hacker News • Recommends

On “Ask HN: Recommendation for a SWE looking to get up to speed with latest on AI”

It sounds like you want more broad stuff, not necessarily learning how to train models. More like learning to use them and how they work. link and Hacker's Guide to LLMs by Jeremy from Fast.ai - link State of GPT by Karpathy - link to this video LLMs by 3b1b - link Visualizing transformers by 3b1b - link How ChatGPT trained - link AI in a nutshell - link How Carlini uses LLMs - link For staying updated: X/Twitter & Bluesky. Go and follow people that work at OpenAI, Anthropic, Google DeepMind, and xAI. Podcasts: No Priors, Generally Intelligent, Dwarkesh Patel, Sequoia's "Training Data"
View on Hacker News

tintoron Hacker News • Recommends

On “RLHF Book”

Thanks. Here is presentation by Karpathy explaining different stages of LLM training. Explains many details in a form suitable for beginners. link to this video
View on Hacker News

jiggawattson Hacker News • Mentions

On “Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?”

Chat GPT 4 has ongoing training, such as using Reinforcement Learning from Human Feedback (RLHF) to tune it to provide "better" responses, "safer" answers, and to generally obey the system prompts. There's a release every few weeks. Yes, I've noticed too that recently it has become very "cagey", qualifying everything to death with "As an AI model...". A paper1 that took snapshots monthly mentioned that as the initial bulk self-supervised learning went on, the model became smarter, as expected. However, once the "clicker training" was imposed on top to make it behave, its peak capabilities were reduced. I'm not sure if it's in the paper or the associated video, but the author mentioned that…
View on Hacker News

raon Hacker News • Mentions

On “AI: First New UI Paradigm in 60 Years, Jakob Nielsen”

Kaparthy explained how LLMs can retrospectively assess their own output and judge if they were wrong. Source: link to this video
View on Hacker News

staredon Hacker News • Mentions

On “Google Gemini Eats the World”

Promises are cheap - even if made by researchers and engineers deeply believing that they can beat OpenAI. Let alone any assurances given by PR or marketing departments. And don't fool ourselves - everything announced publicly is prepared or at least approved by such. Computational power alone is not the only resource. It is also the training process itself (see this talk by Andrej Karpathy, link to this video , at 1 min) and, obviously, data and its quality. I will be convinced only after Google demonstrates that Gemini is better than GPT4 (in some, or all, tasks).
View on Hacker News

sk11001on Hacker News • Mentions

On “Ask HN: Give 8 year old access to Chat GPT?”

> They just produce something that looks like a human wrote it real good. That's what the base model does. To get an LLM assistant there are additional training phases to make it conversational, helpful and more correct - link to this video
View on Hacker News

xianshouon Hacker News • Mentions

On “Self-Rewarding Language Models”

Last May, Karpathy mentioned that the next step in LLM performance would likely take inspiration from AlphaGo's Monte Carlo-style tree traversal to enable deeper and more self-correcting reasoning: link to this video But if this approach holds up, it suggests that the most valuable part of the AlphaGo project that applies to LLM development is in fact reinforcement learning through self-play. Why not create a leaderboard of reward-generating language models that constantly "play" against each other, where model selection frequency is based on ELO, and update on the results? What if the Open LLM leaderboard ( link ) evolved into a constantly improving pool of such models? This also alleviates data sca…
View on Hacker News