Inside GPT – Large Language Models Demystified • Alan Smith • GOTO 2024

Alan Smith demystifies the architecture of Generative Pre-trained Transformers, explaining embeddings, attention mechanisms, and how ChatGPT works.

This presentation was recorded at GOTO Amsterdam 2024. #GOTOcon #GOTOams gotoams.nl Alan Smith - AI Developer, Trainer, Mentor & Evangelist at Active Solution @CloudCastsAlanSmith RESOURCES twitter.com/alansmith linkedin.com/in/alan-smith-68a8491 ABSTRACT Natural language processing using generative pre-trained transformers (GPT) algorithms is a rapidly evolving field that offers many opportunities and challenges for application developers. But what is a generative pre-trained transformer, and how does it work? How can you leverage the latest advances in GPT algorithms to create engaging and useful applications? Can my business benefit from creating a GPT powered chat bot? In this demo intensive session Alan will take a deep dive into the architecture of GPT algorithms and the inner workings of ChatGPT. The journey will begin by looking at the fundamental concepts of natural language processing, such as word embedding, vectorization and tokenization. He will then demonstrate how you can apply these techniques to train a GPT2 model that can generate song lyrics, showing the internals of how word sequences are predicted. Alan will then shift the focus to larger language models, such as ChatGPT and GPT4, demonstrating their power, capabilities, and limitations. The use of hyperparameters such as temperature and frequency penalty will be explained and their effect on the generated output demonstrated. Join me for this session if you want to learn how to harness the power of GPT algorithms in your own solutions. [...] TIMECODES 00:00 Intro 01:59 GPT sequence prediction 03:48 Prompt engineering 04:45 Demo: ChatGPT2 10:45 Processing text 19:27 Demo: Word2Vec dimensionality reduction 21:07 Transformer architecture 23:58 Demo: GPT2 input embedding 27:00 Self attention 31:08 Demo: GPT2 multi-head attention 32:52 Attention example 33:49 Demo: GPT2 next token prediction 44:06 Parameters 46:21 Thanks for explanations & inspiration 46:57 Outro Read the full abstract here: gotoams.nl/2024/sessions/3145 RECOMMENDED BOOKS Alex Castrounis • AI for People and Business • amzn.to/3NYKKTo Phil Winder • Reinforcement Learning • amzn.to/3t1S1VZ Holden Karau, Trevor Grant, Boris Lublinsky, Richard Liu & Ilan Filonenko • Kubeflow for Machine Learning • amzn.to/3JVngcx Kelleher & Tierney • Data Science (The MIT Press Essential Knowledge series) • amzn.to/3AQmIRg Lakshmanan, Robinson & Munn • Machine Learning Design Patterns • amzn.to/2ZD7t0x Lakshmanan, Görner & Gillard • Practical Machine Learning for Computer Vision • amzn.to/3m9HNjP twitter.com/GOTOcon linkedin.com/company/goto- instagram.com/goto_con facebook.com/GOTOConferences #LLM #LargeLanguageModel #LimitationsOfLLMs #InsideGPT #GPT #GPT4 #GPT2 #GenAI #AI #GenerativeAI #CUDA #CommonCrawl #LSTMs #AlanSmith CHANNEL MEMBERSHIP BONUS Join this channel to get early access to videos