Skip to content
Hallway
0:00 / 0:00

#gpt-4#agi#microsoftresearch

Sparks of AGI: early experiments with GPT-4

Sebastien Bubeck

24K subscribers • 1 video on Hallway

YouTube
  • Recommended by 8 people: 7 on Hacker News and 1 on Lobsters.
  • Presented by Sébastien Bubeck, Microsoft researcher
  • Linked in the descriptionarXiv 2303.12712

Intermediate. Assumes working knowledge.

1.7M views3 years ago8 recommendations

AI and machine learningTransformers and LLMsNeural networks and deep learningAI safety and impact

Sébastien Bubeck presents early experiments with GPT-4, exploring its capabilities, limitations, and potential signs of artificial general intelligence.

The new wave of AI systems, ChatGPT and its more powerful successors, exhibit extraordinary capabilities across a broad swath of domains. In light of this, we discuss whether artificial INTELLIGENCE has arrived. Paper available here: arxiv.org/abs/2303.12712 Video recorded at MIT on March 22nd, 2023

Why it's here

8 people recommend this

What people say. Highly recommended as an essential lecture for understanding the capabilities, emergent behaviors, and limitations of early GPT-4.

lhlon Hacker News • Recommends

On “The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT”

…me of these a look: Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, et al. “Sparks of Artificial General Intelligence: Early Experiments with GPT-4.” arXiv, March 27, 2023. link . or watching Sebastien Bubeck's recent talk he gave describing what GPT-4 can do that previous LLMs couldn't: link to this video Geoffrey Hinton recently gave a very interesting interview and he specifically wanted to address the "auto-complete" topic: link Here's another way that Ilya Sutskever recently described it (comparing GPT 4 to 3): link I'd also recommend this recent Sam Bowman article that does a goood job reviewing some of the surprising recent develop…
View on Hacker News

iamflimflam1on Hacker News • Recommends

On “What GPT-4 Does Is Less Like “Figuring Out” and More Like “Already Knowing””

I used to say this as well, I’ve written blog posts and made videos saying exactly this. But - I would strongly recommend trying GPT4. There’s a very good video that is worth watching as well that may not change your mind, but will certainly make you think: link to this video The thing to remember is that these models are unbelievably huge and deep. We don’t know what is really happening in the layers or what it has really learned. Thinking that it’s a simple language model that is just predicting the next most likely word is unwise.
View on Hacker News

neomon Hacker News • Recommends

On “ChatGPT Egg Balancing Task Convinced Microsoft That AGI Is Closer”

Highly highly highly recommend listen to this lecture by Sebastien Bubeck (the dude who wrote the paper), it's extremely interesting: link to this video
View on Hacker News

Topfion Hacker News • Recommends

On “Gemini 3.0 spotted in the wild through A/B testing”

…ck and easy to present understanding of a models ability to output code with a rather open ended prompt that needs a high degree of coherence and were a lot of layers depend/build on each other. Helps that these are eye catching (literally as the output is visual) and easy to grasp. Same reason a lot of hype is created around the web desktops. 0 link to this video…
View on Hacker News

jadoon Lobsters • Recommends

Sparks of AGI: early experiments with GPT-4
5View on Lobsters

ftxbroon Hacker News • Recommends

On “The AI revolution is powered by these contractors making $15 an hour”

Here are two. First one is Nathan Labenz who ended up Red Teaming for OpenAI during the six months after the base model was fully trained but before it was released (especially relevant parts are after 45 minutes): link Other one is Sebastien Bubeck who put Sydney into Bing before she was tamed down some notches; he explains the lobotomization process in a particularly poignant way, where they had the GPT keep trying to draw a unicorn even as it was progressively deadened: link to this video
View on Hacker News

pmoriartyon Hacker News • Recommends

On “LLMs are not greater than the sum of their parts: Stanford researchers”

You might find this video1 on why "GPT is smarter than you think" interesting. They're older, but the "Sparks of AGI" paper2 and video3 are also well worth your time. 1 - link 2 - link 3 - link to this video
View on Hacker News

nazkaon Hacker News • Recommends

On “GPT4 Can’t Ace MIT”

You should take a look at this video. He is a researcher at Microsoft and had accès to private version of ChatGPT. He literally claims that ChatGPT 4 is not as good as before. His talk actually demonstrates the different evolutions. link to this video
View on Hacker News

adnmcq999on Hacker News • Mentions

On “Teaching ChatGPT to Speak My Son’s Invented Language”

I just watched a video where the guy touches on GPT-4 limitations and one of those is simple math. He asks it some order of operations question and it outputs the correct answer but only if it does it step by step. It then apologizes and says its original incorrect answer was “a typo.” Video: link to this video
View on Hacker News

d0mineon Hacker News • Mentions

On “OpenAI’s Red Team: the experts hired to ‘break’ ChatGPT”

It is not hypocritical: whatever "Open"AI does to make GPT-4 "safer", makes it dumber. Specific example is drawing a unicorn link to this video
View on Hacker News

HarHarVeryFunnyon Hacker News • Mentions

On “MiniGPT-4”

I got the 1T GPT-4 number from here - this is the video that goes with the Microsoft "Sparks of AGI" paper, by a Microsoft researcher that had early access to GPT-4 as part of their relationship with OpenAI. link to this video
View on Hacker News

revelioon Hacker News • Mentions

On “The AI revolution is powered by these contractors making $15 an hour”

The relevant comment from the second video is here: link to this video He says that GPT-4 started out being much better than GPT-3.5 at this task (draw a unicorn in TikZ), as the model trained longer it got to the point of drawing a great unicorn, but as it was tuned for "safety" it regressed to the point of being no better than GPT-3 was. Quite sad, in a way. It's not totally clear why there should be a tradeoff between "safety" and drawing skill / intelligence in this way. But it's interesting to speculate by analogy to humans.
View on Hacker News

tzekidon Hacker News • Mentions

On “Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?”

Here's the full talk0 from a Microsoft lead researcher that worked with the early / uncensored version of GPT-4. Simplified, tuning it for censorship heavily limits the dimensionality the model can move in to find an answer which means worse results in general for some reason. 0: link to this video
View on Hacker News

jodrellblankon Hacker News • Mentions

On “XB, or eXtreme Bullshitting: a less misleading name for LLMs”

link to this video about 5 minutes in. > People say it doesn't have a world model but it's not as clean cut as that, it absolutely could build an internal representation of the world and act on it as it progresses through the sentence temporally. Beware of trillion-dimensional space and its surprises, it's very hard for humans to reason about. ... We shouldn't think about those neural networks as learning simple concepts like 'Paris is the capital of France'; it's doing much more like operators, it's learning algorithms. Inside it, it's not just retrieving information, not at all, it's built internal representation that allows it to reproduce the data that it has seen succinctly. Really you shouldn't…
View on Hacker News

londons_exploreon Hacker News • Mentions

On “GPT-4 API General Availability”

I think the clearest evidence is Microsofts paper where they show abilities at various stages during training1... But in a talk 2, they give more details... The unicorn gets worse during the finetuning process. 2: link to this video 1: link
View on Hacker News

uLogMichealon Hacker News • Mentions

On “ChatGPT Now Losing Users”

And there's an army of people that dismiss concerns, discussion, and evidence presented on this topic regardless of what is provided. On subjective topics people can always dispute examples. Some people abuse the courtesy of those that provide such examples, you've further proven that here. Regardless of if we should benchmark imagery with something that was claimed to be multimodal, can you genuinely not see the difference here? link Maybe your internal prompt is primed to disagree regardless of what is presented? Edit: link to this video even mentions what you claim not to see. Safety degrades the model.
View on Hacker News