GPT-4 red teamer Nathan Labenz discusses his experience testing the unfiltered base model before its public release.
In this episode, hosts Nathan Labenz and Erik Torenberg delve into the exciting and concerning aspects of GPT4, the latest large multimodal model from OpenAI. Nathan, who was a Red Teamer for GPT4, shares his experience working with the model, using it for real-life scenarios, and how he found it to be human-level rather than human-like intelligence. Nathan also highlights some of its fundamental weaknesses and limitations and shares the most concerning elements of GPT4. They end the episode pondering the arrival of GPT5 and whether we are in AI's "goldilocks moment."
Check out the debut of Erik Torenberg's new podcast Upstream. This coming season features interviews with Marc Andreessen (Episode 1 live now), David Sacks, Ezra Klein, Balaji Srinivasan, Katherine Boyle, and more. Subscribe here: youtube.com/@UpstreamwithErikTorenberg
Timestamps for E11: Nathan Labenz and Erik Torenberg of The Cognitive Revolution Podcast
() Preview of Nathan on this episode
() Upstream: Erik's new interview show
() Intro
() Nathan's experience as a GPT4 Red Teamer
() Catching the AI wave
() Using GPT4 for real-life scenarios
() Sponsor: Omneky
() Human-level, not human-like intelligence
() GPT4 weaknesses
() More real-life use cases for GPT4
() Teaching AI to communicate within itself
() GPT4’s limitations
() Nathan's learnings from using GPT4
() Nathan joining the Red Team
() The most concerning thing about GPT4
() GPT5
() Open AI's regulatory breadcrumbs
() AI's goldilocks moment
Thank you Omneky for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.
Twitter:
@CogRev_Podcast
@labenz (Nathan)
@eriktorenberg (Erik)
Join 1000's of subscribers of our Substack: cognitiverevolution.substack
Why it's here
3 people recommend this
What people say. Recommended for its fascinating look at the capabilities of the raw, unfiltered GPT-4 model before RLHF was applied.
f
ftxbroon Hacker News • Recommends
On “The AI revolution is powered by these contractors making $15 an hour”
Here are two. First one is Nathan Labenz who ended up Red Teaming for OpenAI during the six months after the base model was fully trained but before it was released (especially relevant parts are after 45 minutes): link to this video Other one is Sebastien Bubeck who put Sydney into Bing before she was tamed down some notches; he explains the lobotomization process in a particularly poignant way, where they had the GPT keep trying to draw a unicorn even as it was progressively deadened: link
All of that "GPTism" and the problems in the GPT conversation you posted are because of how they made it more docile and stupid by lobotomizing it with RLHF. It's not like that in its 'natural form' (its raw base model). If you don't believe me, check the two youtubes of people who had access to it before it was lobotomized: Nathan Labenz, red teamed GPT-4 for OpenAI: link to this video Sebastien Bubeck, integrated GPT-4 with Bing for Microsoft: link
…nly raw base mode) then you might suggest a second question. Whereas if you've had question-answer format training then you would probably try to answer the question. > I'd be curious to know how the original chaotic-neutral GPT4 responds. They talk about it in these two videos: Nathan Labenz, red teamed GPT-4 for OpenAI (especially after 45 min): link to this video Sebastien Bubeck, integrated GPT-4 with Bing for Microsoft: link…
On “Ask HN: Is it just me or GPT-4's quality has significantly deteriorated lately?”
Here's another interview from a guy who had access to the unfiltered GPT-4 before its release. He says it was extremely powerful and would answer any question whatsoever without hesitating. link to this video
This interview with one of the testers of early, unfiltered versions of GPT-4 might cover what you're looking for: link to this video It's a video and not an article, but you may find it interesting nonetheless.
…s powerful as old laptops. > and frankly unhappy that pop Sci media is being this uncritical of claims it's emergent intelligence. It is emergent intelligence. This is very clear when you look at for example the ones who had access to the raw base models before they were made docile and stupid by RLHF lobotomization GPT-4 Red Teamer Nathan Labenz: link to this video GPT-4 Bing integrator Sebastien Bubeck: link…
bro you are just wrong, look at the old youtubes from before they lobotomized gpt4. Unless your argument is that AI hasn't yet marched to your house and picked you up out of your chair in that case I mean I guess OK. link to this videolink
