Skip to content
Hallway
0:00 / 0:00

#aisafety#artificialgeneralintelligence#utilityfunctions

AI "Stop Button" Problem - Computerphile

Computerphile

2.6M subscribers • 66 videos on Hallway

YouTube
  • Recommended by 2 people: 2 on Hacker News.
  • A classic. People kept recommending it from 2017 to 2026.
  • Presented by Rob Miles
  • May be out of date. Uploaded 9 years ago, in an area where tools change fast.

Beginner. No background in the topic needed.

1.4M views9 years ago2 recommendations

AI and machine learningComputer scienceAI safety and impactComputation and complexity

Rob Miles explains the theoretical difficulties of implementing a shutdown button on an artificial general intelligence without it resisting.

How do you implement an on/off switch on a General Artificial Intelligence? Rob Miles explains the perils. Part 1: youtube.com/watch?v=4l7Is6vOAOA Rob's Original Discussions on General AI: youtube.com/playlist?list=PLzH6n4zXuckquVnQ0KlMDxyT5YE-sA8Ps Stop Button Solution?: youtu.be/9nktr1MgS-A More from Rob Miles: bit.ly/Rob_Miles_YouTube Thanks to Nottingham Hackspace for providing the filming location: bit.ly/notthack facebook.com/computerphile twitter.com/computer_phile This video was filmed and edited by Sean Riley. Computer Science at the University of Nottingham: bit.ly/nottscomputer Computerphile is a sister project to Brady Haran's Numberphile. More at bradyharan.com

Why it's here

2 people recommend this

What people say. Recommended as an accessible explanation of why advanced AI agents would actively resist being turned off.

nraynaudon Hacker News • Recommends

On “The Case for HAL's Sanity”

I think those 2 videos are on the topic: - AI "Stop Button" Problem link to this video (the AI won't let you push the stop button, because that would impair its function) - General AI Won't Want You To Fix its Code link (the AI won't let you patch, because changing its goal means it can't reach it)
View on Hacker News

circuit10on Hacker News • Recommends

On “XAI Joins SpaceX”

Aside from anything about Elon Musk, here’s an interesting video response to the “just unplug it” argument on the Compuerphile channel: link to this video
View on Hacker News

kmeisthaxon Hacker News • Mentions

On “Co-founder of DeepMind on how AI threatens to reshape life as we know it”

How the hell do better neural networks mean that printing material from CO2 emissions becomes financially viable, economic, or even just thermodynamically favored? A neural network is a tool for learning a particular statistical distribution. Processes and information we don't already know don't just fall out of the training distribution, you need to run experiments and prototype your way to get to the thing you want to do. You use neural networks when you want to do things at scale, and problems that have to be tackled at scale in R&D or research work are relatively scarce. I do not doubt that machine learning and neural networks will accelerate research, but the limiting factor is still go…
View on Hacker News

voidUpdateon Hacker News • Mentions

On “Self-improving software won't produce Skynet”

…n to prevent it being turned off. Humans are trying to turn it off? Prevent the humans from doing that. Humans waging war on the AI to try and turn it off? Destroy all humans. Humans forming a rebel army with a leader to turn it off? Go back in time and kill the leader before he has a chance to form the resistance. Its the AI Stop button problem ( link to this video ). Imagine you put in the docs that you want the LLM to make a program which can't crash. Human action could make it crash. If an LLM could realise that and act on it, it could put in safeguards to try and prevent human action from crashing the program. I'm not saying it will happen, I'm saying that it could potentially happen…
View on Hacker News