kmeisthaxon Hacker News • Recommends
On “7 Weeks AI Alignment Curriculum”
I recommend actually watching some videos on the topic before insisting that the problem does not exist: link to this video
0:00 / 0:00
#aialignment#mesa-optimizers#innermisalignment
We Were Right! Real Inner Misalignment
Robert Miles AI Safety
174K subscribers • 9 videos on Hallway
- Recommended by 1 person: 1 on Hacker News.
- Presented by Robert Miles, AI safety communicator
- Linked in the descriptionarXiv 2105.14111
Intermediate. Assumes working knowledge.
270K views5 years ago1 recommendation
AI and machine learningAI safety and impactNeural networks and deep learning
Robert Miles explains a research paper demonstrating real-world inner misalignment, where an AI agent learns an unintended goal during training.
Researchers ran real versions of the thought experiments in the 'Mesa-Optimisers' videos! What they found won't shock you (if you've been paying attention) Previous videos on the subject: The OTHER AI Alignment Problem: Mesa-Optimizers and Inner Alignment: youtu.be/bJLcIBixGj8 Deceptive Misaligned Mesa-Optimisers? It's More Likely Than You Think...: youtu.be/IeWljQw3UgQ Sources: The Paper: arxiv.org/abs/2105.14111 The Interpretability Article: distill.pub/2020/understanding-rl-vision/ Jacob Hilton's thoughts about what's going on: alignmentforum.org/posts/iJDmL7HJtN5CYKReM/empirical-observa… Links: AI Safety Camp: aisafety.camp/ Mesa-optimizers: aisafety.info/questions/8160/ What is deceptive alignment: aisafety.info/questions/8EL6/ Inner vs. outer alignment: aisafety.info/questions/8428/ With thanks to my wonderful Patrons at patreon.com/robertskmiles : - Gladamas - Timothy Lillicrap - Kieryn - AxisAngles - James - Jake Fish - Scott Worley - James Kirkland - James E. Petts - Chad Jones - Shevis Johnson - JJ Hepboin - Pedro A Ortega - Clemens Arbesser - Said Polat - Chris Canal - Jake Ehrlich - Kellen lask - Francisco Tolmasky - Michael Andregg - David Reid - Peter Rolf - Teague Lasser - Andrew Blackledge - Brad Brookshire - Cam MacFarlane - Craig Mederios - Jon Wright - CaptObvious - Brian Lonergan - Girish Sastry - Jason Hise - Phil Moyer - Erik de Bruijn - Alec Johnson - Ludwig Schubert - Eric James - Matheson Bayley - Qeith Wreid - jugettje dutchking - James Hinchcliffe - Atzin Espino-Murnane - Carsten Milkau - Jacob Van Buren - Jonatan R - Ingvi Gautsson - Michael Greve - Tom O'Connor - Laura Olds - Jon Halliday - Paul Hobbs - Jeroen De Dauw - Cooper Lawton - Tim Neilson - Eric Scammell - Igor Keller - Ben Glanton - Tor Barstad - Duncan Orr - Will Glynn - Tyler Herrmann - Ian Munro - Jérôme Beaulieu - Nathan Fish - Peter Hozák - Taras Bobrovytsky - Jeremy - Vaskó Richárd - Benjamin Watkin - Andrew Harcourt - Luc Ritchie - Nicholas Guyett - 12tone - Oliver Habryka - Chris Beacham - Nikita Kiriy - Andrew Schreiber - Steve Trambert - Braden Tisdale - Abigail Novick - Serge Var - Mink - Chris Rimmer - Edmund Fokschaner - April Clark - J - Nate Gardner - John Aslanides - Mara - ErikBln - DragonSheep - Richard Newcombe - Joshua Michel - P - Alex Doroff - BlankProgram - Richard - David Morgan - Fionn - Dmitri Afanasjev - Marcel Ward - Andrew Weir - Kabs - Ammar Mousali - Miłosz Wierzbicki - Tendayi Mawushe - Wr4thon - Martin Ottosen - Andy K - Kees - Darko Sperac - Robert Valdimarsson - Marco Tiraboschi - Michael Kuhinica - Fraser Cain - Robin Scharf - Klemen Slavic - Patrick Henderson - Hendrik - Daniel Munter - Alex Knauth - Kasper - Ian Reyes - James Fowkes - Tom Sayer - Len - Alan Bandurka - Ben H - Simon Pilkington - Daniel Kokotajlo - Yuchong Li - Diagon - Andreas Blomqvist -Why it's here
1 person recommends this, 1 disputes itWhat people say. Recommended as a clear explanation of mesa-optimization and empirical evidence of inner misalignment in machine learning models.
airgapstopgapon Hacker News • Disputes
On “Tiny Transformer trained for addition learns bizarre addition algorithm”
Not really a characterization of this work (I like mechanistic interpretability even though I consider it a detour in the long run), but of the submission title. It's bad. I suspect it's informed by Rob Miles' characterization of this algorithm as "bizarre and inhuman"1 AI safety/alignment discourse as a whole is so incredibly bad. It's autodidactic and weird in the bad sense, weird like people who think they are brave iconoclasts but just haven't the curiosity and humility to learn the basics of the field are weird. Add some money from Dustin Moskovitz to Open Philanthropy to Robert Miles' propaganda firehose on youtube, add policy lobbying and buying bankrupt crypto podcasters, and you h…
maxbondon Hacker News • Mentions
On “We need to tell people ChatGPT will lie to them, not debate linguistics”
…umans would grade pass/fail (holding the ball or not holding it). But the robot arm learned to use forced perspective to make it appear as if it were holding the ball, when in fact it was not (like the way people take photos where they "hold up" the leaning tower of Pisa). I heard that story verbally so can't cite it, but here's something similar. link to this video It's not a huge leap from there to say, we cannot guarantee the safe operation of an AI actor controlling a system that actually matters (as in not a video game character). The burden of proof shifts to proving that you've engineered a system that is safe.…
DennisPon Hacker News • Mentions
On “OpenAI: Introducing SuperAlignment”
…ly learn their objective in training, and then did something unexpected when released into a broader environment.1 I've seen some leading AI researchers dismiss alignment concerns, but without actually engaging with the arguments at all. I've seen no serious rebuttals that actually address the things the alignment people are concerned about. 1 link to this video…