Skip to content
0:00 / 0:00

#geoffreyhinton#deepbeliefnetworks#restrictedboltzmannmachines

Recent Developments in Deep Learning

Google TechTalks

349K subscribers • 56 videos on Hallway

YouTube
  • Recommended by 3 people: 3 on Hacker News.
  • Presented by Geoffrey Hinton, Professor at University of Toronto
  • May be out of date. Uploaded 16 years ago, in an area where tools change fast.

Advanced. Deep dive for people who know the field.

84K views16 years ago3 recommendations

AI and machine learningNeural networks and deep learningClassic machine learning

Geoffrey Hinton explains how to efficiently learn deep networks from unlabeled data and introduces multiplicative interactions into basic learning modules.

Google Tech Talk March 19, 2010 ABSTRACT Presented by Geoff Hinton, University of Toronto. Deep networks can be learned efficiently from unlabeled data. The layers of representation are learned one at a time using a simple learning module that has only one layer of latent variables. The values of the latent variables of one module form the data for training the next module. Although deep networks have been quite successful for tasks such as object recognition, information retrieval, and modeling motion capture data, the simple learning modules do not have multiplicative interactions which are very useful for some types of data. The talk will show how to introduce multiplicative interactions into the basic learning module in a way that preserves the simple rules for learning and perceptual inference. The new module has a structure that is very similar to the simple cell/complex cell hierarchy that is found in visual cortex. The multiplicative interactions are useful for modeling images, image transformations, and different styles of human walking. Speaker bio: cs.toronto.edu/~hinton/bio.html

Why it's here

3 people recommend this

What people say. Highly recommended as an excellent, slightly technical introduction to deep belief networks and unsupervised learning by a pioneer of the field.

Devilboyon Hacker News • Recommends

On “Building high-level features using large scale unsupervised learning”

Have you seen the online courses? link (From one of the authors of this paper!) link link Prof. Hinton's videos are very watchable: link link to this video

tylerhobbson Hacker News • Recommends

On “Microsoft Research make breakthrough in audio speech recognition”

There's a very good Google Tech Talk by Geoff Hinton (who has worked closely with Dahl on a lot of this research and developed some of the key algorithms in this field) that explains how to build deep belief networks using layers of RBMs: link That video focuses on handwritten digit recognition, but it's great for understanding the basics. There's a second Google Tech Talk video from a few years later that talks directly about phoneme recognition as well: link to this video

markovblingon Hacker News • Recommends

On “Is Google Cornering the Market on Deep Learning?”

There's a great Google Tech Talk by Geoff Hinton that serves as a slightly technical introduction to Deep Learning if anyone's interested: link to this video

Dn_Abon Hacker News • Mentions

On “Jeff Hawkins talk on modeling neocortex and its impact on machine intelligence”

I would remove one maybe both muches. I agree that in general SVMs are better but they are also every bit as black box. Deep belief networks though, are on another level. literally, svms are depth 2, dbn's are depth unbounded, i think. link to this video

utungaon Hacker News • Mentions

On “Machine Learning Video Library - Learning From Data (Abu-Mostafa)”

If you enjoyed Geoff Hinton's talk you will probably find the theano 'deep learning' library to be of use. Still undergoing quite a lot of iteration but powerful and you get to run your stuff on the GPU for added fun. link . Incidentally Hinton gave another google tech talk in 2010 link to this video .

iandanforthon Hacker News • Mentions

On “Large Scale Visual Recognition Challenge 2011 - Results”

Hinton's team (SuperVision) uses an interesting 'dropout' technique. He gave a Google Tech Talk on this back in June. link And an older talk that covers some of what a deep convolutional net is: link to this video

cs702on Hacker News • Mentions

On “Geoffrey Hinton has joined Google”

Something really interesting must be happening with AI at Google, because in the past few months both Ray Kurzweil (the best-known proponent of the singularity) and Geoff Hinton (the crazy-talented individual who invented deep-belief networks using interconnected "restricted Boltzman machines"1) have joined the company. -- 1 For an overview of deep belief networks, see these videos: link , link to this video , and link

zackmorrison Hacker News • Mentions

On “Research Priorities for Artificial Intelligence: An Open Letter”

…ome unstable. I've seen very little about this, except for a video where (as I recall) a multilayer neural network can be trained while holding certain connections at a constant value. That way the network can be trained to do different tasks and then put back into those modes by setting those training connections to various values during runtime: link to this video Eventually another neural net could be trained to know which values to hold to get different behaviors. It sounds like one of the goals is to reuse logic for similar tasks, and eventually be able to adapt in novel situations like our brains do. I believe there was also a bit about how time figured into all of this, but I haven’t watched it…

crystal_revengeon Hacker News • Mentions

On “2025: The Year in LLMs”

That must have been a long time back. Having lived through the time when web pages were served through CGI and mobile phones only existed in movies, when SVMs where the new hotness in ML and people would write about how weird NNs were, I feel like I've seen a lot more concrete progress in the last few decades than this year. This year honestly feels quite stagnant. LLMs are literally technology that can only reproduce the past. They're cool, but they were way cooler 4 years ago. We've taken big ideas like "agents" and "reinforcement learning" and basically stripped them of all meaning in order to claim progress. I mean, do you remember Geoffrey Hinton's RBM talk at Google in 2010? 0 That w…

u/imasharkamain r/compsci • Mentions

On “Significant advances in automatic speech recognition enabled by deep neural networks.”

Hinton's Tech Talks are highly watchable by the way. link to this video (2007) link to this video (2010)