Skip to content
Hallway
0:00 / 0:00

#nvidia#cuda#deeplearninghardware

HC2023-K2: Hardware for Deep Learning

hotchipsvideos

11K subscribers • 1 video on Hallway

YouTube
  • Recommended by 2 people: 2 on Hacker News.
  • Presented by Bill Dally, Chief Scientist at NVIDIA

Advanced. Deep dive for people who know the field.

14K views3 years ago2 recommendations

HardwareAI and machine learningCPUs and computer architectureGPUs, shaders and CUDANeural networks and deep learning

Bill Dally traces the history of GPU modifications for machine learning and explains the physical and architectural challenges of scaling deep learning hardware.

Keynote 2, Hot Chips 2023, Tuesday, August 29, 2023 Bill Dally, NVIDIA Bill describes many of the challenges of building hardware optimized for Deep Learning and how NVIDIA in particular is addressing these challenges, particularly for training workloads. He looks at the history of how GPUs have been modified and enhanced to make them more applicable for machine learning.

Why it's here

2 people recommend this

What people say. Recommended for its clear explanation of the energy costs of data movement versus computation in modern AI hardware.

Footposton Hacker News • Recommends

On “Neuromorphic computing”

…"I keep getting those calls from those people who claim they are doing neuromorphic computing and they claim there is something magical about it because it's the way that the brain works ... but it's truly more like building an airplane by putting feathers on it and flapping with the wings!" From "Hardware for Deep Learning" HotChips 2023 keynote. link to this video This is at 21:28. The whole talk is brilliant and worth watching.…
View on Hacker News

tverbeureon Hacker News • Recommends

On “Nvidia, one of tech's hottest companies, is fine with remote work”

I think your biggest mistake is ignoring the amount of software that runs on top of the hardware. There were times in the past where others had a higher number of FLOPS, but it didn’t make a dent in the usage of CUDA. My standard assumption about anything in tech (and beyond) is that you don’t win on first principles alone (“parallel design is well understood”). It’s the details that make you stand out. Bill Daly gave a great presentation at Hot Chips a few weeks ago that explains how Nvidia’s AI performance was only in small parts due to process improvement. The rest was doing things smarter: matrix sparsity, different number representations and so forth. link to this video
View on Hacker News

gdiamoson Hacker News • Mentions

On “3D-Stacked CMOS Takes Moore's Law to New Heights”

Says the hotchips keynote - link to this video - data that is moved is about 100x higher power than data that is local You have a good point about intermediate data. This would not apply to all algorithms.
View on Hacker News