Mohamed S. Abdelfattah discusses the limitations of FPGAs for deep learning workloads and highlights specific areas where they can succeed.
Speaker: Mohamed S. Abdelfattah, Cornell University
There have been many attempts to use FPGAs to accelerate deep neural networks (DNNs), including many by the speaker of this talk. Some of these attempts ended up facing direct competition from GPUs and ASICs that are hyper-tuned for DNNs–inevitably, FPGAs often lose in that competition. However, there are many promising research directions in which FPGAs are indeed the best platform to accelerate parts of a deep learning workload. This talk will discuss several emerging paradigms in which FPGA strengths can be successfully leveraged for accelerating deep learning workloads. I will focus on (1) Automated DNN-HW codesign, (2) Using FPGA lookup tables as DNN building blocks and (3) The role of embedded networks on-chip in FPGA-powered datacenters.
Speaker Bio: Mohamed Abdelfattah is an Assistant Professor at Cornell Tech and in the Electrical and Computer Engineering Department at Cornell University. His research interests include deep learning systems, automated machine learning, hardware-software codesign, reconfigurable computing, and FPGA architecture. Mohamed’s goal is to design the next generation of machine-learning-centric computer systems for both datacenters and mobile devices.
Mohamed received his BSc from the German University in Cairo, his MSc from the University of Stuttgart, and his PhD from the University of Toronto. His PhD was supported by the Vanier Canada Graduate Scholarship and he received three best paper awards for his work on embedded networks-on-chip for FPGAs. His PhD work garnered much industrial interest and has since been adopted by multiple semiconductor companies in their latest FPGAs. After his PhD, Mohamed spent time at Intel’s programmable solutions group, and most recently at Samsung where he led a research team focused on hardware-aware automated machine learning.
----------------------------------------------------------
For more videos subscribe to the YouTube channel youtube.com/channel/UCGXVSHxK6PZKFiP9-XtjxtA!
For more information visit crossroadsfpga.org/seminars.html
----------------------------------------------------------
Website: crossroadsfpga.org
The Intel/VMware Crossroads 3D-FPGA Academic Research Center is jointly supported by Intel and VMware. The center is committed to public and free dissemination of its research outcome.
----------------------------------------------------------
Chapters
Introduction
GPU vs. DLA for DNN Acceleration
Arithmetic: Block Minifloat
Programming the Accelerator
Instruction Decode in HW
VLIW Network-on-Chip
Configurability: Custom Kernels
Customize Hardware for each DNN
Graph Compiler
Scheduling and Allocation
PART I: A Retrospective on FPGA Overlay for DNNS
Design Space Exploration Automated Codesi
AutoML: Neural Architecture Search (NAS)
AutoML: Hardware-Aware NAS
Hardware-Aware NAS Results
Why it's here
8 people recommend this
What people say. Highly recommended by engineers as a definitive, realistic retrospective on why FPGAs struggle to compete with GPUs for deep learning.
On “Is it possible to run Llama inference on an FPGA for (potentially) better performance?”
Here is an in-depth retrospective of Intel's attempt to outdo Nvidia in deep learning using FPGAs. The key points are at 13:20 to 18:20.
To summarise, Intel put together a dream team of researchers to make an FPGA that could outperform Pascal. They came up with numerous innovations that reduced the overhead of performing floating point operations on the FPGA only for Nvidia to do something very similar in their next GPU iteration (Volta) which could be leveraged by all of the existing Nvidia CUDA ecosystem. In any given task an ASIC will be more efficient than an FPGA at the same throughput and logic implementation.
On “Text classification (NLP) on FPGA. Is this possible??”
FPGAs are (not) Good at Deep Learning
According to this talk by a researcher who worked on it extensively, they fall short of GPUs for deep learning tasks. I myself am just trying to figure out if FPGAs give some significant advantages for RL training in the case where the data is generated directly on chip and avoids the friction of being moved onto the chip across multiple memory hierarchies, I definitely wouldn't try competing with GPUs in what they are good at directly.
You would do much better to instead share the video you linked at the start of the blog post than a 10 minute read of a post that you admittedly used ChatGPT and Claude to write.
To save others from AI slop and one click, and to reduce the impressions your post receives:
FPGAs are (not) good at deep learning by Dr. Mohamed S. Abdelfattah
On “We build an FPGA Startup - can you give me feedback pls!”
link to this video
Somebody posted this ^ here a few months ago.
Disclaimer - I have only a few years of FPGA experience, and non at all using FPGAs for AI. All from theoretical knowledge and my observations.
Basically FPGAs have no long-term merit for deploying specific apps. They are good for quick prototyping / POCs, or for VERY diverse loads (given you pre-designed an acceleration framework for each type of load).
The market today as a whole goes towards ASICs that are targeted at accelerating specific task-families and that allow software programmability.
The traditional "comfort zone" of FPGA accelerators remains where there's a need for a niche / specific computation and there's no financic…
On “When do you consider that using an fpga is a better choice than a microprocessor? ”
I don't believe this is true. If we're talking top of the line performance, GPUs are far and away the superior option, both in terms of raw performance and perf/Watt.
There's a talk on YouTube titled FPGAs are (not) good at deep learning which goes in depth into how and why FPGAs are losing compared to modern GPUs.
