[SPCL_Bcast] Cloud-Scale Inference on FPGAs at Microsoft Bing

Steve Reinhardt discusses Microsoft's Project Catapult and Project Brainwave, detailing how FPGAs accelerate DNN inference for Bing.

Speaker: Steve Reinhardt Venue: SPCL_Bcast, recorded on 8 April, 2021 Abstract: Microsoft's Project Catapult began nearly a decade ago, leading to the widespread deployment of FPGAs in Microsoft's data centers for application and network acceleration. Project Brainwave began five years later, applying those FPGAs to accelerate DNN inference for Bing and later other Microsoft cloud services. FPGA flexibility has enabled the Brainwave architecture to evolve rapidly, keeping pace with rapid developments in the DNN model space. The low cost of updating FPGA-based designs also enables greater risk taking, facilitating innovations such as our Microsoft Floating Point (MSFP) data format. FPGAs with hardened support for MSFP will provide a new level of performance for Brainwave. These AI-optimized FPGAs also introduce a new point in the hardware spectrum between general-purpose devices and domain-specific accelerators. Going forward, a key challenge for accelerator architects will be finding the right balance between hardware specialization, hardware configurability, and software programmability. See spcl.inf.ethz.ch/Bcast/ for more talks.