This week marked Habana’s second year of showing working hardware and demonstrating leading AI performance at the AI Hardware Summit. At last year’s summit, Habana stepped out of stealth to introduce the company and our Goya Inference Processor. This year at the Summit marked more progress for the company and its products—inference AND training.
This week Goya demonstrated that it continues to lead the inference processor market with performance on ResNet-50 for image classification, now with 15,453 images-per-second and latency of 1.0ms vs the T4, which is reported to have roughly one-third the inference throughput speed of Goya—with throughput of 5,013 and 26ms latency. Goya also demonstrated its BERT language processing performance; on BERT, Goya demonstrated 1,527 sentences-per-second and 16.3 ms latency vs. T4’s 738 sentences-per-second and 32.4ms—both for 24-Batch. In addition, Habana demonstrated Goya’s SSD (Single Shot MultiBox Detector) functionality live at the summit.
Also featured at this year’s summit was the Gaudi processor, introduced in June, demonstrating its ResNet-50 image classification training performance of 1,650 images-per-second on the industry’s only training processor to feature on-chip integrated RoCE RDMA. Habana’s innovative use of Ethernet enables unprecedented scaling of AI training systems—the topic of Habana’s “sold out” invitation-only workshop at the summit: A Standards-Based Approach to Scaling AI Datacenter Capacity and Performance.
As a strong proponent of standardization of AI datacenter hardware, Habana also participated in one of the summit’s key industry panels, “Developments and Impacts of Emerging Industry Standards on the AI Datacenter,”alongside industry leaders from Facebook, Google, Alibaba and Uber. The consensus of the panelist: for the AI industry to address datacenter customer needs for greater performance and increased flexibility, industry standards such as Ethernet and the OCP-OAM for hardware will be useful.
The Habana team was pleased to contribute to the Summit with our live hardware demonstrations, presentation and panel participation, and we look forward to many successful collaborations with customers and partners we engaged at the conference.