Habana Labs AI inference processor that can outrun the fastest GPUs

Pure-AI processing is here

To meet the ever-increasing demand for
AI workloads, the industry needs a new kind
of processing.
Legacy CPU and converted-GPU
architectures are not delivering the
required performance.
Introducing the first purpose-built AI
processing, ready for production today.

$graph of AI processor images\second$ $graph of AI processor images\second$

Hardware:
• 2-socket Intel Xeon
Platinum 8180 processors
• Int-8 precision

Software
• Intel Optimized Caffe ResNet-50
• Batch sizes = 64
• 4 Instances

Latency not reported
Power not reported
Source

GPU: TESLA V100;
GPU Memory: 16GB
CPU: E5-2698 v4 2.2Ghz
TensorRT 5.0
Synthetic dataset
Container – 18.09-py3
Precision: INT8
Batch size: 8
Total scenario power: 178W
Efficiency (images/sec/watt): 18

Source

Hardware
Goya HL-100 PCIe Card

Software
Ubuntu 16.04, SynapseAI 0.1.4
Precision: INT8
Batch size: 10

Total scenario power=100W
Efficiency (images/sec/watt): 151
Latency: 1.3msec including drivers

General purpose AI Processor
(AIP) – supporting any topology

Image
recognition

Neural machine
translation

Sentiment
analysis

Recommendation
system

Habana inference processor in action

Image recognition (ResNet-50)

Single Habana Goya™ processor, 100W, 1.3ms latency

Neural machine translation

Workloads run on a single Habana processor

Sentiment analysis

Workloads run on a single Habana processor

Recommendation system

Workloads run on a single Habana processor

* Topologies were compiled from a standard neural network framework
using SynapseAI graph compiler and executed on Habana Goya processor

One processor doesn't fit all

To optimize efficiency, Habana offers separate processors for training and inference workloads.
The inference processor supports models trained
by any processor (GPU, TPU, CPU,
and Habana Gaudi).

Inferenceavailable now

Learn more

TrainingHL-2000 will be sampling in Q2-2019
Training performance linearly scalable to thousands
of devices
2 Terabit/second interconnect bandwidth per device

Seamless integration
with software frameworks

Habana’s seamless integration with popular
frameworks and ONNX exchange format allows
developers
to quickly use complex AI resources
without having to be experts.

For power users looking for further
customization
and optimization, Habana offers rich
Software Development Tools and a Neural
Network Profiler.

AI performance, not stories

Image recognition (ResNet-50)

Neural machine translation

Sentiment analysis

Recommendation system