Nvidia’s flagship AI chip reportedly 4.5x faster than the previous champ

Nvidia announced yesterday that its upcoming H100 “Hopper” Tensor Core GPU set new performance records during its debut in the industry-standard MLPerf benchmarks, delivering results up to 4.5 times faster than the A100, which is currently Nvidia’s fastest production AI chip.

The MPerf benchmarks (technically called “MLPerfTM Inference 2.1“) measure “inference” workloads, which demonstrate how well a chip can apply a previously trained machine learning model to new data. A group of industry firms known as the MLCommons developed the MLPerf benchmarks in 2018 to deliver a standardized metric for conveying machine learning performance to potential customers.

In particular, the H100 did well in the BERT-Large benchmark, which measures natural language-processing performance using the BERT model developed by Google. Nvidia credits this particular result to the Hopper architecture’s Transformer Engine, which specifically accelerates training transformer models. This means that the H100 could accelerate future natural language models similar to OpenAI’s GPT-3, which can compose written works in many different styles and hold conversational chats.

Nvidia positions the H100 as a high-end data center GPU chip designed for AI and supercomputer applications such as image recognition, large language models, image synthesis, and more. Analysts expect it to replace the A100 as Nvidia’s flagship data center GPU, but it is still in development. US government restrictions imposed last week on exports of the chips to China brought fears that Nvidia might not be able to deliver the H100 by the end of 2022 since part of its development is taking place there.

Nvidia clarified in a second Securities and Exchange Commission filing last week that the US government will allow continued development of the H100 in China, so the project appears back on track for now. According to Nvidia, the H100 will be available “later this year.” If the success of the previous generation’s A100 chip is any indication, the H100 may power a large variety of groundbreaking AI applications in the years ahead.