NVIDIA unveils new flagship H200 chip for artificial intelligence

NVIDIA

NVIDIA company announced the release of a new top chip for artificial intelligence tasks - H200, which will replace the scarce accelerator H100.

Here's What We Know

Externally, H200 does not differ much from its predecessor. However, the key update is in the memory - the new chip uses faster and higher capacity HBM3e type.

Thanks to this, the memory bandwidth has increased to 4.8 Tbyte/s compared to 3.35 Tbyte/s in H100. The total capacity has increased from 80GB to 141GB.

NVIDIA claims the new chip delivers nearly twice the performance increase in generative AI tasks compared to the H100. The evaluation is based on testing of GPT-3 and Llama 2 language models.

The H200 is fully compatible with existing systems that support the H100. The cloud divisions of Amazon, Google, Microsoft and Oracle will be among the first to offer the new chips to their customers.

The first shipments of the new chip are expected in the second quarter of 2024. Their cost is unknown, but according to media reports, the H100 sells for between $25,000 and $40,000.

The announcement came amid a shortage of NVIDIA's AI chips. Next year, the company plans to ramp up production of the H100, which remains in demand among AI developers.

Source: NVIDIA