What is the Nvidia B100?

by Stephen M. Walker II, Co-Founder / CEO

What is the Nvidia B100?

The NVIDIA B100 is a high-performance Tensor Core GPU designed for data center and cloud-based applications, optimized for AI workloads. It offers unprecedented performance, scalability, and security, making it a game-changer for large-scale AI and HPC workloads. The B100 is built on the cutting-edge NVIDIA Hopper GPU architecture, which introduces several innovations that make it more powerful, efficient, and programmable than any previous GPU.

Key features of the NVIDIA B100 include:

  • Architecture — The B100 is built on the 4 nm process and features 14,592 CUDA cores, 456 tensor cores, and 80 GB of HBM2e memory.
  • Performance — The GPU operates at a frequency of 1,095 MHz, which can be boosted up to 1,755 MHz, and has a memory running at 1,593 MHz.
  • Connectivity — The B100 supports PCIe Gen 5 and NVLink for ultra-high bandwidth and low-latency communication with other GPUs and devices.
  • Multi-Instance GPU (MIG) — The B100 can be partitioned into right-sized slices for workloads that don't require a full GPU, allowing multiple workloads to run on the same GPU for optimal efficiency.
  • AI Acceleration — The B100 includes the new Transformer Engine and NVIDIA AI Enterprise, which optimizes the development and deployment of accelerated AI workflows.
  • Price — The flagship B100 GPU is priced at around $30,000 (average).

The NVIDIA B100 is primarily used in data centers and for AI workloads, offering a significant performance leap compared to previous generations. It is widely used in large-scale AI and HPC applications, powering data centers with an order of magnitude speedup over the prior NVIDIA NVLink.

What is the B100 price and demand?

The Nvidia B100 GPU, designed for generative AI and high-performance computing (HPC), is priced around $30,000 on average as of August 2023. Market demand and retailer pricing strategies can cause significant price variations, with listings on eBay ranging from $39,995 to nearly $46,000, and prices in China reaching up to $70,000.

Due to its high demand for generative AI applications, Nvidia anticipates shipping 550,000 units of the B100 GPU globally in 2023. The combination of this strong demand and limited supply may result in elevated market prices.

In response to the surging demand for AI and HPC applications, Nvidia is ramping up its B100 GPU production, aiming to ship between 1.5 to 2 million units in 2024, a significant jump from the 550,000 units forecasted for 2023. This scale-up is part of Nvidia's strategy to leverage the ongoing AI boom and enhance its supply chain capabilities to ensure the availability of its advanced B100 compute GPUs.

With a base price of $25,000 per unit, Nvidia's projected shipment of 2 million B100 GPUs in 2024 could generate a minimum of $50 billion in revenue. This figure could increase substantially with higher shipment volumes.

Key Features and Specifications

The B100 GPU has a maximum thermal design power (TDP) of up to 700W, which is configurable to 300-350W. It supports up to 7 Multi-Instance GPUs (MIGs) at 10GB each. The B100 GPU features fourth-generation Tensor Cores, which perform matrix computations faster and more efficiently than ever before. This allows the B100 to handle a broader range of AI and HPC tasks with ease.

The B100 GPU delivers up to 5x faster AI training and 30x faster AI inference speedups on large language models compared to the previous A100 generation. It also accelerates exascale workloads with a dedicated Transformer Engine for trillion-parameter language models.

The B100 GPU can be partitioned down to right-sized Multi-Instance GPU (MIG) partitions for smaller jobs. With Hopper Confidential Computing, this scalable compute power can secure sensitive applications on shared data center infrastructure.

The NVIDIA Hopper architecture, on which the B100 is built, delivers unprecedented performance, scalability, and security to every data center. Hopper builds upon prior generations with new compute core capabilities, such as the Transformer Engine, and faster networking to power the data center with an order of magnitude speedup over the prior generation.

Performance

The B100 GPU has set new records on all eight tests in the latest MLPerf training benchmarks, excelling on a new MLPerf test for generative AI. It delivered the highest performance on every benchmark, including large language models, recommenders, computer vision, medical imaging, and speech recognition.

The B100 GPU also set new at-scale performance records for AI training. Optimizations across the full technology stack enabled near-linear performance scaling on the demanding LLM test as submissions scaled from hundreds to thousands of B100 GPUs.

Use Cases

The B100 GPU is suitable for a wide range of use cases. It is ideal for applications that require high-performance computing, such as complex AI models and scientific research. It is also a perfect match for PCIe expansions and GPU servers.

The B100 GPU is particularly effective for generative AI and large language models (LLMs). It has been used to set new records in the MLPerf training benchmarks, demonstrating its superior performance in these areas.

Conclusion

The NVIDIA B100 Tensor Core GPU represents a significant step forward in GPU technology. With its advanced architecture, fourth-generation Tensor Cores, and the ability to deliver lightning-fast AI training and inference speedups on large language models, it is one of the most powerful, programmable, and power-efficient GPUs to date. It is an excellent choice for organizations that require high-performance computing capabilities, making it a game-changing solution for those working with complex AI models.

More terms

What is a subject-matter expert?

A subject-matter expert (SME) in a specific domain is an individual with extensive knowledge and expertise in that particular field where machine learning is applied. This expertise could be in fields such as healthcare, finance, transportation, or e-commerce, among others. SMEs in these domains are typically well-versed in the specific challenges, data types, and regulatory requirements of their field, and they have a deep understanding of both theoretical and practical aspects of their industry. They are often involved in developing new machine learning applications, improving existing systems, and conducting research and development in their domain.

Read more

What is brute-force search?

Brute-force search, also known as exhaustive search or generate and test, is a general problem-solving technique and algorithmic paradigm that systematically enumerates all possible candidates for a solution and checks each one for validity. This approach is straightforward and relies on sheer computing power to solve a problem.

Read more

It's time to build

Collaborate with your team on reliable Generative AI features.
Want expert guidance? Book a 1:1 onboarding session from your dashboard.

Start for free