NVIDIA's full GPU range compared: A100 vs. L40S vs. H100 vs. H200 vs. B200 vs. B300 vs. Vera Rubin

We previously covered the difference between CPUs and GPUs and wanted to go further into our research, especially on GPUs. If you are a developer, researcher, or enthusiast who works with Artificial Intelligence (AI), Machine Learning (ML), or Deep Learning (DL), you know how important it is to have a powerful and reliable Graphics Processing Unit (GPU) to handle the complex computations required by these applications.

NVIDIA is one of the leading manufacturers of GPUs, and they have been constantly innovating and improving their products to meet the growing demands of the AI and ML community.

Throughout this blog, we will compare seven of NVIDIA's most advanced and high-performance GPUs available on Civo: the A100, L40S, H100, H200, B200, B300, and Vera Rubin. We will look at each GPU's key specifications, features, and performance, see how they stack up against each other on various benchmarks and metrics, and provide some recommendations on which to choose depending on your needs.

An overview of the NVIDIA GPU range

NVIDIA produces several top-tier GPUs suited for a variety of workloads, from graphics and animation through to frontier AI training and inference at scale. This section provides a brief overview of each GPU available on Civo.

• NVIDIA A100 GPU: Introduced with the Ampere architecture, the A100 is a versatile GPU designed for a broad range of data center applications, balancing performance and flexibility.
• NVIDIA L40S GPU: The L40S, part of the Ada Lovelace architecture, offers strong graphics and AI performance and is designed for workloads that combine rendering, media, and inference.
• NVIDIA H100 Tensor Core GPU: With the Hopper architecture, the H100 pushes the boundaries of GPU performance, targeting the most demanding AI and ML training applications.
• NVIDIA H200 GPU: The H200 builds on the same Hopper architecture as the H100 but pairs it with significantly more HBM3e memory, making it better suited for large LLMs and memory-intensive inference workloads.
• NVIDIA B200 GPU: The first GPU in NVIDIA's Blackwell generation, the B200 delivers a step change in inference performance with the new NVFP4 precision format, 192GB of HBM3e, and NVLink 5 interconnect for rack-scale AI infrastructure.
• NVIDIA B300 GPU (Blackwell Ultra): The highest-performance GPU in the Blackwell lineup, the B300 pushes the B200's architecture to its ceiling with 288GB of HBM3e, 1.5x more NVFP4 compute, and 2x faster attention-layer processing, built for reasoning models and frontier inference.
• NVIDIA Vera Rubin GPU: NVIDIA's next-generation platform, succeeding Blackwell entirely. Built on TSMC's N3 process with HBM4 memory and NVLink 6, Vera Rubin is designed for the era of large-scale agentic AI and long-reasoning workloads. Available for early access reservation at Civo with delivery from Q1 2027.

Here is a summary table of the main characteristics of each GPU:

*Vera Rubin CUDA and Tensor Core counts have not yet been officially published by NVIDIA. Performance is measured in petaFLOPS — 50 PFLOPS dense FP4 inference — rather than core counts.

Beyond the numbers, what do these differences mean for users? Let's look at that.

CUDA Cores and Tensor Cores

The core counts on NVIDIA GPUs are a significant indicator of parallel processing power. CUDA Cores are the general-purpose processors that handle standard computing tasks, while Tensor Cores are specialized for accelerating machine learning and AI workloads. The more of these cores a GPU has, the more parallel computations it can perform simultaneously, crucial for demanding AI and ML applications.

The higher CUDA and Tensor Core counts of the H100 (and, to some extent, the L40S) deliver faster parallel processing compared to the A100, with performance improvements scaling with workload parallelism.

With the Blackwell generation (B200 and B300), NVIDIA introduced fifth-generation Tensor Cores with support for the new NVFP4 precision format. Despite having fewer CUDA Cores than the L40S, the B200 and B300 significantly outperform all Hopper-generation GPUs on AI inference tasks, demonstrating that raw core count has become a less meaningful comparison metric across generations. The B200 and B300 share the same 20,480 CUDA Cores and 640 Tensor Cores, but the B300's Tensor Cores are driven harder at the chip level to deliver 50% more NVFP4 throughput.

For Vera Rubin, NVIDIA has not published individual core counts, pivoting instead to system-level performance metrics (petaFLOPS, tokens per second, and performance per watt). This reflects a broader industry shift toward treating the full-stack system, GPU, interconnect, memory, and software, as the unit of performance rather than the chip in isolation.

Memory type and size

The type, size, and speed of a GPU's memory determine what applications it can optimally support. Larger, faster memory options mean bigger models can run without offloading, and more concurrent requests can be served.

• A100: 40GB or 80GB of HBM2e, ample for many workloads at its time of release, though constrained for models above 70B parameters.
• L40S: 48GB GDDR6 with ECC, fast for graphics workloads but lower bandwidth than HBM, which limits throughput on data-intensive AI inference.
• H100: 80GB of HBM2e, sufficient for many LLM inference workloads, with high bandwidth well-suited for training at scale.
• H200: 141GB of HBM3e, the same Hopper chip as the H100, but with significantly more memory. The step up to HBM3e also raises bandwidth by 43%. The H200 was NVIDIA's answer to growing demand for larger KV cache capacity in inference workloads before Blackwell arrived.
• B200: 192GB of HBM3e across eight high-speed stacks, 2.4x more than the H100, with 8 TB/s of bandwidth. Models up to approximately 300B parameters can reside fully on a single B200 without offloading.
• B300: 288GB of HBM3e, the same memory type and bandwidth as the B200, but with 50% more capacity. This enables models like DeepSeek-R1 (671B MoE) to run on fewer GPUs, with less tensor parallelism overhead and lower per-request latency.
• Vera Rubin: 288GB of HBM4 per GPU, with 22 TB/s of memory bandwidth, 2.75x more bandwidth than the B200/B300, and the same capacity. The upgrade to HBM4 is about keeping the GPU fed quickly enough to match the Rubin chip's substantially higher compute throughput.

Memory bandwidth

Transferring data efficiently between memory and processor cores is crucial. Higher bandwidth reduces the likelihood of GPU compute sitting idle waiting for data, a common bottleneck in large-model inference.

• A100: 2,039 GB/s, a strong baseline for its generation.
• L40S: 864 GB/s, lower than the other GPUs in this comparison due to GDDR6 memory. Fine for graphics and mixed workloads; a constraint for memory-bound AI inference.
• H100: 3,350 GB/s, a meaningful step forward, well-matched to the H100's compute.
• H200: 4,800 GB/s, the same chip as the H100, but HBM3e over HBM2e raises bandwidth by 43%. Combined with the larger memory pool, this makes the H200 meaningfully better for large-model inference despite having the same compute silicon.
• B200 and B300: 8,000 GB/s each, more than double the H200. Both share the same memory bandwidth despite the B300 having more total memory, so the B300's advantage is headroom and model capacity rather than raw throughput.
• Vera Rubin: 22,000 GB/s, 2.75x the B200/B300. This bandwidth is matched to the Rubin chip's substantially higher FP4 compute throughput, ensuring memory access doesn't become the constraint for the next generation of large models.

Sparsity support

Sparsity support lets the GPU skip zero values in sparse AI models, effectively doubling performance for workloads where many weight values are near zero.

All seven GPUs support sparsity. The H100 and later generations handle it more efficiently due to architectural improvements in the Hopper and Blackwell Tensor Engines. For the B200, B300, and Vera Rubin, sparsity is combined with NVFP4 precision to deliver substantially higher effective throughput than raw dense compute figures suggest.

MIG capability

Multi-Instance GPU (MIG) partitions a single GPU into multiple isolated instances, each with guaranteed compute, memory, and bandwidth. This is valuable in multi-tenant environments or when running several smaller workloads simultaneously.

• A100 and H100: Both support MIG, allowing the GPU to be split into up to 7 instances.
• L40S: Does not support MIG, a meaningful limitation for teams needing workload isolation or shared access.
• H200: Supports MIG with the same topology as the H100.
• B200 and B300: Both support MIG. Given their larger memory pools, instance sizes are proportionally larger, a B300 MIG instance with 34GB of memory has more headroom than an equivalent H100 partition.
• Vera Rubin: Expected to support MIG based on NVIDIA's architecture cadence, though specific instance configurations have not yet been published.

Performance benchmark

Let's look at the real-world performance profile for each GPU.

• NVIDIA A100: The A100 has been tested extensively and is known for its significant performance gains in AI and deep learning tasks. In language model training, the A100 is approximately 1.95x to 2.5x faster than the V100 using FP16 Tensor Cores. It also scored 446 points on OctaneBench, claiming the title of fastest GPU at the time of that benchmark.
• NVIDIA L40S: The L40S delivers A100-level AI performance across a variety of training and inference workloads in the MLPerf benchmark. However, with only 48GB of VRAM, it underperforms for large language models with significantly high parameter counts compared to the A100's 80GB variant. It shows a 26% performance improvement in Geekbench OpenCL over its predecessor.
• NVIDIA H100: The H100 shows a significant leap in computational power, particularly in FP64 and FP32 metrics. Optimized for large language models, it surpasses the A100 in specific areas, offering up to 30x better inference performance. It has also demonstrated improvements of up to 54% with software optimizations in MLPerf 3.0 benchmarks.
• NVIDIA H200: The H200 uses the same Hopper compute die as the H100, so raw training throughput is comparable. The performance improvement is concentrated in inference: the larger 141GB HBM3e memory pool allows bigger models to run without offloading, and higher memory bandwidth reduces latency for memory-bound LLM inference. For teams running models that previously required multi-GPU H100 setups, the H200 often collapses that to fewer GPUs.
• NVIDIA B200: MLPerf Inference v4.1 results showed Blackwell delivering twice the LLM training performance of Hopper. The B200's NVFP4 precision format delivers 10 petaFLOPS of dense compute—5x more than the H100 or H200's FP8 throughput, enabling meaningfully higher tokens-per-second on inference workloads. For teams moving from H100 to B200, inference throughput improvements of 3-5x are common, depending on model and batch size.
• NVIDIA B300 (Blackwell Ultra): MLPerf Inference v6.0 (April 2026) confirmed Blackwell Ultra systems achieved the highest throughput across the widest range of models and scenarios. On DeepSeek-R1, Blackwell Ultra delivered 2.5 million tokens per second, representing 2.7x higher token throughput compared to Blackwell submissions from six months earlier, driven by NVIDIA TensorRT-LLM software updates. The B300 also delivers up to 50x higher throughput per megawatt and 35x lower cost per token than Hopper for low-latency agentic workloads.
• NVIDIA Vera Rubin: Full MLPerf results for Vera Rubin are not yet available, as the platform is entering production now. NVIDIA has confirmed that the Rubin GPU delivers 50 petaFLOPS of dense FP4 inference, 5x the B200 and 3.3x the B300, and can train mixture-of-experts models with one-quarter the number of GPUs compared to Blackwell. When deployed in NVL72 configurations alongside NVIDIA Groq 3 LPX racks, NVIDIA states that Vera Rubin delivers up to 35x inference performance per watt for trillion-parameter models relative to Blackwell.

Which GPU is right for you?

The best GPU for you will depend on your specific use case, preferences, and budget. Here are some general guidelines that may help you make a decision:

Deploying your first GPU instance on Civo

Getting started with Civo GPU compute is straightforward. The process from account creation to a running GPU instance takes just a couple of minutes:

1. Create a Civo account at civo.com: New accounts receive $250 in free credit to explore the platform
2. Navigate to GPU instances in the Civo dashboard and select your preferred GPU
3. Choose your configuration: Select region, instance size, and whether you need on-demand or committed pricing
4. Select your base image: Civo provides pre-configured ML images with PyTorch, TensorFlow, CUDA, and cuDNN pre-installed, or bring your own container image
5. Connect and run : SSH directly into your instance or use one-click Jupyter access to begin your workload immediately

For teams looking to automate GPU provisioning as part of a CI/CD pipeline or MLOps workflow, Civo's API and Terraform provider support programmatic instance management - provision, configure, and tear down GPU instances without touching the dashboard.

What makes Civo's GPU cloud different

GPU access is only part of the equation. The infrastructure surrounding the GPU (orchestration, networking, developer experience, and support) determines how quickly your team goes from provisioning to production.

Kubernetes-native from the ground up

Civo's infrastructure is native to Kubernetes, ensuring rapid deployment, dynamic auto-scaling, and a modern networking architecture that grows with your workloads. Teams benefit from bare metal performance through Kubernetes by eliminating hypervisors from the stack and deploying containerized workloads directly.

This matters for ML teams because Kubernetes is the operational layer most production AI pipelines are already built around. There is no translation layer, no proprietary orchestration system to learn, and no lock-in to a custom scheduling interface.

Transparent pricing with no egress fees

One of the most significant hidden costs in GPU cloud is data egress - moving large model artifacts, training datasets, and inference outputs in and out of the provider's network. Civo eliminates egress fees entirely, which means the cost you see when you provision a GPU instance is the cost you pay.

For teams running iterative training workflows or serving models that require frequent checkpoint uploads, this is not a minor consideration - it is a meaningful component of total cost of ownership.

On-demand availability without quota friction

In the GPU market right now, "on-demand" often means something closer to "available eventually, probably, if you've planned ahead."

Quota approvals, regional availability constraints, and waitlists are routine frustrations with hyperscaler GPU access. Civo's on-demand model means genuine self-serve access. You can provision an A100 or H100 instance through the Civo dashboard or API without submitting a support ticket or waiting for quota approval.

Pre-configured ML frameworks

Civo GPU instances come with pre-installed popular ML frameworks, including PyTorch, TensorFlow, CUDA, and cuDNN, all readily available - with one-click Jupyter access and the ability to connect to GPU instances directly from a browser.

For teams that want to move from provisioning to running their first training job in minutes rather than hours, this removes a significant amount of setup friction.

Why GPU choice matters for ML teams in 2026

Not all GPU deployments are equal. In a market where NVIDIA GPUs span six distinct architectures with wildly different memory capacities and compute profiles, the choice of which GPU to run is as important as whether to use GPU compute at all.

Reasoning models, which generate long chains of thought before producing an answer, have made memory capacity and attention throughput central constraints in a way that standard LLM inference never did. A GPU that was the right call in 2024 for your use case may not be the right call today if the models you're running have doubled in parameter count or your context windows have grown by 10x.

The practical implication: benchmark your workload against the GPU you're considering, not against a GPU from two generations ago. The jump from H100 to B300 is not incremental, it's a different class of infrastructure designed for a different class of problem.

Civo makes it straightforward to access the full NVIDIA GPU range, from A100 through to B300 today, and Vera Rubin from Q1 2027, on transparent, predictable pricing with no egress surprises.

Explore the full Civo GPU range →

Summary

In this blog, we've detailed a comparison of seven of NVIDIA's current and next-generation GPUs, A100, L40S, H100, H200, B200, B300, and Vera Rubin, spanning five distinct architectural generations. We've explored their architectures, memory systems, and performance profiles to help you understand how they compare across the workloads that matter in 2026.

Check out how you can access each GPU with Civo AI.

Whether you're deciding on the best GPU for your next project or planning your infrastructure strategy for 2027 and beyond, we have the right solutions to meet your compute needs.

FAQs