How the two GPUs differ
H100 is an NVIDIA data center GPU based on the Hopper architecture. A100 is the previous generation, based on Ampere. H100 adds FP8 support and the Transformer Engine, designed to run transformer training and inference faster than A100.
In the cloud, H100 is mostly offered with 80GB of VRAM, while A100 comes with 40GB or 80GB. A 40GB A100 and an 80GB A100 can hold different model sizes, so check the VRAM column together with the price.
Why hourly price alone misleads
A100 usually costs less per GPU-hour than H100. What you actually pay is:
Job cost = price per GPU-hour × number of GPUs × hours the job runs
If H100 finishes the same job in fewer hours, the total can be lower even with a higher hourly price. If the job is limited by data loading or networking rather than GPU compute, H100's speed advantage shrinks.
How to choose
- Check that the model fits in GPU memory. If 40GB is not enough, drop the 40GB A100.
- Run a short version of the same job on both GPUs and measure throughput, such as tokens per second or time per step.
- Estimate total job hours from that throughput and compare totals with the formula above.
- Confirm the provider has the GPU available for the period you need.
How to check on Perhour
The H100 and A100 price pages list per GPU-hour prices by provider, and the H100 vs A100 comparison page shows the gap between the lowest prices. Prices exclude VAT and show the collection date.