GPU Infrastructure Glossary

All-reduce is a collective communication operation that aggregates data from all participating processes and distributes...

Bare metal refers to GPU servers accessed without virtualisation or hypervisor overhead — the customer receives direct h...

Colocation (colo) is a data centre service model where the facility provider supplies power, cooling, physical security,...

Capacity planning is the discipline of forecasting GPU compute demand and aligning infrastructure procurement, deploymen...

Cluster utilisation measures the percentage of available GPU capacity that is actively generating revenue at any given t...

Distributed training is the practice of splitting a deep learning training workload across multiple GPUs — within a node...

Edge compute deploys GPU processing capacity at the network edge — closer to end users and data sources rather than in c...

GPU-as-a-Service is a cloud delivery model where GPU compute capacity is rented on-demand or via reservation, rather tha...

GPU memory (VRAM, specifically HBM — High Bandwidth Memory) is the on-chip memory that stores model parameters, activati...

A hyperscaler is a cloud infrastructure provider operating at massive global scale — specifically AWS (Amazon), Azure (M...

InfiniBand is a high-bandwidth, low-latency networking technology developed by Mellanox (now NVIDIA Networking) that ser...

Immersion cooling is a thermal management technique where IT equipment is fully submerged in a thermally conductive but ...

An inference endpoint is a deployed model serving layer that accepts input data and returns predictions or generated con...

Liquid cooling encompasses any data centre thermal management approach that uses liquid — typically water or a dielectri...

A neocloud is a GPU-focused cloud provider that emerged outside the hyperscaler ecosystem to serve AI and high-performan...

NVLink is NVIDIA's proprietary high-speed interconnect for GPU-to-GPU communication within a single node. Unlike InfiniB...

Network topology describes the physical and logical arrangement of interconnections between nodes in a GPU cluster. The ...

Network bandwidth is the maximum data transfer rate of a network connection, measured in gigabits per second (Gb/s) or t...

Network latency is the time delay for data to travel between two points in a network, measured in microseconds (µs) or m...

Pipeline parallelism distributes different layers of a neural network across multiple GPUs or nodes, with each stage pro...

Power density measures the electrical power consumed per unit of data centre floor space, typically expressed as kilowat...

PUE is the ratio of total facility energy to IT equipment energy, measuring how efficiently a data centre delivers power...

Quantisation reduces the numerical precision of model weights and activations — from 32-bit floating point (FP32) to 16-...

Reserved instances are GPU compute resources purchased via a time-bound commitment — typically 1, 6, 12, or 36 months — ...

Spot instances are GPU compute resources offered at variable, discounted pricing with the caveat that the provider can r...

Sovereign compute refers to nationally controlled GPU and AI infrastructure operated within a country's borders, subject...

Tensor parallelism is a distributed computing strategy that splits individual neural network layers across multiple GPUs...

A training cluster is a tightly coupled array of GPU nodes connected via high-bandwidth interconnects — typically Infini...

Unit economics in GPU infrastructure refers to the revenue, cost, and margin analysis at the per-GPU or per-MW level. Th...

WUE measures the litres of water consumed per kilowatt-hour of IT energy, quantifying a data centre's water footprint. T...