CPU, NPU, GPU, TPU, DPU and QPU: Key Differences and When to Use Each
Prompt & Circuit Editorial
May 30, 2026 · 12 Min Read

In short: CPUs are general-purpose and run everything else; GPUs provide massive parallelism for training, inference and graphics; TPUs are Google's cloud accelerators for large-scale machine learning; NPUs run AI inference efficiently on devices; DPUs offload networking, storage and security work from server CPUs; and QPUs are experimental quantum processors still in the research phase. Pick by compute pattern, latency budget, power envelope and software ecosystem.
CPU: Central Processing Unit
- Purpose: General-purpose processor that runs the operating system, application logic and control flow.
- Design: A relatively small number of powerful cores with large caches, out-of-order execution, branch prediction and SIMD/vector extensions.
- Strengths: Low latency on branchy, sequential code; strong single-thread performance; broad instruction-set and software support (x86-64, Arm, RISC-V).
- Typical uses: Everything that is not a good fit for an accelerator: web and application servers, databases, orchestration, data preparation, and the host side of every accelerated system.
GPU: Graphics Processing Unit
- Purpose: Built for graphics, now the default general-purpose parallel processor for AI and high-performance computing.
- Design: Thousands of simple cores grouped into streaming multiprocessors (NVIDIA) or compute units (AMD), with high-bandwidth memory. Recent data-centre GPUs add matrix units (for example NVIDIA Tensor Cores) for low-precision AI math.
- Strengths: High throughput on data-parallel work, very high memory bandwidth, and the most mature AI software ecosystem (CUDA, ROCm, and framework support in PyTorch and JAX).
- Typical uses: Model training and inference, rendering, scientific simulation, video processing and parallel analytics.
TPU: Tensor Processing Unit
- Purpose: Google-designed ASIC for machine-learning training and inference, available through Google Cloud (Compute Engine, GKE and Vertex AI).
- Design: Each chip has one or more TensorCores. Each TensorCore contains matrix-multiply units (MXUs) built as systolic arrays, plus vector and scalar units. Per Google's documentation, MXUs are 256 x 256 on TPU v6e and TPU7x (Ironwood) and 128 x 128 on earlier generations, taking bfloat16 inputs and accumulating in FP32. Chips are linked by high-speed interconnects into slices and pods.
- Generations: Google currently documents TPU7x (Ironwood), v6e, v5p, v5e, v4, v3 and v2. All are data-centre parts; none is an edge device.
- Software: Google lists JAX and PyTorch as supported frameworks. Note that the PyTorch/XLA project is deprecated in favour of TorchTPU, and its repository is scheduled to be archived on 15 November 2026, so check current guidance before starting a new PyTorch-on-TPU project.
- Typical uses: Large-scale training and high-volume inference for teams already on Google Cloud.
NPU: Neural Processing Unit
- Purpose: Power-efficient AI inference, mostly on devices. "NPU" is a generic term; vendors use their own names for these blocks.
- Design: Arrays of multiply-accumulate units with on-chip memory, optimised for low-precision integer and floating-point math (INT8 and lower is common).
- Strengths: High inference throughput per watt and low latency without a round trip to the cloud, which also helps privacy.
- Typical uses: Phones, laptops, cameras, robots and IoT devices. NPUs are now built into many phone and PC system-on-chips. Add-on accelerators also exist: Google's Coral line uses an Edge TPU, and the Raspberry Pi AI Kit paired a 13 TOPS Hailo-8L module with the Pi (it is no longer in production; Raspberry Pi now points buyers to the AI HAT+).
- Limits: NPUs are inference-oriented and usually need quantised models and vendor toolchains. They do not replace GPUs for training.
DPU: Data Processing Unit
- Purpose: A programmable infrastructure processor that takes networking, storage and security work off the server's main CPU. Related device categories include SmartNICs and IPUs.
- Design: NVIDIA describes its BlueField DPUs as combining a CPU, a high-speed network interface and hardware acceleration engines.
- Current examples: NVIDIA lists BlueField-3 as a 400 Gb/s platform for software-defined networking, storage and security, and BlueField-4 as an 800 Gb/s platform aimed at large AI data centres.
- Typical uses: Software-defined networking and overlays, RDMA, NVMe over Fabrics and GPUDirect Storage, isolation and zero-trust enforcement in multi-tenant clouds, and keeping GPUs fed with data.
- Limits: DPUs are a data-centre technology. They pay off at scale and add operational complexity for small fleets.
QPU: Quantum Processing Unit
- Purpose: Runs quantum circuits on qubits, which use superposition and entanglement to represent and process information differently from classical bits.
- Design: Several competing physical approaches, including superconducting circuits (which need cryogenic cooling), trapped ions, neutral atoms and photonics. QPUs are used as co-processors from a classical computer, usually through cloud services and SDKs.
- State of the art: Google's 105-qubit Willow chip (announced December 2024) showed error rates falling as its error-correcting code grew, which Google describes as operating "below threshold." Google also notes that the random-circuit-sampling benchmark Willow excelled at "has yet to demonstrate practical commercial applications."
- Typical uses today: Research and prototyping in chemistry and materials simulation, optimisation and algorithm development. Theoretical speed-ups exist for specific problems (Shor's algorithm for factoring is the classic example), but these need far larger, error-corrected machines than exist today.
- Bottom line: Not a production compute target for architects in 2026.
Summary table
| Processor | Primary focus | Core design | Typical precision | Where it runs |
|---|---|---|---|---|
| CPU | General purpose | Few powerful out-of-order cores | FP64/FP32/INT | Everywhere |
| GPU | Parallel compute and AI | Thousands of simple cores plus matrix units | FP32 down to FP8 and lower | Workstations, data centres |
| TPU | ML training and inference | Systolic-array MXUs | bfloat16 inputs, FP32 accumulate | Google Cloud |
| NPU | On-device inference | MAC arrays with on-chip memory | INT8 and lower | Phones, PCs, edge devices |
| DPU | Networking, storage, security offload | CPU cores plus network and acceleration engines | Not applicable | Data-centre servers |
| QPU | Quantum circuits | Physical qubits | Qubits | Labs and cloud services |
Choosing the right unit
| Scenario | Good fit | Why |
|---|---|---|
| Application servers, databases, orchestration | CPU | Branchy, varied work; universal software support |
| Training or fine-tuning models | GPU, or TPU on Google Cloud | Dense matrix math at scale |
| High-volume inference in the cloud | GPU or TPU | Throughput per dollar; benchmark both on your model |
| On-device inference (phone, camera, laptop) | NPU | Low power, low latency, data stays local |
| Multi-tenant networking and storage at scale | DPU | Frees CPU cores, adds isolation |
| Quantum algorithm research | QPU via a cloud service | Experimental; no production advantage yet |
Hybrid patterns
- CPU plus GPU or TPU: The CPU handles I/O, preprocessing and control; the accelerator runs the model.
- GPU plus DPU: In AI clusters, DPUs handle networking and storage traffic so GPUs and host CPUs spend their time on compute.
- Cloud plus edge: Train on GPUs or TPUs, then quantise and deploy to NPUs on devices.
Practical tip: Prototype on GPUs, which have the widest availability and software support. Move to TPUs or NPUs when you have a stable model and a measured cost, latency or power problem that they solve.
How to compare accelerators
Published TOPS and TFLOPS figures are peak numbers at a specific precision and are not comparable across vendors. Before committing:
- Benchmark your own model, at the precision and batch size you will run in production.
- Compare cost per useful result (per million tokens, per image, per training run), not cost per hour.
- Count software effort: compilers, quantisation, and framework support often decide the outcome.
- Check availability and quotas in your region.
FAQ
Can I run PyTorch on a TPU? Yes. Google lists PyTorch alongside JAX for TPUs. The PyTorch/XLA bridge is being replaced by TorchTPU, so follow Google's current guidance.
Do NPUs replace GPUs? No. NPUs are designed for efficient inference on quantised models. Training and large-scale serving remain GPU and TPU territory.
Are DPUs only for data centres? Primarily. Their value comes from offloading infrastructure work across many servers.
Is a QPU ready for production? No. Current systems are research and prototyping tools.
Final thoughts
Understanding why each processor is designed the way it is helps you avoid the common mistake of reaching for the most expensive silicon by default. Match the compute pattern, power envelope and software stack to the workload, then measure.
Sources
Responses (0)
Sign in to join the conversation and share your thoughts.

Prompt & Circuit Editorial
Contributing Editor
Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.
Continue Reading

Sharing One GPU Across ECS Tasks with NVIDIA MPS: Limits and Trade-offs
Amazon ECS can now pack several tasks onto a single NVIDIA GPU using MPS. Here is how placement works, where the isolation ends, and what to check before you adopt it.

DRA on EKS Auto Mode and Karpenter: Choosing Between Device Plugins and Dynamic Resource Allocation for GPUs
Amazon EKS Auto Mode and Karpenter now support Dynamic Resource Allocation for NVIDIA GPUs and EFA devices. Here is what changes for GPU scheduling, and where the documented limits still push you back to device plugins.

GPU Partitioning for SageMaker Real-Time Endpoints: Packing Small Models onto One GPU
SageMaker AI can now split a GPU into MIG partitions behind a real-time endpoint. Here is what it changes for small-model hosting, and the constraints to plan around.