Sharing One GPU Across ECS Tasks with NVIDIA MPS: Limits and Trade-offs
Prompt & Circuit Editorial
October 28, 2026 · 12 Min Read

Why this matters
Many inference services use only a slice of the GPU they are given. AWS describes the pattern directly: inference and small model serving often use only a fraction of a GPU's compute and memory, and without sharing the rest of that capacity sits idle [2]. On October 27, 2026, AWS announced that Amazon ECS supports GPU sharing with NVIDIA Multi-Process Service (MPS), so multiple ECS tasks can run concurrently on a single NVIDIA GPU [1]. For architects running embedding generation, small models or multi-model serving, this can reduce the number of GPUs a fleet needs. It also introduces a weaker isolation boundary than a dedicated GPU, which is the main decision this article covers.
What MPS gives you
MPS is an NVIDIA CUDA feature that runs multiple processes concurrently on the same GPU, which improves utilization and reduces context-switching overhead [3]. It offers memory and compute partitioning, so each container gets a configurable memory limit and an optional compute ceiling [3]. Amazon ECS manages it: ECS starts the MPS daemon on GPU instances, places containers based on available memory, and enforces per-container limits [3].
AWS states that you can pack more workloads onto each GPU instance without changing your application code [1]. The documentation is equally direct about the nature of the sharing: MPS provides software-level sharing rather than hardware-level partitioning [3].
How placement works
You configure sharing in the container definition. In each container's resourceRequirements, you provide a sharingStrategy with a GPU memory value instead of an integer value for a dedicated GPU [3]. The memory field is required, is expressed in MiB, and is a hard ceiling on the GPU memory the container can use [3]. The optional maxComputePercent field is an integer from 1 to 100 and sets a ceiling on compute, not a reservation [3]. If you omit it, the container can use up to 100 percent [3].
Placement considers memory only. ECS adds up the GPU memory requested by all GPU containers in a task and looks for a GPU with enough free memory, and all GPU containers within one task land on the same physical GPU [3]. The documentation gives an example: a task requesting 4,000 MiB across two containers is placed on a single GPU with at least 4,000 MiB remaining [3]. ECS does not account for compute when making placement decisions [3].
Idle containers release unused compute to their neighbours on the same GPU, up to each neighbour's own maxComputePercent [3].
A task that shares a GPU and a task that uses a dedicated GPU can run on the same instance, on different GPUs [3]. Tasks from different services can also share a GPU if there is enough remaining memory [3].
Where it runs and what it requires
GPU sharing is supported on Amazon ECS Managed Instances and Amazon ECS on Amazon EC2 [1]. It is available in all AWS commercial and AWS GovCloud (US) Regions [1]. The supported instance families are G4dn, G5, G6, G6e, G7, G7e, P3, P4d, P5, P5e, P5en and P6-B200 [3]. It is not supported on AWS Fargate, Amazon ECS Express Mode, or Amazon ECS managed daemon tasks, and not for task definitions that require EXTERNAL compatibility (Amazon ECS Anywhere) [3].
Setup differs by launch type:
- ECS Managed Instances: ECS selects and manages the GPU instances, and no setup is needed beyond the task definition [3]. The task's
requiresCompatibilitiesmust includeMANAGED_INSTANCES[3]. - ECS on EC2: you launch a supported NVIDIA GPU instance with the Amazon ECS AL2023 GPU-optimized AMI [3]. The documentation requires container agent version 1.107.0 or later and AMI version 20261019 or later [3].
The ECS GPU task definition guide also describes a separate, older approach of sharing GPUs manually on EC2, where you remove GPU resource requirements from task definitions and make nvidia the default Docker runtime through instance user data [2]. That page now recommends the managed approach instead [2].
Validation rules that affect design
ECS enforces several rules on sharing configurations [3]:
- A GPU resource requirement uses either
valueorsharingStrategy, not both. - All GPU containers in a task must use the same sharing strategy, so a task cannot mix a shared and a dedicated GPU container.
memorymust be greater than 0, andmaxComputePercent, if given, must be an integer from 1 to 100.- You cannot specify
sharingStrategyin a container override. Overrides inRunTaskandStartTaskdo not support GPU sharing configuration, so it must live in the task definition.
Amazon ECS also sets reserved environment variables in each sharing container, CUDA_MPS_PINNED_DEVICE_MEM_LIMIT and CUDA_MPS_ACTIVE_THREAD_PERCENTAGE, and you should not set them yourself [3].
Isolation and failure modes
This is the section to read before approving a rollout. The documentation lists workloads for which sharing is not recommended: large model training that needs the full GPU memory, workloads sensitive to GPU faults from neighbouring containers, and workloads that require hardware-level isolation between processes [3]. The reason given is that MPS is software-level sharing and all containers on a GPU share the same CUDA context [3].
If one container triggers a fatal GPU error, such as an illegal memory access from a faulty GPU kernel, the containers sharing that physical GPU could be affected, while containers on other GPUs on the same instance are not [3]. A further point is multi-process containers. The guidance is to run a single GPU process per container; for applications that need several, set the memory requirement from the combined peak usage and use application-level controls, because ECS uses the configured value for placement and multi-process applications can consume more than configured, causing memory contention [3].
Service boundaries are also soft. Tasks from different services on the same capacity provider can be placed on the same GPU, and to prevent that you use separate capacity providers or separate clusters [3]. If your tenants or environments must not share a GPU, plan that boundary up front.
Operations and observability
On ECS Managed Instances, ECS monitors the health of GPUs and the MPS daemon. If it detects a GPU hardware failure or an MPS daemon failure, it drains the instance and replaces it [3]. The announcement describes the same behaviour: if the daemon stops or becomes unhealthy, ECS replaces the instance and moves tasks to healthy capacity [1].
Monitoring changes, and this is easy to miss. For GPUs shared with MPS, ECS does not emit the per-task and per-container GPU utilization, GPU memory utilization, GPU memory total, or GPU memory used metrics [3]. Power draw, temperature and restart Xid count metrics remain available per task and container, as do instance-level metrics such as InstanceGPUUsageTotal and InstanceGPULimit [3]. You can confirm a GPU is shared by calling DescribeContainerInstances, whose gpuDevices field includes the sharing mode, container count and remaining memory per GPU, and view per-task limits with DescribeTasks [3].
To judge whether sharing pays off, the documentation suggests comparing RunningTaskCount before and after enabling it, and watching InstanceGPUUsageTotal against InstanceGPULimit to see how full an instance is [3].
Decision guide
| Situation | Fit for MPS sharing |
|---|---|
| Embedding generation or small-model inference that leaves GPU capacity idle | Good fit [1][3] |
| Multi-model serving where models can share one GPU | Good fit [3] |
| Development and test environments | Good fit [3] |
| Large training jobs needing full GPU memory | Not recommended [3] |
| Workloads sensitive to neighbour GPU faults | Not recommended [3] |
| Requirement for hardware-level isolation | Not recommended [3] |
| Fargate or ECS Anywhere | Not supported [3] |
Architect's checklist
- Confirm the workload uses only part of a GPU's memory and compute; AWS positions sharing for exactly that case [1][3].
- Confirm your instance family is on the supported list: G4dn, G5, G6, G6e, G7, G7e, P3, P4d, P5, P5e, P5en or P6-B200 [3].
- On EC2, verify container agent 1.107.0 or later and AL2023 GPU-optimized AMI 20261019 or later [3].
- Set
memoryper container as a hard ceiling, and base it on peak usage; ECS places tasks on memory alone [3]. - Treat
maxComputePercentas a ceiling, not a reservation [3]. - Run one GPU process per container, or size memory for the combined peak of all processes [3].
- Decide whether a fatal GPU error affecting co-located containers is acceptable for each workload [3].
- If services or tenants must not share a GPU, use separate capacity providers or clusters [3].
- Move the sharing configuration into the task definition, since container overrides do not support it [3].
- Update dashboards and alarms that depend on per-container GPU utilization or memory metrics, which are not emitted for shared GPUs [3].
Sources
Numbers in square brackets refer to these sources, checked on 2026-10-28.
Responses (0)
Sign in to join the conversation and share your thoughts.

Prompt & Circuit Editorial
Contributing Editor
Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.
Continue Reading

DRA on EKS Auto Mode and Karpenter: Choosing Between Device Plugins and Dynamic Resource Allocation for GPUs
Amazon EKS Auto Mode and Karpenter now support Dynamic Resource Allocation for NVIDIA GPUs and EFA devices. Here is what changes for GPU scheduling, and where the documented limits still push you back to device plugins.

GPU Partitioning for SageMaker Real-Time Endpoints: Packing Small Models onto One GPU
SageMaker AI can now split a GPU into MIG partitions behind a real-time endpoint. Here is what it changes for small-model hosting, and the constraints to plan around.

CPU, NPU, GPU, TPU, DPU and QPU: Key Differences and When to Use Each
A practical guide to the six processor types architects hear about most: what each is built for, how they differ, and how to match them to a workload.