AI Infrastructureamazon-ecs

Sharing One GPU Across ECS Tasks with NVIDIA MPS: Limits and Trade-offs

P&C

Prompt & Circuit Editorial

October 28, 2026 · 12 Min Read

Sharing One GPU Across ECS Tasks with NVIDIA MPS: Limits and Trade-offs

Why this matters

Many inference services use only a slice of the GPU they are given. AWS describes the pattern directly: inference and small model serving often use only a fraction of a GPU's compute and memory, and without sharing the rest of that capacity sits idle [2]. On October 27, 2026, AWS announced that Amazon ECS supports GPU sharing with NVIDIA Multi-Process Service (MPS), so multiple ECS tasks can run concurrently on a single NVIDIA GPU [1]. For architects running embedding generation, small models or multi-model serving, this can reduce the number of GPUs a fleet needs. It also introduces a weaker isolation boundary than a dedicated GPU, which is the main decision this article covers.

What MPS gives you

MPS is an NVIDIA CUDA feature that runs multiple processes concurrently on the same GPU, which improves utilization and reduces context-switching overhead [3]. It offers memory and compute partitioning, so each container gets a configurable memory limit and an optional compute ceiling [3]. Amazon ECS manages it: ECS starts the MPS daemon on GPU instances, places containers based on available memory, and enforces per-container limits [3].

AWS states that you can pack more workloads onto each GPU instance without changing your application code [1]. The documentation is equally direct about the nature of the sharing: MPS provides software-level sharing rather than hardware-level partitioning [3].

How placement works

You configure sharing in the container definition. In each container's resourceRequirements, you provide a sharingStrategy with a GPU memory value instead of an integer value for a dedicated GPU [3]. The memory field is required, is expressed in MiB, and is a hard ceiling on the GPU memory the container can use [3]. The optional maxComputePercent field is an integer from 1 to 100 and sets a ceiling on compute, not a reservation [3]. If you omit it, the container can use up to 100 percent [3].

Placement considers memory only. ECS adds up the GPU memory requested by all GPU containers in a task and looks for a GPU with enough free memory, and all GPU containers within one task land on the same physical GPU [3]. The documentation gives an example: a task requesting 4,000 MiB across two containers is placed on a single GPU with at least 4,000 MiB remaining [3]. ECS does not account for compute when making placement decisions [3].

Idle containers release unused compute to their neighbours on the same GPU, up to each neighbour's own maxComputePercent [3].

A task that shares a GPU and a task that uses a dedicated GPU can run on the same instance, on different GPUs [3]. Tasks from different services can also share a GPU if there is enough remaining memory [3].

Where it runs and what it requires

GPU sharing is supported on Amazon ECS Managed Instances and Amazon ECS on Amazon EC2 [1]. It is available in all AWS commercial and AWS GovCloud (US) Regions [1]. The supported instance families are G4dn, G5, G6, G6e, G7, G7e, P3, P4d, P5, P5e, P5en and P6-B200 [3]. It is not supported on AWS Fargate, Amazon ECS Express Mode, or Amazon ECS managed daemon tasks, and not for task definitions that require EXTERNAL compatibility (Amazon ECS Anywhere) [3].

Setup differs by launch type:

  • ECS Managed Instances: ECS selects and manages the GPU instances, and no setup is needed beyond the task definition [3]. The task's requiresCompatibilities must include MANAGED_INSTANCES [3].
  • ECS on EC2: you launch a supported NVIDIA GPU instance with the Amazon ECS AL2023 GPU-optimized AMI [3]. The documentation requires container agent version 1.107.0 or later and AMI version 20261019 or later [3].

The ECS GPU task definition guide also describes a separate, older approach of sharing GPUs manually on EC2, where you remove GPU resource requirements from task definitions and make nvidia the default Docker runtime through instance user data [2]. That page now recommends the managed approach instead [2].

Validation rules that affect design

ECS enforces several rules on sharing configurations [3]:

  • A GPU resource requirement uses either value or sharingStrategy, not both.
  • All GPU containers in a task must use the same sharing strategy, so a task cannot mix a shared and a dedicated GPU container.
  • memory must be greater than 0, and maxComputePercent, if given, must be an integer from 1 to 100.
  • You cannot specify sharingStrategy in a container override. Overrides in RunTask and StartTask do not support GPU sharing configuration, so it must live in the task definition.

Amazon ECS also sets reserved environment variables in each sharing container, CUDA_MPS_PINNED_DEVICE_MEM_LIMIT and CUDA_MPS_ACTIVE_THREAD_PERCENTAGE, and you should not set them yourself [3].

Isolation and failure modes

This is the section to read before approving a rollout. The documentation lists workloads for which sharing is not recommended: large model training that needs the full GPU memory, workloads sensitive to GPU faults from neighbouring containers, and workloads that require hardware-level isolation between processes [3]. The reason given is that MPS is software-level sharing and all containers on a GPU share the same CUDA context [3].

If one container triggers a fatal GPU error, such as an illegal memory access from a faulty GPU kernel, the containers sharing that physical GPU could be affected, while containers on other GPUs on the same instance are not [3]. A further point is multi-process containers. The guidance is to run a single GPU process per container; for applications that need several, set the memory requirement from the combined peak usage and use application-level controls, because ECS uses the configured value for placement and multi-process applications can consume more than configured, causing memory contention [3].

Service boundaries are also soft. Tasks from different services on the same capacity provider can be placed on the same GPU, and to prevent that you use separate capacity providers or separate clusters [3]. If your tenants or environments must not share a GPU, plan that boundary up front.

Operations and observability

On ECS Managed Instances, ECS monitors the health of GPUs and the MPS daemon. If it detects a GPU hardware failure or an MPS daemon failure, it drains the instance and replaces it [3]. The announcement describes the same behaviour: if the daemon stops or becomes unhealthy, ECS replaces the instance and moves tasks to healthy capacity [1].

Monitoring changes, and this is easy to miss. For GPUs shared with MPS, ECS does not emit the per-task and per-container GPU utilization, GPU memory utilization, GPU memory total, or GPU memory used metrics [3]. Power draw, temperature and restart Xid count metrics remain available per task and container, as do instance-level metrics such as InstanceGPUUsageTotal and InstanceGPULimit [3]. You can confirm a GPU is shared by calling DescribeContainerInstances, whose gpuDevices field includes the sharing mode, container count and remaining memory per GPU, and view per-task limits with DescribeTasks [3].

To judge whether sharing pays off, the documentation suggests comparing RunningTaskCount before and after enabling it, and watching InstanceGPUUsageTotal against InstanceGPULimit to see how full an instance is [3].

Decision guide

SituationFit for MPS sharing
Embedding generation or small-model inference that leaves GPU capacity idleGood fit [1][3]
Multi-model serving where models can share one GPUGood fit [3]
Development and test environmentsGood fit [3]
Large training jobs needing full GPU memoryNot recommended [3]
Workloads sensitive to neighbour GPU faultsNot recommended [3]
Requirement for hardware-level isolationNot recommended [3]
Fargate or ECS AnywhereNot supported [3]

Architect's checklist

  • Confirm the workload uses only part of a GPU's memory and compute; AWS positions sharing for exactly that case [1][3].
  • Confirm your instance family is on the supported list: G4dn, G5, G6, G6e, G7, G7e, P3, P4d, P5, P5e, P5en or P6-B200 [3].
  • On EC2, verify container agent 1.107.0 or later and AL2023 GPU-optimized AMI 20261019 or later [3].
  • Set memory per container as a hard ceiling, and base it on peak usage; ECS places tasks on memory alone [3].
  • Treat maxComputePercent as a ceiling, not a reservation [3].
  • Run one GPU process per container, or size memory for the combined peak of all processes [3].
  • Decide whether a fatal GPU error affecting co-located containers is acceptable for each workload [3].
  • If services or tenants must not share a GPU, use separate capacity providers or clusters [3].
  • Move the sharing configuration into the task definition, since container overrides do not support it [3].
  • Update dashboards and alarms that depend on per-container GPU utilization or memory metrics, which are not emitted for shared GPUs [3].

Sources

Numbers in square brackets refer to these sources, checked on 2026-10-28.

  1. Amazon ECS now supports GPU sharing with NVIDIA Multi-Process Service (MPS) - AWS
  2. Amazon ECS task definitions for GPU workloads - Amazon Elastic Container Service
  3. Share GPUs on Amazon ECS - Amazon Elastic Container Service
Tags:amazon-ecsgpunvidia-mpsinferencecost
Share Article

Responses (0)

Sign in to join the conversation and share your thoughts.

P&C

Prompt & Circuit Editorial

Contributing Editor

Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.