> ## Documentation Index
> Fetch the complete documentation index at: https://docs.getlimina.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS ECS GPU Checklist

> The host, agent, and task definition settings ECS requires before it will run the Limina GPU container

Both ECS deployment guides ([automated](/installation/aws/automated-ecs-guide.md) and [manual](/installation/aws/manual-ecs-guide.md)) deploy the CPU container. Swapping the image tag is not enough since ECS only exposes a GPU to a container when the host, the ECS agent and the task definition have all been configured for it. If any of the configurations are missed, the task will start, fail its GPU check and exit.

> [!WARNING]
>
> GPU tasks must use the EC2 launch type. Fargate does not support either GPUs or the `sharedMemorySize` setting that the GPU container requires.

## Requirements

| # | Requirement | Where it is configured | Value |
| - | - | - | - |
| 1 | GPU instance type | EC2 launch template | `g4dn.4xlarge` or larger |
| 2 | GPU-optimized ECS AMI | EC2 launch template | `/aws/service/ecs/optimized-ami/amazon-linux-2023/gpu/recommended/image_id` |
| 3 | GPU support in the ECS agent | EC2 user data, written to `/etc/ecs/ecs.config` | `ECS_ENABLE_GPU_SUPPORT=true` |
| 4 | Root volume for the GPU image | EC2 launch template | “150 GiB” or more, `gp3` |
| 5 | A GPU reserved for the container | Task definition, `resourceRequirements` | `{"type": "GPU", "value": "1"}` |
| 6 | 4 GiB of shared memory | Task definition, `linuxParameters.sharedMemorySize` | `4096` (MiB) |
| 7 | 56 GiB of RAM for the `gpu` build | Task definition, `memory` | `57344` (MiB) |
| 8 | Startup time | Task definition, ECS service, agent config | `startPeriod` `300`, grace period `600`, `ECS_CONTAINER_START_TIMEOUT=10m` |
| 9 | EC2 launch type | Task definition | `EC2`, with Fargate unchecked |
| 10 | A GPU build of the image | Task definition | `<version>-gpu` |

> [!NOTE]
>
> The 56 GiB figure applies to the `gpu` and `gpu-synthetic` builds which include file support. The text-only `gpu-text` build requires 28 GiB so a `memory` of `28672` is enough for it. The container also needs an Nvidia GPU with compute capability 7.0 or higher and at least 16GB VRAM. Please refer to [Prerequisites and System Requirements](/installation/prerequisites-and-system-requirements.md) and [Grabbing the Image](/installation/grabbing-the-image.md).

## Apply the Changes

**1. Confirm GPU quota and availability**

The "Running On-Demand G and VT instances" quota is expressed in vCPUs and is `0` in many new accounts. A `g4dn.4xlarge` requires 16. GPU instance types are also absent from some regions and availability zones which surfaces as `InsufficientInstanceCapacity` from the Auto Scaling group.

```shell AWS Command wrap theme={"theme":"poimandres"}
aws ec2 describe-instance-type-offerings --location-type availability-zone --filters Name=instance-type,Values=g4dn.4xlarge --query "InstanceTypeOfferings[].Location" --output text --region <your-region>
```

**2. Use the GPU-optimized AMI**

The standard ECS AMI has no Nvidia driver and no Nvidia container runtime even on a GPU instance. Set the launch template's AMI to the Systems Manager parameter which resolves to the current GPU AMI in whichever region you deploy to:

```shell Launch template AMI theme={"theme":"poimandres"}
resolve:ssm:/aws/service/ecs/optimized-ami/amazon-linux-2023/gpu/recommended/image_id
```

To pick the AMI by hand, search “Community AMIs” for `al2023-ami-ecs-gpu` and confirm the owner is `amazon`; these are published by Amazon, not by a Marketplace seller. AMI IDs differ per region so never copy one between regions. In the automated guide, set `cluster.yaml`'s `ECSAMI` parameter to the same path.

**3. Enable GPU support in the ECS agent**

The agent only advertises the host's GPUs when `ECS_ENABLE_GPU_SUPPORT` is set. The timeouts are not GPU requirements but the defaults are often too short for the first pull of the larger GPU image.

```shell User data theme={"theme":"poimandres"}
#!/bin/bash
cat << EOF >> /etc/ecs/ecs.config
ECS_CLUSTER=environment-name-cluster
ECS_ENABLE_GPU_SUPPORT=true
ECS_IMAGE_PULL_INACTIVITY_TIMEOUT=10m
ECS_CONTAINER_START_TIMEOUT=10m
EOF
```

Do not set it on a host with no GPU as the agent then fails to initialize and never registers with the cluster.

**4. Add the GPU settings to the task definition**

ECS assigns GPUs only when the container definition requests them, pinning device IDs and injecting `NVIDIA_VISIBLE_DEVICES`.

```json Container Definition lines wrap theme={"theme":"poimandres"}
{
  "image": "<your-registry>/deid:<version>-gpu",
  "resourceRequirements": [{ "type": "GPU", "value": "1" }],
  "linuxParameters": { "sharedMemorySize": 4096 },
  "memory": 57344,
  "ulimits": [
    { "name": "memlock", "softLimit": -1, "hardLimit": -1 },
    { "name": "stack", "softLimit": 67108864, "hardLimit": 67108864 }
  ],
  "healthCheck": { "startPeriod": 300 }
}
```

In the console these are: the GPU limit under "Resource allocation limits"; the shared memory size under "Docker configuration"; and "Resource limits (Ulimits)". The license file still has to be mounted at `/app/license/license.json`.

`memory` sets the cgroup limit that the container reads so it has to meet the requirement on its own and stay below the instance's registered capacity or the task is never placed. A `g4dn.4xlarge` registers 63596 MiB of its 65536 MiB leaving `57344` roughly 6 GiB of headroom.

**5. Extend the service health check grace period**

Set `healthCheckGracePeriodSeconds` to `600` otherwise the load balancer marks the target unhealthy and ECS kills the task while the image is still being pulled.

> [!NOTE]
>
> `main.yaml` exposes neither the instance type nor the AMI as root-stack parameters so GPU deployments cannot use the Quickstart command. Use the "Optional - Customize deployment" step of the [automated ECS guide](/installation/aws/automated-ecs-guide.md) to edit `cluster.yaml` and `service.yaml`, then host them in your own S3 bucket.

## Verify the Deployment

**1. Check the host**

On the ECS host, run `nvidia-smi` and `grep GPU /etc/ecs/ecs.config`.

**2. Confirm the agent registered the GPU**

The container instance should report a `GPU` resource listing one device ID per GPU. While it is empty, no task definition change will help.

```shell AWS Command wrap theme={"theme":"poimandres"}
aws ecs describe-container-instances --cluster environment-name-cluster --container-instances $(aws ecs list-container-instances --cluster environment-name-cluster --query "containerInstanceArns[0]" --output text) --query "containerInstances[].registeredResources[?name=='GPU']"
```

**3. Check the task logs**

A container that found its GPU logs the Triton Inference Server startup; one that did not logs `The container is unable to detect an NVIDIA GPU` and exits. Logs require an `awslogs` configuration and a task execution role with `AmazonECSTaskExecutionRolePolicy` without which the task fails on a logger error instead.

> [!NOTE]
>
> The GPU container is sized for roughly 16 concurrent requests, compared with 1 for the CPU container. Please visit the [recommended concurrency levels](/configuration-and-operations/container-management/concurrency.md) page before tuning your client or autoscaling policy.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.