aws-solutions-library-samples/global-capacity-orchestrator-on-aws

Global Capacity Orchestrator (GCO) runs accelerated workloads — LLM training and inference, batch ML, HPC — on EKS Auto Mode clusters in as many AWS Regions as you configure, behind one IAM-authenticated API, CLI and MCP server. It finds where NVIDIA GPU, Trainium, Inferentia and CPU capacity actually is, places jobs there, and serves inference endpoints with automatic cross-Region failover.

A real gco session, reproducible from demo/live_demo.sh: fleet status, capacity discovery, four schedulers plus KEDA, shared storage, a globally replicated vector store and live LLM inference. More recordings are under See it running.

Why GCO?

Running accelerated workloads across Regions means finding capacity, standing up clusters, wiring authentication, handling failover, and keeping job outputs after the pods exit. GCO packages all of it as one platform you deploy with a single command:

• Capacity-aware placement. gco capacity reads spot placement scores, spot price history, On-Demand Capacity Reservations and Capacity Blocks for ML, and auto-Region submission builds on them. For guaranteed capacity it buys a Capacity Block or creates an ODCR, and gco nodepools generates the NodePool that runs your pods on it.
• One authenticated front door. A SigV4 API, the gco CLI and an MCP server reach every Region with the AWS credentials you already have, with no kubeconfig distribution.
• Accelerators on demand. EKS Auto Mode NodePools for NVIDIA GPU (x86 and Arm), EFA, Trainium, Inferentia and CPU launch nodes only when a workload needs them.
• Inference in every Region. One command deploys vLLM, SGLang or Triton endpoints everywhere, with automatic failover through Global Accelerator in the commercial aws partition.
• A curated ecosystem. KEDA, Volcano, Kueue, KubeRay and Kubeflow Trainer are on by default and Slurm, YuniKorn, Argo CD and Crossplane are opt-in; EFS ships with every cluster, and FSx for Lustre, Valkey and Aurora pgvector are a toggle away; Prometheus, Grafana, OpenCost and MLflow are on by default for observability and cost tracking.
• Agent-ready. gco autopilot opens Claude Code, OpenAI Codex or OpenCode on Amazon Bedrock with the GCO MCP server already wired in.

Get started

Prerequisites

• An AWS account you can create infrastructure in, with credentials the AWS CLI can use
• Git and a container runtime: Docker, Finch or Podman (Colima also works)

The dev container brings everything else — Python 3.14, Node.js 24, CDK, kubectl, the AWS CLI with the Session Manager plugin, and Docker CLI + Buildx — at pinned versions. A host install is possible but advanced; see More ways to run GCO below.

Deploy

git clone https://github.com/aws-solutions-library-samples/global-capacity-orchestrator-on-aws.git
cd global-capacity-orchestrator-on-aws
./scripts/setup-dev-alias.sh   # builds the dev container and installs the `gco` shell function
source ~/.zshrc                # or ~/.bashrc — the script prints which file it updated
gco stacks deploy-all -y       # stands up every Region in cdk.json; billing starts here

deploy-all bootstraps CDK where needed, deploys the global control plane and one regional stack per Region, and then installs the cluster add-ons in the background.

Run your first workload

gco capacity check --instance-type g4dn.xlarge --region us-east-1   # where is capacity right now?
gco jobs submit-sqs examples/simple-job.yaml --region us-east-1     # submit a job over SQS
gco jobs list --all-regions                                         # watch it run
gco jobs logs hello-gco -n gco-jobs -r us-east-1                    # read its output

Serve a model from every Region with one command:

gco inference deploy my-llm -i vllm/vllm-openai:v0.32.0 --gpu-count 1 -e MODEL=Qwen/Qwen2.5-1.5B-Instruct
gco inference status my-llm                # rollout state in every Region
gco inference invoke my-llm -p "Hello"     # an OpenAI-compatible completion
gco inference scale my-llm --replicas 3    # scale it out

Let an agent drive

gco autopilot                     # default Claude Code session
gco autopilot --engine codex      # or an OpenAI Codex session
gco autopilot --engine opencode   # or an OpenCode session (Kimi K3)

Every engine runs on Amazon Bedrock with your AWS credentials and the GCO MCP server plus the recommended companion MCP servers wired in, so you can simply ask: “deploy everything”, “where is p5 capacity cheapest right now?” The Autopilot Guide covers sessions, opt-in tool groups and dry runs.

Clean up

gco stacks destroy-all -y   # destroys every stack, then sweeps known leftovers

Teardown is not an account-wide emptiness guarantee: retained ECR repositories, resources configured for retention, and unexpected resources can remain. See gco stacks destroy-all for the exact cleanup scope.

The Quick Start Guide walks through every step with the output to expect, and the CLI Reference lists every command.

See it running

Real terminal recordings, not mock-ups. Each one is reproducible from the script linked beneath it; the live demo itself plays at the top of this page.

Install the MCP server

GCO ships with the GCO MCP server — an MCP server exposing 151 tools by default (up to 210 with feature flags) that index the whole project: docs, examples, source code, K8s manifests, and scripts. Connect it to an AI-powered IDE with MCP support and explore GCO conversationally — “How does region recommendation work?”, “Walk me through the inference deployment flow” — or let it drive real operations. One click adds it (pinned to the latest release, no clone needed) to your client; uv and AWS credentials are the only prerequisites. Other clients and options: setup guide.

Architecture Overview

Figure 1: GCO multi-region reference architecture — global control plane and workload entry

One CDK app deploys a global control plane (the API Gateway entry point, Global Accelerator in aws, shared DynamoDB state, the model bucket and monitoring) and one regional stack per Region: an EKS Auto Mode cluster with GCO’s platform services, NodePools, storage and job queue. Architecture Details is the full deep dive, and the AWS Solutions Guidance presents the reference architecture in the AWS Solutions Library.

Security Model

Six complementary controls protect every backend request: IAM authentication at API Gateway, TLS trust separation, a request-bound HMAC, private backend exposure, freshness and integrity validation in the backend middleware, and IRSA / EKS Pod Identity for pod-level AWS access.

Key Features

Everything GCO does, grouped by area; expand one for the details and its guide.

AWS Services in this Guidance

GCO is published in the AWS Solutions Library as the Guidance for EKS AutoMode Clusters with Global Capacity Orchestrator on AWS. It is built from 31 AWS services.

Sample Cost Table

A single-Region deployment with default settings carries roughly $210 a month of fixed platform cost before any workload runs. GPU instances dominate real spend and scale with the hours they run (one on-demand g5.xlarge around the clock is about $734 a month in us-east-1), and multi-Region deployments scale linearly. gco costs summary tracks what you actually spend.

Supported AWS Regions

GCO deploys to any AWS Region in the aws, aws-cn or GovCloud partitions, as many as you list in cdk.json under deployment_regions.regional. GPU availability varies by Region, so gco capacity recommend-region --gpu is a good first stop.

Documentation

Prefer a website? The project wiki is a short orientation site — what GCO is, how it works, what you can run, and where to go deeper — published from this repository with the live coverage reports for Python, Bash and Node.js embedded.

Project Tenets

GCO is guided by the prioritized project tenets, beginning with workload, data, and account safety and anchored by the north star One API. Every Accelerator. Any Region. The tenets define how the project resolves trade-offs across truthful state, security, regional behavior, accelerator policy, automation, recovery, operations, cost, and maintainability. Earlier tenets outrank later ones; durable exceptions require an Architecture Decision Record.

Project Structure

Contributing

See CONTRIBUTING.md for development setup, testing, the GitHub Actions CI/CD layout, the release process, and dependency scanning schedules. The whole test-suite runs from the same dev container scripts/setup-dev-alias.sh builds:

docker run --rm -v $(pwd):/workspace -w /workspace gco-dev pytest tests/ -v

Support

• Check Troubleshooting for common issues
• Review CloudWatch logs for Lambda and EKS errors
• Open an issue on GitHub

Security

GCO implements defense in depth across the six controls of its Security Model, checks its infrastructure against five cdk-nag rule packs, and scans container images and dependencies in CI.

Reporting a vulnerability: do not open a public GitHub issue; follow the responsible disclosure process in .github/SECURITY.md.

License

See the LICENSE file for details.