Last week, the latest AWS Heroes arrived! AWS Heroes are amazing technical experts who generously share their insights, best practices, and innovative solutions to help others. The AWS GenAI Lofts are in full swing with San Francisco and São Paulo open now, and London, Paris, and Seoul coming in the next couple of months. Here’s […]

AWS Weekly Roundup: Amazon DynamoDB, AWS AppSync, Storage Browser for Amazon S3, and more (September 9, 2024)

Generate photorealistic images with exceptional detail. Enhance visuals effortlessly across industries with improved multi-subject prompts, stunning image quality, and impeccable typography.

Stability AI’s best image generating models now in Amazon Bedrock

With AWS Parallel Computing Service, run HPC workloads at virtually any scale effortlessly; leverage Slurm and managed clusters to accelerate simulations.

Announcing AWS Parallel Computing Service to run HPC workloads at virtually any scale

AWS Community Days have been in full swing around the world. I am going to put the spotlight on AWS Community Day Argentina where Jeff Barr delivered the keynote, talks and shared his nuggets of wisdom with the community, including a fun story of how he once followed Bill Gates to a McDonald’s! I encourage […]

AWS Weekly Roundup: Amazon EC2 X8g Instances, Amazon Q generative SQL for Amazon Redshift, AWS SDK for Swift, and more (Sep 23, 2024)

The AWS User Group Japan (JAWS-UG) hosted JAWS PANKRATION 2024 themed ‘No Border’. This is a 24-hour online event where AWS Heroes, AWS Community Builders, AWS User Group leaders, and others from around the world discuss topics ranging from cultural discussions to technical talks. One of the speakers at this event, Kevin Tuei, an AWS […]

AWS Weekly Roundup: S3 Conditional writes, AWS Lambda, JAWS Pankration, and more (August 26, 2024)

Unleash unparalleled computing prowess and energy efficiency for memory-intensive workloads, from in-memory databases to real-time analytics, with C8g and M8g instances powered by AWS Graviton4 processors.

Run your compute-intensive and general purpose workloads sustainably with the new Amazon EC2 C8g, M8g instances

Amazon Web Services (AWS) AI code assistant Amazon Q Developer named a Leader in Gartner's first Magic Quadrant for its Ability to Execute and Completeness of Vision, driving innovation across the software development lifecycle with enterprise-grade controls.

AWS named as a Leader in the first Gartner Magic Quadrant for AI Code Assistants

Amazon Elastic Kubernetes Service (EKS) support in Amazon SageMaker HyperPod

 — purpose-built infrastructure engineered with resilience at its core for foundation model (FM) development. This new capability enables customers to orchestrate HyperPod clusters using EKS, combining the power of 

‘s resilient environment designed for training large models. Amazon SageMaker HyperPod helps efficiently scale across more than a thousand artificial intelligence (AI) accelerators, reducing training time by up to 40%.

Amazon SageMaker HyperPod now enables customers to manage their clusters using a Kubernetes-based interface. This integration allows seamless switching between Slurm and Amazon EKS for optimizing various workloads, including training, fine-tuning, experimentation, and inference. The CloudWatch Observability EKS add-on provides comprehensive monitoring capabilities, offering insights into CPU, network, disk, and other low-level node metrics on a unified dashboard. This enhanced observability extends to resource utilization across the entire cluster, node-level metrics, pod-level performance, and container-specific utilization data, facilitating efficient troubleshooting and optimization.

 has become a go-to solution for AI startups and enterprises looking to efficiently train and deploy large scale models. It is compatible with 

SageMaker’s distributed training libraries

 software optimizations that help reduce training time by up to 20%. SageMaker HyperPod automatically detects and repairs or replaces faulty instances, enabling data scientists to train models uninterrupted for weeks or months. This allows data scientists to focus on model development, rather than managing infrastructure.

The integration of Amazon EKS with Amazon SageMaker HyperPod uses the advantages of Kubernetes, which has become popular for machine learning (ML) workloads due to its scalability and rich open-source tooling. Organizations often standardize on Kubernetes for building applications, including those required for generative AI use cases, as it allows reuse of capabilities across environments while meeting compliance and governance standards. Today’s announcement enables customers to scale and optimize resource utilization across more than a thousand AI accelerators. This flexibility enhances the developer experience, containerized app management, and dynamic scaling for FM training and inference workloads.

Amazon EKS support in Amazon SageMaker HyperPod strengthens resilience through deep health checks, automated node recovery, and job auto-resume capabilities, ensuring uninterrupted training for large scale and/or long-running jobs. Job management can be streamlined with the optional 

, designed for Kubernetes environments, though customers can also use their own CLI tools. Integration with 

 provides advanced observability, offering deeper insights into cluster performance, health, and utilization. Additionally, data scientists can use tools like Kubeflow for automated ML workflows. The integration also includes Amazon SageMaker managed MLflow, providing a robust solution for experiment tracking and model management.

At a high level, Amazon SageMaker HyperPod cluster is created by the cloud admin using the 

 and is fully managed by the HyperPod service, removing the undifferentiated heavy lifting involved in building and optimizing ML infrastructure. Amazon EKS is used to orchestrate these HyperPod nodes, similar to how Slurm orchestrates HyperPod nodes, providing customers with a familiar Kubernetes-based administrator experience.

Let’s explore how to get started with Amazon EKS support in Amazon SageMaker HyperPod


I start by preparing the scenario, checking the 

, and creating an Amazon EKS cluster with a single 

, configured with VPC and storage resources.

To create and manage Amazon SageMaker HyperPod clusters, I can use either the 

. Using the AWS CLI, I specify my cluster configuration in a JSON file. I choose the Amazon EKS cluster created previously as the orchestrator of the SageMaker HyperPod Cluster. Then, I create the cluster worker nodes that I call “worker-group-1”, with a private 

 to enable automatic node recovery and for 

The AWS command returns the ARN of the new HyperPod cluster.

I then verify the HyperPod cluster status in the 

Alternatively, you can check the cluster status using the AWS CLI running the 

aws sagemaker describe-cluster --cluster-name my-hyperpod-cluster

Once the cluster is ready, I can access the SageMaker HyperPod cluster nodes. For most operations, I can use 

 commands to manage resources and jobs from my development environment, using the full power of Kubernetes orchestration while benefiting from SageMaker HyperPod’s managed infrastructure. On this occasion, for advanced troubleshooting or direct node access, I use 

 (SSM) to log into individual nodes, following the instructions in the 

Access your SageMaker HyperPod cluster nodes

To run jobs on the SageMaker HyperPod cluster orchestrated by EKS, I follow the steps outlined in the 

Run jobs on SageMaker HyperPod cluster through Amazon EKS page

. You can use the HyperPod CLI and the native 

 command to find avaible HyperPod clusters and submit training jobs (Pods). For managing ML experiments and training runs, you can use 

Finally, in the SageMaker Console, I can view the 

 of recently added EKS clusters, providing a comprehensive overview of my SageMaker HyperPod environment.

And I can monitor cluster performance and health metrics using 


Here are some key things you should know about Amazon EKS support in Amazon SageMaker HyperPod:

 – This integration provides a more resilient training environment with deep health checks, automated node recovery, and job auto-resume. SageMaker HyperPod automatically detects, diagnoses, and recovers from faults, allowing you to continually train foundation models for weeks or months without disruption. This can reduce training time by up to 40%.

 provides detailed metrics and logs for your containerized applications and microservices. This enables comprehensive monitoring of cluster performance and health.

 – This launch includes a custom HyperPod CLI for job management, Kubeflow Training Operators for distributed training, Kueue for scheduling, and integration with SageMaker Managed MLflow for experiment tracking. It also works with SageMaker’s distributed training libraries, which provide Model Parallel and Data Parallel optimizations to significantly reduce training time. These libraries, combined with auto-resumption of jobs, enable efficient and uninterrupted training of large models.

 – This integration enhances developer experience and scalability for FM workloads. Data scientists can efficiently share compute capacity across training and inference tasks. You can use your existing Amazon EKS clusters or create and attach new ones to HyperPod compute, bring your own tools for job submission, queuing and monitoring.

To get started with Amazon SageMaker HyperPod on Amazon EKS, you can explore resources such as the 

. This release is generally available in the AWS Regions where Amazon SageMaker HyperPod is available except Europe(London). For pricing information, visit the 

This blog post was a collaborative effort. I would like to thank Manoj Ravi, Adhesh Garg, Tomonori Shimomura, Alex Iankoulski, Anoop Saha, and the entire team for their significant contributions in compiling and refining the information presented here. Their collective expertise was crucial in creating this comprehensive article.

Amazon SageMaker HyperPod introduces Amazon EKS support

Every week, there’s a new Amazon Web Services (AWS) community event where you can network, learn something new, and immerse yourself in the community. When you’re in a community, everyone grows together, and no one is left behind. Last week was no exception. I can highlight the Dach AWS Community Day where Viktoria Semaan closed […]

AWS Weekly Roundup: Jamba 1.5 family, Llama 3.2, Amazon EC2 C8g and M8g instances and more (Sep 30, 2024)

AI21's Jamba 1.5 models enable high-performance long-context language processing up to 256K tokens, with JSON output support and multilingual capabilities across 9 languages.

Jamba 1.5 family of models by AI21 Labs is now available in Amazon Bedrock

Hello, everyone! It’s been an interesting week full of AWS news as usual, but also full of vibrant faces filling up the rooms in a variety of events happening this month. Let’s start by covering some of the releases that have caught my attention this week. My Top 3 AWS news announcements of the week […]

AWS Weekly Roundup: Oracle Database@AWS, Amazon RDS, AWS PrivateLink, Amazon MSK, Amazon EventBridge, Amazon SageMaker and more

Pushing the boundaries of generative AI, Meta unveils Llama 3.2, a groundbreaking language model family featuring enhanced capabilities, broader applicability, and multimodal image support, now available in Amazon Bedrock.

Introducing Llama 3.2 models from Meta in Amazon Bedrock: A new generation of multimodal vision and lightweight models

New memory-optimized X8g instances offer up to 3 TiB DDR5 memory, 192 vCPUs, and 50 Gbps network bandwidth, designed for memory-intensive workloads like databases, analytics, and caching with unparalleled price/performance and efficiency.

Now available: Graviton4-powered memory-optimized Amazon EC2 X8g instances

The AWS Heroes program recognizes outstanding individuals who are making meaningful contributions within the AWS community. These technical experts generously share their insights, best practices, and innovative solutions to help others create efficiencies and build faster on AWS. Heroes are thought leaders who have demonstrated a commitment to empowering the broader AWS community through their […]

The latest AWS Heroes have arrived – September 2024

With the arrival of September, AWS re:Invent 2024 is now 3 months away and I am very excited for the new upcoming services and announcements at the conference. I remember attending re:Invent 2019, just before the COVID-19 pandemic. It was the biggest in-person re:Invent with 60,000+ attendees and it was my second one. It was […]

AWS Weekly Roundup: AWS Parallel Computing Service, Amazon EC2 status checks, and more (September 2, 2024)

Near real-time data replication: RDS for MySQL zero-ETL with Redshift streamlines data ingestion with filtering, multiple integrations, and CloudFormation support for customized insights without managing complex pipelines.

Amazon RDS for MySQL zero-ETL integration with Amazon Redshift, now generally available, enables near real-time analytics

Gain indispensable data engineering expertise through a hands-on specialization by DeepLearning.AI and AWS. This professional certificate covers ingestion, storage, querying, modeling, and more.

Data engineering professional certificate: New hands-on specialization by DeepLearning.AI and AWS

Secure your mission-critical data with S3 Express One Zone's server-side encryption using KMS keys, combining top-notch performance and robust security for regulatory compliance.

Amazon S3 Express One Zone now supports AWS KMS with customer managed keys

The 2024 Gartner Magic Quadrant positions AWS as a Leader, reflecting our commitment to diverse virtual desktop solutions and operational excellence - driving innovation for remote and hybrid workforces.

AWS named as a Leader in the 2024 Gartner Magic Quadrant for Desktop as a Service (DaaS)

Discover how this high-performance remote protocol securely streams graphics-intensive apps from cloud to device, with QUIC, client enhancements, and extended OS support.