Amazon Web Services, known as AWS, has just revealed interesting details about its key artificial intelligence service. According to Julie White, AWS's chief marketing officer, more than half of the Bedrock service runs on its own Trainium chips. She shared this information in an interview with The Information’s TITV. Bedrock allows customers to access artificial intelligence models from companies such as Anthropic and other providers.
Previously, Amazon executives had not shared such details about Trainium's use in Bedrock. The Bedrock service also runs on Nvidia graphics processors, but AWS is now relying heavily on its in-house technology. Trainium is not a GPU, but a specialized chip designed to accelerate the training and inference of artificial intelligence models. This approach allows AWS to achieve better gross margins from AI because Trainium is cheaper than Nvidia GPUs.
Aggressive discounts on Trainium servers
AWS offers Trainium-powered cloud servers at significantly lower prices than those featuring Nvidia chips. For example, EC2 Trn2 instances with Trainium2 deliver up to four times the performance of the first-generation Trainium. These instances achieve a 30–40% better price-performance ratio than top-tier EC2 instances with Nvidia GPUs, such as P5e or P5en.
Trainium2 chips have up to 96 GB of HBM3e memory per chip and support advanced interconnects such as NeuronLink and EFA. This enables scaling to as many as 100,000 chips for training large artificial intelligence models. In real-world generative AI workloads, they achieve up to a 40% better price-performance ratio. AWS therefore sells these servers at discounts of up to 50% compared to Nvidia, reducing training costs by as much as half.
Customer benefits and compatibility
Bedrock customers use Trainium to optimize latency in generative AI. For example, Anthropic's Claude 3.5 Haiku model runs 60% faster on Trainium2. The chips support frameworks such as PyTorch and JAX, making migration and deployment easier for developers.
AWS continues to offer Nvidia GPUs but is increasing the share of Trainium to reduce its dependence on external suppliers. This approach delivers operating cost savings and strengthens AWS's position in scalable solutions for large models with high memory requirements.
Trainium2 is well suited for workloads with high memory requirements and large models. AWS thus offers an ecosystem in which customers can easily switch between technologies. Trainium instances provide competitive training and inference performance, while the discounts make this option attractive to companies seeking affordable cloud solutions.
Source: theinformation.com
