SoftBank Launches Infrinia AI Cloud OS for GPU Cloud Services

Infrinia Cloud OS meets growing global demands

SoftBank’s Infrinia AI Cloud OS is a software stack custom-designed for AI data centres, enabling data centre operators to deliver Kubernetes-as-a-service (KaaS) and inference-as-a-service (Inf-aaS) in multi-tenant settings. The software stack is expected to reduce total cost of ownership (TCO) and streamline day-to-day complexities, particularly when compared to options developed internally and custom-made stacks, with a projected reduction of 30% in TCO.

Why This Matters

The technical reality of managing GPU cloud services is far more complex than ideal models suggest, with many organizations struggling to balance the needs of different users, from fully managed systems to affordable AI inference without direct GPU management. This complexity can lead to significant costs, with some estimates suggesting that inefficient GPU management can result in costs upwards of $100,000 per year for large-scale deployments.

Key Insights

• Infrinia AI Cloud OS automates every layer of the underlying infrastructure, from low-level server settings to storage, networking, and Kubernetes itself, allowing for more efficient management of GPU cloud services.
• The use of KaaS and Inf-aaS enables faster and more scalable access to AI model inference through managed services, reducing the need for manual intervention and minimizing delays.
• Automated node allocation, based on NVIDIA NVLink domains, helps reduce delays and improves GPU-to-GPU bandwidth for larger scale, distributed workloads, making it an attractive solution for organizations with complex AI workloads.

Practical Applications

• Use Case: SoftBank plans to incorporate Infrinia Cloud OS into its existing GPU cloud offerings, providing customers with a streamlined and efficient way to access AI services.
• Pitfall: Organizations that fail to adopt automated management solutions like Infrinia AI Cloud OS may struggle with inefficient GPU management, leading to increased costs and reduced performance.

References:

• https://www.cloudcomputing-news.net/news/infrinia-ai-for-cloud-kaas-released-by-softbank/