Lyft Rearchitects ML Platform with Hybrid AWS SageMaker-Kubernetes Approach

Lyft Rearchitects ML Platform with Hybrid AWS SageMaker-Kubernetes Approach

Lyft rearchitected its machine learning platform, LyftLearn, into a hybrid system leveraging AWS SageMaker for offline tasks and retaining Kubernetes for online model serving. The platform supports hundreds of millions of daily predictions and thousands of training jobs, previously running entirely on Kubernetes.

The shift addresses the growing operational complexity of managing a fully custom Kubernetes-based ML infrastructure at scale, freeing up engineering resources for core platform development. Maintaining separate infrastructures for training and serving allows for optimized resource allocation and cost efficiency.

Why This Matters

Traditional monolithic ML platforms often struggle with balancing operational overhead and control. Maintaining custom Kubernetes infrastructure for both training and serving requires significant engineering effort, especially as scale increases. Lyft’s approach recognizes that managed services like SageMaker can effectively handle the complexity of batch processing, while Kubernetes remains ideal for real-time inference where existing tooling and performance are critical.

Key Insights

• LyftLearn operational complexity: Increased with scale, requiring custom orchestration and state synchronization.
• SageMaker benefits: Reduced engineering overhead by handling cluster autoscaling and event-driven state management.
• Seekable OCI (SOCI): Enables Kubernetes-comparable startup times for frequent model retraining (every 15 minutes).

Working Example

# Example Dockerfile for cross-platform compatibility
FROM python:3.9-slim-buster

# Install necessary dependencies
RUN pip install --no-cache-dir pandas scikit-learn

# Copy your model training script
COPY train.py .

# Set entrypoint
ENTRYPOINT ["python", "train.py"]

Practical Applications

• E-commerce: Utilizing SageMaker for batch model training and Kubernetes for real-time personalization.
• Pitfall: Over-reliance on a single platform can lead to vendor lock-in and hinder innovation.