Slack AI: The Path to Multi-Cloud

Slack evolved from AWS SageMaker to a multi-cloud LLM infrastructure over three years.

• •SageMaker provided enterprise security and FedRamp compliance but faced scaling latency, GPU scarcity, and over-provisioning costs
• •Migrated to Amazon Bedrock for operational simplicity and immediate access to latest models with Provisioned Throughput and On-Demand options
• •Hybrid routing strategy keeps latency-sensitive features on dedicated capacity while moving bursty workloads to On-Demand infrastructure
• •Spillover Pattern automatically directs excess requests to on-demand endpoints during peak usage to prevent capacity drops
• •Achieved zero-incident migration through compliance verification, extensive load testing, A/B testing, and gradual traffic shifts