Customize timeouts for faster automatic failover on Vercel AI Gateway | Endigest
Vercel
Get the latest tech trends every morning
Receive daily AI-curated summaries of engineering articles from top tech companies worldwide.
Vercel AI Gateway now supports per-inference provider timeouts to enable faster automatic failover between AI providers.
- •Configurable timeouts (in milliseconds) can be set per provider using `providerTimeouts` inside `providerOptions.gateway`
- •If a provider fails to start responding within the configured timeout, AI Gateway aborts the request and falls back to the next available provider
- •Currently available in beta for BYOK (Bring Your Own Key) credentials only; system provider timeout support is coming soon
- •The `order` field can be combined with `providerTimeouts` to control both provider sequence and failover speed
- •Some providers do not support stream cancellation, so timed-out requests may still incur charges depending on the provider
This summary was automatically generated by AI based on the original article and may not be fully accurate.
Google Cloud
Databricks
Stripe