AI & Tech··1 sources verified
AWS Lambda Adds GPU Inference, Sharing One Model Across Up to 48 Concurrent Invocations
Summarised by Relevant News AI · Read time: 3 min
AWS Lambda now runs Event Functions on EC2 G-series GPU instances through Lambda Managed Instances, with up to 48 concurrent invocations sharing one model per GPU. Prebuilt vLLM and SGLang base images need no handler code, and asynchronous invocations can run for up to 90 minutes. Billing is the instance price plus a 9.75 percent fee, and early hands-on tests report an eight-minute publish time and no response streaming.
Why it matters: Teams serving open-weight models can now run GPU inference on serverless infrastructure, but they should weigh the 9.75 percent fee and the missing response streaming against their latency and cost needs.
All sources