Serverless GPUs with NVIDIA NIM support is now generally
available in Azure Container Apps. Serverless GPUs enable you to seamlessly run
your AI workloads on-demand with automatic scaling, optimized cold start,
per-second billing with scale down to zero when not in use, and reduced
operational overhead to support easy real-time custom model inferencing and other
GPU related tasks.
Serverless
GPUs accelerate the speed of AI development teams by allowing customers to
focus on core AI code and less on managing infrastructure when using GPUs. This
provides an excellent middle layer option between Azure AI model catalog's serverless APIs and hosting
custom models on managed compute. This feature will allow customers to build
their own serverless API endpoints for inferencing custom AI models. Customers
can also provision on-demand GPU-powered Jupyter Notebooks or run other ML workloads
with larger computation needs that are ephemeral in nature. It provides full
data governance as customer’s data never leaves the boundaries of the container
while still providing a managed, serverless platform from which to build your
applications.
This GA release of Serverless GPUs also adds support for NVIDIA NIMs, NVIDIA’s purpose-built containers for a wide range of
AI scenarios. These out-of-the-box solutions support scenarios from large
language models to visual language models, and models for speech, image
generation, and much more.
Learn more: