By
default, the AI toolchain operator add-on now deploys model inferencing
workloads using the high-throughput vLLM serving engine. vLLM
offers a significant speed-up to process incoming requests and flexibility to
use OpenAI-compatible APIs, DeepSeek R1 models, and a range of pre-trained
HuggingFace models. Alternatively, you can switch from vLLM to HuggingFace
Transformers for KAITO inference deployments at any time.
Learn
more.