amd unsloth npu support 100 agentic ai apps amd unsloth npu support 100 agentic ai apps

AMD and Unsloth Brings NPU Support to 100+ Agentic AI Apps

Unsloth Desktop now taps AMD XDNA 2 NPUs for local inference on Ryzen AI PCs.

AMD and Unsloth are bringing local NPU inference to more than 100 agentic AI applications and workflows on supported Ryzen AI PCs.

The AMD XDNA 2 NPUs are the first NPUs supported by Unsloth Desktop, which uses an API that connects compatible apps to supported models running on the NPU.

Unsloth covers both model development and deployment. It offers open-source tools that speed up fine-tuning and cut GPU memory requirements, plus a desktop app for training and inference without writing code.

How It Works

A compatible app sends a model request to the Unsloth Desktop API. Lemonade and FastFlowLM handle model serving and inference on the XDNA 2 NPU, then the response goes back to the app. The NPU takes on the supported model inference, while the app’s other functions keep using the CPU and GPU as needed.

Supported Tools

The Unsloth Desktop API provides an OpenAI-compliant endpoint for compatible apps. Tools include Hermes Agent, OpenClaw, Open WebUI, Cline, LangChain, and GitHub Copilot.

Agents can carry out multistep tasks across connected tools. With local NPU inference, they get a more power-efficient way to process model requests along the way, while the CPU and GPU stay available for orchestration and other apps.

Leave a Reply

Your email address will not be published. Required fields are marked *