GenieX — Run LLMs Locally on Qualcomm Snapdragon Devices

Introduction

GenieX is Qualcomm's open-source SDK for deploying language models and vision-language models directly on Snapdragon hardware. It uses the Hexagon NPU for inference acceleration, enabling on-device AI applications that run without cloud connectivity and with lower latency than server-based alternatives.

What GenieX Does

• Runs quantized LLMs (Llama 3, Qwen 3, Gemma, Granite) on Snapdragon NPU, GPU, and CPU
• Supports vision-language models for image understanding tasks on mobile devices
• Provides Python, Go, Kotlin, and Swift bindings for cross-platform integration
• Automatically selects the optimal compute backend based on available Snapdragon hardware
• Delivers token generation at speeds suitable for interactive chat on mobile devices

Architecture Overview

GenieX sits on top of Qualcomm's QNN (Qualcomm Neural Network) runtime, which maps model operations to the best available accelerator. Models are compiled into QNN-optimized formats during a preparation step, then loaded at runtime into the Hexagon DSP or Adreno GPU memory. The SDK handles memory management, tokenization, and sampling, exposing a high-level generate API that streams tokens back to the caller.

Self-Hosting & Configuration

• Install the Python package via pip or use the native SDKs for mobile platforms
• Download pre-optimized model bundles from the GenieX model hub or convert your own
• Select the compute target (npu, gpu, cpu, or auto) at initialization time
• Configure context length, temperature, and top-p sampling parameters
• Use the streaming API for token-by-token output in chat applications

Key Features

• Hexagon NPU acceleration for power-efficient inference on Snapdragon chipsets
• Support for both text-only and multimodal vision-language models
• Multi-language SDK with Python, Kotlin, Swift, and Go bindings
• Model preparation pipeline that optimizes GGUF or Hugging Face models for QNN
• On-device inference with no cloud dependency and full data privacy

Comparison with Similar Tools

• Ollama — desktop-focused CPU/GPU inference; GenieX targets mobile NPUs for power efficiency
• llama.cpp — portable C++ inference; GenieX adds Qualcomm NPU optimization on top
• MLX — Apple Silicon only; GenieX targets the Qualcomm Snapdragon ecosystem
• ExecuTorch — PyTorch mobile runtime; GenieX uses Qualcomm-specific QNN for deeper hardware integration
• MediaPipe LLM — Google's on-device inference; GenieX offers wider model format support on Snapdragon

FAQ

Q: Which Snapdragon chipsets are supported? A: Snapdragon 8 Gen 2 and newer mobile platforms, plus Snapdragon X Elite and X Plus for PCs. Older chipsets fall back to GPU or CPU inference.

Q: What model sizes can run on a phone? A: 1B to 8B parameter models with 4-bit quantization run well. Larger models require Snapdragon PC-class hardware with more memory.

Q: Is it free to use? A: Yes. GenieX is open source and free for commercial and non-commercial use.

Q: Can I fine-tune models through GenieX? A: GenieX is inference-only. Fine-tune models with standard tools, then convert to the supported format for on-device deployment.

Sources

• https://github.com/qualcomm/GenieX