# GenieX — Run LLMs Locally on Qualcomm Snapdragon Devices

> An SDK by Qualcomm for running large language models and vision-language models on-device using the Hexagon NPU, Adreno GPU, and Kryo CPU found in Snapdragon-powered phones, PCs, and IoT devices.

## Install

Save in your project root:

# GenieX — Run LLMs Locally on Qualcomm Snapdragon Devices

## Quick Use
```python
from geniex import GenieX
gx = GenieX(model='llama3-8b', device='npu')
response = gx.generate('Explain quantum computing in simple terms.')
print(response)
```

## Introduction
GenieX is Qualcomm's open-source SDK for deploying language models and vision-language models directly on Snapdragon hardware. It uses the Hexagon NPU for inference acceleration, enabling on-device AI applications that run without cloud connectivity and with lower latency than server-based alternatives.

## What GenieX Does
- Runs quantized LLMs (Llama 3, Qwen 3, Gemma, Granite) on Snapdragon NPU, GPU, and CPU
- Supports vision-language models for image understanding tasks on mobile devices
- Provides Python, Go, Kotlin, and Swift bindings for cross-platform integration
- Automatically selects the optimal compute backend based on available Snapdragon hardware
- Delivers token generation at speeds suitable for interactive chat on mobile devices

## Architecture Overview
GenieX sits on top of Qualcomm's QNN (Qualcomm Neural Network) runtime, which maps model operations to the best available accelerator. Models are compiled into QNN-optimized formats during a preparation step, then loaded at runtime into the Hexagon DSP or Adreno GPU memory. The SDK handles memory management, tokenization, and sampling, exposing a high-level generate API that streams tokens back to the caller.

## Self-Hosting & Configuration
- Install the Python package via pip or use the native SDKs for mobile platforms
- Download pre-optimized model bundles from the GenieX model hub or convert your own
- Select the compute target (npu, gpu, cpu, or auto) at initialization time
- Configure context length, temperature, and top-p sampling parameters
- Use the streaming API for token-by-token output in chat applications

## Key Features
- Hexagon NPU acceleration for power-efficient inference on Snapdragon chipsets
- Support for both text-only and multimodal vision-language models
- Multi-language SDK with Python, Kotlin, Swift, and Go bindings
- Model preparation pipeline that optimizes GGUF or Hugging Face models for QNN
- On-device inference with no cloud dependency and full data privacy

## Comparison with Similar Tools
- **Ollama** — desktop-focused CPU/GPU inference; GenieX targets mobile NPUs for power efficiency
- **llama.cpp** — portable C++ inference; GenieX adds Qualcomm NPU optimization on top
- **MLX** — Apple Silicon only; GenieX targets the Qualcomm Snapdragon ecosystem
- **ExecuTorch** — PyTorch mobile runtime; GenieX uses Qualcomm-specific QNN for deeper hardware integration
- **MediaPipe LLM** — Google's on-device inference; GenieX offers wider model format support on Snapdragon

## FAQ
**Q: Which Snapdragon chipsets are supported?**
A: Snapdragon 8 Gen 2 and newer mobile platforms, plus Snapdragon X Elite and X Plus for PCs. Older chipsets fall back to GPU or CPU inference.

**Q: What model sizes can run on a phone?**
A: 1B to 8B parameter models with 4-bit quantization run well. Larger models require Snapdragon PC-class hardware with more memory.

**Q: Is it free to use?**
A: Yes. GenieX is open source and free for commercial and non-commercial use.

**Q: Can I fine-tune models through GenieX?**
A: GenieX is inference-only. Fine-tune models with standard tools, then convert to the supported format for on-device deployment.

## Sources
- https://github.com/qualcomm/GenieX

---
Source: https://tokrepo.com/en/workflows/asset-f43a17ca
Author: AI Open Source