Cactus
Hybrid edge-cloud AI engine providing local LLM, VLM, and transcription for mobile and wearable devices.
Run quantized LLMs, vision models, and speech transcription natively on mobile and wearables.
Cactus is a high-performance inference engine tailored for mobile devices and wearables, supporting text, speech, and vision models. It suits mobile developers and engineers looking to deploy local AI pipelines with optional cloud handoff.
What it is
It supplies a zero-copy computation graph, CPU/GPU kernels across multiple architectures, and custom rotation-based quantization techniques ranging from 4-bit down to 1-bit. The engine exposes OpenAI-compatible APIs and handles automatic RAG alongside on-device tool calling via its Needle model.
Who it's for
It suits developers embedding on-device intelligence into iOS, Android, macOS, or embedded applications. It does not suit teams requiring purely cloud-hosted inference without local hardware constraints.
Key features
Rotation-Based Quantization
Custom quantization techniques scale models from 4-bit down to 1-bit weight precision.
Zero-Copy Computation Graph
Optimized Cactus Graph handles tensor operations, matrix multiplication, and attention locally.
OpenAI-Compatible APIs
Cactus Engine provides standard chat completion, streaming, vision, and tool calling interfaces.
Hybrid Cloud Handoff
Automatically routes difficult queries to the cloud based on local model confidence thresholds.
Up and running in 3 steps
Our honest verdict
What's great
- Native execution across Apple, Samsung, and Pixel hardware via optimized SIMD kernels
- Supports multi-modal workflows including vision models and audio transcription
- Provides bindings for Swift, Kotlin, Flutter, React Native, Rust, and Python
- Includes auto-RAG capabilities directly inside the model initialization pipeline
Where it falls short
- Requires compiling C++ dependencies and setting up local build toolchains
- Model conversion from HuggingFace can be experimental depending on architecture
- Running heavy multi-modal models locally consumes significant device RAM
Pricing & license
The software is open source with no stated license fee in the repository metadata. The only cost is running it yourself on local hardware or paying for cloud handoff API usage.
Open remote AI-training roles
870 open remote AI-training roles, updated November 14, 2026; most pay $20–$170 an hour. Zekai lists them free; the application goes to the employer’s platform.
Alternatives
Use it, or have us set it up
FAQ
What model architectures are supported?
Liquid, Gemma, Whisper, Parakeet, and Qwen model families are explicitly tested, alongside any HuggingFace model via conversion.
Does Cactus support cloud fallback?
Yes, it includes a hybrid mode that routes hard queries to the cloud when local model confidence drops below a set threshold.
