AngelSlim
Model compression toolkit for efficient LLM, VLM, and diffusion workflows.
Comprehensive toolkit for LLM, VLM, and diffusion model compression, quantization, and distillation.
AngelSlim is an open-source model compression toolkit engineered by Tencent for large models like Hunyuan, Qwen, and DeepSeek. It suits audio, AI, and model infrastructure engineers needing to reduce model footprints through advanced quantization and speculative decoding.
What it is
The toolkit provides a unified framework for quantization algorithms like INT4, FP8, and NVFP4, alongside speculative decoding frameworks like Eagle3, DFlare, and DFly. It also integrates sparse attention mechanisms and scale-only quantization-aware distillation to optimize large model deployment.
Who it's for
It suits machine learning engineers and researchers deploying massive language or multimodal models on constrained hardware. It does not suit audio practitioners looking for high-level digital audio workstation plugins or non-technical creators.
Key features
Diverse Quantization
Supports FP8, INT8, INT4, NVFP4, and custom ternary quantization algorithms.
Speculative Decoding
Implements Eagle3, DFlare, DFly, and MTP frameworks for end-to-end inference speedups.
Sparse Attention
Includes Stem and MInference block-sparse attention algorithms to accelerate long-context prefill.
Quantized Distillation
Provides quantization-aware distillation support on Megatron-Core for distributed training.
Up and running in 3 steps
Our honest verdict
What's great
- Integrates mainstream compression algorithms into a single unified framework
- Supports extreme quantization methods down to 1.25-bit and 2-bit model versions
- Provides specialized speculative decoding strategies like block-diffusion layer-wise fusion
- Enables large model quantization like Qwen3-235B and DeepSeek-R1 on a single GPU
Where it falls short
- Requires advanced technical skill in deep learning frameworks and distributed training
- Does not state a specific open-source license in its repository metadata
- Running massive compressed models still requires significant GPU or system memory resources
Pricing & license
The software is provided via its GitHub repository, and the only cost is running it yourself on your own hardware.
Open remote AI-training roles
870 open remote AI-training roles, updated November 14, 2026; most pay $20–$170 an hour. Zekai lists them free; the application goes to the employer’s platform.
