pytorch-optimizer
A production-focused PyTorch optimization toolkit with 100+ optimizers.
Access 100+ optimizers, schedulers, and loss functions behind a consistent API.
This library offers over 100 optimizers, 10+ learning rate schedulers, and 10+ loss functions for PyTorch through a consistent API. It is built for deep learning engineers who want to experiment with recent research variants without rewriting optimizer boilerplate code.
What it is
The package groups various optimization algorithms, loss functions, and schedulers like AdaBelief, AdamP, and Lookahead into a unified loader. You can load components directly by name, use helper creation functions, or compile optimizer steps for fused updates.
Who it's for
It suits machine learning practitioners and researchers looking to test multiple non-standard optimizers quickly within PyTorch. It does not suit beginners who are unfamiliar with PyTorch training loops or those who only require standard native optimizers.
Key features
100+ Optimizers
Access a broad collection of recent research optimizers behind a single API.
Consistent Loaders
Load optimizers, learning rate schedulers, and loss functions by name or helper functions.
Optional Compilation
Compile optimizer steps to automatically fuse updates for supported algorithms.
Ecosystem Integrations
Works with optional dependencies like bitsandbytes, q-galore-torch, and torchao.
Up and running in 3 steps
Our honest verdict
What's great
- Eliminates the need to copy-paste or write custom optimizer boilerplate.
- Includes built-in support for Gradient Centralization and Lookahead.
- Supports automatic step compilation for select optimizers.
- Provides filtering helpers to easily discover available components.
Where it falls short
- Requires maintaining compatibility with specific Python and PyTorch versions.
- Optional backend integrations like bitsandbytes require manual separate installation.
- Running heavy experimental optimizers may introduce unexpected training instability.
Pricing & license
Free under its license, with no paid hosted option provided. The only cost is running your own hardware infrastructure for model training.
Open remote AI-training roles
880 open remote AI-training roles, updated November 15, 2026; most pay $20–$175 an hour. Zekai lists them free; the application goes to the employer’s platform.
Alternatives
Use it, or have us set it up
FAQ
What Python and PyTorch versions are required?
You need Python 3.10+ and PyTorch 2.1+ to use the package.
Can I compile optimizer steps for faster execution?
Yes, setting compile=True lets PyTorch automatically fuse optimizer updates for supported algorithms.
