Trajectory's Multi-LoRA Stack Boosts Experiment Throughput

Trajectory, in collaboration with UC Berkeley Sky Lab and Anyscale, has launched a concurrent Multi-LoRA training stack that reportedly boosts experiment throughput by 2.81 times, available in the open-source NovaSky-AI/SkyRL repository.

This development could significantly enhance the efficiency of continually learning language models by integrating live production feedback, paving the way for more adaptive AI systems.

AI researchers and organizations seeking to implement more efficient and adaptive language model training could find this concurrent multi-LoRA stack particularly impactful.

The increased throughput comes at the cost of higher per-step latency, and the current system has only been tested on mid-sized models, not frontier-scale parameters.