Tilde ORG

Aurora optimizer release

An efficient implementation of the NSA (Native Sparse Attention) kernel

Memory optimized Mixture of Experts

Fused KL divergence from hidden states for knowledge distillation

Attention variant with per-channel multiplicative decay

Engine for collecting, uploading, and downloading model activations

Applying SAEs for fine-grained control

Compositional Muon release

Compositional Muon is a Python library that improves how transformer AI models train their attention layers by using each matrix's partner to shape its update, leading to more precise learning steps.

🚀 Efficient implementations for emerging model architectures

verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

Train transformer language models with reinforcement learning.