Future Machine Learning and System Lab ORG

CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization

Research code for OSCAR, a 2-bit KV cache compression method for LLM inference that uses offline spectral covariance-aware rotation and plugs into SGLang.

Repeat Until Done

No description.