MambaDecoderBlock
Python
★ 5
updated 6mo ago
MambaDecoderBlock is a novel decoder architecture that replaces traditional self-attention mechanisms with Mamba state space models, augmented by Mixture of Experts (MoE) layers.
No plain-English explanation yet — one is being written right now. Check back in a minute.