xlite-dev ORG

📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉

A collection of 200+ working CUDA GPU programming examples organized by difficulty, covering high-performance matrix multiplication and Flash Attention, aimed at developers who want to learn GPU kernel development from first principles.

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉

🛠A lite C++ AI toolkit: 100+ models with MNN, ORT and TRT, including Det, Seg, Stable-Diffusion, Face-Fusion, etc.🎉

📚A curated list of Awesome Diffusion Inference Papers with Codes: Sampling, Cache, Quantization, Parallelism, etc.🎉

📚《统计学习方法-李航: 笔记》 200页PDF，公式细节讲解🎉

🤖FFPA: Extends FlashAttention-2 via Split-D for large headdims, 1.5x~3×↑🎉 vs SDPA, up to 430T🎉 on H200.

💎An easy-to-use PyTorch library for face landmarks detection: training, evaluation, inference, and 100+ data augmentations.🎉

⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.

🔥Robust Video Matting C++ inference toolkit with ONNXRuntime、MNN、NCNN and TNN, via lite.ai.toolkit.

YOLO5Face 2021 with MNN/NCNN/TNN/ONNXRuntime

NanoDet、NanoDet-Plus with ONNXRuntime/MNN/TNN/NCNN C++.

A forked version of flux-fast that makes flux-fast even faster with cache-dit, 3.3x speedup on NVIDIA L20.

Super fast accurate face detector ! SCRFD(CVPR 2021) with MNN/TNN/NCNN/ONNXRuntime C++.

⚡️Qwen-Image 4.8x🎉 speedup with Hybrid Acceleration for low VRAM GPUs

FSANet: 1 Mb!! Head Pose Estimation with MNN、TNN and ONNXRuntime C++.

🔥LongCat-Video 1.7x🎉 speedup: cache acceleration and 4/8-bits weight only.

☕️ A vscode extension for netron, support *.pdmodel, *.nb, *.onnx, *.pb, *.h5, *.tflite, *.pth, *.pt, *.mnn, *.param, etc.

YOLOX with NCNN/MNN/TNN/ONNXRuntime C++.

YOLOP with ONNXRuntime C++/MNN/TNN/NCNN

MGMatting with MNN/TNN/ONNXRuntime C++, GPU/CPU, support dynamic shape.

SpargeAttention: A training-free sparse attention that can accelerate any model inference.

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

A PyTorch-native inference engine with cache, parallelism, quantization for Diffusion Transformers.

A forked version of flux-fast that makes flux-fast even faster with cache-dit.

[ICLR2025 Spotlight] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

SSRNet: 190 Kb!! Super fast Age Estimation with MNN/TNN/ONNXRuntime C++.

CUDA Templates and Python DSLs for High-Performance Linear Algebra

A Quirky Assortment of CuTe Kernels

Cross-architecture CUDA kernels for SVDQuant (W4A4 with low-rank correction)

SGLang is a fast serving framework for large language models and vision language models.

🤗 Diffusers: State-of-the-art diffusion models for image, video, and audio generation in PyTorch and FLAX.

Pick Qwen, GLM, MiniMax, Xiaomi MiMo, Moonshot Kimi & Tencent Hunyuan models from the Copilot Chat model picker. Vision, thinking, BYOK.

No description.

Model Compression Toolbox for Large Language Models and Diffusion Models

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

PyTorch native quantization and sparsity for training and inference

A framework for efficient model inference with omni-modality models

Cache-DiT Node for Comfyui

A parallelism VAE avoids OOM for high resolution image generation

Qwen-Image is a powerful image generation foundation model capable of complex text rendering and precise image editing.

No description.

HunyuanImage-3.0: A Powerful Native Multimodal Model for Image Generation

HunyuanImage-2.1: An Efficient Diffusion Model for High-Resolution (2K) Text-to-Image Generation​

Code for Draft Attention

The cli version of lite.ai.toolkit

Pick DeepSeek V4 from the Copilot Chat model picker — and keep everything else Copilot already gives you.

通过集成国内主流原生大模型提供商，为开发者提供更加丰富、更适合本土需求的 AI 编程助手选择。 目前已内置支持 智谱AI、MiniMax、MoonshotAI、DeepSeek、阿里云百炼、快手万擎、火山方舟、腾讯云、Xiaomi MiMo 等原生大模型提供商。 此外，扩展插件已适配支持 OpenAI 与 Anthropic 的 API 接口兼容模型，支持自定义接入任何提供兼容接口的第三方云服务模型。

FlashInfer: Kernel Library for LLM Serving

Fast and memory-efficient exact attention

A high-throughput and memory-efficient inference and serving engine for LLMs

Wan: Open and Advanced Large-Scale Video Generative Models

PTX ISA 9.1 documentation converted to searchable markdown. Includes Claude Code skill for CUDA development.

Distributed Compiler based on Triton for Parallel Systems

NVIDIA cuTile learn

Quantized Attention that achieves speedups of 2.1-3.1x and 2.7-5.1x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.

FlagGems is an operator library for large language models implemented in the Triton Language.

Kandinsky 5.0: A family of diffusion models for Video & Image generation

[NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation

No description.

Wan: Open and Advanced Large-Scale Video Generative Models

Enjoy the magic of Diffusion models!

[NeurIPS 2024] Boosting the performance of consistency models with PCM!

Qwen-Image-Lightning: Speed up Qwen-Image model with distillation

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Template for the Read the Docs tutorial