[ICLR 2025] Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization

Awesome-RL-Reasoning

双方向型ビジュアル量子教材

A simple, performant and scalable Jax LLM!

YaRN: Efficient Context Window Extension of Large Language Models

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

Examples and recipes for Llama 2 model

Utilities for Training Very Large Models

No description.

NanoGPT (124M) in 2 minutes

Domain-specific language designed to streamline the development of high-performance GPU/CPU/Accelerators kernels

AI agents running research on single-GPU nanochat training automatically

Train the smallest LM you can that fits in 16MB. Best model wins!

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Config files for my GitHub profile.

Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

The best ChatGPT that $100 can buy.

Tile primitives for speedy kernels

No description.

🚧 Accepting Task Submissions 🚧

verl: Volcano Engine Reinforcement Learning for LLMs

SGLang is a fast serving framework for large language models and vision language models.

A high-throughput and memory-efficient inference and serving engine for LLMs

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

DeepEP: an efficient expert-parallel communication library

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

Scalable data pre processing and curation toolkit for LLMs

A PyTorch native platform for training generative AI models

A framework for few-shot evaluation of language models.

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

PyTorch building blocks for the OLMo ecosystem

No description.

NCCL Tests

No description.

No description.

Ongoing research training transformer models at scale

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

No description.

Robust recipes to align language models with human and AI preferences

Project of llm evaluation to Japanese tasks

Inference code for LLaMA models

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Empowers LLMs with the ability to see and draw.

No description.

No description.

Multilingual Large Language Models Evaluation Benchmark

No description.

Evaluation of Finnish generative models

A framework for the evaluation of autoregressive code generation language models.

No description.

No description.

distributed trainer for LLMs

No description.

LLM training code for MosaicML foundation models

The RedPajama-Data repository contains code for preparing large datasets for training large language models.