binmakeswell

Colossal-AI: A Unified Deep Learning System for Big Model Era

Building your own video generation model like OpenAI's Sora

Accuracy 77%. Large batch deep learning optimizer LARS for ImageNet with PyTorch and ResNet, using Horovod for distribution. Optional accumulated gradient and NVIDIA DALI dataloader.

ColossalChat is the project to implement LLM with RLHF, powered by the Colossal-AI project.

No description.

Examples of training models with hybrid parallelism using ColossalAI

A family of open-sourced Mixture-of-Experts (MoE) Large Language Models

A Python library transfers PyTorch tensors between CPU and NVMe

Repo for external large-scale work

A curated list of references for MLOps

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

A curated list of research in machine learning systems (MLSys). Paper notes are also provided.

Deep Learning and deep reinforcement learning research papers and some codes

No description.

Scalable PaLM implementation of PyTorch

Optimizing Protein Structure Prediction Model Training and Inference on GPU Clusters

Sky Computing: Accelerating Geo-distributed Computing in Federated Learning

Performance benchmarking with ColossalAI

No description.

PyTorch implementation of LAMB for ImageNet/ResNet-50 training