Wentao Guo

CD-GraB is a distributed gradient balancing framework that aims to find distributed data permutation with provably better convergence guarantees than Distributed Random Reshuffling (D-RR). https://arxiv.org/pdf/2302.00845.pdf.

This is the starter code for the Assignment 1 in COS 568. Due Feb 26 (Thursday) midnight. Submission: https://www.gradescope.com/courses/1255765

No description.

torch-optimizer -- collection of optimizers for Pytorch

No description.

No description.

No description.

No description.

Training materials associated with NVIDIA's CUDA Training Series (www.olcf.ornl.gov/cuda-training-series/)

No description.

Material for gpu-mode lectures

Experiment of using Tangent to autodiff triton

Collection of papers on state-space models

No description.

Tips for Writing a Research Paper using LaTeX

Building blocks for foundation models.

No description.

No description.

No description.

MeZO: Fine-Tuning Language Models with Just Forward Passes. https://arxiv.org/abs/2305.17333

QLoRA: Efficient Finetuning of Quantized LLMs

PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks

Inequalities in Homophilic Directed ScaleFree Networks

Source code for the GraB paper.

Decentralized SGD and Consensus with Communication Compression: https://arxiv.org/abs/1907.09356

Train 🤗transformers with DeepSpeed: ZeRO-2, ZeRO-3

Image Deblurring using Generative Adversarial Networks

Shōgun

No description.

PyTorch library to facilitate development and standardized evaluation of neural network pruning methods.

Transformer Models in NLP