Jia-Wei Jiang

A toolkit to run Ray applications on Kubernetes

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

SkyRL: A Modular Full-stack RL Library for LLMs

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 16+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.

My solutions and supplemental resources for CS224N in the spring of 2024.

Predict the electricity production and consumption of prosumers in Estonia.

Another hydra-powered torch template for quick data science project setup for my personal use.

玉山人工智慧公開挑戰賽2021冬季賽 - 信用卡消費類別推薦8th solution.

Google - Fast or Slow? Predict AI Model Runtime - 33rd solution.

Setup a single-host slurm playground for local development and testing.

In this repo, I try to record the practicing journey of CS224W, including the implementations of Colab and homeworks. Some supplementary resources are also provided in each practice.

Simple news scraper

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

A shared flyte demo registry facilitating collaboration with my friends

The preliminary round of Taiwan Intelligent Manufacturing Big Data Analysis Competition (IMBD2022).

The final round of projectB of Taiwan Intelligent Manufacturing Big Data Analysis Competition (IMBD2022).

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

SGLang is a fast serving framework for large language models and vision language models.

TokenSpeed is a speed-of-light LLM inference engine.

OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)

KubeRay utilities for dev

NVIDIA DRA Driver for GPUs

Start managing my dotfiles, nothing special

Production-Grade Container Scheduling and Management

Tensors and Dynamic neural networks in Python with strong GPU acceleration

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

No description.

No description.

Mirror of Apache Mahout

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

NVIDIA GPU metrics exporter for Prometheus leveraging DCGM

Implementing a ChatGPT-like LLM in PyTorch from scratch, step by step

Nano vLLM

Supercharge Your LLM with the Fastest KV Cache Layer

CUDA/Metal accelerated language model inference

Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O

NVIDIA device plugin for Kubernetes

Example DRA driver that developers can fork and modify to get them started writing their own.

My learning notes/codes for ML SYS.

No description.

A Datacenter Scale Distributed Inference Serving Framework

Deploy an LLM model serving stack in an air-gapped environment.

Extensible Python SDK for developing Flyte tasks and workflows. Simple to get started and learn and highly extensible.

Informally speaking.

Play around with Flyte agent to better understand how it works.

You like pytorch? You like micrograd? You love tinygrad! ❤️

No description.

Slurm: A Highly Scalable Workload Manager

CookieCutter template for getting started with Flyte python projects

Walk through AI roadmap by project-based learning.

Just my exploration of RAG, nothing special.

Just hands-on data structures and algorithms.

A simulated dbms for stock brokerage.

Association rule implementation for course Data Mining