10-day current streak·34-day longest streak
-
vllm ★ PINNED ⑂
A high-throughput and memory-efficient inference and serving engine for LLMs
Python ★ 0 50m agoExplain → -
llm-d ★ PINNED ⑂
llm-d is a Kubernetes-native high-performance distributed LLM inference framework
★ 0 1y agoExplain → -
deepsparse ★ PINNED ⑂
Neural network inference engine that delivers GPU-class performance for sparsified models on CPUs
★ 0 4y agoExplain → -
advos ★ PINNED
RISC-V OS in Rust with hardware support for SiFive's HiFive1 board
Rust ★ 1 2y agoExplain → -
torch_bitmask ★ PINNED
Implementations of bitmask compression for weight sparsity in PyTorch
Python ★ 5 2y agoExplain → -
learned_indexes
Experiments on ideas proposed in Tim Kraska's "The Case for Learned Index Structures"
Python ★ 10 2y agoExplain → -
vllm-wheels
No description.
Python ★ 6 4h agoExplain → -
bfc
No description.
Brainfuck ★ 3 1y agoExplain → -
MPT-Medical-Chatbot ⑂
This is a medical bot built using MPT and Sentence Transformers. The bot is powered by DeepSparse, Langchain, and Chainlit. The bot runs on a decent CPU machine with a minimum of 16GB of RAM.
Python ★ 3 2y agoExplain → -
benchmark-arena
No description.
HTML ★ 2 1y agoExplain → -
torch-fp8
No description.
Python ★ 2 2y agoExplain → -
hf_model_stats
No description.
Python ★ 1 9h agoExplain → -
meTile ⑂
python-based eDSL for efficient Metal Shading Language code generation
★ 1 3mo agoExplain → -
cnpy ⑂
Single header-only library to read and write Numpy files in C/C++
C++ ★ 1 5y agoExplain → -
TTS ⑂
🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production
★ 1 2y agoExplain → -
ultralytics-deepsparse ⑂
YOLOv8 🚀 in PyTorch > ONNX > CoreML > TFLite
★ 1 3y agoExplain → -
YOLOv5-and-DeepSparse-in-CPP
No description.
C++ ★ 1 3y agoExplain → -
speculators ⑂
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
Python ★ 0 4d agoExplain → -
vllm-dashboard ⑂
No description.
★ 0 7d agoExplain → -
Mooncake ⑂
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
C++ ★ 0 16h agoExplain → -
ci-infra ⑂
This repo hosts code for vLLM CI & Performance Benchmark infrastructure.
★ 0 14d agoExplain → -
pingboard
No description.
JavaScript ★ 0 26d agoExplain → -
vllm-gguf-plugin ⑂
vLLM Quantization plugin for GGUF
★ 0 1mo agoExplain → -
humming ⑂
No description.
Python ★ 0 1mo agoExplain → -
InstantTensor ⑂
An ultra-fast, distributed Safetensors loader
★ 0 2mo agoExplain → -
mgoin.github.io
No description.
HTML ★ 0 3mo agoExplain → -
shepherd
No description.
JavaScript ★ 0 6mo agoExplain → -
binfer ⑂
inference in bun
★ 0 6mo agoExplain → -
dvc.org ⑂
🔗 DVC website and documentation
★ 0 4y agoExplain → -
prime-rl ⑂
Async RL Training at Scale
★ 0 8mo agoExplain → -
SpecForge ⑂
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.
★ 0 9mo agoExplain → -
loc-blocks-vllm
No description.
JavaScript ★ 0 9mo agoExplain → -
vllm-ci-status
No description.
TypeScript ★ 0 10mo agoExplain → -
flashinfer ⑂
FlashInfer: Kernel Library for LLM Serving
★ 0 9mo agoExplain → -
llmgoin
No description.
Python ★ 0 1y agoExplain → -
gh-activity
No description.
Python ★ 0 1y agoExplain → -
uv ⑂
An extremely fast Python package and project manager, written in Rust.
★ 0 1y agoExplain → -
pdf-vlm
No description.
Python ★ 0 1y agoExplain → -
GPTAQ
No description.
Python ★ 0 2y agoExplain → -
transformers ⑂
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.
Python ★ 0 1y agoExplain → -
outlines ⑂
Structured Text Generation
★ 0 1y agoExplain → -
outlines-core ⑂
Structured generation in Rust
★ 0 1y agoExplain → -
pytorch ⑂
Tensors and Dynamic neural networks in Python with strong GPU acceleration
★ 0 1y agoExplain → -
mistral-evals ⑂
No description.
★ 0 1y agoExplain → -
flash-attention ⑂
Fast and memory-efficient exact attention
★ 0 1y agoExplain → -
huggingface.js ⑂
Utilities to use the Hugging Face Hub API
★ 0 1y agoExplain → -
BitBLAS ⑂
BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.
★ 0 2y agoExplain → -
dev_env
Holds dotfiles, scripts, and notes to quickly construct my preferred development environment.
Shell ★ 0 3mo agoExplain → -
webgl_signed_distance_fields
No description.
JavaScript ★ 0 2y agoExplain → -
AutoGPTQ ⑂
An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
★ 0 2y agoExplain → -
llmperf ⑂
LLMPerf is a library for validating and benchmarking LLMs
★ 0 2y agoExplain → -
clip-retrieval ⑂
Easily compute clip embeddings and build a clip retrieval system with them
★ 0 2y agoExplain → -
sparsegpt ⑂
Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".
Python ★ 0 2y agoExplain → -
llama-cpp-python ⑂
Python bindings for llama.cpp
★ 0 2y agoExplain → -
lm-evaluation-harness ⑂
A framework for few-shot evaluation of autoregressive language models.
Python ★ 0 11mo agoExplain → -
mteb ⑂
MTEB: Massive Text Embedding Benchmark
★ 0 2y agoExplain → -
amsterdam-demo
No description.
Python ★ 0 2y agoExplain → -
tinystories-sparsify
No description.
★ 0 2y agoExplain → -
memory-bandwidth-benchmark
No description.
Shell ★ 0 2y agoExplain → -
benchmark-sparsezoo
No description.
Python ★ 0 2y agoExplain → -
langchain ⑂
⚡ Building applications with LLMs through composability ⚡
Python ★ 0 2y agoExplain → -
optimum ⑂
🏎️ Accelerate training and inference of 🤗 Transformers with easy to use hardware optimization tools
Python ★ 0 3y agoExplain → -
BiSeNet ⑂
Add bisenetv2. My implementation of BiSeNet
★ 0 3y agoExplain → -
whisper ⑂
No description.
Jupyter Notebook ★ 0 3y agoExplain → -
onnx ⑂
Open standard for machine learning interoperability
★ 0 3y agoExplain → -
inference_policies ⑂
Issues related to MLPerf™ Inference policies, including rules and suggested changes
★ 0 3y agoExplain → -
inference ⑂
Reference implementations of MLPerf™ inference benchmarks
Python ★ 0 2y agoExplain → -
Model-Compression-Research-Package ⑂
A library for researching neural networks compression and acceleration methods.
★ 0 3y agoExplain → -
onnx_transformers ⑂
Accelerated NLP pipelines for fast inference on CPU. Built with Transformers and ONNX runtime.
★ 0 5y agoExplain → -
generative
No description.
JavaScript ★ 0 5y agoExplain → -
Satelight-Image-Semantic-Segmentation-Unet-Tensorflow-keras ⑂
Collection of different Unet Variant suchas VggUnet, ResUnet, DenseUnet, Unet. AttUnet, MobileNetUnet, NestedUNet, R2AttUNet, R2UNet, SEUnet, scSEUnet, Unet_Xception_ResNetBlock
★ 0 5y agoExplain → -
onnxruntime ⑂
ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
C++ ★ 0 5y agoExplain → -
hyphae ⑂
No description.
★ 0 12y agoExplain → -
rol
Game of Life implemented in Rust
Rust ★ 0 2y agoExplain → -
SSD ⑂
High quality, fast, modular reference implementation of SSD in PyTorch
Python ★ 0 6y agoExplain → -
tiniest-analytics ⑂
VERY simple cross-platform C++ analytics for games (using Google Analytics)
C++ ★ 0 6y agoExplain → -
community ⑂
Database for all community Rack plugins
Python ★ 0 8y agoExplain → -
webserver
No description.
HTML ★ 0 8y agoExplain → -
students ⑂
Background information/Interests
★ 0 8y agoExplain → -
dolphin ⑂
Dolphin is a GameCube / Wii emulator, allowing you to play games for these two platforms on PC with improvements.
C++ ★ 0 11y agoExplain → -
3D_plane_intersection
No description.
★ 0 11y agoExplain → -
VM_C
Virtual CPU written in C++
C++ ★ 0 11y agoExplain → -
rpg_txt
A text-based fully-fledged RPG meant to be played in a terminal.
C++ ★ 0 12y agoExplain → -
grid-hack
The game I am working on for AP Computer Science
Java ★ 0 13y agoExplain →
No repos match these filters.