5-day longest streak
BlinkDL A minimalist deep learning library in Javascript using WebGL + asm.js. Runs in your browser. Currently it is a proof-of-concept (inference only). Note: Convolution is buggy when memories overlap.…
BlinkDL
A minimalist deep learning library in Javascript using WebGL + asm.js. Runs in your browser.
Currently it is a proof-of-concept (inference only). Note: Convolution is buggy when memories overlap.
The WebGL backend is powered by weblas: https://github.com/waylonflinn/weblas.
Example
https://withablink.coding.me/goPolicyNet/ : a weiqi (baduk, go) policy network in AlphaGo style:
const N = 19;
const NN = N * N;
const nFeaturePlane = 8;
const nFilter = 128;
const x = new BlinkArray();
x.Init('weblas');
x.nChannel = nFeaturePlane;
x.data = new Float32Array(nFeaturePlane * NN);
for (var i = 0; i < NN; i++)
x.data[5 * NN + i] = 1; // set feature plane for empty board
// pre-act residual network with 6 residual blocks
const bak = new Float32Array(nFilter * NN);
x.Convolution(nFilter, 3);
x.CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak).CopyTo(bak);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.BatchNorm().ReLU().Convolution(nFilter, 3);
x.Add(bak);
x.BatchNorm().ReLU().Convolution(1, 1).Softmax();
Usage
Todo
- [x] Convolution (3x3_pad_1 and 1x1), BatchNorm, ReLU, Softmax
- [ ] Pooling layer
- [ ] FC layer
- [ ] Strided convolution
- [ ] Transposed convolution
- [ ] Webworker and async
- [ ] Faster inference with weblas pipeline, WebGPU, WebAssembly
- [ ] Memory manager
- [ ] Training
-
RWKV-LM ★ PINNED
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Python ★ 15k 4d agoExplain → -
Albatross ★ PINNED
RWKV-7 7.2B fp16 15000+ tps decoding @ single 5090
Cuda ★ 117 11d agoExplain → -
ChatRWKV ★ PINNED
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
Python ★ 9.5k 1mo agoExplain → -
RWKV-CUDA ★ PINNED
The CUDA version of the RWKV language model ( https://github.com/BlinkDL/RWKV-LM )
Cuda ★ 232 6mo agoExplain → -
modded-nanogpt-rwkv ★ PINNED ⑂
RWKV-7: Surpassing GPT
Python ★ 104 1y agoExplain → -
LinearAttentionArena ★ PINNED
Here we will test various linear attention designs.
Python ★ 62 2y agoExplain → -
AI-Writer
AI 写小说，生成玄幻和言情网文等等。中文预训练生成模型。采用我的 RWKV 模型，类似 GPT-2 。AI写作。RWKV for Chinese novel generation.
Python ★ 3.8k 1y agoExplain → -
Hua
Hua is an AI image editor with Stable Diffusion (and more).
★ 359 3y agoExplain → -
nanoRWKV ⑂
RWKV in nanoGPT style
Python ★ 196 2y agoExplain → -
minGPT-tuned ⑂
A *tuned* minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
Jupyter Notebook ★ 118 4y agoExplain → -
BlinkDL.github.io
A collection of State of the Art results in AI / ML / DL / RL / CV / NLP.
★ 90 8y agoExplain → -
BlinkDL
A minimalist deep learning library in Javascript using WebGL + asm.js. Run convolutional neural network in your browser.
JavaScript ★ 87 8y agoExplain → -
fast.c
Prepare for DeekSeek R1 inference: Benchmark CPU, DRAM, SSD, iGPU, GPU, ... with efficient code.
C ★ 73 1y agoExplain → -
RWKV-v2-RNN-Pile
RWKV-v2-RNN trained on the Pile. See https://github.com/BlinkDL/RWKV-LM for details.
Python ★ 67 3y agoExplain → -
SmallInitEmb
LayerNorm(SmallInit(Embedding)) in a Transformer to improve convergence
Python ★ 61 4y agoExplain → -
BookCNN
《深度卷积网络：原理与实践》现已在淘宝天猫京东当当发售. 这里是其中的代码下载.
Jupyter Notebook ★ 57 7y agoExplain → -
Agen
Agen is a minimalist language for agent loops and state machines.
Python ★ 49 3mo agoExplain → -
WorldModel
Let us make Psychohistory (as in Asimov) a reality, and accessible to everyone. Useful for LLM grounding and games / fiction / business / finance / governance, and can align agents with human too.
★ 40 3y agoExplain → -
LM-Trick-Questions
Here we collect trick questions and failed tasks for open source LLMs to improve them.
★ 32 3y agoExplain → -
Basis
The Basis Programming Language
Python ★ 26 7y agoExplain → -
BlinkToDo
A minimalist ToDo.txt page. 如果你的ToDo有一百项以上，试试这个基于txt的极简事项管理工具。
JavaScript ★ 24 8y agoExplain → -
RWKV.com
No description.
JavaScript ★ 14 21h agoExplain → -
zoology ⑂
Understand and test language model architectures on synthetic tasks.
Python ★ 11 1y agoExplain → -
AntiAging
List of Anti-aging Research
★ 11 3y agoExplain → -
Nala
The Nala markup, to turn a "Natural Language" sentence into a code-like statement. Nala 标注，将自然语言变为编程语言。
★ 9 4y agoExplain → -
PathTracingJS
Path tracing demo with JS in your web browser. 用浏览器JS做路径跟踪渲染。
JavaScript ★ 7 8y agoExplain → -
Model_Leaderboard
Leaderboard of AI models.
HTML ★ 4 2y agoExplain → -
BlinkColorTheme
A colorful theme for HTML+JS+CSS.
CSS ★ 4 7y agoExplain → -
MathBook
一个较为系统的数学笔记（graduate level）
★ 2 7y agoExplain → -
BasisLang.com
BasisLang.com
HTML ★ 1 7y agoExplain → -
FastChat ⑂
An open platform for training, serving, and evaluating large languages. Release repo for Vicuna and FastChat-T5.
Python ★ 0 3y agoExplain → -
OpenAudioTest
No description.
Python ★ 0 3y agoExplain → -
libai ⑂
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
★ 0 3y agoExplain → -
gpt-neox ⑂
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
Python ★ 0 4y agoExplain → -
ProjectIvory
Project Ivory is a simple forum written a few years ago.
★ 0 6y agoExplain →
No repos match these filters.