Hikari_localLLM

NVFP4 KV cache for vLLM on SM120 (RTX PRO 6000) via FlashInfer FA2 explicit-SF-stride patch — ~1.5x fp8 pool at ~95-104% speed

A patch for vLLM that enables NVFP4 KV cache compression on Blackwell-architecture GPUs like the RTX PRO 6000, fitting roughly 78 percent more context into the same GPU memory.

High-concurrency goal-driven swarm harness for Step-3.7-Flash: conversational front door, stigmergic board, AIMD admission, recall, skills, web UI.

GLM-5.2 UD-Q2 decode speedup: CPU∥GPU MoE expert-split (harness, specs, de-risk)

FP8 vs NVFP4 (4-bit) KV cache on Terminal-Bench 2.0 — no measurable accuracy loss, 1.78x more KV capacity. Full results + verify.py.

SGLang NVFP4 (fp4_e2m1) KV cache for Blackwell SM120 (RTX PRO 6000): FlashInfer FA2 kernel patches + native FP4 pool + hybrid-SWA wiring + per-layer global-scale auto-calibration. 1.778x KV capacity, ~4% decode cost. Validated end-to-end on Step-3.7-Flash 198B (cuda-graph, TP=2). Small models hit the 4-bit precision floor (use fp8 KV).

Gemma-4-31B representation engineering: dataset-free personality control + process-level sycophancy ablation

llama.cpp fork: GLM-5.2 CPU∥GPU MoE expert-split decode speedup

DeepSeek-V4-Flash-180B (REAP K160) × Terminal-Bench 2.0 — reproducible evaluation deliverable