NielsRogge

This repository contains demos I made with the Transformers library by HuggingFace.

A collection of hands-on Jupyter notebooks showing how to apply HuggingFace Transformers models for text, image classification, object detection, and document understanding, all runnable free in Google Colab.

This repository contains an overview of important follow-up works based on the original Vision Transformer (ViT) by Google.

A repository containing general tutorials I'd like to share with the world.

🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

Repository containing awesome resources regarding Hugging Face tooling.

Transforming textual descriptions into process models using deep learning

A tiny package supporting distributed computation of COCO metrics for PyTorch models.

Short README about myself.

No description.

UniLM - Unified Language Model Pre-training / Pre-training for NLP and Beyond

A package containing utils for the PyTorch version of the Tapas algorithm.

a state-of-the-art-level open visual language model

Some notes I took when learning about diffusion models.

A repository showcasing the entire workflow of putting computer vision models in production.

Notebooks using the Hugging Face libraries 🤗

Large Language-and-Vision Assistant built towards multimodal GPT-4 level capabilities.

Reference implementation of Mistral AI 7B v0.1 model.

The official repository for MedSAM: Segment Anything in Medical Images.

Official codebase used to develop Vision Transformer, MLP-Mixer, LiT and more.

RF-DETR is a real-time object detection model architecture developed by Roboflow, released under the Apache 2.0 license.

ImageBind One Embedding Space to Bind Them All

[ICASSP 2024] 🍵 Matcha-TTS: A fast TTS architecture with conditional flow matching

This repository is meant for parsing evaluation results from Hugging Face models, and opening pull requests on the hub to display them at leaderboards.

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

No description.

This repository contains the assignments I made during the 2019 version of the Deep Learning for Computer Vision course taught at the University of Michigan.

Official repository for our work on micro-budget training of large-scale diffusion models.

YOLOv10: Real-Time End-to-End Object Detection

Model training and evaluation code for our dataset PubTables-1M, developed to support the task of table extraction from unstructured documents.

Implementation of Nougat Neural Optical Understanding for Academic Documents

PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"

🤗 ml-intern: an open-source ML engineer that reads papers, trains models, and ships ML models

Simple, safe way to store and distribute tensors

A repository with various baselines for the agentic-document-ai project.

Your own personal AI assistant. Any OS. Any Platform. The lobster way. 🦞

veRL: Volcano Engine Reinforcement Learning for LLM

Utilities to use the Hugging Face Hub API

Tips for releasing research code in Machine Learning (with official NeurIPS 2020 recommendations)

Tools for extracting tables and results from Machine Learning papers

No description.

VisualQuality-R1 is the first open-sourced NR-IQA model can accurately describe and rate the image quality.

No description.

My solutions to the practical assignments of CS224n (Natural Language Processing with Deep Learning) [Stanford University-Winter 2019]

No description.

OpenMMLab Computer Vision Foundation

DarkIR: Robust Low-Light Image Restoration [Official PyTorch Implementation]

Official implementation of the paper “MagicDriveDiT: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control”

Official implementation of "GST: Precise 3D Human Body from a Single Image with Gaussian Splatting Transformers"

[TPAMI'23] Unifying Flow, Stereo and Depth Estimation

DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

Train transformer language models with reinforcement learning.

From anything to mesh like human artists. Official impl. of "MeshAnything V2: Artist-Created Mesh Generation With Adjacent Mesh Tokenization"

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction"

Place where folks can contribute to 🤗 community events

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch

Official PyTorch implementation of LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding (ACL 2022)

Detection Transformers with Assignment

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

Dense Prediction Transformers

An implementation of DCGAN, leveraging the HuggingFace ecosystem for getting data and pushing to the hub.

You Only Look at One Sequence (NeurIPS 2021)

Pix2Seq - A general framework for turning RGB pixels into semantically meaningful sequences

🤗 Fast, efficient, open-access datasets and evaluation metrics in PyTorch, TensorFlow, NumPy and Pandas

OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

A repository for research on medium sized language models.

Official implementation of "Separate Anything You Describe"

Scenic: A Jax Library for Computer Vision Research and Beyond

Image processing in Python

PyTorch code and models for the DINOv2 self-supervised learning method.

Faster Arbitrarily-Shaped Text Detector with Minimalist Kernel Representation

PyTorch image models, scripts, pretrained weights -- (SE)ResNet/ResNeXT, DPN, EfficientNet, MixNet, MobileNet-V3/V2, MNASNet, Single-Path NAS, FBNet, and more

Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

An MCP server to search for flights.

A benchmark for LLMs on complicated tasks in the terminal

This repository is an official implementation of the paper "LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection".

EfficientSAM3 compresses SAM3 into lightweight, edge-friendly models via progressive knowledge distillation for fast promptable concept segmentation and tracking.

[DEIMv2] Real Time Object Detection Meets DINOv3

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

No description.

:alarm_clock: AI conference deadline countdowns

A lightweight, powerful framework for multi-agent workflows

ICLR2024 Spotlight: curation/training code, metadata, distribution and pre-trained models for MetaCLIP; CVPR 2024: MoDE: CLIP Data Experts via Clustering

Official code for the paper: Depth Anything At Any Condition

[CVPR 2025 Best Paper Award Candidate] VGGT: Visual Geometry Grounded Transformer

[ECCV 2022] ByteTrack: Multi-Object Tracking by Associating Every Detection Box

Code to repro PE and PLM

A blazing fast inference solution for text embeddings models

A jump start solution using GKE or Cloud Run with Cloud SQL and VertexAI

Implementation for SimDINO/SimDINOv2

Official code of DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning

OmniMamba: Efficient and Unified Multimodal Understanding and Generation via State Space Models

YOLOE: Real-Time Seeing Anything

YOLOv12: Attention-Centric Real-Time Object Detectors

This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API.

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

Repository for Show-o, One Single Transformer to Unify Multimodal Understanding and Generation.

Large Language Model Text Generation Inference

This codebase stores the complete artifacts and describes how to reproduce or extend the results from the paper "Style over Substance: Failure modes of LLM judges in alignment benchmarking", including the MisMo-Bench meta-benchmark.

🤗 LeRobot: End-to-end Learning for Real-World Robotics in Pytorch

Music repair method to convert lossy MP3 compressed music to lossless music.

open-source multimodal large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities.

Official pytorch implementation for "LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models"

No description.

ZIM: Zero-Shot Image Matting for Anything

The official repo for the paper "VeCLIP: Improving CLIP Training via Visual-enriched Captions"

Official implementation of 'CLIP-DINOiser: Teaching CLIP a few DINO tricks' paper.

[ECCV 2024] official code for "Long-CLIP: Unlocking the Long-Text Capability of CLIP"

A library for easily evaluating machine learning models and datasets.

No description.

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

Swin2SR: SwinV2 Transformer for Compressed Image Super-Resolution and Restoration at the Advances in Image Manipulation (AIM) workshop ECCV 2022, Tel Aviv

Code for the Interspeech 2021 paper "AST: Audio Spectrogram Transformer".

This repository contains the code of the CVPR 2022 paper "Image Segmentation Using Text and Image Prompts".

Per-Pixel Classification is Not All You Need for Semantic Segmentation (NeurIPS 2021, spotlight)

No description.

GIT: A Generative Image-to-text Transformer for Vision and Language

OpenMMLab Semantic Segmentation Toolbox and Benchmark.

GLPDepth PyTorch Implementation: Global-Local Path Networks for Monocular Depth Estimation with Vertical CutDepth

This repository is an official implementation of the ICCV 2021 paper "Conditional DETR for Fast Training Convergence". (https://arxiv.org/abs/2108.06152)

Datasets, Transforms and Models specific to Computer Vision

OpenMMLab Image Classification Toolbox and Benchmark

VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

[NeurIPS 2022] Official code for "Focal Modulation Networks"

Language Modeling with the H3 State Space Model

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.

Easily compute clip embeddings and build a clip retrieval system with them

PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

Personalize Segment Anything Model (SAM) with 1 shot in 10 seconds

Boosting Image Matting with Pretrained Plain Vision Transformers

OpenMMLab Model Deployment Framework

This is a collection of our NAS and Vision Transformer work.

OpenMMLab Detection Toolbox and Benchmark

DataComp: In search of the next generation of multimodal datasets

The official implementation of SPTS v2: Single-Point Text Spotting

Code for T-MARS data filtering

The official implementation of "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

No description.

Toolkit for linearizing PDFs for LLM datasets/training

The repository provides code for running inference and finetuning with the Meta Segment Anything Model 3 (SAM 3), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

AI-assisted academic posters.

Code & data for TaxCalcBench

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

A collection of tutorials on state-of-the-art computer vision models and techniques. Explore everything from foundational architectures like ResNet to cutting-edge models like RF-DETR, YOLO11, SAM 3, and Qwen3-VL.

No description.

Official code and models for Video Encoder-only Mask Transformer (VidEoMT).

⏰ AI conference deadline countdowns

SGLang is a fast serving framework for large language models and vision language models.

CVE cache of the official CVE List in CVE JSON 5 format

[CVPR 2025 Highlight] Official code and models for Encoder-only Mask Transformer (EoMT).

CUDA accelerated rasterization of gaussian splatting

Get started with building Fullstack Agents using Gemini 2.5 and LangGraph

Official repository for "AnyCam: Learning to Recover Camera Poses and Intrinsics from Casual Videos" (CVPR 2025)

[CVPR 2025 Best Paper Nomination] FoundationStereo: Zero-Shot Stereo Matching

DSPy: The framework for programming—not prompting—language models

[CVPR 2025] "DiffusionSfM: Predicting Structure and Motion via Ray Origin and Endpoint Diffusion" official implementation.

The simplest, fastest repository for training/finetuning small-sized VLMs.

Official Code for "LoftUp: Learning a Coordinate-Based Feature Upsampler for Vision Foundation Models"

Code for "Scaling Language-Free Visual Representation Learning" paper (Web-SSL).

A TTS model capable of generating ultra-realistic dialogue in one pass.

Code for BLT research paper

[AAAI'25] DLF: Disentangled-Language-Focused Multimodal Sentiment Analysis

The official implementation for the CVPR'2025 paper Dynamic Updates for Language Adaptation in Visual-Language Tracking

Official PyTorch Implementation of Opt-CWM: Self-Supervised Learning of Motion Concepts by Optimizing Counterfactuals.

No description.

[CVPR'25] Official repository of Sonata: Self-Supervised Learning of Reliable Point Representations

This repo contains the code for our paper An Image is Worth 32 Tokens for Reconstruction and Generation

A Conversational Speech Generation Model

[CVPR2025] HVI: A New Color Space for Low-light Image Enhancement && "You Only Need One Color Space: An Efficient Network for Low-light Image Enhancement"

PyTorch implementation of FractalGen https://arxiv.org/abs/2502.17437

Unified automatic quality assessment for speech, music, and sound.

Make websites accessible for AI agents

Helper library for writing API clients.

No description.

No description.

Pytorch demo code and models for Multi-HMR

[ECCV 2024] OneRestore: A Universal Restoration Framework for Composite Degradation

Adding guardrails to large language models.

API Client for paperswithcode.com

Open source codebase powering the HuggingChat app

Official Implementation of Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Prediction

Official implementation of the 2024 ECCV paper SHIC: Shape-Image Correspondences with no Keypoint Annotation

ECCV 2024 & GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes

Text-to-Music Generation with Rectified Flow Transformers

[IJCAI'24] Beyond Alignment: Blind Video Face Restoration via Parsing-Guided Temporal-Coherent Transformer

StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text

[CVPR 2023] Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolatio

[AAAI 2023 Oral] CoMAE: Single Model Hybrid Pre-training on Small-Scale RGB-D Datasets

VFIMamba: Video Frame Interpolation with State Space Models

[ECCV 2024] DoubleTake: Geometry Guided Depth Estimation

CounTR: Transformer-based Generalised Visual Counting

No description.

Official PyTorch Implementation of "Scalable Autoregressive Image Generation with Mamba"

No description.

No description.

Silero VAD: pre-trained enterprise-grade Voice Activity Detector

VGGSfM: Visual Geometry Grounded Deep Structure From Motion

CoTracker is a model for tracking any point (pixel) on a video.