OpenGVLab ORG

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

[ECCV2024] VideoMamba: State Space Model for Efficient Video Understanding

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

Unofficial Implementation of DragGAN - "Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold" （DragGAN 全功能实现，在线Demo，本地部署试用，代码、模型已全部开源，支持Windows, macOS, Linux）

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

[CVPR 2023 Highlight] InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

VisionLLM Series

Official implementation of SAM-Med2D

[ICLR 2026 Oral] ScaleCUA is the open-sourced computer use agents that can operate on cross-platform environments (Windows, macOS, Ubuntu, Android).

[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

[CVPR 2024] Deformable Convolution v4

Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!

[ICLR 2025 Spotlight] Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures

[ICLR2026] VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling

[ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World"

[ICLR 2025 Spotlight] OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text

[CVPR 2023] Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners

Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model

[T-PAMI 2025] PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm

[ICCV2023] UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer

[ICCV2023 Oral] Unmasked Teacher: Towards Training-Efficient Video Foundation Models

[ACL 2025 Main] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

[NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI Agents

The suite of modeling video with Mamba

InternVL-U is a 4B-parameter unified multimodal model (UMM) that brings multimodal understanding, reasoning, image generation, image editing into a single framework.

[NIPS2025] VideoChat-R1 & R1.5: Enhancing Spatio-Temporal Perception and Reasoning via Reinforcement Fine-Tuning

MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer

This repo is official implementation of HumanBench (CVPR2023)

Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model

ControlLLM: Augment Language Models with Tools by Searching on Graphs

General Vision Benchmark, GV-B, a project from OpenGVLab

No description.

No description.

[ICML2025] The code and data of Paper: Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

Official PyTorch implementation of UniHCP

[ICCV 2025] GUIOdyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUIOdyssey consists of 8,834 episodes from 6 mobile devices, spanning 6 types of cross-app tasks, 212 apps, and 1.4K app combos.

An official implementation of "Hulk: A Universal Knowledge Translator for Human-Centric Tasks"

[CVPR 2024 Champions][ICLR 2025] Solutions for EgoVis Chanllenges in CVPR 2024

[ACL 2024] ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.

[NeurIPS 2024] Needle In A Multimodal Haystack (MM-NIAH): A comprehensive benchmark designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.

ZeroGUI: Automating Online GUI Learning at Zero Human Cost

[ICML 2024] | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

[NeurIPS 2024 Spotlight ⭐️ & TPAMI 2025] Parameter-Inverted Image Pyramid Networks (PIIP)

[CVPR 2025] Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training

Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeed

[ICCV 23]An approach to enhance the efficiency of Vision Transformer (ViT) by concurrently employing token pruning and token merging techniques, while incorporating a differentiable compression rate.

Sequential Diffusion Language Model (SDLM) enhances pre-trained autoregressive language models by adaptively determining generation length and maintaining KV-cache compatibility, achieving high efficiency and throughput.

[ICLR2025] MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

No description.

Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model

[CVPR 2023] implementation of Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information.

Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces

「AAAI 2024」 Referred by Multi-Modality: A Unified Temporal Transformers for Video Object Segmentation

[CVPR 2024] Data and benchmark code for the EgoExoLearn dataset

Awesome-DragGAN: A curated list of papers, tutorials, repositories related to DragGAN

Official Implementation of "Denoising Diffusion Semantic Segmentation with Mask Prior Modeling"

[ICLR 2025] TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning

[NeurIPS 2024] Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning

No description.

[ICML 2026] GenExam: A Multidisciplinary Text-to-Image Exam

A curated list of the papers, repositories, tutorials, and anythings related to the large language models for tools

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment

[ICML2023] Long-Term Rhythmic Video Soundtracker

[ICCV2025] V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

[CVPR 2025] PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models

No description.

Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning

[CVPR 2023]Implementation of Siamese Image Modeling for Self-Supervised Vision Representation Learning

Make Your Training Flexible: Towards Deployment-Efficient Video Models

[CVPR 2025] Docopilot: Improving Multimodal Models for Document-Level Understanding

[NIPS2023]Implementation of Foundation Model is Efficient Multimodal Multitask Model Selector

Learning 1D Causal Visual Representation with De-focus Attention Networks

No description.

[ICCV 2025] A Benchmark for Multi-Step Reasoning in Long Narrative Videos

[CVPR 2024] DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model

No description.

BESA is a differentiable weight pruning technique for large language models.

Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction

No description.

No description.

No description.

Official implementation of: Learning Goal-Oriented Language-Guided Navigation with Self-Improving Demonstrations at Scale

Champion Solutions repository for Perception Test challenges in ICCV2023 workshop.

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

No description.

[ICLR 2026] RIVER: A Real-Time Interaction Benchmark for Video LLMs

No description.

Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

This repo takes the initial step towards leveraging text learning for online action detection without explicit human supervision.

The first unified, efficient, and extensible evaluation toolkit for evaluating image generation and editing models across multiple benchmarks.