Max Bain

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval [ICCV'21]

Large-scale text-video dataset. 10 million captioned short videos.

Story-Based Retrieval with Contextual Embeddings. Largest freely available movie video dataset. [ACCV'20]

Implementations of Transformers for Video

Condensed Movies Challenge 2021

A Clip-Hitchiker's Guide to Long Video Retrieval [Arxiv 2022]

LAVIS - A One-stop Library for Language-Vision Intelligence

A pytorch implemented classifier for Multiple-Label classification

OpenAI Whisper ASR Webservice API

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

Simple Diarization model

No description.

An open-source AI agent that brings the power of Gemini directly into your terminal.

Multimodal language model benchmark, featuring challenging examples

No description.

CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

Easily create large video dataset from video urls

Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

Automated Audiovisual Behaviour Recognition in Wild Primates

Video embeddings for retrieval - code for the paper "Use What You Have: Video retrieval using representations from collaborative experts"

Pytorch port of Google Research's VGGish model used for extracting audio features.

A simple command line tool to show GPU usage on a SLURM cluster

PyTorch image models, scripts, pretrained weights -- (SE)ResNet/ResNeXT, DPN, EfficientNet, MixNet, MobileNet-V3/V2, MNASNet, Single-Path NAS, FBNet, and more

Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Robust Speech Recognition via Large-Scale Weak Supervision

No description.

Extract video features from raw videos using multiple GPUs. We support RAFT and PWC flow frames as well as S3D, I3D, R(2+1)D, VGGish, CLIP, ResNet features.

Mapping a variable-length sentence to a fixed-length vector using BERT model

Hydra is a framework for elegantly configuring complex applications