Qingsong Liu

Monkey (LMM); 多模态大模型 华科小猴子

Blind Source Separation for Audio Recognition Tasks

No description.

一个基于nano banana pro🍌的原生AI PPT生成应用，迈向＂Vibe PPT＂; 支持上传任意模板图片，上传任意素材&智能解析，一句话/大纲/页面描述自动生成PPT，口头修改指定区域、一键导出可编辑ppt - An AI-native slides generator based on nano banana pro🍌

High-fidelity tool to convert PDF files and images into fully editable PowerPoint presentations (.pptx). Unlike basic converters, it uses AI-powered structure extraction (MinerU) to accurately reconstruct text, layouts, and images, ensuring the output is not just a screenshot but a professional, modifiable document.

A Datacenter Scale Distributed Inference Serving Framework

使用Github Action将国外的Docker镜像转存到阿里云私有仓库，供国内服务器使用，免费易用

Instant, Concurrent, Secure & Lightweight Sandbox for AI Agents.

General-purpose web UI for Kubernetes clusters

Open Source Image and Video Restoration Toolbox for Super-resolution, Denoise, Deblurring, etc. Currently, it includes EDSR, RCAN, SRResNet, SRGAN, ESRGAN, EDVR, BasicVSR, SwinIR, ECBSR, etc. Also support StyleGAN2, DFDNet.

INTERSPEECH2023: Target Active Speaker Detection with Audio-visual Cues

The repository for IEEE CVPR 2023 (A Light Weight Model for Active Speaker Detection)

ACM MM 2021: 'Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection'

Visual Voice Activity Detection software, based on PyTorch and 3DI

Accepted by CVPR Workshop 2024

Out-of-the-box DeepSeek OCR document parsing Web Studio

VI-LayoutXLM（Visual-feature Independent LayoutXLM）

XFUND: A Multilingual Form Understanding Benchmark

该项目是为了使用layoutlmv3针对中文图片训练和推理。 其中主要解决三个问题： 1.数据标准化成可以的训练数据集格式 2.layoutlmv3-base-chinese 分词修改 2.超过512长度的文本切分和滑窗操作

支持国密SM2/SM3/SM4/SM9/SSL的密码工具箱

LLM-powered framework for deep document understanding, semantic retrieval, and context-aware answers using RAG paradigm.

An on-premises, OCR-free unstructured data extraction, markdown conversion and benchmarking toolkit. (https://idp-leaderboard.org/)

A lightweight LMM-based Document Parsing Model

Industry leading face manipulation platform

ICCV 2023 "Neural Video Depth Stabilizer" (NVDS) & TPAMI 2024 "NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimation" (NVDS+)

Port of Facebook's LLaMA model in C/C++

OpenCV - 3rdparty

Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research

Full stack, modern web application template. Using FastAPI, React, SQLModel, PostgreSQL, Docker, GitHub Actions, automatic HTTPS and more.

[NAACL 2024] Visually Guided Generative Text-Layout Pre-training for Document Intelligence

No description.

Whisper realtime streaming for long speech-to-text transcription and translation

An open-source implementaion for fine-tuning Qwen2-VL-2B and Qwen2-VL-7B.

A simple FastAPI Server to run XTTSv2

[ECCV2024] This is an official inference code of the paper "Glyph-ByT5: A Customized Text Encoder for Accurate Visual Text Rendering" and "Glyph-ByT5-v2: A Strong Aesthetic Baseline for Accurate Multilingual Visual Text Rendering""

Medical SAM 2: Segment Medical Images As Video Via Segment Anything Model 2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

RecordRTC is WebRTC JavaScript library for audio/video as well as screen activity recording. It supports Chrome, Firefox, Opera, Android, and Microsoft Edge. Platforms: Linux, Mac and Windows.

LLM based TTS model, providing inference/training/deployment full-stack ability.

Multilingual Voice Understanding Model

FunCodec is a research-oriented toolkit for audio quantization and downstream applications, such as text-to-speech synthesis, music generation et.al.

State-of-the-art audio codec with 90x compression factor. Supports 44.1kHz, 24kHz, and 16kHz mono/stereo audio.

No description.

No description.

Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

ChatTTS is a generative speech model for daily dialogue.

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Alibaba DAMO Academy.

The official code for the CVPR 2024 paper: Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer

AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation

A multimodal large-scale model, which performs close to the closed-source Qwen-VL-PLUS on many datasets and significantly surpasses the performance of the open-source model Qwen-VL-7B-Chat.

On the Hidden Mystery of OCR in Large Multimodal Models (OCRBench)

FiT: Flexible Vision Transformer for Diffusion Model

No description.

PhotoMaker

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Unofficial Implementation of Animate Anyone

Official implementations for paper: DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models

No description.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models (LLM-grounded Diffusion: LMD)

a list of demo websites for automatic music generation research

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

Generative Models by Stability AI

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

Emu: An Open Multimodal Generalist

An Open-source Toolkit for LLM Development

No description.

A PDF to Markdown converter

🔊 Text-Prompted Generative Audio Model

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

No description.

Implementation of Nougat Neural Optical Understanding for Academic Documents

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!

🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

An example Flask Application that has a progress bar using EventSource

Simple Example. Flask File Upload with jQuery Ajax.

An open-source framework for training large multimodal models.

pix2tex: Using a ViT to convert images of equations into LaTeX code.

Large Language-and-Vision Assistant built towards multimodal GPT-4 level capabilities.

Large Language Model Text Generation Inference

4 bits quantization of LLaMA using GPTQ

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

Transformer related optimization, including BERT, GPT

LAVIS - A One-stop Library for Language-Vision Intelligence

pycorrector is a toolkit for text error correction. 文本纠错，Kenlm，ConvSeq2Seq，BERT，MacBERT，ELECTRA，ERNIE，Transformer，T5等模型实现，开箱即用。

The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

No description.

No description.

No description.

Flask website integrated with Tesseract-OCR for reading multiple images, extracting text from them, and saving to Word, PDF, or txt file 🖼🡆🆎 [finished]

the cpp version of 3ddfav2

Modern spell checking library - accurate, fast, multi-language

No description.

FACIAL: Synthesizing Dynamic Talking Face With Implicit Attribute Learning. ICCV, 2021.

OCR evaluation brought to you by University of Alicante

Connectionist Temporal Classification (CTC) decoder with dictionary and language model for TensorFlow.

Connectionist Temporal Classification (CTC) decoding algorithms: best path, prefix search, beam search and token passing. Implemented in Python.

This github contains the network architectures of NeuralVoicePuppetry.

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.

Automatic Panorama Stitching From Scratch

Domain Adaptation as a Problem of Inference on Graphical Models

Implementation of "Duration Informed Attention Network for Multimodal Synthesis" (https://arxiv.org/pdf/1909.01700.pdf) paper.

Real-World Super-Resolution via Kernel Estimation and Noise Injection

ZBar bar code reader.

Magenta: Music and Art Generation with Machine Intelligence

A PyTorch implementation of Conv-TasNet described in "TasNet: Surpassing Ideal Time-Frequency Masking for Speech Separation" with Permutation Invariant Training (PIT).

No description.

A community-driven framework for managing your zsh configuration. Includes 120+ optional plugins (rails, git, OSX, hub, capistrano, brew, ant, macports, etc), over 120 themes to spice up your morning, and an auto-update tool so that makes it easy to keep up with the latest updates from the community.

Messing around hamming distance.

No description.

Code for Face Anti-spoofing Attack Detection Challenge @ CVPR2019

Open-Source SURF Library

C++ re-implementation of fast Discriminative Scale Space Tracking.

Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier

No description.

Fruits-360: A dataset of images containing fruits

Repository for the MPI operator.

No description.

No description.

Caffe Implementation of Google's MobileNets (v1 and v2)

Machine Learning Toolkit for Kubernetes

The ELK stack powered by Docker and Compose.

Elasticsearch Fluentd Kibana playground

Production-Grade Container Scheduling and Management

Deep Learning library for Python. Convnets, recurrent neural networks, and more. Runs on Theano and TensorFlow.

A web application to build your blog on GitHub

Python client for kubernetes

Go client for Kubernetes.

TensorFlow on GPU-scheduled Kubernetes

gpu monitor stats feed to influxdb

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Go, and more

PArallel Distributed Deep LEarning

OpenBLAS is an optimized BLAS library based on GotoBLAS2 1.13 BSD version.

Ready to use implementations of various Deep Learning algorithms using TensorFlow.

A TensorFlow implementation of DeepMind's WaveNet paper

TensorFlow implementation of Pixel Recurrent Neural Networks

An all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.)

This is now the official location of the Merlin project.

This is now the official location of the Kaldi project.

This is a speech analysis, modification and synthesis system

A YAML parser and emitter in C++

SaltStack for Kubernetes

Install k8s on coreos which is in a bare metal machine

HTTP proxy written in Go. COW can automatically identify blocked sites and use parent proxies to access.

MARY TTS -- an open-source, multilingual text-to-speech synthesis system written in pure java

Fast Recurrent Networks Library

ShadowsocksR update rss

Run Kubernetes and an nginx service on a Vagrant cluster of 3 CoreOS nodes.

中文自然语言处理工具包 Toolkit for Chinese natural language processing

Named Entity Recognition Tool

Transition-based word segmentation using neural networks based on package https://github.com/SUTDNLP/LibN3L

Tree-structured Long Short-Term Memory networks (http://arxiv.org/abs/1503.00075)

G2P with Tensorflow

RNN for acoustic speech generation

Mycroft's TTS engine, based on CMU's Flite (Festival Lite)

Speech Signal Processing - a small collection of routines in Python to do signal processing

Chrome extension to append sci-hub.org to tab domain.

Phonetic and phonological vocoding platform

A Cooperative Voice Analysis Repository for Speech Technologies

Praat: Doing Phonetics By Computer

Rocaloid UTAU Compatible Engine

Ian Goodfellow, Yoshua Bengio and Aaron Courville's deep learning book Chinese translation

Another Chinese Translation of Neural Networks and Deep Learning

This is the idlak branch from kaldi

RNNLIB is a recurrent neural network library for sequence learning problems. Forked from Alex Graves work http://sourceforge.net/projects/rnnl/

A recurrent neural network library for sequence learning problems.

No description.

Computational Network Toolkit (CNTK)

A high-quality speech analysis, manipulation and synthesis system

shell scripts to install different version of OpenCV in different distributions of Linux

Mel cepstral distortion (MCD) computations in python.

:microscope: Nano size Theano LSTM module

A project for conversion between Traditional and Simplified Chinese

The Fastest Fourier Transform in the South

Personal changes to festvox. I hope they reach upstream at some point.

The ultimate vim distribution

Collection of tests to check which C++11 features are supported by a specific (pre-installed) compiler.

Caffe: a fast framework for deep learning. For the most recent version checkout the dev branch. For the latest stable release checkout the master branch.

Frontend system for HMM-based speech synthesis models generated by HTS.

Copy of the US English STRAIGHT-based HTS demo for statistical parametric speech synthesis.

A git mirror of kaldi-trunk

my vim configuration (lot of python, autocompletition, fuzzy finder, debugger, ...)

Festival speech synthesis repository

Autoregressive HMM version of the HTS demo for statistical speech synthesis (includes autoregressive clustering)

dump ustc email and append to gmail