Tom Aarsen Claim this profile

Multilingual Sentence & Image Embeddings with BERT

NLTK Source

Efficient few-shot learning with Sentence Transformers

SpanMarker for Named Entity Recognition

Extend existing LLMs way beyond the original training length with constant memory usage, without retraining

Gather module dependencies of source code

Twitch Bot for generating messages based on what it learned from chat

Convert English text from written expressions into spoken forms

A Python wrapper allowing convenient access to the Noun Project API.

Sphinx theme for NLTK

Natural Language Inflection in English

Twitch Bot focusing on aggregating votes and averages from Twitch chat, with a GUI

Python Wrapper for easily connecting to Twitch and setting up a chat bot.

Twitch bot to save API information from chat messages and more in a database.

Twitch Bot allowing chat to play AI Dungeon

Twitch bot which displays weather information of any town

Twitch Bot to look up urban dictionary definitions and examples.

Send singular messages from a Bot account to chat

Twitch Bot focusing on aggregating votes and averages from Twitch chat

Model2Vec: Distill a Small Fast Model from any Sentence Transformer

Compare the performance of a Portfolio Reallocation Bot against holding on to the coins, for any (number of) Binance coins.

The easiest way to get started with LlamaIndex

BM25S is an ultra-fast lexical search library that implements BM25 using scipy

Twitch Bot that allows chat to keep track of a Death Counter

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Baichuan, Mixtral, Gemma, etc.) on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max). A PyTorch LLM library that seamlessly integrates with llama.cpp, HuggingFace, LangChain, LlamaIndex, DeepSpeed, vLLM, FastChat, ModelScope, etc.

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Angle-optimized Text Embeddings | 🔥 SOTA on STS and MTEB Leaderboard

Twitch Bot which can hilariously modify its input using Google Translate

Twitch bot that will suggest what you should get for dinner!

Twitch bot which create a queue for Mario Maker 2 Levels from chat, with a GUI

Rule-based spelling correction as query pre-processing on IR systems

Public repo for HF blog posts

Late Interaction Models Training & Retrieval

[EMNLP2025] "LightRAG: Simple and Fast Retrieval-Augmented Generation"

A lightweight, local-first, and 🆓 experiment tracking library from Hugging Face 🤗

LlamaIndex (GPT Index) is a data framework for your LLM applications

Backend for www.tomaarsen.com

Open-source search and retrieval database for AI applications.

Fine-tuning & Reinforcement Learning for LLMs. 🦥 Train OpenAI gpt-oss, DeepSeek, Qwen, Llama, Gemma, TTS 2x faster with 70% less VRAM.

LLM-powered Markdown editor

Nearly Inference Free Embeddings: make your RAG queries 500x faster

Frontend for www.tomaarsen.com

Retrieval Augmented Generation (RAG) chatbot powered by Weaviate

Twitch Bot to keep track of gift subscriptions

No description.

AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark

[ICLR 2024]EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modeling(https://arxiv.org/abs/2310.04691)

ColBERT: state-of-the-art neural search (SIGIR'20, TACL'21, NeurIPS'21, NAACL'22, CIKM'22, ACL'23, EMNLP'23)

Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone

Solution to a maximum independent set problem

just a bunch of useful embeddings

Twitch bot that periodically responds to messages with a rhyme

This repository contains an easy and intuitive approach to few-shot classification using sentence-transformers or spaCy models, or zero-shot classificaiton with Huggingface..

Twitch Bot to detect and rate DIAL-in checks.

Allow chat to generate some horrible recipes

Twitch bot which semi-randomly picks Mario Maker 2 Levels from chat, with a GUI

Simple Twitch Bot which semi-randomly picks a recent chatter from Chat.

Pythonic implementation of a Neural Network using Gradient descent

No description.

The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

The package used to build the documentation of our Hugging Face repos

Give your agents the power of the Hugging Face ecosystem

No description.

🦜🔗 Docs for LangChain projects

GenAI Agent Framework, the Pydantic way

Build compute kernels and load them from the Hub.

A lightweight, lightning-fast, in-process vector database

Benchmark for vector databases.

All the open source things related to the Hugging Face Hub.

No description.

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

Train transformer language models with reinforcement learning.

MTEB: Massive Text Embedding Benchmark

Utilities to use the Hugging Face hub API

No description.

🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

Hybrid search engine, combining best features of text and semantic search worlds

Data for the MTEB leaderboard

DSPy: The framework for programming—not prompting—foundation models

No description.

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

No description.

Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

The GPU-powered AI application database. Get your app to market faster using the simplicity of SQL and the latest NLP, ML + LLM models.

8-bit CUDA functions for PyTorch

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

No description.

⚡ Building applications with LLMs through composability ⚡

Generalist model for NER (Extract any entity types from texts)

Machine translated multilingual STS benchmark dataset.

No description.

Frontend components, documentation and information hosted on the Hugging Face website.

Landmark Attention: Random-Access Infinite Context Length for Transformers

Efficient Streaming Language Models with Attention Sinks

A multi-backend utility for benchmarking Transformers and Diffusers with support for hardware specific Optimum optimizations/quantization schemas

Website for the Weaviate vector database

No description.

Tensors and Dynamic neural networks in Python with strong GPU acceleration

💫 Industrial-strength Natural Language Processing (NLP) in Python

OpenChat: Less is More for Open-source Models

Salesforce open-source LLMs with 8k sequence length.

This repository contains an easy and intuitive approach to use SetFit in combination with spaCy.

The Official Python Client for Galileo

Create UIs for your machine learning model in Python in 3 minutes

🎨 Imagine what Picasso could have done with AI. Self-host your StableDiffusion API.

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

Running large language models like OPT-175B/GPT-3 on a single GPU. Focusing on high-throughput large-batch generation.

✨ Rubrix, open-source framework for data-centric NLP. Data annotation and monitoring for enterprise NLP

No description.

One Embedder, Any Task: Instruction-Finetuned Text Embeddings

The WeightWatcher tool for predicting the accuracy of Deep Neural Networks

A Python utility / library to sort imports.

This repository, called fast sentence transformers, contains code to run 5X faster sentence transformers using tools like quantization and ONNX.

This repository contains an easy and intuitive approach to few-shot NER using most similar expansion over spaCy embeddings. Now with entity scoring.

Fuzzy matching and more functionality for spaCy.

Universal Dependencies online documentation

Asent is a python library for performing efficient and transparent sentiment analysis using spaCy.

Graph-Based Data Science: an abstraction layer in Python for building knowledge graphs, integrated with popular graph libraries – atop Pandas, RDFlib, pySHACL, RAPIDS, NetworkX, iGraph, PyVis, pslpython, pyarrow, etc.

Implementation of the ClausIE information extraction system for python+spacy

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

🚴 Call stack profiler for Python. Shows you why your code is slow!

Modules to convert numbers to words. 42 --> forty-two

Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

Python implementation of TextRank for phrase extraction and summarization of text documents

A Rust CLI tool to graph GitHub Contribution Graphs

Compiler for Simple Programming Language (SPL), written in Python

Variable Elimination in Bayesian Networks

Solution to an perfect bipartite matching problem

7th place entry to the Hotel-ID 2022 Kaggle challenge

NLTK Data

Accurately generate all possible forms of an English word e.g "election" --> "elect", "electoral", "electorate" etc.

Command-line program to download videos from YouTube.com and other video sites

huntr.dev browser extension, allowing you to view bounty rewards on a repository as you browse GitHub.

Awesome autocompletion, static analysis and refactoring library for python

Repository of the Bayesian Networks course at RU

NLTK Website

Generate readable documentation for users based on source code with epytext-formatted docstrings, with support for zope.interface.

Python client for the Twitter 'search Tweets' and 'count Tweets' endpoints (v2/Labs/premium/enterprise). Now supports Twitter API v2 /recent and /all search endpoints.

bTCP: basic Transmission Control Protocol

Prosodic: a metrical-phonological parser, written in Python. For English and Finnish, with flexible language support.

Updated, simplified prosodic

WordNet Interface

Brainfuck Interpreter for generating LaTeX commands of program executions