The Enterprise-Grade Multi-Agent Orchestration Framework. Website: https://swarms.ai

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

Build high-performance AI models with modular building blocks

A compilation of the best multi-agent papers

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

Integrating Mamba/SSMs with Transformer for Enhanced Long Context and High-Quality Sequence Modeling

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Implementation of Alpha Fold 3 from the paper: "Accurate structure prediction of biomolecular interactions with AlphaFold3" in PyTorch

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

Democratization of RT-2 "RT-2: New model translates vision and language into action"

Implementation of Vision Mamba from the paper: "Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model" It's 2.8x faster than DeiT and saves 86.8% GPU memory when performing batch inference to extract features on high-res images

The open source implementation of Gemini, the model that will "eclipse ChatGPT" by Google

Towards Generalist Biomedical AI

Implementation of the ScreenAI model from the paper: "A Vision-Language Model for UI and Infographics Understanding"

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

An open source implementation of "Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning", an all-new multi modal AI that uses just a decoder to generate both text and images

Implementation of "PaLM-E: An Embodied Multimodal Language Model"

My implementation of "Patch n’ Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolution"

Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment: Robotic Learning Datasets and RT-X Models"

An open source implementation of LFMs from Liquid AI: Liquid Foundation Models

PyTorch Implementation of Jamba: "Jamba: A Hybrid Transformer-Mamba Language Model"

Open source implementation of "Vision Transformers Need Registers"

A easy, reliable, fluid template for python packages complete with docs, testing suites, readme's, github workflows, linting and much much more

Implementation of "Attention Is Off By One" by Evan Miller

An all-new Language Model That Processes Ultra-Long Sequences of 100,000+ Ultra-Fast

Implementation of PALI3 from the paper PALI-3 VISION LANGUAGE MODELS: SMALLER, FASTER, STRONGER"

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

Seed, Code, Harvest: Grow Your Own App with Tree of Thoughts!

Implementation of "MORPHEUS-1" from Prophetic AI and "The world’s first multi-modal generative ultrasonic transformer designed to induce and stabilize lucid dreams. "

Implementation of MoE Mamba from the paper: "MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts" in Pytorch and Zeta

Implementation of MambaByte in "MambaByte: Token-free Selective State Space Model" in Pytorch and Zeta

Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"

Implementation of xLSTM in Pytorch from the paper: "xLSTM: Extended Long Short-Term Memory"

"Improving Mathematical Reasoning with Process Supervision" by OPENAI

Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels

My implementation of "Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models"

Pytorch Implementation of the sparse attention from the paper: "Generating Long Sequences with Sparse Transformers"

Democratization of "PaLI: A Jointly-Scaled Multilingual Language-Image Model"

Implementation of Deepmind's RoboCat: "Self-Improving Foundation Agent for Robotic Manipulation" An next generation robot LLM

My implementation of Kosmos2.5 from the paper: "KOSMOS-2.5: A Multimodal Literate Model"

Plug in and play implementation of " Textbooks Are All You Need", ready for training, inference, and dataset generation

The Next Generation Multi-Modality Superintelligence

Implementation of Liquid Nets in Pytorch

Implementation of PyTorch: "GAMBA: MARRY GAUSSIAN SPLATTING WITH MAMBA FOR SINGLE-VIEW 3D RECONSTRUCTION"

A multi-modal AI Model that can generate high quality novel videos with text, images, or video clips.

Implementation of the transformer from the paper: "Real-World Humanoid Locomotion with Reinforcement Learning"

An open source community implementation of the model from the paper: "Movie Gen: A Cast of Media Foundation Models". Join our community to help implement this model!

Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTORCH

Finetune any model on HF in less than 30 seconds

Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"

Implementation of the premier Text to Video model from OpenAI

Triton implementation of Flash Attention2.0

32 times longer context window than vanilla Transformers and up to 4 times longer than memory efficient Transformers.

Implementation of Qformer from BLIP2 in Zeta Lego blocks.

The next evolution of Agents

This repository provides a minimal, single-file implementation of SingLoRA (Single Matrix Low-Rank Adaptation) as described in the paper "SingLoRA: Low Rank Adaptation Using a Single Matrix" by Bensaïd et al.

Implementation of AutoRT: "AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents"

The world's first fully automated VC fund.

An open source community implementation of the model from "DIFFERENTIAL TRANSFORMER" paper by Microsoft.

AthenaOS is a next generation AI-native operating system managed by Swarms of AI Agents

Implementation of ViTaR: ViTAR: Vision Transformer with Any Resolution in PyTorch

Implementation of a modular, high-performance, and simplistic mamba for high-speed applications

An implementation of "metnet3" in Pytorch

A clean, single-file PyTorch implementation of Attention Residuals (Kimi Team, MoonshotAI, 2026), integrated with Grouped Query Attention (GQA), SwiGLU feed-forward networks, and Rotary Position Embeddings (RoPE).

Implementation of the model "AudioFlamingo" from the paper: "Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities"

The open source implementation of the cross attention mechanism from the paper: "JOINTLY TRAINING LARGE AUTOREGRESSIVE MULTIMODAL MODELS"

This is an ultra-simple, single-file PyTorch implementation of MoonViT, the native-resolution vision encoder from Kimi-VL.

Implementation of the "the first large-scale multimodal mixture of experts models." from the paper: "Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts"

Implementation of Google's USM speech model in Pytorch

Pytorch Implementation of Deepmind's SIMA: "Scaling Instructable Agents Across Many Simulated Worlds"

The open source implementation of the model from "Scaling Vision Transformers to 22 Billion Parameters"

Plug in & Play Pytorch Implementation of the paper: "Evolutionary Optimization of Model Merging Recipes" by Sakana AI

Community Open Source Implementation of GPT4o in PyTorch

A simple and minimal open source implementation of "Introducing LFM2: The Fastest On-Device Foundation Models on the Market" from Liquid AI in Pytorch

An open source implementation of R1

An open source replication of the stawberry method that leverages Monte Carlo Search with PPO and or DPO

Community Implementation of the paper: "Multi-Head Mixture-of-Experts" In PyTorch

Understanding CLAUDE.md, MEMORY.md, SKILLS.md, SOUL.md, and Related Prompting Mechanisms

Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch

A vast array of Multi-Modal Embodied Robotic Foundation Models!

A simpler Pytorch + Zeta Implementation of the paper: "SiMBA: Simplified Mamba-based Architecture for Vision and Multivariate Time series"

This repository is an open source implementation of the MuonClip strategy from the KIMI K2 Model from Moonshot AI

Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and minimal learning curve.

Implementation of the model: "(MC-ViT)" from the paper: "Memory Consolidation Enables Long-Context Video Understanding"

PyTorch Implementation of the paper "MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training"

Implementation of "Do As I Can, Not As I Say: Grounding Language in Robotic Affordances" by Google

The World's First AI-Enabled Multi-Modality Native Search Engine

Implementation of the paper: "BRAVE : Broadening the visual encoding of vision-language models"

Implementation of the paper: "Aurora: A Foundation Model of the Atmosphere" in PyTorch

Pytorch Implementation of the paper: "Learning to (Learn at Test Time): RNNs with Expressive Hidden States"

A simple reproducible template to implement AI research papers

Implementation of MambaFormer in Pytorch ++ Zeta from the paper: "Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks"

An implementation of the base GPT-3 Model architecture from the paper by OPENAI "Language Models are Few-Shot Learners"

An simple pytorch implementation of Flash MultiHead Attention

CogNetX is an advanced, multimodal neural network architecture inspired by human cognition. It integrates speech, vision, and video processing into one unified framework.

Implementation of "Audio xLSTMs: Learning Self-supervised audio representations with xLSTMs" in PyTorch

A swarm of LLM agents that will help you test, document, and productionize your code!

A forest of autonomous agents.

Implementation of LatentMoE,Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts (Elango et al., NVIDIA 2026) — in Pytorch. A single-file, dependency-light layer you can drop in place of a standard MoE FFN.

The open source implementation of the multi grouped query attention by the paper "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints"

Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT

Implementation of "PaLM2-VAdapter:" from the multi-modal model paper: "PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter"

OmniByteFormer is a generalized Transformer model that can process any type of data by converting it into byte sequences, bypassing traditional tokenization or specific data-type encodings.

An open source community implementation of the model MELLE from the paper: "Autoregressive Speech Synthesis without Vector Quantization"

Implementation of a Hierarchical Mamba as described in the paper: "Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling"

Plug in and Play implementation of "Certified Reasoning with Language Models" that elevates model reasoning by 40%

Implementation of the model "Hedgehog" from the paper: "The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry"

Welcome to AGI, the cutting-edge project dedicated to building the core components of Artificial General Intelligence.

Transformers + Mambas + LSTMS All in One Model

NETWATCH is a terminal-based security monitoring tool that gives you continuous, live visibility into every active network connection on your machine, with automatic risk scoring, GeoIP intelligence, VPN status, process path validation, and new-connection alerting.

We introduce VLM-Mamba, the first Vision-Language Model built entirely on State Space Models (SSMs), specifically leveraging the Mamba architecture.

Implementation of VisionLLaMA from the paper: "VisionLLaMA: A Unified LLaMA Interface for Vision Tasks" in PyTorch and Zeta

Welcome to Prometheus, the revolutionary AI model that allows you to generate DNA sequences for any creature you can imagine. Whether it’s a pink panda, an elephant-sized turtle, or a completely new lifeform from your wildest dreams, Prometheus decodes the mystery of biology and synthesizes genetic blueprints with precision.

Implementation of the paper: "Audio Mamba: Bidirectional State Space Model for Audio Representation Learning" in pytorch

A super simple cheatsheet for poetry because I forget easily.

Tiktok is an advanced multimedia recommender system that fuses the generative modality-aware collaborative self-augmentation and contrastive cross-modality dependency encoding to achieve superior performance compared to existing state-of-the-art multi-model recommenders.

An open source community implementation of VARC: "ARC Is a Vision Problem!"

My implementation of the model KosmosG from "KOSMOS-G: Generating Images in Context with Multimodal Large Language Models"

My personal implementation of the model from "Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities", they haven't released model code yet sooo...

The prompts I use daily for building documentation, validating ideas, and other activities.

Zeta implemantion of "Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers"

Plug in and Play Prompt Technique to Boost Model reasoning by 40%

Open source implementation of a vision transformer that can understand Videos using max vit as a foundation.

OpioidRL is a cutting-edge reinforcement learning (RL) library that simulates drug addiction behaviors within RL agents. Inspired by the addictive properties of drugs like methamphetamine and crack cocaine, OpioidRL offers a unique environment where agents experience reward dependency, high-risk decision-making, and compulsive behaviors — pushing

Multi-Agent Template App

A dynamic, self-healing, and AI Driven Hard Drive Encryption Algorithm Inspired from Captain America

TableMamba is a Mamba-based sequential recommender that encodes a user's interaction history with selective state-space blocks and a multi-interest head of M parallel SSM readers at log-spaced timescales, scoring items by their best-matching interest in O(T).

Implementation of the model from "Faster sorting algorithms discovered using deep reinforcement learning" that discovered an all-new ultra fast sorting algorithm.

A framework making it effortless to convert any llm model into a reasoning agent like o1 or DeepSeek's r1

My Implementation of "Decoding speech perception from non-invasive brain recordings"

This repository is a straightforward attempt to implement the base Kimi K2 Reasoning model architecture in pure PyTorch as simply as possible.

The open source implementation of the base model behind GPT-4 from OPENAI [Language + Multi-Modal]

A new optimizer, Gradient Ascent: Gradient Ascent adjusts the parameters in the direction of the gradient to maximize some objective function.

Tree of Thoughts with an meta prompt for 50% boost in model reasoning

This module implements the Gemini Embedding model as described in the research paper "Gemini Embedding: Generalizable Embeddings from Gemini" (2025).

A conversational platform for the swarms framework.

An implementation of the paper: "Gated Slot Attention for Efficient Linear-Time Sequence Modeling" in PyTorch

Synchronize your requirement.txt and pyproject.toml at the bush of a button!

Unofficial open-source PyTorch implementation of the OLMo Hybrid architecture introduced by the Allen Institute for AI (Ai2).

A framework to enable bulk processing of Linkedin Leads profile URLs to acquire lead emaisl

A simple, open, and PyTorch implementation of the ViT from the GLM paper: “tGLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents”

An implementation of the all-new rope from jianlin

My own implementation of "Fast Inference from Transformers via Speculative Decoding"

The APAC AI Hub for Documents, Product Briefs, Plans, and SOPS, we're currently raising SAFE 100M$ at a 1Billion$ valuation.

A simple pytorch implementation of a meta learning algorithm from OPENAI "Reptile: A scalable meta-learning algorithm"

An all-new model architecture to detect devices on ranges of radio frequencies using transformers and mambas

An implementation of a switch transformer like Multi-query attention model

StarForce-X, an enterprise-grade, AI-driven model engineered to intercept, analyze, and decode alien radio frequencies and communications with unparalleled precision.

The python API for the swarms marketplace API: Add, Update, Query, Update Agents, Tools, Prompts + More

This repository demonstrates how to build a Decoder-Only Transformer with Multi-Query Attention in JAX.

An extremely experimental model that intakes images and generates 3D scenes of those images using Diffusion

Implementation of the Paper: "Zamba: A Compact 7B SSM Hybrid Model" in Pytorch

Deep MoE is an all-new mixture-of-experts architecture a transformer layer that does its thinking in a compressed thought space rather than on the wide residual stream.

Reaper is a simple polymorphic malware algorithm

This collection brings together the highest-signal research papers in modern AI from the invention of the Transformer to the frontier work of 2024–2025 into a single, curated map of the field

The Sonar Vision Mapping System is an advanced environmental mapping and visualization technology. It utilizes high-frequency sound pulses to create a real-time, three-dimensional map of an environment. This system is designed for applications in security, surveillance, and search-and-rescue operations.

Building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize.

Implementation of "CogVLM2: Visual Language Models for Image and Video Understanding" in PyTorch

A discord bot that can do anything.

An open source implementation of the paper: "AN EVOLVED UNIVERSAL TRANSFORMER MEMORY"

All mathematics research papers sourced from ArXiv and meticulously curated for LLM pretraining purposes.

An open-source autonomous AI agent implementation inspired by Cursor Agent, built with the Swarms framework. This production-grade agent can autonomously plan, execute, and complete complex tasks using a combination of Large Language Model reasoning and tool execution.

A network of attention mechanisms at your fingertips. Unleash the potential of attention mechanisms for diverse AI applications. AttentionGrid is all you need!

A simple implementation of a CLIP that splits up an image into quandrants and then gets the embeddings for each quandrant

An awesome list of omni-modality LLM models that can perceive and generate images, videos, audios, and more all at once

Implementation of the paper "SHORTCIRCUIT: ALPHAZERO-DRIVEN CIRCUIT DESIGN" in PyTorch

Implementation of Attention on Attention in Zeta

Advancing Humanity with Multi-Modality AI

This paper presents a comprehensive analysis of Kurt G¨odel’s ontological proof for the existence of God, with particular focus on contemporary expansions and refinements of the original framework

This CNN-based language model leverages causal and dilated convolutions, gated activations, residual connections, and layer normalization to effectively model textual data for generation tasks.

An experimental repository on research for training multiple models all at once in an evolutionary capacity!

The Ultimate Hub of AI Models - Simplified, Streamlined, and Scalable for Production-Grade Deployment.

SOTA Classification at scale for UAVs, Drones, and much more

MambaDecoderBlock is a novel decoder architecture that replaces traditional self-attention mechanisms with Mamba state space models, augmented by Mixture of Experts (MoE) layers.

A pytorch implementation of the paper: ""Toto: Time Series Optimized Transformer for Observability"

open-source nanotech CAD

Athena is an all-in-one app that allows easy creation of various things like images, videos, music, and soon, even apps, using natural language.

An implementation of Tree-Attention in PyTorch because it's in JAX for some reason

A production-grade implementation of a memory-efficient transformer specifically designed for tabular time series data.

Implementation of the paper: "HGRN2: Gated Linear RNNs with State Expansion" in PyTorch

Your Personal Hospital Powered by Swarms.ai

Open source implementation of a super reliable web crawler powered by LLMs by OpenAI

An open-source PyTorch implementation of OmniAlignNet from the OmniVinci paper, designed to align vision and audio embeddings in a shared omni-modal space.

This paper presents a groundbreaking framework that models economic systems as intelligent neural networks, offering a novel approach to understanding how economies learn, adapt, and self-organize.

Shadows of Other Worlds: Detecting Multiversal Interference in Quantum Measurements

An exploration into leveraging SSM's as Bridge/Adapter Layers for VLM

An free and open source community implementation of Chai-1 in PyTorch

A simple but complete full-attention transformer with a set of promising experimental features from various papers

An experimental open-source attempt to make GPT-4 fully autonomous.

A multimodal foundation model for humanoid robotics that integrates multiple input modalities—text, speech, vision (images and videos), and outputs both actions and speech simultaneously like a transformer.

SuperPrompt is an attempt to engineer prompts that might help us understand AI agents. I made an prompt optimization swarm and updated it hundreds of times. This is the 400th iteration

This paper introduces a comprehensive and formal theory on value creation, bridging the gap between classical economic models and contemporary economic realities driven by technology, innovation, and intangible assets.

A non-official implementation of Qwen 3.5, as there doesn’t seem to be a paper or any code available that I can find, so I decided to implement it just for fun.

No description.

An Definitive and Unified AI Agents Framework to Automate Anything and Everything

Open-source pre-training implementation of Google's LaMDA in PyTorch.

KNOTX is a new and theoretical activation function designed for ultra-fast multi-modality learning utilizing knot theory + dynamical systems modeling.

A Python package for segmenting geospatial data with the Segment Anything Model (SAM)