Dustin Franklin

Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.

A guided tutorial library for running computer vision AI on NVIDIA Jetson edge devices, covering object detection, segmentation, pose estimation, and training custom models.

Machine Learning Containers for NVIDIA Jetson and JetPack-L4T

Deep learning inference nodes for ROS / ROS2 with support for NVIDIA Jetson and TensorRT

Deep reinforcement learning GPU libraries for NVIDIA Jetson TX1/TX2 with PyTorch, OpenAI Gym, and Gazebo robotics simulator.

C++/CUDA/Python multimedia utilities for NVIDIA Jetson

ROS nodes and Gazebo model for NVIDIA JetBot with Jetson Nano

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

ASR/NLP/TTS deep learning inference library for NVIDIA Jetson using PyTorch and TensorRT

Configuration scripts and install guides for NVIDIA Jetson platform

Tegra Unmanned Retrieval Bot gen2, with NVIDIA Jetson and deep reinforcement learning

PDF slides about NVIDIA's Jetson embedded platform and deep learning.

Zero-copy multimodal vector DB with CUDA and CLIP/SigLIP

MobileNetV1, MobileNetV2, VGG based SSD/SSD-lite implementation in PyTorch. Out-of-box support for retraining on Open Images dataset. ONNX and Caffe2 support. Experiment Ideas like CoordConv.

GUI tool for collecting & labeling data from live camera feed

CLIP and SigLIP models optimized with TensorRT with a Transformers-like API

Deep reinforcement learning-driven robotic controller and navigation library using Torch.

Training of image classification models with PyTorch

Training of semantic segmentation networks with PyTorch

PyTorch super resolution model with RGB support and ONNX exporter

No description.

Training of object detection networks with PyTorch

Training of visual odometry estimation networks using PyTorch

Code for accompanying our OpenCV video tutorial series.

NIMs powershell for launching local AI microservices and open-stack model templates

MonoDepth Estimation - ICRA 2018 "Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image"

OpenVLA: An open-source vision-language-action model for robotic manipulation.

No description.

No description.

Jetson AI Lab - LLM Inference

Datasets, Transforms and Models specific to Computer Vision

A project that optimizes Whisper for low latency inference using NVIDIA TensorRT

ROS2 nodes for Pololu Maestro, Jrk, and Simple Motor Controller

A collection ROS projects utilizing foundation models.

8-bit CUDA functions for PyTorch, modified to build on NVIDIA Jetson

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

Normalized Direct Linear Transform (DLT) for 4-point Homography Estimation

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

ORB_SLAM2 with GPU Enhancement running on NVIDIA Jetson TX1. Focus on ROS part.

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

Python module for CUDA accelerated SIFT on GPUs

robosuite: A Modular Simulation Framework and Benchmark for Robot Learning

Real-time pose estimation accelerated with NVIDIA TensorRT

A continuous action space version of A3C LSTM in pytorch plus A3G design

Unified framework for robot learning built on NVIDIA Isaac Sim

[RSS 2024] 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations

Port of Facebook's LLaMA model in C/C++

Python bindings for llama.cpp

ROS 2 wrapper beta for the ZED SDK

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Download ImageNet images from list of IDs and URLs

[RSS 2023] Diffusion Policy Visuomotor Policy Learning via Action Diffusion

This code corresponds to simulation environments used as part of the MimicGen project.

cuML - RAPIDS Machine Learning Library

A training framework for Stable Baselines3 reinforcement learning agents, with hyperparameter optimization and pre-trained agents included.

Python bindings for the Gazebo multi-robot simulator.

Algorithm-agnostic computer vision message types for ROS.

Toolkit for efficient experimentation with Speech Recognition, Text2Speech and NLP

KenLM: Faster and Smaller Language Model Queries

Building a homography dataset and training with PyTorch

⚙️ ROS 2 packages to interact with the nanosaur hardware

No description.

A command line toolkit to generate maps, point clouds, 3D models and DEMs from drone, balloon or kite images. 📷

No description.

No description.

Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. Please follow Documentation/SubmittingPatches procedure for any of your improvements.

📦 nanosaur CLI to manage and control nanosaur

👨‍💻 run nanosaur in a simulated world!

🖼️ Perception for nanosaur. Works with Isaac ROS

🦕 nanosaur is a little robot, powered by ROS 2. Made for NVIDIA Jetson

Python Docker Hub API client library

cuDF - GPU DataFrame Library

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Dash components for building draggable and resizable grid layouts. Based on the well-known STRML/react-grid-layout.