9-day longest streak
VINAY JOGANI Machine Learning Engineer | Real-Time Voice AI & Agentic Systems | LLM Engineering, MLOps & Data Infrastructure Machine Learning Engineer at RSK IT International, building real-time voice agents…
VINAY JOGANI
Machine Learning Engineer | Real-Time Voice AI & Agentic Systems | LLM Engineering, MLOps & Data Infrastructure
Machine Learning Engineer at RSK IT International, building real-time voice agents running in production at drive-thrus, toll booths, and parking lots. MS in Information Systems, Northeastern University (August 2025). I bridge research and production: 4 peer-reviewed publications on explainable AI and adversarial robustness, open-source AI infrastructure on PyPI and npm, and production systems from streaming audio pipelines to agentic LLM workflows.
Resume: View Resume (PDF)
---
About Me
I build ML systems that survive contact with production. Right now that means real-time voice AI at RSK IT International: streaming audio pipelines with 80-120 ms chunking, VAD and directed-speech gating, ASR/NLU integration, and intent routing into POS, tolling, and gate-control systems, holding end-to-end turn times under ~500 ms across 3 live sites, deployed on-prem and in hybrid edge/cloud setups for 3 production customers and 2 POCs.
Alongside that, I design and ship open-source AI infrastructure focused on making LLM and agentic systems reliable: evals, guardrails, and observability. Earlier, I did AI research at Northeastern's Amal Lab (medical image segmentation, 18TB+ multi-omics processing, multi-GPU training) and clinical data science at Brigham and Women's Hospital (transformer NLP over 10,000+ pharma reports, market microstructure analysis).
---
Open-Source AI Infrastructure
| Project | What it does |
|---|---|
| tracewall | Flight recorder + firewall for AI coding agents: records every tool call, tracks lethal-trifecta taint, blocks prompt-injection exfiltration. Zero dependencies, on PyPI. |
| spareloop | Scheduler + optimizer for AI coding CLIs (Claude Code, Codex, Cursor): learns your usage rhythm, queues work into spare capacity, prewarms your usage window. On npm. |
| whichmodel | Describe your task, get one AI model recommendation backed by benchmarks and real developer sentiment. Zero-dependency, CI-validated dataset. |
Featured Projects
| Project | Highlights |
|---|---|
| Bank Reconciliation Agent | Production agentic pipeline (Claude tool calling + rules): 90.6% accuracy / 92.5% F1 on a golden eval suite, invariants, traces, replay |
| Agentic RAG for 10-K Analysis | ReAct agent routing SQL + semantic search, 89% accuracy graded by LLM-as-a-judge |
| Healthcare Data Pipeline | 10M+ patient records: Kafka + Spark Streaming, dbt medallion (31 models), Great Expectations, Terraform, Prometheus/Grafana |
| Skin Cancer Classification at Scale | 91.8% across 35 conditions on 245K images; 3.37x training speedup via PyTorch DDP on 4x A100 + AMP |
| NBA Injury Prediction System | Full MLOps: FastAPI at sub-100ms p95, 1000+ RPS, MLflow, Prometheus/Grafana, K8s |
| BankNifty Vectorized Backtester | 10.3M rows of 1-minute options data processed in under 8 seconds, zero loops |
---
Experience
Machine Learning Engineer, RSK IT International (September 2025 – Present)
- Engineered low-latency streaming audio pipelines for drive-thru, toll-booth, and parking-lot voice agents across 3 live sites (80-120 ms chunking, VAD/directed-speech gating, ASR, intent routing), holding end-to-end turn times under ~500 ms
- Architected and deployed serving topologies for 3 production customers and 2 POCs across on-prem edge and hybrid edge/cloud, with containerized ASR/NLU microservices and gRPC/WebRTC streaming
- Built a RAG-based post-sales QA helper over ~75 internal runbooks and a three-agent ticket-routing layer, cutting misrouted tickets and cross-team handoffs
AI/ML Research Assistant, Amal Lab, Northeastern University (June 2024 – August 2025)
- Engineered Med-SAM medical image segmentation for multi-modal datasets (MRI, CT, histopathology) with fine-tuned transformer architectures
- Processed 18TB+ TCGA multi-omics data on Spark and Dask for biomarker discovery across 33+ cancer types
- Trained a production skin-cancer classifier to 97.28% accuracy with ensembles and external validation across 3 datasets
Research Data Scientist, Brigham and Women's Hospital (August 2024 – December 2024)
- Built transformer-based NLP analytics (BERT, RoBERTa) over 10,000+ pharmaceutical reports linking media sentiment to regulatory outcomes
- Designed a HIPAA-compliant meta-analysis framework across ClinicalTrials.gov and PubMed with statistical modeling
---
Education
Northeastern University, Boston, MA
Master of Science in Information Systems (August 2025)
*Focused Coursework:* Advanced Data Science & Architecture, Parallel Machine Learning & AI, LLM with Knowledge Graph Databases, Natural Language Engineering, AI Generative Modeling with focus in Finance
Veermata Jijabai Technological Institute, Mumbai, India
Bachelor of Technology in Information Technology (June 2023)
---
Technical Skills
Programming: Python, SQL, C++, TypeScript, Java, R, JavaScript, MATLAB, Cypher
LLM & Agentic AI: Claude API (tool calling, MCP), GPT-4o, LangChain, RAG, FAISS, ChromaDB, Sentence Transformers, CLIP, Prompt Engineering, LLM-as-a-Judge Evaluation, Agent Orchestration, Evals & Guardrails
Voice & Real-Time Systems: ASR/NLU Integration, Voice Activity Detection, gRPC/WebRTC Streaming, Real-Time Audio Chunking, Session State Machines, On-Prem/Edge/Hybrid Deployment
Machine Learning & Deep Learning: PyTorch, TensorFlow, Keras, Scikit-learn, XGBoost, LightGBM, CNN, Vision Transformers, GANs, Reinforcement Learning (PPO, DQN), Distributed Data Parallel, Multi-GPU Training, Optuna
Data Engineering: Apache Spark, PySpark, Kafka, Airflow, dbt, Dask, Stream Processing, ETL/ELT, Great Expectations, Medallion Architecture
Databases & Cloud: PostgreSQL, MySQL, MongoDB, Neo4j, Snowflake, BigQuery, Redis, Supabase, AWS (EC2, S3, Lambda), GCP (BigQuery, GCS, Pub/Sub)
MLOps & Production: Docker, Kubernetes, MLflow, CI/CD, GitHub Actions, Model Monitoring, Prometheus, Grafana, Terraform, FastAPI, gRPC, Package Publishing (PyPI, npm)
Statistics & Experimentation: A/B Testing, Bayesian Inference, Sequential Testing, Multi-Armed Bandits, Causal Inference, Time Series (ARIMA, GARCH), Monte Carlo
Quantitative Finance: Options Pricing (Black-Scholes, Heston, Monte Carlo), Portfolio Optimization, Vectorized Backtesting, Statistical Arbitrage, VaR/Expected Shortfall
---
Publications
- Analysis of Explainable AI Methods on Medical Image Classification - IEEE ICAECT 2023
- Adversarial Attacks and Defenses for Skin Cancer Classification - IEEE ICONAT 2023
- Intrusion Detection: A Deep Learning Approach - IEEE ICEEICT 2023
- Image Captioning Using Transformer: VisionAid - IRJET 2022
What Makes Me Different
I don't just build models, I build the entire system around them: streaming ingestion, serving, evals, monitoring, and the deployment that makes it real. I've published research AND shipped production code that paying customers run every day. I understand both the math and the engineering.
---
Contact
- Email: [email protected]
- Portfolio: https://vinayjogani14.github.io/
- LinkedIn: linkedin.com/in/vinayjogani
- X: @jogani_vinay
- Google Scholar: View Profile
- Scopus ID: 58030923600
- ORCiD: 0009-0005-9568-5747
-
spareloop ★ PINNED
Stop wasting your AI coding CLI usage windows. Queue tasks for spare capacity + prewarm your 5-hour window so resets land when you need them. Claude Code, Codex CLI, Cursor.
TypeScript ★ 6 6d agoExplain → -
tracewall ★ PINNED
Flight recorder and firewall for AI agents — records every tool call, tracks lethal-trifecta taint across a session, and blocks the prompt-injection exfiltration path. Zero deps, cross-harness.
Python ★ 5 6d agoExplain → -
Healthcare-Data-Pipeline ★ PINNED
Enterprise healthcare data pipeline: real-time streaming (Kafka+Spark) + batch ETL processing 10M+ patient records. Airflow orchestration, dbt transformations, Great Expectations validation. SCD Type 2, Terraform IaC (Snowflake/GCP), Prometheus/Grafana monitoring. Production-grade with full testing & documentation.
Python ★ 1 9mo agoExplain → -
Skin-Cancer-Classification-Using-High-Parallel-Machine-Learning ★ PINNED
High-performance skin cancer classification using EfficientNet-B3 achieving 91% accuracy across 35 conditions on 245K images. Implemented multi-GPU parallelism with 4× NVIDIA A100s for 3.3× training speedup and 84% parallel efficiency optimization.
Jupyter Notebook ★ 1 1y agoExplain → -
NBA-Injury-Prediction-System ★ PINNED
NBA player injury prediction system with 78% AUC-ROC. Complete MLOps pipeline: 4.5K+ games processed with 55+ engineered features, FastAPI serving <100ms predictions, Redis caching, MLflow experiment tracking, Prometheus/Grafana monitoring. Production-ready with Docker/K8s deployment, automated testing (90%+ coverage), and real-time API endpoints.
Jupyter Notebook ★ 1 9mo agoExplain → -
bank-recon-agent ★ PINNED
Production-grade bank reconciliation agent with invariants, retries, traces, replays, and evals
Python ★ 1 1mo agoExplain → -
Smart-Order-Routing-and-Trading-Algorithms-using-Agile-Methodology
Quantitative trading simulator with smart order routing (SOR) and execution algorithms (VWAP, TWAP, POV). Features modular Streamlit backtesting framework analyzing execution costs and market microstructure with real-time Yahoo Finance data integration.
Python ★ 3 1y agoExplain → -
Supply-Chain-Management-using-Knowledge-Graphs-and-LLM
Interactive supply chain dashboard using Neo4j knowledge graphs and Streamlit for real-time analysis. Features natural language querying with LLMs translating to Cypher queries, enabling conversational analytics for supplier traceability and shipment management.
Python ★ 3 1y agoExplain → -
Deep-Reinforcement-Learning-Pairs-Trading
Implemented a Deep Q-Network (DQN) based reinforcement learning system for automated pairs trading. Built a custom Gym-like environment with real-time feature engineering on stock spreads, volatility, and rolling means, enabling dynamic decision-making and optimized portfolio performance in varying market conditions.
Jupyter Notebook ★ 3 1y agoExplain → -
transcript-edit-intelligence
AI-assisted video edit detector — NLP + FAISS RAG CLI mirroring Descript Underlord
Python ★ 2 1mo agoExplain → -
Copy-Move-Forgery-Detection
Built a CNN-based image forgery detection system using CASIA 2.0 dataset, achieving 96.9% accuracy. Deployed model on AWS EC2 for accessibility, with SVM for classification and a user-friendly GUI web app that allows users to test images for copy-move manipulation detection in real time.
JavaScript ★ 2 1y agoExplain → -
VinayJogani14.github.io
No description.
HTML ★ 1 3d agoExplain → -
VinayJogani14
My personal repository
★ 1 3d agoExplain → -
vllm ⑂
A high-throughput and memory-efficient inference and serving engine for LLMs
Python ★ 1 1mo agoExplain → -
semantic-dedup-poc-datologyai
Semantic deduplication POC for DatologyAI Solutions Engineer assignment - CLIP embeddings for e-commerce product catalog curation
HTML ★ 1 5mo agoExplain → -
snowconvert-ai-challenge
snowconvert-ai-challenge
TSQL ★ 1 3mo agoExplain → -
fireworks-rag-agent
Fireworks Assignment: Vinay Jogani
Python ★ 1 3mo agoExplain → -
whichmodel
Describe your task, get one AI model recommendation, backed by benchmarks and real developer sentiment.
Python ★ 1 6d agoExplain → -
Nifty-Options-Backtest
Qode Assessment 2
Python ★ 1 3mo agoExplain → -
BankNifty-Strangle-Backtest
Qode_Assesment
Python ★ 1 3mo agoExplain → -
Sports-Betting-AB-Testing
A/B testing platform with frequentist (t-test, chi-square), Bayesian, and sequential testing (O'Brien-Fleming). ML models: Random Forest for CLV prediction and churn classification. SQL analytics: window functions, CTEs, cohort retention, RFM segmentation. PostgreSQL (100K users, 1M bets). Stack: Python, scikit-learn, pandas, statsmodels, scipy.
Jupyter Notebook ★ 1 9mo agoExplain → -
KellyBet-Analytics-Platform
A machine learning-powered sports analytics platform that optimizes betting strategies using the Kelly Criterion across multiple sports. Integrates real-time odds data with custom ML models to provide data-driven bankroll management and predictive insights, achieving 87% prediction accuracy
Python ★ 1 10mo agoExplain → -
agents ⑂
A powerful framework for building realtime voice AI agents 🤖🎙️📹
★ 1 5mo agoExplain → -
cookbook ⑂
Recipes and resources for building, deploying, and fine-tuning generative AI with Fireworks.
Jupyter Notebook ★ 1 8mo agoExplain → -
trial ⑂
No description.
★ 1 7mo agoExplain → -
Financial-Dasboard
Full stack financial analytics system: PostgreSQL database with normalized tables processing 1,750+ transactions. Built 12 advanced SQL queries (CTEs, window functions, recursive queries). 5-page interactive Tableau dashboard. Python ETL pipeline with pandas. Automated reporting. Demonstrates database design, statistical analysis & BI development.
Python ★ 1 9mo agoExplain → -
Personal-Finance-API
Production-ready microservices API for personal finance management with expense tracking, budgets, and analytics. Built with FastAPI, PostgreSQL, and Redis. Features 4 independent services, 90%+ test coverage, CI/CD with GitHub Actions, Docker/Kubernetes deployment, rate limiting, and Redis caching. Demonstrates enterprise software engineering.
Python ★ 1 9mo agoExplain → -
CodeBuddy-A-Natural-Language-Code-Explanation-Generator
AI-powered code explanation system using fine-tuned CodeT5 with Retrieval-Augmented Generation. Generates natural language explanations for Python code with 24% BLEU improvement over GPT-4, interactive Q&A capabilities, and sub-500ms response times on 2M+ training examples.
Python ★ 1 10mo agoExplain → -
trading-signals
Smart trading assistant powered by Roboquant and Streamlit. Analyze stocks with 4 technical indicators (RSI, MACD, SMA, Bollinger Bands), get buy/sell signals, price targets, and risk management suggestions. Real-time data from Yahoo Finance with interactive charts.
Python ★ 1 10mo agoExplain → -
SorosAI-Intelligent-Biography-and-Pairs-Trading-System
Developed an AI-powered system combining a biography knowledge assistant with an intelligent pairs trading bot. The solution parses unstructured text for contextual Q&A and applies machine learning with statistical arbitrage for automated trade decision-making
Python ★ 1 11mo agoExplain → -
swimlane-chatbot
Domain-specific chatbot parsing Swimlane diagrams and OpenAPI specs into structured event sequences. Features transformer-based semantic retrieval with FAISS vector store and modular production pipeline for enterprise workflow automation.
Python ★ 1 1y agoExplain → -
Jogani_Vinay_002839145_labs
No description.
Java ★ 1 2y agoExplain → -
Volatility-Surface-Modelling
Volatility surface modeling with Black-Scholes and Heston calibration using SciPy/CVXPY optimization. Built arbitrage-detecting trading strategies with improved Sharpe ratios, comprehensive VaR/ES risk analysis, and real-time Bloomberg/yFinance data integration.
Python ★ 1 1y agoExplain → -
Reinforcement-Learning-for-Portfolio-Management
Custom Gymnasium RL environment for portfolio management across 5 synthetic assets with realistic trading costs and volatility. PPO agent trained with Stable-Baselines3 achieved 23% return improvement over baselines while maintaining 90% diversification consistency.
Jupyter Notebook ★ 1 1y agoExplain → -
Options-pricing-model
Advanced options pricing models implementing Black-Scholes, Binomial Tree, and Monte Carlo methods. Features real-time data analysis via yfinance API, multi-method comparison tools, and optimized computation with NumPy/SciPy for scalable financial modeling.
Jupyter Notebook ★ 1 1y agoExplain → -
Deep-Reinforcement-Learning-Algorithm
Deep reinforcement learning trading algorithm using OpenAI Gym environment with financial time-series data and technical indicators. Built with stable_baselines3, featuring Bayesian hyperparameter optimization for high-frequency trading decisions and performance visualization on historical market data.
Jupyter Notebook ★ 1 1y agoExplain → -
Algorithmic-Insider-Trading-Detector
Developed a pipeline to detect potential insider trading by integrating SEC Form 4 filings with stock market data via yfinance. Applied ETL processes, engineered financial features (e.g., moving averages, PVT, buy-sell ratios), and trained ML models (Random Forest, SVM) to flag anomalous trading patterns.
Jupyter Notebook ★ 1 1y agoExplain → -
Deep-Fake-Detection-GAN-s
Built a DeepFake detection system using GAN-based models to classify manipulated content via perceptual cues. Achieved 91% accuracy with a plain-frames model, outperforming MRI-GAN SSIM at 74%. Explored loss functions, hyper-parameter tuning, and perceptual metrics to improve robustness against synthetic media.
Jupyter Notebook ★ 1 1y agoExplain → -
Mean-Variance-Optimization-with-ESG-scores
Developed an ESG-integrated mean-variance optimizer using PyPortfolioOpt on S&P 500 stocks. Applied covariance shrinkage and CAPM-based returns to generate efficient frontiers under ESG thresholds, reducing portfolio volatility by 22% and improving Sharpe ratio by 18%, proving the value of sustainable investing.
Jupyter Notebook ★ 1 1y agoExplain → -
Trading-with-Technical-and-Sentimental-Analysis
Developed an end-to-end stock analysis and recommendation system combining technical indicators with sentiment analysis from Reddit and Twitter. Automated data pipeline integrated Yahoo Finance API, NASDAQ data via FTP, and social media insights to rank top 5 daily stock picks.
Jupyter Notebook ★ 1 1y agoExplain → -
Data_Science_Projects
A collection of applied data science projects covering causal inference, exploratory data analysis, and visualization. Includes studies on air pollution impact, product recommendations, FIFA player analytics, Titanic survival insights, and tennis performance metrics.
Jupyter Notebook ★ 1 1y agoExplain →
No repos match these filters.
More creators on gitmyhub
standardgalactic AlexTheAnalyst MorvanZhou cloudwu kenjinote