19-day current streak·60-day longest streak
oh hey! 👋 I'm James, an engineer / data scientist from Chicago. My time on GitHub is mostly spent writing Python, R, C++, and shell scripts on projects for data…
oh hey! 👋
I'm James, an engineer / data scientist from Chicago. My time on GitHub is mostly spent writing Python, R, C++, and shell scripts on projects for data scientists and data engineers. My time off GitHub is spent with family, at hip hop shows, and watching reality TV.
:shipit: open source stuff I'm maintaining
- LightGBM: a lightweight gradient boosting machine
- lightgbm-dask-testing: containerized setup for testing LightGBM's Dask interface locally and on Amazon ECS
- pkgnet: R package for analyzing an R package's dependencies
- pydistcheck: linter that finds portability issues in Python package distributions (wheels, sdists, and conda packages)
- uptasticsearch: an R data frame client for Elasticsearch
✋ some other open source stuff I've contributed on in the past
- hamilton: a "micro-framework" for feature engineering in Python
- prefect: a workflow management thing in Python that plays nicely with Dask
- xgboost: another gradient boosting machine
😊 open source contributions I'm proud of
click for details
The pull requests and none-code contributions below were chosen to showcase the types of software work I've done. This list is not exhaustive.
Features
- adapting
lightgbmandxgboosttoscikit-learn1.6:
- setting up
condapackages forlegate-boost,legate-dataframe, andlegate-raft: https://github.com/rapidsai/legate-boost/issues/115 - replacing LightGBM's
setup.pywithscikit-build-corefor PEP 517/518 compatibility: https://github.com/microsoft/LightGBM/pull/5759 - upstreaming
dask-lightgbminto LightGBM and guiding community discussion with Dask, XGBoost maintainers
- adding
Webhookstorage toprefect: https://github.com/PrefectHQ/prefect/pull/3000 - adding
autoconf-based builds of LightGBM's R package: https://github.com/microsoft/LightGBM/pull/3188 - making
snowflake-connector-pythoncompatible withpyjwt1.x and 2.x: https://github.com/snowflakedb/snowflake-connector-python/pull/604 - allow tight control over ports in LightGBM distributed traiining with Dask: https://github.com/microsoft/LightGBM/pull/3994
- cut compiled size of
{lightgbm}by ignoring CLI-only objects: https://github.com/microsoft/LightGBM/pull/3566 - allow use of multiple image pull secrets in
prefectkubernetes agent: https://github.com/PrefectHQ/prefect/pull/3596 - replace single-shot HTTP requests with
httr::RETRY()in various R packages
{sergeant} (one example): https://github.com/hrbrmstr/sergeant/pull/42
Docs
- tutorial on distributed LightGBM training with Dask: https://github.com/microsoft/LightGBM/pull/4030
- early stopping example in XGBoost Dask docs: https://github.com/dmlc/xgboost/pull/6501
- detailed information on how LightGBM parameters affect training speed: https://github.com/microsoft/LightGBM/pull/3628
- guide on how to find valid memory and CPU combinations for ECS / Fargate clusters in
dask-cloudprovider: https://github.com/dask/dask-cloudprovider/pull/156
Bug Fixes
- diagnosing and fixing a
scikit-build-core+ CMake build issue whose root cause was "CPython bakes in information about which compiler it was built with, and now that's an absolute path"
- fixing
dtypesetting and tests acrosspredict()calls inlightgbm, required deep investigation (https://github.com/microsoft/LightGBM/pull/7140#discussion_r2725023963) - fixing OpenMP conflicts in
lightgbm:
- detecting debug symbols in
pandas2.0 wheels: https://github.com/pandas-dev/pandas/issues/51900 - prevent
condafrom "downgrading" Python from CPython to PyPy, while also reducing the risk of a subtle networking error made worse by unpredictability in when Dask garbage collects objects (https://github.com/microsoft/LightGBM/pull/5510) - create a reproducible example for
lightgbmloading failing withGLIBCXXcompatibility errors: https://github.com/microsoft/LightGBM/issues/5106#issuecomment-1121925896 - fix
jupyter_serverconda-forge feedstock recipe to prevent broken environments: https://github.com/conda-forge/jupyter_server-feedstock/pull/84 - make multioutput behavior of
dask-mlregression metrics consistent withscikit-learn: https://github.com/dask/dask-ml/pull/820 - fix saving Dask Random Forest models in
cuml: https://github.com/rapidsai/cuml/pull/3388 - fix checks for availability of
mm_mallocin{lightgbm}autoconf-based builds: https://github.com/microsoft/LightGBM/pull/3510 - fix broken plots in
{lightgbm}'s docs site: https://github.com/microsoft/LightGBM/pull/3508 - factor out dependency on
gendef.exefor compiling XGBoost and LightGBM R packages with Visual Studio compilers and R 4.0:
{xgboost}: https://github.com/dmlc/xgboost/pull/5764
- {lightgbm}: https://github.com/microsoft/LightGBM/pull/3065
Infrastructure / CI
- helping with various migrations for all of the RAPIDS libraries:
fmt / spdlog: https://github.com/rapidsai/build-planning/issues/56
- Dropping Python 3.9: https://github.com/rapidsai/build-planning/issues/88
- CUDA 12.5: https://github.com/rapidsai/build-planning/issues/73
- Adding Python 3.12: https://github.com/rapidsai/build-planning/issues/40
- Adding Python 3.11: https://github.com/rapidsai/build-planning/issues/3
- switching LightGBM's Python package jobs to
manylinux_2_28: https://github.com/microsoft/LightGBM/pull/5580 - automatically publish
prefect-saturnto PyPI when a new release is created: https://github.com/saturncloud/prefect-saturn/pull/7 - moving LightGBM CI jobs from Travis to GitHub Actions:
- move
{uptasticsearch}CI to GitHub Actions: https://github.com/uptake/uptasticsearch/pull/217 - add CI job testing
{lightgbm}within ASAN and UBSAN sanitizers: https://github.com/microsoft/LightGBM/pull/3439 - reduce data loading work in LightGBM tests by caching data loading calls: https://github.com/microsoft/LightGBM/pull/3486
- add Dockerfile to build an image for testing the Apache Arrow R package: https://github.com/apache/arrow/pull/2770
💰 things I do for money
- Sr. Software Engineer at NVIDIA, working on RAPIDS (https://github.com/rapidsai)
- adjunct instructor at Marquette University, where I teach "Intro to R Programming" (https://github.com/jameslamb/intro-to-r)
:computer: conference talks
I've given talks on Dask, LightGBM, R, Python packaging, and other random stuff. For a full list and links to videos, see https://github.com/jameslamb/talks#gallery.
:microphone: talk to me!
My DMs are open if you want to talk about open source, data science careers, *Bravo shows*, or anything else.
- :link: LinkedIn: https://www.linkedin.com/in/jameslamb1/
- 🦋 Bluesky: https://bsky.app/profile/jameslamb.bsky.social
-
LightGBM ★ PINNED ⑂
A fast, distributed, high performance gradient boosting (GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks. It is under the umbrella of the DMTK(http://github.com/microsoft/dmtk) project of Microsoft.
C++ ★ 3 2mo agoExplain → -
talks ★ PINNED
Conference talks, meetup talks, and misc. writing
CSS ★ 26 7d agoExplain → -
pydistcheck ★ PINNED
Linter that finds portability issues in Python package distributions (wheels, sdists, conda packages).
Python ★ 44 4d agoExplain → -
lightgbm-dask-testing ★ PINNED
Test LightGBM's Dask integration on different cluster types
Jupyter Notebook ★ 12 7d agoExplain → -
pkgnet ★ PINNED ⑂
R package for analyzing other R packages via graph representations of their dependencies
★ 0 5y agoExplain → -
uptasticsearch ★ PINNED ⑂
An Elasticsearch client tailored to data science workflows.
R ★ 0 5mo agoExplain → -
rsixygen ▣
A (prototype) project for creating Roxygen documentation for R6 objects
R ★ 10 6y agoExplain → -
intro-to-r
Materials for my "Intro to R Programming" course
HTML ★ 9 2d agoExplain → -
doppel-cli ▣
Test framework for comparing the consistency of library APIs
Python ★ 8 2y agoExplain → -
conference-notes
No description.
★ 4 1y agoExplain → -
bigforecast ▣
Forecasting of globally important economic time series using lots of data and cool statistical stuff. Final project for "W251: Scaling Up!" in UC-Berkeley MIDS program
Jupyter Notebook ★ 4 9y agoExplain → -
ci-tools ▣
Miscellaneous scripts for use in continuous integration (CI) pipelines
Shell ★ 3 5y agoExplain → -
oss-report ▣
Make a report on one or more users' open source contributions
JavaScript ★ 3 6y agoExplain → -
dot-files
misc. configs and scripts
Shell ★ 2 4mo agoExplain → -
jameslamb
No description.
★ 1 23d agoExplain → -
lgb-glibc-demo
Code and notes from investigating https://github.com/microsoft/LightGBM/issues/5106
Dockerfile ★ 1 4y agoExplain → -
py_dep_summary ▣
Check licenses of your dependencies and get a lil CSV of the information
Shell ★ 1 7y agoExplain → -
prefect-cloud-repro ▣
Notes and code used to create reproducible examples when reporting Prefect Cloud issues
Python ★ 1 5y agoExplain → -
webhook-storage-integration-tests ▣
burner repo to hold integration tests for a prefect PR
Python ★ 1 6y agoExplain → -
ci-imgs ⑂
No description.
Dockerfile ★ 0 1d agoExplain → -
cuml ⑂
cuML - RAPIDS Machine Learning Library
Python ★ 0 1d agoExplain → -
cuopt ⑂
GPU accelerated decision optimization
Cuda ★ 0 1d agoExplain → -
cuvs ⑂
cuVS - a library for vector search and clustering on the GPU
Cuda ★ 0 1d agoExplain → -
cudf ⑂
cuDF - GPU DataFrame Library
C++ ★ 0 1d agoExplain → -
velox ⑂
A composable and fully extensible C++ execution engine library for data management systems.
★ 0 8d agoExplain → -
sccache ⑂
Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
★ 0 7d agoExplain → -
raft ⑂
RAFT contains fundamental widely-used algorithms and primitives for machine learning and information retrieval. The algorithms are CUDA-accelerated and form building blocks for more easily writing high performance applications.
Cuda ★ 0 1d agoExplain → -
cuvs-lucene ⑂
A Lucene codec for vector search and clustering on the GPU
★ 0 1mo agoExplain → -
kvikio ⑂
KvikIO - High Performance File IO
C++ ★ 0 1d agoExplain → -
jupyterlab-nvdashboard-feedstock ⑂
A conda-smithy repository for jupyterlab-nvdashboard.
★ 0 13d agoExplain → -
jupyterlab-nvdashboard ⑂
A JupyterLab extension for displaying dashboards of GPU usage.
TypeScript ★ 0 15d agoExplain → -
integration ⑂
RAPIDS - combined conda package & integration tests for all of RAPIDS libraries
Shell ★ 0 28d agoExplain → -
cugraph-docs ⑂
cuGraph Docs - RAPIDS Graph Analytics Documentation
Shell ★ 0 28d agoExplain → -
pytorch ⑂
Tensors and Dynamic neural networks in Python with strong GPU acceleration
★ 0 25d agoExplain → -
rapidsmpf ⑂
No description.
C++ ★ 0 1d agoExplain → -
cugraph-gnn ⑂
No description.
Cuda ★ 0 1d agoExplain → -
pre-commit ⑂
A framework for managing and maintaining multi-language pre-commit hooks.
★ 0 1mo agoExplain → -
workflows ⑂
No description.
★ 0 1mo agoExplain → -
sphinx-theme ⑂
This repository contains a Sphinx theme used for RAPIDS documentation
★ 0 1mo agoExplain → -
spdx-license-builder ⑂
No description.
★ 0 1mo agoExplain → -
rvc ⑂
No description.
★ 0 1mo agoExplain → -
rapids-triton ⑂
No description.
★ 0 1mo agoExplain → -
rapids-examples ⑂
No description.
★ 0 1mo agoExplain → -
node ⑂
GPU-accelerated data science and visualization in node
★ 0 1mo agoExplain → -
multi-gpu-tools ⑂
No description.
★ 0 1mo agoExplain → -
legate-boost ⑂
GBM implementation on Legate
★ 0 1mo agoExplain → -
gqe ⑂
No description.
★ 0 1mo agoExplain → -
gpu-bdb ⑂
RAPIDS GPU-BDB
★ 0 1mo agoExplain → -
gha-tools ⑂
User tools for RAPIDS GitHub Actions
★ 0 13d agoExplain → -
detect-weak-linking ⑂
No description.
★ 0 1mo agoExplain → -
dask-upstream-testing ⑂
No description.
★ 0 1mo agoExplain → -
build-planning ⑂
Tracking for RAPIDS-wide build tasks
★ 0 1mo agoExplain → -
shared-workflows ⑂
Reusable GitHub Actions workflows for RAPIDS CI
★ 0 8d agoExplain → -
shared-actions ⑂
No description.
★ 0 1mo agoExplain → -
sagemaker-feedstock ⑂
A conda-smithy repository for sagemaker.
★ 0 3mo agoExplain → -
winget-pkgs ⑂
The Microsoft community Windows Package Manager manifest repository
★ 0 3mo agoExplain → -
nixpkgs ⑂
Nix Packages collection & NixOS
★ 0 3mo agoExplain → -
macports-ports ⑂
The MacPorts ports tree
★ 0 3mo agoExplain → -
catboost ⑂
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
★ 0 3mo agoExplain → -
homebrew-core ⑂
🍻 Default formulae for the missing package manager for macOS (or Linux)
★ 0 3mo agoExplain → -
optuna ⑂
A hyperparameter optimization framework
★ 0 3mo agoExplain → -
shap ⑂
A game theoretic approach to explain the output of any machine learning model.
★ 0 3mo agoExplain → -
SynapseML ⑂
Simple and Distributed Machine Learning
★ 0 3mo agoExplain → -
parsnip ⑂
A tidy unified interface to models
★ 0 3mo agoExplain → -
velox-testing ⑂
No description.
★ 0 1mo agoExplain → -
ucxx ⑂
No description.
C++ ★ 0 1d agoExplain → -
rmm ⑂
RAPIDS Memory Manager
C++ ★ 0 1d agoExplain → -
nx-cugraph ⑂
GPU Accelerated Backend for NetworkX
Jupyter Notebook ★ 0 1mo agoExplain → -
nvforest ⑂
Highly optimized and lightweight library for decision tree inference on NVIDIA GPUs and CPUs
C++ ★ 0 1d agoExplain → -
cuxfilter ⑂
GPU accelerated cross filtering with cuDF.
Python ★ 0 16d agoExplain → -
dask-cuda ⑂
Utilities for Dask and CUDA interactions
Python ★ 0 2d agoExplain → -
scikit-learn ⑂
scikit-learn: machine learning in Python
★ 0 4mo agoExplain → -
python-organizer-conferences ⑂
List of Python Conferences around the World
★ 0 4mo agoExplain → -
fvdb-core-feedstock ⑂
A conda-smithy repository for fvdb-core.
★ 0 4mo agoExplain → -
conference-data ⑂
Conference data for www.confs.tech
★ 0 4mo agoExplain → -
rapids.ai ⑂
rapids.ai web site
★ 0 1mo agoExplain → -
rapids-dask-dependency ⑂
No description.
Python ★ 0 1mo agoExplain → -
rapids-cmake ⑂
No description.
CMake ★ 0 1mo agoExplain → -
docker ⑂
Dockerfile templates for creating RAPIDS Docker Images
Shell ★ 0 29d agoExplain → -
devcontainers ⑂
No description.
Shell ★ 0 1mo agoExplain → -
cugraph ⑂
cuGraph - RAPIDS Graph Analytics Library
Cuda ★ 0 1d agoExplain → -
cucim ⑂
cuCIM - RAPIDS GPU-accelerated image processing library
Jupyter Notebook ★ 0 1d agoExplain → -
deployment ⑂
RAPIDS Deployment Documentation
★ 0 29d agoExplain → -
wheel-stub ⑂
wheel stub package build backend
★ 0 5mo agoExplain → -
dependency-file-generator ⑂
No description.
★ 0 1mo agoExplain → -
open-source-events ⑂
A list of Open Source conferences or events to look out for.
★ 0 5mo agoExplain → -
conference.scipy.org ⑂
Landing page and blog for conference.scipy.org
★ 0 5mo agoExplain → -
scipy-conference ⑂
SciPy Conference Materials
★ 0 6mo agoExplain → -
pre-commit-hooks ⑂
No description.
Python ★ 0 1mo agoExplain → -
pip ⑂
The Python package installer
★ 0 5mo agoExplain → -
legate-dataframe ⑂
Distributed cuDF on Legate
★ 0 1mo agoExplain → -
gputreeshap ⑂
No description.
★ 0 1mo agoExplain → -
rapids-xgboost-feedstock ⑂
A conda-smithy repository for xgboost.
★ 0 5mo agoExplain → -
distributed ⑂
A distributed task scheduler for Dask
★ 0 1mo agoExplain → -
rchk-containers ⑂
Docker containers for R-hub
★ 0 6mo agoExplain → -
pydistcheck-feedstock ⑂
A conda-smithy repository for pydistcheck.
★ 0 5mo agoExplain → -
rapids-logger ⑂
A logging interface for RAPIDS built on spdlog
★ 0 1mo agoExplain → -
rapids-metadata ⑂
Metadata about RAPIDS libraries
Python ★ 0 1d agoExplain → -
ucx-wheels ⑂
No description.
★ 0 13d agoExplain → -
rapids-cli ⑂
CLI utilities for RAPIDS
★ 0 1mo agoExplain → -
rapids-build-backend ⑂
RAPIDS-specific wrapper around PEP 517 build backends
★ 0 8d agoExplain → -
tempest-extremes-feedstock ⑂
A conda-smithy repository for tempest-extremes.
★ 0 6mo agoExplain → -
python-cudnn-frontend-feedstock ⑂
A conda-smithy repository for python-cudnn-frontend.
★ 0 6mo agoExplain → -
r-lightgbm-feedstock ⑂
A conda-smithy repository for r-lightgbm.
★ 0 6mo agoExplain → -
lightgbm-feedstock ⑂
A conda-smithy repository for lightgbm.
★ 0 3mo agoExplain → -
NVTX ⑂
The NVIDIA® Tools Extension SDK (NVTX) is a C-based Application Programming Interface (API) for annotating events, code ranges, and resources in your applications.
★ 0 6mo agoExplain → -
docs ⑂
RAPIDS Documentation Site
★ 0 1mo agoExplain → -
python-package-guide ⑂
Python packaging made simple. Recommendations & guidance curated by the pyOpenSci community
★ 0 11mo agoExplain → -
groundhog ▣
Service and client libraries for interacting with SRTM elevation data
Python ★ 0 5y agoExplain → -
MSAE_Alumni_2015 ▣
This repo holds files relating to a presentation given at the Marquette University MSAE Alumni Event, held in Milwaukee, WI, 18 April 2015.
CSS ★ 0 6y agoExplain → -
ML4EVIEWS ▣
A repo to make some common tools and algorithms from data science pipelines available as EViews add-ins.
xBase ★ 0 1y agoExplain → -
r-uptasticsearch-feedstock ⑂
A conda-smithy repository for r-uptasticsearch.
★ 0 6mo agoExplain → -
xgboost ⑂
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow
C++ ★ 0 3mo agoExplain → -
skills ⑂ ▣
tool for making skill recommendations
Jupyter Notebook ★ 0 8y agoExplain → -
chessviz ▣
W209 Final Project: Chess Viz
Jupyter Notebook ★ 0 9y agoExplain → -
two_sigma_connect ⑂ ▣
No description.
Jupyter Notebook ★ 0 9y agoExplain → -
FREDHEAD ▣
FREDHEAD: A Simple Dashboard for St. Louis FED Data
CSS ★ 0 11y agoExplain → -
add_static_search_to_pkgdown ▣
WIP - experimental project aiming to add static search to a pkgdown site for an R package
JavaScript ★ 0 7y agoExplain → -
MSAE_Alumni_2016 ▣
Repository with files for a presentation entitled "Economics as A Science", to be delivered at Marquette University in April 2016.
CSS ★ 0 6y agoExplain → -
oddish ▣
trying to learn how to use R and c++ without Rcpp
C++ ★ 0 6y agoExplain → -
bedazzle ▣
Experimenting with Bazel to build a C library shared between R and Python packages
Python ★ 0 6y agoExplain → -
crangraph ▣
An application for analyzing and visualizing the dependencies between R packages on CRAN.
Jupyter Notebook ★ 0 6y agoExplain →
No repos match these filters.