Hendrik Makait Claim this profile

Parallel computing with task scheduling

A distributed task scheduler for Dask

No description.

Instrument Dask with OpenTelemetry

An Apache Iceberg integration for Dask

Slides and resources for my talk on "Observability for Distributed Computing with Dask"

Spark SQL demo for AIM-3 Scalable Data Science at TU Berlin

Prefect integrations with the Dask execution framework.

narwhals integrations for datafusion-python

Python-based implementation of a desktop-cleaning agent based on https://systems-engineering-for-agentic-applications.resonatehq.io/chapters/chapter-2

🍻 A CLI workflow for the administration of macOS applications distributed as binaries

Lightweight and extensible compatibility layer between dataframe libraries!

Data Structures and algorithms in Rust

lzarray is an experiment combining learnings from 50 years of database research with a Rust-based implementation to create a next-generation xarray-like

Professional Website

Extensible SQL Lexer and Parser for Rust

Apache Arrow DataFusion SQL Query Engine

No description.

Rust implementation of the Monkey programming language based on https://interpreterbook.com/

Implementation of linked lists based on https://rust-unofficial.github.io/too-many-lists/

Apache PyIceberg

Apache Iceberg

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

Profile the dask distributed scheduler with py-spy and viztracer

Implementation of a grep-like CLI tool based on chapter 12 of the Rust book

Open-source, cloud-native transactional tensor storage engine

Implementation of the Writing an OS in Rust series at os.phil-opp.com.

Userspace for the open-source QMK keyboard firmware.

A specification that python filesystems should adhere to.

A Dask-based ETL showcase based on the TPC-DI benchmark

A functional standard library for Python.

Sampling profiler for Python programs

Deploy Dask on job schedulers like PBS, SLURM, and SGE

Cloud provider cluster managers for Dask. Supports AWS, Google Cloud Azure and more...

N-D labeled arrays and datasets in Python

dask-pytorch-ddp is a Python package that makes it easy to train PyTorch models on dask clusters using distributed data parallel.

Scalable Machine Learning with Dask

Distributed SQL Engine in Python using Dask

Extended pickling support for Python objects

Sphinx theme for Dask documentation

Repository for tooling around Dask Maintenance

Serialization library for Exceptions and Tracebacks.

Workbench for performance optimizing Dask's P2P shuffling

ABlog for blogging with Sphinx

The governance process and model for Dask

Official Rust implementation of Apache Arrow

Fast multi-threaded, hybrid-out-of-core query engine focussing on DataFrame front-ends

A Delta Lake reader for Dask

Dask development blog

Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication. Languages currently supported include C, C++, Java, JavaScript, Python, and Ruby.

Github Action for automatically labelling PRs for a smooth review flow

No description.

Dask Tutorial by Coiled

Notebook for showcasing P2P shuffling/rechunking at Dask Demo Day 2023-03-16

Is the GIL seeing someone else? How's about repetitively calling and seeing how long it takes to answer?

Useful Mutable Mappings

Coming soon

The Apache Kafka C/C++ library

Scripts and notebooks for Dask Demo Day

Apache Iceberg

Collections of algorithms and data structures implemented in Python

The Python programming language

No description.

Dask tutorial

This repo contains a short version of a dask tutorial.

Apache Superset is a Data Visualization and Data Exploration Platform

Distributed compute platform implemented in Rust, powered by Apache Arrow.

:crab: Small exercises to get you used to reading and writing Rust code!

The easiest way to automate your data

Mirror of Apache Kafka

Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)

My notes for Prof. Klaus Obermayer's "Machine Intelligence 1 - Supervised Learning" course at the TU Berlin

Causal Analysis Pipeline for Air Pollution Data for BDAPRO SS 2019

No description.

First practical task for In-Memory Databases on Modern Hardware

No description.

An extension of Yahoo's Benchmarks

Student repository for the CppND OOP course project

No description.

Streamline Hackathon 2018

Next Generation Python Package Repository

A microframework based on Werkzeug, Jinja2 and good intentions

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more