Martin Marek

LLM pretraining in pure JAX

pretrained LLMs in pure JAX

📄Small Batch Size Training for Language Models

📄 Forgetting in Language Models: Capacity, Optimization, and Self-Generated Replay

Research code for a paper studying how AI language models forget old knowledge when trained on new data, and how mixing in synthetic self-generated examples can prevent most of that forgetting.

📸Night mode on any camera. Based on HDR+.

A PyTorch implementation of HDR+ with GPU support.

📄Image-Based Parking Space Occupancy Classification: Dataset and Baseline

A simple implementation of Hamiltonian Monte Carlo in JAX.

A pure Python utility for maintaining preemptible TPUs

📄Can a Confident Prior Replace a Cold Posterior?

Personal (unpolished) notes on various models / algorithms.

No description.

A machine learning compiler for GPUs, CPUs, and ML accelerators

No description.

Minimal yet performant LLM examples in pure JAX

(private research codebase)

Easily time your Python code

My bachelor's thesis on Bayesian Neural Networks (BNNs).

"On Functional Priors and Cold Posteriors in Bayesian Neural Networks"