BigScience Workshop ORG

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

Petals lets you run massive AI language models like Llama 3.1 (405B) on consumer hardware by splitting the model across multiple volunteer computers over the internet, like BitTorrent for AI inference.

Toolkit for creating, sharing and using natural language prompts.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

Crosslingual Generalization through Multitask Finetuning

Tools for curating biomedical training data for large-scale language modeling

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Code used for sourcing and cleaning the BigScience ROOTS corpus

A framework for few-shot evaluation of autoregressive language models.

No description.

Tools for managing datasets for governance and training.

Libraries, Archives and Museums (LAM)

BLOOM+1: Adapting BLOOM model to support a new unseen language

Code and Data for Evaluation WG

This directory gathers the tools developed by the Data Sourcing Working Group

Experiments on including metadata such as URLs, timestamps, website descriptions and HTML tags during pretraining.

No description.

A repository for `codecarbon` logs.

No description.

Alternative to https://github.com/Dynalon/mdwiki-seed

Evaluation for Shades of Bias in Text

A repo for running model shrinking experiments

Framework for BLOOM probing

PII Processing code to detect and remediate PII in BigScience datasets. Reference implementation for the PII Hackathon

Scripts to prepare catalogue data

BigScience working group on language models for historical texts

🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

No description.

Managing your machine learning lifecycle with MLflow and Amazon SageMaker

A list of BigScience publications

No description.

scaling-laws-tokenization

Tools for evaluating model robustness and consistency

Generate statistics over datasets used in the context of BS

No description.

Track emissions from Compute and recommend ways to reduce their impact on the environment.