McGill NLP ORG

Code for 'LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders'

Llama-3 agents that can browse the web by following instructions and talking to you

Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"

Code for paper "The Markovian Thinker: Architecture-Agnostic Linear Scaling of Reasoning"

Code for the paper "VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment"

WebLINX is a benchmark for building web navigation agents with conversational capabilities

Large medical text dataset curated for abbreviation disambiguation, designed for natural language understanding pre-training in the medical domain

ACL 2022: An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models.

Code for the paper "The Impact of Positional Encoding on Length Generalization in Transformers", NeurIPS 2023

Code and Data for "Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering"

Code for `LLM2VEC-GEN: Generative Embeddings from Large Language Models`

Code and data for reproducing baselines for TopiOCQA, an open-domain conversational question-answering dataset

No description.

No description.

AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories

Code and data for the paper "LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs"

Code and data for ImageCoDe, a contextual vison-and-language benchmark

Code and data for the paper: Learning Action and Reasoning-Centric Image Editing from Videos and Simulation

Code and data setup for the paper "Are Diffusion Models Vision-and-language Reasoners?"

Code for "Can Retriever-Augmented Language Models Reason? The Blame Game Between the Retriever and the Language Model", EMNLP Findings 2023

SafeArena is a benchmark for assessing the harmful capabilities of web agents

Large Scale Benchmark of Large Language Models on African Languages

TACL 2025: Investigating Adversarial Trigger Transfer in Large Language Models

No description.

Code for the paper "Modelling Latent Translations for Cross-Lingual Transfer"

Synthetic Data Generation for Evaluation

No description.

FeedbackQA: Improving Question Answering Post-Deployment with Interactive Feedback

Forecasting Downstream Performance of LLMs With Proxy Metrics

NAACL 2024: Evaluating In-Context Learning of Libraries for Code Generation

The StatCan Dialogue Dataset: Retrieving Data Tables through Conversations with Genuine Intents

Code for `Exploiting Instruction-Following Retrievers for Malicious Information Retrieval`

No description.

Agent-as-Annotators: Structured Distillation of Web Agent Capabilities

No description.

BRIDGE: Predicting Human Task Completion Time From Model Performance

Code and data for the paper 'Scope Ambiguities in Large Language Models'.

EMNLP 2023: MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretations

A repository for code related to a project mapping Universal Dependencies syntactic forms to meaning representations.

Repo for "Language Models Largely Exhibit Human-like Constituent Ordering Preferences"

Experiments and data for the paper "When and why vision-language models behave like bags-of-words, and what to do about it?" Oral @ ICLR 2023

No description.

No description.

No description.

Code for safety evaluations of StarCoder.

No description.

No description.

Data and code for the paper "Humans and LLMs Diverge on Probabilistic Inferences"

No description.

Agents for science platform

No description.

🤗 Diffusers: State-of-the-art diffusion models for image and audio generation in PyTorch

Template for creating project webpages based on jekyll/minimal-mistakes

No description.

No description.

No description.

[ECCV 2026] Official repository for "CRAG-MM-Diagnostics: Enabling Stage-Wise Analysis of Knowledge-Intensive VQA"

No description.

No description.

Workshop on AI for Science

No description.

No description.

No description.

No description.

No description.

A fork of JailbreakBench: An Open Robustness Benchmark for Jailbreaking Language Models [NeurIPS 2024 Datasets and Benchmarks Track]

No description.

Evaluation dataset for our NAACL 2025 paper on "Does Generative AI speak Nigerian-Pidgin?: Issues about Representativeness and Bias for Multilingualism in LLMs"

No description.

A multicultural, open-source benchmark dataset for 16 African languages with utterances generated by native speakers across diverse domains.

No description.

Template for creating project docs with mkdocs-material