Adrien Barbaresi

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Fast and robust date extraction from web pages, with Python or on the command-line

Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

Rust code and command-line utility to compute and visualize prime sequences

Clean, filter and sample URLs to optimize data collection – Python & command-line – Deduplication, spam, content and language filters

Curated list of open-access/open-source/off-the-shelf resources and tools developed with a particular focus on German

Faster, modernized fork of the language identification tool langid.py

integrates spatial and textual data processing tools into a modular software package which features preprocessing, geocoding, disambiguation and visualization

Extraction of a German Reddit Corpus

A collection of awesome web crawler,spider in different languages

List of libraries, tools and APIs for web scraping and data processing.

Filtering and Language-identification for URL Crawling Seeds (FLUCS) a.k.a. FLUX-Toolchain

Diverse tools used with Twitter data

A readability parser which can extract title, content, images from html pages

Software for humanities scholars using quantitative or computational methods.

NLTK Data

Material zum Aufbau eines deutschsprachigen COVID-19-Webkorpus / Building a corpus in German dedicated to coronavirus

Experiments to modernize the LaTeX class of the JLCL

Automatically exported from code.google.com/p/zeitcrawler

Experiments conducted on the occasion of the VarDial shared tasks

Old prototype for toponym extraction in historical texts written in German

A fast pattern-based URL compression for lists of links

Perform crawls of social networks (identi.ca, reddit, friendfeed) to gather internal and external links and identify their language

A bridge between Lichess bots and chess engines

A chess library for Python, with move generation and validation, PGN parsing and writing, Polyglot opening book reading, Gaviota tablebase probing, Syzygy tablebase probing, and UCI/XBoard engine communication

🚀 A list of Haystack Integrations, maintained by the community or deepset.

No description.

Automatically exported from code.google.com/p/gps-corpus-builder

Automatically exported from code.google.com/p/equipe-crawler

Explore, visualize and publish corpora as CSS/XHTML documents

Visualization of the most frequent words in the German federal election in 2021

A one-pass FSA valency-oriented chunker for German (proof of concept)

LAnguage-CLassified OpenSubtitles