Crawlab Team ORG

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

A web-based platform for running, scheduling, and monitoring web scrapers written in any language across multiple machines, upload your crawler code and manage everything from one dashboard.

Article publishing platform that automatically distributes your articles to various media channels

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

An intelligent web service to automatically detect web content and extract information from it.

SDK for Crawlab, including SDK for different programming languages such as Python, Node.js and Java, and a CLI Tool written in Python.

Backend core modules for Crawlab

Documentation for Crawlab

🎉 A Vue.js 3.0 UI Library made by Crawlab team

Frontend for Crawlab

Crawlab Go SDK

SDK for Crawlab AI

No description.

Resources and tools for evaluating the performance and behavior of web crawling systems

This is a golang implementation of various BM25 algorithms. It is a port of `dorianbrown/rank_bm25`

No description.

Plugin for notifications | 消息通知插件

gRPC for Crawlab

A complete Golang client for SeaweedFS

Convert HTML to JSON. Can also (intelligently) convert HTML tables to JSON (using table headers (if available) as keys in the resulting JSON).

Backend database module for Crawlab

Public plugins repo for Crawlab

No description.

Backend file system module for Crawlab

Version Control System (VCS) for Crawlab

Trace stack tools

Plugin for managing dependencies | 依赖管理插件

Plugin framework for Crawlab

No description.

No description.

No description.

Plugins and extensions for Crawlab

Plugin for spider assistance | 爬虫助手插件

Python SDK for Crawlab

No description.

AI-powered scrapy plugin

Backend tools for Crawlab

Intelligence service to monitor pricing info for cloud server providers

Detect and classify pagination links

A tool project that supports dependency injection (DI) in Golang

Open Source License Key Generation and Verification Tool written in Go

Log module for Crawlab

HTML5 file drag-drop zone with vue3

Tutorials for Crawlab

Java SDK for Crawlab

Node.js SDK for Crawlab

:lemon: Gin wrapper with OpenAPI 3 spec generation

End-to-End (E2E) tests for Crawlab

A Go implementation of the Model Context Protocol (MCP), enabling seamless integration between LLM applications and external data sources and tools.

No description.

No description.

No description.

Vuepress plugin for mermaid.js diagrams

No description.

No description.

Resources served on Github

No description.

Plugins for Crawlab

Guide to safely crawl web pages instead of POP (Prison Oriented Programming)