Bruce Tang

A collection of awesome web crawler,spider in different languages

A curated reference list of web crawling and scraping tools organized by programming language, browse it to find the right library before starting a data collection project.

可视化任务调度系统，精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)

all kinds of scrapy demo

all kinds of demos of tensorflow code

Python web visualize build on the awesome web framework sanic

darknet yolov3 tiny train model demo

一键抓取cnbeta 首页的所有消息

Simple DAG-based job scheduler in Python

tensorflow gpu docker file to build docker image

demo of using dotnet core and Nancy

No description.

Visual scraping for Scrapy

DHT Spider + BitTorrent Client = P2P Spider

A service daemon to run Scrapy spiders

using vibora and all kinds of message queue

all kinds of docker library ,such as opencv 3 , pytorch ,deep learning library

A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

A flexible python library for building your own cron-like system, with REST APIs and a Web UI.

Scrapy, a fast high-level web crawling & scraping framework for Python.

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

a flask profiler which watches endpoint calls and tries to make some analysis.

The code of the spider

豆瓣相册助手，选择相册，下载图片，一键帮助

Generates index page like mod_autoindex.

Docker Apache Airflow

Python extension to capture video with video4linux2 (fork of https://launchpad.net/python-v4l2-capture)

在scrapyd基础上新增权限验证、爬虫运行信息统计、界面重构、，并增加排序、筛选过滤等多个API

🚂 12306 购票助手，支持分布式，多账号，多任务购票以及 Web 页面管理

Prometheus Exporter for Redis Metrics. Supports Redis 2.x, 3.x, 4.x, and 5.x

Pholcus is a distributed high-concurrency crawler software written in pure golang

No description.

capture SSL/TLS text content without CA cert by eBPF.

for brucedone.com blog comments here, feel free to comment on it :)

chatGPT私有化部署

:fire: Linux下C++轻量级Web服务器

A captcha library that generates audio and image CAPTCHAs.

A Python wrapper for working with Scrapyd's API.

用 Markdown 写公众号文章，一键转换为精美排版并自动上传到微信草稿箱。支持 AI 多主题样式和批量发布，让公众号写作像发朋友圈一样简单。

An open and reliable container runtime

A parallel implementation of gzip for modern multi-processor, multi-core machines.

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化的设计和执行爬虫任务。别名：ServiceWrapper面向Web应用的智能化服务封装系统。

Tiny BitTorrent client written in Go

High-level tracing language for Linux eBPF

🎉 A Vue.js 3 UI Library made by Element team

分布式代理ip共享系统

Apache TubeMQ

Coupon and voucher center

Caffe2 is a lightweight, modular, and scalable deep learning framework.

flask non-block web scheduler demo

Python implementation of directed acyclic graph