Hydra
简单但绝不简陋的 Python3 爬虫项目，集合 HelloGitHub 所有平台的账号信息。会基础的 Python 语法就能看懂！
A Python web scraper that aggregates HelloGitHub's own content stats across Chinese platforms like WeChat and Zhihu, doubling as a scraping tutorial.
Hydra is a Python web scraping project created by the HelloGitHub team to collect and aggregate their own content and account data from multiple Chinese platforms. The platforms supported include WeChat, Zhihu, Juejin, Toutiao, CSDN, and others. The purpose is to let the team's authors see how their published articles are performing across all the places where HelloGitHub posts content, such as view counts and publication dates.
The project is also intended as a learning resource. The README is written in Chinese and describes the project as a practical example that covers web scraping, database operations, common third-party libraries, parsing web pages and APIs, writing unit tests, mocking network requests, error monitoring, code quality automation, and GitHub Actions. The idea is that anyone who knows basic Python syntax can read through the code and learn from it.
To run the project, you clone the repository, create a local configuration file, install dependencies using Poetry (a Python package manager), and then run individual scrapers for specific platforms using a command-line argument. The project requires Python 3.7.5 or newer.
The README is in Chinese. The project is licensed under a Creative Commons license that allows sharing with attribution but prohibits commercial use and modifications.
Where it fits
- Track how HelloGitHub's articles perform across WeChat, Zhihu, Juejin, and other platforms.
- Learn practical Python web scraping, database, and testing techniques from real code.
- Run an individual platform scraper locally with Poetry and a config file.