text_classification
一个文本分类的项目，这个项目实现了三种文本分类的方法，从一开始的随机森林，到fasttext，最后是基于BERT预训练模型训练出自己的文本分类模型；还包括模型的压缩，比如模型量化，模型蒸馏等操作；是一个完整的项目
A Chinese news headline classifier comparing traditional ML, FastText, BERT, and a distilled small model across 10 categories.
This Python project builds a classifier that sorts Chinese news headlines into one of ten categories: finance, real estate, stocks, education, science, society, politics, sports, gaming, and entertainment. The source data is described as coming from the Toutiao news platform and contains 200,000 labeled examples split into 180,000 for training and 10,000 each for validation and testing.
The repository offers three main approaches to the classification problem. The first uses traditional machine learning: headlines are tokenized with the jieba Chinese word segmenter, converted into TF-IDF feature vectors, and fed to a random forest classifier. The second uses FastText, a lightweight neural model that trains on word n-gram features and exposes a REST API via Flask for real-time inference. The third fine-tunes a full Chinese BERT model, a pre-trained deep language model with 110 million parameters, by adding a single linear classification layer on top.
A fourth approach explores knowledge distillation. Here the large BERT model acts as a teacher and a much smaller TextCNN model acts as a student. During training the student learns from both the labeled data and the teacher's output probabilities. The final student model is far smaller and faster than BERT while retaining much of its accuracy, which makes it more practical for devices with limited computing power.
The BERT model is large, around 390 MB before quantization and 146 MB after applying PyTorch dynamic quantization, and must be downloaded separately from the official source. The README notes that the random forest and FastText paths have been verified to work, but the knowledge distillation path may still contain bugs. The code detects and supports CUDA, Apple Silicon (MPS), and CPU automatically. Each sub-project includes a Flask API for serving predictions over HTTP.
The README is written in Chinese. The project is licensed under MIT.
Where it fits
- Compare traditional machine learning, FastText, and BERT approaches on the same text classification task.
- Learn how knowledge distillation can shrink a large BERT model into a smaller, faster student model.
- Serve trained text classifiers over an HTTP API using the included Flask setup.