Paddle-Lite
PaddlePaddle High Performance Deep Learning Inference Engine for Mobile and Edge (飞桨高性能深度学习端侧推理引擎）
Paddle Lite is Baidu's AI inference engine that runs optimized deep learning models on phones, embedded devices, and edge hardware where memory and processing power are limited.
Paddle Lite is an inference engine from Baidu's PaddlePaddle team designed to run AI models on phones, embedded devices, and other edge hardware where resources like memory and processing power are limited. Inference means taking a trained AI model and using it to make predictions on new data, which is a different and lighter task than training the model in the first place.
The tool takes a model trained with the PaddlePaddle deep learning framework and optimizes it for a specific target device. This optimization step applies techniques like quantization, which reduces the numerical precision of the model's weights to shrink file size and speed up computation, and operator fusion, which combines multiple processing steps into single efficient operations. Models from other frameworks such as TensorFlow and PyTorch can be converted to the required format using a companion conversion tool before being passed through this same pipeline.
Once optimized, the model can run on a very wide range of hardware. The list of supported targets includes Android and iOS phones, standard ARM-based embedded Linux boards, and specialized AI accelerator chips from manufacturers including Qualcomm, Huawei, MediaTek, Cambricon, and Intel. This breadth is one of Paddle Lite's distinguishing characteristics: a single development workflow can target many different devices without rewriting the application code.
Developer APIs are available in C++, Java, and Python. Pre-built libraries for common platforms are available for direct download, which avoids the need to compile from source. Demo Android apps covering tasks like image classification, object detection, face landmark detection, and face mask detection are available as downloadable APK files for quick testing.
The README is written primarily in Chinese, with an English version available in a separate file.
Where it fits
- Run an image classification or object detection model on an Android phone without a server
- Deploy a PaddlePaddle or converted TensorFlow model to an ARM embedded Linux board
- Shrink and speed up a deep learning model using quantization for resource-constrained edge devices