beats
:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
A collection of lightweight data shippers you install on servers to collect logs, metrics, and network traffic and forward them to Elasticsearch or Logstash for storage and analysis.
Beats is a collection of small programs you install on your servers to collect data and send it to a central location for storage and analysis. Each individual Beat is purpose-built for one kind of data: log files, system metrics, network traffic, Windows event logs, or uptime checks. The collected data gets forwarded to Elasticsearch (a search and storage engine) or to Logstash (a data processing pipeline), where it can be explored through Kibana (a dashboard tool). All three of those belong to the same Elastic Stack family.
The design goal is minimal resource use. Beats are written in Go, have a small installation size, and require no additional software to run on the host. You drop them onto a server, configure what to collect, and they start shipping data.
The repository contains several officially supported Beats. Filebeat watches log files and ships new entries as they appear. Metricbeat pulls performance numbers from the operating system and from services like databases or web servers. Packetbeat inspects network packets to show which services are talking to each other and how fast. Heartbeat periodically pings URLs or services to check whether they are up. Auditbeat reads Linux audit logs and monitors file changes. Winlogbeat ships Windows Event logs. Osquerybeat runs Osquery, a tool that lets you query system information using SQL-like statements.
The repository also includes libbeat, a Go library for building your own custom Beat if none of the official ones cover your needs. A community list of third-party Beats exists on the Elastic website. Pre-compiled binaries and packages for supported platforms are available for download without building from source.
Documentation and setup guides for each Beat live on the Elastic website rather than in this repository. Support questions go to the Elastic discussion forums; GitHub issues are reserved for confirmed bugs and feature requests.
Where it fits
- Ship log files from your servers to Elasticsearch in real time using Filebeat with minimal CPU and memory overhead.
- Monitor system CPU, memory, and disk metrics across your fleet using Metricbeat and visualize them in Kibana.
- Check whether your services and URLs are up by running Heartbeat to periodically ping them.
- Build a custom data shipper for a data source not covered by official Beats using the libbeat Go library.