fluentd

Fluentd is an open-source tool for collecting logs. When software runs, it constantly produces small records of what is happening: a user signed in, a request failed, a job finished. These records are called events or logs, and in a real system they come from many different programs and machines at once. Fluentd's job is to gather all of those events from their various sources and then send them on to wherever you want them stored or analyzed.

The README describes Fluentd as a way to unify your logging infrastructure. Instead of each program writing logs in its own format to its own location, Fluentd sits in the middle and acts as a single collection layer. It can read events coming in and write them out to many kinds of destinations, including plain files, traditional databases, NoSQL stores, cloud services, and big-data systems like Hadoop. The point is to have one consistent place that handles the flow of log data.

Getting started is shown as a few short commands. You install Fluentd as a Ruby gem, generate a sample configuration, start the program pointed at that configuration, and then pipe a small test message in to confirm it is working. The behavior of Fluentd is driven by its configuration file, which is where you describe the sources to read from and the destinations to write to.

The rest of the README is aimed at people who want to work on Fluentd itself. It lists the prerequisites, such as a recent version of Ruby and git, and explains how to install the project's dependencies with Bundler and how to run its test suite. It also points to the official website, documentation, community Slack, and discussion forum for more help. Fluentd is written in Ruby, is released under the Apache License version 2.0, and is a project hosted under the Cloud Native Computing Foundation. The README notes that a third-party security audit of the project was carried out.