> For the complete documentation index, see [llms.txt](https://kchuene.gitbook.io/data-communication-project/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://kchuene.gitbook.io/data-communication-project/archive/fluentd.md).

# Fluentd

Containerized Fluentd service for honeypot logs reception and processing.

## Pre-Configuration

Create `./fluentd-conf/fluent.conf`:

{% code overflow="wrap" lineNumbers="true" %}

```xml
# Input plugin
<source>
   # Spin up http listener
   @type http # TODO: must be HTTP(S)
   port 9880
   bind 0.0.0.0 
</source>

# Output plugin
<match **>
   @type stdout
</match>
```

{% endcode %}

{% hint style="warning" %}
Make sure the config file is named `fluent.conf` without the **'d'**, otherwise the service won't startup successfully.
{% endhint %}

## Installation

**Pull image**

{% code overflow="wrap" lineNumbers="true" %}

```
sudo docker pull fluent/fluentd:edge-debian
```

{% endcode %}

**Install MongoDB plugin**

Install `fluent-plugin-gelf-hs` on top of the base image. It is an important plugin that allows **Fluentd** to communicate with **Graylog**.

Create and edit `Fluentd-Dockerfile`&#x20;

{% code overflow="wrap" lineNumbers="true" %}

```
# Extending from base image with gelf-hs plugin
FROM fluent/fluentd:edge-debian 
USER root
RUN gem install fluent-plugin-gelf-hs
USER fluent
```

{% endcode %}

Build the extended image

{% code overflow="wrap" lineNumbers="true" %}

```
sudo docker build -f Fluentd-Dockerfile -t local/fluentd:edge-debian-gelf-hs ./
```

{% endcode %}

**Compose configuration**

{% code overflow="wrap" lineNumbers="true" %}

```yaml
services:
   fluentd:
      image: local/fluentd:edge-debian-gelf-hs # Extended image
      restart: always
      ulimits:
         nofile:
            soft: 65536
            hard: 65536
      volumes:
         - ./fluentd-conf:/fluentd/etc
      ports:
         - 9880:9880
```

{% endcode %}

**Create + Run container**

{% code overflow="wrap" lineNumbers="true" %}

```
sudo docker compose -f fluentd-compose.yaml up -d
```

{% endcode %}

**Confirm plug installation**

{% code overflow="wrap" lineNumbers="true" %}

```
sudo docker exec fluentd gem list | grep "gelf-hs"  
```

{% endcode %}

You should see something like `fluent-plugin-gelf-hs (1.0.8)`.

## Connecting to Graylog

**Prepare Graylog for Fluentd**

Graylog accepts data via so called **Inputs**, so we need to setup one for **Fluentd**. Logged in, navigate to `http://localhost:9000/system/inputs`, in the interface:

{% code overflow="wrap" lineNumbers="true" %}

```
System > Inputs
```

{% endcode %}

From the dropdown, select `GELF UDP` then click `Launch new input`

{% code overflow="wrap" lineNumbers="true" %}

```
GELF UDP > Launch new input
```

{% endcode %}

A modal dialogue should pop up, Uncheck the `Global` box, select the `Node` and fill the `Title`, then click `Save`&#x20;

{% code overflow="wrap" lineNumbers="true" %}

```
Uncheck Global > Node > Title > Save
```

{% endcode %}

On the right of you newly created **Local input** click `Set-up input`, get through the stages `Routing`, `Launch` and `Diagnosis` and there after your **Input** should indicate `RUNNING`.

{% code overflow="wrap" lineNumbers="true" %}

```
Routing > Create Stream > Title
```

{% endcode %}

{% code overflow="wrap" lineNumbers="true" %}

```
Launch > Start Input
```

{% endcode %}

{% code overflow="wrap" lineNumbers="true" %}

```
Diagnosis > Launch Input Diagnosis
```

{% endcode %}

**Configure Fluentd**

Create or edit `fluentd.conf` (from whichever project folder you created), configure **Fluentd** as follows:

{% code overflow="wrap" lineNumbers="true" %}

```
<source>
   @type forward
   tag honeypot 
</source>

<match **>
   @type gelf
   host 127.0.0.1
   port 12201
   
   <buffer>
      flush_interval 5s
   </buffer>
</match>
```

{% endcode %}

**Recreate the Fluentd container**

{% code overflow="wrap" lineNumbers="true" %}

```
sudo docker compose up fluentd --force-recreate
```

{% endcode %}

{% embed url="<https://docs.fluentd.org/installation/before-install>" %}

{% embed url="<https://docs.fluentd.org/container-deployment/install-by-docker>" %}

{% embed url="<https://docs.fluentd.org/configuration/config-file>" %}

{% embed url="<https://hub.docker.com/r/fluent/fluentd>" %}

{% embed url="<https://docs.fluentd.org/how-to-guides/graylog>" %}

{% embed url="<https://rubygems.org/gems/fluent-plugin-gelf-hs/versions/1.0.8>" %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://kchuene.gitbook.io/data-communication-project/archive/fluentd.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.