Installing the Jenkins App

What type of data does Jenkins ingest?

The Jenkins Integration collects data from two sources:

  • Fluent Bit - Fluent Bit monitors the Jenkins logs written to the local filesystem.

  • Jenkins Statistics Gatherer Plugin collects telemetry about each job and publishes it locally to Fluent Bit.

To ingest data, install and configure the required agents on each host as described below. For more about exploring this data, see Jenkins Integration.

Setup

Note

This setup follows the same procedure as creating a data connection in the Jenkins app. If you already created a data connection in the app, you do not need to perform these steps again.

The Jenkins Integration uses Fluent Bit and the Jenkins Statistics Gatherer Plugin to send logs and metrics to Observe.

Installing the Observe App for Jenkins

Install the Jenkins App for Observe using the App section under Workspace Setting.

Jenkins app

Figure 1 - Jenkins app card

Installing and Configuring Fluent Bit

You need the following items:

  • Your Observe Customer ID

  • A datastream token - for details on creating a token, see Data streams

  • One or more Linux Jenkins hosts to monitor

  • The location of the Jenkins logs directory (such as /var/lib/jenkins/)

Note

Some Observe instances may optionally use a name instead of Customer ID; if this is the case for your instance, contact your Observe Data Engineer to discuss implementation. A stem name will work as is, but a DNS redirect name may require client configuration.

The instructions work with the following platforms:

  • Amazon Linux 2, Amazon Linux 2023

  • Ubuntu 20.04 LTS

  • Centos 7+

Run the following installation script at the command prompt of your Linux hosts. Update the module_path path to your Jenkins log directory if needed. This script installs Fluent Bit on your Jenkins host and configures it to send data to Observe.

If you want to review the script parameters, you can find the install script here.

curl "https://raw.githubusercontent.com/observeinc/linux-host-configuration-scripts/main/observe_configure_script.sh" | bash
-s -- --customer_id ${OBSERVE_CUSTOMER?} --ingest_token ${OBSERVE_TOKEN?} --collection_endpoint ${OBSERVE_COLLECTION_ENDPOINT?} --module "jenkins" --module_path "/var/lib/jenkins"

Statistics Gatherer Plugin for Jenkins

The Jenkins integration collects additional data using the Statistics Gatherer plugin for Jenkins

After you install the Statistics Gatherer plugin, configure it to send data to your local Fluent Bit installation which you configured in the previous step. The plugin then forwards the data to Observe.

Once you have installed the plugin, configure the Statistics Gatherer as follows:

  1. In the Jenkins UI, navigate to Manage Jenkins

  2. Select Configure system and scroll to Statistics Gatherer.

  3. Configure the following fields, using the URLs below:

    Example of configured Statistics Gatherer

    Figure 2 - Jenkins URLs 

    Queue URL:
http://localhost:8888/jenkins/queue/

Build URL:
http://localhost:8888/jenkins/build/

Project URL:
http://localhost:8888/jenkins/project/

BuildSteps URL:
http://localhost:8888/jenkins/buildsteps/

ScmCheckoutInfo URL:
http://localhost:8888/jenkins/scmcheckoutInfo/

  4. Select Advanced Settings and enable HTTP publishing

    Example of configured Statistics Gatherer Advanced Section

Figure 3 - Configured Statistics Gatherer Advanced Section Example

You have now configured the Jenkins app and sent your Jenkins data to Observe.