Jenkins Integration installation guide

What data does it ingest?

The Jenkins Integration collects data from two sources:

  • Fluent Bit - Fluent Bit monitors the Jenkins logs written to the local filesystem.

  • Jenkins Statistics Gatherer Plugin collects telemetry about each job and publishes it locally to Fluent Bit.

To ingest data, install and configure the needed agents on each host as described below. For more about exploring this data, see Jenkins Integration.

Setup

Note

This setup is the same as creating a data connection in the Jenkins app. If you have already done this in the app, you do not need to perform these steps again.

The Jenkins Integration uses Fluent Bit and the Jenkins Statistics Gatherer Plugin to send logs and metrics to Observe.

Observe App for Jenkins

Install the Jenkins App for Observe using the App section under Workspace Setting. You may also contact us for assistance in installing and configuring the app for your environment.

Jenkins app

Fluent Bit installation and configuration

You need the following items:

  • Your Observe Customer ID

  • A datastream token - for details on creating a token, see Data streams

  • One or more Linux Jenkins hosts to monitor

  • The location of the Jenkins logs directory (such as /var/lib/jenkins/)

The instructions work with the following platforms:

  • Amazon Linux 2

  • Ubuntu 20.04 LTS

  • Centos 7+

Run the following installation script at the command prompt of your Linux hosts. (Update the module_path path to your Jenkins log directory if needed.) This script installs Fluent Bit on your Jenkins host and configures it to send data to Observe.

If you would like to review the script parameters, you may find the install script here.

curl "https://raw.githubusercontent.com/observeinc/linux-host-configuration-scripts/main/observe_configure_script.sh" | bash
-s --customer_id ${OBSERVE_CUSTOMER?} --ingest_token ${OBSERVE_TOKEN?} --collection_endpoint ${OBSERVE_COLLECTION_ENDPOINT?} --module "jenkins" --module_path "/var/lib/jenkins"

Statistics Gatherer Plugin for Jenkins

The Jenkins integration collects additional data using the Statistics Gatherer plugin for Jenkins

After you install the statistics gatherer, configure it to send data to your local Fluent Bit installation, configured in the previous step. It then forwards the data to Observe.

Once you have installed the plugin, configure the statistics gatherer as follows:

  1. In the Jenkins UI, navigate to Manage Jenkins

  2. Select Configure system and scroll to Statistics Gatherer

  3. Configure the following fields, using the URLs below:

    Example of configured Statistics Gatherer
    Queue URL:
    http://localhost:8888/jenkins/queue/
    
    Build URL:
    http://localhost:8888/jenkins/build/
    
    Project URL:
    http://localhost:8888/jenkins/project/
    
    BuildSteps URL:
    http://localhost:8888/jenkins/buildsteps/
    
    ScmCheckoutInfo URL:
    http://localhost:8888/jenkins/scmcheckoutInfo/
    
  4. Select Advanced Settings and enable HTTP publishing

    Example of configured Statistics Gatherer Advanced Section

You have now configured the Jenkins app and sent your Jenkins data to Observe.