Skip to main content

Use Splunk

Splunk is a third-party monitoring solution compatible with Besu. A Splunk server can receive Besu logs and enable complex search, visualization, and analysis.

Splunk can aggregate multiple logs in one place and run complex queries without being connected to the machine running Besu to read the standard output.

Options for running Splunk and Besu are:

Developer Quickstart with Splunk​

To view the Quickstart network logs in Splunk:

  1. Start the Developer Quickstart with Besu, selecting Splunk monitoring.
  2. Open the Splunk UI.

Splunk Connect for Ethereum Docker Compose​

To run a development Besu node and connect it to Splunk Enterprise, use the Splunk Connect for Ethereum demonstration Docker Compose environment provided by Splunk.

Requirements​

info

A Splunk license is not required to use the Splunk Connect for Ethereum demonstration.

Steps​

  1. Clone the Splunk Connect for Ethereum repository:

    git clone https://github.com/splunk/splunk-connect-for-ethereum.git
    cd splunk-connect-for-ethereum

  2. Start the demonstration environment by following the Splunk Connect for Ethereum repository README.

    note

    Splunk enterprise takes some time to start.

    Run docker ps and wait for the STATUS of the 3 containers to be Up [number] seconds (healthy).

    CONTAINER ID        IMAGE                        COMMAND                  CREATED             STATUS                    PORTS                                                                            NAMES
    127600dd1173        splunkdlt/ethlogger:latest   "ethlogger"              53 seconds ago      Up 51 seconds (healthy)                                                                                    ethlogger
    88dfcee683c4        splunk/splunk:latest         "/sbin/entrypoint.sh…"   53 seconds ago      Up 52 seconds (healthy)   8065/tcp, 8088-8089/tcp, 8191/tcp, 9887/tcp, 9997/tcp, 0.0.0.0:18000->8000/tcp   splunk
    111b0c6d6072        hyperledger/besu:1.4.4       "besu"                   53 seconds ago      Up 52 seconds (healthy)   8545-8547/tcp, 30303/tcp                                                         besu

Use Splunk Enterprise as a Docker container​

Prerequisites​

info

A Splunk license is not required to use the trial version of the Splunk Docker image. The image is not suitable for production use and has restrictions on daily log volume.

note

If running Besu as a Docker container, consider using Splunk Connect for Ethereum Docker Compose or Kubernetes instead of the Splunk Enterprise trial container.

Steps​

  1. Start the Splunk Enterprise container:

    docker run \
    -e SPLUNK_START_ARGS=--accept-license \
    -e SPLUNK_HEC_TOKEN=11111111-1111-1111-1111-1111111111113 \
    -e SPLUNK_PASSWORD=changeme \
    --rm \
    -p8080:8000 -p8088:8088 \
    -d \
    --name splunk-demo \
    splunk/splunk:latest

    Once the service is started, connect on http://localhost:8080/ and login as the admin user with a password of changeme.

    tip

    To follow the logs of the Splunk container:

    docker logs -f splunk-demo

  2. Create the Besu index:

    1. In the Splunk Web interface, navigate to the index list in the settings.
    2. Create an event index with an Index Name of besu.
    3. Leave other fields with the default values.
    4. Save the besu index.

  3. Run Besu. To start a Besu node running in development mode, run the following command:

    LOGGER=Splunk \
    SPLUNK_URL=https://localhost:8088 \
    SPLUNK_TOKEN=11111111-1111-1111-1111-1111111111113 \
    SPLUNK_SKIPTLSVERIFY=true \
    besu \
    --network=dev \
    --miner-coinbase=0xfe3b557e8fb62b89f4916b721be55ceb828dbd73 \
    --miner-enabled \
    --logging=trace

    The environment variables specified send the Besu logs to Splunk. Only LOGGER, SPLUNK_URL, SPLUNK_TOKEN and SPLUNK_SKIPTLSVERIFY are required in this example. The complete list of options is in the Splunk options reference table.

  4. In the Splunk Web interface, navigate to the search page. Type index="besu" in the search field. Log events sent by Besu are displayed.

    Congratulations! You can now play with the search and other Splunk features to explore your Besu logs.

    Splunk search page

  5. Stop Besu with ++ctrl+c++. Stop the Splunk container with docker stop splunk-demo.

Run a Splunk Enterprise instance​

Prerequisites​

Steps​

  1. Follow the steps in the Splunk Enterprise documentation to download, install, and run Splunk Enterprise.

  2. After logging into the Splunk Enterprise Web interface, navigate to the settings to:

    1. Create an HTTP Event Collector.
    2. Create an event index named besu.

  3. Run Besu as in step 3 in using Splunk on Docker. Set the SPLUNK_URL value to match the HTTP Event Collector address and port.

    You can display logs and use the search engine as in step 4 in using Splunk on Docker.

Splunk options reference​

NameDescriptionRequired
LOGGERSet to Splunk to activate sending logs to Splunk.Yes
HOSTCurrent host. If in a Docker environment, the default value is the docker container ID. Otherwise, the default value is localhost.No
SPLUNK_URLURL of the Splunk HTTP Event Collector. For example, use https://localhost:8088Yes
SPLUNK_TOKENAuthentication token, usually of the form 11111111-1111-1111-1111-111111111111Yes
SPLUNK_INDEXIndex to store logs. Defaults to besuNo
SPLUNK_SOURCESource of the logs. Defaults to besuNo
SPLUNK_SOURCETYPESource type of the logs. Defaults to besuNo
SPLUNK_BATCH_SIZE_BYTESSize of a log batch in bytes. Defaults to 65536No
SPLUNK_BATCH_SIZE_COUNTSize of a log batch in number of events. Defaults to 1000No
SPLUNK_BATCH_INTERVALInterval at which to send log batches. Defaults to 500No
SPLUNK_SKIPTLSVERIFYWhether to check the Splunk instance TLS certificate when sending data. Defaults to falseNo