Skip to main content

Use metrics to monitor node performance

To enable the Prometheus monitoring and alerting service to access Hyperledger Besu metrics, use the --metrics-enabled option. Use Grafana to visualize the collected data. See the sample Besu Full Grafana dashboard.

The Besu example networks have monitoring with Prometheus and Grafana configured.

Use Prometheus to monitor the number of blocks your Besu node is behind the chain head, and to alert you that your node is not keeping up with the chain head.

This recording shows examples of monitoring Hyperledger Besu.

Install Prometheus​

To use Prometheus with Besu, install the Prometheus main component. On MacOS, install with Homebrew:

brew install prometheus
tip

You can also install:

  • Exporters that send system metrics to Prometheus to monitor non-Besu-specific items such as disk and CPU usage.
  • Other Prometheus components, such as the Alert Manager. Additional configuration is not required for these components because Prometheus handles and analyzes data directly from the feed.

Set up and run Prometheus with Besu​

To configure Prometheus and run with Besu:

  1. Configure Prometheus to poll Besu. For example, add the following YAML fragment to the scrape_configs block of the prometheus.yml file:

    - job_name: besu
      scrape_interval: 15s
      scrape_timeout: 10s
      metrics_path: /metrics
      scheme: http
      static_configs:
        - targets:
            - localhost:9545

    Prometheus requires 3 MB of space per node per hour for metrics, with a scrape_interval of 15 seconds.

  2. Start Besu with the --metrics-enabled option. To start a single node for testing with metrics enabled, run the following command:

    besu --network=dev --miner-enabled --miner-coinbase <COINBASE ADDRESS> --rpc-http-cors-origins="all" --rpc-http-enabled --metrics-enabled

    To specify the host and port on which Prometheus accesses Besu, use the --metrics-host and --metrics-port options. The default host and port are 127.0.0.1 (localhost) and 9545.

    danger

    To avoid DNS rebinding attacks, if running Prometheus on a different host than your Besu node (any host other than localhost), add the hostname that Prometheus uses to --host-allowlist. For example, if Prometheus is configured to get metrics from http://besu.local:8008/metrics, then besu.local must be in --host-allowlist.

  3. In another terminal, run Prometheus specifying the prometheus.yml file:

    prometheus --config.file=prometheus.yml

  4. View the Prometheus graphical interface.

    tip

    Use a log ingestion tool, such as Logstash, to parse the logs and alert you to configured anomalies.

Run Prometheus with Besu in push mode​

The --metrics-enabled option enables Prometheus polling of Besu, but sometimes metrics are hard to poll (for example, when running inside Docker containers with varying IP addresses). To enable Besu to push metrics to a Prometheus push gateway, use the --metrics-push-enabled option.

To configure Prometheus and run with Besu pushing to a push gateway:

  1. Configure Prometheus to read from a push gateway. For example, add the following YAML fragment to the scrape_configs block of the prometheus.yml file:

    - job_name: push-gateway
      metrics_path: /metrics
      scheme: http
      static_configs:
        - targets:
            - localhost:9091

  2. Start the push gateway. You can deploy the push gateway using the Docker image:

    docker pull prom/pushgateway
    docker run -d -p 9091:9091 prom/pushgateway

  3. Start Besu specifying the --metrics-push-enabled option and port of the push gateway:

    besu --network=dev --miner-enabled --miner-coinbase <COINBASE ADDRESS> --rpc-http-cors-origins="all" --rpc-http-enabled --metrics-push-enabled --metrics-push-port=9091 --metrics-push-host=127.0.0.1

  4. In another terminal, run Prometheus specifying the prometheus.yml file:

    prometheus --config.file=prometheus.yml

  5. View the Prometheus graphical interface.

View Prometheus graphical interface​

  1. Open a Web browser to http://localhost:9090 to view the Prometheus graphical interface.

  2. Choose Graph from the menu bar and click the Console tab below.

  3. From the Insert metric at cursor drop-down, select a metric such as besu_blockchain_difficulty_total or ethereum_blockchain_height and click Execute. The values display.

  4. Click the Graph tab to view the data as a time-based graph. The query string displays below the graph. For example, {ethereum_blockchain_height{instance="localhost:9545",job="prometheus"}.

View the metrics list​

Run the following command to view the full list of available metrics:

curl http://localhost:9545/metrics

Update the host and port if you are not using the default values.

Each metric, such as besu_blockchain_chain_head_gas_limit, starts with a metric category prefix. Metrics specific to Besu use the besu_ prefix, followed by another metric category. You can enable metric categories using the --metrics-category command line option.