Updated 2020-03-20

High availability of JSON-RPC and RPC Pub/Sub APIs

To enable high availability to the RPC Pub/Sub API over WebSockets or the JSON-RPC API, run and synchronize more than one Hyperledger Besu nodes to the network. Use a load balancer to distribute requests across nodes in the cluster that are ready to receive requests.

Load Balancer

Important

We do not recommend putting bootnodes behind a load balancer.

Determining when a node is ready

Use the readiness endpoint to determine when a node is ready.

Note

The minimum number of peers and number of blocks from the best known block for determining if a node considered ready is deployment specific.

Transaction nonces

Besu obtains the nonce for the next transaction using eth_getTransactionCount. The nonce depends on the transactions in the transaction pool. If sending eth_getTransactionCount and eth_sendRawTransaction requests for a specific account to more than one node, the eth_getTransactionCount results might be incorrect.

Note

If using private transactions, retrieve the nonce using priv_getTransactionCount or priv_getEeaTransactionCount and send the private transactions using eea_sendRawTransaction.

To get correct nonces when distributing requests across a cluster, either:

  • Track the next nonce outside of the Besu node (as MetaMask does)
  • Configure the load balancer in sticky mode to send requests from a specific account to a single node, unless that node is unavailable.

Subscriptions

You can subscribe to events using:

We recommend using RPC Pub/Sub over WebSockets because WebSockets connections associate with a specific node and do not require using the load balancer in sticky mode.

If using filters over HTTP, configure the load balancer in sticky mode to associate the subscription with a specific node.

Recovering from dropped subscriptions

Dropped subscriptions can occur because of:

  • A disconnected WebSockets connection
  • The removal of the node serving the subscription from the ready pool.

If there is a dropped subscription, missed events might occur while reconnecting to a different node. To recover dropped messages, create another subscription and follow the process for that subscription type:

New headers

To request information on blocks from the last block before the subscription dropped to the first block received from the new subscription, use eth_getBlockByNumber.

Logs

To request logs from the block number of the last log received before the subscription dropped to the current chain head, use eth_getLogs.

New pending transactions

To request all pending transactions for the new node, use txpool_besuTransactions.

Note

Nodes do not all store the same pending transactions.

Dropped pending transactions

To request all pending transactions for the new node, use txpool_besuTransactions.

Note

Nodes do not all store the same pending transactions.

Syncing

The syncing state of each node is specific to that node. To retrieve the syncing state of the new node, use eth_syncing.