Updated 2019-11-11

High Availability of JSON-RPC and RPC Pub/Sub APIs

To enable high availability to the RPC Pub/Sub API over WebSockets or the JSON-RPC API run and synchronize multiple Hyperledger Besu nodes to the network. Use a load balancer to distribute requests across nodes in the cluster that are ready to receive requests.

Load Balancer

Important

We do not recommend putting bootnodes behind a load balancer.

Determining When a Node is Ready

Use the readiness endpoint to determine when a node is ready.

Note

The minimum number of peers and number of blocks from the best known block for a node to be considered ready is deployment specific.

Transaction Nonces

The nonce for the next transaction is obtained using eth_getTransactionCount. The nonce depends on the transactions in the transaction pool. If eth_getTransactionCount and eth_sendRawTransaction requests for a specific account are sent to multiple nodes, the eth_getTransactionCount results can be incorrect.

Note

If using private transactions, priv_getTransactionCount or priv_getEeaTransactionCount are used to obtain the nonce and eea_sendRawTransaction to send private transactions.

To get correct nonces when distributing requests across a cluster, do one of:

  • Track the next nonce outside of the Besu node (as MetaMask does)
  • Configure the load balancer in sticky mode so requests from a specific account are sent to a single node unless that node is unavailable.

Subscriptions

You can subscribe to events using:

We recommend using RPC Pub/Sub over WebSockets because WebSockets connections are associated a specific node and do not require using the load balancer in sticky mode.

If using filters over HTTP, configure the load balancer in sticky mode to associate the subscription with a specific node.

Recovering from Dropped Subscriptions

Subscriptions can be dropped if:

  • WebSockets connection is disconnected
  • Node serving the subscription is removed from the ready pool

If a subscription is dropped, events can be missed while reconnecting to a different node. To recover dropped messages, create another subscription and follow the process for that subscription type:

New Headers

Use eth_getBlockByNumber to request information on blocks from the last block before the subscription dropped to the first block received from the new subscription.

Logs

Use eth_getLogs to request logs from the block number of the last log received before the subscription dropped to the current chain head.

New Pending Transactions

Use txpool_besuTransactions to request all pending transactions for the new node.

Note

Nodes do not all store the same pending transactions.

Dropped Pending Transactions

Use txpool_besuTransactions to request all pending transactions for the new node.

Note

Nodes do not all store the same pending transactions.

Syncing

The syncing state of each node is specific to that node. Use eth_syncing to retrieve the syncing state of the new node.