Improved Fee Estimations in the Hiro Wallet and Stacks API 🎯

What Are Transaction Fees and Fee Rates?

A transaction fee is the amount of STX a user sends when submitting a transaction in order to incentivize miners to process it. The fee rate of a transaction is the fee per unit of the transaction’s “size”. The current Stacks node computes a transaction’s size using various parameters: its write count, write length, read count, read length, runtime, and size in number of bytes.

With a few exceptions, a default mining node considers transactions in order of fee rate, highest first. A higher (and thus more competitive) fee rate will increase the likelihood of a user’s transaction getting included in an upcoming block. This consideration grows more important in times of network congestion. However, a user also does not want to overpay. To avoid doing so, they must have a good sense of what a competitive transaction fee rate is relative to what others are paying on the network. This is why we built fee estimation features in the Stacks API and Hiro Wallet.

The fee estimation API endpoint analyzes historical transaction data to compute these estimates. There are three fee rate options available to a user: low, medium, and high. Each of these categories represents a specific percentile of the fee rates computed from looking at the transactions included in the past few blocks. Fee rates are ordered from smallest to largest and then estimates are set at the 5th, 50th and 95th percentiles. The low estimate corresponds to the fifth percentile, medium corresponds to the fiftieth percentile, and high corresponds to the ninety-fifth percentile.

A Healthy Fee Market

Transaction fees are essential in a blockchain. They incentivize miners to process and include transactions in blocks, and prevent the occurrence of spam transactions in the network. As the network matures, we expect that users will pay higher transaction fees for time-sensitive transactions or do replace-by-fee to bump up the priority of their transactions. This competitive pricing naturally leads to the emergence of a robust fee market, which is a sign of a healthy blockchain network.

On the Stacks network, we have been seeing short bursts of high congestion, often centered around major contract launches. Fee markets can help soften these bursts. A user with a less urgent use case may be inclined to wait for the network to free up, to avoid having to pay high fees to ensure inclusion of their transaction in a block.

The Nuts and Bolts

This section will cover the technical details of our efforts on fee estimation. Skip ahead to Guidance for users if you aren’t concerned with implementation details!

Past Work

Those keeping a close eye on Hiro products may be aware that we rolled out a version of fee estimation back in November 2021. The Hiro Wallet presented estimates from an older version of the API endpoint. That API endpoint returned data from the fee estimator, which computed erroneously high fee estimates. This happened for two main reasons:

1. Fee rates were not weighted by the size of the related transaction. Thus, STX transfers, which are minuscule by transaction size, had relatively huge fee rates, which greatly skewed fee rate estimates.
2. Outlier blocks disproportionately swung estimates. These blocks seemed to be mostly filled with STX transfers, which, as mentioned, had relatively high fee rates. In the initial fee estimator, we used exponential windowing to incorporate fee data from blocks, which allowed data from outlier blocks to persist for longer than desired.

To read more about the reasons for the inaccurate estimates and see some relevant data, you can check out this GitHub issue.

How the Estimator Works & Improvements

As mentioned above, the fee estimate consists of three rates: a low, middle, and high estimate. Each of these corresponds to a certain percentile of observed fee rates (5, 50, and 95 respectively). After a block is mined, the miner notifies the fee estimator and provides it with a receipt of the block. This receipt has information about the included transactions and their execution costs. The fee estimator then updates its estimates for the 5th, 50th, and 95th percentile of fee rates based on this new data.

In the newer implementation of the fee estimator, we took special care to fix the issues seen in the past. Here are each of the modifications and related reasoning:

1. If a Stacks block isn’t full, the fee estimator will “fill” the remaining block space with a dummy transaction that uses the minimum fee rate. This is done to account for the value of empty space. If blocks are often half-empty, a user wouldn’t want to match the fee rate of the occupied part of the block - they could use a lower fee rate and be fairly confident that there will be enough block space for their transactions to be included.
   ‍
2. The fee rates are now weighted by transaction size during calculation. The relative size of the transaction is used to determine how much fee estimates should be affected by that fee rate. In practice, this means that heavier transactions with high fee rates have a greater chance of nudging the high estimate than the same number of light transactions with high fee rates.
   ‍
3. Previously, the fee estimator used exponential decay for block data. In practice, this approach allowed outlier fee rates to impact fee estimates for longer than desired. In the new approach, the fee estimator computes estimates from data from the past 5 blocks by default, although the exact size of the window is configurable to miners.
   ‍
4. When a user requests estimates, the fee estimator now returns a “fuzzed” version of the low, middle, and high estimate - some random noise is added to the baseline estimate. This is done to introduce some variation into the fee rates, to prevent the network converging on a single fee rate. This ensures that the fee rate dynamically adjusts according to network congestion levels. In periods of congestion, transactions that used the higher fee rates would be included in the next block, nudging the fee estimates up. As congestion clears, the transactions with lower estimates will also be included in the block, bringing the fee rate estimate back down again.

Future Improvements

This work analyzes transactions in recent blocks to provide users with estimates for what a competitive fee rate might look like. However, in reality, a user’s competition is against the other transactions in the mempool, the group of unmined transactions that a miner considers in packaging its next block. Thus, future fee rate estimates may compute low, medium, and high estimates based on a transaction’s live competition by computing the fee rates of the transactions in the mempool itself.

Guidance for Users

Improving the Hiro Wallet Experience

The fee estimator work was included in the Stacks 2.05.0.1.0 blockchain release in early February as well as the most recent release of the Hiro Wallet.

From a user experience perspective, a more effective fee estimation protocol means that users will be able to better price their transactions. If they want to ensure their transaction is prioritized over other ones, using the “high” estimate will increase the likelihood their transaction will be confirmed more quickly. Effective fee estimations will give users more of an idea of the relative “attractiveness” of their transaction to miners, without users having to overspend to be confident in their transaction’s inclusion in the next block.

Access Estimates Through the API

Developers can access these estimates through the fee estimation API endpoint. The endpoint returns all three fee estimates: high, middle, and low.