GIP: 0051
Title: Exponential Query Fee Rebates for Indexers
Authors: Justin Grana, Howard Heaton, Tomás Migone
Created: 2023-03-01
Updated: 2023-03-29
Stage: Draft
Abstract
A Cobb-Douglas-inspired mechanism is currently used by The Graph to distribute query fee rebates to indexers that serve queries. We propose replacing this mechanism with an exponential rebate function that alleviates multiple shortcomings of Cobb-Douglas. The proposed rebate mechanism 1) ensures indexers no longer need to anticipate how much other indexers will stake (i.e. no “gaming”), 2) makes optimal staking decisions clear (i.e. significantly reduces cognitive load) and 3) significantly reduces frictional costs associated with serving queries. Specifically, at current stake levels, query fee burn would be less than 7%, which is in stark contrast to the current mechanism which historically has burned over 50% of query fees.
Motivation
For the protocol to operate effectively, query prices must be low enough to be justifiable by consumers, yet indexers must be able to obtain enough revenue from query fees to be profitable. The current Cobb-Douglas rebate mechanism is opaque, injects unnecessary complexity and does not clearly define optimal staking levels. This inhibits indexers from profitably serving queries as their revenue is dependent on optimizing for this mechanism. Furthermore, since the mechanism does not clearly characterize optimal staking, equilibrium stake levels are difficult to anticipate, which in turn jeopardizes the security of the network. The task at hand is to alleviate these shortcomings as much as possible.
High-Level Description
This GIP proposes a query fee rebate scheme that is based solely on the ratio between an indexer’s stake on a subgraph and the query fees that the indexer has generated serving queries for that subgraph. As the indexer stakes more (relative to the query fees), they are rebated a larger percentage of the fees. For a sufficiently large stake to query ratio, indexers receive essentially all of the query fees they generate.
With this proposed change, individual indexer staking decisions uniquely determine query fee rebates. This reduces the need for indexers to predict what other indexers will stake. We anticipate reducing this cognitive friction facilitates indexer efficiency and will ultimately lead to a better and more efficient experience for both indexers and consumers.
Exponential Rebates Formula
We formalize the proposal as follows. Let q_{ij} be the query fees generated by indexer i on subgraph j and let s_{ij} be indexer i's allocated stake on subgraph j. Then, the query fees rebated to agent i that were generated on subgraph j are
(\mbox{Rebate to i-th indexer for j-th Subgraph}) = R_{ij}(q_{ij}, s_{ij}) = \left(1-\alpha e^{-\lambda \frac{s_{ij}}{q_{ij}}}\right)q_{ij}
where 0\leq\alpha\leq1 and \lambda>0 are protocol parameters. Total query fees rebated to the i-th indexer over all subgraphs is
(\text{Total Rebate to i-th indexer}) =\displaystyle \sum_jR_{ij}(q_{ij}, s_{ij}).
Intuitively, (1-\alpha e^{-\lambda \frac{s_{ij}}{q_{ij}}}) represents the proportion of query fees served by the i-th indexer on subgraph j that is rebated to the indexer. Importantly, only the i-th indexer can claim query fees it generates and its rebate does not depend on the staking levels of any other indexer.
Based on assumptions of indexers’ required returns, we believe setting \lambda=0.6 and \alpha=1 ensures there will be enough stake to secure the network without putting too much of a staking burden on indexers. Using data beginning from the start of calendar year 2021, under this exponential mechanism less than 7% of query fees would have been burned as compared to over 50% under the Cobb-Douglas mechanism.
With this proposed change, individual indexer staking decisions uniquely determine query fee rebates. This reduces the need for indexers to predict what other indexers will stake. We anticipate reducing this cognitive friction facilitates indexer efficiency and will ultimately lead to a better and more efficient experience for both indexers and consumers.
Indexer Optimization Guide
A critical benefit of the exponential rebate mechanism is that indexers can directly compute how much they will earn in rebates for any level of stake. The figure below illustrates the returns as a function of stake ratio. The horizontal axis represents how much an indexer stakes relative to the amount of query fees it generated (\frac{s_{ij}}{q_{ij}}) and the vertical axis represents the proportion of query fees the indexer receives in rebates. With a stake ratio of 4, the indexer receives 90% of generated query fees in rebates. With a stake ratio of 6, the indexer receives 97% of generated query fees in rebates and with a stake ratio of 8, the indexer receives over 99% of generated query fees in rebates. For calendar year 2022, the average stake ratio was significantly greater than 1000 and 90% of the time indexers collected rebates, their stake ratio was over 1000. This implies that at current stake levels, effectively all query fees will be returned to indexers.
Protocol simplifications
Changing the rebates formula from the current Cobb-Douglas (CD) to the exponential rebates approach described by this GIP has nice side benefits in terms of simplifying the protocol. Most notably, rebate pools no longer serve a purpose; so, they can safely be removed. This implies allocation and query fee life cycles can be simplified.
With CD, the rebates an indexer gets depend on the global network state (other indexers’ states); both the total amount staked and total query fees collected at the protocol level have an impact in the rebates amount. In practice, this means query fees need to be periodically accumulated by the protocol, held for some period of time, then released to indexers following the CB formula. Rebate pools are the construct that support this, acting as “buckets” where query fees are collected until they are distributed to indexers.
Query fees currently go through the following lifecycle:
- An indexer calls
AllocationExchange.redeemwhich collects the query fee amount specified by a voucher into a rebate pool. The query fees are held by theStakingcontract and are associated to the allocation (subgraph) they were generated by. Fees can be collected while the allocation is inActiveorClosedstate. - An indexer calls
Staking.claim, which applies the CD formula on the rebate pool to calculate the rebate amount before transferring the rebate to the indexer. Note that this can only happen for allocations in theFinalizedstate which is defined asStaking.channelDisputeEpochsamount of epochs after the allocation was closed .
With the new exponential rebates formula, indexers can only claim rebates for query fees they generated themselves. Furthermore, the amount they get only depends on their own state, which is defined by s_{ij} and {q_{ij}}. This means query fees can instantly be rebated to indexers whenever they redeem their vouchers, thus removing the need for rebate pools.
In summary, switching to exponential rebates brings the following additional benefits:
- Rebate pools are no longer needed.
- Allocations states
ClaimedandFinalizedare no longer needed. - Query fees can be collected with a single transaction by calling
AllocationExchange.redeem. The secondStaking.claimtransaction is no longer needed. - The
Staking.channelDisputeEpochsparameter is no longer needed.
Exponential rebates with multiple gateways
Decentralizing the gateway is an ongoing effort for The Graph core dev teams. There are some extra caveats to consider if we think of a future with multiple gateways as part of the protocol. Thus, it is important for our proposed changes address this.
Suppose an indexer generates query fees for a given subgraph on multiple gateways {G_1,G_2,...,G_N}. When redeeming a voucher from gateway {G_k}, the query fee value that is used on the exponential calculation would not be the total amount of fees generated on the subgraph, {q_{ij}}, but rather the portion of those fees that were generated through gateway {G_k}: {q_{ijk}} where {q_{ij}=\sum_k {q_{ijk}}}.
In practice, since rebates are governed by the staking ratio \frac{s_{ij}}{q_{ij}} , and {q_{ijk}}<{q_{ij}}, indexers will need less stake to keep the same amount of query fees in a multi gateway scenario (or they will be pocketing a higher amount of fees for the same stake). Initially this might seem good for decentralization as it incentivizes indexers to run queries through as many gateways as possible to maximize rebates; however, in a fully decentralized world this means gateways could game this to bypass the rebates formula (splitting vouchers into smaller chunks can guarantee a very high staking ratio and, thus, guarantee rebates of 100% ).
This can be remediated by keeping track of distributed rebates ({D_{ij}}) and fees collected ({q_{ijk}}) on a subgraph basis, and using the accumulated amount of fees instead of the voucher fees for each successive rebates calculation. Any extra rebates that have been given out are deducted from the rebates to distribute.
The process for each voucher an indexer redeems is as follows (note that this applies to both multiple vouchers from different gateways or from the same one.):
- Update the accumulated fees collected: {q_{ij}}={q_{ij}}+{q_{ijk}}
- Compute R_{ij}(q_{ij}, s_{ij}) using the new value of {q_{ij}} instead of {q_{ijk}}
- Compute the delta of rebates to distribute as \Delta R = R_{ij}-D_{ij}
- Update the distributed rebates: D_{ij}=R_{ij}
There is a downside to this approach, which is that it makes it less intuitive to know the rebates amount beforehand. (Note, however, it is still completely possible to calculate the correct value using on-chain data.) An example of this is an indexer redeeming two vouchers for the same allocation with the same amount; the second redeem will net them less rebates than the first one (while the grand total will be correct).
Detailed Specification
Spec for staking contract changes
The staking contract will be upgraded to a new implementation containing the following changes:
- Add
lambdaNumeratorandlambdaDenominatorto contract storage. ForalphaNumeratorandalphaDenominatorwe will re-use the storage variables currently in use for the Cobb-Douglas formula. - Deprecate
rebatesstorage variable and updatecloseAllocation()function to remove rebate pools logic. - Remove
claim()function - Update
collect()function to perform operations that were done byclaim()- Calculate rebated fees
- Distribute rebates to delegators and indexer
- Re-stake rebated fees only if
rewardsDestinationis not set for the indexer
- Deprecate
channelDisputeEpochsstorage variable and remove associated setters/getters - Remove
ClaimedandFinalizedstates from Allocation life cycle, affectsgetAllocationState() - Add
distributedRebatesto theAllocationstruct to keep track of rebate distribution across multiple voucher collections.
Spec for rebates staking lib
The rebates library for the staking contract will be updated:
- Remove Cobb-Douglas implementation
- Update
redeem()to calculate rebates using exponential formula described in this GIP.
Backwards Compatibility + Upgrade Path
The switch to exponential rebates in itself is not significant as it only modifies how rebates are calculated, which is a self-contained upgrade that does not affect contract state. However, the removal of rebate pools presents a notable change from how rebates operate today, one which makes this upgrade not backwards compatible. Once the upgrade is completed indexers will not be able to claim rebates from old rebate pools. Due to this, some care must be taken to minimize potential loses for indexers and make the transition smoother.
We propose making both changes at once, i.e. directly switch to exponential rebates and remove rebate pools in a single upgrade. This greatly simplifies the operational overhead (orchestrating ops, communication with indexers, code audits, etc.) when compared to a potential two stage upgrade (first one to switch rebates to exponential, second one to remove pools after a grace period). This presents some transitional challenges described below.
Unclaimed open pools
Today there are roughly 38,000 open pools with rebates waiting to be claimed. These most likely remain unclaimed due to the rebate amount being so small it’s not enough to cover the gas cost for the claim transaction.
We anticipate most indexers will forfeit these fees as they should, for the most part, be insignificant for them. A timely communication of the upgrade plan will ensure indexers have enough time to evaluate wether or not they want to claim any residual fees before it’s too late.
Pools caught in transition
There is another situation where pools/allocations may be caught in between the transition and, thus, lost. Fees cannot be claimed from a rebate pool until the allocation they are being claiming for is in the Finalized state. Currently this takes ~7 days after the allocation is closed (defined by Staking.channelDisputeEpochs). Indexers will lose any collected fees if the upgrade hits during that time window as the claim function will be removed before they can reclaiming the rebates.
To minimize this risk, we propose reducing Staking.channelDisputeEpochs from the current ~7 days to ~1 day, this will give indexers more flexibility so they can plan their query collecting/claiming accordingly and minimize loses. With a clear upgrade date on sight, indexers can close allocations up to one day before the upgrade and have enough time to claim any collected fees as long as they don’t collect any new fees during that last day.
Re-staking flag removal
An additional consideration for indexers is re-staking behavior when collecting rebates after the upgrade. In order to keep the same Staking.collect interface (which avoids having to upgrade the AllocationExchange contract) rebated fees for indexers will automatically be re-staked unless there is a rewardsDestination set. This mimics the way indexing rewards re-staking works.
Since rewardsDestination will control both indexing rewards and query fee re-staking behavior, indexers wanting to re-stake one but withdraw the other will need to toggle the rewards destination address before/after their collect or closeAllocation calls, which can be done using multi-call. For example, to have indexing rewards re-staked but query fees withdrawn to an address:
- Set rewards destination address by calling
setRewardsDestination(withdrawAddress) - For rebates collection just call
collectand rebates will be transferred towithdrawAddress - For indexing rewards re-staking, indexer will need to perform the following multi-call:
setRewardsDestination(address(0))closeAllocation(...)setRewardsDestination(withdrawAddress)
Dependencies
None.
Rationale and Alternatives
As partly noted above, the proposed mechanism
- simplifies indexer actions (with respect to what must be done to collect fees),
- simplifies smart contract code,
- requires minimal changes to graph node code,
- reduces indexer cognitive load, and
- increases indexer revenues without burdening consumers (by reducing burnt query fees).
Numerous alternative mechanisms were considered. One of these has a similar functional form to Cobb-Douglas today, which prevents some “taking of funds” dynamics of whales from small indexers. However, this scheme still suffers from other forms of gaming, has a high cognitive load, and burns a nontrivial amount of query fees.
Another considered alternative is similar to the proposal, but with a different functional form. That scheme has a hard constraint on required stake ratio. If the stake ratio is above the set threshold, all query fees are returned. If the stake ratio is below the threshold, then the amount of query fees rebated would equal the division of the stake by the required stake ratio (i.e. less than 100% of queries would be returned). This scheme is not preferred due to the “sharp” nature of its rebate effects and its sensitivity with respect to the protocol parameters.
Risks and Security Considerations
Due to the simple nature of the proposed change to exponential rebates we do not anticipate any notable risks. Before deployment, all relevant smart contracts will be audited by trusted third parties. The following table registers some risks we identified related to the removal of rebate pools:
| Risk | Impact | Likelihood | Criticality | Mitigation |
|---|---|---|---|---|
| Indexer does not claim their open rebates before the upgrade | Rebated fees are lost. | High | Very Low | Upgrade date to be communicated with enough time for indexers to claim any open rebates. |
| Indexer cannot claim rebates of pools caught in transition (for allocations not finalized) | Rebated fees are lost. | Medium | Medium | Reduce Staking.channelDisputeEpochs to ~1 day to provide more flexibility. Promptly communicate upgrade date so indexers can plan accordingly (not collect fees they cannot claim). |
Future Directions
None.
Copyright Waiver
Copyright and related rights waived via CC0.