Edge & Node to spin up an "Upgrade Indexer"

General Ecosystem Updates
,
1

Dear community,

we wanted to let you know that, over the next couple of months, Edge & Node will be spinning up what we call an “upgrade Indexer”. This upgrade Indexer will act as a fallback for subgraph queries that cannot be handled by other Indexers in the network. It will support many of the chains that are today only supported on the hosted service.

What does upgrade Indexer mean?

This Indexer will help bootstrap chains on the network and improve the experience of upgrading subgraphs from the hosted service by ensuring that there is an Indexer ready to serve queries right after a subgraph or a new subgraph version is published to the network.

To achieve this, the upgrade Indexer will support many of the chains that are only supported on the hosted service today, and will have many of the hosted service subgraphs already synced.

What does this mean for the Indexer community?

Chains that are currently only supported on the hosted service will soon be available on the network as well. We expect this to lead to more subgraphs to be published on the network, available for Indexers to index and serve in return for query fees.

Our upgrade Indexer will allocate only the minimum amount per subgraph and will not be collecting indexing rewards, only query fees for queries served. This is to avoid being competitive with other Indexers in the network. We expect the upgrade Indexer to be a temporary participant in the network until the respective chains and subgraphs are sufficiently serviced by other Indexers. For chains and subgraphs that are supported by enough Indexers with a high quality of service, the upgrade Indexer is merely a fallback.

We welcome Indexers to join us in supporting the process of upgrading more subgraphs to the network. To make this as easy as possible, we will publicly share a list of relevant chains and subgraphs in advance, to enable the Indexer community to pre-index them as well, and help make switching to the network a smooth experience for everyone.

What does this mean for subgraph developers and data consumers?

Once the upgrade Indexer is live, subgraphs that are upgraded from the hosted service or published from Subgraph Studio can be queried on the network right away, with no lead time being required to index them first.

The upgrade Indexer will also enable chains on the network that are only supported on the hosted service today or that don’t attract enough Indexers due to a lack of indexing rewards.

As we enter the sunrise of decentralized data, the upgrade Indexer isn’t just a safety net, but the bridge that will enable all subgraphs to upgrade to The Graph Network and take us one step further in making The Graph the Internet’s unbreakable foundation of open data.

12 Likes
2

The Graph has continually made progress on decentralization since the launch of The Graph Network. That said, one thing we cannot ignore as devs in the ecosystem is the fact that the majority of query traffic today is happening on the hosted service. If it is a hosted service, it is not web3. We are 100% committed to getting all of that traffic supported fully on the network as quickly and as smoothly as possible – this is our north star.

We spent a lot of time iterating on the best possible plan to realize this north star, and the upgrade Indexer is the first step towards reaching this objective. This plan was developed to serve the network and thereby Indexers: more traffic on The Graph Network is a win for the Indexer community, representing a new wave of network users and query fees. Additionally, the upgrade Indexer will bring further transparency to the network, making it easier to see which chains have the most highly-used subgraphs, and should therefore help inform which chains should pass through the governance process for full network support (i.e. indexing rewards) more quickly and transparently.

To double click on Jannis’s point above: the upgrade Indexer is not a competitive participant on the network, but rather a facilitator to empower subgraph users to upgrade to the network, while also serving as a fallback mechanism.

The indexer community on The Graph Network should be the backbone to web3, not any hosted service, not the upgrade indexer.

We are committed to ensuring The Graph is the internet’s unbreakable foundation of open data to create a world powered by individual autonomy, shared sovereignty and limitless collaboration – and we’re excited and grateful to be on this journey with all of you.

11 Likes
3

How is the “upgrade indexer” different from the existing “backstop indexer”?

1 Like
4

The strategy appears to be solid, and we are fully embracing it. The introduction of this special indexer will facilitate a smoother transition for subgraph users and allow decentralized indexers ample time to catch up on indexing and bolster their infrastructure. While we’re relying on E&N’s assurance that this special indexer won’t create unfair competition, there’s no reason to doubt their word at this point.

Any additional transparency from E&N regarding current subgraph traffic and migration plans would help indexers prepare more effectively. While this information has been closely guarded in the past for valid reasons, we may be nearing a pivotal moment in subgraph migration where sharing a bit more information with the community would be beneficial.

8 Likes
5

The introduction of this special indexer will facilitate a smoother transition for subgraph users and allow decentralized indexers ample time to catch up on indexing and bolster their infrastructure.

We don’t see it as so special but we also didn’t want to spin it up without letting you all know. :slight_smile:

Any additional transparency from E&N regarding current subgraph traffic and migration plans would help indexers prepare more effectively. While this information has been closely guarded in the past for valid reasons, we may be nearing a pivotal moment in subgraph migration where sharing a bit more information with the community would be beneficial.

I fully agree. We’ll share future plans for upgrading more folks to the network as well as the chains and subgraphs included in the upgrade Indexer with plenty of lead time.

When it comes to subgraphs on the hosted service, our thinking is to share a live list (or table) of those that are public—with meta information like chain name/ID, entity count, database size, query volume—to help Indexers decide which of them they want to pick up.

7 Likes
6

How is the “upgrade indexer” different from the existing “backstop indexer”?

The “backstop Indexer” that we run also acts as an altruistic fallback for when there are no other suitable Indexers available to serve a query, but it only indexes subgraphs after they are published on the network. It also operates only on L1.

The “upgrade Indexer” most likely will run only on L2. The main difference is that it will support additional chains and that it will have subgraphs from the hosted service pre-indexed to ensure immediate availability on the network if and when these subgraphs are published.

2 Likes
7

Thanks for clarification. Is all documentation being updated to explain all this?

1 Like
8

As we know, the network has better availability numbers than the hosted service today. When the upgrade indexer comes online will that show that the network performance has decreased (all other things being equal). What is the plan to communicate this drop on overall performance? Will we publish dashboard with and without the upgrade indexer?

2 Likes
9

Thanks for clarification. Is all documentation being updated to explain all this?

Not yet, but when our plans become more concrete, we’ll make sure that hosted service users understand that switching to the network will not incur a waiting period before they can use their subgraphs there.

As we know, the network has better availability numbers than the hosted service today. When the upgrade indexer comes online will that show that the network performance has decreased (all other things being equal). What is the plan to communicate this drop on overall performance? Will we publish dashboard with and without the upgrade indexer?

I wouldn’t expect the network performance to decrease, especially if other indexers help supply the most used subgraphs.

2 Likes
10

How long will the upgrade indexer take before it’s live?

Which subgraphs are you going to prioritize pre-indexing?

Does this address current pain points for the state of the indexer migration? Maybe I don’t understand, but it seems that the pre-indexing is the only potentially ‘helpful’ thing to other indexers.

Can other indexers pre-indexing subgraphs as well?

1 Like
12

Soon-ish :wink:

Most, perhaps all subgraphs that are on the hosted service today.

Can you elaborate what you mean by “state of the indexer migration”?

Yes, see my earlier reply: Edge & Node to spin up an "Upgrade Indexer" - #5 by jannis

4 Likes
13

Hi everyone,

we promised that we would share data for public hosted service subgraphs that the upgrade Indexer will be serving ahead of time, so that any Indexer can proactively pre-index these as well.

As Indexers, you can now apply for access to this data via the following form: Apply For Access To Hosted Service Subgraph Stats.

Note that we’ll end the application process on Dec 1, so be sure to fill it out by end of November.

indexer

3 Likes