Why is there billing from the first query?

Questions & Feedback
1

I am a fairly new developer of DeFi, coming from the Web Dev world.

I’ve been going in circles for weeks, trying to figure out how to query for UniV3 Liquidity Pairs given a pair of tokens… a task quite trivial with UniV2 and the factory contract… After weeks of asking and asking around, I concluded that the only feasible way is through using thegraph protocol via GraphQL queries.

I have now realised that in order to perform even a single query on TheGraph, I am required to pay in GRT.

This has perplexed and confused me in multiple levels…

  1. I know and understand there are costs involved in maintaining this infrastructure. Everyone knew this when TheGraph started and positioned itself as the “single-source-of-truth” for many queries… Is this now a matter of TheGraph leveraging their unique position in expense of the developers?

  2. Why aren’t DAOs participating in the costs instead of collecting fees from the Developers?

  3. How are the DAOs actively supporting this business model, milking the people who put their work in to make tooling, applications and services for DeFi?

  4. Why isn’t there a common sense tier system like most commercial services have? Where they offer a free tier for up an amount of resource usage and beyond that billing starts, something sensible?

I could go on and on with issues and trying to understand justification, but for this to be so widely supported and noone raising the issue, I must be missing something fundamental…

So I am prepared to get embarrassed with what I am missing here…

1 Like
2

The Graph Network is technically a paid service, you pay for the queries you run through the network.
There’s still the “free” service, which is what was known as the “hosted service”, which is still functioning (probably where you were querying the UniV2 data) and will still be available for some time, as far as I understand.

Whether there’s a UniV3 deployment on the hosted service that you can query for free I’m not sure, you could try reaching out to the Uniswap team. As far as I understand, the Uniswap V3 on the The Graph Network wasn’t deployed by them, but it’s using their code :thinking:

EDIT: There’s actually deployments on the hosted service, which are linked in the uniswap official documentation
Uniswap official docs for subgraph data: Uniswap Subgraph Data | Uniswap
Uniswap V3 on hosted service: https://thegraph.com/legacy-explorer/subgraph/uniswap/uniswap-v3

1 Like
3

Thank you Juanmardefago for the reply.

The issue I posted appears as “two-fold” but it really isn’t. It appears as if I have a technical problem in fetching data for UniV3. But that’s not really the case at all. Thank you for the provided resources, though, that helps.

The real issue here is the politics involved in the decision making to move the cost of running this service (thegraph) to the end-users (developers).

In a world where the entire global industry, horizontally, vertically and through any slice or segment you wanna cut it, are subsidising development adoption and are thirsty for talent, thegraph is effectively doing the exact opposite by taxing application development and platform (of DAOs) adoption.

Let’s be clear on what is going on here, the main beneficiary in this service are the DAOs. The whole premise of this project (TheGraph) is to make information more accessible. The mission statement of TheGraph is:

“The Graph’s mission is to make serverless applications possible and to make building on Web3 accessible to anyone.”

How does the decision to raise a paywall on the data bode with the mission statement and the fact that the main beneficiaries are the DAOs?

Don’t get me wrong here, I have been in the tech industry for over 20 years and I know what the costs are and what paying bills is and means. And that’s why I can confidently say, that the costs of the entire TheGraph’s operation can be covered by DAOs and they wouldn’t even flinch at the cost.

This isn’t about me being frugal, this is about the wrongs I see on DeFi as a new engineer trying to ramp up. This one takes the cake because it’s not “just another blockchain indexer”. It is 100% supported and exists because of, the entire DeFi community.

It is the DAOs that put in very significant resources to build the schemas and how the graph will populate, and it is with these resources that “TheGraph” is what it is today.

Deciding to move the costs of TheGraph infrastructure and operations to the developers while at the same time DAOs are yearning for developer adoption and application development, is quite an interesting position.

1 Like
4

This is a really good question. There should be a freemium tier for devs to just play around with. Thanks for raising this point.

5

I think this is great feedback, and I appreciate you taking the time to share the insight. I don’t have anything to add to the discussion at this point, but I wanted to leave a comment here so that I can follow along/come back if I had something to add.

6

Hi @thanpolas! I think you raise quite a few interesting points here.

Firstly - can you clarify who / what you mean by “DAOs”? There are quite a lot of types of organisation building on Ethereum, and I don’t know if it makes sense to treat them all as one group.

When you say “the costs of the entire TheGraph’s operation can be covered by DAOs and they wouldn’t even flinch at the cost”, I think that is slightly at odds to the overall approach of the network. Firstly, there isn’t one single infrastructure provider - there is a network of indexers who are incurring cost to index and serve subgraphs. The Graph Protocol aims to create a self-sustaining ecosystem where indexing and serving subgraphs is a sustainable business for indexers, while providing a robust & affordable solution for developers. Right now the bulk of the compensation for indexers is indexing rewards, but query fees are a key aspect of the network design. Is your suggestion that all query fees to indexers should be paid by DeFi protocols, or some larger shift in the overall model?

You say that “It is the DAOs that put in very significant resources to build the schemas and how the graph will populate, and it is with these resources that “TheGraph” is what it is today”, but actually it is more egalitarian than that. It is true that some of the most-used subgraphs were developed by large Defi protocols, but there are thousands of subgraphs built for specific dapps by subgraph developers, and that customisability and variety is one of the real strengths of The Graph. Ethereum is a diverse & interesting place, and so subgraphs have to be diverse and interesting too.

To address you primary point, there are a bunch of participants in The Graph ecosystem, and a few types of developers. Subgraph developers build and maintain subgraphs, Dapp developers build applications on top of those subgraphs (and of course often developers will wear both hats).

It is true that the network introduces costs to both types of developers that weren’t there before.

Subgraph developers now have to pay transaction costs to publish their subgraphs to the network, they may have to lock up some GRT as signal, and may have to pay GRT to upgrade their subgraphs. But as a subgraph developer just starting out, you can deploy your subgraph to the subgraph studio, where it will get indexed by some Edge & Node indexers for testing and development purposes, before publishing to the network. It won’t work for production-scale dapps, but it is like the free tier suggestion from your original message.

However it is the case that on the network, Dapp developers who didn’t develop the subgraph they want to use (which I believe is the category you are in) now have an upfront cost from the first query outside the explorer, which is a valid concern and real barrier to entry. I think there are potential solutions to this (a faucet, some kind of community-subsidised limit) which it would be great to explore.

2 Likes
7

Hello Adam,

thank you for taking the time to give a thorough reply. Let me start by answering your specific questions:

can you clarify who / what you mean by “DAOs”

As you correctly mentioned later, I am referring to all the DeFi protocols that participate in the subgraph.

Is your suggestion that all query fees to indexers should be paid by DeFi protocols, or some larger shift in the overall model?

Yes, all fees to indexers should be paid by DeFi protocols. And that is not “at odds to the overall approach of the network”.

TheGraph’s [new] role is to collect and distribute the fees and create and enforce the fee policies. So, there is a centralized entity taking care of that, albeit it can be a DAO itself (I am not sure to what degree, as I said, I am new to the space).

So, it’s just a decision to be made. That decision, is even more easy to be made considering the points you make further down your reply:

Subgraph developers build and maintain subgraphs, Dapp developers build applications on top of those subgraphs (and of course often developers will wear both hats).

So, effectively, the biggest percentage of the cost, is already handled by the DeFi Protocols.

My point and belief still stands:

“the costs of the entire TheGraph’s operation can be covered by DAOs and they wouldn’t even flinch at the cost”

And that would be fair, as the DeFi protocols are the main beneficiaries in this service. Paywalling the service, doesn’t really serve anyone, nor thegraph’s mission itself. Nor does it help new protocols that are trying to get a footing on the space, to be taxed by the moment they want to submit their first subgraph. A subsidy has to exist there too if we care for the ecosystem at large.

I understand that big consumers of thegraph, outside the DeFi protocols, should weigh in the costs too, but that’s a good problem to have, when it presents itself. So far, as I understand it, the vast majority of the users of thegraph, are the DeFi protocols themselves to feed their frontends. Correct me if I am wrong here.

There are some other technical problems, like how it is not really fair to charge by the query when you are dealing with GraphQL (not all queries are equal), or the choice of a highly volatile token as the main unit of payment, but let’s not get technical at this point.

The issue at hand is of a larger, political scale, and requires big moves before you get to the nitty gritty.

8

I am more in line with what @adamfuller is presenting than what you are presenting @thanpolas and I’ll try to explain why.

Adam is putting forth the idea of a faucet or community-subsidized initial amount of queries. When reading your initial post, this was my first thought as well. The devs still get their initial usage covered so that they can ramp up and develop their PoC. From there, it can either be shown to be relevant to the point of making sense to have a bit of funding put in by the dev themselves, monetized through the dapp(s) it is serving data to, a grant can be requested if it is a communal resource, or perhaps it can be shopped around to be funded by community members or other dapps.

By saying that you think DAOs of DeFi protocols should bare this cost is (to me) perhaps a bit too short sighted. What if next month NFT platforms are the real consumers of The Graph, but DeFi DAOs are paying for it to run? And then a year later another “hot” area explodes and begins to be the biggest consumer? I don’t think the DeFi DAOs would be too happy to be taking on the cost, and then any new major consumer market would have to be approached for funding, etc.
This would also make The Graph beholden to the needs/wants/pivots of those marketplaces. If DeFi protocol X wanted certain features modified because it would fit their needs better, and they are one of the main funders, then the autonomy of The Graph would be getting compromised a bit.

The Foundation and The Council have a position to steer the protocol as a whole towards serving the widest market, as well as is possible (or else fear that The Graph would not maintain its relevance and utility to devs). Once you start to have a narrower segment with outsized control, this dynamic shifts.

This was perhaps a long winded way of saying that the value of autonomy and self-funding goes very deep, and getting funding from deep pockets who “wouldn’t flinch at the costs” could put things at odds with the mission.

Note: in case my response comes off as a bit flippant, it is not meant to be at all. I fully agree with you that this issue needs to be addressed and I am really happy you brought it up!

6 Likes
9

Thank you for the thoughtful post Josh,

First, let’s get the faucet thing out of the way. It adds friction and overloads complexity, on an already overcomplicated system. From a dev UX perspective it’s just horrendous, complicated and doesn’t really make sense. You might as well give a few GRT coupons, only spendable on thegraph, to each developer account / API key on a monthly basis (otherwise known as “a free tier”).

Now, in regards to the DeFi protocols paying for the service. I don’t understand the point you are making:

What if next month NFT platforms are the real consumers of The Graph

Well, then, it’s the NFT platform that should pay. What am I missing here? I know it can be tracked in detail, which graphs attract the most queries, and split the costs fairly this way.

I don’t think the DeFi DAOs would be too happy to be taking on the cost, and then any new major consumer market would have to be approached for funding, etc.

Has thegraph actually asked the DAOs? Because I am pretty sure (I actually know), if we make the case, it is a pretty compelling one. You’d be surprised to find that DAOs yearn for developer adoption and development. They even pay big money for it, through grants.

This was perhaps a long winded way of saying that the value of autonomy and self-funding goes very deep, and getting funding from deep pockets who “wouldn’t flinch at the costs” could put things at odds with the mission.

The “mission”, is already at odds with the current state. Let me recap, so we have a clear case:

  1. The Graph has raised a paywall on data that was primarily created out of DAOs work (some have actually paid huge funds for the graphs to be created).
  2. The Graph arbitrarily and unilaterally, chose to use GRT as the token of billing.
    1. That’s simply said, quite a self-serving decision.
    2. As a business, I don’t know what my expenses will be as they depend on a volatile asset. One month I am paying $10k, the next month for the same service I am paying $12k. You can’t expect serious business adoption on this premise.
  3. The unit of billing, the query, is an unfair unit as it favours the advanced engineers that know how to combine and aggregate multiple GQL queries into a single call at the expense of less experienced or simply un-optimised applications.

So, whose “autonomy” exactly are we referring to in this context? DeFi’s or TheGraph’s? If it’s the graph’s autonomy… what exactly does the word mean in this context? What is the graph? Is it a DAO? Is it a business? There is no suggestion that the costs of the service should not be paid for.

I understand there is an attempt to promote “decentralization”, but the reality is, what has been decentralized is simply the “backend” (indexers) of the graph service, a clever way to pass-on infrastructure costs.

The Graph service, is very much centralized as “the service” itself goes through a single, aggregating, gateway and unique domain name. That’s a far cry from having an actual decentralized service.

To conclude, what I am suggesting is, that the graph, touches base with its constituents and stake holders. Hear them out, and together formulate a strategy and policies to move forward.

I am speaking on behalf of noone but myself. I just know what I know, and I know with high confidence, that the current state of things, is wrong from multiple points of view. Points that I’ve made through the course of this and the previous posts of mine.

I am positive the graph will realise that this is an unsustainable way forward and change course.

With respect,
Thanos

10

Hello! Thanks for the follow-up @Josh-StreamingFast and @thanpolas.

I am conscious that the scope of the discussion has gone from quite a focused consideration of UX and barrier-to-entry (i.e. how can we make sure that developers who want to query subgraphs can do so) to a larger consideration of the overall architecture of the protocol.

To stick on the initial topic, as I said before I agree that this is something that we should address. I agree with you that faucets are not a great UX, so something that looks more like an automatic “free tier” could definitely make sense, and I think there is some interesting thinking to be done there.

As you mention later on, the gateway is currently a centralised service, so a short-term “free-tier” solution could be quite easily achieved via the gateway. However to address the reason you brought it up (“The Graph service, is very much centralized”) - the current architecture is not the end-state for that reason, as the overall approach is one of progressive decentralisation over time. For this reason, solving the “subgraph consumer barrier-to-entry” problem in the gateway is maybe not a desirable approach.

To quickly jump back to some of your broader points:

  • It seems that you would generally prefer the subgraph creators pay for query fees, rather than consumers? (i.e. “I know it can be tracked in detail, which graphs attract the most queries, and split the costs fairly this way.”) Subgraph creators have already invested time and money in creating subgraphs (as you point out), so I am interested that you think they should pay even more if their subgraphs are widely used, and wanted to verify that was your recommendation?
    On that point, with the current gateway architecture, it is the case that developers can and do pay for query fees on their subgraphs (see billing: Billing on the Subgraph Studio | Graph Docs), though for the most part that is being paid for end users of their dapps, rather than for other developers to query the subgraphs for their own purposes.

  • The use of GRT as a utility token in the protocol is pretty core to the overall protocol design. Obviously GRT price volatility does introduce some challenges, but that is not a unique problem to The Graph protocol - the same is true of Ethereum itself of course (where the 20% price move you give in your 10 to 12K example happened in the last 7 days). I am interested in what your preferred approach would be? My personal perspective is that there are interesting solutions to be built managing volatility and costs between tokens for web3 developers in general.

  • Whether the query is a good unit of billing is certainly an interesting topic, though it is a pretty common pattern by which to pay for APIs. You describe it as an unfair unit, partly because it penalises less optimised applications, but that in particular would seem like a good thing (subgraph consumers should be incentivised to build optimised solutions).

On your final recommendation: “To conclude, what I am suggesting is, that the graph, touches base with its constituents and stake holders. Hear them out, and together formulate a strategy and policies to move forward.”

As a community we try to stay in touch, either here on the forum, or on Discord, or during the Monthly core devs meetings. I certainly welcome you to join the next core dev meeting in the first week of next month. As I said at the start of this message, it seems that you have misgivings that are deeper than simply wanting to be able to make some queries without paying GRT. Like all web3 protocols, The Graph Network is a work in progress, and can certainly improve - see this thread for information on the GIP process: GIP-0001 and Getting Started with GIPs, GRPs, GRCs, etc. Would you be able to summarise your suggested change as a GIP, even if very briefly? That would give the community something more concrete to react to than this forum thread, which by its nature is more discursive.

Thanks again for taking the time to reflect on The Graph Protocol!
Adam

2 Likes
11

Correct me if I’m wrong, but couldn’t a DAO could sponsor a subgraph essentially by creating an API key and plugging it into an API on their end, thus all traffic from their API to the subgraph would go through their account? Otherwise it is up to DApp whether or not it makes sense for them to cover the query fees.

Presumably with the current price model (which is established by each individual indexer), I would imagine this route would be a more cost efficient route than having to host your own archive nodes to query from. In fact, I think smaller 3rd party DApps probably benefit the most from this.

12

Thank you for the elaborate response @adamfuller,

The topic got broadened as I had to make wider approach, to support where I am coming from and my motivation on the subject.

I am happy to learn that this is not the “end-state”, could you point me to resources where I can view the “master plan” to decentralise the service?

It seems that you would generally prefer the subgraph creators pay for query fees, rather than consumers?

Yes, that is what I am suggesting. Consider this:

“thegraph” does not exist. Yet the protocols need to have this service. They’d implement and host it on their own, taking up all the implementation and hosting cost. Wether they’d consider monetisation of that service, is a second order problem, and most likely they would offer it for free, much like it already is happening with protocols that offer their own APIs to access indexed and real-time data (1inch comes to mind).

Again, it is the protocols that are the beneficiaries in having developers build applications using the protocols’ data as well as populating their own frontends.

The use of GRT as a utility token in the protocol is pretty core to the overall protocol design.

It was made to be core. Preferred approach? A stablecoin of your choice.

In regards to the Query as a unit of billing, yea, it is a big subject and probably shouldn’t be addressed in this context.

There are no misgivings on my part, I believe I’ve been courteous and respectful in my remarks. I was indeed frustrated with what I discovered but that has neither been expressed nor it still stands as I found different solutions to my problem.

I cared enough about the subject to spend all this time to write all of these posts. I trust the topic has been exhausted at this point.

Wether I join on thegraph’s governance, is a different kind of engagement, that I don’t intend on taking on at this time.

I hope my comments are received in good faith and will help the graph protocol be an excellent service and contributor to the DeFi ecosystem.