File Data Sources are live on Graph Node v0.30

Introducing a highly-requested feature for NFT devs: File Data Sources!

NFT developers are now presented with an exciting new opportunity to enhance their dapps. File Data Sources change the way NFT metadata is queried, allowing for the retrieval of multiple data sources simultaneously. In other words, now you can build subgraphs that retrieve IPFS file metadata!

One of the challenges faced by NFT developers has been the slow retrieval of data from IPFS, which has previously caused indexing to halt until the requested file was found and retrieved. This often takes several minutes. With the release of Graph Node v0.30, parallel retrievals and retries have been made possible, improving indexing performance and allowing for a smoother experience for users of NFT dapps.

File Data Sources is available to users of the Subgraph Studio, offering a new level of capabilities for NFT developers. This feature is a significant step forward in the development of NFT technology, and its implementation can bring many benefits to NFT dapps. NFT developers are encouraged to take advantage of this opportunity and start using File Data Sources in their subgraphs and queries today.

Learn more in the announcement tweet.

7 Likes

Hello @Brian! Is there a resource where one can understand how do we integrate these File Data Sources using SubStreams?

1 Like

Also @Brian , when is the IPNS support planned extension to file data sources expected to be launched?

2 Likes

hey @nicog.

There isn’t currently a way to use File Data Sources with Substreams-powered subgraphs as Substreams-powered subgraphs stream entity changes directly into the Graph Node store, but we are working on some enhancements to Graph Node which may change that. Do you have a specific use case in mind?

Similarly for IPNS support - support for Arweave files will be added in the upcoming Graph Node release (0.33.x), but IPNS introduces some additional complexity as a file may change. Similarly, if you have a specific use case in mind that would ensure that we take that into account.

2 Likes

Thank you so much for your response @adamfuller! Im really looking forward to have IPNS added to the next Graph Node release. However, it is indeed more complex than adding IPFS hashes as a file source so I would love your guidance.

My use case is the following: I would like to have entities within my subgraphs storing values of IPNS names (instead of IPFS hashes). The IPNS name would point to a directory which could have both metadata files (JSON) as well as regular files so it would be ideal if the mapping (given the fact that you say substreams won’t be able to do it) could index this IPNS name and access its files?

The reasoning behid this is that given an IPFS hash I could be able to identify what’s the IPNS name that is referencing it.

Does it make sense?

2 Likes

Hi @nicog - it’s Arweave which is included, there isn’t yet support for non-deterministic file data sources.

Thanks for explaining your use case - for IPNS, as the referenced files can change, how would you ideally expect the data to be refreshed? And how many IPNS names would likely be involved?

1 Like