[Proposal: seeking feedback] Optimizing the experience for Subgraph Developers through user research

Hi all, I’m Georgia Rakusen, co-founder at OpenUX. We have submitted a grant proposal and are seeking some feedback from the community here.

Project summary
This proposal is related to the New Era announcement, and the focus on improving developer experiences.

The success of The Graph ecosystem relies on the ability for developers to successfully deploy widely adopted projects with subgraphs, and for those subgraphs to become beneficial to other projects. Understanding the experience of developers at different stages of their The Graph engagement is vital for the organization to deliver the right resources, tools and support - at the right time - so that projects can be ultimately successful. Projects which have the right kind of support are able to effectively execute their plans and become widely adopted, and the success of these individual developer projects results in the success of the The Graph ecosystem as a whole.

Over 10-12 weeks, OpenUX will conduct an in-depth discovery user research study on the experiences of a) web3 developers brand new to subgraphs, b) those who have recently begun creating their own and maintaining them, and c) those developers on successful projects already well versed in subgraph APIs, to understand the goals, needs, pains and behaviors of these different cohorts. This qualitative research will comprise in-depth 1-1 interviews with profiled and screened participants. The data will be analyzed, and a report published which contains both tactical and strategic recommendations to The Graph, to support the mission of growing the ecosystem.

Following the delivery of the report, we will activate external stakeholders in The Graph community to engage with the findings and ideate novel solutions for some of the challenges and opportunities identified.

Breakdown of milestones and activities
The research study comprises 4 main phases or milestones, with a total duration of approximately ~10-12 weeks. For all phases, at least 2-3 team members will be working on the activities. Georgia Rakusen will be lead researcher across all activities.

Milestone 1: Study Design (~2-3 weeks, e.g. Jan 8-Jan 26 2024)

Stakeholder consultations and planning

  • OpenUX will run workshops with the wider community and interview relevant stakeholders from within The Graph core team to gather assumptions and research questions to test. These will inform the study design.
  • We will conduct our own internal onboarding to simulate the experience of new developers, mapping the existing user journey from awareness all the way to deployed projects.
  • We will drive awareness of the research study across the ecosystem to support understanding of the objectives, seek new input, answer questions and build trust in our approach.

Study participant recruitment

  • Finding the right participants is essential to the success of the study. We will write screeners and publish these on social channels, as well as utilize existing recruitment platforms such as web3ux, and utilizing personal connections.
  • We will ask The Graph core team to support us in this effort by sharing our screener across their network (you have a relationship with some of these developers already and official promotion of the research builds trust and increases engagement).
  • Participants who complete the screener and are suitable will then be invited to book an interview with us.
  • We will offer a paid incentive (~$150) for every participant who completes a session with us.

Creation of interview guide

  • Using the outputs from the stakeholder consultations and planning, we will write an interview guide that will be used in the 1-1 interviews. This will address the main research questions and assumptions, and be written in a manner which avoids bias.

Milestone 1 budget: $8,250

Milestone 2: Data gathering (~2-3 weeks, e.g. Jan 29-Feb 16)

In-depth interviews with ~20 profiled and screened participants

  • In-depth interviews will take place over video call. These sessions will be recorded. An experienced user researcher will facilitate the interview, with internal stakeholders at Edge & Node invited to observe the sessions.
  • All sessions will be recorded, and made available for internal use only at The Graph.
  • Participants who complete a session with us will be paid ~$150 at the end of the call, via crypto or PayPal.

Initial synthesis of data

  • Recorded interviews will be uploaded to a GDPR-compliant research repository, for transcription and initial review.

Milestone 2 budget: $16,200 (including $3,000 for participant incentives)

Milestone 3: Analysis (~2-3 weeks, e.g. Feb 19-March 8)

Fine comb analysis

  • Transcripts and recordings will be coded, compared and collated to identify the answers to the research questions and assumptions, and to identify the “unknown unknowns” - new insights that were unexpected but highly relevant to the ecosystem.

User journey mapping

  • Our assumptions-based user journey map created in Milestone 1 will be adjusted and refined based on the real user data accumulated.

Creation of report and recommendations

  • We will produce a professional report detailing all the findings, including the answers to the research questions and assumptions, novel insights, and recommendations. It will include (anonymized) participant quotes, and an up-to-date user journey map. Anything that identifies individual participants will be stripped from all reporting.

Milestone 3 budget: $10,725

Milestone 4: Integration (~2-3 weeks, e.g. March 11-March 29)

Final report and recommendations published

  • OpenUX will host the report making it freely available online.

Facilitation of workshops for internal stakeholders

  • OpenUX will run multiple ‘working with the findings’ workshops for internal stakeholders, answering questions about the study, and helping the team prioritize the actions.

Community engagement

  • Great ideas come from many quarters. There are many ecosystem contributors (including grant-funded community projects outside of The Graph) trying to improve the experiences of developers, who are well placed to ideate on solutions from the research insights. OpenUX will facilitate these conversations and workshops.

Continued awareness-building

  • We will write a series of accompanying blog posts which break down the research report and dive deep into specific problem areas and solutions.
  • We will apply to speak at The Graph-affiliated and other blockchain events to disseminate the learnings from the study and raise awareness of the efforts of The Graph to elevate the web3 developer experience.

Milestone 4 budget: $5,500

Total budget for entire project: $40,675.00 USD

How we will deliver impact
Our research will provide much-needed robust insights into the experiences of developers, and will provide a source of independent truth that gives voice to the ‘silent majority’ of developers (as opposed to the loud minority who will actively reach out on Discord).

We will deliver both tactical and strategic insights that help internal decision-making at the organization, helping you optimize the experience for developers, with confidence about the choices you make.

As a result of implementing the recommendations from the study, developers will have a better experience getting up to speed with subgraphs, and therefore more likely to use The Graph in their project, leading to greater proliferation of your APIs and other services.

Why we want to do this work
The ability to query on-chain data is essential for successful web3 applications. OpenUX exists to support teams with building great user experiences that lead to mass adoption. However, developers are users too and need great experiences designed for them so that they can execute effectively. Supporting infrastructure projects like The Graph is vital for the ultimate success of web3.

About OpenUX
OpenUX is a web3-native collective of the most experienced UX researchers working in the space today. Our’s and our members’ previous clients include: MetaMask, ConsenSys, Protocol Labs, MakerDAO, Liquity, Yearn, dHedge, Stacks Bitcoin ecosystem, Lens Protocol, NEAR, Solana, Infura, Pocket Network and more. Collectively we have conducted research interviews with more than 2000 individual web3/crypto users and developers across all use cases (NFTs, DAOs, infrastructure, DeFi, staking and more).

We see user research as a public good in web3, and are dedicated to publishing our findings.

Personnel on this project

  • Georgia Rakusen
  • Eric Aker
  • Henry Harboe
  • Victor Lansman
  • Manansh Shukla

Feedback welcome!

1 Like