Overview
This publication is a basic introduction to The Graph Indexer Infrastructure provisioning with the Ansible automation.
Indexer infrastructure is complex and includes few essential components: PostgreSQL, graph-node, indexer-service, and indexer-agent.
We wrote separate ansible roles for each component which allows describing infrastructure with the help of Ansible playbooks. All code is available in the repository.
Following, we will show how to use sample Ansible playbooks to configure and deploy Indexer infrastructure.
Keep in mind that some variables contain sensitive information, keep them safe or encrypt with the Ansible Vault.
All examples are using root user, but any user with sudo permissions should work.
Prerequisites
Set up passwordless SSH to all hosts that will be a part of The Graph Indexer Infrastructure. You can use the ssh-copy-id command to install your public SSH key on each host.
Install ansible on your local machine, that is, the machine you use to ssh into the remote machines. See the Ansible documentation for help with installation.
Roles
The Ansible role is a set of tasks to configure a particular service on a targeted host. We’ve created separate roles for each component.
All roles are located in the roles directory.
Each role has its variables, default values are set in the defaults/main.yml file inside the role directory.
Sample playbooks
Ansible Playbooks offer a repeatable, re-usable, simple configuration management and multi-machine deployment system, one that is well suited to deploying complex applications. If you need to execute a task with Ansible more than once, write a playbook and put it under source control. Then you can use the playbook to push out new configurations or confirm the configuration of remote systems.
Playbooks consist of one or more plays, or groups of tasks/roles, that operate on a set of targeted hosts declared in the inventory file.
Besides the inventory file, each sample playbook needs to adjust some variables.
When all set ansible playbooks can execute the defined tasks on the targeted hosts.
For better understanding, it is good to read Ansible docs.
Single node
Single-node playbook deploys all Indexer components on a single host. It could be used for testing purposes or running a small/medium size Indexer.
Inventory
Inventory for a single-node setup is very simple, you just need to copy the inventory example file and set the IP of your targetted host.
$ cp single-node.inventory.example single-node.inventory
Variables
Copy example variables file and adjust values for your Indexer.
$ cp vars/single-node-testnet.yml.example vars/single-node-testnet.yml
Description of all variables could be found here.
Run playbook
$ ansible-playbook -i single-node.inventory single-node-testnet.yml -u root -b -e @vars/single-node-testnet.yml
Multiple nodes
Multi-node sample playbook utilizes 7 different servers to run each component on its own server. Connectivity between them will be configured with the Wireguard VPN.
Inventory
Multi-node inventory isn’t so straightforward, but we’ll get you covered.
Copy example inventory file:
$ cp multi-node.inventory.example multi-node.inventory
Now you need to generate pairs of Wireguard keys for each host and set them in inventory:
private_key=$(wg genkey)
public_key=$(echo $private_key | wg pubkey)
echo "private key: $private_key"
echo "public key: $public_key"
As you could notice all Wireguard addresses are predefined and if you want to change them don’t also forget to change related variables.
Variables
Multi-node playbook utilizes 3 variables files.
The first one is very similar to the one used in a single node, so you can just copy the example file and adjust values for your Indexer.
$ cp vars/multie-node-testnet.yml.example vars/multi-node-testnet.yml
Two other files are related to the PostgreSQL configuration: graph-node db and agent/service db. The only thing you need to adjust is the password, leave all other params untouched.
Run playbook
Because the multi-node playbook connects to multiple hosts, we recommend that you check connectivity to each host before running the playbook.
$ ansible thegraph -m ping -i multi-node.inventory -u root
If everything went fine, now you can run the playbook:
$ ansible-playbook -i multi-node.inventory multi-node-testnet.yml -u root -e @vars/multi-node-testnet.yml
Connect to Wireguard VPN
To get access to your Indexer you need to connect to its Wireguard VPN. Generate Wireguard config by running:
$ ansible-playbook -i multi-node.inventory wg_local.yml -u root
As a result, it will generate wireguard.conf that can be used to connect to VPN.
Summary
Now you know how to run sample playbooks and get your own indexer up and running with the single command.
If you want to run a more customized setup, you can easily reuse Ansible roles and create your own playbook for your needs.