Deploy and Configure Graph-node

Search…
This guide provides instructions for:
Installing graph-node, query-node
Where relevant, configuring a systemd unitfile for each component, so they auto-restart on failure.
Acknowledgements and Disclaimer
The contents of this guide has been pulled together from a variety of sources. It has been tested on Ubuntu Server 18.04 and 20.04. Your mileage may vary. The exact versions of Graph components needed for mainnet/testnet are documented here - many of the commands include specific versions of the Graph software and you will need to crosscheck and use the latest versions approved for the network you are deploying on.
Prerequisites
First and foremost, it is assumed that you have decided on your architecture per the earlier part of this guide series - VMs or containers, storage sizing and redundancy, Eth node choice. At all times, this guide will use the reference architecture from the first page for all instructions. This guide is not intended for absolute beginners. It assumes some knowledge of using a linux terminal. Before you get started you will need to have your Ubuntu server instance up and running and up to date. Your server will require an internet connection. This guide assumes that you are logged into the server using a non-root account with SUDO access. Security will not be covered in this guide.
Note: If you are using Linux containers and being economical with the resources you assign to them, you need to be sure that you supply them with enough memory and compute power to install and build rust packages, or you will see strange, un-diagnosable behavior when compiling the source code. 4 vcpu and 2GB of memory should be sufficient in most cases.
graph-node installation
Per the reference architecture, two of these nodes should be deployed. To avoid repetition, the guide will only run through the first install. We will also build a query-node, which is simply a graph-node with indexing turned off and graphql query logging turned on, for use in serving queries to consumers and analyzing the performance of served queries.
Additional packages that may need to be installed:
Rust - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh Git - sudo apt install git To support Rust compiles - sudo apt-get install -y clang libpq-dev libssl-dev pkg-config build-essential
Make a directory graph-node in home: mkdir ~/graph-node
Build graph-node
Download and compile the tagged release of graph-node using the following command:
cd ~/ && rm -rf ~/graph-node && git clone https://github.com/graphprotocol/graph-node ~/graph-node && cd ~/graph-node && git checkout v0.21.1 && cargo build --release
In the above case we are checking out the v0.21.1 tag. Make sure and check that you are using the right version for the network you are deploying to as per the link in the Acknowledgements section.
Configure graph-node startup script
Run the following to create the graph-node startup script
1
tee ~/graph-startup.sh<<EOF
2
​
3
#!/bin/bash
4
source $HOME/.cargo/env
5
cd $HOME/graph-node
6
​
7
cargo run -p graph-node --release -- \\
8
        --node-id graph-test-node-0 \\
9
        --postgres-url postgresql://postgres:[email protected]:5432/graph-node \\
10
        --ethereum-rpc mainnet:http://graph-test-eth-0:8545 \\
11
        --ipfs https://ipfs.testnet.thegraph.com
12
​
13
EOF
Copied!
Make the script executable by running chmod +x graph-startup.sh
As a preliminary test, you can manually execute the script ./graph-startup.sh and should see the initial compilation process kick off (this will take some time)
1
$ ./graph-startup.sh 
2
   Compiling libc v0.2.80
3
   Compiling proc-macro2 v1.0.24
4
   Compiling syn v1.0.48
5
   Compiling log v0.4.11
6
   Compiling lazy_static v1.4.0
7
     Building [============>            ] 10/650...
Copied!
Once compilation is complete the graph-node should start and will look something like:
1
Feb 15 17:27:04.779 INFO Graph Node version: v0.21.1 (8915fd15c 2021-01-11)
2
Feb 15 17:27:04.779 INFO Generating configuration from command line arguments
3
Feb 15 17:27:04.809 INFO Starting up
4
Feb 15 17:27:04.810 INFO Trying IPFS node at: https://ipfs.testnet.thegraph.com/
5
Feb 15 17:27:04.829 INFO Creating transport, capabilities: archive, trace, url: http://graph-test-eth-0:8545, network: mainnet
6
Feb 15 17:27:04.961 INFO Connecting to Postgres, weight: 1, conn_pool_size: 10, url: postgresql://postgres:[email protected]:5432/graph-node, pool: main, shard: primary
7
Feb 15 17:27:05.070 INFO Pool successfully connected to Postgres, pool: main, shard: primary, component: Store
8
Feb 15 17:27:05.137 INFO Waiting for other graph-node instances to finish migrating, shard: primary, component: Store
9
Feb 15 17:27:05.144 INFO Running migrations, shard: primary, component: Store
10
Feb 15 17:27:05.160 INFO Successfully connected to IPFS node at: https://ipfs.testnet.thegraph.com/
11
Feb 15 17:27:07.861 INFO Migrations finished, shard: primary, component: Store
12
Feb 15 17:27:07.884 INFO Connecting to Ethereum..., capabilities: archive, trace, network: mainnet
13
Feb 15 17:27:07.900 INFO Connected to Ethereum, capabilities: archive, trace, network_version: 1, network: mainnet
14
Feb 15 17:27:07.918 INFO Creating LoadManager in disabled mode, component: LoadManager
15
Feb 15 17:27:07.921 INFO Starting block ingestors
16
Feb 15 17:27:07.921 INFO Starting block ingestor for network, network_name: mainnet
17
Feb 15 17:27:07.934 INFO Starting JSON-RPC admin server at: http://localhost:8020, component: JsonRpcServer
18
Feb 15 17:27:07.936 INFO Started all subgraphs, component: SubgraphRegistrar
19
Feb 15 17:27:07.937 INFO Starting GraphQL HTTP server at: http://localhost:8000, component: GraphQLServer
20
Feb 15 17:27:07.938 INFO Starting index node server at: http://localhost:8030, component: IndexNodeServer
21
Feb 15 17:27:07.941 INFO Starting metrics server at: http://localhost:8040, component: MetricsServer
22
Feb 15 17:27:07.942 INFO Starting GraphQL WebSocket server at: ws://localhost:8001, component: SubscriptionServer
23
Feb 15 17:27:08.027 INFO Downloading latest blocks from Ethereum. This may take a few minutes..., network_name: mainnet, component: BlockIng....
Copied!
graph-node systemd configuration
Once confident that your node runs correctly, it is important to configure the node to start automatically and restart itself if a failure occurs. To achieve this we can used systemd to manage the node process.
Run the following command to create your graphindexer unit file
1
sudo tee /etc/systemd/system/graphindexer.service<<EOF
2
​
3
[Unit]
4
Description=Graph Indexer Node
5
After=network.target 
6
Wants=network.target
7
[Service]
8
User=user
9
Group=user
10
WorkingDirectory=/home/user/
11
StandardOutput=journal
12
StandardError=journal
13
Type=simple
14
Restart=always
15
RestartSec=5
16
ExecStart= /home/user/graph-startup.sh
17
​
18
[Install]
19
WantedBy=default.target
20
EOF
Copied!
Note how the unitfile called the script you made earlier. Keep in mind that this script contains your Postgres database password for the postgres user.
Reload the systemd daemon to include the new service unitfile
1
sudo systemctl daemon-reload
Copied!
Start the graph-node service
1
sudo systemctl start graphindexer.service
Copied!
Check that graph-node is running as expected
1
sudo systemctl status graphindexer.service
Copied!
You can also tail the logs for more details
1
sudo journalctl --follow -o cat -u graphindexer.service
Copied!
When you are satisfied that the node is running correctly, you can enable it for automatic restart on fail/system restart:
1
sudo systemctl enable graphindexer.service
Copied!
query-node installation
Follow the graph-node installation steps above to build a query node. Stop just before you hit the Congifure graph-node startup script section.
Configure query-node startup script
Run the following to create the query-node startup script
1
tee ~/graph-startup.sh<<EOF
2
​
3
#!/bin/bash
4
source $HOME/.cargo/env
5
cd $HOME/graph-node
6
​
7
export GRAPH_LOG_QUERY_TIMING="gql"
8
export DISABLE_BLOCK_INGESTOR="true"
9
​
10
cargo run -p graph-node  --release -- --debug \\
11
        --node-id query0 \\
12
        --postgres-url postgresql://postgres:[email protected]:5432/graph-node \\
13
        --ethereum-rpc mainnet:http://graph-test-eth-0:8545 \\
14
        --ipfs https://ipfs.testnet.thegraph.com
15
​
16
EOF
Copied!
Note the application of two environment variables GRAPH_LOG_QUERY_TIMING and DISABLE_BLOCK_INGESTOR these are the variables that configure the graph node to act as a query node. Adding graph log query timing allows the extraction of graphql performance data on subgraph queries. Disabling the block ingestor means this node will not be able to index subgraphs. Its only role will be to collect query performance data and serve queries to the end user.
Make the script executable by running chmod +x graph-startup.sh
As per your experience deploying a standard graph node, you can manually execute the script ./graph-startup.sh and should see the initial compilation process kick off (this will take some time)
Once compilation is complete the node should start running, looking something like:
1
Mar 23 16:36:21.014 INFO Graph Node version: v0.21.1 (8915fd15c 2021-01-11)
2
Mar 23 16:36:21.017 INFO Generating configuration from command line arguments
3
Mar 23 16:36:21.165 INFO Starting up
4
Mar 23 16:36:21.165 INFO Trying IPFS node at: https://ipfs.testnet.thegraph.com/
5
Mar 23 16:36:21.206 INFO Creating transport, capabilities: archive, trace, url: http://graph-test-eth-0:8545, network: mainnet
6
Mar 23 16:36:21.273 DEBG Cleaning up large notifications after about 300s, channel: store_events, component: NotificationListener
7
Mar 23 16:36:21.384 INFO Connecting to Postgres, weight: 1, conn_pool_size: 10, url: postgresql://postgres:[email protected]:5432/graph-node, pool: main, shard: primary
8
Mar 23 16:36:21.690 INFO Successfully connected to IPFS node at: https://ipfs.testnet.thegraph.com/
9
Mar 23 16:36:21.808 INFO Pool successfully connected to Postgres, pool: main, shard: primary, component: Store
10
Mar 23 16:36:21.845 INFO Waiting for other graph-node instances to finish migrating, shard: primary, component: Store
11
Mar 23 16:36:21.851 INFO Running migrations, shard: primary, component: Store
12
Mar 23 16:36:21.917 INFO Migrations finished, shard: primary, component: Store
13
Mar 23 16:36:21.918 DEBG Using postgres host order [Main], shard: primary, component: Store
14
Mar 23 16:36:21.918 DEBG Cleaning up large notifications after about 300s, channel: chain_head_updates, component: ChainHeadUpdateListener > NotificationListener
15
Mar 23 16:36:21.936 INFO Connecting to Ethereum..., capabilities: archive, trace, network: mainnet
16
Mar 23 16:36:21.946 INFO Connected to Ethereum, capabilities: archive, trace, network_version: 1, network: mainnet
17
Mar 23 16:36:21.989 INFO Creating LoadManager in disabled mode, component: LoadManager
18
Mar 23 16:36:22.014 INFO Starting JSON-RPC admin server at: http://localhost:8020, component: JsonRpcServer
19
Mar 23 16:36:22.018 INFO Started all subgraphs, component: SubgraphRegistrar
20
Mar 23 16:36:22.021 INFO Starting GraphQL HTTP server at: http://localhost:8000, component: GraphQLServer
21
Mar 23 16:36:22.022 INFO Starting index node server at: http://localhost:8030, component: IndexNodeServer
22
Mar 23 16:36:22.023 INFO Starting metrics server at: http://localhost:8040, component: MetricsServer
23
Mar 23 16:36:22.023 INFO Starting GraphQL WebSocket server at: ws://localhost:8001, component: SubscriptionServer
Copied!
You can now use the graph-node systemd configuration section of this guide to automate restarting of the query node.
You have successfully built two graph-nodes and one query node.
Deploy and Configure Database
Deploy and Configure Indexer stack
Last modified 1yr ago
Copy link
Contents
Acknowledgements and Disclaimer
Prerequisites
graph-node installation
graph-node systemd configuration
query-node installation
Configure query-node startup script