Mastering Subgraphs
#1 Tools:
Protofire Subgraph Toolkit:
-<https://github.com/protofire/subgraph-toolkit>
Gnosis Subgraphs Monitor:
-<https://github.com/gnosis/thegraph-subgraphs-monitor>
Synthetix Subgraph-results-pager:
-<https://github.com/justinjmoses/graph-results-pager>
Utility to get paged results from The Graph endpoints
GraphProtocol-utils:
-<https://github.com/Amxx/graphprotocol-utils>
mStable Subgraph Utilities:
-<https://github.com/mstable/mStable-subgraphs-monorepo/tree/master/packages/utils>
Dennison's Subgraph Health Check:
-~~https://subgraphtools.com/~~- Community videos:
- Wildcards:<https://www.youtube.com/watch?v=_4iipzOwq-U>
- Aragon:<https://www.youtube.com/watch?v=JNqN3fek6FY>
- PoolTogether workshop @ETHDenver:<https://www.youtube.com/watch?v=GqU_-ffyz0Q&t=21143s-> Community Articles:
-<https://medium.com/protofire-blog/subgraph-development-part-1-understanding-and-aggregating-data-ef0c9a61063d>
-<https://medium.com/protofire-blog/subgraph-development-part-2-handling-arrays-and-identifying-entities-30d63d4b1dc6>
- How to create an Ethereum DeFi realtime Dashboard using Google Data Studio:<https://towardsdatascience.com/how-to-create-a-ethereum-defi-realtime-dashboard-a60c23b527f7>
- Result:<https://datastudio.google.com/reporting/c7806832-8ba6-4250-9c27-8dab1238247b/page/6zXD>
- Creating subgraphs:
-<https://theethernaut.substack.com/p/creating-a-dark-side-subgraph>
-<https://theethernaut.substack.com/p/developer-superpowers-with-thegraph>
- Time travel queries:<https://blocklytics.org/blog/ethereum-blocks-subgraph-made-for-time-travel/>
- Date strings in AssemblyScript:<https://medium.com/blockrocket/how-to-create-date-strings-using-assemblyscript-in-the-graph-f7871f48e92d-> Scaffold-eth:<https://github.com/austintgriffith/scaffold-eth/tree/graph-dev>
- Create-eth-app:<https://github.com/PaulRBerg/create-eth-app>
- Entity names should be singular:
type UserClaimedReward @entityinstead oftype UserClaimedRewards @entity
A1. SubGraph composability is still something we are looking into. Till then u will not be able to query to Blockchains that is xDai as well as MainNet using one Graph. However, u can create two subgraphs for two chains and display them on a single front-end so in that case, no end-user is affected. For that u can use -https://www.graphql-tools.com/docs/schema-stitching/
Q3. Deploying Subgraph with HardHat
Q4. Error- Handler skipped due to execution failure, error: wasm trap: unreachable wasm backtrace
A4. array access out of bounds
Q5. Deployed my Subgraph on BSC chain but it's not Syncing
A5. There is a problem with the BSC nodes, tracking on our status page:https://status.thegraph.com/
Q6. Is there a Block-Ethereum Network subgraph
Q7. Any idea how to add multiple sources?
A7. Yes it is possible-
dataSources:
- kind: ethereum/contract name: ContractOne network: xdai mapping: kind: ethereum/events ....
- kind: ethereum/contract name: ContractTwo network: xdai mapping: kind: ethereum/events
Q8. Subgraph is too slow indexing. How can I optimise:
A8.
Ways to optimise your subgraph:
a) Only use event handlers
b) Start by optimising the smart contracts so that everything you need for indexing is included in the event parameters
c) Avoid contract calls in the handlers - possible if you have everything or nearly everything in the event payloads, see b)
f) Avoid ipfs calls
d) Avoid unnecessary
.loadsof entity - possible when no data from the entity needs to be read first to do the updatee) Avoid large / boundless arrays in entity properties
Also consider splitting the subgraph up into separate subgraphs which can sync in parallel
To go more in-depth, I recommend
,
and
In general, as other degens already pointed out: Avoid eth-calls, focus on event-handlers, avoid unnecessary data loading.
Q9. How can I index ether transaction in the subgraph?
A9. Transactions can not be indexed. You can currently only index on the basis of contract events, contract calls and blocks
Q10. How Create an Entity to be saved in store
let transfer = new Transfer(id) // Transfer is a schema
// Set properties on the entity, using the event parameters
transfer.from = event.params.from
transfer.to = event.params.to
transfer.amount = event.params.amount
// Save the entity to the store
transfer.save()
Q11. Load and update Entity from the store
let id = event.transaction.hash.toHex() // or however the ID is constructed
let transfer = Transfer.load(id)
if (transfer == null) {
transfer = new Transfer(id)
}
transfer.from = events.params.from
transfer.to = ...
transfer.amount = ...
transfer.save()
Q12. Delete an Entity
import { store } from '@graphprotocol/graph-ts'
...
let id = event.transaction.hash.toHex()
store.remove('Transfer', id)
Q13. How to query logs locally
You can query the index-node endpoint to fetch fatal Error information:
If it's the current version of your subgraph:
curl --location --request POST '<https://api.thegraph.com/index-node/graphql>' --data-raw '{"query":"{ indexingStatusForCurrentVersion(subgraphName: \\"<SUBGRAPH-NAME>\\") { subgraph synced fatalError { message } nonFatalErrors {message } } }"}'
If it's the pending version of your subgraph:
curl --location --request POST '<https://api.thegraph.com/index-node/graphql>' --data-raw '{"query":"{ indexingStatusForPendingVersion(subgraphName: \\"<SUBGRAPH-NAME>\\") { subgraph fatalError { message } nonFatalErrors {message } } }"}'
Q14. Delegation FAQ
Q15. Deploying subgraphs from contract address
graph init --from-contract 0xabEFBc9fD2F806065b4f3C237d4b59D9A97Bcac7 --network mainnet \\
--contract-name Token --index-events
Q16. How to connect locally deployed graph node to ganache
change the docker network to use the host
or use the docker links to the host
run a dockerized ganache
Use
mainnet:http://host.docker.internal:7545If that works, all good.
Otherwise put in the IP address following these instructions:
CONTAINER_ID=$(docker container ls | grep graph-node | cut -d' ' -f1) docker exec $CONTAINER_ID /bin/bash -c 'apt install -y iproute2 && ip route' | awk '/^default via /{print $3}'Q17. Is there a way to move new subgraph we deploy to a different endpoint ?
If you deploy under a different name, but with the exact same hash, the underlining deployment will be reused so there will be no sync time
Q18. Steps for deployment
Step1: npm install -g @graphprotocol/graph-cli
Step2: yarn global add @graphprotocol/graph-cli
Step4:
graph deploy \ --debug \ --node https://api.thegraph.com/deploy/ \ --ipfs https://api.thegraph.com/ipfs/ \ <SUBGRAPH_NAME>
Q19. TimeTravel queries might fail if A19.
1.Block not Onchain
- 1.ReOrg
Q20. How to debug when deploying graph-node locally A.20 RUST_LOG : debug in docker-compose file
For newer graph-node versions (>0.22.0): GRAPH_LOG: info
Q21. Data source within a block are ordered using the following process:
- 1.Event and call triggers are first ordered by transaction index(as per the sequence of trx in dapp) within the block.
- 2.Event and call triggers with in the same transaction are ordered using a convention: event triggers first then call triggers
- 3.Block triggers are run after event and call triggers, in the order they are defined in the manifest.
Q22: Arweave support
Q23: HTTP error creating the subgraph: ECONNREFUSED
Are you trying to deploy to the hosted service? There, you need to create your subgraph through the UI first:https://thegraph.com/explorer/dashboard
Q24: IndexingStatus local graph-node
Run these query:
{
indexingStatusForCurrentVersion(subgraphName: "aave/protocol") {
node
synced
health
fatalError {
message
}
chains {
network
chainHeadBlock {
number
}
earliestBlock {
number
}
latestBlock {
number
}
lastHealthyBlock {
number
}
}
entityCount
}
}
Q25. Command to track RPC and rectify the block it is in
curl -X POST -H "Content-Type: application/json" --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest", false],"id":1}'https://rpc-graph-mainnet.maticvigil.com/v1/7d25a9c3aca826c9d24fed0c273f686b28f4b0e5
Q26: "message": "Store error: store error: type Subscription already exists in the input schema"
The
schema.graphql probably contains a type Subscription. Example. This is a reserved word. Find another word.Q27 How to deprecate a subgraph from the network
Assume this is one of the ones they want to deprecate:
they should go here
and call `deprecateSubgraph with their own address as the first param, and the subgraph number as 1 . and it should work!
Q28. Recover Subgraph from depreciated subgraphs
!! YOU CAN RECOVER YOUR GRT !! Step 1: Go to the GraphProxyhttps://etherscan.io/address/0xadca0dd4729c8ba3acf3e99f3a9f471ef37b6825#writeProxyContract Step 2:Find the "withdraw" function Step 3: Put the graph account as 0xacfe4511ce883c14c4ea40563f176c3c09b4c47c with subgraphNumber = 1 Step 4: Write the txn (you can connect Etherscan to your metamask wallet)
Q29: API Key security
if my dapp frontend uses the graph for query, do I want to write my query key into the frontend directly? If we pay query fees for users, will malicious users cause our query fees to be very high?
→
This is actually a very frequently asked question and the answer has different dimensions:
- Currently yes, the recommended approach for a dapp is to add the key to the front-end and actually expose it. That said, you can limit that key to a host-name, like yourdapp.io, and subgraphs.
- The gateway is currently run by Edge and Node and we monitor the queries. If we detect abuses or you report an abuse, we can block IPs, etc.
- This system is actually not that different of running a server/API by yourself: It is also open to the public and one can try to attack it.
- Long term we envision that users will pay for their queries directly.
- 1.Is there a limit to graph results? because i can see max upto 100 results, nothing after that?
I think it displays only the first 100 by default, but try to query with first: 1000, skip: 1000 to view the next ones
Q31. Check for EIP_1898 compatibility for an rpc curl -X POST --data '{"jsonrpc":"2.0","method":"eth_call","params":[{"data":"0x0902f1ac","gas":"0x2faf080","to":"0x58f876857a02d6762e0101bb5c46a8c1ed44dc16"},{"blockHash":"0xab9316c3461cb0377666b2d442d7a5fb3d327eeb6934d17263fa42ef5cc385c6"}],"id":1033989}'https://bsc-dataseed-archive-graph.nariox.org/
Q32 Getting CORS error on graph API Ans) The CORS errors are intermittent, we are narrowing them down but basically it is a problem with the router. Could happen when we deploy new releases or if we see an unexpected load spikes. Usually, it resolves by itself after a while like it did for you. Feel free to ping us here if you see any other problems anytime.
How to deploy on Staging
- 1.
- 2.graph auth --product hosted-service-staging e8986681830b4ebcb0c236749f40b101 --ipfshttps://api.staging.thegraph.com/ipfs/ --nodehttps://api.staging.thegraph.com/deploy/
- 3.graph deploy pranavdaa/openzappelinsubgraph --access-token <Token> --ipfshttps://api.staging.thegraph.com/ipfs/ --nodehttps://api.staging.thegraph.com/deploy/
- 1.{app="index-node",cluster="hosted-service-production",pod="index-node-community-0"} |= "QmNtawNYjWwUzcrnCjV87jvtjQWtvezKe5PnyFq46JfMuP"
- 2.{app="index-node",cluster="hosted-service-production",pod=~"index-node-community-.*"}
- 3.{app="index-node",cluster="hosted-service-production",pod="index-node-community-0"} != "QmNtawNYjWwUzcrnCjV87jvtjQWtvezKe5PnyFq46JfMuP" |= "handleBlock" |~ "total.*"
- 4.{cluster="hosted-service-production",app_kubernetes_io_name="ingress-nginx"} |= "/subgraphs/name/uniswap/uniswap-v2"
- 5.{cluster="hosted-service-production",app=~"query-node.*"
For Subgraphs deployed on the network serving any issues
- 1.{cluster=~"mainnet-indexer-01-us-central|mainnet-indexer-02-europe-west|mainnet-indexer-03-us-west|mainnet-indexer-04-us-east|mainnet-indexer-05-eu-central|mainnet-indexer-07-asia-northeast",deployment="QmWK7Phe4L6H3dzXHh48G1TrYDugeAVaFroUke3y2c7Kai"}
graphman is a command line tool that is part of graph-node (when you build graph-node, graphman is also built) It's available in all the graph-node docker images, and on all our index-node-* and query-node-* pods. It uses the normal graph-node code to expose some useful admin functionality in a CLI.
You can get more insight into how a query performs by running
GRAPH_LOG_QUERY_TIMING=sql,gql GRAPH_LOG=debug graphman query Qm... 'query { .. }' for example in index-node-community-quarantine-0That will print timing information about each SQL query that gets run for a GraphQL query, and you can then throw that SQL into
psql and do an explain on itFor quickswap, I looked into some of the slow queries a while ago, and tried created some indexes. For that order by query, often creating an index on
token(trade_volume_usd, id) helps with those (or a gist index on token(block_range, trade_volume_usd, id), but for quickswap neither of these brought a significant improvementThe easiest way to play with it is if you log into a pod in staging:
> kubectl --context=staging exec index-node-community-quarantine-0 -ti -- /bin/bash
# unset GRAPH_LOG
# graphman help
# graphman info -s lutter/dice2win
You can also run it against your local
graph-node; it requires a config file, a simple example for one is here in config.simple.tomlcurl -X POST --data '{"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest", false],"id":1}' -H "Content-Type: application/json" [<https://rpc.testnet.moonbeam.network>](<https://rpc.testnet.moonbeam.network/>)- Have postgres installed:
brew install postgresql cargo buildcargo install --bin graphman --path node --locked
curl --location --request POST 'https://api.thegraph.com/index-node/graphql' --data-raw '{"query":"{ indexingStatusForPendingVersion(subgraphName: \"bigwin-official/bunny_v2\") { subgraph fatalError { message } nonFatalErrors {message } } }"}'
IPFS issue info
Making graphman run with indexer login
Hosted service
Last modified 1yr ago