Create a Subgraph
Create a Subgraph
Before being able to use the Graph CLI, you need to create your subgraph in Subgraph Studio. You will then be able to setup your subgraph project and deploy it to the platform of your choice. Note that subgraphs that do not index Ethereum mainnet will not be published to The Graph Network.
The graph init
command can be used to set up a new subgraph project, either from an existing contract on any of the public Ethereum networks or from an example subgraph. This command can be used to create a subgraph on the Subgraph Studio by passing in graph init --product subgraph-studio
. If you already have a smart contract deployed to Ethereum mainnet or one of the testnets, bootstrapping a new subgraph from that contract can be a good way to get started. But first, a little about the networks The Graph supports.
Supported Networks
The Graph Network supports subgraphs indexing mainnet Ethereum:
mainnet
Remember that you will not be able to publish a subgraph that indexes a non-mainnet network to the decentralized Graph Network in Subgraph Studio.
From An Existing Contract
The following command creates a subgraph that indexes all events of an existing contract. It attempts to fetch the contract ABI from Etherscan and falls back to requesting a local file path. If any of the optional arguments are missing, it takes you through an interactive form.
The <SUBGRAPH_SLUG>
is the ID of your subgraph in Subgraph Studio, it can be found on your subgraph details page.
From An Example Subgraph
The second mode graph init
supports is creating a new project from an example subgraph. The following command does this:
The example subgraph is based on the Gravity contract by Dani Grant that manages user avatars and emits NewGravatar
or UpdateGravatar
events whenever avatars are created or updated. The subgraph handles these events by writing Gravatar
entities to the Graph Node store and ensuring these are updated according to the events. The following sections will go over the files that make up the subgraph manifest for this example.
The Subgraph Manifest
The subgraph manifest subgraph.yaml
defines the smart contracts your subgraph indexes, which events from these contracts to pay attention to, and how to map event data to entities that Graph Node stores and allows to query. The full specification for subgraph manifests can be found here.
For the example subgraph, subgraph.yaml
is:
The important entries to update for the manifest are:
description
: a human-readable description of what the subgraph is. This description is displayed by the Graph Explorer when the subgraph is deployedrepository
: the URL of the repository where the subgraph manifest can be found. This is also displayed by The Graph Explorer.features
: a list of all used feature names.dataSources.source
: the address of the smart contract the subgraph sources, and the ABI of the smart contract to use. The address is optional; omitting it allows to index matching events from all contracts.dataSources.source.startBlock
: the optional number of the block that the data source starts indexing from. In most cases, we suggest using the block in which the contract was created.dataSources.mapping.entities
: the entities that the data source writes to the store. The schema for each entity is defined in the schema.graphql file.dataSources.mapping.abis
: one or more named ABI files for the source contract as well as any other smart contracts that you interact with from within the mappings.dataSources.mapping.eventHandlers
: lists the smart contract events this subgraph reacts to and the handlers in the mapping—./src/mapping.ts in the example—that transform these events into entities in the store.dataSources.mapping.callHandlers
: lists the smart contract functions this subgraph reacts to and handlers in the mapping that transform the inputs and outputs to function calls into entities in the store.dataSources.mapping.blockHandlers
: lists the blocks this subgraph reacts to and handlers in the mapping to run when a block is appended to the chain. Without a filter, the block handler will be run every block. An optional call-filter can be provided by adding afilter
field withkind: call
to the handler. This will only run the handler if the block contains at least one call to the data source contract.
A single subgraph can index data from multiple smart contracts. Add an entry for each contract from which data needs to be indexed to the dataSources
array.
The triggers for a data source within a block are ordered using the following process:
Event and call triggers are first ordered by transaction index within the block.
Event and call triggers within the same transaction are ordered using a convention: event triggers first then call triggers, each type respecting the order they are defined in the manifest.
Block triggers are run after event and call triggers, in the order they are defined in the manifest.
These ordering rules are subject to change.
Getting The ABIs
The ABI file(s) must match your contract(s). There are a few ways to obtain ABI files:
If you are building your own project, you will likely have access to your most current ABIs.
If you are building a subgraph for a public project, you can download that project to your computer and get the ABI by using
truffle compile
or using solc to compile.You can also find the ABI on Etherscan, but this isn't always reliable, as the ABI that is uploaded there may be out of date. Make sure you have the right ABI, otherwise running your subgraph will fail.
The GraphQL Schema
The schema for your subgraph is in the file schema.graphql
. GraphQL schemas are defined using the GraphQL interface definition language. If you've never written a GraphQL schema, it is recommended that you check out this primer on the GraphQL type system. Reference documentation for GraphQL schemas can be found in the GraphQL API section.
Defining Entities
Before defining entities, it is important to take a step back and think about how your data is structured and linked. All queries will be made against the data model defined in the subgraph schema and the entities indexed by the subgraph. Because of this, it is good to define the subgraph schema in a way that matches the needs of your dapp. It may be useful to imagine entities as "objects containing data", rather than as events or functions.
With The Graph, you simply define entity types in schema.graphql
, and Graph Node will generate top level fields for querying single instances and collections of that entity type. Each type that should be an entity is required to be annotated with an @entity
directive. By default, entities are mutable, meaning that mappings can load existing entities, modify them and store a new version of that entity. Mutability comes at a price, and for entity types for which it is known that they will never be modified, for example, because they simply contain data extracted verbatim from the chain, it is recommended to mark them as immutable with @entity(immutable: true)
. Mappings can make changes to immutable entities as long as those changes happen in the same block in which the entity was created. Immutable entities are much faster to write and to query, and should therefore be used whenever possible.
Good Example
The Gravatar
entity below is structured around a Gravatar object and is a good example of how an entity could be defined.
Bad Example
The example GravatarAccepted
and GravatarDeclined
entities below are based around events. It is not recommended to map events or function calls to entities 1:1.
Optional and Required Fields
Entity fields can be defined as required or optional. Required fields are indicated by the !
in the schema. If a required field is not set in the mapping, you will receive this error when querying the field:
Each entity must have an id
field, which must be of type Bytes!
or String!
. It is generally recommended to use Bytes!
, unless the id
contains human-readable text, since entities with Bytes!
id's will be faster to write and query as those with a String!
id
. The id
field serves as the primary key, and needs to be unique among all entities of the same type. For historical reasons, the type ID!
is also accepted and is a synonym for String!
.
For some entity types the id
is constructed from the id's of two other entities; that is possible using concat
, e.g., let id = left.id.concat(right.id)
to form the id from the id's of left
and right
. Similarly, to construct an id from the id of an existing entity and a counter count
, let id = left.id.concatI32(count)
can be used. The concatenation is guaranteed to produce unique id's as long as the length of left
is the same for all such entities, for example, because left.id
is an Address
.
Built-In Scalar Types
GraphQL Supported Scalars
We support the following scalars in our GraphQL API:
Enums
You can also create enums within a schema. Enums have the following syntax:
Once the enum is defined in the schema, you can use the string representation of the enum value to set an enum field on an entity. For example, you can set the tokenStatus
to SecondOwner
by first defining your entity and subsequently setting the field with entity.tokenStatus = "SecondOwner"
. The example below demonstrates what the Token entity would look like with an enum field:
More detail on writing enums can be found in the GraphQL documentation.
Entity Relationships
An entity may have a relationship to one or more other entities in your schema. These relationships may be traversed in your queries. Relationships in The Graph are unidirectional. It is possible to simulate bidirectional relationships by defining a unidirectional relationship on either "end" of the relationship.
Relationships are defined on entities just like any other field except that the type specified is that of another entity.
One-To-One Relationships
Define a Transaction
entity type with an optional one-to-one relationship with a TransactionReceipt
entity type:
One-To-Many Relationships
Define a TokenBalance
entity type with a required one-to-many relationship with a Token entity type:
Reverse Lookups
Reverse lookups can be defined on an entity through the @derivedFrom
field. This creates a virtual field on the entity that may be queried but cannot be set manually through the mappings API. Rather, it is derived from the relationship defined on the other entity. For such relationships, it rarely makes sense to store both sides of the relationship, and both indexing and query performance will be better when only one side is stored and the other is derived.
For one-to-many relationships, the relationship should always be stored on the 'one' side, and the 'many' side should always be derived. Storing the relationship this way, rather than storing an array of entities on the 'many' side, will result in dramatically better performance for both indexing and querying the subgraph. In general, storing arrays of entities should be avoided as much as is practical.
Example
We can make the balances for a token accessible from the token by deriving a tokenBalances
field:
Many-To-Many Relationships
For many-to-many relationships, such as users that each may belong to any number of organizations, the most straightforward, but generally not the most performant, way to model the relationship is as an array in each of the two entities involved. If the relationship is symmetric, only one side of the relationship needs to be stored and the other side can be derived.
Example
Define a reverse lookup from a User
entity type to an Organization
entity type. In the example below, this is achieved by looking up the members
attribute from within the Organization
entity. In queries, the organizations
field on User
will be resolved by finding all Organization
entities that include the user's ID.
A more performant way to store this relationship is through a mapping table that has one entry for each User
/ Organization
pair with a schema like
This approach requires that queries descend into one additional level to retrieve, for example, the organizations for users:
This more elaborate way of storing many-to-many relationships will result in less data stored for the subgraph, and therefore to a subgraph that is often dramatically faster to index and to query.
Adding comments to the schema
As per GraphQL spec, comments can be added above schema entity attributes using double quotations ""
. This is illustrated in the example below:
Defining Fulltext Search Fields
Fulltext search queries filter and rank entities based on a text search input. Fulltext queries are able to return matches for similar words by processing the query text input into stems before comparing them to the indexed text data.
A fulltext query definition includes the query name, the language dictionary used to process the text fields, the ranking algorithm used to order the results, and the fields included in the search. Each fulltext query may span multiple fields, but all included fields must be from a single entity type.
To add a fulltext query, include a _Schema_
type with a fulltext directive in the GraphQL schema.
The example bandSearch
field can be used in queries to filter Band
entities based on the text documents in the name
, description
, and bio
fields. Jump to GraphQL API - Queries for a description of the fulltext search API and more example usage.
Feature Management: From
specVersion
0.0.4
and onwards,fullTextSearch
must be declared under thefeatures
section in the subgraph manifest.
Languages supported
Choosing a different language will have a definitive, though sometimes subtle, effect on the fulltext search API. Fields covered by a fulltext query field are examined in the context of the chosen language, so the lexemes produced by analysis and search queries vary from language to language. For example: when using the supported Turkish dictionary "token" is stemmed to "toke" while, of course, the English dictionary will stem it to "token".
Supported language dictionaries:
Ranking Algorithms
Supported algorithms for ordering results:
Writing Mappings
The mappings transform the Ethereum data your mappings are sourcing into entities defined in your schema. Mappings are written in a subset of TypeScript called AssemblyScript which can be compiled to WASM (WebAssembly). AssemblyScript is stricter than normal TypeScript, yet provides a familiar syntax.
For each event handler that is defined in subgraph.yaml
under mapping.eventHandlers
, create an exported function of the same name. Each handler must accept a single parameter called event
with a type corresponding to the name of the event which is being handled.
In the example subgraph, src/mapping.ts
contains handlers for the NewGravatar
and UpdatedGravatar
events:
The first handler takes a NewGravatar
event and creates a new Gravatar
entity with new Gravatar(event.params.id.toHex())
, populating the entity fields using the corresponding event parameters. This entity instance is represented by the variable gravatar
, with an id value of event.params.id.toHex()
.
The second handler tries to load the existing Gravatar
from the Graph Node store. If it does not exist yet, it is created on-demand. The entity is then updated to match the new event parameters before it is saved back to the store using gravatar.save()
.
Recommended IDs for Creating New Entities
Every entity has to have an id
that is unique among all entities of the same type. An entity's id
value is set when the entity is created. Below are some recommended id
values to consider when creating new entities. NOTE: The value of id
must be a string
.
event.params.id.toHex()
event.transaction.from.toHex()
event.transaction.hash.toHex() + "-" + event.logIndex.toString()
We provide the Graph Typescript Library which contains utilies for interacting with the Graph Node store and conveniences for handling smart contract data and entities. You can use this library in your mappings by importing @graphprotocol/graph-ts
in mapping.ts
.
Code Generation
In order to make it easy and type-safe to work with smart contracts, events and entities, the Graph CLI can generate AssemblyScript types from the subgraph's GraphQL schema and the contract ABIs included in the data sources.
This is done with
but in most cases, subgraphs are already preconfigured via package.json
to allow you to simply run one of the following to achieve the same:
This will generate an AssemblyScript class for every smart contract in the ABI files mentioned in subgraph.yaml
, allowing you to bind these contracts to specific addresses in the mappings and call read-only contract methods against the block being processed. It will also generate a class for every contract event to provide easy access to event parameters, as well as the block and transaction the event originated from. All of these types are written to <OUTPUT_DIR>/<DATA_SOURCE_NAME>/<ABI_NAME>.ts
. In the example subgraph, this would be generated/Gravity/Gravity.ts
, allowing mappings to import these types with.
In addition to this, one class is generated for each entity type in the subgraph's GraphQL schema. These classes provide type-safe entity loading, read and write access to entity fields as well as a save()
method to write entities to store. All entity classes are written to <OUTPUT_DIR>/schema.ts
, allowing mappings to import them with
Note: The code generation must be performed again after every change to the GraphQL schema or the ABIs included in the manifest. It must also be performed at least once before building or deploying the subgraph.
Code generation does not check your mapping code in src/mapping.ts
. If you want to check that before trying to deploy your subgraph to the Graph Explorer, you can run yarn build
and fix any syntax errors that the TypeScript compiler might find.
Data Source Templates
A common pattern in Ethereum smart contracts is the use of registry or factory contracts, where one contract creates, manages, or references an arbitrary number of other contracts that each have their own state and events. The addresses of these sub-contracts may or may not be known upfront and many of these contracts may be created and/or added over time. This is why, in such cases, defining a single data source or a fixed number of data sources is impossible and a more dynamic approach is needed: data source templates.
Data Source for the Main Contract
First, you define a regular data source for the main contract. The snippet below shows a simplified example data source for the Uniswap exchange factory contract. Note the NewExchange(address,address)
event handler. This is emitted when a new exchange contract is created on-chain by the factory contract.
Data Source Templates for Dynamically Created Contracts
Then, you add data source templates to the manifest. These are identical to regular data sources, except that they lack a pre-defined contract address under source
. Typically, you would define one template for each type of sub-contract managed or referenced by the parent contract.
Instantiating a Data Source Template
In the final step, you update your main contract mapping to create a dynamic data source instance from one of the templates. In this example, you would change the main contract mapping to import the Exchange
template and call the Exchange.create(address)
method on it to start indexing the new exchange contract.
Note: A new data source will only process the calls and events for the block in which it was created and all following blocks, but will not process historical data, i.e., data that is contained in prior blocks.
If prior blocks contain data relevant to the new data source, it is best to index that data by reading the current state of the contract and creating entities representing that state at the time the new data source is created.
Data Source Context
Data source contexts allow passing extra configuration when instantiating a template. In our example, let's say exchanges are associated with a particular trading pair, which is included in the NewExchange
event. That information can be passed into the instantiated data source, like so:
Inside a mapping of the Exchange
template, the context can then be accessed:
There are setters and getters like setString
and getString
for all value types.
Start Blocks
The startBlock
is an optional setting that allows you to define from which block in the chain the data source will start indexing. Setting the start block allows the data source to skip potentially millions of blocks that are irrelevant. Typically, a subgraph developer will set startBlock
to the block in which the smart contract of the data source was created.
Note: The contract creation block can be quickly looked up on Etherscan:
Search for the contract by entering its address in the search bar.
Click on the creation transaction hash in the
Contract Creator
section.Load the transaction details page where you'll find the start block for that contract.
Call Handlers
While events provide an effective way to collect relevant changes to the state of a contract, many contracts avoid generating logs to optimize gas costs. In these cases, a subgraph can subscribe to calls made to the data source contract. This is achieved by defining call handlers referencing the function signature and the mapping handler that will process calls to this function. To process these calls, the mapping handler will receive an ethereum.Call
as an argument with the typed inputs to and outputs from the call. Calls made at any depth in a transaction's call chain will trigger the mapping, allowing activity with the data source contract through proxy contracts to be captured.
Call handlers will only trigger in one of two cases: when the function specified is called by an account other than the contract itself or when it is marked as external in Solidity and called as part of another function in the same contract.
Note: Call handlers are not supported on Rinkeby, Goerli, or Ganache. Call handlers currently depend on the Parity tracing API and these networks do not support it.
Defining a Call Handler
To define a call handler in your manifest, simply add a callHandlers
array under the data source you would like to subscribe to.
The function
is the normalized function signature to filter calls by. The handler
property is the name of the function in your mapping you would like to execute when the target function is called in the data source contract.
Mapping Function
Each call handler takes a single parameter that has a type corresponding to the name of the called function. In the example subgraph above, the mapping contains a handler for when the createGravatar
function is called and receives a CreateGravatarCall
parameter as an argument:
The handleCreateGravatar
function takes a new CreateGravatarCall
which is a subclass of ethereum.Call
, provided by @graphprotocol/graph-ts
, that includes the typed inputs and outputs of the call. The CreateGravatarCall
type is generated for you when you run graph codegen
.
Block Handlers
In addition to subscribing to contract events or function calls, a subgraph may want to update its data as new blocks are appended to the chain. To achieve this a subgraph can run a function after every block or after blocks that match a pre-defined filter.
Supported Filters
The defined handler will be called once for every block which contains a call to the contract (data source) the handler is defined under.
The absence of a filter for a block handler will ensure that the handler is called every block. A data source can only contain one block handler for each filter type.
Mapping Function
The mapping function will receive an ethereum.Block
as its only argument. Like mapping functions for events, this function can access existing subgraph entities in the store, call smart contracts and create or update entities.
Anonymous Events
If you need to process anonymous events in Solidity, that can be achieved by providing the topic 0 of the event, as in the example:
An event will only be triggered when both the signature and topic 0 match. By default, topic0
is equal to the hash of the event signature.
Transaction Receipts in Event Handlers
Starting from specVersion
0.0.5
and apiVersion
0.0.7
, event handlers can have access to the receipt for the transaction which emitted them.
To do so, event handlers must be declared in the subgraph manifest with the new receipt: true
key, which is optional and defaults to false.
Inside the handler function, the receipt can be accessed in the Event.receipt
field. When the receipt
key is set to false
or omitted in the manifest, a null
value will be returned instead.
Experimental features
Starting from specVersion
0.0.4
, subgraph features must be explicitly declared in the features
section at the top level of the manifest file, using their camelCase
name, as listed in the table below:
For instance, if a subgraph uses the Full-Text Search and the Non-fatal Errors features, the features
field in the manifest should be:
Note that using a feature without declaring it will incur a validation error during subgraph deployment, but no errors will occur if a feature is declared but not used.
IPFS on Ethereum Contracts
A common use case for combining IPFS with Ethereum is to store data on IPFS that would be too expensive to maintain on-chain, and reference the IPFS hash in Ethereum contracts.
Given such IPFS hashes, subgraphs can read the corresponding files from IPFS using ipfs.cat
and ipfs.map
. To do this reliably, it is required that these files are pinned to an IPFS node with high availability.
Note: The Graph Network does not yet support
ipfs.cat
andipfs.map
, and developers should not deploy subgraphs using that functionality to the network via the Studio.
Feature Management:
ipfsOnEthereumContracts
must be declared underfeatures
in the subgraph manifest. For non EVM chains, thenonDeterministicIpfs
alias can also be used for the same purpose.
When running a local Graph Node, the GRAPH_ALLOW_NON_DETERMINISTIC_IPFS
environment variable must be set in order to index subgraphs using this experimental functionality.
Non-fatal errors
Indexing errors on already synced subgraphs will, by default, cause the subgraph to fail and stop syncing. Subgraphs can alternatively be configured to continue syncing in the presence of errors, by ignoring the changes made by the handler which provoked the error. This gives subgraph authors time to correct their subgraphs while queries continue to be served against the latest block, though the results might be inconsistent due to the bug that caused the error. Note that some errors are still always fatal. To be non-fatal, the error must be known to be deterministic.
Note: The Graph Network does not yet support non-fatal errors, and developers should not deploy subgraphs using that functionality to the network via the Studio.
Enabling non-fatal errors requires setting the following feature flag on the subgraph manifest:
The query must also opt-in to querying data with potential inconsistencies through the subgraphError
argument. It is also recommended to query _meta
to check if the subgraph has skipped over errors, as in the example:
If the subgraph encounters an error, that query will return both the data and a graphql error with the message "indexing_error"
, as in this example response:
Grafting onto Existing Subgraphs
When a subgraph is first deployed, it starts indexing events at the genesis block of the corresponding chain (or at the startBlock
defined with each data source) In some circumstances; it is beneficial to reuse the data from an existing subgraph and start indexing at a much later block. This mode of indexing is called Grafting. Grafting is, for example, useful during development to get past simple errors in the mappings quickly or to temporarily get an existing subgraph working again after it has failed.
Note: Grafting requires that the Indexer has indexed the base subgraph. It is not recommended on The Graph Network at this time, and developers should not deploy subgraphs using that functionality to the network via the Studio.
A subgraph is grafted onto a base subgraph when the subgraph manifest in subgraph.yaml
contains a graft
block at the top-level:
When a subgraph whose manifest contains a graft
block is deployed, Graph Node will copy the data of the base
subgraph up to and including the given block
and then continue indexing the new subgraph from that block on. The base subgraph must exist on the target Graph Node instance and must have indexed up to at least the given block. Because of this restriction, grafting should only be used during development or during an emergency to speed up producing an equivalent non-grafted subgraph.
Because grafting copies rather than indexes base data, it is much quicker to get the subgraph to the desired block than indexing from scratch, though the initial data copy can still take several hours for very large subgraphs. While the grafted subgraph is being initialized, the Graph Node will log information about the entity types that have already been copied.
The grafted subgraph can use a GraphQL schema that is not identical to the one of the base subgraph, but merely compatible with it. It has to be a valid subgraph schema in its own right, but may deviate from the base subgraph's schema in the following ways:
It adds or removes entity types
It removes attributes from entity types
It adds nullable attributes to entity types
It turns non-nullable attributes into nullable attributes
It adds values to enums
It adds or removes interfaces
It changes for which entity types an interface is implemented
Feature Management:
grafting
must be declared underfeatures
in the subgraph manifest.
Last updated