The Graph Academy
Master The GraphBug BountyExplorer
  • Welcome to The Graph Hub
  • FAQs
  • Official Docs
    • Get Started
    • About The Graph
      • Introduction
      • Network Overview
    • Developer
      • Mastering Subgraphs
      • Quick Start
      • Define a Subgraph
      • Create a Subgraph
      • Publish a Subgraph to the Decentralized Network
      • Query The Graph
      • Querying from an Application
      • Distributed Systems
      • AssemblyScript API
      • AssemblyScript Migration Guide
      • GraphQL API
      • Unit Testing Framework
      • Quick and easy subgraph debugging using forks
      • Deprecating a Subgraph
    • Indexer
      • What are Indexers?
      • Revenue Streams
      • Reward Distribution
      • Allocation Lifecycles
      • Indexing & Querying
      • Hardware requirements
      • IPFS hash converter
      • Vulnerabilities
      • Indexer Subgraph Selection Guide
      • Testnet
        • Graph Protocol Testnet Docker Compose
        • Graph Protocol Testnet Baremetal
          • Architecture Considerations
          • Deploy and Configure Database
          • Deploy and Configure Graph-node
          • Deploy and Configure Indexer stack
      • Monitoring
        • Ethereum Node Chainhead Monitoring - no third parties
      • Best Practices
        • Failed subgraphs - Manually Closing Allocations
    • Delegator
      • Choosing Indexers
        • Network Page
        • Tools
        • Guides
      • Reward Statuses
    • Curator
    • Subgraph Studio
      • How to Use the Subgraph Studio
      • Deploy a Subgraph to the Subgraph Studio
      • Billing on the Subgraph Studio
      • Managing your API keys
      • Subgraph Studio FAQs
      • Transferring Subgraph Ownership
    • The Graph Explorer
    • Supported Networks
      • Building Subgraphs on NEAR
      • Building Subgraphs on Cosmos
      • Building Subgraphs on Arweave
  • The Graph Ecosystem
    • Network
      • Core Developer Teams
        • Edge & Node
        • Streamingfast
        • Figment
        • Semiotic
        • The Guild
        • GraphOps
      • The Graph Foundation
      • The Graph Council
      • Graph Advocates
      • Graph Advocates DAO
    • Infrastructure
      • Mainnet
      • Testnet
      • Network Migration
      • Multichain Migration
      • Firehose
      • Substreams
      • The Graph Client
Powered by GitBook
On this page
  • Purpose
  • Motivation
  • Capabilities

Was this helpful?

  1. The Graph Ecosystem
  2. Infrastructure

Firehose

PreviousMultichain MigrationNextSubstreams

Last updated 2 years ago

Was this helpful?

A Firehose is a gRPC service providing an ordered, yet fork-aware, stream of blocks, developed by The Graph's core developers to better support performant indexing at scale. This is not currently an indexer requirement, but Indexers are encouraged to familiarise themselves with the technology, ahead of full network support. Learn more here: .

Purpose

What is it?

Firehose is a core component of StreamingFast’s suite of open-source blockchain technologies.

Streaming-first & Files-based

Firehose provides a files-based and streaming-first approach to processing blockchain data.

Blockchain Data Extraction

Firehose is responsible for extracting data from blockchain nodes in a highly efficient manner. Firehose also consumes, processes, and streams blockchain data to consumers of nodes running Firehose-enabled, instrumented blockchain client software.

Speed Increases Across Blockchains

Firehose makes paramount improvements in the speed and performance of data availability for any blockchain.

The full enables low-latency processing of real-time blockchain data in the form of binary data streams. is another application in the suite that works with Firehose to execute massive operations on historical blockchain data, in an extremely parallelized manner.

Motivation

Why does it exist?

Firehose was created to increase the speed and performance of blockchain data extraction from problems encountered in deployed, real-world applications.

Firehose Prevents Downtime

Companies experienced up to three week-long periods of downtime due to the reprocessing of blockchain data in a linear fashion. Firehose was designed for highly efficient parallelized node data processing, at a massively large scale, circumventing these unwanted and problematic downtimes.

Unrivaled Blockchain Data Processing Speeds

Firehose was designed to process blockchain at speeds that were previously unseen and thought to be impossible.

Resolving Slow JSON-RPC Responses

Another factor that heavily contributed to the design of Firehose is the brittleness and slow response times of, often inconsistent, JSON-RPC systems.

Capabilities

How it works

The Firehose instrumentation service is added to a node for efficient capture and simple storage of blockchain data.

Data Extraction, Storage, & Access

Firehose extracts, transforms and saves blockchain data in a highly performant file-based strategy. Blockchain developers can then access data extracted by Firehose through binary data streams.

Firehose Single Source of Truth (SSOT)

Firehose provides a single source of truth for developers looking to utilize blockchain data for their blockchain application development efforts.

The Graph & Firehose

Technical Overview

To get started with Firehose, the first step is to learn about its core concepts and technical architecture.

Existing Firehose Users

Experienced node operators can get up and running by using one of the pre-instrumented blockchain node codebases provided by StreamingFast. Look in the Firehose Setup section of the Firehose documentation for further information.

Custom Firehose Setups & New Chains

Lastly, developers can learn how to implement custom Firehose nodes in the Integrate new chains section of the Firehose documentation.

Firehose is intended to stand as a replacement for The Graph’s original blockchain data extraction layer. is an indexing protocol used for the organization of blockchain data.

Firehose Documentation by Streamingfast
StreamingFast software suite
Substreams
The Graph