Blockchain Infrastructure: Nodes, RPC, Indexing
Subgraph stopped indexing events at 3:47 AM. By morning users see stale balances, transactions "hang" in UI, support flooded with tickets. Reason: handler in subgraph crashed on one transaction with non-standard event log — entire index went down. Blockchain infrastructure doesn't forgive gaps in observability.
RPC Layer Architecture
Every dApp-to-blockchain interaction goes through RPC — JSON-RPC API provided by a node. Three options:
Managed providers — Alchemy, QuickNode, Infura, Ankr. Minimal ops overhead, SLA, built-in monitoring. Limitations: rate limits (Alchemy Free: 300 RU/sec), vendor lock-in, potential downtime during provider incidents. For most projects — correct choice to start.
Own nodes — full control, no rate limits, no third-party dependency. Cost: archive node Ethereum is 2.5–3TB SSD (2025), requires powerful server and DevOps support. Sync from scratch on Ethereum through Geth/Nethermind — 3–7 days. Justified at high load or latency requirements.
Hybrid — own node as primary, managed provider as fallback. Standard for protocols with TVL from $10M.
| Provider | Strength | Limitation |
|---|---|---|
| Alchemy | Supernode, Enhanced APIs, webhooks | Expensive on high-volume |
| QuickNode | Low latency, multi-chain | Pricier than Alchemy on basic plan |
| Infura | Historical reliability | Rate limits on free, single 2020 downtime hit half of DeFi |
| Ankr | Cheap, 40+ chains | Less stable |
Ethereum Node Clients
Execution clients: Geth (most used), Nethermind (C#, fast sync), Besu (Java, enterprise), Erigon (fastest sync, efficient archive mode).
Consensus clients (post-Merge): Lighthouse (Rust), Prysm (Go), Teku (Java), Nimbus (Nim). After The Merge each node needs execution + consensus client pair.
For DevOps: eth-docker — Docker Compose configs for all client combinations. Monitoring via Grafana + Prometheus is mandatory, standard dashboard in each client repo.
The Graph: Event Indexing
The Graph Protocol — decentralized indexing. Subgraph describes which events from which contracts to index and how to transform into GraphQL schema.
Subgraph structure:
-
subgraph.yaml— manifest: contract addresses, startBlock, events to handle -
schema.graphql— GraphQL schema entities -
src/mapping.ts— AssemblyScript event handlers
AssemblyScript handlers are NOT TypeScript. No nullable types, no closures, no many standard APIs. Error in handler stops subgraph indexing on that transaction. Important: add try-catch on operations that may fail (e.g., store.get() for entity that might not exist).
Hosted Service vs Decentralized Network
Graph Hosted Service (free, centralized) deprecated for Subgraph Studio + Graph Network. For production: deploy on Graph Network with GRT curation signal — subgraph gets indexers proportional to curation.
Alternatives: Ponder (TypeScript, self-hosted, easier debug), Envio (ultra-fast, EVM + non-EVM), Subsquid (TypeScript, own network), Moralis Streams (managed, webhook-based).
Webhooks and Real-Time Notifications
Alchemy Webhooks and QuickNode Streams provide real-time events via HTTP webhook or WebSocket. For address monitoring, new transactions, mints — faster than polling RPC.
Tenderly — monitoring platform. Set alerts on contract events, balance changes, function calls with specific parameters. Transaction simulation via Tenderly API is invaluable for debugging.
Monitoring and Observability
Minimal monitoring stack for protocol:
On-chain: OpenZeppelin Defender Sentinel — watches contract events, calls webhook or Autotask on trigger. Forta Network — community bots detect anomalies (large withdrawals, flash loans, governance attacks).
Infrastructure: Grafana + Prometheus for nodes, Datadog or Grafana Cloud for managed metrics. Alerts on: node lagged 10+ blocks, RPC latency > 500ms, subgraph lag > 100 blocks.
Uptime: Better Uptime or PagerDuty on RPC endpoint and subgraph health endpoint (The Graph provides _meta { hasIndexingErrors, block { number } }).
Multichain Infrastructure
Protocol on 5 chains = 5 separate RPC endpoints, 5 subgraphs, 5 monitoring configs. Manageable but needs deployment automation.
For subgraph multi-network deploy: graph deploy --network mainnet, graph deploy --network arbitrum-one etc with shared codebase and network-specific addresses in separate config files.
Chainlink CCIP and LayerZero for cross-chain messaging require monitoring both chains and intermediate relayer transactions. Reorg on source chain after confirmed mint on target chain — classic bridge problem. Solution: wait for finality (on Ethereum ~15 minutes post-Merge for economic finality) before confirming on target chain.
Infrastructure Setup Process
Provider selection — based on chains, request volume, latency requirements.
RPC configuration — primary + fallback, load balancing for high load.
Subgraph development — manifest → schema → handlers → test on local Graph Node → testnet deploy → mainnet.
Monitoring — Tenderly alerts on critical events, Grafana dashboard for infrastructure metrics.
Runbook — documentation: what to do when subgraph falls behind, RPC downtime, node desync.
Timelines
- RPC setup and basic monitoring: 1–2 weeks
- Subgraph for one protocol: 2–4 weeks
- Self-hosted node with monitoring: 2–3 weeks
- Full infrastructure (multi-chain, monitoring, runbooks): 6–10 weeks