Development of Blockchain Incident Response System
Hack in DeFi is not breach in traditional sense. No brute force, no phishing. Attack takes one transaction (sometimes several), money leaves irreversibly in seconds. By the time team learns of incident — damage already done. Only way to minimize losses — detect attack in real time (ideally before execution) and have pre-prepared stop mechanisms.
Ronin Bridge (March 2022, $625M) didn't notice hack for 6 days. Euler Finance (March 2023, $197M) reacted quickly, negotiated return in 2 weeks — atypical happy ending. Difference in incident response approach.
Architecture of response system
System consists of three layers:
Detection (monitoring) — on-chain and off-chain anomaly monitoring in real time.
Reaction (circuit breaker) — set of on-chain mechanisms for immediate function halt.
Communication and recovery — notification processes, damage assessment, response coordination.
Circuit Breaker contracts
Pause mechanism
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.20;
import "@openzeppelin/contracts/security/Pausable.sol";
import "@openzeppelin/contracts/access/AccessControl.sol";
contract ProtocolCore is Pausable, AccessControl {
bytes32 public constant GUARDIAN_ROLE = keccak256("GUARDIAN_ROLE");
bytes32 public constant PAUSER_ROLE = keccak256("PAUSER_ROLE");
// Emergency pause: separate functions, not entire protocol
mapping(bytes4 => bool) public functionPaused;
event FunctionPaused(bytes4 indexed selector, address indexed by);
event FunctionUnpaused(bytes4 indexed selector);
event EmergencyWithdrawTriggered(address indexed asset, uint256 amount);
modifier notFunctionPaused() {
require(!functionPaused[msg.sig], "Function paused");
_;
}
// Global halt — only through timelock (except Guardian)
function pause() external {
require(
hasRole(PAUSER_ROLE, msg.sender) || hasRole(GUARDIAN_ROLE, msg.sender),
"Not authorized"
);
_pause();
}
// Resume — only through timelock governance
function unpause() external onlyRole(DEFAULT_ADMIN_ROLE) {
_unpause();
}
// Point pause specific function
function pauseFunction(bytes4 selector) external onlyRole(GUARDIAN_ROLE) {
functionPaused[selector] = true;
emit FunctionPaused(selector, msg.sender);
}
// Critical: withdraw funds to safe wallet on compromise
function emergencyWithdraw(
address asset,
address safeDestination
) external onlyRole(GUARDIAN_ROLE) whenPaused {
require(safeDestination != address(0), "Invalid destination");
uint256 balance = IERC20(asset).balanceOf(address(this));
if (balance > 0) {
IERC20(asset).safeTransfer(safeDestination, balance);
emit EmergencyWithdrawTriggered(asset, balance);
}
}
}
Rate limiter
Not all DeFi attacks happen in one transaction. Often attack is series of transactions. Rate limiter slows fund outflow:
contract RateLimiter {
struct RateLimit {
uint256 maxAmount; // max per period
uint256 periodDuration; // period length in seconds
uint256 currentAmount; // spent in current period
uint256 periodStart; // current period start
}
mapping(address => RateLimit) public limits; // per-asset limits
event RateLimitExceeded(address indexed asset, uint256 requested, uint256 available);
function _checkAndUpdateRateLimit(address asset, uint256 amount) internal {
RateLimit storage limit = limits[asset];
if (limit.maxAmount == 0) return; // limit not set
// Reset period if expired
if (block.timestamp >= limit.periodStart + limit.periodDuration) {
limit.currentAmount = 0;
limit.periodStart = block.timestamp;
}
uint256 available = limit.maxAmount - limit.currentAmount;
if (amount > available) {
emit RateLimitExceeded(asset, amount, available);
revert("Rate limit exceeded");
}
limit.currentAmount += amount;
}
}
Monitoring system
On-chain event monitoring
interface MonitoringRule {
eventSignature: string;
condition: (event: Log) => boolean;
severity: 'low' | 'medium' | 'high' | 'critical';
action: 'alert' | 'auto-pause' | 'notify-guardian';
}
const rules: MonitoringRule[] = [
{
eventSignature: 'Withdrawal(address,uint256)',
condition: (e) => BigInt(e.data) > LARGE_WITHDRAWAL_THRESHOLD,
severity: 'high',
action: 'alert',
},
{
eventSignature: 'OwnershipTransferred(address,address)',
condition: () => true, // any ownership transfer — critical
severity: 'critical',
action: 'notify-guardian',
},
];
Invariant monitoring
Most reliable detector — continuous financial invariant checking:
interface ProtocolInvariant {
name: string;
check: (state: ProtocolState) => boolean;
description: string;
}
const invariants: ProtocolInvariant[] = [
{
name: 'total_supply_consistency',
check: (s) => s.totalShares * s.pricePerShare <= s.totalAssets * 1.001,
description: 'Total shares value never exceeds total assets',
},
{
name: 'no_sudden_tvl_drop',
check: (s) => s.currentTVL >= s.previousTVL * 0.8,
description: 'TVL didn't drop more than 20% per block',
},
];
Runbook and processes
T+0 (first alert):
- Guardian on-call gets Telegram/PagerDuty alert
- Assessment: false positive or real attack? 2-3 minutes max
- If doubt — pause immediately, analyze later
T+5 minutes:
- Pause executed (or decision not to pause)
- Core team notification
- On-chain investigation start (Tenderly transaction trace)
T+30 minutes:
- Public notification via Twitter/Discord: "We are investigating an issue"
- Damage assessment
- Coordination with other protocols (if cross-protocol attack)
T+24-48 hours:
- Public post-mortem
- Recovery plan
- Exchange coordination (freeze attacker addresses)
Guardian key security
Guardian key that can pause protocol is itself vulnerability. If compromised — attacker pauses protocol and extorts money for unblocking.
Recommendations:
- Guardian — Gnosis Safe 2/3 or 3/5, not EOA
- Hardware wallet for signing (Ledger/Trezor)
- Geographically distributed keyholders
- Regular drill exercises (quarterly)
- Monitor Guardian multisig itself
Development timeline: 2-2.5 months. On-chain development (1-2 weeks), monitoring system (2-3 weeks), runbook and testing (2-3 weeks), infrastructure integration (1 week).







