Proposal: Implement Database Purge for TTL Nodes

Validators
,
1

Summary

Casper validators currently operate in two modes:

  • Archival nodes: Maintain the full history of the blockchain.

  • TTL (recent-data) nodes: Maintain only recent data within a defined time-to-live (TTL).

While TTL nodes are designed to store a smaller dataset, in practice their database continues to grow indefinitely because there is no purge mechanism to remove old, unnecessary data. This results in large and inefficient databases even for TTL nodes, which increases storage requirements and operational costs for validators.

To resolve this, I propose the implementation of a database purge feature that automatically removes expired data from TTL nodes.

Problem

  • TTL nodes are intended to hold only recent blockchain data.

  • Currently, old data accumulates without removal, leading to large databases even on TTL nodes.

  • This increases storage usage, slows down operations, and creates unnecessary overhead for validators.

  • Without purge functionality, the difference between TTL and archival nodes is diminished over time.

Proposed Change

  1. Introduce a purge mechanism in the Casper database layer for TTL nodes.

    • Periodically remove blocks, state data, and other entries that are older than the configured TTL.

    • Ensure the process does not affect network consensus or validator performance.

  2. Configuration option:

    • Allow node operators to configure purge intervals or thresholds (e.g., “keep only the last X eras/blocks”).

    • Ensure archival nodes remain unaffected, preserving full history.

Benefits

  • Keeps TTL node databases compact and aligned with their purpose.

  • Reduces disk space requirements and improves validator efficiency.

  • Lowers validator operational costs, encouraging decentralization.

  • Prevents TTL nodes from unintentionally becoming semi-archival.

2

Long time ago Joe said it will be hard to implement with actual LMDB schema. It was a discussed on The Casper Validators Guild group on telegram quite not long time ago (2 months?). Then i proposed to use PostgreSQL/MariaDB to optimize switch block and increase TPS. Maybe its time to check how other blockains have resolved that (for example Monero use LMDB, they prune old data what we select based on time, and rest are copy to new DB, and replace).