DBAtlas · Database operations atlas

cluster_block_exception: ClusterBlockException[blocked by: [TOO_MANY_REQUESTS/12/disk usage exceeded flood-stage watermark, index has read-only-allow-delete block]] (before 7.x: FORBIDDEN/12/index read-only / allow delete (api))

Elasticsearch

What does it mean?

Because disk usage exceeded the flood stage watermark (default 95%; on large disks determined by max_headroom), Elasticsearch placed a read_only_allow_delete block on indices that have a shard on the affected node, to keep the disk from filling up completely. Because of this block, writes to the index (index/update/delete document) are not possible; only deleting the index itself is allowed. ES 8.x returns this error with HTTP 429.

Why does it happen?

Disk usage on at least one node exceeded the cluster.routing.allocation.disk.watermark.flood_stage threshold. Even if the block is removed manually, Elasticsearch reapplies it while the disk is still full. Large indices without log rotation, uncleaned snapshots or insufficient disk capacity are typical causes.

How to fix it

First free up disk space: delete old/unneeded indices (DELETE /old-index), take a snapshot and remove indices, or grow the disk. In 7.4+, the block is removed automatically when disk usage drops below the high watermark; to clear it immediately without waiting, remove it manually: PUT /_all/_settings {"index.blocks.read_only_allow_delete": null}. For a permanent fix, delete/archive old data automatically with ILM (Index Lifecycle Management) and review the watermark settings (disk.watermark.low/high/flood_stage).

Open in DBAtlas →

More from this engine