high_disk_watermark_exceeded: high disk watermark [90%] exceeded / flood stage disk watermark [95%] exceeded

What does it mean?

A node's disk usage exceeded the high watermark (default 90%) or flood stage (default 95%) threshold. When the high watermark is exceeded, Elasticsearch tries to move shards from that node to other nodes; when flood stage is exceeded, it makes indices read-only.

Why does it happen?

The disk is filling up: continuously growing log/metric indices, old indices never deleted, an uncleaned snapshot repository, an undersized disk, or a single node overfilling due to unbalanced shard distribution.

How to fix it

Inspect disk usage: GET _cat/allocation?v. Delete old indices or snapshot and remove them, and grow the disk. You can temporarily raise the thresholds (carefully; in 8.x set all three together because of max_headroom validation): PUT _cluster/settings {"persistent":{"cluster.routing.allocation.disk.watermark.low":"88%","cluster.routing.allocation.disk.watermark.high":"93%","cluster.routing.allocation.disk.watermark.flood_stage":"96%"}}. Once resolved, revert to the defaults by setting the same settings to null (transient settings are deprecated; do not use them). Permanent fix: lifecycle management with ILM, regular index cleanup and capacity planning. If indices became read-only due to flood stage, remove the block separately.