High latency for 5.5% namespaces in gcp-us-central1
Incident Report for turbopuffer
Resolved
Between 20:00 and 01:04 UTC 5.5% of namespaces in gcp-us-central1 may have experienced latency to the point that it may have caused timeouts. The root cause was a maxed out NVMe volume on a single query node in gcp-us-central1. The team's taking additional actions to address the root cause of the high NVMe utilization.
Posted Dec 01, 2024 - 20:00 UTC