etcdHighCommitDurations warning

[aug70]

I have a single master on a Kubernetes cluster and I'm getting below warning most of the time.
I'm not sure how to resolve this warning message?

etcdHighCommitDurations (1 active)
name: etcdHighCommitDurations
expr: histogram_quantile(0.99, rate(etcd_disk_backend_commit_duration_seconds_bucket{job=~".*etcd.*"}[5m])) > 0.25
for: 10m
labels:
severity: warning
annotations:
description: etcd cluster "{{ $labels.job }}": 99th percentile commit durations {{ $value }}s on etcd instance {{ $labels.instance }}.
summary: etcd cluster 99th percentile commit durations are too high.

When I execute below query

kubectl exec -it etcd-master \
-n kube-system -- etcdctl  \
--insecure-skip-tls-verify=true \
--cert=/etc/kubernetes/pki/etcd/server.crt  \
--key=/etc/kubernetes/pki/etcd/server.key \
--endpoints=https://10.35.0.105:2379 \
member list -w table endpoint status

The result is

+------------------+---------+--------+--------------------------+--------------------------+------------+
|        ID        | STATUS  |  NAME  |        PEER ADDRS        |       CLIENT ADDRS       | IS LEARNER |
+------------------+---------+--------+--------------------------+--------------------------+------------+
| b0e3eeee643de299 | started | master | https://10.35.0.105:2380 | https://10.35.0.105:2379 |      false |
+------------------+---------+--------+--------------------------+--------------------------+------------+

More on volumes

| Name |  Read Only | Mount Path | Sub Path | Source Type | Source Name
| etcd-data | false | /var/lib/etcd | - | HostPath | /var/lib/etcd |
| etcd-certs | false | /etc/kubernetes/pki/etcd | - | HostPath | /etc/kubernetes/pki/etcd |

[ghouscht]

This usually indicates poor disk performance. See the Disk section in https://etcd.io/docs/v3.5/metrics:

High disk operation latencies (wal_fsync_duration_seconds or backend_commit_duration_seconds) often indicate disk issues. It may cause high request latency or make the cluster unstable.

There is also a section dedicated to disks in the hardware recommendations: https://etcd.io/docs/v3.5/op-guide/hardware/

[aug70]

Thanks, I have been able to resolve this by zfs set sync=disabled rpool and make it disappear.