Skip to main content

Troubleshooting: Volumes Stuck in Attach/Detach Loop When Using Longhorn on OKD

· 2 min read
Jack Lin

Applicable versions​

All Longhorn versions.

Symptoms​

All volumes stuck in Attach/Detach loop. By using dmesg on storage nodes you can see errors like the following:

[Sat Dec 10 18:52:01 2022] audit: type=1400 audit(1670698321.515:7214): avc:  denied  { dac_override } for  pid=231579 comm="iscsiadm" capability=1  scontext=system_u:system_r:iscsid_t:s0 tcontext=system_u:system_r:iscsid_t:s0 tclass=capability permissive=0
[Sat Dec 10 18:52:01 2022] audit: type=1300 audit(1670698321.515:7214): arch=c000003e syscall=83 success=no exit=-13 a0=55b9035185c0 a1=1f8 a2=ffffffffffffff00 a3=0 items=0 ppid=231163 pid=231579 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iscsiadm" exe="/usr/sbin/iscsiadm" subj=system_u:system_r:iscsid_t:s0 key=(null)
[Sat Dec 10 18:52:01 2022] audit: type=1327 audit(1670698321.515:7214): proctitle=697363736961646D002D6D00646973636F76657279002D740073656E6474617267657473002D700031302E3133312E312E31363

Troubleshooting: Velero restores Longhorn PersistentVolumeClaim stuck in the Pending state when using the Velero CSI Plugin version before v0.4.0

· 2 min read
Ray Chang

Applicable versions​

All Longhorn versions.

Symptoms​

PersistentVolumeClaim is stuck in the Pending state when restoring Longhorn with Velero with Velero CSI Plugin version before v0.4.0.

Reason​

For Longhorn versions using longhornio/csi-provisioner:v2.1.2, when it processes a PVC to provision the volume, Longhorn CSI provisioner will only recognize the volume.beta.kubernetes.io/storage-provisioner annotation which will be tagged together with volume.kubernetes.io/storage-provisioner to each PVC via Kubernetes after determining the storage provisioner. The PVC with these annotations will be backed up together via Velero.