Skip to main content

Troubleshooting: Unexpected expansion leads to degradation or attach failure

· 8 min read
Eric Weber

Applicable versions

Confirmed in:

  • Longhorn v1.3.2 - v1.3.3
  • Longhorn v1.4.0 - v1.4.2
  • Longhorn v1.5.0

Potentially mitigated in:

  • Longhorn v1.4.3
  • Longhorn v1.5.1

Complete fix planned in:

  • Longhorn v1.4.x
  • Longhorn v1.5.x
  • Longhorn v1.6.0

Symptoms

While the root cause is always the same, symptoms can vary depending on other factors (e.g. whether there are multiple

Troubleshooting: Failure to delete orphaned Pod volume directory

· 2 min read
Ray Chang

Applicable versions

All Longhorn versions.

Kubernetes versions before v1.28. A backported PR to v1.27 is awaiting merging.

Symptoms

In the event of a worker node failure, while hosting active Pods, the Pods are gracefully evicted as the node undergoes downtime and awaits restoration. During this period, the kubelet, which is responsible for managing the node, will generate the following error messages at regular intervals of two seconds.

orphaned pod <pod-uid> found, but error not a directory occurred when trying to remove the volumes dir