Input/Output Errors During XFS Filesystem Access In CentOS/RHEL 7

The Problem

While accessing a directory mounted on XFS filesystem, it is throwing Input/Output errors on any command as in the example below:

[root@OL7-VM mnt]# cd /u01
[root@OL7-VM u01]# ll
ls: cannot open directory .: Input/output error

[root@OL7-VM u01]# cd /mnt
[root@OL7-VM mnt]# ll
ls: cannot open directory .: Input/output error

The Solution

It appears that the issue was caused by some type of corruption on the filesystem that caused it became unresponsive. Run file system check and repair the filesystem corruption.

Possible scenarios:

1. The affected device is a physical volume of the LVM where the root filesystem is mounted on it.
2. The affected device is a physical volume of the LVM where the root filesystem is not mounted on it
3. The affected device is not a physical volume of the LVM configuration but root filesystem is mounted on it
4. The affected device is not a physical volume of the LVM configuration where the root filesystem is not mounted on it but another filesystem is

For scenario 1 and 3:

1. Reboot the system and boot with Rescue mode

NOTE: Choose ‘Skip‘ instead of ‘Continue‘ while booting CentOS/RHEL 7 into rescue mode to make sure file system of system are not mounted when booting into rescue mode.

2. Run the following commands to activate the LVM.

# lvm pvscan 
# lvm vgscan
# lvm lvscan
# lvm lvchange -ay /dev/mapper/vgName-lvName

3. Run file system check and repair.

# xfs_repair /dev/deviceName

4. Reboot the system

# shutdown -r now

For scenario 2 and 4:

There is no need to boot to rescue mode, the filesystem check and repair can run while the system is up and running. Just make sure the system is not in production.

1. Unmount the filesystem

# umount /file_systemName

2. Run file system check and repair

# xfs_repair /dev/deviceName

3. Reboot the system.

# shutdown -r now