SysRq utility
In case your kernel freezes or/and hangs, you can use ‘magic’ SysRq tool. The ‘magic’ SysRq key allows instant access to several essential kernel commands, including Sync, Unmount, kill processes, showMem, showPc and reBoot. According to the Linux kernel documentation:
It is a ‘magical’ key combo you can hit to which the kernel will respond regardless of whatever else it is doing, even if the console is unresponsive.
How to Enable SysRq
The SysRq key can be activated by setting the ‘Magic SysRq’. You have to make sure that the kernel.sysrq in sysctl.conf is set properly:
kernel.sysrq = 1 (SysRq turned ON) kernel.sysrq = 0 (SysRq turned OFF)
Note:
On CentOS/RHEL 7 you should make the changes into “/usr/lib/sysctl.d/50-default.conf”
The above settings ensure that SysRq is enabled by default after system boot (By default it is disabled in CentOS/RHEL). To enable it, run in right away in the current session, use the below command:
# echo 1 > /proc/sys/kernel/sysrq
And to disable it again:
echo 0 > /proc/sys/kernel/sysrq
NOTE: Since enabling SysRq gives someone with physical console access extra abilities, it is recommended to disable it when not troubleshooting a problem or to ensure that physical console access is properly secured.
How to trigger a SysRq event
There are several ways to trigger a SysRq event. On most architectures SysRq events can be triggered from the console with the following key combination:
Alt+PrintScreen+[CommandKey]
The PrintScreen is the Key used as SysRq key in most of Linux the systems.
To access a non responsive system, press Alt+SysRq and H to get a list of options. Different kernel version may have different options. To get the supported options for your current kernel, you can run the command below as root:
# echo h > /proc/sysrq-trigger
and check dmesg output. Below is an example on RHEL 7 system:
# dmesg [ 4338.124035] SysRq : HELP : loglevel(0-9) reboot(b) crash(c) terminate-all-tasks(e) memory-full-oom-kill(f) kill-all-tasks(i) thaw-filesystems(j) sak(k) show-backtrace-all-active-cpus(l) show-memory-usage(m) nice-all-RT-tasks(n) poweroff(o) show-registers(p) show-all-timers(q) unraw(r) sync(s) show-task-states(t) unmount(u) force-fb(V) show-blocked-tasks(w) dump-ftrace-buffer(z)
Types of SysRq events
| Key | Function |
|---|---|
| m | dump information about memory allocation |
| t | dump thread state information |
| p | dump current CPU registers and flags |
| c | intentionally crash the system (kernel panic). Useful for capturing a vmcore through kdump. |
| s | immediately sync all mounted filesystems |
| u | immediately remount all filesystems read-only |
| b | immediately reboot the machine |
| o | immediately power off the machine (if configured and supported) |
| f | start the Out Of Memory Killer (OOM) |
| w | dumps tasks that are in uninterruptible (blocked) state |
There are several SysRq events that can be triggered once the SysRq facility is enabled. These vary somewhat between kernel versions, but there are a few that are commonly used:
How to trigger a SysRq event from root shell
If you have a root shell on the machine (and the system is responding enough for you to do so), you can also write the command key character to the /proc/sysrq-trigger file. This is useful for triggering this info when you are not on the system console or for triggering it from scripts.
# echo 'm' > /proc/sysrq-trigger
This method has the additional benefit of working even when kernel.sysrq is set to 0.
