We need to inform you that access to all NHR@FAU systems had to be taken offline immediately.
NHR@FAU follows many other HPC centers who took the same action.
The reason for this is a potential security issue in the Linux kernel that requires urgent investigation and patching.
What does this mean for you?
- Login Access: You cannot log into any NHR@FAU system at this time.
Also access via Jupyterhub is currently not possible.
- Running and Queued Jobs:
– We try to keep already running batch jobs running.
– Interactive jobs had to be killed.
– Waiting jobs will be delayed as all clusters are reserved.
We will roll out a fix as soon as Ubuntu / RedHat / AlmaLinux provide patches for the different issues. Further information will be announced as it becomes available.
Thank you for your understanding and cooperation.
In the meantime, also RRZE published a general notice: https://www.rrze.fau.de/2026/07/warnung-wieder-eine-local-privilege-escalation-im-linux-kernel-veroeffentlicht-2026-07-09/
Update 2026-07-09 11:30: We by now have produced a seemingly working patched kernel for our AlmaLinux 9 systems. Assuming no further problems or new exploits show up, our AlmaLinux 9 systems (Alex, parts of Fritz, Helma) might be back soon. There is still no ETA whatsoever for all other systems.
Update 2026-07-09 16:00: Batch processing on Helma has been resumed. Login to the frontends is also possible again, but please note that the csnhr is still unavailable, so you can only connect if you connect directly via IPv6. Otherwise, you’ll have to wait until csnhr is back too.
Update 2026-07-09 18:00: Batch processing on Alex has been resumed. Login to the frontends is also possible again, with the same caveat as on Helma.
Update 2026-07-09 18:15: Batch processing on some of the AlmaLinux 9 nodes in Fritz has been resumed. Login to the frontends is also possible again, with the same caveat as on Helma. Work will continue tomorrow.
Update 2026-07-10 09:15: Batch processing on Woody has been resumed. Login to the frontends is also possible again, with the same caveat as on Helma.
Update 2026-07-10 09:50: csnhr is available again. The caveats from the updates above no longer apply – login to the frontends of those clusters that are back should now work normally again.
Update 2026-07-10 14:50: over the last hours, the rest of Fritz has been brought back online.
Update 2026-07-10 18:45: large parts of TinyGPU and TinyFAT have been brought back online.
Still to be done: parts of the Testcluster & final works on all systems