[resolved] Problems with Homedirectories

checkmark-sunshine

There are currently problems with the homedirectories under /home/hpc and /home/vault.

Attempts to write there may fail with error messages like “no space left on device”, “disk quota exceeded” or even “permission denied”. This can also happen on attempts to delete files. Simply reading files is not affected.

We’re on it.

Update 1: These should be gone for now since about 14:20, but the underlying problem is NOT fixed yet, so these may reoccour.

Final Update 2026-10-08 10:00: the system is stable again. However, the root cause for the problems were users that produce excessive change rates (massive amounts of everchanging files) that our existing hardware just cannot handle anymore. As a result, we have had to change a few settings that will reduce the level of comfort on these filesystems:

  • the quota for the number of files on /home/hpc / $HOME has been reduced from 500.000 to 300.000.
  • snapshots on /home/hpc / $HOME will no longer happen every 30 minutes. The minimum interval is now 2 hours. The timespan covered by the 2-hour-snapshots has been reduced from 24 to 12 hours. The timespan covered by the daily snapshots has been reduced from 7 to 4 days. The timespan covered by the weekly snapshots has been reduced from 4 to 3 weeks.
  • on /home/vault / $VAULT, the timespan covered by the weekly snapshots has been reduced from 4 to 3 weeks.