Troubleshooting and Triage FAQ
Student Cannot Write Files
Check the number of inodes the student is using. We have limited to 60K
/home/support/bin/inodecount
Disk Full
Local Disk is Full
First thing to understand is that ICS Computing Support does not delete user data. We help users clean up their own data.
Identify where your local storage is at:
# df -l Filesystem 1K-blocks Used Available Use% Mounted on udev 264130348 12 264130336 1% /dev tmpfs 52828320 2312 52826008 1% /run /dev/dm-2 24478084 23271192 0 100% / none 4 0 4 0% /sys/fs/cgroup none 5120 0 5120 0% /run/lock none 264141592 102440 264039152 1% /run/shm none 102400 0 102400 0% /run/user /dev/sdb1 967320 38440 862528 5% /boot /dev/mapper/vg_02_zen-lv_zen_disk00 7028490240 411913704 6616576536 6% /srv/disk00 /dev/mapper/vg_03_zen_ssd-lv_zen_disk01 1556166248 746202144 809964104 48% /srv/disk01
We see that / is full. Try to identify what is filling it up using du. Run du, use the -x switch so that du does not
du -xsh /
Of course, that -x switch is useless if we want to know which directory is eating up all of our storage:
# du -xsh /var /scratch.old /usr /scratch.puppet-bak/ bin boot etc export lib lib32 lib64 lost+found lv_scratch
Login Failures
Running Testaccess
Run the command `/home/support/bin/testaccess <username>` to determine if the user is on the appropriate lists. /home
This user is allowed to login:
# /home/support/bin/testaccess lopes Username Status ---------------------------------------- lopes ACCESS PERMITTED
This user would not be allowed to login:
# /home/support/bin/testaccess yuzok Username Status ---------------------------------------- yuzok ACCESS DENIED
TestAccess: Access Denied
Make sure the user is a member of the appropriate netgroup.
Look in /var/log/secure
If user is in the correct netgroup run: sss_cache -E Try testaccess again
Could not set limit for 'nofile'
Ticket #63897
It turns out that unlimited is not a proper value for nofiles in /etc/security/limits.d/10-owner-limits.conf. Set this to best practices 63536 at maximum (recommended 1024 for openlab and 4096 for most research hosts).
May 18 09:46:15 styx su: pam_limits(su-l:session): Could not set limit for 'nofile': Operation not permitted