Troubleshooting and Triage FAQ

Student Cannot Write Files

Check the number of inodes the student is using. We have limited to 60K


Disk Full

Local Disk is Full

First thing to understand is that ICS Computing Support does not delete user data. We help users clean up their own data.

Identify where your local storage is at:

# df -l
Filesystem                               1K-blocks      Used  Available Use% Mounted on
udev                                     264130348        12  264130336   1% /dev
tmpfs                                     52828320      2312   52826008   1% /run
/dev/dm-2                                 24478084  23271192          0 100% /
none                                             4         0          4   0% /sys/fs/cgroup
none                                          5120         0       5120   0% /run/lock
none                                     264141592    102440  264039152   1% /run/shm
none                                        102400         0     102400   0% /run/user
/dev/sdb1                                   967320     38440     862528   5% /boot
/dev/mapper/vg_02_zen-lv_zen_disk00     7028490240 411913704 6616576536   6% /srv/disk00
/dev/mapper/vg_03_zen_ssd-lv_zen_disk01 1556166248 746202144  809964104  48% /srv/disk01

We see that / is full. Try to identify what is filling it up using du. Run du, use the -x switch so that du does not

du -xsh /

Of course, that -x switch is useless if we want to know which directory is eating up all of our storage:

# du -xsh /var /scratch.old /usr /scratch.puppet-bak/ bin boot etc export lib lib32 lib64 lost+found lv_scratch

Login Failures

Running Testaccess

Run the command `/home/support/bin/testaccess <username>` to determine if the user is on the appropriate lists. /home

This user is allowed to login:

# /home/support/bin/testaccess lopes
Username            Status
lopes               ACCESS PERMITTED

This user would not be allowed to login:

# /home/support/bin/testaccess yuzok
Username            Status
yuzok               ACCESS DENIED

TestAccess: Access Denied

Make sure the user is a member of the appropriate netgroup.

Look in /var/log/secure

If user is in the correct netgroup run: sss_cache -E Try testaccess again

Could not set limit for 'nofile'

Ticket #63897

It turns out that unlimited is not a proper value for nofiles in /etc/security/limits.d/10-owner-limits.conf. Set this to best practices 63536 at maximum (recommended 1024 for openlab and 4096 for most research hosts).

May 18 09:46:15 styx su: pam_limits(su-l:session): Could not set limit for 'nofile': Operation not permitted
troubleshooting_and_triage_faq.txt · Last modified: 2019/10/04 14:57 by wdcohen
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0