Patch Name: PHKL_23122 Patch Description: s700_800 11.04 (VVOS) ufs inode hangs&race of mkdir,umount Creation Date: 01/01/30 Post Date: 01/03/17 Hardware Platforms - OS Releases: s700: 11.04 s800: 11.04 Products: N/A Filesets: OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_32,v=HP OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_64,v=HP Automatic Reboot?: Yes Status: General Release Critical: Yes PHKL_23122: PANIC Category Tags: defect_repair general_release critical panic Path Name: /hp-ux_patches/s700_800/11.X/PHKL_23122 Symptoms: PHKL_23122: Port base HP-UX patch PHKL_22635 for VVOS. Based on PHKL_22635: ( SR: 8606135807 CR: JAGad04939 ) Data Page fault due to a race between mkdir( ) and unmount on a filesystem with quotas. The stack trace for this race condition looks like: stack trace for event 0 crash event was a panic panic+0x14 dqput+0x38 dqrele+0xb8 ieget+0x1b4 in_get+0x30 iget+0x18 ialloc+0x114 dirmakeinode+0xc8 direnter+0x1e4 ufs_mkdir+0x4c vns_create+0x168 vn_create+0xcc mkdir+0x80 syscall+0x480 $syscallrtn+0x0 Based on PHKL_21512: ( SR: 8606125420 CR: JAGac40810 ) Note that for this patch the symptoms are the same as the patch it supersedes. This patch addresses a second, and similar, race condition. Processes reading/writing inodes on a ufs file system may sleep (and hang) indefinitely waiting for the inode lock to become available. Based on PHKL_19773: ( SR: 8606100744 CR: JAGab31988 ) Processes reading/writing inodes on a ufs file system may sleep (and hang) indefinitely waiting for the inode lock to become available. Defect Description: PHKL_23122: Ported HP-UX patch PHKL_22635 for VVOS. Resolution: Ported HP-UX patch PHKL_22635. Based on PHKL_22635: ( SR: 8606135807 CR: JAGad04939 ) This is due to a race between two threads one running mkdir( ) and the other umount( ) at the same time, on a filesystem with quotas on. umount( ) grabs the filesystem semaphore, but has not yet gotten rid of the quota (dquot) structure. At this time mkdir( ) seeing the dquot structure still valid tries to execute dqrele(), but blocks or sleeps because the filesystem semaphore is held by umount( ). umount( ) finishes what it is doing, including getting rid of the dquot structure. It then releases the filesystem semaphore. dqrele() now gets the filesystem semaphore and continues. It dosen't check again to see if the dquot structure is still valid. At this point it is trying to release a dquot for a filesystem that has already been unmounted, and thus panics. Resolution: When releasing the quota (dquot) structure for an inode pointer verify that the dquot structure is valid both before and after acquiring the filesystem semaphore. Based on PHKL_21512: The problem is race condition where an inode is being purged. There are two possibilities during this process: (In PHKL_19773 we only addressed the first possible race condition) 1. Before the inode been moved to freelist, several threads find this inode on the head of the incore inode list, and sleep on it waiting to do their read/write; 2. After the inode been moved to freelist, the first thread which was waiting for it was awakened and locked the inode, but at same time other threads trying to get a free inode will get the same inode from freelist, and they will all sleep on that inode since it has been locked by the first thread. Both conditions cause multiple threads sleeping on one inode of which the identity or status is about to be changed (to be moved from inode list to freelist, and/or from freelist to inode list). If the inode identity or status has been changed during the sleep time, after waking up and retrying, a thread will get a new inode which is different from the old one it was sleeping on last time, and continue processing with the new inode without waking up the rest of threads that are still sleeping on the old inode. Those threads will hang forever. Resolution: The solution is waking up the next sleeper which is still sleeping on the old inode if the new inode is different from the old. Based on PHKL_19773: ( SR: 8606100744 CR: JAGab31988 ) The problem is race condition where an inode is being purged. During this process, several threads find this inode on the head of the incore inode list, and sleep on it waiting to do their read/write. When the inode is being purged, it is placed on the freelist, and yet another thread finds it there and changes the inode's identity. While the threads sleeping on the original identity will properly check for this condition, and find a different inode once awakened, the problem is that if a number of threads are sleeping on the original identity, they may not all be awakened. So, some threads may hang forever. Resolution: The solution is to wake up all readers and writers sleeping on an incore inode when a purge of that inode is done. When this happens, the processes will realize the identity of the inode has changed, and will properly look for a different inode. SR: 8606100744 8606125420 8606135807 Patch Files: OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_32,v=HP: /usr/conf/lib/libhp-ux.a(quota_ufs.o) /usr/conf/lib/libufs.a(ufs_inode.o) OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_64,v=HP: /usr/conf/lib/libhp-ux.a(quota_ufs.o) /usr/conf/lib/libufs.a(ufs_inode.o) what(1) Output: OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_32,v=HP: /usr/conf/lib/libhp-ux.a(quota_ufs.o): $Source: kern/ufs/quota_ufs.c, hpuxsysufs, vvos_rose , rose0196 $ $Date: 01/01/10 08:57:34 $ $Rev ision: 1.10 PATCH_11.04 (PHKL_23122) $ /usr/conf/lib/libufs.a(ufs_inode.o): $Source: kern/ufs/ufs_inode.c, hpuxsysufs, vvos_rose , rose0196 $ $Date: 01/01/10 11:27:08 $ $Rev ision: 1.41 PATCH_11.04 (PHKL_23122) $ OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_64,v=HP: /usr/conf/lib/libhp-ux.a(quota_ufs.o): $Source: kern/ufs/quota_ufs.c, hpuxsysufs, vvos_rose , rose0196 $ $Date: 01/01/10 08:57:34 $ $Rev ision: 1.10 PATCH_11.04 (PHKL_23122) $ /usr/conf/lib/libufs.a(ufs_inode.o): $Source: kern/ufs/ufs_inode.c, hpuxsysufs, vvos_rose , rose0196 $ $Date: 01/01/10 11:27:08 $ $Rev ision: 1.41 PATCH_11.04 (PHKL_23122) $ cksum(1) Output: OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_32,v=HP: 2860488559 6684 /usr/conf/lib/libhp-ux.a(quota_ufs.o) 2712107185 26260 /usr/conf/lib/libufs.a(ufs_inode.o) OS-Core.CORE2-KRN,fr=B.11.04,fa=HP-UX_B.11.04_64,v=HP: 1192967662 15080 /usr/conf/lib/libhp-ux.a(quota_ufs.o) 1078652771 62984 /usr/conf/lib/libufs.a(ufs_inode.o) Patch Conflicts: None Patch Dependencies: s700: 11.04: PHKL_19142 s800: 11.04: PHKL_19142 Hardware Dependencies: None Other Dependencies: None Supersedes: None Equivalent Patches: PHKL_22635: s700: 11.00 s800: 11.00 Patch Package Size: 150 KBytes Installation Instructions: Please review all instructions and the Hewlett-Packard SupportLine User Guide or your Hewlett-Packard support terms and conditions for precautions, scope of license, restrictions, and, limitation of liability and warranties, before installing this patch. ------------------------------------------------------------ 1. Back up your system before installing a patch. 2. Login as root. 3. Copy the patch to the /tmp directory. 4. Move to the /tmp directory and unshar the patch: cd /tmp sh PHKL_23122 5. Run swinstall to install the patch: swinstall -x autoreboot=true -x patch_match_target=true \ -s /tmp/PHKL_23122.depot By default swinstall will archive the original software in /var/adm/sw/save/PHKL_23122. If you do not wish to retain a copy of the original software, use the patch_save_files option: swinstall -x autoreboot=true -x patch_match_target=true \ -x patch_save_files=false -s /tmp/PHKL_23122.depot WARNING: If patch_save_files is false when a patch is installed, the patch cannot be deinstalled. Please be careful when using this feature. For future reference, the contents of the PHKL_23122.text file is available in the product readme: swlist -l product -a readme -d @ /tmp/PHKL_23122.depot To put this patch on a magnetic tape and install from the tape drive, use the command: dd if=/tmp/PHKL_23122.depot of=/dev/rmt/0m bs=2k Special Installation Instructions: This patch depends on base patch PHKL_19142. For successful installation please insure that PHKL_19142 is already installed, or that PHKL_19142 is included in the same depot with this patch and PHKL_19142 is selected for installation.