Patch Name: PHKL_30557 Patch Description: s700_800 11.11 ufalloc;VxFS3.5;SPP fragmentation;AIO;EVP Creation Date: 04/04/22 Post Date: 04/05/11 Hardware Platforms - OS Releases: s700: 11.11 s800: 11.11 Products: N/A Filesets: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP Automatic Reboot?: Yes Status: General Release Critical: No (superseded patches were critical) PHKL_30541: PANIC PHKL_30317: HANG PHKL_29826: PANIC Debug kernel panics while walking a list of ~60000 open file descriptors. PHKL_25871: HANG Mutithreaded proceses may hang in socket close(2) Category Tags: defect_repair enhancement general_release critical panic halts_system manual_dependencies Path Name: /hp-ux_patches/s700_800/11.X/PHKL_30557 Symptoms: PHKL_30557: ( SR:8606275342 CR:JAGae39417 ) This product update is a member of a set needed to enable the optional HP-UX VFS-AdvSysCall and HP-UX FUSER-Enh features. Upon installation, the HP-UX VFS-AdvSysCall and HP-UX FUSER-Enh bundle (NEWFUSER11i) will install the full set of product update (including this one) to enable the VFS-AdvSysCall and FUSER-Enh features. If the HP-UX VFS-AdvSysCall or HP-UX FUSER-Enh product is not installed, this product update will have no impact on your system. PHKL_30541: ( SR:8606342243 CR:JAGaf03150 ) Panic occurs during an ioctl(2) or close on an event port device. The stack traces can look like: soo_select+0x8 unp_poll_handler+0x30 so_poll_switch+0x1c evp_dp_poll+0x1d8 evp_ioctl+0xcc spec_ioctl+0xac vno_ioctl+0x98 ioctl+0x118 or so_eventreg+0x78 evp_dereg_objhdr+0x64 evp_close+0x188 call_open_close+0x28c closed+0xe4 spec_close+0x64 vn_close+0x48 vno_close+0x20 closef+0x64 PHKL_30317: ( SR:8606156998 CR:JAGad26332 ) A multithreaded process may hang with one thread inside exit and another thread continuously looping inside the AIO code. The stack trace of the thread sleeping in exit code can look like: _sleep+0x214 thread_halt_wait+0x198 for_specific_threads+0xc0 process_wide_halt_wait+0x2c exit+0x3a0 The stack trace of the thread looping inside the AIO code can look like: sleep_spinunlock+0x60 aio_shutdown_fd_begin+0x160 close+0x28 PHKL_29826: ( SR:8606326634 CR:JAGae88911 ) Processes get deactivated due to memory pressure. The processes don't get reactivated even when free memory is increased. The memory pressure is undue as there are free memory pages on the superpage pool. Reports generated using kmeminfo tool show a lot of memory on the superpage free pool. ( SR:8606306720 CR:JAGae69754 ) When there are ~60000 file descriptors open for a process then a close on a file descriptor will lead to any of the below symptoms. 1. panic on a debuf kernel. The stack trace may be similar to the following. panic+0x6c log_ticks+0x31c set_it+0x170 clock_int+0x2d0 mp_ext_interrupt+0x3f4 ivti_patch_to_nop3+0x0 spinunlock+0x48 vno_lockrelease+0x10c closef+0x24 close+0xe0 syscall+0x790 syscallrtn+0x0 2. performance loss on a perf kernel. PHKL_25995: ( SR:8606217733 CR:JAGad86885 ) Duplicate ( SR:8606217874 CR:JAGad87024 ) Duplicate ( SR:8606212631 CR:JAGad81817 ) Enhancement: This product update is a member of a set needed to enable Fast File Descriptor allocation. The full list of product updates required for this feature are: PHKL_25995, PHKL_25993, PHKL_25994, PHKL_25996. If any member of this set of product updates is not installed, this product update will have no impact on your system. Performance decreases when a large number of file descriptors are open and as more file descriptors are needed, there is an increase in the time spent in the open(2) call. PHKL_25871: ( SR:8606178928 CR:JAGad48153 ) A process may hang when attempting to perform a close(2) on a file descriptor created by the socket(2) system call. PHKL_25611: ( SR:8606222220 CR:JAGad91335 ) This patch is a member of a set of patches needed to enable the eventport pseudo driver feature delivered in PHKL_25468. The eventport driver patch specifies the full set of required patches for this new feature. If the eventport pseudo driver patch (or superseding patch) is not installed, this change will have no impact on you system. PHKL_29759: ( SR:8606329864 CR:JAGae90987 ) When applications that use chroot() system call are operated on a directory with the following properties 1. When accessed, autoFS will mount a filesystem on this directory using direct map. 2. The mount operation has not taken place on this directory when chroot() is called on it. many unusual failures are encountered. One such application is ftp where operations like put, mput, etc., will fail with an error message when performed on a directory with the above mentioned properties. An example of such error message is shown below. Aug 8 15:17:00 q9j9a ftpd[23183]: Could not determine cwdir: No such file or directory. A simple application code in this case will look like the following. chroot("/users/nfsu1") chdir("/dir1") where "/users/nfsu1" is a direct mount point managed by AutoFS. ( SR:8606330941 CR:JAGae92062 ) This product update is a member of a set needed to enable the optional HP-UX VFS-Enh feature. Upon installation, the HP-UX VFS-Enh bundle (EnhancedVFS) will install the full set of product updates (including this one) to enable the VFS-Enh feature. If the HP-UX VFS-Enh product is not installed, this product update will have no impact on your system. PHKL_28999: ( SR:8606295554 CR:JAGae59236 ) Automount fails to mount cdfs with PHKL_26239 and returns EBUSY. The problem is noticed when mounting non-NFS file systems using direct maps and not with indirect maps. This problem can also cause Block-Level Incremental Backups (BLIB) performed with the Veritas NetBackup application to fail with errno 16 (EBUSY). PHKL_27800: ( SR:8606263709 CR:JAGae28034 ) This product update is member of a set needed to enable the optional HP-UX support for VxFS 3.5 Cluster mount feature. Upon installation, the HP-UX BaseVxFS bundle will install the full set of product updates (including this one) necessary to enable the support for VxFS 3.5 Cluster mount feature. If the HP-UX VxFS 3.5 product is not installed, this product update will have no impact on your system. ( SR:8606198319 CR:JAGad67509 ) This product update is member of a set needed to enable the optional HP-UX VFS support for VxFS 3.5. Upon Installation, the HP-UX Base VxFS bundle will install the full set of product updates (including this one) necessary to enable the VFS support for VxFS 3.5. If the HP-UX VxFS 3.5 product is not installed, this product update will have no impact on your system. PHKL_26239: ( SR:8606247419 CR:JAGae13857 ) Enhancement: This product update is a member of a set which improve the performance of mount(2) and umount(2). The full list of product updates that address this performance improvement are: PHKL_26239, PHKL_26694, PHKL_26695, PHKL_26698, and PHKL_27211. Each of these product updates may be installed independently of the others. Each provides an incremental performance improvement. ( SR:8606241982 CR:JAGae09237 ) Enhancement: Provide device major numbers that are suitable for use in /etc/mnttab. ( SR:8606260639 CR:JAGae24962 ) opendir(3C) performance may be quite slow when there are a lot of directories involving long path names. ( SR:8606178913 CR:JAGad48138 ) Either or both of the following symptoms may appear on a given system: 1. Service Guard may experience umount(2) failures. 2. System throughput and/or response time may degrade when CPUs are added to a configuration that executes a significant number of calls to getcwd(3). PHKL_23239: ( SR:8606175337 CR:JAGad44579 ) Overall system throughput degradation when getmount_entry() and getmount_cnt() hit contention for the filesys_sema (an alpha semaphore). PHKL_26749: ( SR:8606248238 CR:JAGae14638 ) creat(2) system call fails with ENOENT. PHKL_26099: ( SR:8606233922 CR:JAGae03145 ) Applications using rename(2) may run slower than necessary. Defect Description: PHKL_30557: ( SR:8606275342 CR:JAGae39417 ) This product update contains minor enhancements required to enable the HP-UX VFS-AdvSysCall and HP-UX FUSER-Enh. Resolution: This patch enables new functions that will be called by the HP-UX VFS-AdvSysCall and HP-UX FUSER-Enh. PHKL_30541: ( SR:8606342243 CR:JAGaf03150 ) 1) A flag in the file descriptor was used to indicate if the file descriptor is registered with an event port. This implementation cannot fully support registering a file descriptor with multiple event ports. 2) A race condition exists between registration and deregistration of a file descriptor with the same event port. This can result in the file descriptor not properly flagged to indicate its registration with the event port. Resolution: 1) The flag in the file descriptor structure is replaced by a counter to keep track of how many event ports the file descriptor is registered. 2) The new counter is always incremented (decremented) accordingly upon successful registration (deregistration). PHKL_30317: ( SR:8606156998 CR:JAGad26332 ) The exiting thread was waiting for the thread looping inside the AIO code to finish. The thread inside the AIO code will call sleep to let the AIO operation finish. As the process is exiting the sleep will immediately be interrupted and the thread loops continuously calling sleep and being interrupted. Resolution: The thread which was looping inside the AIO code sets the EINTR after it is interrupted from its sleep and returns. PHKL_29826: ( SR:8606326634 CR:JAGae88911 ) When maxfiles tunable is set to 60000, approximately 15 pages are required for a table of pointers on 64 bit kernel. These 15 pages are carved from the superpage pool from a 16 page area, the remaining 1 page gets used by normal arena allocations. When the 15 pages are freed it cannot coalesce thus adding an entry on the 8 page free list, the 4 page free list, the 2 page freelist, and the 1 page free list. The next time we need 15 pages we must carve up a new 16 page and the process repeats. Thus leading to superpage pool fragmentation. Resolution: A request for anything between 14 and 16 pages is rounded up to 16 pages. This will lead to an extra allocation of maximum 2 pages, but fragmentation of superpage pool will not happen in case when maxfiles is set to 60000. ( SR:8606306720 CR:JAGae69754 ) When a process has ~60000 open file descriptors and it does a close on any of the file descriptors, it walks each file descriptor to check for a flag. This is done while holding a spinlock. Since the spinlock is held for a long time, it leads to a panic on a debug kernel and performance loss on a perf kernel. Resolution: When a lock is acquired on a file, a flag is set. On first close of the file if the flag is set, the lock is released for the whole file. PHKL_25995: ( SR:8606217733 CR:JAGad86885 ) Duplicate ( SR:8606217874 CR:JAGad87024 ) Duplicate ( SR:8606212631 CR:JAGad81817 ) As a user program opens a large number of file descriptors, more time is spent in the file allocation routines because of the current linear algorithm which results in a performance decrease. Resolution: The solution to this problem is to implement a more efficient algorithm for managing file descriptors. PHKL_25871: ( SR:8606178928 CR:JAGad48153 ) In a mutlithreaded situtation, socket close(2) can cause a program to hang as a result of a race condition between the close(2) and the creat(2) of the socket. Resolution: Resolution involved several kernel related subsystems. We now check to see if a thread in waiting in a state that could cause a process to block and we abort the call to the routine that would cause a hang. The patches listed in other dependencies are necessary to allow and detect an error code that is necessary for detection of the waiting thread. PHKL_25611: ( SR:8606222220 CR:JAGad91335 ) This change contains minor enhancements required to support the eventport feature. Resolution: Enhancements added include a file descriptor subsytem interface used by the eventport driver and respective eventport driver callbacks. PHKL_29759: ( SR:8606329864 CR:JAGae90987 ) When the system call chroot() is exercised on a directory with the following properties 1. When accessed, autoFS will mount a filesystem on this directory using direct map. 2. The mount operation has not taken place on this directory when chroot() is called on it. the effective root directory is not assigned with respect to the automounted filesystem. This results in many applications (that use chroot()) failing with different error messages. Resolution: Modified chroot() routine in such a way that the effective root directory is assigned with respect to the automounted NFS filesystem. ( SR:8606330941 CR:JAGae92062 ) This product update contains minor enhancements required to enable the HP-UX VFS-Enh. Resolution: This patch enables new functions that will be called by the HP-UX VFS-Enh. PHKL_28999: ( SR:8606295554 CR:JAGae59236 ) A flag is incorrectly checked when adding a VFS structure outside of the normal mount code path. The flag is properly checked in the normal mount code path, so the additional check is not needed. Resolution: The fix is to remove the check of the flag when adding a vfs structure. PHKL_27800: ( SR:8606263709 CR:JAGae28034 ) This product update contains minor enhancements required to enable the HP-UX support for VxFS 3.5 Cluster mount. Resolution: Enhancements added to disallow mounting another file system within a VxFS 3.5 Cluster mount. ( SR:8606198319 CR:JAGad67509 ) This product update contains minor enhancements required to enable the HP-UX VFS support for VxFS 3.5. Resolution: Enhancements added to remove VxFS 3.5 entries from VFS switch table. PHKL_26239: ( SR:8606247419 CR:JAGae13857 ) This product update contains minor changes necessary to improve the performance of mount(2) and umount(2) for all file systems. We incorrectly held a lock to serialize all access to all file systems for mount/umount service calls. These file systems specific operations are already serialized where necessary, and the additional higher level serialization results in unnecessary performance degradation. Resolution: Removed the unnecessary use of a file system lock that used to unconditionally serialize all access to all file systems for mount/umount service calls. ( SR:8606241982 CR:JAGae09237 ) Device major numbers are currently assigned the value 0xff. This value can not be used in /etc/mnttab, which needs a value between 0x0 and 0x7f. Resolution: Provide device major numbers that are in the 0x0 to 0x7f range. ( SR:8606260639 CR:JAGae24962 ) opendir(3C) uses open(2) and stat(2) calls, each with path name parameters. With large numbers of directories, there are multiple calls to open(2) and stat(2). Resolution: open(2) already has access to the information provided by stat(2) to opendir(3C). opendir(3C) uses open(2) with a new parameter to gain access to this additional information, thus avoiding the additional call to stat(2). ( SR:8606178913 CR:JAGad48138 ) The mechanism for retrieving information about mounted file systems from the kernel is extremely inefficient. Its cost is essentially a quadratic function of the number of mounted file systems. Resolution: Added a new system call to retrieve information for all currently mounted file systems, which makes the performance linear with the number of mounted file systems. PHKL_23239: ( SR:8606175337 CR:JAGad44579 ) It is not efficient to use the alpha semaphore to serialize access to the list of vfs structs. Resolution: Use the rootvfs_lock (a simple spinlock) instead of the filesys_sema to serialize access to the list of vfs structs. PHKL_26749: ( SR:8606248238 CR:JAGae14638 ) creat(2) races with unlink(2) allowing a ENOENT failure. Resolution: The function that creates a file, vn_create(), was modified to return a referenced vnode to the open function, vn_open(), thereby eliminating the original race with unlink(2). PHKL_26099: ( SR:8606233922 CR:JAGae03145 ) rename(2) always gets the file system semaphore, even though it is not needed for mp safe file systems, which include HFS and JFS. Resolution: rename(2) will only get the file system semaphore for file systems that are not mp safe. Enhancement: Yes PHKL_30557: Support added for VFS-AdvSysCall and FUSER-Enh. PHKL_29759: This patch enables new functions that will be called by the HP-UX VFS-Enh. PHKL_27800: Enhancements added to disallow mounting another file system within a VxFS 3.5 Cluster mount and to remove VxFS 3.5 entries from VFS switch table, when this product is configured. Additional enhancements were delivered in a patch this one has superseded. Please review the Defect Description text for more information. SR: 8606156998 8606175337 8606178913 8606178928 8606198319 8606212631 8606217733 8606217874 8606222220 8606233922 8606241982 8606247419 8606248238 8606260639 8606263709 8606275342 8606295554 8606306720 8606326634 8606329864 8606330941 8606342243 Patch Files: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP: /usr/conf/lib/libfs.a(kern_dscrp.o) /usr/conf/lib/libfs.a(vfs.o) /usr/conf/lib/libfs.a(vfs_scalls.o) /usr/conf/lib/libfs.a(vfs_vnode.o) /usr/conf/lib/libio.a(aio_subr.o) OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP: /usr/conf/lib/libfs.a(kern_dscrp.o) /usr/conf/lib/libfs.a(vfs.o) /usr/conf/lib/libfs.a(vfs_scalls.o) /usr/conf/lib/libfs.a(vfs_vnode.o) /usr/conf/lib/libio.a(aio_subr.o) what(1) Output: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP: /usr/conf/lib/libfs.a(kern_dscrp.o): kern_dscrp.c $Date: 2004/03/02 22:32:08 $Revision: r 11.11/11 PATCH_11.11 (PHKL_30541) /usr/conf/lib/libfs.a(vfs.o): vfs.c $Date: 2004/03/31 02:45:07 $Revision: r11.11/9 PATCH_11.11 (PHKL_30557) /usr/conf/lib/libfs.a(vfs_scalls.o): vfs_scalls.c $Date: 2003/12/04 01:20:26 $Revision: r 11.11/6 PATCH_11.11 (PHKL_29826) /usr/conf/lib/libfs.a(vfs_vnode.o): vfs_vnode.c $Date: 2002/05/22 10:09:42 $Revision: r1 1.11/4 PATCH_11.11 (PHKL_26239) /usr/conf/lib/libio.a(aio_subr.o): aio_subr.c $Date: 2004/01/19 21:36:06 $Revision: r11 .11/1 PATCH_11.11 (PHKL_30317) OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP: /usr/conf/lib/libfs.a(kern_dscrp.o): kern_dscrp.c $Date: 2004/03/02 22:32:08 $Revision: r 11.11/11 PATCH_11.11 (PHKL_30541) /usr/conf/lib/libfs.a(vfs.o): vfs.c $Date: 2004/03/31 02:45:07 $Revision: r11.11/9 PATCH_11.11 (PHKL_30557) /usr/conf/lib/libfs.a(vfs_scalls.o): vfs_scalls.c $Date: 2003/12/04 01:20:26 $Revision: r 11.11/6 PATCH_11.11 (PHKL_29826) /usr/conf/lib/libfs.a(vfs_vnode.o): vfs_vnode.c $Date: 2002/05/22 10:09:42 $Revision: r1 1.11/4 PATCH_11.11 (PHKL_26239) /usr/conf/lib/libio.a(aio_subr.o): aio_subr.c $Date: 2004/01/19 21:36:06 $Revision: r11 .11/1 PATCH_11.11 (PHKL_30317) cksum(1) Output: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP: 201448106 30744 /usr/conf/lib/libfs.a(kern_dscrp.o) 2364918112 25048 /usr/conf/lib/libfs.a(vfs.o) 1500804575 36732 /usr/conf/lib/libfs.a(vfs_scalls.o) 1915805180 10904 /usr/conf/lib/libfs.a(vfs_vnode.o) 369192802 14160 /usr/conf/lib/libio.a(aio_subr.o) OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP: 3106979486 70248 /usr/conf/lib/libfs.a(kern_dscrp.o) 3714746862 69912 /usr/conf/lib/libfs.a(vfs.o) 3906719746 94256 /usr/conf/lib/libfs.a(vfs_scalls.o) 1390053917 25696 /usr/conf/lib/libfs.a(vfs_vnode.o) 1510911537 32840 /usr/conf/lib/libio.a(aio_subr.o) Patch Conflicts: None Patch Dependencies: s700: 11.11: PHCO_27120 s800: 11.11: PHCO_27120 Hardware Dependencies: None Other Dependencies: PHKL_30542 along with this patch provides the fix for the defects in event port driver reported in JAGaf03150. PHKL_25995: To enable the Fast File Descriptor Allocation enhancement, the following product updates must be installed: PHKL_25993, PHKL_25994, PHKL_25995, PHKL_25996. These product updates may be installed in any order. If any of these product updates are not installed, this patch will have no impact on your system. PHKL_25871: To enable the changes required to fix JAGad48153, the following must also be installed: PHKL_25840, PHKL_25842, PHNE_25644, PHNE_25084. These product updates may be installed in any order. If any of these product updates are not installed, this patch will have no impact on your system. PHKL_26239: This product update, along with PHCO_27910 (libc) and PHKL_26467 (new system call), provide the enabling performance enhancement for accessing information about mounted file systems. The following set of product updates: PHCO_26242, PHCO_27120, PHCO_26331, PHCO_26332, PHCO_26333 and PHCO_26334 each take advantage of this performance enhancement, and thus provide incremental performance improvements. Each of these product updates may be installed independently of each other, and in any order. The three enabling product updates, plus at least one of the others are necessary to see an improvement in performance. None are required if this enhancement is not desired. PHKL_26239: opendir(3C) performance requires product update PHCO_27910 (libc). Each of these product updates may be installed independently of each other, in any order, or not at all if this enhancement is not desired. Supersedes: PHKL_30541 PHKL_30317 PHKL_29826 PHKL_29759 PHKL_28999 PHKL_27800 PHKL_26749 PHKL_26239 PHKL_26099 PHKL_25995 PHKL_25871 PHKL_25611 PHKL_23239 Equivalent Patches: None Patch Package Size: 200 KBytes Installation Instructions: Please review all instructions and the Hewlett-Packard SupportLine User Guide or your Hewlett-Packard support terms and conditions for precautions, scope of license, restrictions, and, limitation of liability and warranties, before installing this patch. ------------------------------------------------------------ 1. Back up your system before installing a patch. 2. Login as root. 3. Copy the patch to the /tmp directory. 4. Move to the /tmp directory and unshar the patch: cd /tmp sh PHKL_30557 5. Run swinstall to install the patch: swinstall -x autoreboot=true -x patch_match_target=true \ -s /tmp/PHKL_30557.depot By default swinstall will archive the original software in /var/adm/sw/save/PHKL_30557. If you do not wish to retain a copy of the original software, include the patch_save_files option in the swinstall command above: -x patch_save_files=false WARNING: If patch_save_files is false when a patch is installed, the patch cannot be deinstalled. Please be careful when using this feature. For future reference, the contents of the PHKL_30557.text file is available in the product readme: swlist -l product -a readme -d @ /tmp/PHKL_30557.depot To put this patch on a magnetic tape and install from the tape drive, use the command: dd if=/tmp/PHKL_30557.depot of=/dev/rmt/0m bs=2k Special Installation Instructions: None