Patch Name: PHKL_30796 Patch Description: s700_800 11.11 vm preemption point, pdc, vhand, MtIOscan Creation Date: 04/04/24 Post Date: 04/06/17 Hardware Platforms - OS Releases: s700: 11.11 s800: 11.11 Products: N/A Filesets: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP Automatic Reboot?: Yes Status: General Release Critical: Yes PHKL_30796: HANG PHKL_28410: HANG PHKL_25361: PANIC The panic is caused by a Service Guard TOC. Category Tags: defect_repair enhancement general_release critical panic halts_system Path Name: /hp-ux_patches/s700_800/11.X/PHKL_30796 Symptoms: PHKL_30796: ( SR:8606352268 CR:JAGaf13073 ) A system may experience performance degradation under memory pressure that may appear as a hang, if a large shared region is being accessed by many processes. Vhand takes a long time in walking the region causing other processes to block for tens of seconds, possibly minutes. PHKL_30681: ( SR:8606271813 CR:JAGae35992 ) This product update is a member of a set needed to enable the optional HP-UX MtIOscan feature. Upon installation, the HP-UX MtIOscan bundle (MtIOscan11i) will install the full set of product updates (including this one) to enable the MtIOscan feature. PHKL_28410: ( SR:8606259131 CR:JAGae23449 ) Applications which use large memory objects may experience intermittent hangs lasting up to several minutes. This occurs only on systems with patch PHKL_23946 (or superseding) installed. PHKL_25361: ( SR:8606194618 CR:JAGad63826 ) On K-Class and slower systems, Service Guard may TOC the system when a process that uses more than a gigabyte of memory exits. The stack trace of the TOC may look similar to the following: pdc_call+0x174 hversion_pdc_call+0xa4 diag2_ioctl+0x170c spec_ioctl+0xac vno_ioctl+0x90 ioctl+0x1e4 syscall+0x394 syscallinit+0x54c or... pdc_call+0x198 pa_generic_psm_pdc_rtc+0x34 psi_rtc+0xb4 checkclock+0x24 invoke_callouts_for_self+0xc0 sw_service+0xb0 mp_ext_interrupt+0x150 ihandler+0x904 splnopreempt+0x8 idle+0x504 swidle+0x20 PHKL_23946: ( SR:8606128017 CR:JAGac78818 ) Time critical functions, such as real-time heartbeat process threads, may timeout. For example, ServiceGuard may unexpectedly failover and other appliations may suddently show reduced response time when a large database is being initialized, or when there is significant system memory pressure. This problem will most likely be seen on systems with one or two processors. Defect Description: PHKL_30796: ( SR:8606352268 CR:JAGaf13073 ) The problem lies in the algorithm used to walk the large region's b-tree. This causes vhand to spend unreasonable time in walking the region. The b-tree walker walks the range of pages specified as its arguments. However, before it can walk the range it must traverse up to the starting point in the tree. Currently the starting point is being reached by searching depth first when it could have been searched breadth first. This causes unnecessary tree traversal that can become significant for large regions. Resolution: Modify the walker algorithm to search the starting point of the walk breadth first instead of depth first. PHKL_30681: ( SR:8606271813 CR:JAGae35992 ) This product update contains minor enhancements required to enable the HP-UX MtIOscan feature. Resolution: Enhancements added to support creation of threads before physical swap initialization in the bootpath. This will allow the "ioscan" to be performed in a parallel fashion. This feature will significantly reduce boot times for systems with large I/O configurations. PHKL_28410: ( SR:8606259131 CR:JAGae23449 ) Patch PHKL_23946 introduced kernel preemption for one long kernel path. The preemption feature does not scale well with large memory objects. vhand is affected when working on those objects and may slow down significantly: vhand is holding a lock on the memory object while doing a lot of unnecessary work. The application is waiting on the lock. This leads to the application being unresponsive for up to several minutes. Resolution: Remove some unnecessary checking for preemption, thus improving the performance of vhand for large objects. This does not take away the effectiveness of the preemption feature. PHKL_25361: ( SR:8606194618 CR:JAGad63826 ) The problem is caused by an exiting process flooding the bus with cache flush instructions. This in turn may cause processes executing firmware code to wait until the exiting process is finished before continuing. If the Service Guard process is scheduled behind the process executing the firmware code, the Service Guard process may not have the chance to run for many seconds, causing it to TOC the system when it does eventually run. Resolution: The exiting process now does its cache flushing less aggressively and allows processes executing firmware code to run at the same time. If such processes are allowed to run, the Service Guard process scheduled after those processes will be able to maintain contact with the other Service Guard nodes in the cluster and it will not TOC the system. PHKL_23946: ( SR:8606128017 CR:JAGac78818 ) Time critical process threads will not be scheduled to run until there is an available processor. There are instances in the kernel where a process thread may not be scheduled to run even when it is the highest priority process thread. This occurs when another process thread is executing on a processor and must complete a time consuming task before releasing the processor. There are several such places in the kernel; this patch addresses one specific kernel path. When the system is under memory pressure, the pager addresses this pressure by freeing a specified number of pages. The pager will run until the quota of free pages is met. Similarly, applications needing to mlock or plock large amounts of memory will cause mlock/plock to execute until the entire request has been satisfied. No other process threads can be scheduled on the processors executing these threads until the work is completed. The amount of work performed along these kernel paths is proportional to the amount of memory involved (being mlocked, plocked, or paged) and thus impacts the delay time of waiting process threads. Resolution: This patch enables kernel preemption along a kernel path shared by the pager, mlock and plock: If a process thread has been executing within the kernel for over a specified period of time, the kernel will preempt this thread and schedule a higher priority thread to run. The preempted thread will be placed back on the run queue to resume where it left off in the kernel once rescheduled. The scheduler dictates the kernel execution time allotted to a single process and the policies governing which process priorities may preempt the current process. This enhancement may not necessarily improve performance or response time for all processes as the pager or mlocking process thread may not be executing along these preemptible paths. Enhancement: No (superseded patches contained enhancements) PHKL_30681: Support added for MtIOscan. PHKL_28410: Enhancements were delivered in a patch this one has superseded. Please review the Defect Description text for more information. SR: 8606128017 8606194618 8606259131 8606271813 8606352268 Patch Files: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP: /usr/conf/lib/libvm.a(vm_page.o) /usr/conf/lib/libvm.a(vm_region.o) /usr/conf/lib/libvm.a(vm_vfd.o) OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP: /usr/conf/lib/libvm.a(vm_page.o) /usr/conf/lib/libvm.a(vm_region.o) /usr/conf/lib/libvm.a(vm_vfd.o) what(1) Output: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP: /usr/conf/lib/libvm.a(vm_page.o): vm_page.c $Date: 2004/04/02 01:21:05 $Revision: r11. 11/1 PATCH_11.11 (PHKL_30681) /usr/conf/lib/libvm.a(vm_region.o): vm_region.c $Date: 2004/04/02 01:21:05 $Revision: r1 1.11/1 PATCH_11.11 (PHKL_30681) /usr/conf/lib/libvm.a(vm_vfd.o): vm_vfd.c $Date: 2004/04/23 23:12:24 $Revision: r11.1 1/7 PATCH_11.11 (PHKL_30796) OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP: /usr/conf/lib/libvm.a(vm_page.o): vm_page.c $Date: 2004/04/02 01:21:05 $Revision: r11. 11/1 PATCH_11.11 (PHKL_30681) /usr/conf/lib/libvm.a(vm_region.o): vm_region.c $Date: 2004/04/02 01:21:05 $Revision: r1 1.11/1 PATCH_11.11 (PHKL_30681) /usr/conf/lib/libvm.a(vm_vfd.o): vm_vfd.c $Date: 2004/04/23 23:12:24 $Revision: r11.1 1/7 PATCH_11.11 (PHKL_30796) cksum(1) Output: OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_32,v=HP: 313668415 12592 /usr/conf/lib/libvm.a(vm_page.o) 3261913756 9364 /usr/conf/lib/libvm.a(vm_region.o) 3279905521 16204 /usr/conf/lib/libvm.a(vm_vfd.o) OS-Core.CORE2-KRN,fr=B.11.11,fa=HP-UX_B.11.11_64,v=HP: 122920565 28760 /usr/conf/lib/libvm.a(vm_page.o) 2970303981 17872 /usr/conf/lib/libvm.a(vm_region.o) 661187542 38856 /usr/conf/lib/libvm.a(vm_vfd.o) Patch Conflicts: None Patch Dependencies: None Hardware Dependencies: None Other Dependencies: None Supersedes: PHKL_30681 PHKL_28410 PHKL_25361 PHKL_23946 Equivalent Patches: None Patch Package Size: 90 KBytes Installation Instructions: Please review all instructions and the Hewlett-Packard SupportLine User Guide or your Hewlett-Packard support terms and conditions for precautions, scope of license, restrictions, and, limitation of liability and warranties, before installing this patch. ------------------------------------------------------------ 1. Back up your system before installing a patch. 2. Login as root. 3. Copy the patch to the /tmp directory. 4. Move to the /tmp directory and unshar the patch: cd /tmp sh PHKL_30796 5. Run swinstall to install the patch: swinstall -x autoreboot=true -x patch_match_target=true \ -s /tmp/PHKL_30796.depot By default swinstall will archive the original software in /var/adm/sw/save/PHKL_30796. If you do not wish to retain a copy of the original software, include the patch_save_files option in the swinstall command above: -x patch_save_files=false WARNING: If patch_save_files is false when a patch is installed, the patch cannot be deinstalled. Please be careful when using this feature. For future reference, the contents of the PHKL_30796.text file is available in the product readme: swlist -l product -a readme -d @ /tmp/PHKL_30796.depot To put this patch on a magnetic tape and install from the tape drive, use the command: dd if=/tmp/PHKL_30796.depot of=/dev/rmt/0m bs=2k Special Installation Instructions: PHKL_23946: There are three patches providing similar kernel preemption enhancements: PHKL_23908, PHKL_23944, PHKL_23946. Each of these may be useful in preventing a time critical process thread time-out. However, these patches, installed either individually or collectively, may not necessarily improve performance or response time for all processes as the process threads causing the delay may not be executing along these preemptible paths. Each of these patches is independent of the others; they may be installed separately or in any combination, and in any order. Each provides kernel preemption for a specific long running kernel path.