Complete report #21613
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffff81325907
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff811b2ac3
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x93
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff811ebd54
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x1a4
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff811b1a37
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0xc37
 | 
    
  
    
      | 
  5
 | 
  
 | 
  vmlinux | 
    0xffffffff811b7dc5
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x305
 | 
    
  
    
      | 
  6
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff81692a84
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x154
 | 
    
  
    
      | 
  7
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff81692e66
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  8
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8169250b
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x1b
 | 
    
  
    
      | 
  9
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8168eff8
       Build id: 3.10.0-514.21.1.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#7 stuck for 21s! [mysqld:1202]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache dm_mirror dm_region_hash dm_log dm_mod iosf_mbi crc32_pclmul ghash_clmulni_intel ext4 aesni_intel lrw gf128mul mbcache jbd2 glue_helper bochs_drm ttm ablk_helper drm_kms_helper cryptd ppdev syscopyarea sysfillrect parport_pc sysimgblt shpchp fb_sys_fops drm parport pcspkr sg virtio_balloon i2c_piix4 i2c_core nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk crct10dif_pclmul ata_piix virtio_pci crct10dif_common virtio_ring crc32c_intel libata serio_raw virtio floppy
CPU: 7 PID: 1202 Comm: mysqld Tainted: G             L ------------   3.10.0-514.21.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff88042c25ce70 ti: ffff88042064c000 task.ti: ffff88042064c000
RIP: 0010:[<ffffffff81325907>]  [<ffffffff81325907>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff88042064fd60  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000200000000 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 00007fa987800000 RDI: ffff88040e600000
RBP: ffff88042064fd98 R08: 0000000000000000 R09: 00000000000000a9
R10: ffff88043ffd7800 R11: 00007fa987800058 R12: ffff88043ffd9008
R13: ffff88042c25ce70 R14: ffff88042c25ce70 R15: ffff88042c25ce70
FS:  00007faaa38a5900(0000) GS:ffff88043fdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa987800058 CR3: 00000004293f7000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811b2ac3 ffff8804212151e0 0000000000000000 00007fa987800000
ffff8804212151e0 ffff88042bd4e400 ffff88042db51368 ffff88042064fdf8
ffffffff811ebd54 000000a92bd4e400 00007fa987800058 00007fa987800058
Call Trace:
[<ffffffff811b2ac3>] ? clear_huge_page+0x93/0x140
[<ffffffff811ebd54>] do_huge_pmd_anonymous_page+0x1a4/0x520
[<ffffffff811b1a37>] handle_mm_fault+0xc37/0xfe0
[<ffffffff811b7dc5>] ? do_mmap_pgoff+0x305/0x3c0
[<ffffffff81692a84>] __do_page_fault+0x154/0x450
[<ffffffff81692e66>] trace_do_page_fault+0x56/0x150
[<ffffffff8169250b>] do_async_page_fault+0x1b/0xd0
[<ffffffff8168eff8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 6b 01 d6 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
       
    
      
        Complete report #1029505
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffffb65ab567
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffffb6402d52
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x72
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffffb6445f41
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x151
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffb640257e
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0xdde
 | 
    
  
    
      | 
  5
 | 
  
 | 
  vmlinux | 
    0xffffffffb6405904
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0xf4
 | 
    
  
    
      | 
  6
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffb69bf7d3
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x213
 | 
    
  
    
      | 
  7
 | 
  
 | 
  vmlinux | 
    0xffffffffb6407fe7
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x327
 | 
    
  
    
      | 
  8
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffb69bfbb6
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  9
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffb69bf112
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  10
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffb69bb7e8
       Build id: 3.10.0-1160.119.1.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#5 stuck for 49s! [php-fpm:4993]
Modules linked in: xt_multiport tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6table_nat nf_nat_ipv6 tun bridge stp llc ip6table_mangle ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_nat nf_nat_ipv4 nf_nat iptable_mangle ipt_REJECT nf_reject_ipv4 binfmt_misc nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack libcrc32c iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul sg ghash_clmulni_intel aesni_intel virtio_balloon lrw gf128mul i2c_piix4 glue_helper ablk_helper pcspkr joydev cryptd ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_scsi bochs_drm failover drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common virtio_pci crc32c_intel virtio_ring serio_raw drm_panel_orientation_quirks virtio floppy fuse
CPU: 5 PID: 4993 Comm: php-fpm Kdump: loaded Tainted: G             L ------------   3.10.0-1160.119.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
task: ffff8cc4cb8e1080 ti: ffff8cb8356f8000 task.ti: ffff8cb8356f8000
RIP: 0010:[<ffffffffb65ab567>]  [<ffffffffb65ab567>] clear_page_c_e+0x7/0x10
RSP: 0018:ffff8cb8356fbd28  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffb69b7920 RCX: 0000000000001000
RDX: 0000000000000000 RSI: ffff8cc4cb8e1080 RDI: ffff8cc3f4c7f000
RBP: ffff8cb8356fbd58 R08: ffff8cb8356f8000 R09: 0000000000000001
R10: 0000000000000003 R11: 0000000000000293 R12: 0000000000000000
R13: ffff8cc6fe68b840 R14: ffff8cc70329acc0 R15: ffff8cc6fe5476c0
FS:  00007f85cefe8880(0000) GS:ffff8cc703280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000c00064d000 CR3: 0000000035652000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffffb6402d52>] ? clear_huge_page+0x72/0x140
[<ffffffffb6445f41>] do_huge_pmd_anonymous_page+0x151/0x510
[<ffffffffb640257e>] handle_mm_fault+0xdde/0x1190
[<ffffffffb6405904>] ? unmap_region+0xf4/0x150
[<ffffffffb69bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffb6407fe7>] ? do_munmap+0x327/0x480
[<ffffffffb69bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffb69bf112>] do_async_page_fault+0x22/0x100
[<ffffffffb69bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 56 40 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 a1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 92 2f 00 00 66 90 eb ee 66 66 66 90 66 66 66 90 66 66 
       
    
      
        Complete report #986888
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffffbc5ab3d7
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffffbc402c92
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x72
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_wp_page
     
    
  
 | 
  vmlinux | 
    0xffffffffbc4472a1
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x801
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffbc40216c
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0xa8c
 | 
    
  
    
      | 
  5
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffbc9bf7d3
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x213
 | 
    
  
    
      | 
  6
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffbc9bfbb6
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  7
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffbc9bf112
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  8
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffbc9bb7e8
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#40 stuck for 28s! [vcontact2:96666]
Modules linked in: tcp_lp nls_utf8 isofs xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ext4 mbcache jbd2 nfit libnvdimm iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr sg joydev virtio_balloon i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif ata_generic crct10dif_generic pata_acpi
bochs_drm virtio_net net_failover drm_kms_helper virtio_scsi failover syscopyarea sysfillrect ahci sysimgblt fb_sys_fops ata_piix ttm libahci crct10dif_pclmul crct10dif_common virtio_pci drm virtio_ring libata serio_raw crc32c_intel drm_panel_orientation_quirks virtio floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 40 PID: 96666 Comm: vcontact2 Tainted: G             L ------------   3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04/01/2014
task: ffffa1bdb220b180 ti: ffffa1e8a4dd4000 task.ti: ffffa1e8a4dd4000
RIP: 0010:[<ffffffffbc5ab3d7>]  [<ffffffffbc5ab3d7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffffa1e8a4dd7cf8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 00007f2fa0e00000 RDI: ffffa1489d200000
RBP: ffffa1e8a4dd7d28 R08: 0000000000000000 R09: fffff6a15b750000
R10: ffffa1f2ffb24660 R11: 0000000000019ca0 R12: fffff6a15b748000
R13: 00ffa1e3d3b244c0 R14: ffffa1bdb220b180 R15: ffffa1e8a4dd7ca0
FS:  00007f3c5bcb0740(0000) GS:ffffa1ef7a000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f2fa0fafbc0 CR3: 00000047626de000 CR4: 0000000000760fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffbc402c92>] ? clear_huge_page+0x72/0x140
[<ffffffffbc4472a1>] do_huge_pmd_wp_page+0x801/0xd40
[<ffffffffbc40216c>] handle_mm_fault+0xa8c/0x1190
[<ffffffffbc9bf7d3>] __do_page_fault+0x213/0x510
[<ffffffffbc9bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffffbc9bf112>] do_async_page_fault+0x22/0x100
[<ffffffffbc9bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 e6 41 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 b1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 a2 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
       
    
      
        Complete report #705203
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffff921ab3d7
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff92002c92
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x72
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff92045e81
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x151
 | 
    
  
    
      | 
  4
 | 
  
 | 
  vmlinux | 
    0xffffffff920089ec
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x20c
 | 
    
  
    
      | 
  5
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff920024be
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0xdde
 | 
    
  
    
      | 
  6
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff925bf7d3
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x213
 | 
    
  
    
      | 
  7
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff925bfbb6
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  8
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff925bf112
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  9
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff925bb7e8
       Build id: 3.10.0-1160.88.1.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#4 stuck for 44s! [ActionThreadPoo:20364]
Modules linked in: ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_ipv4 nf_log_common nf_nat_ftp xt_REDIRECT nf_nat_redirect nf_conntrack_ftp xt_LOG xt_limit xt_multiport xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_conntrack tun iptable_nat nf_conntrack_ipv4 bridge nf_defrag_ipv4 nf_nat_ipv4 stp llc ip6table_mangle ip6table_raw ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_nat ebtable_filter nf_conntrack ebtables libcrc32c devlink iptable_mangle ip6table_filter ip6_tables iptable_raw ipt_REJECT nf_reject_ipv4 iptable_filter ipmi_devintf ipmi_msghandler sunrpc dm_mirror dm_region_hash dm_log dm_mod sb_edac ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd sg joydev pcspkr virtio_balloon parport_pc parport i2c_piix4
ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi virtio_net net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy fuse
CPU: 4 PID: 20364 Comm: ActionThreadPoo Kdump: loaded Tainted: G             L ------------   3.10.0-1160.88.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.1-0-ga5cab58e9a3f-prebuilt.qemu.org 04/01/2014
task: ffff8f40d9911080 ti: ffff8f40de548000 task.ti: ffff8f40de548000
RIP: 0010:[<ffffffff921ab3d7>]  [<ffffffff921ab3d7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8f40de54bd28  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffffffff925b7c58 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8f3ea1304000
RBP: ffff8f40de54bd58 R08: 0000037f9571e7c0 R09: 0000000000000001
R10: 000000000000000c R11: 0000000000000000 R12: ffffffff91ed54f9
R13: ffff8f40de54bc98 R14: 0000000000000000 R15: ffff8f40ffd1acc0
FS:  00007f3696cfa700(0000) GS:ffff8f40ffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f35437ffd90 CR3: 000000079d814000 CR4: 0000000000360fe0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff92002c92>] ? clear_huge_page+0x72/0x140
[<ffffffff92045e81>] do_huge_pmd_anonymous_page+0x151/0x510
[<ffffffff920089ec>] ? mmap_region+0x20c/0x680
[<ffffffff920024be>] handle_mm_fault+0xdde/0x1190
[<ffffffff925bf7d3>] __do_page_fault+0x213/0x510
[<ffffffff925bfbb6>] trace_do_page_fault+0x56/0x150
[<ffffffff925bf112>] do_async_page_fault+0x22/0x100
[<ffffffff925bb7e8>] async_page_fault+0x28/0x30
Code: bc 85 d2 7f 95 89 d0 eb b4 e8 e6 41 cf ff 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab e9 b1 2f 00 00 90 b9 00 10 00 00 31 c0 <f3> aa e9 a2 2f 00 00 66 90 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
       
    
      
        Complete report #566297
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffff95b94c17
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff959f6752
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x72
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff95a387d8
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x178
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff959f5fb8
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0xc28
 | 
    
  
    
      | 
  5
 | 
  
 | 
  vmlinux | 
    0xffffffff95912280
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x90
 | 
    
  
    
      | 
  6
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff95f8f653
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x213
 | 
    
  
    
      | 
  7
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff95f8fa26
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  8
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff95f8efa2
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  9
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff95f8b7a8
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#141 stuck for 21s! [bwa-mem2:139542]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14306(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core qxl snd_hwdep ttm drm_kms_helper snd_seq syscopyarea snd_seq_device sysfillrect snd_pcm sysimgblt fb_sys_fops snd_timer
ppdev iTCO_wdt iTCO_vendor_support crc32_pclmul ghash_clmulni_intel snd drm aesni_intel parport_pc lrw gf128mul glue_helper ablk_helper joydev pcspkr nfsd lpc_ich virtio_balloon soundcore drm_panel_orientation_quirks i2c_i801 virtio_rng cryptd parport nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net net_failover virtio_console virtio_blk failover ahci libahci crct10dif_pclmul virtio_pci crct10dif_common virtio_ring crc32c_intel libata serio_raw virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 141 PID: 139542 Comm: bwa-mem2 Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff9224b7b4a100 ti: ffff922938bbc000 task.ti: ffff922938bbc000
RIP: 0010:[<ffffffff95b94c17>]  [<ffffffff95b94c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff922938bbfd28  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 00007f6916600000 RDI: ffff92af0d600000
RBP: ffff922938bbfd58 R08: 0000000000000002 R09: 000000000001fd08
R10: ffff94177ffd7000 R11: 0000000000000000 R12: ffff9224b7b4a100
R13: ffff9224b7b4a100 R14: ffff9224b7b4a100 R15: ffff9224b7b4a100
FS:  00007f6ceced7700(0000) GS:ffff940fad940000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6916618010 CR3: 00000004e3768000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff959f6752>] ? clear_huge_page+0x72/0x140
[<ffffffff95a387d8>] do_huge_pmd_anonymous_page+0x178/0x4c0
[<ffffffff959f5fb8>] handle_mm_fault+0xc28/0xfb0
[<ffffffff95912280>] ? futex_wake+0x90/0x180
[<ffffffff95f8f653>] __do_page_fault+0x213/0x500
[<ffffffff95f8fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff95f8efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff95f8b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 66 66 66 90 66 66 66 90 66 66 
       
    
      
        Complete report #524151
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffff82394c17
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff821f6752
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x72
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff822387d8
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x178
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff821f5fb8
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0xc28
 | 
    
  
    
      | 
  5
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8278f653
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x213
 | 
    
  
    
      | 
  6
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8278fa26
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  7
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8278efa2
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  8
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8278b7a8
       Build id: 3.10.0-1160.15.2.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#210 stuck for 26s! [python:104303]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14203(E) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_pinned_14108(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm qxl snd_timer ttm drm_kms_helper iTCO_wdt syscopyarea
crc32_pclmul ghash_clmulni_intel aesni_intel lrw sysfillrect gf128mul sysimgblt glue_helper ppdev fb_sys_fops ablk_helper snd iTCO_vendor_support joydev cryptd parport_pc drm virtio_rng pcspkr i2c_i801 lpc_ich soundcore virtio_balloon parport drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net net_failover virtio_console virtio_blk failover crct10dif_pclmul ahci virtio_pci libahci crct10dif_common serio_raw crc32c_intel libata virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 210 PID: 104303 Comm: python Kdump: loaded Tainted: P            EL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8b1f6e3b2100 ti: ffff8b18c03f4000 task.ti: ffff8b18c03f4000
RIP: 0010:[<ffffffff82394c17>]  [<ffffffff82394c17>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff8b18c03f7d28  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 00007f9978200000 RDI: ffff8b9e63000000
RBP: ffff8b18c03f7d58 R08: 0000000000000000 R09: 000000000001fd08
R10: ffff8d037ffd8000 R11: 0000000000000000 R12: ffff8b1f6e3b2100
R13: ffff8b1f6e3b2100 R14: ffff8b1f6e3b2100 R15: ffff8b1f6e3b2100
FS:  00007f9b0b508740(0000) GS:ffff8cfbaea80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9978200000 CR3: 0000000988f9c000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
[<ffffffff821f6752>] ? clear_huge_page+0x72/0x140
[<ffffffff822387d8>] do_huge_pmd_anonymous_page+0x178/0x4c0
[<ffffffff821f5fb8>] handle_mm_fault+0xc28/0xfb0
[<ffffffff8278f653>] __do_page_fault+0x213/0x500
[<ffffffff8278fa26>] trace_do_page_fault+0x56/0x150
[<ffffffff8278efa2>] do_async_page_fault+0x22/0xf0
[<ffffffff8278b7a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 1b 63 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 66 66 66 90 66 66 66 90 66 66 
       
    
      
        Complete report #270836
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffff8132fec7
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff811b34b6
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x76
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff811ed9e9
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x1a9
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff811b258f
       Build id: 3.10.0-693.el7.x86_64
 | 
    0xd0f
 | 
    
  
    
      | 
  5
 | 
  
 | 
  xfs | 
    0xffffffffc02308e7
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x87
 | 
    
  
    
      | 
  6
 | 
  
 | 
  vmlinux | 
    0xffffffff8120019d
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x8d
 | 
    
  
    
      | 
  7
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816affb4
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x154
 | 
    
  
    
      | 
  8
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816b0396
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  9
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816afa2a
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x1a
 | 
    
  
    
      | 
  10
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816ac538
       Build id: 3.10.0-693.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#10 stuck for 36s! [rpm:1248]
Modules linked in: overlay(T) dm_mirror dm_region_hash dm_log dm_mod edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw parport_pc gf128mul glue_helper ablk_helper cryptd sg pcspkr virtio_balloon parport joydev i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic virtio_net virtio_console virtio_blk pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul ata_piix drm floppy serio_raw crct10dif_common crc32c_intel libata virtio_pci virtio_ring virtio i2c_core
CPU: 10 PID: 1248 Comm: rpm Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: RDO OpenStack Compute, BIOS 1.11.0-2.el7 04/01/2014
task: ffff880c0b95af70 ti: ffff880be2fcc000 task.ti: ffff880be2fcc000
RIP: 0010:[<ffffffff8132fec7>]  [<ffffffff8132fec7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff880be2fcfd60  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000200000000 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 00007f0173c00000 RDI: ffff880bdc458000
RBP: ffff880be2fcfd98 R08: 0000000000000001 R09: 000000000001a1d8
R10: ffff880c09f8c5e8 R11: 00000000000000a9 R12: ffff880c3ffd9008
R13: ffff880c0b95af70 R14: ffff880c0b95af70 R15: ffff880c0b95af70
FS:  00007f017f1e5840(0000) GS:ffff880c0f480000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0173c00500 CR3: 0000000bfcf6b000 CR4: 00000000003406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff811b34b6 ffff880be2e45cf0 0000000000000000 00007f0173c00000
ffff880be2e45cf0 ffff880c0a8c4b00 00007f0173c00500 ffff880be2fcfdf8
ffffffff811ed9e9 ffff880be2fcfe08 ffffffff000000a9 ffff880c09f8c5e8
Call Trace:
[<ffffffff811b34b6>] ? clear_huge_page+0x76/0x110
[<ffffffff811ed9e9>] do_huge_pmd_anonymous_page+0x1a9/0x530
[<ffffffff811b258f>] handle_mm_fault+0xd0f/0xfa0
[<ffffffffc02308e7>] ? xfs_file_aio_read+0x87/0x100 [xfs]
[<ffffffff8120019d>] ? do_sync_read+0x8d/0xd0
[<ffffffff816affb4>] __do_page_fault+0x154/0x450
[<ffffffff816b0396>] trace_do_page_fault+0x56/0x150
[<ffffffff816afa2a>] do_async_page_fault+0x1a/0xd0
[<ffffffff816ac538>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 7b 78 d5 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
       
    
      
        Complete report #235271
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c
     
    
  
 | 
  vmlinux | 
    0xffffffffa1158817
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffffa0fc83f6
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x76
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffffa10072b0
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x170
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa0fc7972
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x892
 | 
    
  
    
      | 
  5
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa151a597
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x197
 | 
    
  
    
      | 
  6
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa151a9d6
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  7
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa1519f62
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  8
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa1516798
       Build id: 3.10.0-862.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#5 stuck for 28s! [java:2030]
Modules linked in: ppdev kvm_intel kvm irqbypass sg parport_pc joydev virtio_balloon pcspkr parport i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif sr_mod crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata 8139too virtio_pci i2c_core floppy 8139cp virtio_ring serio_raw virtio mii dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 2030 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
task: ffff8be960b91fa0 ti: ffff8be904ef0000 task.ti: ffff8be904ef0000
RIP: 0010:[<ffffffffa1158817>]  [<ffffffffa1158817>] clear_page_c+0x7/0x10
RSP: 0018:ffff8be904ef3d48  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8be904ef3fd8 RCX: 0000000000000200
RDX: 0000000000000000 RSI: ffff8be960b91fa0 RDI: ffff8be8d97f6000
RBP: ffff8be904ef3d80 R08: ffff8be904ef0000 R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: ffff8be904ef3fd8
R13: ffff8be960b91fa0 R14: ffffffffa151291c R15: ffff8be904ef3d18
FS:  00007f186078e700(0000) GS:ffff8be9ff280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000c2180000 CR3: 0000000ff736e000 CR4: 00000000000006e0
Call Trace:
[<ffffffffa0fc83f6>] ? clear_huge_page+0x76/0x110
[<ffffffffa10072b0>] do_huge_pmd_anonymous_page+0x170/0x4a0
[<ffffffffa0fc7972>] handle_mm_fault+0x892/0x9b0
[<ffffffffa151a597>] __do_page_fault+0x197/0x4f0
[<ffffffffa151a9d6>] trace_do_page_fault+0x56/0x150
[<ffffffffa1519f62>] do_async_page_fault+0x22/0xf0
[<ffffffffa1516798>] async_page_fault+0x28/0x30
Code: 4c 29 ea 39 da 89 d1 7f c4 85 d2 7f 9d 89 d0 eb bc 0f 1f 00 e8 2b 8c d3 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f 
       
    
      
        Complete report #220988
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c_e
     
    
  
 | 
  vmlinux | 
    0xffffffffa4d8ffa7
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffffa4bf48f2
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x72
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffffa4c36140
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x170
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa4bf43c2
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x892
 | 
    
  
    
      | 
  5
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa5188653
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x213
 | 
    
  
    
      | 
  6
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa5188a26
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  7
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa5187fa2
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x22
 | 
    
  
    
      | 
  8
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffffa51847a8
       Build id: 3.10.0-1062.9.1.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#1 stuck for 25s! [rpm:6559]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter tcp_diag inet_diag nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack overlay libcrc32c bridge stp llc sunrpc dm_mirror dm_region_hash dm_log nfit libnvdimm iosf_mbi kvm_intel kvm dm_mod ppdev virtio_balloon irqbypass sg crc32_pclmul joydev ghash_clmulni_intel parport_pc parport aesni_intel pcspkr i2c_piix4 lrw gf128mul glue_helper ablk_helper cryptd ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi virtio_net virtio_blk cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common libata crc32c_intel
serio_raw virtio_pci floppy virtio_ring drm_panel_orientation_quirks virtio
CPU: 1 PID: 6559 Comm: rpm Kdump: loaded Tainted: G             L ------------   3.10.0-1062.9.1.el7.x86_64 #1
Hardware name: OpenStack Foundation OpenStack Nova, BIOS 1.10.1-1ubuntu1~cloud0 04/01/2014
task: ffff940257650000 ti: ffff94022568c000 task.ti: ffff94022568c000
RIP: 0010:[<ffffffffa4d8ffa7>]  [<ffffffffa4d8ffa7>] clear_page_c_e+0x7/0x10
RSP: 0000:ffff94022568fd48  EFLAGS: 00010246
RAX: 0000000000000000 RBX: 0000000000000001 RCX: 0000000000001000
RDX: 0000000000000000 RSI: 00007f7792600000 RDI: ffff940221a00000
RBP: ffff94022568fd78 R08: 0000000000000001 R09: 000000000001fd08
R10: ffff9402fffda000 R11: 0000000000000000 R12: ffff9402fffdb000
R13: 0000014100000000 R14: ffff9402fffdb008 R15: ffff940257650000
FS:  00007f779f1e2840(0000) GS:ffff9402ffc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f77927ffff0 CR3: 000000036a2fc000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
[<ffffffffa4bf48f2>] ? clear_huge_page+0x72/0x140
[<ffffffffa4c36140>] do_huge_pmd_anonymous_page+0x170/0x4d0
[<ffffffffa4bf43c2>] handle_mm_fault+0x892/0x9b0
[<ffffffffa5188653>] __do_page_fault+0x213/0x500
[<ffffffffa5188a26>] trace_do_page_fault+0x56/0x150
[<ffffffffa5187fa2>] do_async_page_fault+0x22/0xf0
[<ffffffffa51847a8>] async_page_fault+0x28/0x30
Code: bc 0f 1f 00 e8 2b b7 d0 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 <f3> aa c3 66 0f 1f 44 00 00 eb ee 0f 1f 84 00 00 00 00 00 0f 1f 
       
    
      
        Complete report #18416
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page
     
    
  
 | 
  vmlinux | 
    0xffffffff8132ffb2
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x12
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff811b3426
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x76
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff811ed969
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x1a9
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff811b24ff
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0xd0f
 | 
    
  
    
      | 
  5
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816b0074
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x154
 | 
    
  
    
      | 
  6
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816b0456
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  7
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816afaea
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x1a
 | 
    
  
    
      | 
  8
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff816ac5f8
       Build id: 3.10.0-693.5.2.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [php:7779]
Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ppdev sg virtio_balloon i2c_piix4 shpchp parport_pc joydev parport pcspkr nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix
drm libata i2c_core serio_raw virtio_pci virtio_ring virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 7779 Comm: php Not tainted 3.10.0-693.5.2.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.10.2-0-g5f4c7b1-prebuilt.qemu-project.org 04/01/2014
task: ffff88008f2f2f70 ti: ffff88007dfd8000 task.ti: ffff88007dfd8000
RIP: 0010:[<ffffffff8132ffb2>]  [<ffffffff8132ffb2>] clear_page+0x12/0x40
RSP: 0000:ffff88007dfdbd60  EFLAGS: 00010216
RAX: 0000000000000000 RBX: 0000000200000000 RCX: 000000000000003f
RDX: 0000000000000000 RSI: 00007f2318200000 RDI: ffff880019800000
RBP: ffff88007dfdbd98 R08: 0000000000000000 R09: 000000000001a170
R10: ffff88007ddb5ca8 R11: 00000000000000a9 R12: ffff88013ffd8008
R13: ffff88008f2f2f70 R14: ffff88008f2f2f70 R15: ffff88008f2f2f70
FS:  00007f235e480840(0000) GS:ffff88013fc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f2318200000 CR3: 000000007dcc8000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffffffff811b3426 ffff8800a03bf608 0000000000000000 00007f2318200000
ffff8800a03bf608 ffff88009bb6e400 00007f2318200000 ffff88007dfdbdf8
ffffffff811ed969 ffff8800000000a9 ffff8800000000a9 ffff88007ddb5ca8
Call Trace:
[<ffffffff811b3426>] ? clear_huge_page+0x76/0x110
[<ffffffff811ed969>] do_huge_pmd_anonymous_page+0x1a9/0x530
[<ffffffff811b24ff>] handle_mm_fault+0xd0f/0xfa0
[<ffffffff816b0074>] __do_page_fault+0x154/0x450
[<ffffffff816b0456>] trace_do_page_fault+0x56/0x150
[<ffffffff816afaea>] do_async_page_fault+0x1a/0xd0
[<ffffffff816ac5f8>] async_page_fault+0x28/0x30
Code: f3 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f 1f 44 00 00 31 c0 b9 40 00 00 00 66 0f 1f 84 00 00 00 00 00 ff c9 <48> 89 07 48 89 47 08 48 89 47 10 48 89 47 18 48 89 47 20 48 89 
       
    
      
        Complete report #2086
        
  
  
    | Frame # | 
    Function | 
    
      Binary | 
    
    Source or offset | 
    Line | 
  
  
  
    
      | 
  1
 | 
   
      clear_page_c
     
    
  
 | 
  vmlinux | 
    0xffffffff813250c7
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x7
 | 
    
  
    
      | 
  2
 | 
  
 | 
  vmlinux | 
    0xffffffff811b23d3
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x93
 | 
    
  
    
      | 
  3
 | 
   
      do_huge_pmd_anonymous_page
     
    
  
 | 
  vmlinux | 
    0xffffffff811eb664
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x1a4
 | 
    
  
    
      | 
  4
 | 
   
      handle_mm_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff811b1347
       Build id: 3.10.0-514.el7.x86_64
 | 
    0xc37
 | 
    
  
    
      | 
  5
 | 
   
      __do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff81691a94
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x154
 | 
    
  
    
      | 
  6
 | 
  
 | 
  vmlinux | 
    0xffffffff810295ec
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x15c
 | 
    
  
    
      | 
  7
 | 
   
      trace_do_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff81691e76
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x56
 | 
    
  
    
      | 
  8
 | 
   
      do_async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8169151b
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x1b
 | 
    
  
    
      | 
  9
 | 
   
      async_page_fault
     
    
  
 | 
  vmlinux | 
    0xffffffff8168e0b8
       Build id: 3.10.0-514.el7.x86_64
 | 
    0x28
 | 
    
  
  NMI watchdog: BUG: soft lockup - CPU#1 stuck for 26s! [rpm:18253]
Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic iosf_mbi crc32_pclmul snd_hda_intel ghash_clmulni_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device cryptd snd_pcm snd_timer snd pcspkr parport_pc virtio_balloon parport sg i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect
crc32c_intel sysimgblt ata_piix fb_sys_fops serio_raw ttm drm virtio_pci virtio_ring virtio libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 18253 Comm: rpm Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff880005639f60 ti: ffff8801c375c000 task.ti: ffff8801c375c000
RIP: 0010:[<ffffffff813250c7>]  [<ffffffff813250c7>] clear_page_c+0x7/0x10
RSP: 0000:ffff8801c375fd60  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8801c375c000 RCX: 0000000000000200
RDX: 0000000000000000 RSI: 0000000000e00000 RDI: ffff880120295000
RBP: ffff8801c375fd98 R08: fffffffffffffff3 R09: 00000000000000a9
R10: ffff880239fd5000 R11: 0000000000e00000 R12: 000000000480a540
R13: ffff880000000000 R14: ffff8801c375ffd8 R15: 0000000004810000
FS:  00007f69eeaa8840(0000) GS:ffff880239c80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000008ec90001 CR3: 0000000224bba000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffffffff811b23d3 ffff88022e5cb038 0000000000000000 0000000000e00000
ffff88022e5cb038 ffff8801d7ac8000 ffff8801d472d000 ffff8801c375fdf8
ffffffff811eb664 000000a9d7ac8000 0000000000e00000 0000000000e00000
Call Trace:
[<ffffffff811b23d3>] ? clear_huge_page+0x93/0x140
[<ffffffff811eb664>] do_huge_pmd_anonymous_page+0x1a4/0x520
[<ffffffff811b1347>] handle_mm_fault+0xc37/0xfe0
[<ffffffff81691a94>] __do_page_fault+0x154/0x450
[<ffffffff810295ec>] ? __switch_to+0x15c/0x4c0
[<ffffffff81691e76>] trace_do_page_fault+0x56/0x150
[<ffffffff8169151b>] do_async_page_fault+0x1b/0xd0
[<ffffffff8168e0b8>] async_page_fault+0x28/0x30
Code: 4c 29 ea 39 da 89 d1 7f c4 85 d2 7f 9d 89 d0 eb bc 0f 1f 00 e8 0b 05 d6 ff 90 90 90 90 90 90 90 90 90 90 90 b9 00 02 00 00 31 c0 <f3> 48 ab c3 0f 1f 44 00 00 b9 00 10 00 00 31 c0 f3 aa c3 66 0f