Info

Permalink
Function
generic_exec_single
First occurrence
2020-09-22
Last occurrence
2024-09-07
State
NEW
Components
kernel
Quality
-29

Statistics

Operating system Count
CentOS 7 3,547
Architecture Count
x86_64 3,547
Related packages Count
kernel 3,547
 0:3.10.0-957.el7 1,019
 0:3.10.0-1062.1.1.el7 686
 0:3.10.0-1160.el7 272
 0:3.10.0-862.el7 271
 0:3.10.0-693.5.2.el7 130
 0:3.10.0-1062.el7 120
 0:3.10.0-1062.9.1.el7 110
 0:3.10.0-693.el7 88
 0:3.10.0-1160.53.1.el7 85
 0:3.10.0-1127.13.1.el7 52
 0:3.10.0-957.12.1.el7 44
 0:3.10.0-862.11.6.el7 44
 0:3.10.0-1160.62.1.el7 36
 0:3.10.0-957.27.2.el7 34
 0:3.10.0-957.1.3.el7 28
 0:3.10.0-1127.19.1.el7 27
 0:3.10.0-1160.45.1.el7 27
 0:3.10.0-1160.11.1.el7 22
 0:3.10.0-1160.76.1.el7 21
 0:3.10.0-1160.15.2.el7 21
 0:3.10.0-1062.4.1.el7 19
 0:3.10.0-1160.6.1.el7 19
 0:3.10.0-1127.8.2.el7 18
 0:3.10.0-1160.25.1.el7 18
 0:3.10.0-1160.95.1.el7 18
 0:3.10.0-1160.102.1.el7 17
 0:3.10.0-1160.59.1.el7 17
 0:3.10.0-957.12.2.el7 17
 0:3.10.0-957.21.3.el7 17
 0:3.10.0-1160.49.1.el7 16
 0:3.10.0-1160.42.2.el7 14
 0:3.10.0-957.10.1.el7 14
 0:3.10.0-862.14.4.el7 14
 0:3.10.0-1160.71.1.el7 13
 0:3.10.0-1062.12.1.el7 13
 0:3.10.0-1062.1.2.el7 13
 0:3.10.0-693.21.1.el7 10
 0:3.10.0-1160.83.1.el7 9
 0:3.10.0-1160.41.1.el7 9
 0:3.10.0-957.5.1.el7 8
 0:3.10.0-693.17.1.el7 7
 0:3.10.0-1062.18.1.el7 7
 0:3.10.0-1127.el7 7
 0:3.10.0-1160.119.1.el7 7
 0:3.10.0-1160.90.1.el7 7
 0:3.10.0-693.11.6.el7 6
 0:3.10.0-1127.18.2.el7 6
 0:3.10.0-1160.21.1.el7 6
 0:3.10.0-862.3.2.el7 6
 0:3.10.0-1160.118.1.el7 6
 0:3.10.0-1160.88.1.el7 5
 0:3.10.0-1160.31.1.el7 5
 0:3.10.0-1160.36.2.el7 5
 0:3.10.0-514.26.2.el7 4
 0:3.10.0-862.3.3.el7 4
 0:3.10.0-1160.24.1.el7 3
 0:3.10.0-1160.66.1.el7 3
 0:3.10.0-957.21.2.el7 3
 0:3.10.0-1160.80.1.el7 2
 0:3.10.0-862.9.1.el7 2
 0:3.10.0-1160.114.2.el7 2
 0:3.10.0-1160.92.1.el7 2
 0:3.10.0-1127.10.1.el7 2
 0:3.10.0-1160.99.1.el7 2
 0:3.10.0-1160.81.1.el7 2
 0:3.10.0-693.11.1.el7 1
 0:3.10.0-693.2.2.el7 1
 0:3.10.0-862.6.3.el7 1
 0:3.10.0-1160.105.1.el7 1
 0:3.10.0-1160.108.1.el7 1
 0:3.10.0-1062.4.3.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1197544
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f932a
Build id: 3.10.0-693.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f942f
Build id: 3.10.0-693.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813226f5
Build id: 3.10.0-693.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99c3
Build id: 3.10.0-693.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b5444
Build id: 3.10.0-693.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff811b5a01
Build id: 3.10.0-693.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff811b7a02
Build id: 3.10.0-693.el7.x86_64
0x292
15
vm_munmap
vmlinux 0xffffffff811b7c45
Build id: 3.10.0-693.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff811b8d52
Build id: 3.10.0-693.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! [ActorSystem:3969]
Modules linked in: qax_tq_base(OE) binfmt_misc xt_set ip_set_hash_ip cfg80211 rfkill uinput gsch(OE) redirfs(OE) nf_conntrack_netlink xt_addrtype br_netfilter dsa_filter(POE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ilocker(OE) overlay(T) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cirrus ttm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops snd_hda_codec_generic snd_hda_intel snd_hda_codec
drm virtio_net snd_hda_core snd_hwdep snd_seq snd_seq_device virtio_console pcspkr i2c_piix4 snd_pcm joydev snd_timer virtio_balloon virtio_scsi sg shpchp i2c_core snd soundcore virtio_pci virtio_ring virtio nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi ata_piix libata serio_raw floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: qax_tq_base]
CPU: 7 PID: 3969 Comm: ActorSystem Tainted: P           OEL ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: inspur Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-2.module_kos5+70+5e037788 04/01/2014
task: ffff8803bc3cdee0 ti: ffff8803bc3e4000 task.ti: ffff8803bc3e4000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8803bc3e7c60  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff8803bc3e7c30 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff8803bc3e7cb0 R08: ffffffff816ce420 R09: ffffffff811bd768
R10: ffff88043fdd9bc0 R11: ffffea00105b93c0 R12: 0000000000000000
R13: 0000000000001000 R14: 0000000000000050 R15: 0000000000000001
FS:  00007f05bffff700(0000) GS:ffff88043fdc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f05fcdcc000 CR3: 000000032b7d3000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff8803bc3e7d38
0000000000000003 000000008964d66e ffff8803bc3e7ca8 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff8803bc3e7ce0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
[<ffffffff811b5a01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7a02>] do_munmap+0x292/0x470
[<ffffffff811b7c45>] vm_munmap+0x65/0xb0
[<ffffffff811b8d52>] SyS_munmap+0x22/0x30
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #1196962
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f932a
Build id: 3.10.0-693.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f942f
Build id: 3.10.0-693.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813226f5
Build id: 3.10.0-693.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99c3
Build id: 3.10.0-693.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b5444
Build id: 3.10.0-693.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff811b5a01
Build id: 3.10.0-693.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff811b7a02
Build id: 3.10.0-693.el7.x86_64
0x292
15
vm_munmap
vmlinux 0xffffffff811b7c45
Build id: 3.10.0-693.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff811b8d52
Build id: 3.10.0-693.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 196s! [qaxsafed:7281]
Modules linked in: qax_tq_base(OE) udp_diag binfmt_misc tcp_diag inet_diag xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ilocker(OE) nfsv3 nfs fscache gsch(OE) redirfs(OE) tun dsa_filter(POE) ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sb_edac edac_core coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev
pcspkr joydev vmw_balloon sg i2c_piix4 parport_pc parport shpchp vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul mptspi crct10dif_common crc32c_intel scsi_transport_spi libata mptscsih serio_raw e1000 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: qax_tq_base]
CPU: 4 PID: 7281 Comm: qaxsafed Tainted: P        W  OEL ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8800a942dee0 ti: ffff88000a4f4000 task.ti: ffff88000a4f4000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88000a4f7c60  EFLAGS: 00000202
RAX: 00000000000008fd RBX: ffff88024bb3d780 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff88000a4f7cb0 R08: 0000000000000001 R09: ffffffff811bd768
R10: ffff88043fd19bc0 R11: ffffea0008ef9580 R12: ffff88000a4f7c28
R13: ffff8800a942dee0 R14: ffff8800a942dee0 R15: ffff8800a942dee0
FS:  00007f13d1767700(0000) GS:ffff88043fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f13e4a8c000 CR3: 0000000222e75000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff88000a4f7d38
0000000000000002 0000000001d539fd 0000000001d539fd 0000000000000006
ffffffff8106e4a0 ffffffff8106e4a0 ffff88000a4f7ce0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
[<ffffffff811b5a01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7a02>] do_munmap+0x292/0x470
[<ffffffff811b7c45>] vm_munmap+0x65/0xb0
[<ffffffff811b8d52>] SyS_munmap+0x22/0x30
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #1036179
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa851d9c2
Build id: 3.10.0-1160.81.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffa8481ab0
Build id: 3.10.0-1160.81.1.el7.x86_64
0x130
3
? leave_mm
vmlinux 0xffffffffa8481ab0
Build id: 3.10.0-1160.81.1.el7.x86_64
0x130
4
? leave_mm
vmlinux 0xffffffffa8481ab0
Build id: 3.10.0-1160.81.1.el7.x86_64
0x130
5
smp_call_function_single
vmlinux 0xffffffffa851dadf
Build id: 3.10.0-1160.81.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa879bf55
Build id: 3.10.0-1160.81.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa851e09b
Build id: 3.10.0-1160.81.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa8481f48
Build id: 3.10.0-1160.81.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa8481fbb
Build id: 3.10.0-1160.81.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa85f9f0e
Build id: 3.10.0-1160.81.1.el7.x86_64
0xae
11
unmap_page_range
vmlinux 0xffffffffa85fd1b9
Build id: 3.10.0-1160.81.1.el7.x86_64
0x979
12
unmap_single_vma
vmlinux 0xffffffffa85fd541
Build id: 3.10.0-1160.81.1.el7.x86_64
0x81
13
unmap_vmas
vmlinux 0xffffffffa85fef99
Build id: 3.10.0-1160.81.1.el7.x86_64
0x49
14
unmap_region
vmlinux 0xffffffffa86054ee
Build id: 3.10.0-1160.81.1.el7.x86_64
0xbe
15
? __vma_rb_erase
vmlinux 0xffffffffa8605b31
Build id: 3.10.0-1160.81.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffa8607b75
Build id: 3.10.0-1160.81.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffa8607db5
Build id: 3.10.0-1160.81.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffa86090e2
Build id: 3.10.0-1160.81.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffa8bc539a
Build id: 3.10.0-1160.81.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#8 stuck for 23s! [zmlogger:119553]
Modules linked in: vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event vfat fat snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel sg aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev ppdev vmw_balloon tpm_crb parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci ata_piix libahci nfit crct10dif_pclmul mptsas libata crct10dif_common crc32c_intel scsi_transport_sas e1000 libnvdimm mptscsih mptbase serio_raw drm_panel_orientation_quirks fuse
CPU: 8 PID: 119553 Comm: zmlogger Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.81.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware7,1/440BX Desktop Reference Platform, BIOS VMW71.00V.14410784.B64.1908150010 08/15/2019
task: ffff96687babb180 ti: ffff9667ce938000 task.ti: ffff9667ce938000
RIP: 0010:[<ffffffffa851d9c2>]  [<ffffffffa851d9c2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9667ce93bb10  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9667ce93bae0 RCX: 000000000000000b
RDX: ffffffffa8c151d0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9667ce93bb58 R08: ffffffffa8c151c8 R09: ffff9668ddfbdea0
R10: ffff9667cd1af0e0 R11: 0000000000000246 R12: 000000000000000c
R13: 0000000000001000 R14: ffff966bb8c18000 R15: ffffffffc035462b
FS:  00007f4191279740(0000) GS:ffff966bed800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4191207560 CR3: 000000008bd06000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa8481ab0>] ? leave_mm+0x130/0x130
[<ffffffffa8481ab0>] ? leave_mm+0x130/0x130
[<ffffffffa8481ab0>] ? leave_mm+0x130/0x130
[<ffffffffa851dadf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa879bf55>] ? cpumask_next_and+0x35/0x50
[<ffffffffa851e09b>] smp_call_function_many+0x22b/0x270
[<ffffffffa8481f48>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa8481fbb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa85f9f0e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa85fd1b9>] unmap_page_range+0x979/0xc80
[<ffffffffa85fd541>] unmap_single_vma+0x81/0x100
[<ffffffffa85fef99>] unmap_vmas+0x49/0x90
[<ffffffffa86054ee>] unmap_region+0xbe/0x150
[<ffffffffa8605b31>] ? __vma_rb_erase+0x121/0x230
[<ffffffffa8607b75>] do_munmap+0x2a5/0x480
[<ffffffffa8607db5>] vm_munmap+0x65/0xb0
[<ffffffffa86090e2>] SyS_munmap+0x22/0x30
[<ffffffffa8bc539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 15 a9 48 89 df e8 d7 62 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #931174
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa2916ae2
Build id: 3.10.0-1160.76.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffa287ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffa287ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffffa287ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffffa2916bff
Build id: 3.10.0-1160.76.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa2b86845
Build id: 3.10.0-1160.76.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa29171ab
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa287e258
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa287e2cb
Build id: 3.10.0-1160.76.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa29ee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa29f016f
Build id: 3.10.0-1160.76.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa29f0293
Build id: 3.10.0-1160.76.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffa29f9a84
Build id: 3.10.0-1160.76.1.el7.x86_64
0xf4
14
? xfs_iunlock
xfs 0xffffffffc03153d4
Build id: 3.10.0-1160.76.1.el7.x86_64
0xe4
15
? __vma_rb_erase
vmlinux 0xffffffffa29fa041
Build id: 3.10.0-1160.76.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffa29fc025
Build id: 3.10.0-1160.76.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffa29fc265
Build id: 3.10.0-1160.76.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffa29fd562
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffa2f99f92
Build id: 3.10.0-1160.76.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [I2Availability:2293]
Modules linked in: nbd(E) ch osst st mhvtl(OE) tcp_lp nls_utf8 isofs target_core_mod xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc crc32_pclmul ghash_clmulni_intel ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg parport_pc parport vmw_vmci i2c_piix4 binfmt_misc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix
nfit libata serio_raw libnvdimm mptspi scsi_transport_spi e1000 mptscsih drm_panel_orientation_quirks mptbase dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: sfs]
CPU: 3 PID: 2293 Comm: I2Availability Tainted: G        W  OEL ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff91ef8fd72100 ti: ffff91ef78710000 task.ti: ffff91ef78710000
RIP: 0010:[<ffffffffa2916ae2>]  [<ffffffffa2916ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff91ef78713c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff91ef78713bf0 RCX: 0000000000000005
RDX: ffffffffa30141d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff91ef78713c70 R08: ffffffffa30141c8 R09: ffffffffa2a03064
R10: ffff91ef93edf0a0 R11: ffffdf7300ef9140 R12: ffffffffa2f90778
R13: ffff91ef78713c70 R14: 0000000000001000 R15: 00000000000001dd
FS:  00007f1382656700(0000) GS:ffff91ef93ec0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f139506d000 CR3: 0000000039588000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa287ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2916bff>] smp_call_function_single+0x5f/0xa0
[<ffffffffa2b86845>] ? cpumask_next_and+0x35/0x50
[<ffffffffa29171ab>] smp_call_function_many+0x22b/0x270
[<ffffffffa287e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa287e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa29ee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa29f016f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa29f0293>] tlb_finish_mmu+0x23/0x30
[<ffffffffa29f9a84>] unmap_region+0xf4/0x140
[<ffffffffc03153d4>] ? xfs_iunlock+0xe4/0x120 [xfs]
[<ffffffffa29fa041>] ? __vma_rb_erase+0x121/0x220
[<ffffffffa29fc025>] do_munmap+0x2a5/0x480
[<ffffffffa29fc265>] vm_munmap+0x65/0xb0
[<ffffffffa29fd562>] SyS_munmap+0x22/0x30
[<ffffffffa2f99f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 55 a3 48 89 df e8 d7 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #916508
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb0116562
Build id: 3.10.0-1160.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffb007e6a0
Build id: 3.10.0-1160.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffb007e6a0
Build id: 3.10.0-1160.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffffb007e6a0
Build id: 3.10.0-1160.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffffb011667f
Build id: 3.10.0-1160.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb0385de5
Build id: 3.10.0-1160.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb0116c2b
Build id: 3.10.0-1160.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb007e868
Build id: 3.10.0-1160.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb007e8db
Build id: 3.10.0-1160.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb01edd06
Build id: 3.10.0-1160.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb01ef7cf
Build id: 3.10.0-1160.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb01ef8f3
Build id: 3.10.0-1160.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffb01f90e4
Build id: 3.10.0-1160.el7.x86_64
0xf4
14
? do_sync_write
vmlinux 0xffffffffb024d060
Build id: 3.10.0-1160.el7.x86_64
0x90
15
? __vma_rb_erase
vmlinux 0xffffffffb01f96a1
Build id: 3.10.0-1160.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffb01fb6d5
Build id: 3.10.0-1160.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffb01fb915
Build id: 3.10.0-1160.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffb01fcbb2
Build id: 3.10.0-1160.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffb0793f92
Build id: 3.10.0-1160.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 67s! [redis-server:2543]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock bnep sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm btusb btrtl btbcm snd_timer btintel snd bluetooth soundcore sg iosf_mbi crc32_pclmul rfkill ghash_clmulni_intel aesni_intel lrw gf128mul
glue_helper ablk_helper ppdev cryptd joydev vmw_balloon pcspkr parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix crct10dif_pclmul drm crct10dif_common mptspi libata nfit crc32c_intel scsi_transport_spi libnvdimm mptscsih e1000 serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 2543 Comm: redis-server Kdump: loaded Tainted: G             L ------------   3.10.0-1160.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9e1d59213180 ti: ffff9e1d580ac000 task.ti: ffff9e1d580ac000
RIP: 0010:[<ffffffffb0116562>]  [<ffffffffb0116562>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9e1d580afc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9e1d580afbf0 RCX: 0000000000000008
RDX: ffffffffb0814910 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9e1d580afc70 R08: ffffffffb0814908 R09: ffffffffb02026ac
R10: ffff9e1d6d75f0a0 R11: fffff1238aa0d3c0 R12: ffff9e1d580afc80
R13: 0000000000000000 R14: ffff9e1d580afbf0 R15: 0000000000000db3
FS:  00007fd7824f9780(0000) GS:ffff9e1d6d740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd782510000 CR3: 0000000419340000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb007e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb007e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb007e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb011667f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb0385de5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb0116c2b>] smp_call_function_many+0x22b/0x270
[<ffffffffb007e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb007e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb01edd06>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb01ef7cf>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb01ef8f3>] tlb_finish_mmu+0x23/0x30
[<ffffffffb01f90e4>] unmap_region+0xf4/0x140
[<ffffffffb024d060>] ? do_sync_write+0x90/0xe0
[<ffffffffb01f96a1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb01fb6d5>] do_munmap+0x2a5/0x480
[<ffffffffb01fb915>] vm_munmap+0x65/0xb0
[<ffffffffb01fcbb2>] SyS_munmap+0x22/0x30
[<ffffffffb0793f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 13 d5 b0 48 89 df e8 f7 69 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #896616
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8cb1db82
Build id: 3.10.0-1160.99.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8ca81b10
Build id: 3.10.0-1160.99.1.el7.x86_64
0x130
3
? leave_mm
vmlinux 0xffffffff8ca81b10
Build id: 3.10.0-1160.99.1.el7.x86_64
0x130
4
? leave_mm
vmlinux 0xffffffff8ca81b10
Build id: 3.10.0-1160.99.1.el7.x86_64
0x130
5
smp_call_function_single
vmlinux 0xffffffff8cb1dc9f
Build id: 3.10.0-1160.99.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8cd9c5d5
Build id: 3.10.0-1160.99.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8cb1e25b
Build id: 3.10.0-1160.99.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8ca81fa8
Build id: 3.10.0-1160.99.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8ca8201b
Build id: 3.10.0-1160.99.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8cbfa07e
Build id: 3.10.0-1160.99.1.el7.x86_64
0xae
11
unmap_page_range
vmlinux 0xffffffff8cbfd369
Build id: 3.10.0-1160.99.1.el7.x86_64
0x979
12
unmap_single_vma
vmlinux 0xffffffff8cbfd6f1
Build id: 3.10.0-1160.99.1.el7.x86_64
0x81
13
unmap_vmas
vmlinux 0xffffffff8cbff149
Build id: 3.10.0-1160.99.1.el7.x86_64
0x49
14
unmap_region
vmlinux 0xffffffff8cc0587e
Build id: 3.10.0-1160.99.1.el7.x86_64
0xbe
15
? ftrace_regs_call
vmlinux 0xffffffff8d1c91c4
Build id: 3.10.0-1160.99.1.el7.x86_64
0x5
16
? __vma_rb_erase
vmlinux 0xffffffff8cc05ec1
Build id: 3.10.0-1160.99.1.el7.x86_64
0x121
17
do_munmap
vmlinux 0xffffffff8cc07f15
Build id: 3.10.0-1160.99.1.el7.x86_64
0x2a5
18
vm_munmap
vmlinux 0xffffffff8cc08155
Build id: 3.10.0-1160.99.1.el7.x86_64
0x65
19
SyS_munmap
vmlinux 0xffffffff8cc09482
Build id: 3.10.0-1160.99.1.el7.x86_64
0x22
20
tracesys
vmlinux 0xffffffff8d1c562e
Build id: 3.10.0-1160.99.1.el7.x86_64
0xa6
NMI watchdog: BUG: soft lockup - CPU#24 stuck for 23s! [liagent:9761]
Modules linked in: tcp_diag udp_diag inet_diag ip6table_filter ip6_tables xt_owner iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptsas drm ata_piix scsi_transport_sas nfit mptscsih crct10dif_pclmul libata crct10dif_common crc32c_intel libnvdimm mptbase serio_raw vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 24 PID: 9761 Comm: liagent Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.99.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9ddf196f4200 ti: ffff9ddfdd05c000 task.ti: ffff9ddfdd05c000
RIP: 0010:[<ffffffff8cb1db82>]  [<ffffffff8cb1db82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9ddfdd05fb10  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff9ddfdd05fae0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 0000000000000020 RDI: 0000000000000286
RBP: ffff9ddfdd05fb58 R08: ffffffff8d217f40 R09: 00003ffffffff000
R10: ffff9de3aee6ad80 R11: 0000000000000202 R12: ffff9ddfdd05ffd8
R13: 0000000000000000 R14: 00000000945d4028 R15: ffff9ddfdd05ffd8
FS:  00007fd0c3fff700(0000) GS:ffff9de3bf800000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd0d3698000 CR3: 00000008213b2000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff8ca81b10>] ? leave_mm+0x130/0x130
[<ffffffff8ca81b10>] ? leave_mm+0x130/0x130
[<ffffffff8ca81b10>] ? leave_mm+0x130/0x130
[<ffffffff8cb1dc9f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8cd9c5d5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8cb1e25b>] smp_call_function_many+0x22b/0x270
[<ffffffff8ca81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ca8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8cbfa07e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff8cbfd369>] unmap_page_range+0x979/0xc80
[<ffffffff8cbfd6f1>] unmap_single_vma+0x81/0x100
[<ffffffff8cbff149>] unmap_vmas+0x49/0x90
[<ffffffff8cc0587e>] unmap_region+0xbe/0x150
[<ffffffff8d1c91c4>] ? ftrace_regs_call+0x5/0x81
[<ffffffff8cc05ec1>] ? __vma_rb_erase+0x121/0x230
[<ffffffff8cc07f15>] do_munmap+0x2a5/0x480
[<ffffffff8cc08155>] vm_munmap+0x65/0xb0
[<ffffffff8cc09482>] SyS_munmap+0x22/0x30
[<ffffffff8d1c562e>] tracesys+0xa6/0xcc
Code: 89 de 48 03 14 c5 40 19 75 8d 48 89 df e8 97 67 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #860205
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9e716532
Build id: 3.10.0-1160.31.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9e67ddd0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff9e67ddd0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff9e67ddd0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff9e71664f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9e9864f5
Build id: 3.10.0-1160.31.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9e716bfb
Build id: 3.10.0-1160.31.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9e67e258
Build id: 3.10.0-1160.31.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9e67e2cb
Build id: 3.10.0-1160.31.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9e7ee176
Build id: 3.10.0-1160.31.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9e7efc3f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9e7efd63
Build id: 3.10.0-1160.31.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff9e7f9554
Build id: 3.10.0-1160.31.1.el7.x86_64
0xf4
14
? xfs_iunlock
xfs 0xffffffffc03f03c4
Build id: 3.10.0-1160.31.1.el7.x86_64
0xe4
15
? __vma_rb_erase
vmlinux 0xffffffff9e7f9b11
Build id: 3.10.0-1160.31.1.el7.x86_64
0x121
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [commons-pool-Ev:2480]
Modules linked in: xt_CHECKSUM tun devlink xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_mark ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc squashfs loop sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper vmw_balloon cryptd joydev sg pcspkr vmw_vmci parport_pc i2c_piix4 parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic
pata_acpi vmwgfx sd_mod drm_kms_helper crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common syscopyarea sysfillrect ahci sysimgblt fb_sys_fops crc32c_intel ttm serio_raw libahci ata_piix nfit drm vmxnet3 vmw_pvscsi libnvdimm libata drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 2480 Comm: commons-pool-Ev Kdump: loaded Tainted: G             L ------------   3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff9ff5fd9f6300 ti: ffff9ff5fd9e4000 task.ti: ffff9ff5fd9e4000
RIP: 0010:[<ffffffff9e716532>]  [<ffffffff9e716532>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9ff5fd9e7c20  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff9ff5fd9e7bf0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff9ff5fd9e7c70 R08: ffffffff9ee137c0 R09: ffffffff9e802b7c
R10: ffff9ff63fd1f0a0 R11: fffff3aa0cc48380 R12: ffffffff9ed8c778
R13: ffff9ff5fd9e7c70 R14: 0000000000001000 R15: 000000000000009e
FS:  00007faba5e8b700(0000) GS:ffff9ff63fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fac1a45a000 CR3: 000000032b4aa000 CR4: 00000000000607e0
Call Trace:
[<ffffffff9e67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9e67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9e67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9e71664f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9e9864f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9e716bfb>] smp_call_function_many+0x22b/0x270
[<ffffffff9e67e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9e67e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9e7ee176>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9e7efc3f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9e7efd63>] tlb_finish_mmu+0x23/0x30
[<ffffffff9e7f9554>] unmap_region+0xf4/0x140
[<ffffffffc03f03c4>] ? xfs_iunlock+0xe4/0x120 [xfs]
[<ffffffff9e7f9b11>] ? __vma_rb_erase+0x121/0x220
Complete report #829254
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8cd1db12
Build id: 3.10.0-1160.92.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8cc81b10
Build id: 3.10.0-1160.92.1.el7.x86_64
0x130
3
? leave_mm
vmlinux 0xffffffff8cc81b10
Build id: 3.10.0-1160.92.1.el7.x86_64
0x130
4
? leave_mm
vmlinux 0xffffffff8cc81b10
Build id: 3.10.0-1160.92.1.el7.x86_64
0x130
5
smp_call_function_single
vmlinux 0xffffffff8cd1dc2f
Build id: 3.10.0-1160.92.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8cf9c485
Build id: 3.10.0-1160.92.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8cd1e1eb
Build id: 3.10.0-1160.92.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8cc81fa8
Build id: 3.10.0-1160.92.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8cc8201b
Build id: 3.10.0-1160.92.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8cdfa00e
Build id: 3.10.0-1160.92.1.el7.x86_64
0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8cdfbb8f
Build id: 3.10.0-1160.92.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8cdfbcc3
Build id: 3.10.0-1160.92.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff8ce05844
Build id: 3.10.0-1160.92.1.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff8ce05e51
Build id: 3.10.0-1160.92.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff8ce07ea5
Build id: 3.10.0-1160.92.1.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff8ce080e5
Build id: 3.10.0-1160.92.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff8ce09412
Build id: 3.10.0-1160.92.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff8d3c539a
Build id: 3.10.0-1160.92.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kubelet:29384]
Modules linked in: xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipport dummy nf_tables ip6table_mangle ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 nf_conntrack_netlink br_netfilter tcp_diag udp_diag inet_diag unix_diag af_packet_diag netlink_diag mmfs26(OE) mmfslinux(OE) tracedev(OE) overlay(T) ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink xt_set ip_set xt_addrtype iptable_nat nf_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_CHECKSUM xt_conntrack iptable_mangle ipt_rpfilter xt_comment xt_mark iptable_raw sunrpc ext4 mbcache jbd2
ppdev pcspkr joydev virtio_balloon parport_pc parport i2c_piix4 ip_vs_wrr ip_vs_wlc ip_vs_sh ip_vs_sed ip_vs_rr ip_vs_pe_sip nf_conntrack_sip ip_vs_nq ip_vs_lc ip_vs_lblcr ip_vs_lblc ip_vs_ftp nf_nat ip_vs_dh ip_vs nf_conntrack binfmt_misc ip_tables xfs libcrc32c virtio_console virtio_blk virtio_scsi virtio_net net_failover failover ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci drm_panel_orientation_quirks virtio_ring floppy virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 29384 Comm: kubelet Kdump: loaded Tainted: G        W  OEL ------------ T 3.10.0-1160.92.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff9c5c0d204200 ti: ffff9c5a8d678000 task.ti: ffff9c5a8d678000
RIP: 0010:[<ffffffff8cd1db12>]  [<ffffffff8cd1db12>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9c5a8d67bc20  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9c5a8d67bbf0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff9c5a8d67bc70 R08: ffffffff8d413e48 R09: ffffffff8ce0eebf
R10: ffff9c582cedf0a0 R11: ffffec805655e4c0 R12: ffffffff8d3bb7e8
R13: ffff9c5a8d67bc70 R14: 0000000000000128 R15: 0000000000000001
FS:  00007f8b8cb8c700(0000) GS:ffff9c582cec0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8bbdec2000 CR3: 000000080d55c000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8cc81b10>] ? leave_mm+0x130/0x130
[<ffffffff8cc81b10>] ? leave_mm+0x130/0x130
[<ffffffff8cc81b10>] ? leave_mm+0x130/0x130
[<ffffffff8cd1dc2f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8cf9c485>] ? cpumask_next_and+0x35/0x50
[<ffffffff8cd1e1eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8cc81fa8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8cc8201b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8cdfa00e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffff8cdfbb8f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8cdfbcc3>] tlb_finish_mmu+0x23/0x40
[<ffffffff8ce05844>] unmap_region+0xf4/0x150
[<ffffffff8ce05e51>] ? __vma_rb_erase+0x121/0x230
[<ffffffff8ce07ea5>] do_munmap+0x2a5/0x480
[<ffffffff8ce080e5>] vm_munmap+0x65/0xb0
[<ffffffff8ce09412>] SyS_munmap+0x22/0x30
[<ffffffff8d3c539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 95 8d 48 89 df e8 c7 66 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #813770
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff95b16ae2
Build id: 3.10.0-1160.76.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff95a7ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff95a7ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff95a7ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff95b16bff
Build id: 3.10.0-1160.76.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff95d86845
Build id: 3.10.0-1160.76.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff95b171ab
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff95a7e258
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff95a7e2cb
Build id: 3.10.0-1160.76.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff95bee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff95bf016f
Build id: 3.10.0-1160.76.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff95bf0293
Build id: 3.10.0-1160.76.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff95bf9a84
Build id: 3.10.0-1160.76.1.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff95bfa041
Build id: 3.10.0-1160.76.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff95bfc025
Build id: 3.10.0-1160.76.1.el7.x86_64
0x2a5
NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [rpm:63088]
NMI watchdog: BUG: soft lockup - CPU#10 stuck for 22s! [rpm:63088]
Modules linked in: nf_conntrack_netlink tcp_diag udp_diag inet_diag ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ext4 mbcache jbd2 bridge stp llc ip_set binfmt_misc ppdev joydev sg virtio_balloon parport_pc parport i2c_piix4 pcspkr xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix
libata virtio_pci floppy virtio_ring serio_raw virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 10 PID: 63088 Comm: rpm Kdump: loaded Tainted: G             L ------------   3.10.0-1160.76.1.el7.x86_64 #1
Modules linked in: nf_conntrack_netlink tcp_diag udp_diag inet_diag ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ext4 mbcache jbd2 bridge stp llc ip_set binfmt_misc ppdev joydev sg virtio_balloon parport_pc parport i2c_piix4 pcspkr xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net net_failover virtio_console virtio_scsi virtio_blk failover cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix
libata virtio_pci floppy virtio_ring serio_raw virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 10 PID: 63088 Comm: rpm Kdump: loaded Tainted: G             L ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9af70d7b2100 ti: ffff9af3ed0a8000 task.ti: ffff9af3ed0a8000
RIP: 0010:[<ffffffff95b16ae2>]  [<ffffffff95b16ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9af3ed0abc20  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9af3ed0abbf0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff9af3ed0abc70 R08: ffffffff96214e48 R09: ffffffff95c02f68
R10: ffff9afa5d09f0a0 R11: ffffe2c15b5bc0c0 R12: ffff9af357d8d4e8
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  00007fb593add840(0000) GS:ffff9afa5d080000(0000) knlGS:0000000000000000
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9af70d7b2100 ti: ffff9af3ed0a8000 task.ti: ffff9af3ed0a8000
RIP: 0010:[<ffffffff95b16ae2>]  [<ffffffff95b16ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9af3ed0abc20  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9af3ed0abbf0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff9af3ed0abc70 R08: ffffffff96214e48 R09: ffffffff95c02f68
R10: ffff9afa5d09f0a0 R11: ffffe2c15b5bc0c0 R12: ffff9af357d8d4e8
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
FS:  00007fb593add840(0000) GS:ffff9afa5d080000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb593af6000 CR3: 0000000134792000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff95a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff95a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff95a7ddd0>] ? leave_mm+0x120/0x120
[<ffffffff95b16bff>] smp_call_function_single+0x5f/0xa0
[<ffffffff95d86845>] ? cpumask_next_and+0x35/0x50
[<ffffffff95b171ab>] smp_call_function_many+0x22b/0x270
[<ffffffff95a7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff95a7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff95bee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff95bf016f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff95bf0293>] tlb_finish_mmu+0x23/0x30
[<ffffffff95bf9a84>] unmap_region+0xf4/0x140
[<ffffffff95bfa041>] ? __vma_rb_erase+0x121/0x220
[<ffffffff95bfc025>] do_munmap+0x2a5/0x480
Complete report #786774
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa091da02
Build id: 3.10.0-1160.83.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffa0881af0
Build id: 3.10.0-1160.83.1.el7.x86_64
0x130
3
? leave_mm
vmlinux 0xffffffffa0881af0
Build id: 3.10.0-1160.83.1.el7.x86_64
0x130
4
? leave_mm
vmlinux 0xffffffffa0881af0
Build id: 3.10.0-1160.83.1.el7.x86_64
0x130
5
smp_call_function_single
vmlinux 0xffffffffa091db1f
Build id: 3.10.0-1160.83.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa0b9bf95
Build id: 3.10.0-1160.83.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa091e0db
Build id: 3.10.0-1160.83.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa0881f88
Build id: 3.10.0-1160.83.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa0881ffb
Build id: 3.10.0-1160.83.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa09f9f4e
Build id: 3.10.0-1160.83.1.el7.x86_64
0xae
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa09fba8f
Build id: 3.10.0-1160.83.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa09fbbc3
Build id: 3.10.0-1160.83.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffa0a05564
Build id: 3.10.0-1160.83.1.el7.x86_64
0xf4
14
? blk_finish_plug
vmlinux 0xffffffffa0b6c0e4
Build id: 3.10.0-1160.83.1.el7.x86_64
0x14
15
? __vma_rb_erase
vmlinux 0xffffffffa0a05b71
Build id: 3.10.0-1160.83.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffa0a07bb5
Build id: 3.10.0-1160.83.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffa0a07df5
Build id: 3.10.0-1160.83.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffa0a09122
Build id: 3.10.0-1160.83.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffa0fc539a
Build id: 3.10.0-1160.83.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 24s! [pool:24985]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg joydev vmw_balloon pcspkr parport_pc parport snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm snd_timer snd soundcore vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm serio_raw e1000 ata_piix mptspi nfit scsi_transport_spi libata mptscsih libnvdimm mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 24985 Comm: pool Kdump: loaded Tainted: G        W    L ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff9cb293f03180 ti: ffff9cb425a20000 task.ti: ffff9cb425a20000
RIP: 0010:[<ffffffffa091da02>]  [<ffffffffa091da02>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9cb425a23c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9cb425a23bf0 RCX: 0000000000000001
RDX: ffffffffa10138d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9cb425a23c70 R08: ffffffffa10138c8 R09: 000000018040003f
R10: 0000000000000001 R11: ffff9cb3dc626540 R12: ffff9cb425a23bd8
R13: ffffffffa08e90cc R14: ffff9cb425a23bd0 R15: 0000000000000001
FS:  00007ff055e53700(0000) GS:ffff9cb4356c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f70e2fea05a CR3: 0000000074fe6000 CR4: 00000000003607e0
Call Trace:
[<ffffffffa0881af0>] ? leave_mm+0x130/0x130
[<ffffffffa0881af0>] ? leave_mm+0x130/0x130
[<ffffffffa0881af0>] ? leave_mm+0x130/0x130
[<ffffffffa091db1f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa0b9bf95>] ? cpumask_next_and+0x35/0x50
[<ffffffffa091e0db>] smp_call_function_many+0x22b/0x270
[<ffffffffa0881f88>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa0881ffb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa09f9f4e>] tlb_flush_mmu_tlbonly+0xae/0xd0
[<ffffffffa09fba8f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa09fbbc3>] tlb_finish_mmu+0x23/0x40
[<ffffffffa0a05564>] unmap_region+0xf4/0x150
[<ffffffffa0b6c0e4>] ? blk_finish_plug+0x14/0x50
[<ffffffffa0a05b71>] ? __vma_rb_erase+0x121/0x230
[<ffffffffa0a07bb5>] do_munmap+0x2a5/0x480
[<ffffffffa0a07df5>] vm_munmap+0x65/0xb0
[<ffffffffa0a09122>] SyS_munmap+0x22/0x30
[<ffffffffa0fc539a>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 55 a1 48 89 df e8 d7 62 29 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #778134
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8bb0779e
Build id: 3.10.0-862.el7.x86_64
0xfe
2
? leave_mm
vmlinux 0xffffffff8ba76e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff8ba76e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff8ba76e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff8bb078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8bd4a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8bb07e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8ba77018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8ba77088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff8bbc0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff8bbc1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff8bbca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc05c4bec
Build id: 3.10.0-862.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffff8bbcabe1
Build id: 3.10.0-862.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff8bbccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffff8bbcce35
Build id: 3.10.0-862.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff8bbce052
Build id: 3.10.0-862.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff8c11f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [AliYunDunMonito:7340]
Modules linked in: AliSecNetFlt64(OE) AliSecProcFilter64(OE) AliSecGuard(OE) tcp_diag inet_diag tun devlink nfsv3 nfs_acl nfs lockd grace fscache ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon ppdev joydev nfit pcspkr libnvdimm parport_pc parport sg shpchp i2c_piix4 vmw_vmci binfmt_misc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul crct10dif_common mptspi scsi_transport_spi libata crc32c_intel serio_raw mptscsih mptbase vmxnet3 i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: AliSecNetFlt64]
CPU: 1 PID: 7340 Comm: AliYunDunMonito Kdump: loaded Tainted: G           OE  ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff98404cfbeeb0 ti: ffff984029768000 task.ti: ffff984029768000
RIP: 0010:[<ffffffff8bb0779e>]  [<ffffffff8bb0779e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff98402976bc30  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff98402976bc00 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff98402976bc78 R08: ffffffff8c214060 R09: ffffffff8bbd35bc
R10: ffff98423fc5ba40 R11: ffffe63c1abf8bc0 R12: 0000000000000001
R13: ffffffff8bc61e45 R14: ffffffff8c11a925 R15: ffff98402976bbb8
FS:  00007f8eed5d6700(0000) GS:ffff98423fc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8ef2a0d000 CR3: 000000080ef64000 CR4: 00000000001607e0
Call Trace:
[<ffffffff8ba76e50>] ? leave_mm+0x110/0x110
[<ffffffff8ba76e50>] ? leave_mm+0x110/0x110
[<ffffffff8ba76e50>] ? leave_mm+0x110/0x110
[<ffffffff8bb078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff8bd4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff8bb07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff8ba77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8ba77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8bbc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff8bbc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff8bbca624>] unmap_region+0xf4/0x140
[<ffffffffc05c4bec>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffff8bbcabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff8bbccbf5>] do_munmap+0x295/0x470
[<ffffffff8bbcce35>] vm_munmap+0x65/0xb0
[<ffffffff8bbce052>] SyS_munmap+0x22/0x30
[<ffffffff8c11f7d5>] system_call_fastpath+0x1c/0x21
Code: 48 89 de 48 03 14 c5 a0 53 73 8c 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 
Complete report #777913
Frame # Function Binary Source or offset Line
1
? leave_mm
vmlinux 0xffffffffab276e50
Build id: 3.10.0-862.el7.x86_64
0x110
2
? leave_mm
vmlinux 0xffffffffab276e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffab276e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
smp_call_function_single
vmlinux 0xffffffffab3078af
Build id: 3.10.0-862.el7.x86_64
0x5f
5
? cpumask_next_and
vmlinux 0xffffffffab54a275
Build id: 3.10.0-862.el7.x86_64
0x35
6
smp_call_function_many
vmlinux 0xffffffffab307e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
7
native_flush_tlb_others
vmlinux 0xffffffffab277018
Build id: 3.10.0-862.el7.x86_64
0xb8
8
flush_tlb_mm_range
vmlinux 0xffffffffab277088
Build id: 3.10.0-862.el7.x86_64
0x68
9
tlb_flush_mmu.part.75
vmlinux 0xffffffffab3c0497
Build id: 3.10.0-862.el7.x86_64
0x37
10
tlb_finish_mmu
vmlinux 0xffffffffab3c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
11
unmap_region
vmlinux 0xffffffffab3ca624
Build id: 3.10.0-862.el7.x86_64
0xf4
12
? __vma_rb_erase
vmlinux 0xffffffffab3cabe1
Build id: 3.10.0-862.el7.x86_64
0x121
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 30s! [java:117922]
PKRU: 55555554
Call Trace:
[<ffffffffab276e50>] ? leave_mm+0x110/0x110
[<ffffffffab276e50>] ? leave_mm+0x110/0x110
[<ffffffffab276e50>] ? leave_mm+0x110/0x110
[<ffffffffab3078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffab54a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffab307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffab277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffab277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffab3c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffab3c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffab3ca624>] unmap_region+0xf4/0x140
[<ffffffffab3cabe1>] ? __vma_rb_erase+0x121/0x220
Complete report #710183
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f932a
Build id: 3.10.0-693.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f942f
Build id: 3.10.0-693.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813226f5
Build id: 3.10.0-693.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99c3
Build id: 3.10.0-693.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b5444
Build id: 3.10.0-693.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff811b5a01
Build id: 3.10.0-693.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff811b7a02
Build id: 3.10.0-693.el7.x86_64
0x292
15
vm_munmap
vmlinux 0xffffffff811b7c45
Build id: 3.10.0-693.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff811b8d52
Build id: 3.10.0-693.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [sqlservr:1948]
Modules linked in: tcp_lp fuse xt_nat veth br_netfilter tun overlay(T) ip_set nfnetlink bridge stp llc iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev edac_core iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device pcspkr snd_pcm joydev nfit libnvdimm snd_timer vmw_vmci snd sg shpchp soundcore i2c_piix4 parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm crct10dif_pclmul crct10dif_common drm crc32c_intel ata_piix e1000 libata mptspi scsi_transport_spi mptscsih serio_raw mptbase i2c_core [last unloaded: iptable_filter]
CPU: 2 PID: 1948 Comm: sqlservr Tainted: G        W    L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8800b2470000 ti: ffff88031d31c000 task.ti: ffff88031d31c000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88031d31fc60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff88031d31fc30 RCX: 0000000000000004
RDX: ffffffff816cebb0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff88031d31fcb0 R08: ffffffff816ceba8 R09: ffffffff811bd768
R10: ffff880370299bc0 R11: ffffea000dafcf00 R12: ffff880361b32000
R13: ffffea0005197400 R14: ffff8800b9566400 R15: ffff8800b2470000
FS:  00007f8bea8b9700(0000) GS:ffff880370280000(006b) knlGS:000000006a612000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8bea8df000 CR3: 0000000320bd5000 CR4: 00000000003407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff88031d31fd38
0000000000000003 00000000db24a9fe ffff88031d31fca8 0000000000000003
ffffffff8106e4a0 ffffffff8106e4a0 ffff88031d31fce0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
[<ffffffff811b5a01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7a02>] do_munmap+0x292/0x470
[<ffffffff811b7c45>] vm_munmap+0x65/0xb0
[<ffffffff811b8d52>] SyS_munmap+0x22/0x30
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #656869
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff91316b82
Build id: 3.10.0-1160.62.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9127ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff9127ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff9127ddd0
Build id: 3.10.0-1160.62.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff91316c9f
Build id: 3.10.0-1160.62.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff91586825
Build id: 3.10.0-1160.62.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9131724b
Build id: 3.10.0-1160.62.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9127e258
Build id: 3.10.0-1160.62.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9127e2cb
Build id: 3.10.0-1160.62.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff913ee666
Build id: 3.10.0-1160.62.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff913f012f
Build id: 3.10.0-1160.62.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff913f0253
Build id: 3.10.0-1160.62.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff913f9a44
Build id: 3.10.0-1160.62.1.el7.x86_64
0xf4
14
? do_sync_write
vmlinux 0xffffffff9144da00
Build id: 3.10.0-1160.62.1.el7.x86_64
0x90
15
? __vma_rb_erase
vmlinux 0xffffffff913fa001
Build id: 3.10.0-1160.62.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffff913fc035
Build id: 3.10.0-1160.62.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffff913fc275
Build id: 3.10.0-1160.62.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffff913fd572
Build id: 3.10.0-1160.62.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffff91999f92
Build id: 3.10.0-1160.62.1.el7.x86_64
0x25
20
? system_call_after_swapgs
vmlinux 0xffffffff91999ed5
Build id: 3.10.0-1160.62.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [mysqld:9911]
Modules linked in: dattobd(OE) nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod ppdev vmw_balloon coretemp pcspkr joydev sg vmw_vmci parport_pc parport i2c_piix4 ip_tables xfs libcrc32c ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic vmwgfx crct10dif_common drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm ata_piix mptsas drm scsi_transport_sas libata e1000e serio_raw mptscsih mptbase ptp pps_core drm_panel_orientation_quirks floppy fuse [last unloaded: dattobd]
CPU: 3 PID: 9911 Comm: mysqld Tainted: G        W  OEL ------------ T 3.10.0-1160.62.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff97a56caf8000 ti: ffff97a24df98000 task.ti: ffff97a24df98000
RIP: 0010:[<ffffffff91316b82>]  [<ffffffff91316b82>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff97a24df9bc20  EFLAGS: 00000202
RAX: 00000000000008fd RBX: 00007fada25eb000 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000282
RBP: ffff97a24df9bc70 R08: 0000000000000001 R09: 00003ffffffff000
R10: ffff97a5091e18c8 R11: 0000000000000206 R12: fffff418c7027e80
R13: ffff97a55ad77f50 R14: ffffffff913622b4 R15: ffff97a24df9bbb0
FS:  00007fad8018c700(0000) GS:ffff97a57fcc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fada25ea014 CR3: 0000000428a32000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9127ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9127ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9127ddd0>] ? leave_mm+0x120/0x120
[<ffffffff91316c9f>] smp_call_function_single+0x5f/0xa0
[<ffffffff91586825>] ? cpumask_next_and+0x35/0x50
[<ffffffff9131724b>] smp_call_function_many+0x22b/0x270
[<ffffffff9127e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9127e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff913ee666>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff913f012f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff913f0253>] tlb_finish_mmu+0x23/0x30
[<ffffffff913f9a44>] unmap_region+0xf4/0x140
[<ffffffff9144da00>] ? do_sync_write+0x90/0xe0
[<ffffffff913fa001>] ? __vma_rb_erase+0x121/0x220
[<ffffffff913fc035>] do_munmap+0x2a5/0x480
[<ffffffff913fc275>] vm_munmap+0x65/0xb0
[<ffffffff913fd572>] SyS_munmap+0x22/0x30
[<ffffffff91999f92>] system_call_fastpath+0x25/0x2a
[<ffffffff91999ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 18 f5 91 48 89 df e8 17 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #640377
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f932a
Build id: 3.10.0-693.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f942f
Build id: 3.10.0-693.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813226f5
Build id: 3.10.0-693.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99c3
Build id: 3.10.0-693.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b5444
Build id: 3.10.0-693.el7.x86_64
0xf4
13
? copy_user_generic_unrolled
vmlinux 0xffffffff81330240
Build id: 3.10.0-693.el7.x86_64
0xa0
14
? __vma_rb_erase
vmlinux 0xffffffff811b5a01
Build id: 3.10.0-693.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff811b7a02
Build id: 3.10.0-693.el7.x86_64
0x292
16
vm_munmap
vmlinux 0xffffffff811b7c45
Build id: 3.10.0-693.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff811b8d52
Build id: 3.10.0-693.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [com.alibaba.nac:7324]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) tcp_diag inet_diag xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon joydev vmw_vmci pcspkr sg i2c_piix4 shpchp nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt
fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi scsi_transport_spi libata mptscsih serio_raw e1000 mptbase i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 7324 Comm: com.alibaba.nac Tainted: G             L ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff8801c43b2f70 ti: ffff88006cf54000 task.ti: ffff88006cf54000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88006cf57c60  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffff88017dd114d0 RCX: 0000000000000001
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff88006cf57cb0 R08: ffffffff816ce428 R09: ffffffff811bd85c
R10: ffff880236759bc0 R11: ffffea0006999540 R12: ffff8801bebc9000
R13: ffffea0008245340 R14: ffff880185a9e400 R15: ffff8801c43b2f70
FS:  00007f21060e4700(0000) GS:ffff880236740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f21ec014000 CR3: 00000001c4f3e000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff88006cf57d38
0000000000000003 000000007f542615 ffff88006cf57ca8 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff88006cf57ce0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
[<ffffffff81330240>] ? copy_user_generic_unrolled+0xa0/0xc0
[<ffffffff811b5a01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7a02>] do_munmap+0x292/0x470
[<ffffffff811b7c45>] vm_munmap+0x65/0xb0
[<ffffffff811b8d52>] SyS_munmap+0x22/0x30
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #621414
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810fcf5a
Build id: 3.10.0-693.21.1.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff810fd06f
Build id: 3.10.0-693.21.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813288a5
Build id: 3.10.0-693.21.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810fd61b
Build id: 3.10.0-693.21.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff81070b98
Build id: 3.10.0-693.21.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff81070c68
Build id: 3.10.0-693.21.1.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811b18c3
Build id: 3.10.0-693.21.1.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811b2be5
Build id: 3.10.0-693.21.1.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b9cb4
Build id: 3.10.0-693.21.1.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc028310c
Build id: 3.10.0-693.21.1.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffff811ba271
Build id: 3.10.0-693.21.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff811bc285
Build id: 3.10.0-693.21.1.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffff811bc4c5
Build id: 3.10.0-693.21.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff811bd5d2
Build id: 3.10.0-693.21.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [zk-session-expi:8133]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event sb_edac edac_core iosf_mbi crc32_pclmul snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus aesni_intel lrw snd_seq gf128mul glue_helper ablk_helper snd_seq_device cryptd snd_pcm ppdev vmw_balloon snd_timer snd soundcore pcspkr joydev vmw_vmci sg i2c_piix4 shpchp
nfit libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi crct10dif_pclmul crct10dif_common vmwgfx crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm e1000 mptspi serio_raw drm scsi_transport_spi mptscsih mptbase ata_piix libata i2c_core
CPU: 3 PID: 8133 Comm: zk-session-expi Tainted: G        W    L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff8800a2bd5ee0 ti: ffff880010fe4000 task.ti: ffff880010fe4000
RIP: 0010:[<ffffffff810fcf5a>]  [<ffffffff810fcf5a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff880010fe7c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff880010fe7c00 RCX: 0000000000000002
RDX: ffffffff816dcb70 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff880010fe7c80 R08: ffffffff816dcb68 R09: ffffffff811c20dc
R10: ffff8801396db940 R11: ffffea0000839c80 R12: ffffffff816bb835
R13: ffff880010fe7bc8 R14: ffffea00027c6100 R15: 0000000000000000
FS:  00007fddb2050700(0000) GS:ffff8801396c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fddb06ff000 CR3: 00000001317d8000 CR4: 00000000001607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810fd06f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813288a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fd61b>] smp_call_function_many+0x22b/0x270
[<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81070c68>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811b18c3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b2be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b9cb4>] unmap_region+0xf4/0x140
[<ffffffffc028310c>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffff811ba271>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811bc285>] do_munmap+0x295/0x470
[<ffffffff811bc4c5>] vm_munmap+0x65/0xb0
[<ffffffff811bd5d2>] SyS_munmap+0x22/0x30
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 80 95 01 00 48 89 de 48 03 14 c5 a0 10 b2 81 48 89 df e8 7a 03 24 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #528497
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff88f111ba
Build id: 3.10.0-957.5.1.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff88e7a400
Build id: 3.10.0-957.5.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff88e7a400
Build id: 3.10.0-957.5.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff88e7a400
Build id: 3.10.0-957.5.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff88f112cf
Build id: 3.10.0-957.5.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff89175a45
Build id: 3.10.0-957.5.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff88f1187b
Build id: 3.10.0-957.5.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff88e7a5c8
Build id: 3.10.0-957.5.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff88e7a638
Build id: 3.10.0-957.5.1.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff88fe4427
Build id: 3.10.0-957.5.1.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff88fe5d25
Build id: 3.10.0-957.5.1.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff88fef3c4
Build id: 3.10.0-957.5.1.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff88fef981
Build id: 3.10.0-957.5.1.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff88ff19b5
Build id: 3.10.0-957.5.1.el7.x86_64
0x2a5
15
vm_munmap
vmlinux 0xffffffff88ff1bf5
Build id: 3.10.0-957.5.1.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff88ff2e82
Build id: 3.10.0-957.5.1.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff89574ddb
Build id: 3.10.0-957.5.1.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [automount:60652]
Modules linked in: vfat fat uas usb_storage cmac arc4 md4 nls_utf8 cifs ccm dns_resolver fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun nfsv3 nfs_acl bridge nfs stp llc lockd grace fscache ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter mvfs(OE) vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 snd_rawmidi snd_ac97_codec joydev ac97_bus snd_seq pcspkr snd_seq_device snd_pcm snd_timer snd sg soundcore vmw_vmci i2c_piix4 nfit libnvdimm parport_pc
parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw ata_piix syscopyarea sysfillrect sysimgblt libata fb_sys_fops ttm mptspi scsi_transport_spi mptscsih drm e1000 mptbase drm_panel_orientation_quirks floppy
CPU: 1 PID: 60652 Comm: automount Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-957.5.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff8abdf03130c0 ti: ffff8abd273dc000 task.ti: ffff8abd273dc000
RIP: 0010:[<ffffffff88f111ba>]  [<ffffffff88f111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8abd273dfc30  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffff8abd273dfcc8 RCX: 0000000000000003
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff8abd273dfc78 R08: ffffffff896132c8 R09: ffffffff88ff8648
R10: ffff8abe7645f0a0 R11: ffffd236d723ce80 R12: 0000000000000001
R13: 0000000000001000 R14: ffffffff88fd8408 R15: ffff8abd273dfbb0
FS:  00007f9559dca700(0000) GS:ffff8abe76440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9565f64000 CR3: 0000000622bdc000 CR4: 00000000001607e0
Call Trace:
[<ffffffff88e7a400>] ? leave_mm+0x110/0x110
[<ffffffff88e7a400>] ? leave_mm+0x110/0x110
[<ffffffff88e7a400>] ? leave_mm+0x110/0x110
[<ffffffff88f112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffff89175a45>] ? cpumask_next_and+0x35/0x50
[<ffffffff88f1187b>] smp_call_function_many+0x22b/0x270
[<ffffffff88e7a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff88e7a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffff88fe4427>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff88fe5d25>] tlb_finish_mmu+0x55/0x60
[<ffffffff88fef3c4>] unmap_region+0xf4/0x140
[<ffffffff88fef981>] ? __vma_rb_erase+0x121/0x220
[<ffffffff88ff19b5>] do_munmap+0x2a5/0x480
[<ffffffff88ff1bf5>] vm_munmap+0x65/0xb0
[<ffffffff88ff2e82>] SyS_munmap+0x22/0x30
[<ffffffff89574ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 b4 89 48 89 df e8 da b6 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #495721
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9770779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff97676e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff97676e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff97676e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff977078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9794a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff97707e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff97677018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff97677088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff977c0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff977c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff977ca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc0423bec
Build id: 3.10.0-862.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffff977cabe1
Build id: 3.10.0-862.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff977ccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffff977cce35
Build id: 3.10.0-862.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff977ce052
Build id: 3.10.0-862.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff97d1f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:97407]
Modules linked in: ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq iosf_mbi snd_seq_device crc32_pclmul snd_pcm ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_timer cryptd snd ppdev vmw_balloon soundcore pcspkr joydev sg vmw_vmci i2c_piix4 shpchp nfit libnvdimm parport_pc parport binfmt_misc xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi drm ata_piix crct10dif_pclmul mptscsih libata crct10dif_common crc32c_intel e1000 i2c_core serio_raw mptbase [last unloaded: ip_tables]
CPU: 1 PID: 97407 Comm: java Tainted: G        W    L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9d7f806c8fd0 ti: ffff9d7f8d360000 task.ti: ffff9d7f8d360000
RIP: 0010:[<ffffffff9770779a>]  [<ffffffff9770779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9d7f8d363c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9d7f8d363c00 RCX: 0000000000000004
RDX: ffffffff97e138f0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff9d7f8d363c78 R08: ffffffff97e138e8 R09: ffffffff977d35bc
R10: ffff9d80fd65ba40 R11: ffffe4149988f2c0 R12: 0000000000000001
R13: 0000000000000001 R14: ffffffff97d1a925 R15: ffff9d7f8d363bb8
FS:  00002ae9988e9700(0000) GS:ffff9d80fd640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002ae8fef5a000 CR3: 0000000fdf2ae000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffff97676e50>] ? leave_mm+0x110/0x110
[<ffffffff97676e50>] ? leave_mm+0x110/0x110
[<ffffffff97676e50>] ? leave_mm+0x110/0x110
[<ffffffff977078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9794a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff97707e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff97677018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff97677088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff977c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff977c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff977ca624>] unmap_region+0xf4/0x140
[<ffffffffc0423bec>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffff977cabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff977ccbf5>] do_munmap+0x295/0x470
[<ffffffff977cce35>] vm_munmap+0x65/0xb0
[<ffffffff977ce052>] SyS_munmap+0x22/0x30
[<ffffffff97d1f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 33 98 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #444484
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa5914a22
Build id: 3.10.0-1062.1.2.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffa587d1d0
Build id: 3.10.0-1062.1.2.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffa587d1d0
Build id: 3.10.0-1062.1.2.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffa587d1d0
Build id: 3.10.0-1062.1.2.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffa5914b3f
Build id: 3.10.0-1062.1.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa5b7f475
Build id: 3.10.0-1062.1.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa59150eb
Build id: 3.10.0-1062.1.2.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa587d398
Build id: 3.10.0-1062.1.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa587d40b
Build id: 3.10.0-1062.1.2.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa59ea166
Build id: 3.10.0-1062.1.2.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa59ebc2f
Build id: 3.10.0-1062.1.2.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa59ebd53
Build id: 3.10.0-1062.1.2.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffa59f5544
Build id: 3.10.0-1062.1.2.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffffa59f5b01
Build id: 3.10.0-1062.1.2.el7.x86_64
0x121
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 24s! [7_scheduler:19349]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun bridge stp llc devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci i2c_piix4 parport ip_tables xfs libcrc32c
sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic ahci drm ata_piix libahci libata nfit libnvdimm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 6 PID: 19349 Comm: 7_scheduler Tainted: G        W    L ------------ T 3.10.0-1062.1.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff96e9918ca0e0 ti: ffff96e977df4000 task.ti: ffff96e977df4000
RIP: 0010:[<ffffffffa5914a22>]  [<ffffffffa5914a22>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff96e977df7c20  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff96eb3ffda000 RCX: 0000000000000030
RDX: 0000000000000010 RSI: 0000000000000010 RDI: 0000000000000286
RBP: ffff96e977df7c70 R08: ffffffffa6013a40 R09: ffffffffa59fe7e8
R10: ffff96ea3bd9f0a0 R11: fffffee8440b3440 R12: 0000000000000000
R13: ffff96ea3ffdb008 R14: 0000000000000008 R15: ffff96ea3ffdb000
FS:  00007f5985a2d700(0000) GS:ffff96ea3bd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f597e825000 CR3: 00000001c2720000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa587d1d0>] ? leave_mm+0x110/0x110
[<ffffffffa587d1d0>] ? leave_mm+0x110/0x110
[<ffffffffa587d1d0>] ? leave_mm+0x110/0x110
[<ffffffffa5914b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa5b7f475>] ? cpumask_next_and+0x35/0x50
[<ffffffffa59150eb>] smp_call_function_many+0x22b/0x270
[<ffffffffa587d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa587d40b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa59ea166>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa59ebc2f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa59ebd53>] tlb_finish_mmu+0x23/0x30
[<ffffffffa59f5544>] unmap_region+0xf4/0x140
[<ffffffffa59f5b01>] ? __vma_rb_erase+0x121/0x220
Complete report #442274
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810fb03a
Build id: 3.10.0-693.11.6.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106f270
Build id: 3.10.0-693.11.6.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff8106f270
Build id: 3.10.0-693.11.6.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff8106f270
Build id: 3.10.0-693.11.6.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff810fb13f
Build id: 3.10.0-693.11.6.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813248a5
Build id: 3.10.0-693.11.6.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810fb6d3
Build id: 3.10.0-693.11.6.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106f438
Build id: 3.10.0-693.11.6.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106f508
Build id: 3.10.0-693.11.6.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811aef13
Build id: 3.10.0-693.11.6.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811b0235
Build id: 3.10.0-693.11.6.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b7304
Build id: 3.10.0-693.11.6.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc014654c
Build id: 3.10.0-693.11.6.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffff811b78c1
Build id: 3.10.0-693.11.6.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff811b98c2
Build id: 3.10.0-693.11.6.el7.x86_64
0x292
16
vm_munmap
vmlinux 0xffffffff811b9b05
Build id: 3.10.0-693.11.6.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff811bac12
Build id: 3.10.0-693.11.6.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff816b89fd
Build id: 3.10.0-693.11.6.el7.x86_64
0x16
19
? system_call_after_swapgs
vmlinux 0xffffffff816b889d
Build id: 3.10.0-693.11.6.el7.x86_64
0xca
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [node:1843]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter joydev iTCO_wdt iTCO_vendor_support snd_intel8x0 ppdev snd_ac97_codec iosf_mbi ac97_bus snd_seq sg pcspkr snd_seq_device e1000 snd_pcm lpc_ich parport_pc parport video snd_timer i2c_piix4 snd i2c_core ahci soundcore libahci nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi ata_piix crc32c_intel serio_raw libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1843 Comm: node Tainted: G             L ------------   3.10.0-693.11.6.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff88007a170fd0 ti: ffff88003ca70000 task.ti: ffff88003ca70000
RIP: 0010:[<ffffffff810fb03a>]  [<ffffffff810fb03a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff88003ca73c30  EFLAGS: 00010202
RAX: 00000000000400f6 RBX: 0000000000000001 RCX: 0000000000000000
RDX: 00000000000008fb RSI: 00000000000000fb RDI: 0000000000000286
RBP: ffff88003ca73c80 R08: 0000000000000001 R09: ffffffff811bf628
R10: ffff88007fd1b900 R11: ffffea0001ecdcc0 R12: ffffffff816b9bff
R13: ffffffff816b9c06 R14: ffffffff816b9c0d R15: ffffffff816b9c14
FS:  00007f94c2f42700(0000) GS:ffff88007fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f94c636d000 CR3: 0000000035d1a000 CR4: 00000000000206e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8106f270>] ? leave_mm+0x110/0x110
[<ffffffff8106f270>] ? leave_mm+0x110/0x110
[<ffffffff8106f270>] ? leave_mm+0x110/0x110
[<ffffffff810fb13f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813248a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fb6d3>] smp_call_function_many+0x223/0x260
[<ffffffff8106f438>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106f508>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811aef13>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b0235>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b7304>] unmap_region+0xf4/0x140
[<ffffffffc014654c>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffff811b78c1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b98c2>] do_munmap+0x292/0x470
[<ffffffff811b9b05>] vm_munmap+0x65/0xb0
[<ffffffff811bac12>] SyS_munmap+0x22/0x30
[<ffffffff816b89fd>] system_call_fastpath+0x16/0x1b
[<ffffffff816b889d>] ? system_call_after_swapgs+0xca/0x214
Code: 40 95 01 00 48 89 de 48 03 14 c5 20 8e b1 81 48 89 df e8 0a df 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #386912
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9c716a32
Build id: 3.10.0-1160.41.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9c67ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff9c67ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff9c67ddd0
Build id: 3.10.0-1160.41.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff9c716b4f
Build id: 3.10.0-1160.41.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9c986a25
Build id: 3.10.0-1160.41.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9c7170fb
Build id: 3.10.0-1160.41.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9c67e258
Build id: 3.10.0-1160.41.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9c67e2cb
Build id: 3.10.0-1160.41.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9c7ee676
Build id: 3.10.0-1160.41.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9c7f013f
Build id: 3.10.0-1160.41.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9c7f0263
Build id: 3.10.0-1160.41.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff9c7f9a54
Build id: 3.10.0-1160.41.1.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff9c7fa011
Build id: 3.10.0-1160.41.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff9c7fc045
Build id: 3.10.0-1160.41.1.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff9c7fc285
Build id: 3.10.0-1160.41.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff9c7fd582
Build id: 3.10.0-1160.41.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff9cd95f92
Build id: 3.10.0-1160.41.1.el7.x86_64
0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff9cd95ed5
Build id: 3.10.0-1160.41.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [python2:1307]
Modules linked in: mmfs26(OE) mmfslinux(OE) tracedev(OE) fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nfsv3 nfs_acl nfs lockd grace fscache sunrpc joydev sg virtio_rng virtio_balloon i2c_piix4 openafs(POE) pcspkr edac_mce_amd binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom virtio_net net_failover virtio_console virtio_scsi failover ata_generic pata_acpi qxl drm_kms_helper crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ghash_clmulni_intel ttm ata_piix drm libata
aesni_intel glue_helper lrw gf128mul ablk_helper cryptd serio_raw virtio_pci virtio_ring drm_panel_orientation_quirks virtio floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 1307 Comm: python2 Kdump: loaded Tainted: P           OE  ------------   3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: oVirt RHEL, BIOS 1.14.0-1.el8s 04/01/2014
task: ffff9e9535a1a100 ti: ffff9e982fe2c000 task.ti: ffff9e982fe2c000
RIP: 0010:[<ffffffff9c716a32>]  [<ffffffff9c716a32>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9e982fe2fc20  EFLAGS: 00000202
RAX: 0000000000000010 RBX: ffff9e982fe2fbf0 RCX: 0000000000000030
RDX: 000000000000ffff RSI: 0000000000000010 RDI: 0000000000000282
RBP: ffff9e982fe2fc70 R08: ffffffff9ce137c0 R09: 00003ffffffff000
R10: ffff9e982c9b93c0 R11: 0000000000000246 R12: ffff9e983ffd8000
R13: 0000000000000000 R14: 0000000000000246 R15: 00000001000001c1
FS:  00007f8d03fff700(0000) GS:ffff9e98332c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8d14d71018 CR3: 00000000bb444000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9c67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9c67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9c67ddd0>] ? leave_mm+0x120/0x120
[<ffffffff9c716b4f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9c986a25>] ? cpumask_next_and+0x35/0x50
[<ffffffff9c7170fb>] smp_call_function_many+0x22b/0x270
[<ffffffff9c67e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9c67e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9c7ee676>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9c7f013f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9c7f0263>] tlb_finish_mmu+0x23/0x30
[<ffffffff9c7f9a54>] unmap_region+0xf4/0x140
[<ffffffff9c7fa011>] ? __vma_rb_erase+0x121/0x220
[<ffffffff9c7fc045>] do_munmap+0x2a5/0x480
[<ffffffff9c7fc285>] vm_munmap+0x65/0xb0
[<ffffffff9c7fd582>] SyS_munmap+0x22/0x30
[<ffffffff9cd95f92>] system_call_fastpath+0x25/0x2a
[<ffffffff9cd95ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 15 35 9d 48 89 df e8 67 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #360404
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffadd163d6
Build id: 3.10.0-1062.12.1.el7.x86_64
0x106
2
? leave_mm
vmlinux 0xffffffffadc7e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffadc7e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffadc7e2b0
Build id: 3.10.0-1062.12.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffadd164ef
Build id: 3.10.0-1062.12.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffadf81685
Build id: 3.10.0-1062.12.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffadd16a9b
Build id: 3.10.0-1062.12.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffadc7e478
Build id: 3.10.0-1062.12.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffadc7e4eb
Build id: 3.10.0-1062.12.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffaddec096
Build id: 3.10.0-1062.12.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffaddedb5f
Build id: 3.10.0-1062.12.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffaddedc83
Build id: 3.10.0-1062.12.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffaddf7474
Build id: 3.10.0-1062.12.1.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffffaddf7a31
Build id: 3.10.0-1062.12.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffaddf9a75
Build id: 3.10.0-1062.12.1.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffffaddf9cb5
Build id: 3.10.0-1062.12.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffaddfaf52
Build id: 3.10.0-1062.12.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffae38dede
Build id: 3.10.0-1062.12.1.el7.x86_64
0x25
19
? system_call_after_swapgs
vmlinux 0xffffffffae38de21
Build id: 3.10.0-1062.12.1.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [systemd-logind:1120]
Modules linked in: tcp_lp fuse kbase(OE) xt_NFQUEUE nfnetlink_log nfnetlink nfring(OE) vmsecmod(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter nf_conntrack_ipv6 nf_defrag_ipv6 nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache sunrpc ppdev joydev virtio_balloon sg i2c_piix4 pcspkr parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_net virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci virtio_ring serio_raw floppy virtio
drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmsec_nfq]
CPU: 2 PID: 1120 Comm: systemd-logind Kdump: loaded Tainted: GF          OEL ------------   3.10.0-1062.12.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9c2cf9b81070 ti: ffff9c1f25ce4000 task.ti: ffff9c1f25ce4000
RIP: 0010:[<ffffffffadd163d6>]  [<ffffffffadd163d6>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff9c1f25ce7c20  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff9c1f25ce7bf0 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000282
RBP: ffff9c1f25ce7c70 R08: ffffffffae413548 R09: ffffffffade00718
R10: ffff9c2cfce9f0a0 R11: fffff054bfd631c0 R12: ffff9c2bf4ecf800
R13: ffffffffade68625 R14: ffff9c1f25ce7ba8 R15: 0000000000000246
FS:  00007f6c116fc900(0000) GS:ffff9c2cfce80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6c11716000 CR3: 0000000ff6690000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffadc7e2b0>] ? leave_mm+0x110/0x110
[<ffffffffadc7e2b0>] ? leave_mm+0x110/0x110
[<ffffffffadc7e2b0>] ? leave_mm+0x110/0x110
[<ffffffffadd164ef>] smp_call_function_single+0x5f/0xa0
[<ffffffffadf81685>] ? cpumask_next_and+0x35/0x50
[<ffffffffadd16a9b>] smp_call_function_many+0x22b/0x270
[<ffffffffadc7e478>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffadc7e4eb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffaddec096>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffaddedb5f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffaddedc83>] tlb_finish_mmu+0x23/0x30
[<ffffffffaddf7474>] unmap_region+0xf4/0x140
[<ffffffffaddf7a31>] ? __vma_rb_erase+0x121/0x220
[<ffffffffaddf9a75>] do_munmap+0x2a5/0x480
[<ffffffffaddf9cb5>] vm_munmap+0x65/0xb0
[<ffffffffaddfaf52>] SyS_munmap+0x22/0x30
[<ffffffffae38dede>] system_call_fastpath+0x25/0x2a
[<ffffffffae38de21>] ? system_call_after_swapgs+0xae/0x146
Code: 14 c5 20 e8 94 ae 48 89 df e8 37 23 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f 
Complete report #322904
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff96716f62
Build id: 3.10.0-1127.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9667e730
Build id: 3.10.0-1127.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff9667e730
Build id: 3.10.0-1127.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff9667e730
Build id: 3.10.0-1127.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff9671707f
Build id: 3.10.0-1127.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff96984ee5
Build id: 3.10.0-1127.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9671762b
Build id: 3.10.0-1127.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9667e8f8
Build id: 3.10.0-1127.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9667e96b
Build id: 3.10.0-1127.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff967eddb6
Build id: 3.10.0-1127.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff967ef87f
Build id: 3.10.0-1127.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff967ef9a3
Build id: 3.10.0-1127.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff967f91b4
Build id: 3.10.0-1127.el7.x86_64
0xf4
14
? xfs_iunlock
xfs 0xffffffffc04e5404
Build id: 3.10.0-1127.el7.x86_64
0xe4
15
? __vma_rb_erase
vmlinux 0xffffffff967f9771
Build id: 3.10.0-1127.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffff967fb7a5
Build id: 3.10.0-1127.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffff967fb9e5
Build id: 3.10.0-1127.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffff967fcc82
Build id: 3.10.0-1127.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffff96d92ed2
Build id: 3.10.0-1127.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [Curator-Framewo:58255]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd crc32_pclmul ghash_clmulni_intel aesni_intel lrw soundcore gf128mul glue_helper ppdev ablk_helper cryptd pcspkr joydev vmw_balloon sg parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt nfit fb_sys_fops
ttm ata_piix mptspi libnvdimm drm libata scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih crc32c_intel mptbase e1000 serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 58255 Comm: Curator-Framewo Kdump: loaded Tainted: G        W    L ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
task: ffff92175245a0e0 ti: ffff921830a90000 task.ti: ffff921830a90000
RIP: 0010:[<ffffffff96716f62>]  [<ffffffff96716f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff921830a93c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff921830a93bf0 RCX: 0000000000000001
RDX: ffffffff96e13610 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff921830a93c70 R08: ffffffff96e13608 R09: ffffffff9680277c
R10: ffff92183969f0a0 R11: fffffce602248d80 R12: ffffffff96d8d975
R13: ffff921830a93bb8 R14: 00007f6bdc008000 R15: ffff9217a5635000
FS:  00007f6bcf469700(0000) GS:ffff921839680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6bdc008000 CR3: 0000000035f84000 CR4: 00000000003607e0
Call Trace:
[<ffffffff9667e730>] ? leave_mm+0x120/0x120
[<ffffffff9667e730>] ? leave_mm+0x120/0x120
[<ffffffff9667e730>] ? leave_mm+0x120/0x120
[<ffffffff9671707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff96984ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9671762b>] smp_call_function_many+0x22b/0x270
[<ffffffff9667e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9667e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff967eddb6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff967ef87f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff967ef9a3>] tlb_finish_mmu+0x23/0x30
[<ffffffff967f91b4>] unmap_region+0xf4/0x140
[<ffffffffc04e5404>] ? xfs_iunlock+0xe4/0x120 [xfs]
[<ffffffff967f9771>] ? __vma_rb_erase+0x121/0x220
[<ffffffff967fb7a5>] do_munmap+0x2a5/0x480
[<ffffffff967fb9e5>] vm_munmap+0x65/0xb0
[<ffffffff967fcc82>] SyS_munmap+0x22/0x30
[<ffffffff96d92ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 0f 35 97 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #321087
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9c716f72
Build id: 3.10.0-1127.19.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9c67e730
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff9c67e730
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff9c67e730
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff9c71708f
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9c985005
Build id: 3.10.0-1127.19.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9c71763b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9c67e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9c67e96b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9c7ede36
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9c7ef8ff
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9c7efa23
Build id: 3.10.0-1127.19.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff9c7f9234
Build id: 3.10.0-1127.19.1.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff9c7f97f1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff9c7fb825
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff9c7fba65
Build id: 3.10.0-1127.19.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff9c7fcd02
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff9cd92ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff9cd92e15
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 24s! [grid-timeout-wo:3765]
Modules linked in: tcp_lp fuse xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter overlay(T) vmw_vsock_vmci_transport vsock sunrpc pcspkr ppdev joydev vmw_balloon coretemp sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata crc32c_intel mptspi scsi_transport_spi mptscsih serio_raw mptbase vmxnet3 drm_panel_orientation_quirks
floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 3765 Comm: grid-timeout-wo Kdump: loaded Tainted: G    B        L ------------ T 3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 10/22/2013
task: ffff8c072a333150 ti: ffff8c072a28c000 task.ti: ffff8c072a28c000
RIP: 0010:[<ffffffff9c716f72>]  [<ffffffff9c716f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8c072a28fc20  EFLAGS: 00000202
RAX: 0000000000000040 RBX: fffff875cd4db5c0 RCX: 0000000000000001
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000286
RBP: ffff8c072a28fc70 R08: ffffffff9ce136c8 R09: ffffffff9c80280c
R10: ffff8c076e7df0a0 R11: fffff875cc0efd40 R12: ffff8c072a333150
R13: 0000000000000000 R14: 00000000000280da R15: ffffffff9c83e2b2
FS:  00007f466e0e8700(0000) GS:ffff8c076e7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4674dec000 CR3: 000000020c6c4000 CR4: 00000000000007e0
Call Trace:
[<ffffffff9c67e730>] ? leave_mm+0x120/0x120
[<ffffffff9c67e730>] ? leave_mm+0x120/0x120
[<ffffffff9c67e730>] ? leave_mm+0x120/0x120
[<ffffffff9c71708f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9c985005>] ? cpumask_next_and+0x35/0x50
[<ffffffff9c71763b>] smp_call_function_many+0x22b/0x270
[<ffffffff9c67e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9c67e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9c7ede36>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9c7ef8ff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9c7efa23>] tlb_finish_mmu+0x23/0x30
[<ffffffff9c7f9234>] unmap_region+0xf4/0x140
[<ffffffff9c7f97f1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff9c7fb825>] do_munmap+0x2a5/0x480
[<ffffffff9c7fba65>] vm_munmap+0x65/0xb0
[<ffffffff9c7fcd02>] SyS_munmap+0x22/0x30
[<ffffffff9cd92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff9cd92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 a0 10 35 9d 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #299976
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbc30779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffbc276e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffbc276e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffbc276e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffbc3078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbc54a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffbc307e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbc277018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbc277088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffbc3c0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffbc3c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffbc3ca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffbc3cabe1
Build id: 3.10.0-862.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffbc3ccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
15
vm_munmap
vmlinux 0xffffffffbc3cce35
Build id: 3.10.0-862.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffbc3ce052
Build id: 3.10.0-862.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffbc91f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [systemd:1]
Modules linked in: nls_utf8 isofs xt_conntrack iptable_filter iptable_raw iptable_security iptable_mangle iptable_nat nf_nat_ipv4 nf_nat vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod ext4 iosf_mbi crc32_pclmul ghash_clmulni_intel mbcache aesni_intel jbd2 lrw gf128mul glue_helper ablk_helper cryptd ppdev vmw_balloon pcspkr joydev sg parport_pc parport shpchp vmw_vmci i2c_piix4 nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix mptspi drm scsi_transport_spi libata mptscsih crct10dif_pclmul mptbase crct10dif_common crc32c_intel serio_raw vmxnet3 i2c_core
CPU: 4 PID: 1 Comm: systemd Kdump: loaded Tainted: G        W    L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9b91ba320000 ti: ffff9b91ba2bc000 task.ti: ffff9b91ba2bc000
RIP: 0010:[<ffffffffbc30779a>]  [<ffffffffbc30779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9b91ba2bfc30  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9b91ba2bfc00 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff9b91ba2bfc78 R08: ffffffffbca13660 R09: ffffffffbc3d34c8
R10: ffff9b947fd1ba40 R11: fffff1df82ee5b40 R12: ffffffffbc3a335e
R13: ffff9b91ba2bfba8 R14: ffff9b946a69cca8 R15: fffff1df897c4300
FS:  00007f0017cb8940(0000) GS:ffff9b947fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0017ccb000 CR3: 000000043f9d8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffbc276e50>] ? leave_mm+0x110/0x110
[<ffffffffbc276e50>] ? leave_mm+0x110/0x110
[<ffffffffbc276e50>] ? leave_mm+0x110/0x110
[<ffffffffbc3078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbc54a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffbc307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffbc277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbc277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbc3c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffbc3c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffbc3ca624>] unmap_region+0xf4/0x140
[<ffffffffbc3cabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffbc3ccbf5>] do_munmap+0x295/0x470
[<ffffffffbc3cce35>] vm_munmap+0x65/0xb0
[<ffffffffbc3ce052>] SyS_munmap+0x22/0x30
[<ffffffffbc91f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 f3 bc 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #296065
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa2f16536
Build id: 3.10.0-1160.31.1.el7.x86_64
0x106
2
? leave_mm
vmlinux 0xffffffffa2e7ddd0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffa2e7ddd0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffffa2e7ddd0
Build id: 3.10.0-1160.31.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffffa2f1664f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa31864f5
Build id: 3.10.0-1160.31.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa2f16bfb
Build id: 3.10.0-1160.31.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa2e7e258
Build id: 3.10.0-1160.31.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa2e7e2cb
Build id: 3.10.0-1160.31.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffa2fee176
Build id: 3.10.0-1160.31.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffa2fefc3f
Build id: 3.10.0-1160.31.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffa2fefd63
Build id: 3.10.0-1160.31.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffa2ff9554
Build id: 3.10.0-1160.31.1.el7.x86_64
0xf4
14
? kfree
vmlinux 0xffffffffa30265d6
Build id: 3.10.0-1160.31.1.el7.x86_64
0x106
15
? __vma_rb_erase
vmlinux 0xffffffffa2ff9b11
Build id: 3.10.0-1160.31.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffa2ffbb45
Build id: 3.10.0-1160.31.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffa2ffbd85
Build id: 3.10.0-1160.31.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffa2ffd082
Build id: 3.10.0-1160.31.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffa3595f92
Build id: 3.10.0-1160.31.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [systemd-journal:92751]
Modules linked in: ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6_tables ipt_rpfilter xt_set iptable_raw ip_set_hash_ip ip_set_hash_net ip_set ipip tunnel4 ip_tunnel xt_nat xt_statistic ipt_REJECT nf_reject_ipv4 iptable_mangle xt_comment xt_mark xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype iptable_filter iptable_nat nf_nat_ipv4 nf_nat rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache overlay(T) vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev sg pcspkr vmw_balloon parport_pc parport vmw_vmci i2c_piix4 nfsd auth_rpcgss nf_conntrack_ipv4 nfs_acl nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs lockd
grace sunrpc nf_conntrack libcrc32c ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod crc_t10dif crct10dif_generic syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel ahci ata_piix libahci libata serio_raw nfit libnvdimm vmxnet3 vmw_pvscsi drm_panel_orientation_quirks fuse br_netfilter bridge stp llc
CPU: 1 PID: 92751 Comm: systemd-journal Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1160.31.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/28/2020
task: ffffa08419091080 ti: ffffa0835bda0000 task.ti: ffffa0835bda0000
RIP: 0010:[<ffffffffa2f16536>]  [<ffffffffa2f16536>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffffa0835bda3c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa0835bda3bf0 RCX: 0000000000000001
RDX: ffffffffa36137d0 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffffa0835bda3c70 R08: ffffffffa36137c8 R09: ffffffffa3002b7c
R10: ffffa0841d65f0a0 R11: ffffeac1e0310380 R12: ffffeac1c4d870c0
R13: 0000000100000001 R14: ffffa08419091080 R15: 0000000000000000
FS:  00007f463e566880(0000) GS:ffffa0841d640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f463e575000 CR3: 00000007a8014000 CR4: 00000000001607e0
Call Trace:
[<ffffffffa2e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2e7ddd0>] ? leave_mm+0x120/0x120
[<ffffffffa2f1664f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa31864f5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2f16bfb>] smp_call_function_many+0x22b/0x270
[<ffffffffa2e7e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2e7e2cb>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffa2fee176>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffa2fefc3f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffa2fefd63>] tlb_finish_mmu+0x23/0x30
[<ffffffffa2ff9554>] unmap_region+0xf4/0x140
[<ffffffffa30265d6>] ? kfree+0x106/0x140
[<ffffffffa2ff9b11>] ? __vma_rb_erase+0x121/0x220
[<ffffffffa2ffbb45>] do_munmap+0x2a5/0x480
[<ffffffffa2ffbd85>] vm_munmap+0x65/0xb0
[<ffffffffa2ffd082>] SyS_munmap+0x22/0x30
[<ffffffffa3595f92>] system_call_fastpath+0x25/0x2a
Code: 14 c5 60 15 b5 a3 48 89 df e8 37 71 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f 
Complete report #294898
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffbb11109a
Build id: 3.10.0-957.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffbb07a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffbb07a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffbb07a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffbb1111af
Build id: 3.10.0-957.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffbb375465
Build id: 3.10.0-957.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffbb11175b
Build id: 3.10.0-957.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffbb07a598
Build id: 3.10.0-957.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffbb07a608
Build id: 3.10.0-957.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffbb1e4077
Build id: 3.10.0-957.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffbb1e5975
Build id: 3.10.0-957.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffbb1ef014
Build id: 3.10.0-957.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc046e8bc
Build id: 3.10.0-957.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffffbb1ef5d1
Build id: 3.10.0-957.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffbb1f1605
Build id: 3.10.0-957.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffffbb1f1845
Build id: 3.10.0-957.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffbb1f2ad2
Build id: 3.10.0-957.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffbb774ddb
Build id: 3.10.0-957.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [Runnable elasti:46085]
Modules linked in: rpcsec_gss_krb5 nfsd auth_rpcgss nfs_acl lockd grace veth binfmt_misc nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM tun devlink overlay(T) ip6t_MASQUERADE nf_nat_masquerade_ipv6 xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_mark ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi kvm_intel kvm snd_ens1371
snd_rawmidi snd_ac97_codec ac97_bus snd_seq irqbypass snd_seq_device crc32_pclmul ghash_clmulni_intel aesni_intel snd_pcm lrw gf128mul snd_timer snd glue_helper soundcore ppdev ablk_helper cryptd vmw_balloon sg pcspkr joydev nfit libnvdimm parport_pc parport i2c_piix4 vmw_vmci ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ata_piix drm libata serio_raw mptspi scsi_transport_spi e1000 mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 46085 Comm: Runnable elasti Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
task: ffff9e9d7c991040 ti: ffff9e9f92ff4000 task.ti: ffff9e9f92ff4000
RIP: 0010:[<ffffffffbb11109a>]  [<ffffffffbb11109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9e9f92ff7c30  EFLAGS: 00000202
RAX: 0000000000000040 RBX: 0000000000001000 RCX: 0000000000000001
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff9e9f92ff7c78 R08: ffffffffbb812d88 R09: ffffffffbb1f838c
R10: ffff9ea2e869f0a0 R11: ffffde975196cfc0 R12: 0000000000000001
R13: ffff9e9f92ff7bc8 R14: ffffffffbb76f915 R15: ffff9e9f92ff7bb8
FS:  00007fa30d089700(0000) GS:ffff9ea2e8680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa3cd89a000 CR3: 00000001ca648000 CR4: 00000000001627e0
Call Trace:
[<ffffffffbb07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbb07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbb07a3d0>] ? leave_mm+0x110/0x110
[<ffffffffbb1111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffbb375465>] ? cpumask_next_and+0x35/0x50
[<ffffffffbb11175b>] smp_call_function_many+0x22b/0x270
[<ffffffffbb07a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbb07a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffbb1e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffbb1e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffbb1ef014>] unmap_region+0xf4/0x140
[<ffffffffc046e8bc>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffffbb1ef5d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffbb1f1605>] do_munmap+0x2a5/0x480
[<ffffffffbb1f1845>] vm_munmap+0x65/0xb0
[<ffffffffbb1f2ad2>] SyS_munmap+0x22/0x30
[<ffffffffbb774ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 d4 bb 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #289747
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb751109a
Build id: 3.10.0-957.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffb747a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb747a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb747a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb75111af
Build id: 3.10.0-957.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb7775465
Build id: 3.10.0-957.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb751175b
Build id: 3.10.0-957.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb747a598
Build id: 3.10.0-957.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb747a608
Build id: 3.10.0-957.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffb75e4077
Build id: 3.10.0-957.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffb75e5975
Build id: 3.10.0-957.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffb75ef014
Build id: 3.10.0-957.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc04828bc
Build id: 3.10.0-957.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffffb75ef5d1
Build id: 3.10.0-957.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffb75f1605
Build id: 3.10.0-957.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffffb75f1845
Build id: 3.10.0-957.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffb75f2ad2
Build id: 3.10.0-957.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffb7b74ddb
Build id: 3.10.0-957.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [java:83250]
Modules linked in: veth xt_nat xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock fuse vmhgfs(OE) overlay(T) sunrpc snd_seq_midi snd_seq_midi_event coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_ens1371 lrw snd_rawmidi gf128mul glue_helper snd_ac97_codec ablk_helper cryptd ac97_bus snd_seq snd_seq_device ppdev snd_pcm snd_timer vmw_balloon snd soundcore pcspkr sg joydev parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul
crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw vmwgfx e1000 drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata mptbase drm_panel_orientation_quirks nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 83250 Comm: java Kdump: loaded Tainted: G           OE  ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffffa0d3f7b730c0 ti: ffffa0d401bd8000 task.ti: ffffa0d401bd8000
RIP: 0010:[<ffffffffb751109a>]  [<ffffffffb751109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffffa0d401bdbc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffffa0d401bdbc00 RCX: 0000000000000004
RDX: ffffffffb7c13510 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffffa0d401bdbc78 R08: ffffffffb7c13508 R09: ffffffffb75f838c
R10: ffffa0d41265f0a0 R11: ffffdfb302d88f80 R12: 0000000000000001
R13: ffffa0d401bdbbc8 R14: ffffffffb7b6f915 R15: ffffa0d401bdbbb8
FS:  00007fadb5a8c700(0000) GS:ffffa0d412640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fae566b6000 CR3: 0000000196b28000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb747a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb747a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb747a3d0>] ? leave_mm+0x110/0x110
[<ffffffffb75111af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb7775465>] ? cpumask_next_and+0x35/0x50
[<ffffffffb751175b>] smp_call_function_many+0x22b/0x270
[<ffffffffb747a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb747a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb75e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb75e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffffb75ef014>] unmap_region+0xf4/0x140
[<ffffffffc04828bc>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffffb75ef5d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb75f1605>] do_munmap+0x2a5/0x480
[<ffffffffb75f1845>] vm_munmap+0x65/0xb0
[<ffffffffb75f2ad2>] SyS_munmap+0x22/0x30
[<ffffffffb7b74ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 14 b8 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #285660
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f932a
Build id: 3.10.0-693.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f942f
Build id: 3.10.0-693.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813226f5
Build id: 3.10.0-693.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99c3
Build id: 3.10.0-693.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811ad053
Build id: 3.10.0-693.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae375
Build id: 3.10.0-693.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b5444
Build id: 3.10.0-693.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff811b5a01
Build id: 3.10.0-693.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff811b7a02
Build id: 3.10.0-693.el7.x86_64
0x292
15
vm_munmap
vmlinux 0xffffffff811b7c45
Build id: 3.10.0-693.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff811b8d52
Build id: 3.10.0-693.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff816b4fc9
Build id: 3.10.0-693.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [vmtoolsd:734]
Modules linked in: tcp_lp nls_utf8 isofs fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmhgfs(OE) vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event edac_mce_amd edac_core crc32_pclmul ppdev ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device
snd_pcm pcspkr joydev snd_timer sg snd soundcore vmw_vmci shpchp nfit i2c_piix4 libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect pata_acpi sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi ata_piix serio_raw scsi_transport_spi mptscsih libata e1000 mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 734 Comm: vmtoolsd Tainted: G           OE  ------------   3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff880035d8cf10 ti: ffff8800b85d0000 task.ti: ffff8800b85d0000
RIP: 0010:[<ffffffff810f932a>]  [<ffffffff810f932a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff8800b85d3c60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8800b85d3c30 RCX: 0000000000000001
RDX: ffffffff816ce430 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8800b85d3cb0 R08: ffffffff816ce428 R09: ffffffff811bd768
R10: ffff880193659bc0 R11: ffffea0006468c80 R12: ffff880181a69400
R13: ffffea00059dfb00 R14: ffff8800b943c4c0 R15: ffff880035d8cf10
FS:  00007fc2ab4e1880(0000) GS:ffff880193640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc2ab501000 CR3: 00000000b8744000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff8800b85d3d38
0000000000000003 00000000e1d8ba78 ffff8800b85d3ca8 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff8800b85d3ce0 ffffffff810f942f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f942f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813226f5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99c3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811ad053>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae375>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b5444>] unmap_region+0xf4/0x140
[<ffffffff811b5a01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7a02>] do_munmap+0x292/0x470
[<ffffffff811b7c45>] vm_munmap+0x65/0xb0
[<ffffffff811b8d52>] SyS_munmap+0x22/0x30
[<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 7a da 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #269244
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb250a9fe
Build id: 3.10.0-862.11.6.el7.x86_64
0xfe
2
? leave_mm
vmlinux 0xffffffffb2479c90
Build id: 3.10.0-862.11.6.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb2479c90
Build id: 3.10.0-862.11.6.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb2479c90
Build id: 3.10.0-862.11.6.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb250ab0f
Build id: 3.10.0-862.11.6.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb274ec95
Build id: 3.10.0-862.11.6.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb250b0bb
Build id: 3.10.0-862.11.6.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb2479e58
Build id: 3.10.0-862.11.6.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb2479ec8
Build id: 3.10.0-862.11.6.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffb25c33c7
Build id: 3.10.0-862.11.6.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffb25c4cc5
Build id: 3.10.0-862.11.6.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffb25cdab4
Build id: 3.10.0-862.11.6.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc056abdc
Build id: 3.10.0-862.11.6.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffffb25ce071
Build id: 3.10.0-862.11.6.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffb25d0085
Build id: 3.10.0-862.11.6.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffffb25d02c5
Build id: 3.10.0-862.11.6.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffb25d14e2
Build id: 3.10.0-862.11.6.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffb2b2579b
Build id: 3.10.0-862.11.6.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [cbagentd:1865]
Modules linked in: binfmt_misc veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter gsch(OE) redirfs(OE) event_collector_1_9_8557(POE) ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmw_balloon
ppdev sg pcspkr joydev parport_pc parport vmw_vmci shpchp i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper sd_mod syscopyarea sysfillrect crc_t10dif sysimgblt crct10dif_generic fb_sys_fops ttm drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw i2c_core vmxnet3 vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 5 PID: 1865 Comm: cbagentd Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff95430b5b2f70 ti: ffff9542fab4c000 task.ti: ffff9542fab4c000
RIP: 0010:[<ffffffffb250a9fe>]  [<ffffffffb250a9fe>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff9542fab4fc30  EFLAGS: 00000202
RAX: 000000000000000a RBX: ffff9542fab4fc00 RCX: 0000000000000036
RDX: 00000000000003ff RSI: 000000000000000a RDI: 0000000000000286
RBP: ffff9542fab4fc78 R08: ffffffffb2c13320 R09: ffffffffb25d6d9c
R10: ffff954f8d01ba80 R11: ffffe2c2b6a3e9c0 R12: 0000000000000001
R13: 0000000000000000 R14: ffffffffb2b208e5 R15: ffff9542fab4fbb8
FS:  00007f1377a0b700(0000) GS:ffff954f8d000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f13884ef000 CR3: 000000190072e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb2479c90>] ? leave_mm+0x110/0x110
[<ffffffffb2479c90>] ? leave_mm+0x110/0x110
[<ffffffffb2479c90>] ? leave_mm+0x110/0x110
[<ffffffffb250ab0f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb274ec95>] ? cpumask_next_and+0x35/0x50
[<ffffffffb250b0bb>] smp_call_function_many+0x22b/0x270
[<ffffffffb2479e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb2479ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb25c33c7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffb25c4cc5>] tlb_finish_mmu+0x55/0x60
[<ffffffffb25cdab4>] unmap_region+0xf4/0x140
[<ffffffffc056abdc>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffffb25ce071>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb25d0085>] do_munmap+0x295/0x470
[<ffffffffb25d02c5>] vm_munmap+0x65/0xb0
[<ffffffffb25d14e2>] SyS_munmap+0x22/0x30
[<ffffffffb2b2579b>] system_call_fastpath+0x22/0x27
Code: 48 89 de 48 03 14 c5 60 54 13 b3 48 89 df e8 7a a0 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 
Complete report #229685
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9fb0779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff9fa76e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff9fa76e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff9fa76e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff9fb078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9fd4a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9fb07e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9fa77018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9fa77088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff9fbc0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff9fbc1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff9fbca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc0690bec
Build id: 3.10.0-862.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffff9fbcabe1
Build id: 3.10.0-862.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff9fbccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffff9fbcce35
Build id: 3.10.0-862.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff9fbce052
Build id: 3.10.0-862.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffa011f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [netsectcp:1994]
Modules linked in: esn_cfs(OE) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd soundcore iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_balloon parport_pc parport shpchp vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper crct10dif_pclmul
syscopyarea crct10dif_common sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm ata_piix drm mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 1994 Comm: netsectcp Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2012
task: ffff88709d95cf10 ti: ffff88709d980000 task.ti: ffff88709d980000
RIP: 0010:[<ffffffff9fb0779a>]  [<ffffffff9fb0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88709d983c30  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffff88709d983bc8 RCX: 0000000000000001
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff88709d983c78 R08: ffffffffa0213168 R09: ffffffff9fbd35bc
R10: ffff8870bc65ba40 R11: ffffdcbb008473c0 R12: 0000000000000010
R13: ffffffff9fa59653 R14: ffffffffffffff10 R15: 0000000000000282
FS:  00007f89b0dc1700(0000) GS:ffff8870bc640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8a170b8000 CR3: 0000000076904000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9fa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9fa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9fa76e50>] ? leave_mm+0x110/0x110
[<ffffffff9fb078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff9fd4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff9fb07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff9fa77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9fa77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff9fbc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff9fbc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff9fbca624>] unmap_region+0xf4/0x140
[<ffffffffc0690bec>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffff9fbcabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff9fbccbf5>] do_munmap+0x295/0x470
[<ffffffff9fbcce35>] vm_munmap+0x65/0xb0
[<ffffffff9fbce052>] SyS_munmap+0x22/0x30
[<ffffffffa011f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 73 a0 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #211064
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa2d0779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffa2c76e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffa2c76e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffa2c76e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffa2d078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa2f4a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa2d07e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa2c77018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa2c77088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffa2dc0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa2dc1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffa2dca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffa2dcabe1
Build id: 3.10.0-862.el7.x86_64
0x121
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kubelet:4014]
Modules linked in: veth vxlan ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 nfsv4 dns_resolver nfs ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 fscache nf_nat_ipv6 xt_set xt_mark ip_set_hash_ipport ip_set_hash_ipportip ip_set_bitmap_port ip_set_hash_ipportnet ip_set dummy xt_comment nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink nf_conntrack_ipv4 nf_defrag_ipv4 ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs nf_conntrack vmw_vsock_vmci_transport vsock crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev joydev cryptd parport_pc
vmw_balloon parport sg pcspkr vmw_vmci i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc sch_fq_codel ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci libahci ata_piix mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common i2c_core mptscsih libata crc32c_intel serio_raw mptbase vmxnet3 dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 4014 Comm: kubelet Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff9e7c837d1fa0 ti: ffff9e79382d8000 task.ti: ffff9e79382d8000
RIP: 0010:[<ffffffffa2d0779a>]  [<ffffffffa2d0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9e79382dbc30  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff9e79382dbc00 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff9e79382dbc78 R08: ffffffffa3413160 R09: ffffffffa2dd34c8
R10: ffff9e7cbfd1ba40 R11: ffffee809087fd00 R12: 0000000000000001
R13: ffff9e79382dbbc8 R14: ffffffffa331a925 R15: ffff9e79382dbbb8
FS:  00007f6dd27fc700(0000) GS:ffff9e7cbfd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6e0cd2d000 CR3: 000000041ae6a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffa2c76e50>] ? leave_mm+0x110/0x110
[<ffffffffa2c76e50>] ? leave_mm+0x110/0x110
[<ffffffffa2c76e50>] ? leave_mm+0x110/0x110
[<ffffffffa2d078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffa2f4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffa2d07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffa2c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa2c77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa2dc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa2dc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffa2dca624>] unmap_region+0xf4/0x140
[<ffffffffa2dcabe1>] ? __vma_rb_erase+0x121/0x220
Complete report #181121
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa690a9fa
Build id: 3.10.0-862.11.6.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffa6879c90
Build id: 3.10.0-862.11.6.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffa6879c90
Build id: 3.10.0-862.11.6.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffa6879c90
Build id: 3.10.0-862.11.6.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffa690ab0f
Build id: 3.10.0-862.11.6.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa6b4ec95
Build id: 3.10.0-862.11.6.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa690b0bb
Build id: 3.10.0-862.11.6.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa6879e58
Build id: 3.10.0-862.11.6.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa6879ec8
Build id: 3.10.0-862.11.6.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffa69c33c7
Build id: 3.10.0-862.11.6.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa69c4cc5
Build id: 3.10.0-862.11.6.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffa69cdab4
Build id: 3.10.0-862.11.6.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffa69ce071
Build id: 3.10.0-862.11.6.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffa69d0085
Build id: 3.10.0-862.11.6.el7.x86_64
0x295
15
vm_munmap
vmlinux 0xffffffffa69d02c5
Build id: 3.10.0-862.11.6.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffa69d14e2
Build id: 3.10.0-862.11.6.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffa6f2579b
Build id: 3.10.0-862.11.6.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 23s! [ossec-agentd:24494]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_11906(E) falcon_lsm_pinned_11711(E) falcon_lsm_pinned_11611(E) falcon_lsm_pinned_11312(E) falcon_lsm_pinned_11110(E) falcon_lsm_pinned_10807(E) falcon_lsm_pinned_10703(E) falcon_lsm_pinned_8902(E) symap_custom_3_10_0_862_11_6_el7_x86_64(POE) symev_custom_3_10_0_862_11_6_el7_x86_64(OE) vmw_vsock_vmci_transport vsock ppdev vmw_balloon btrfs sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev raid6_pq pcspkr xor sg i2c_piix4 vmw_vmci parport_pc parport nfit libnvdimm shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc sch_fq_codel binfmt_misc ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi
serio_raw crct10dif_pclmul crct10dif_common crc32c_intel vmxnet3 vmwgfx drm_kms_helper floppy syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi mptscsih i2c_core mptbase dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_kal]
CPU: 6 PID: 24494 Comm: ossec-agentd Kdump: loaded Tainted: P           OE  ------------   3.10.0-862.11.6.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/19/2018
task: ffff881390a50fd0 ti: ffff88187dbb8000 task.ti: ffff88187dbb8000
RIP: 0010:[<ffffffffa690a9fa>]  [<ffffffffa690a9fa>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff88187dbbbc30  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff88187dbbbc00 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000286
RBP: ffff88187dbbbc78 R08: ffffffffa7013320 R09: ffffffffa69d6ca8
R10: ffff8818bfd9ba80 R11: ffffc7c08ca23940 R12: ffff881630d40e40
R13: ffff88172f3389d8 R14: ffffffffc03515a2 R15: ffff88187dbbbba8
FS:  00007fae7ffff700(0000) GS:ffff8818bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fae86de9000 CR3: 0000000207288000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa6879c90>] ? leave_mm+0x110/0x110
[<ffffffffa6879c90>] ? leave_mm+0x110/0x110
[<ffffffffa6879c90>] ? leave_mm+0x110/0x110
[<ffffffffa690ab0f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa6b4ec95>] ? cpumask_next_and+0x35/0x50
[<ffffffffa690b0bb>] smp_call_function_many+0x22b/0x270
[<ffffffffa6879e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa6879ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa69c33c7>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffa69c4cc5>] tlb_finish_mmu+0x55/0x60
[<ffffffffa69cdab4>] unmap_region+0xf4/0x140
[<ffffffffa69ce071>] ? __vma_rb_erase+0x121/0x220
[<ffffffffa69d0085>] do_munmap+0x295/0x470
[<ffffffffa69d02c5>] vm_munmap+0x65/0xb0
[<ffffffffa69d14e2>] SyS_munmap+0x22/0x30
[<ffffffffa6f2579b>] system_call_fastpath+0x22/0x27
Code: c0 96 01 00 48 89 de 48 03 14 c5 60 54 53 a7 48 89 df e8 7a a0 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #168202
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff89716f22
Build id: 3.10.0-1127.10.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8967e730
Build id: 3.10.0-1127.10.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff8967e730
Build id: 3.10.0-1127.10.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff8967e730
Build id: 3.10.0-1127.10.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff8971703f
Build id: 3.10.0-1127.10.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff89984fc5
Build id: 3.10.0-1127.10.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff897175eb
Build id: 3.10.0-1127.10.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8967e8f8
Build id: 3.10.0-1127.10.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8967e96b
Build id: 3.10.0-1127.10.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff897ede16
Build id: 3.10.0-1127.10.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff897ef8df
Build id: 3.10.0-1127.10.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff897efa03
Build id: 3.10.0-1127.10.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff897f9214
Build id: 3.10.0-1127.10.1.el7.x86_64
0xf4
14
? __pollwait
vmlinux 0xffffffff8986330c
Build id: 3.10.0-1127.10.1.el7.x86_64
0x1c
15
? __check_object_size
vmlinux 0xffffffff89849d5a
Build id: 3.10.0-1127.10.1.el7.x86_64
0x1ca
16
? __vma_rb_erase
vmlinux 0xffffffff897f97d1
Build id: 3.10.0-1127.10.1.el7.x86_64
0x121
17
do_munmap
vmlinux 0xffffffff897fb805
Build id: 3.10.0-1127.10.1.el7.x86_64
0x2a5
18
vm_munmap
vmlinux 0xffffffff897fba45
Build id: 3.10.0-1127.10.1.el7.x86_64
0x65
19
SyS_munmap
vmlinux 0xffffffff897fcce2
Build id: 3.10.0-1127.10.1.el7.x86_64
0x22
20
system_call_fastpath
vmlinux 0xffffffff89d92ed2
Build id: 3.10.0-1127.10.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [gnome-shell:2541]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc dm_mirror dm_region_hash dm_log dm_mod crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev pcspkr joydev sg vmw_balloon parport_pc parport vmw_vmci i2c_piix4 ip_tables
xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect nfit sysimgblt fb_sys_fops ttm libnvdimm drm crct10dif_pclmul crct10dif_common crc32c_intel ahci libahci ata_piix mptspi libata scsi_transport_spi mptscsih serio_raw mptbase e1000 drm_panel_orientation_quirks
CPU: 1 PID: 2541 Comm: gnome-shell Tainted: G             L ------------   3.10.0-1127.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9f017b7462a0 ti: ffff9f02f4d64000 task.ti: ffff9f02f4d64000
RIP: 0010:[<ffffffff89716f22>]  [<ffffffff89716f22>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9f02f4d67c20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff9f02f4d67bf0 RCX: 0000000000000001
RDX: ffffffff89e13650 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff9f02f4d67c70 R08: ffffffff89e13648 R09: 00003ffffffff000
R10: ffff9f02c11cc7f0 R11: 0000000000000202 R12: 0000001600000001
R13: 0000000f00000001 R14: 0000000e00000001 R15: 0000000d00000001
FS:  00007f2f1ae07a80(0000) GS:ffff9f02f5640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb9233089e5 CR3: 00000002122c0000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8967e730>] ? leave_mm+0x120/0x120
[<ffffffff8967e730>] ? leave_mm+0x120/0x120
[<ffffffff8967e730>] ? leave_mm+0x120/0x120
[<ffffffff8971703f>] smp_call_function_single+0x5f/0xa0
[<ffffffff89984fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffff897175eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8967e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8967e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff897ede16>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff897ef8df>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff897efa03>] tlb_finish_mmu+0x23/0x30
[<ffffffff897f9214>] unmap_region+0xf4/0x140
[<ffffffff8986330c>] ? __pollwait+0x1c/0xf0
[<ffffffff89849d5a>] ? __check_object_size+0x1ca/0x250
[<ffffffff897f97d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff897fb805>] do_munmap+0x2a5/0x480
[<ffffffff897fba45>] vm_munmap+0x65/0xb0
[<ffffffff897fcce2>] SyS_munmap+0x22/0x30
[<ffffffff89d92ed2>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 20 10 35 8a 48 89 df e8 17 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #159631
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffafb1136a
Build id: 3.10.0-957.12.2.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffafa7a460
Build id: 3.10.0-957.12.2.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffafa7a460
Build id: 3.10.0-957.12.2.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffafa7a460
Build id: 3.10.0-957.12.2.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffafb1147f
Build id: 3.10.0-957.12.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffafd75c05
Build id: 3.10.0-957.12.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffafb11a2b
Build id: 3.10.0-957.12.2.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffafa7a628
Build id: 3.10.0-957.12.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffafa7a698
Build id: 3.10.0-957.12.2.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffafbe4607
Build id: 3.10.0-957.12.2.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffafbe5f05
Build id: 3.10.0-957.12.2.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffafbef5a4
Build id: 3.10.0-957.12.2.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffafbefb61
Build id: 3.10.0-957.12.2.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffafbf1b95
Build id: 3.10.0-957.12.2.el7.x86_64
0x2a5
15
vm_munmap
vmlinux 0xffffffffafbf1dd5
Build id: 3.10.0-957.12.2.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffafbf3062
Build id: 3.10.0-957.12.2.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffb0175ddb
Build id: 3.10.0-957.12.2.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [teamviewerd:10341]
Modules linked in: tcp_lp fuse xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables vmhgfs(OE) ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw ppdev gf128mul glue_helper ablk_helper cryptd vmw_balloon pcspkr joydev sg snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec parport_pc parport
ac97_bus snd_seq snd_seq_device snd_pcm snd_timer snd soundcore vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt serio_raw fb_sys_fops ttm drm ata_piix drm_panel_orientation_quirks libata mptspi e1000 scsi_transport_spi mptscsih mptbase nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 10341 Comm: teamviewerd Kdump: loaded Tainted: G           OE  ------------   3.10.0-957.12.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff8e366fb80000 ti: ffff8e365ee80000 task.ti: ffff8e365ee80000
RIP: 0010:[<ffffffffafb1136a>]  [<ffffffffafb1136a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8e365ee83c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8e365ee83c00 RCX: 0000000000000001
RDX: ffffffffb0212ed0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8e365ee83c78 R08: ffffffffb0212ec8 R09: ffffffffafbf8828
R10: ffff8e36756df0a0 R11: ffffe109c0a578c0 R12: 0000000000000001
R13: 0000000000000000 R14: ffffffffafbe9988 R15: ffff8e365ee83c20
FS:  00007f10f1de8700(0000) GS:ffff8e36756c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f110b6d7000 CR3: 000000021db00000 CR4: 00000000003607e0
Call Trace:
[<ffffffffafa7a460>] ? leave_mm+0x110/0x110
[<ffffffffafa7a460>] ? leave_mm+0x110/0x110
[<ffffffffafa7a460>] ? leave_mm+0x110/0x110
[<ffffffffafb1147f>] smp_call_function_single+0x5f/0xa0
[<ffffffffafd75c05>] ? cpumask_next_and+0x35/0x50
[<ffffffffafb11a2b>] smp_call_function_many+0x22b/0x270
[<ffffffffafa7a628>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffafa7a698>] flush_tlb_mm_range+0x68/0x140
[<ffffffffafbe4607>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffafbe5f05>] tlb_finish_mmu+0x55/0x60
[<ffffffffafbef5a4>] unmap_region+0xf4/0x140
[<ffffffffafbefb61>] ? __vma_rb_erase+0x121/0x220
[<ffffffffafbf1b95>] do_munmap+0x2a5/0x480
[<ffffffffafbf1dd5>] vm_munmap+0x65/0xb0
[<ffffffffafbf3062>] SyS_munmap+0x22/0x30
[<ffffffffb0175ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 bc 74 b0 48 89 df e8 fa b6 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #146599
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff87516842
Build id: 3.10.0-1160.25.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8747e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff8747e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff8747e6a0
Build id: 3.10.0-1160.25.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff8751695f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff87786395
Build id: 3.10.0-1160.25.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff87516f0b
Build id: 3.10.0-1160.25.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8747e868
Build id: 3.10.0-1160.25.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8747e8db
Build id: 3.10.0-1160.25.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff875edf56
Build id: 3.10.0-1160.25.1.el7.x86_64
0xa6
11
unmap_page_range
vmlinux 0xffffffff875f1119
Build id: 3.10.0-1160.25.1.el7.x86_64
0x979
12
unmap_single_vma
vmlinux 0xffffffff875f14a1
Build id: 3.10.0-1160.25.1.el7.x86_64
0x81
13
unmap_vmas
vmlinux 0xffffffff875f2ed9
Build id: 3.10.0-1160.25.1.el7.x86_64
0x49
14
unmap_region
vmlinux 0xffffffff875f92fe
Build id: 3.10.0-1160.25.1.el7.x86_64
0xbe
15
? __vma_rb_erase
vmlinux 0xffffffff875f990f
Build id: 3.10.0-1160.25.1.el7.x86_64
0x13f
16
do_munmap
vmlinux 0xffffffff875fb925
Build id: 3.10.0-1160.25.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffff875fbb65
Build id: 3.10.0-1160.25.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffff875fce62
Build id: 3.10.0-1160.25.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffff87b95f92
Build id: 3.10.0-1160.25.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [MongoDB Compass:15268]
Modules linked in: tcp_lp rfcomm xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter binfmt_misc des_generic md4 nls_utf8 cifs ccm dns_resolver overlay(T) bnep vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event ppdev iosf_mbi crc32_pclmul vmw_balloon snd_ens1371 ghash_clmulni_intel snd_rawmidi snd_ac97_codec ac97_bus btusb snd_seq aesni_intel btrtl btbcm btintel lrw gf128mul glue_helper ablk_helper cryptd bluetooth snd_seq_device snd_pcm joydev pcspkr snd_timer rfkill snd sg soundcore
vmw_vmci i2c_piix4 parport_pc parport squashfs loop ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect nfit sysimgblt fb_sys_fops ttm libnvdimm crct10dif_pclmul ata_piix crct10dif_common crc32c_intel drm mptspi scsi_transport_spi libata e1000 serio_raw mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 15268 Comm: MongoDB Compass Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 02/27/2020
task: ffff9043e74a2100 ti: ffff904a98ba0000 task.ti: ffff904a98ba0000
RIP: 0010:[<ffffffff87516842>]  [<ffffffff87516842>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff904a98ba3b10  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff904a98ba3ae0 RCX: 0000000000000007
RDX: ffffffff87c146d0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff904a98ba3b58 R08: ffffffff87c146c8 R09: 00003ffffffff000
R10: ffff9049861430e0 R11: fff00000f0000000 R12: ffff904abd9bf600
R13: ffffffff874d2ff2 R14: ffff904a98ba3ac8 R15: ffffffff874dae02
FS:  00007fef00ab8a80(0000) GS:ffff9051b3b40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb0f75cd3f0 CR3: 00000003a3dce000 CR4: 00000000003607e0
Call Trace:
[<ffffffff8747e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8747e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8747e6a0>] ? leave_mm+0x120/0x120
[<ffffffff8751695f>] smp_call_function_single+0x5f/0xa0
[<ffffffff87786395>] ? cpumask_next_and+0x35/0x50
[<ffffffff87516f0b>] smp_call_function_many+0x22b/0x270
[<ffffffff8747e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8747e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff875edf56>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff875f1119>] unmap_page_range+0x979/0xc80
[<ffffffff875f14a1>] unmap_single_vma+0x81/0xf0
[<ffffffff875f2ed9>] unmap_vmas+0x49/0x90
[<ffffffff875f92fe>] unmap_region+0xbe/0x140
[<ffffffff875f990f>] ? __vma_rb_erase+0x13f/0x220
[<ffffffff875fb925>] do_munmap+0x2a5/0x480
[<ffffffff875fbb65>] vm_munmap+0x65/0xb0
[<ffffffff875fce62>] SyS_munmap+0x22/0x30
[<ffffffff87b95f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 a0 15 15 88 48 89 df e8 c7 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #138826
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa531117a
Build id: 3.10.0-957.1.3.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffa527a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffa527a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffa527a3d0
Build id: 3.10.0-957.1.3.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffa531128f
Build id: 3.10.0-957.1.3.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa55756d5
Build id: 3.10.0-957.1.3.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa531183b
Build id: 3.10.0-957.1.3.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa527a598
Build id: 3.10.0-957.1.3.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa527a608
Build id: 3.10.0-957.1.3.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffa53e42e7
Build id: 3.10.0-957.1.3.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa53e5be5
Build id: 3.10.0-957.1.3.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffa53ef284
Build id: 3.10.0-957.1.3.el7.x86_64
0xf4
13
? blk_finish_plug
vmlinux 0xffffffffa5546fc4
Build id: 3.10.0-957.1.3.el7.x86_64
0x14
14
? __vma_rb_erase
vmlinux 0xffffffffa53ef841
Build id: 3.10.0-957.1.3.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffa53f1875
Build id: 3.10.0-957.1.3.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffffa53f1ab5
Build id: 3.10.0-957.1.3.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffa53f2d42
Build id: 3.10.0-957.1.3.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffa5974ddb
Build id: 3.10.0-957.1.3.el7.x86_64
0x22
19
? system_call_after_swapgs
vmlinux 0xffffffffa5974d21
Build id: 3.10.0-957.1.3.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [milter-greylist:5319]
Modules linked in: xt_set xt_multiport nf_nat_ftp nf_conntrack_ftp ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ip_set_hash_ip ip_set nfnetlink bridge stp llc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel snd_hda_codec_generic lrw gf128mul glue_helper ablk_helper cryptd ppdev snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm pcspkr joydev snd_timer virtio_balloon snd sg parport_pc soundcore
parport i2c_piix4 xfs libcrc32c sr_mod cdrom virtio_blk virtio_console virtio_net crct10dif_pclmul crct10dif_common crc32c_intel ata_generic pata_acpi serio_raw qxl drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix floppy libata drm virtio_pci virtio_ring virtio drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 5319 Comm: milter-greylist Kdump: loaded Not tainted 3.10.0-957.1.3.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff959aca06d140 ti: ffff959ae4f74000 task.ti: ffff959ae4f74000
RIP: 0010:[<ffffffffa531117a>]  [<ffffffffa531117a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff959ae4f77c30  EFLAGS: 00000202
RAX: 0000000000000002 RBX: ffff959ae4f77bc0 RCX: 000000000000003e
RDX: 0000000000000003 RSI: 0000000000000002 RDI: 0000000000000286
RBP: ffff959ae4f77c78 R08: ffffffffa5a12dc0 R09: 0000000100400025
R10: 00000000d6765301 R11: ffffd45d0459d940 R12: 0000000000000010
R13: ffffffffa52611c6 R14: ffffffffffffff10 R15: 0000000000000286
FS:  00007f83ee7fc700(0000) GS:ffff959bffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000003e32dd0 CR3: 0000000137e18000 CR4: 00000000000406e0
Call Trace:
[<ffffffffa527a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa527a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa527a3d0>] ? leave_mm+0x110/0x110
[<ffffffffa531128f>] smp_call_function_single+0x5f/0xa0
[<ffffffffa55756d5>] ? cpumask_next_and+0x35/0x50
[<ffffffffa531183b>] smp_call_function_many+0x22b/0x270
[<ffffffffa527a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa527a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa53e42e7>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffa53e5be5>] tlb_finish_mmu+0x55/0x60
[<ffffffffa53ef284>] unmap_region+0xf4/0x140
[<ffffffffa5546fc4>] ? blk_finish_plug+0x14/0x40
[<ffffffffa53ef841>] ? __vma_rb_erase+0x121/0x220
[<ffffffffa53f1875>] do_munmap+0x2a5/0x480
[<ffffffffa53f1ab5>] vm_munmap+0x65/0xb0
[<ffffffffa53f2d42>] SyS_munmap+0x22/0x30
[<ffffffffa5974ddb>] system_call_fastpath+0x22/0x27
[<ffffffffa5974d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 f4 a5 48 89 df e8 aa b3 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #122847
Frame # Function Binary Source or offset Line
1
? leave_mm
vmlinux 0xffffffffbb67e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x120
2
? leave_mm
vmlinux 0xffffffffbb67e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffbb67e6a0
Build id: 3.10.0-1160.11.1.el7.x86_64
0x120
4
smp_call_function_single
vmlinux 0xffffffffbb71691f
Build id: 3.10.0-1160.11.1.el7.x86_64
0x5f
5
? cpumask_next_and
vmlinux 0xffffffffbb985fe5
Build id: 3.10.0-1160.11.1.el7.x86_64
0x35
6
smp_call_function_many
vmlinux 0xffffffffbb716ecb
Build id: 3.10.0-1160.11.1.el7.x86_64
0x22b
7
native_flush_tlb_others
vmlinux 0xffffffffbb67e868
Build id: 3.10.0-1160.11.1.el7.x86_64
0xb8
8
flush_tlb_mm_range
vmlinux 0xffffffffbb67e8db
Build id: 3.10.0-1160.11.1.el7.x86_64
0x6b
9
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffbb7edee6
Build id: 3.10.0-1160.11.1.el7.x86_64
0xa6
10
arch_tlb_finish_mmu
vmlinux 0xffffffffbb7ef9af
Build id: 3.10.0-1160.11.1.el7.x86_64
0x3f
11
tlb_finish_mmu
vmlinux 0xffffffffbb7efad3
Build id: 3.10.0-1160.11.1.el7.x86_64
0x23
12
unmap_region
vmlinux 0xffffffffbb7f92c4
Build id: 3.10.0-1160.11.1.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffbb7f9881
Build id: 3.10.0-1160.11.1.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffbb7fb8b5
Build id: 3.10.0-1160.11.1.el7.x86_64
0x2a5
15
vm_munmap
vmlinux 0xffffffffbb7fbaf5
Build id: 3.10.0-1160.11.1.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffbb7fcd92
Build id: 3.10.0-1160.11.1.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffbbd93f92
Build id: 3.10.0-1160.11.1.el7.x86_64
0x25
18
? system_call_after_swapgs
vmlinux 0xffffffffbbd93ed5
Build id: 3.10.0-1160.11.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#12 stuck for 23s! [master:1684]
Call Trace:
[<ffffffffbb67e6a0>] ? leave_mm+0x120/0x120
[<ffffffffbb67e6a0>] ? leave_mm+0x120/0x120
[<ffffffffbb67e6a0>] ? leave_mm+0x120/0x120
[<ffffffffbb71691f>] smp_call_function_single+0x5f/0xa0
[<ffffffffbb985fe5>] ? cpumask_next_and+0x35/0x50
[<ffffffffbb716ecb>] smp_call_function_many+0x22b/0x270
[<ffffffffbb67e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffbb67e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffbb7edee6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffbb7ef9af>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffbb7efad3>] tlb_finish_mmu+0x23/0x30
[<ffffffffbb7f92c4>] unmap_region+0xf4/0x140
[<ffffffffbb7f9881>] ? __vma_rb_erase+0x121/0x220
[<ffffffffbb7fb8b5>] do_munmap+0x2a5/0x480
[<ffffffffbb7fbaf5>] vm_munmap+0x65/0xb0
[<ffffffffbb7fcd92>] SyS_munmap+0x22/0x30
[<ffffffffbbd93f92>] system_call_fastpath+0x25/0x2a
[<ffffffffbbd93ed5>] ? system_call_after_swapgs+0xa2/0x13a
Complete report #117820
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb43167a2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffb427e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffb427e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffffb427e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffffb43168bf
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb45862c5
Build id: 3.10.0-1160.15.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb4316e6b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb427e868
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb427e8db
Build id: 3.10.0-1160.15.2.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb43ede86
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb43ef94f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb43efa73
Build id: 3.10.0-1160.15.2.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffb43f9264
Build id: 3.10.0-1160.15.2.el7.x86_64
0xf4
14
? blk_finish_plug
vmlinux 0xffffffffb45576d4
Build id: 3.10.0-1160.15.2.el7.x86_64
0x14
15
? __vma_rb_erase
vmlinux 0xffffffffb43f9821
Build id: 3.10.0-1160.15.2.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffb43fb855
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffb43fba95
Build id: 3.10.0-1160.15.2.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffb43fcd92
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffb4994f92
Build id: 3.10.0-1160.15.2.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [Thread (pooled):33715]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev snd_ens1371 vmw_balloon snd_rawmidi snd_ac97_codec ac97_bus aesni_intel snd_seq lrw gf128mul glue_helper snd_seq_device ablk_helper cryptd snd_pcm pcspkr joydev
snd_timer sg snd soundcore vmw_vmci i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel drm ata_piix nfit libata libnvdimm serio_raw mptspi scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 33715 Comm: Thread (pooled) Kdump: loaded Tainted: G             L ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff90d15f953180 ti: ffff90d001618000 task.ti: ffff90d001618000
RIP: 0010:[<ffffffffb43167a2>]  [<ffffffffb43167a2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff90d00161bc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff90d00161bbf0 RCX: 0000000000000006
RDX: ffffffffb4a14450 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff90d00161bc70 R08: ffffffffb4a14448 R09: 000000010040000f
R10: 0000000093ff4901 R11: ffff90ce93ff4d40 R12: ffff90d19d69ad40
R13: ffff90d15f9531e8 R14: ffff90cd8a398b42 R15: 0000000000000028
FS:  00007f755f7fe700(0000) GS:ffff90d19d680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000002f900e8 CR3: 000000080c6f0000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffb427e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb427e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb427e6a0>] ? leave_mm+0x120/0x120
[<ffffffffb43168bf>] smp_call_function_single+0x5f/0xa0
[<ffffffffb45862c5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb4316e6b>] smp_call_function_many+0x22b/0x270
[<ffffffffb427e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb427e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb43ede86>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb43ef94f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb43efa73>] tlb_finish_mmu+0x23/0x30
[<ffffffffb43f9264>] unmap_region+0xf4/0x140
[<ffffffffb45576d4>] ? blk_finish_plug+0x14/0x40
[<ffffffffb43f9821>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb43fb855>] do_munmap+0x2a5/0x480
[<ffffffffb43fba95>] vm_munmap+0x65/0xb0
[<ffffffffb43fcd92>] SyS_munmap+0x22/0x30
[<ffffffffb4994f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 13 f5 b4 48 89 df e8 97 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #76800
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9af167a2
Build id: 3.10.0-1160.15.2.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9ae7e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff9ae7e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff9ae7e6a0
Build id: 3.10.0-1160.15.2.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff9af168bf
Build id: 3.10.0-1160.15.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9b1862c5
Build id: 3.10.0-1160.15.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9af16e6b
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9ae7e868
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9ae7e8db
Build id: 3.10.0-1160.15.2.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9afede86
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9afef94f
Build id: 3.10.0-1160.15.2.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9afefa73
Build id: 3.10.0-1160.15.2.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff9aff9264
Build id: 3.10.0-1160.15.2.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff9aff9821
Build id: 3.10.0-1160.15.2.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff9affb855
Build id: 3.10.0-1160.15.2.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff9affba95
Build id: 3.10.0-1160.15.2.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff9affcd92
Build id: 3.10.0-1160.15.2.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff9b594f92
Build id: 3.10.0-1160.15.2.el7.x86_64
0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff9b594ed5
Build id: 3.10.0-1160.15.2.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [sisidsdaemon:2946]
Modules linked in: binfmt_misc falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_10912(E) iptable_filter xt_DSCP iptable_mangle vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev joydev vmw_balloon pcspkr sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi ata_piix scsi_transport_spi nfit mptscsih libata serio_raw mptbase libnvdimm vmxnet3 drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 2946 Comm: sisidsdaemon Kdump: loaded Tainted: P            E  ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff8be6133cb180 ti: ffff8be535d70000 task.ti: ffff8be535d70000
RIP: 0010:[<ffffffff9af167a2>]  [<ffffffff9af167a2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8be535d73c20  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff8be6133cb180 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000286
RBP: ffff8be535d73c70 R08: ffffffff9b6137c0 R09: ffffffff9b002798
R10: ffff8bf4fef1f0a0 R11: fffffdedc48cf040 R12: 00000000000280da
R13: ffffffff9b03e502 R14: ffff8be535d73bd8 R15: 0000000000000000
FS:  00007f670f7fe700(0000) GS:ffff8bf4fef00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6724e95000 CR3: 000000013278a000 CR4: 00000000000407e0
Call Trace:
[<ffffffff9ae7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9ae7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9ae7e6a0>] ? leave_mm+0x120/0x120
[<ffffffff9af168bf>] smp_call_function_single+0x5f/0xa0
[<ffffffff9b1862c5>] ? cpumask_next_and+0x35/0x50
[<ffffffff9af16e6b>] smp_call_function_many+0x22b/0x270
[<ffffffff9ae7e868>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9ae7e8db>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9afede86>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9afef94f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9afefa73>] tlb_finish_mmu+0x23/0x30
[<ffffffff9aff9264>] unmap_region+0xf4/0x140
[<ffffffff9aff9821>] ? __vma_rb_erase+0x121/0x220
[<ffffffff9affb855>] do_munmap+0x2a5/0x480
[<ffffffff9affba95>] vm_munmap+0x65/0xb0
[<ffffffff9affcd92>] SyS_munmap+0x22/0x30
[<ffffffff9b594f92>] system_call_fastpath+0x25/0x2a
[<ffffffff9b594ed5>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 13 b5 9b 48 89 df e8 97 6c 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #72216
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810fcf5a
Build id: 3.10.0-693.21.1.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff810709d0
Build id: 3.10.0-693.21.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff810fd06f
Build id: 3.10.0-693.21.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813288a5
Build id: 3.10.0-693.21.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810fd61b
Build id: 3.10.0-693.21.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff81070b98
Build id: 3.10.0-693.21.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff81070c68
Build id: 3.10.0-693.21.1.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811b18c3
Build id: 3.10.0-693.21.1.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811b2be5
Build id: 3.10.0-693.21.1.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b9cb4
Build id: 3.10.0-693.21.1.el7.x86_64
0xf4
13
? handle_mm_fault
vmlinux 0xffffffff811b6781
Build id: 3.10.0-693.21.1.el7.x86_64
0x691
14
? __vma_rb_erase
vmlinux 0xffffffff811ba271
Build id: 3.10.0-693.21.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff811bc285
Build id: 3.10.0-693.21.1.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffff811bc4c5
Build id: 3.10.0-693.21.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff811bd5d2
Build id: 3.10.0-693.21.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff816c0715
Build id: 3.10.0-693.21.1.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [gnome-shell:2097]
Modules linked in: nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event sb_edac edac_core snd_ens1371 iosf_mbi crc32_pclmul snd_rawmidi ghash_clmulni_intel snd_ac97_codec aesni_intel ac97_bus lrw snd_seq gf128mul glue_helper ablk_helper snd_seq_device cryptd snd_pcm snd_timer ppdev snd vmw_balloon soundcore
joydev sg nfit pcspkr vmw_vmci i2c_piix4 shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata mptspi scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih e1000 crc32c_intel mptbase serio_raw i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 3 PID: 2097 Comm: gnome-shell Tainted: G             L ------------   3.10.0-693.21.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8800bb985ee0 ti: ffff8803fde94000 task.ti: ffff8803fde94000
RIP: 0010:[<ffffffff810fcf5a>]  [<ffffffff810fcf5a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8803fde97c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8803fde97c00 RCX: 0000000000000001
RDX: ffffffff816dc8f0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8803fde97c80 R08: ffffffff816dc8e8 R09: 00007f0bdb81c000
R10: ffff8803e8bc5de0 R11: 0000000000000206 R12: 0000000400000000
R13: 00041e7f00000001 R14: ffff8803fde97c00 R15: 0000000c00000001
FS:  00007f0c3da05ac0(0000) GS:ffff88042d6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f0bdb81b000 CR3: 0000000409720000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810709d0>] ? leave_mm+0x110/0x110
[<ffffffff810fd06f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813288a5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810fd61b>] smp_call_function_many+0x22b/0x270
[<ffffffff81070b98>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff81070c68>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811b18c3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811b2be5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b9cb4>] unmap_region+0xf4/0x140
[<ffffffff811b6781>] ? handle_mm_fault+0x691/0xfa0
[<ffffffff811ba271>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811bc285>] do_munmap+0x295/0x470
[<ffffffff811bc4c5>] vm_munmap+0x65/0xb0
[<ffffffff811bd5d2>] SyS_munmap+0x22/0x30
[<ffffffff816c0715>] system_call_fastpath+0x1c/0x21
Code: 80 95 01 00 48 89 de 48 03 14 c5 a0 10 b2 81 48 89 df e8 7a 03 24 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #67322
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb730779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffb7276e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb7276e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb7276e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb73078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb754a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb7307e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb7277018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb7277088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffb73c0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffb73c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffb73ca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffb73cabe1
Build id: 3.10.0-862.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffb73ccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
15
vm_munmap
vmlinux 0xffffffffb73cce35
Build id: 3.10.0-862.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffb73ce052
Build id: 3.10.0-862.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffb791f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
18
? system_call_after_swapgs
vmlinux 0xffffffffb791f721
Build id: 3.10.0-862.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [systemd-journal:521]
Modules linked in: nfring(OE) tcp_lp fuse xt_NFQUEUE nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack nf_log_ipv4 nf_log_common xt_LOG iptable_filter ip_tables nls_utf8 isofs tun devlink ip_set nfnetlink bridge stp llc sunrpc ppdev sg virtio_balloon joydev parport_pc parport pcspkr i2c_piix4 xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_net virtio_console virtio_blk virtio_scsi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata virtio_pci i2c_core virtio_ring serio_raw virtio floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nfring]
CPU: 1 PID: 521 Comm: systemd-journal Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-862.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.7.5-0-ge51488c-20140602_164612-nilsson.home.kraxel.org 04/01/2014
task: ffff8954716c8000 ti: ffff895275cb0000 task.ti: ffff895275cb0000
RIP: 0010:[<ffffffffb730779a>]  [<ffffffffb730779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff895275cb3c30  EFLAGS: 00000202
RAX: 00000000000000a0 RBX: ffff895275cb3c00 RCX: 0000000000000020
RDX: 00000000ffffffff RSI: 00000000000000a0 RDI: 0000000000000286
RBP: ffff895275cb3c78 R08: ffffffffb7a13b68 R09: ffffffffb73d35bc
R10: ffff895474e5ba40 R11: fffffe9a42aa96c0 R12: ffff895275cb3c38
R13: 00007fd00284b000 R14: fffffe9a48d7eaf0 R15: ffff8952eddff6c8
FS:  00007fd002831880(0000) GS:ffff895474e40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd00284b000 CR3: 0000000034746000 CR4: 00000000000006e0
Call Trace:
[<ffffffffb7276e50>] ? leave_mm+0x110/0x110
[<ffffffffb7276e50>] ? leave_mm+0x110/0x110
[<ffffffffb7276e50>] ? leave_mm+0x110/0x110
[<ffffffffb73078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb754a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffb7307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffb7277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb7277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb73c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffb73c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffb73ca624>] unmap_region+0xf4/0x140
[<ffffffffb73cabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb73ccbf5>] do_munmap+0x295/0x470
[<ffffffffb73cce35>] vm_munmap+0x65/0xb0
[<ffffffffb73ce052>] SyS_munmap+0x22/0x30
[<ffffffffb791f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffffb791f721>] ? system_call_after_swapgs+0xae/0x146
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 f3 b7 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #24609
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff89b16f62
Build id: 3.10.0-1127.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff89a7e730
Build id: 3.10.0-1127.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff89a7e730
Build id: 3.10.0-1127.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff89a7e730
Build id: 3.10.0-1127.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff89b1707f
Build id: 3.10.0-1127.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff89d84ee5
Build id: 3.10.0-1127.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff89b1762b
Build id: 3.10.0-1127.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff89a7e8f8
Build id: 3.10.0-1127.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff89a7e96b
Build id: 3.10.0-1127.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff89beddb6
Build id: 3.10.0-1127.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff89bef87f
Build id: 3.10.0-1127.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff89bef9a3
Build id: 3.10.0-1127.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff89bf91b4
Build id: 3.10.0-1127.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff89bf9771
Build id: 3.10.0-1127.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff89bfb7a5
Build id: 3.10.0-1127.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff89bfb9e5
Build id: 3.10.0-1127.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff89bfcc82
Build id: 3.10.0-1127.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff8a192ed2
Build id: 3.10.0-1127.el7.x86_64
0x25
19
? system_call_after_swapgs
vmlinux 0xffffffff8a192e15
Build id: 3.10.0-1127.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 22s! [gnome-shell:1703]
Modules linked in: tcp_lp vfat fat uas usb_storage fuse binfmt_misc xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc ipmi_ssif iTCO_wdt gpio_ich iTCO_vendor_support intel_powerclamp coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev lpc_ich hpilo hpwdt i7core_edac
ipmi_si sg ipmi_devintf ipmi_msghandler pcc_cpufreq acpi_power_meter ip_tables xfs libcrc32c radeon sd_mod crc_t10dif crct10dif_generic i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel hpsa serio_raw bnx2 drm_panel_orientation_quirks scsi_transport_sas dm_mirror dm_region_hash dm_log dm_mod
CPU: 13 PID: 1703 Comm: gnome-shell Kdump: loaded Tainted: G          I    ------------   3.10.0-1127.el7.x86_64 #1
Hardware name: HP ProLiant DL360 G7, BIOS P68 09/30/2010
task: ffff9dd73466b150 ti: ffff9dd875cac000 task.ti: ffff9dd875cac000
RIP: 0010:[<ffffffff89b16f62>]  [<ffffffff89b16f62>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff9dd875cafc20  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffffffff8a18d678 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000286
RBP: ffff9dd875cafc70 R08: ffffffff8a215180 R09: ffffffff89c02688
R10: ffff9dda1f79f0a0 R11: ffffda2b8c52bf80 R12: ffff9dd91d648640
R13: ffff9dd875cafc48 R14: 00007f43542ff000 R15: 0000000000000029
FS:  00007f43b3411a80(0000) GS:ffff9dda1f780000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f43542ff000 CR3: 000000031197e000 CR4: 00000000000207e0
Call Trace:
[<ffffffff89a7e730>] ? leave_mm+0x120/0x120
[<ffffffff89a7e730>] ? leave_mm+0x120/0x120
[<ffffffff89a7e730>] ? leave_mm+0x120/0x120
[<ffffffff89b1707f>] smp_call_function_single+0x5f/0xa0
[<ffffffff89d84ee5>] ? cpumask_next_and+0x35/0x50
[<ffffffff89b1762b>] smp_call_function_many+0x22b/0x270
[<ffffffff89a7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff89a7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff89beddb6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff89bef87f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff89bef9a3>] tlb_finish_mmu+0x23/0x30
[<ffffffff89bf91b4>] unmap_region+0xf4/0x140
[<ffffffff89bf9771>] ? __vma_rb_erase+0x121/0x220
[<ffffffff89bfb7a5>] do_munmap+0x2a5/0x480
[<ffffffff89bfb9e5>] vm_munmap+0x65/0xb0
[<ffffffff89bfcc82>] SyS_munmap+0x22/0x30
[<ffffffff8a192ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff8a192e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 e0 0f 75 8a 48 89 df e8 17 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #15936
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff895113ea
Build id: 3.10.0-957.21.3.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8947a4e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff8947a4e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff8947a4e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff895114ff
Build id: 3.10.0-957.21.3.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff89775cd5
Build id: 3.10.0-957.21.3.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff89511aab
Build id: 3.10.0-957.21.3.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8947a6a8
Build id: 3.10.0-957.21.3.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8947a718
Build id: 3.10.0-957.21.3.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff895e4687
Build id: 3.10.0-957.21.3.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff895e5f85
Build id: 3.10.0-957.21.3.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff895ef624
Build id: 3.10.0-957.21.3.el7.x86_64
0xf4
13
? do_sync_write
vmlinux 0xffffffff89640da0
Build id: 3.10.0-957.21.3.el7.x86_64
0x90
14
? __vma_rb_erase
vmlinux 0xffffffff895efbff
Build id: 3.10.0-957.21.3.el7.x86_64
0x13f
15
do_munmap
vmlinux 0xffffffff895f1c15
Build id: 3.10.0-957.21.3.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff895f1e55
Build id: 3.10.0-957.21.3.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff895f30e2
Build id: 3.10.0-957.21.3.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff89b75ddb
Build id: 3.10.0-957.21.3.el7.x86_64
0x22
19
? system_call_after_swapgs
vmlinux 0xffffffff89b75d21
Build id: 3.10.0-957.21.3.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 24s! [java:18975]
Modules linked in: tcp_diag udp_diag inet_diag rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ipt_REJECT nf_reject_ipv4 sunrpc sb_edac coretemp ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel vmw_balloon aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc parport vmw_vmci i2c_piix4 xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw ata_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm pata_acpi e1000 mptspi scsi_transport_spi mptscsih mptbase drm_panel_orientation_quirks ata_piix libata floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: nf_defrag_ipv4]
CPU: 0 PID: 18975 Comm: java Kdump: loaded Tainted: G             L ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/14/2014
task: ffff97c4dd3a8000 ti: ffff97c2447a8000 task.ti: ffff97c2447a8000
RIP: 0010:[<ffffffff895113ea>]  [<ffffffff895113ea>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff97c2447abc30  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffff97c2447abbc8 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff97c2447abc78 R08: ffffffff89c13640 R09: 00003ffffffff000
R10: ffff97c2668743b8 R11: 0000000000000206 R12: 0000000000000010
R13: ffffffff8945c403 R14: ffffffffffffff10 R15: 0000000000000282
FS:  00007f23b4f23700(0000) GS:ffff97c5eee00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fcf8d709500 CR3: 0000000363a8c000 CR4: 00000000000407f0
Call Trace:
[<ffffffff8947a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8947a4e0>] ? leave_mm+0x110/0x110
[<ffffffff8947a4e0>] ? leave_mm+0x110/0x110
[<ffffffff895114ff>] smp_call_function_single+0x5f/0xa0
[<ffffffff89775cd5>] ? cpumask_next_and+0x35/0x50
[<ffffffff89511aab>] smp_call_function_many+0x22b/0x270
[<ffffffff8947a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8947a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffff895e4687>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff895e5f85>] tlb_finish_mmu+0x55/0x60
[<ffffffff895ef624>] unmap_region+0xf4/0x140
[<ffffffff89640da0>] ? do_sync_write+0x90/0xe0
[<ffffffff895efbff>] ? __vma_rb_erase+0x13f/0x220
[<ffffffff895f1c15>] do_munmap+0x2a5/0x480
[<ffffffff895f1e55>] vm_munmap+0x65/0xb0
[<ffffffff895f30e2>] SyS_munmap+0x22/0x30
[<ffffffff89b75ddb>] system_call_fastpath+0x22/0x27
[<ffffffff89b75d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 bc 14 8a 48 89 df e8 4a b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #13727
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffa03111ba
Build id: 3.10.0-957.10.1.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffa027a400
Build id: 3.10.0-957.10.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffa027a400
Build id: 3.10.0-957.10.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffa027a400
Build id: 3.10.0-957.10.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffa03112cf
Build id: 3.10.0-957.10.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffa0575b35
Build id: 3.10.0-957.10.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffa031187b
Build id: 3.10.0-957.10.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffa027a5c8
Build id: 3.10.0-957.10.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffa027a638
Build id: 3.10.0-957.10.1.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffa03e4507
Build id: 3.10.0-957.10.1.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffa03e5e05
Build id: 3.10.0-957.10.1.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffa03ef4a4
Build id: 3.10.0-957.10.1.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffa03efa61
Build id: 3.10.0-957.10.1.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffa03f1a95
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a5
15
vm_munmap
vmlinux 0xffffffffa03f1cd5
Build id: 3.10.0-957.10.1.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffa03f2f62
Build id: 3.10.0-957.10.1.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffa0975ddb
Build id: 3.10.0-957.10.1.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 22s! [sidb2:96853]
Modules linked in: cvbf(POE) dm_snapshot dm_bufio binfmt_misc vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 ppdev iosf_mbi crc32_pclmul vmw_balloon ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg vmw_vmci i2c_piix4 nfit libnvdimm parport_pc parport ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw drm_kms_helper syscopyarea vmxnet3 sysfillrect sysimgblt fb_sys_fops ttm drm vmw_pvscsi ata_piix drm_panel_orientation_quirks libata floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: cvblk]
CPU: 5 PID: 96853 Comm: sidb2 Kdump: loaded Tainted: P           OEL ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff94b9e4aed140 ti: ffff94b92bb5c000 task.ti: ffff94b92bb5c000
RIP: 0010:[<ffffffffa03111ba>]  [<ffffffffa03111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff94b92bb5fc30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff94b92bb5fc00 RCX: 0000000000000001
RDX: ffffffffa0a12dd0 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff94b92bb5fc78 R08: ffffffffa0a12dc8 R09: ffffffffa03f8728
R10: ffff94b9ed75f0a0 R11: fffff52bc0d57140 R12: 0000000000000a93
R13: 0000000000000000 R14: ffffffffc0806dd6 R15: ffff94b92bb5fbf0
FS:  00007f8e56a9b700(0000) GS:ffff94b9ed740000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8e8f986000 CR3: 000000024f664000 CR4: 00000000007607e0
PKRU: 55555554
Call Trace:
[<ffffffffa027a400>] ? leave_mm+0x110/0x110
[<ffffffffa027a400>] ? leave_mm+0x110/0x110
[<ffffffffa027a400>] ? leave_mm+0x110/0x110
[<ffffffffa03112cf>] smp_call_function_single+0x5f/0xa0
[<ffffffffa0575b35>] ? cpumask_next_and+0x35/0x50
[<ffffffffa031187b>] smp_call_function_many+0x22b/0x270
[<ffffffffa027a5c8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffa027a638>] flush_tlb_mm_range+0x68/0x140
[<ffffffffa03e4507>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffa03e5e05>] tlb_finish_mmu+0x55/0x60
[<ffffffffa03ef4a4>] unmap_region+0xf4/0x140
[<ffffffffa03efa61>] ? __vma_rb_erase+0x121/0x220
[<ffffffffa03f1a95>] do_munmap+0x2a5/0x480
[<ffffffffa03f1cd5>] vm_munmap+0x65/0xb0
[<ffffffffa03f2f62>] SyS_munmap+0x22/0x30
[<ffffffffa0975ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 b9 f4 a0 48 89 df e8 ca b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #12869
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff88716f72
Build id: 3.10.0-1127.18.2.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8867e730
Build id: 3.10.0-1127.18.2.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff8867e730
Build id: 3.10.0-1127.18.2.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff8867e730
Build id: 3.10.0-1127.18.2.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff8871708f
Build id: 3.10.0-1127.18.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff88984fc5
Build id: 3.10.0-1127.18.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8871763b
Build id: 3.10.0-1127.18.2.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8867e8f8
Build id: 3.10.0-1127.18.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8867e96b
Build id: 3.10.0-1127.18.2.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff887ede36
Build id: 3.10.0-1127.18.2.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff887ef8ff
Build id: 3.10.0-1127.18.2.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff887efa23
Build id: 3.10.0-1127.18.2.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff887f9234
Build id: 3.10.0-1127.18.2.el7.x86_64
0xf4
14
? xfs_iunlock
xfs 0xffffffffc042e344
Build id: 3.10.0-1127.18.2.el7.x86_64
0xe4
15
? __vma_rb_erase
vmlinux 0xffffffff887f97f1
Build id: 3.10.0-1127.18.2.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffff887fb825
Build id: 3.10.0-1127.18.2.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffff887fba65
Build id: 3.10.0-1127.18.2.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffff887fcd02
Build id: 3.10.0-1127.18.2.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffff88d92ed2
Build id: 3.10.0-1127.18.2.el7.x86_64
0x25
20
? system_call_after_swapgs
vmlinux 0xffffffff88d92e15
Build id: 3.10.0-1127.18.2.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 22s! [ManagementAgent:1272]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock fuse sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer sg snd coretemp ppdev soundcore vmw_balloon vmw_vmci joydev pcspkr i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crc32c_intel nfit ata_piix mptspi e1000 libnvdimm scsi_transport_spi
mptscsih libata serio_raw mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 1272 Comm: ManagementAgent Kdump: loaded Tainted: G             L ------------   3.10.0-1127.18.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff9776b5dec1c0 ti: ffff977aacdbc000 task.ti: ffff977aacdbc000
RIP: 0010:[<ffffffff88716f72>]  [<ffffffff88716f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff977aacdbfc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff977aacdbfbf0 RCX: 0000000000000002
RDX: ffffffff88e13950 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff977aacdbfc70 R08: ffffffff88e13948 R09: ffffffff8880280c
R10: ffff977aad71f0a0 R11: ffffde4a509d4b80 R12: ffffffff88d8d975
R13: ffff977aacdbfbb8 R14: 00007fb7587a5000 R15: ffff977aa391b000
FS:  00007fb74ffff700(0000) GS:ffff977aad700000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fb7587a5000 CR3: 000000042cad4000 CR4: 00000000000007e0
Call Trace:
[<ffffffff8867e730>] ? leave_mm+0x120/0x120
[<ffffffff8867e730>] ? leave_mm+0x120/0x120
[<ffffffff8867e730>] ? leave_mm+0x120/0x120
[<ffffffff8871708f>] smp_call_function_single+0x5f/0xa0
[<ffffffff88984fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8871763b>] smp_call_function_many+0x22b/0x270
[<ffffffff8867e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8867e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff887ede36>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff887ef8ff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff887efa23>] tlb_finish_mmu+0x23/0x30
[<ffffffff887f9234>] unmap_region+0xf4/0x140
[<ffffffffc042e344>] ? xfs_iunlock+0xe4/0x120 [xfs]
[<ffffffff887f97f1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff887fb825>] do_munmap+0x2a5/0x480
[<ffffffff887fba65>] vm_munmap+0x65/0xb0
[<ffffffff887fcd02>] SyS_munmap+0x22/0x30
[<ffffffff88d92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff88d92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 a0 10 35 89 48 89 df e8 c7 51 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #12576
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffad30779e
Build id: 3.10.0-862.el7.x86_64
0xfe
2
? leave_mm
vmlinux 0xffffffffad276e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffad276e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffad276e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffad3078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffad54a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffad307e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffad277018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffad277088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffad3c0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffad3c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffad3ca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc033ebec
Build id: 3.10.0-862.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffffad3cabe1
Build id: 3.10.0-862.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffad3ccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffffad3cce35
Build id: 3.10.0-862.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffad3ce052
Build id: 3.10.0-862.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffad91f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
19
? system_call_after_swapgs
vmlinux 0xffffffffad91f721
Build id: 3.10.0-862.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 28s! [java:27030]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_REJECT nf_reject_ipv4 ipt_MASQUERADE tun nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 ebtable_filter nf_defrag_ipv4 ebtables nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntrack ip6table_filter ip6_tables br_netfilter iptable_filter bridge stp llc devlink vmw_vsock_vmci_transport vsock overlay(T) sunrpc coretemp kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon joydev pcspkr vmw_vmci sg i2c_piix4 floppy parport_pc parport shpchp ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi ata_piix scsi_transport_spi drm crct10dif_pclmul
crct10dif_common libata mptscsih crc32c_intel e1000 serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 27030 Comm: java Kdump: loaded Tainted: G             L ------------ T 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8d7e8be43f40 ti: ffff8d7f6dfb0000 task.ti: ffff8d7f6dfb0000
RIP: 0010:[<ffffffffad30779e>]  [<ffffffffad30779e>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff8d7f6dfb3c30  EFLAGS: 00000202
RAX: 0000000000000020 RBX: 0000000000001000 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff8d7f6dfb3c78 R08: ffffffffada14060 R09: ffffffffad3d35bc
R10: ffff8d861ee9ba40 R11: ffffe0e35f715780 R12: 0000000000000001
R13: ffff8d855c6f9f40 R14: ffffffffad91a925 R15: ffff8d7f6dfb3bb8
FS:  00007fc55a317700(0000) GS:ffff8d861ee80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc629c80000 CR3: 00000004aa324000 CR4: 00000000000007e0
Call Trace:
[<ffffffffad276e50>] ? leave_mm+0x110/0x110
[<ffffffffad276e50>] ? leave_mm+0x110/0x110
[<ffffffffad276e50>] ? leave_mm+0x110/0x110
[<ffffffffad3078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffad54a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffad307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffad277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffad277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffad3c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffad3c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffad3ca624>] unmap_region+0xf4/0x140
[<ffffffffc033ebec>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffffad3cabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffad3ccbf5>] do_munmap+0x295/0x470
[<ffffffffad3cce35>] vm_munmap+0x65/0xb0
[<ffffffffad3ce052>] SyS_munmap+0x22/0x30
[<ffffffffad91f7d5>] system_call_fastpath+0x1c/0x21
[<ffffffffad91f721>] ? system_call_after_swapgs+0xae/0x146
Code: 48 89 de 48 03 14 c5 a0 53 f3 ad 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 
Complete report #11517
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8d30aa2a
Build id: 3.10.0-862.14.4.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8d279c90
Build id: 3.10.0-862.14.4.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff8d279c90
Build id: 3.10.0-862.14.4.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff8d279c90
Build id: 3.10.0-862.14.4.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff8d30ab3f
Build id: 3.10.0-862.14.4.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8d54edb5
Build id: 3.10.0-862.14.4.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8d30b0eb
Build id: 3.10.0-862.14.4.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8d279e58
Build id: 3.10.0-862.14.4.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8d279ec8
Build id: 3.10.0-862.14.4.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff8d3c3487
Build id: 3.10.0-862.14.4.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff8d3c4d85
Build id: 3.10.0-862.14.4.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff8d3cdb74
Build id: 3.10.0-862.14.4.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff8d3ce131
Build id: 3.10.0-862.14.4.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff8d3d0145
Build id: 3.10.0-862.14.4.el7.x86_64
0x295
15
vm_munmap
vmlinux 0xffffffff8d3d0385
Build id: 3.10.0-862.14.4.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff8d3d15a2
Build id: 3.10.0-862.14.4.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff8d92579b
Build id: 3.10.0-862.14.4.el7.x86_64
0x22
18
? system_call_after_swapgs
vmlinux 0xffffffff8d9256e1
Build id: 3.10.0-862.14.4.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#13 stuck for 36s! [qtp1850180796-2:24078]
NMI watchdog: BUG: soft lockup - CPU#11 stuck for 21s! [nscd:1117]
Modules linked in: xt_time nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack tcp_diag inet_diag ebtable_filter ebtables kcare(OE) ip6table_filter devlink ip6_tables ipt_REJECT nf_reject_ipv4 xt_multiport iptable_filter xt_owner loop vmw_vsock_vmci_transport vsock xfs libcrc32c ppdev vmw_balloon edac_mce_amd joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables ext4 mbcache jbd2 sd_mod sr_mod crc_t10dif cdrom crct10dif_generic crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix mptspi scsi_transport_spi mptscsih libata serio_raw mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 11 PID: 1117 Comm: nscd Kdump: loaded Tainted: G           OEL ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015
task: ffff9af834c5dee0 ti: ffff9af8aba5c000 task.ti: ffff9af8aba5c000
RIP: 0010:[<ffffffff8d30aa2a>]  [<ffffffff8d30aa2a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9af8aba5fc30  EFLAGS: 00000202
RAX: 0000000000000020 RBX: ffffffff8d9208d5 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000282
RBP: ffff9af8aba5fc78 R08: ffffffff8da14920 R09: ffffffff8d3d6d68
R10: ffff9af8b491ba80 R11: ffffee1a81c99540 R12: ffffee1a86461800
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000002
FS:  00007f8990868700(0000) GS:ffff9af8b4900000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f89a42ba000 CR3: 00000000aa222000 CR4: 00000000000007e0
Call Trace:
[<ffffffff8d279c90>] ? leave_mm+0x110/0x110
[<ffffffff8d279c90>] ? leave_mm+0x110/0x110
[<ffffffff8d279c90>] ? leave_mm+0x110/0x110
[<ffffffff8d30ab3f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8d54edb5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8d30b0eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8d279e58>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8d279ec8>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8d3c3487>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff8d3c4d85>] tlb_finish_mmu+0x55/0x60
[<ffffffff8d3cdb74>] unmap_region+0xf4/0x140
[<ffffffff8d3ce131>] ? __vma_rb_erase+0x121/0x220
[<ffffffff8d3d0145>] do_munmap+0x295/0x470
[<ffffffff8d3d0385>] vm_munmap+0x65/0xb0
[<ffffffff8d3d15a2>] SyS_munmap+0x22/0x30
[<ffffffff8d92579b>] system_call_fastpath+0x22/0x27
[<ffffffff8d9256e1>] ? system_call_after_swapgs+0xae/0x146
Complete report #10895
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb930779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffffb9276e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb9276e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb9276e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb93078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb954a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb9307e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb9277018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb9277088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffffb93c0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffb93c1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffb93ca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? do_sync_write
vmlinux 0xffffffffb941a1f0
Build id: 3.10.0-862.el7.x86_64
0x90
14
? __vma_rb_erase
vmlinux 0xffffffffb93cabe1
Build id: 3.10.0-862.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffb93ccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
16
vm_munmap
vmlinux 0xffffffffb93cce35
Build id: 3.10.0-862.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffb93ce052
Build id: 3.10.0-862.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffb991f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [postgres:1229]
Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul ghash_clmulni_intel snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq ppdev snd_seq_device aesni_intel lrw gf128mul glue_helper snd_pcm ablk_helper cryptd vmw_balloon joydev snd_timer pcspkr snd sg soundcore i2c_piix4 nfit vmw_vmci shpchp libnvdimm parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common drm crc32c_intel mptspi ata_piix scsi_transport_spi mptscsih mptbase serio_raw
libata e1000 i2c_core
CPU: 3 PID: 1229 Comm: postgres Kdump: loaded Tainted: G             L ------------   3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8f3ff213eeb0 ti: ffff8f3fec294000 task.ti: ffff8f3fec294000
RIP: 0010:[<ffffffffb930779a>]  [<ffffffffb930779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8f3fec297c30  EFLAGS: 00000202
RAX: ffff8f3df62eb600 RBX: ffff8f3f73742500 RCX: ffff8f3df62eb610
RDX: ffff8f3ff5619680 RSI: ffff8f3fec297c30 RDI: ffff8f3fec297c30
RBP: ffff8f3fec297c78 R08: 0000000000000001 R09: 000000018040001a
R10: 00000000c238b301 R11: ffffda578808e2c0 R12: 000000000000025f
R13: 0000000000000000 R14: ffffffffc062fad6 R15: ffff8f3fec297bf0
FS:  00007f7b09b4b880(0000) GS:ffff8f3ff56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7b09b7f000 CR3: 0000000226796000 CR4: 00000000001607e0
Call Trace:
[<ffffffffb9276e50>] ? leave_mm+0x110/0x110
[<ffffffffb9276e50>] ? leave_mm+0x110/0x110
[<ffffffffb9276e50>] ? leave_mm+0x110/0x110
[<ffffffffb93078af>] smp_call_function_single+0x5f/0xa0
[<ffffffffb954a275>] ? cpumask_next_and+0x35/0x50
[<ffffffffb9307e5b>] smp_call_function_many+0x22b/0x270
[<ffffffffb9277018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb9277088>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb93c0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffffb93c1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffffb93ca624>] unmap_region+0xf4/0x140
[<ffffffffb941a1f0>] ? do_sync_write+0x90/0xe0
[<ffffffffb93cabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb93ccbf5>] do_munmap+0x295/0x470
[<ffffffffb93cce35>] vm_munmap+0x65/0xb0
[<ffffffffb93ce052>] SyS_munmap+0x22/0x30
[<ffffffffb991f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 f3 b9 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #9608
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff85d0779a
Build id: 3.10.0-862.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff85c76e50
Build id: 3.10.0-862.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff85c76e50
Build id: 3.10.0-862.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff85c76e50
Build id: 3.10.0-862.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff85d078af
Build id: 3.10.0-862.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff85f4a275
Build id: 3.10.0-862.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff85d07e5b
Build id: 3.10.0-862.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff85c77018
Build id: 3.10.0-862.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff85c77088
Build id: 3.10.0-862.el7.x86_64
0x68
10
tlb_flush_mmu.part.75
vmlinux 0xffffffff85dc0497
Build id: 3.10.0-862.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff85dc1c75
Build id: 3.10.0-862.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff85dca624
Build id: 3.10.0-862.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff85dcabe1
Build id: 3.10.0-862.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff85dccbf5
Build id: 3.10.0-862.el7.x86_64
0x295
15
vm_munmap
vmlinux 0xffffffff85dcce35
Build id: 3.10.0-862.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff85dce052
Build id: 3.10.0-862.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff8631f7d5
Build id: 3.10.0-862.el7.x86_64
0x1c
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [systemd:1]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ip6t_rpfilter ebtable_nat ebtable_broute ebtable_filter ebtables iptable_nat nf_nat_ipv4 iptable_mangle iptable_security iptable_raw ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6table_nat nf_nat_ipv6 nf_nat ip6table_mangle ip6table_security ip6table_raw ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 xt_conntrack nf_conntrack ip6table_filter ip6_tables nls_utf8 isofs fuse tun devlink ip_set nfnetlink bridge stp llc vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ac97_bus snd_seq lrw gf128mul snd_seq_device snd_pcm glue_helper ablk_helper snd_timer
cryptd snd soundcore sg ppdev pcspkr nfit joydev vmw_balloon libnvdimm parport_pc parport i2c_piix4 shpchp vmw_vmci xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi scsi_transport_spi libata mptscsih e1000 serio_raw mptbase i2c_core dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 3 PID: 1 Comm: systemd Kdump: loaded Not tainted 3.10.0-862.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8eafb7468000 ti: ffff8eafb7464000 task.ti: ffff8eafb7464000
RIP: 0010:[<ffffffff85d0779a>]  [<ffffffff85d0779a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8eafb7467c30  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff8eafb7467c00 RCX: 0000000000000003
RDX: ffffffff86413670 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff8eafb7467c78 R08: ffffffff86413668 R09: ffffffff85dd34c8
R10: ffff8eb3696dba40 R11: fffff16b4e6294c0 R12: ffffffff85da335e
R13: ffff8eafb7467ba8 R14: ffff8eb361f7cbd0 R15: fffff16b49d53f00
FS:  00007fac05df9940(0000) GS:ffff8eb3696c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fac05e16000 CR3: 0000000524b06000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85c76e50>] ? leave_mm+0x110/0x110
[<ffffffff85d078af>] smp_call_function_single+0x5f/0xa0
[<ffffffff85f4a275>] ? cpumask_next_and+0x35/0x50
[<ffffffff85d07e5b>] smp_call_function_many+0x22b/0x270
[<ffffffff85c77018>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff85c77088>] flush_tlb_mm_range+0x68/0x140
[<ffffffff85dc0497>] tlb_flush_mmu.part.75+0x37/0xe0
[<ffffffff85dc1c75>] tlb_finish_mmu+0x55/0x60
[<ffffffff85dca624>] unmap_region+0xf4/0x140
[<ffffffff85dcabe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff85dccbf5>] do_munmap+0x295/0x470
[<ffffffff85dcce35>] vm_munmap+0x65/0xb0
[<ffffffff85dce052>] SyS_munmap+0x22/0x30
[<ffffffff8631f7d5>] system_call_fastpath+0x1c/0x21
Code: 80 96 01 00 48 89 de 48 03 14 c5 a0 53 93 86 48 89 df e8 4a 87 25 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #9215
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff9db14a22
Build id: 3.10.0-1062.1.2.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff9da7d1d0
Build id: 3.10.0-1062.1.2.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff9da7d1d0
Build id: 3.10.0-1062.1.2.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff9da7d1d0
Build id: 3.10.0-1062.1.2.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff9db14b3f
Build id: 3.10.0-1062.1.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff9dd7f475
Build id: 3.10.0-1062.1.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff9db150eb
Build id: 3.10.0-1062.1.2.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff9da7d398
Build id: 3.10.0-1062.1.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff9da7d40b
Build id: 3.10.0-1062.1.2.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff9dbea166
Build id: 3.10.0-1062.1.2.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff9dbebc2f
Build id: 3.10.0-1062.1.2.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff9dbebd53
Build id: 3.10.0-1062.1.2.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff9dbf5544
Build id: 3.10.0-1062.1.2.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffff9dbf5b01
Build id: 3.10.0-1062.1.2.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff9dbf7b45
Build id: 3.10.0-1062.1.2.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff9dbf7d85
Build id: 3.10.0-1062.1.2.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff9dbf9022
Build id: 3.10.0-1062.1.2.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff9e18cede
Build id: 3.10.0-1062.1.2.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 172s! [systemd:1]
Modules linked in: xt_nat veth nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter overlay(T) xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink binfmt_misc vmw_vsock_vmci_transport vsock sunrpc snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm coretemp snd_timer snd iosf_mbi crc32_pclmul ppdev soundcore ghash_clmulni_intel aesni_intel vmw_balloon lrw gf128mul glue_helper ablk_helper joydev sg cryptd pcspkr parport_pc parport vmw_vmci i2c_piix4 sch_fq_codel ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif
crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptspi ata_piix scsi_transport_spi crct10dif_pclmul crct10dif_common mptscsih libata crc32c_intel serio_raw e1000 drm_panel_orientation_quirks mptbase nfit libnvdimm dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ipmi_msghandler]
CPU: 3 PID: 1 Comm: systemd Kdump: loaded Tainted: G        W    L ------------ T 3.10.0-1062.1.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
task: ffff991efcfc0000 ti: ffff991efcf4c000 task.ti: ffff991efcf4c000
RIP: 0010:[<ffffffff9db14a22>]  [<ffffffff9db14a22>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff991efcf4fc20  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff991efcf4fbf0 RCX: 0000000000000001
RDX: ffffffff9e213550 RSI: 0000000000000080 RDI: 0000000000000282
RBP: ffff991efcf4fc70 R08: ffffffff9e213548 R09: ffffffff9dbfe7e8
R10: ffff991fb56df0a0 R11: fffff3498778e9c0 R12: ffff991efcf4fbb8
R13: ffffffff9dbcb46e R14: ffff991efcf4fb90 R15: ffff991fb0758ca8
FS:  00007f9b733a1940(0000) GS:ffff991fb56c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9b733c9000 CR3: 0000000035c66000 CR4: 00000000001607e0
Call Trace:
[<ffffffff9da7d1d0>] ? leave_mm+0x110/0x110
[<ffffffff9da7d1d0>] ? leave_mm+0x110/0x110
[<ffffffff9da7d1d0>] ? leave_mm+0x110/0x110
[<ffffffff9db14b3f>] smp_call_function_single+0x5f/0xa0
[<ffffffff9dd7f475>] ? cpumask_next_and+0x35/0x50
[<ffffffff9db150eb>] smp_call_function_many+0x22b/0x270
[<ffffffff9da7d398>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff9da7d40b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff9dbea166>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff9dbebc2f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff9dbebd53>] tlb_finish_mmu+0x23/0x30
[<ffffffff9dbf5544>] unmap_region+0xf4/0x140
[<ffffffff9dbf5b01>] ? __vma_rb_erase+0x121/0x220
[<ffffffff9dbf7b45>] do_munmap+0x2a5/0x480
[<ffffffff9dbf7d85>] vm_munmap+0x65/0xb0
[<ffffffff9dbf9022>] SyS_munmap+0x22/0x30
[<ffffffff9e18cede>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 e0 d7 74 9e 48 89 df e8 d7 1a 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #8367
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8d91109a
Build id: 3.10.0-957.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8d87a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff8d87a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff8d87a3d0
Build id: 3.10.0-957.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff8d9111af
Build id: 3.10.0-957.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8db75465
Build id: 3.10.0-957.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8d91175b
Build id: 3.10.0-957.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8d87a598
Build id: 3.10.0-957.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8d87a608
Build id: 3.10.0-957.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff8d9e4077
Build id: 3.10.0-957.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff8d9e5975
Build id: 3.10.0-957.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff8d9ef014
Build id: 3.10.0-957.el7.x86_64
0xf4
13
? xfs_iunlock
xfs 0xffffffffc03538bc
Build id: 3.10.0-957.el7.x86_64
0xec
14
? __vma_rb_erase
vmlinux 0xffffffff8d9ef5d1
Build id: 3.10.0-957.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffff8d9f1605
Build id: 3.10.0-957.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffff8d9f1845
Build id: 3.10.0-957.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffff8d9f2ad2
Build id: 3.10.0-957.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffff8df74ddb
Build id: 3.10.0-957.el7.x86_64
0x22
19
? system_call_after_swapgs
vmlinux 0xffffffff8df74d21
Build id: 3.10.0-957.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [packagekitd:7272]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack iptable_filter sunrpc ppdev joydev pcspkr virtio_balloon parport_pc parport sg i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_scsi virtio_net cirrus serio_raw drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm floppy ata_piix drm libata drm_panel_orientation_quirks virtio_pci virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 7272 Comm: packagekitd Kdump: loaded Tainted: G             L ------------   3.10.0-957.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.1-0-g0551a4be2c-prebuilt.qemu-project.org 04/01/2014
task: ffff9e8b4f3cb0c0 ti: ffff9e8d95a78000 task.ti: ffff9e8d95a78000
RIP: 0010:[<ffffffff8d91109a>]  [<ffffffff8d91109a>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff9e8d95a7bc30  EFLAGS: 00000202
RAX: 0000000000000030 RBX: ffff9e8d95a7bc00 RCX: 0000000000000010
RDX: 0000ffffffffffff RSI: 0000000000000030 RDI: 0000000000000286
RBP: ffff9e8d95a7bc78 R08: ffffffff8e013c80 R09: ffffffff8d9f838c
R10: ffff9e8daeb1f0a0 R11: ffffe8c98ffdd180 R12: ffffffff8df6ef42
R13: ffff9e8d95a7bbb8 R14: 00007f4c0b94b000 R15: ffff9e8da9824000
FS:  00007f4c0b9348c0(0000) GS:ffff9e8daeb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4c0b94b000 CR3: 00000000b941a000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8d87a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8d87a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8d87a3d0>] ? leave_mm+0x110/0x110
[<ffffffff8d9111af>] smp_call_function_single+0x5f/0xa0
[<ffffffff8db75465>] ? cpumask_next_and+0x35/0x50
[<ffffffff8d91175b>] smp_call_function_many+0x22b/0x270
[<ffffffff8d87a598>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8d87a608>] flush_tlb_mm_range+0x68/0x140
[<ffffffff8d9e4077>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffff8d9e5975>] tlb_finish_mmu+0x55/0x60
[<ffffffff8d9ef014>] unmap_region+0xf4/0x140
[<ffffffffc03538bc>] ? xfs_iunlock+0xec/0x130 [xfs]
[<ffffffff8d9ef5d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff8d9f1605>] do_munmap+0x2a5/0x480
[<ffffffff8d9f1845>] vm_munmap+0x65/0xb0
[<ffffffff8d9f2ad2>] SyS_munmap+0x22/0x30
[<ffffffff8df74ddb>] system_call_fastpath+0x22/0x27
[<ffffffff8df74d21>] ? system_call_after_swapgs+0xae/0x146
Code: 00 b7 01 00 48 89 de 48 03 14 c5 60 b9 54 8e 48 89 df e8 1a b2 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #6309
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f98ea
Build id: 3.10.0-514.26.2.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106c430
Build id: 3.10.0-514.26.2.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106c430
Build id: 3.10.0-514.26.2.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106c430
Build id: 3.10.0-514.26.2.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f99ef
Build id: 3.10.0-514.26.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813183b5
Build id: 3.10.0-514.26.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f9f93
Build id: 3.10.0-514.26.2.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106c5f8
Build id: 3.10.0-514.26.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106c6c9
Build id: 3.10.0-514.26.2.el7.x86_64
0x69
10
tlb_flush_mmu.part.61
vmlinux 0xffffffff811ac733
Build id: 3.10.0-514.26.2.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ada25
Build id: 3.10.0-514.26.2.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b4944
Build id: 3.10.0-514.26.2.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff811b4ec1
Build id: 3.10.0-514.26.2.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff811b6e75
Build id: 3.10.0-514.26.2.el7.x86_64
0x245
15
vm_munmap
vmlinux 0xffffffff811b7091
Build id: 3.10.0-514.26.2.el7.x86_64
0x41
16
SyS_munmap
vmlinux 0xffffffff811b8042
Build id: 3.10.0-514.26.2.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff81697809
Build id: 3.10.0-514.26.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [beam.smp:2793]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 iptable_filter xt_conntrack iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle ip_tables ip_set nfnetlink binfmt_misc tun bridge stp llc vmw_vsock_vmci_transport vsock snd_seq_midi snd_seq_midi_event snd_ens1371 snd_rawmidi snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer intel_powerclamp snd coretemp ppdev soundcore sg pcspkr vmw_balloon parport_pc parport shpchp i2c_piix4 vmw_vmci nfsd auth_rpcgss nfs_acl lockd grace sunrpc xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif ata_generic crct10dif_generic crct10dif_common pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops crc32c_intel ttm e1000 drm ata_piix mptspi
serio_raw scsi_transport_spi mptscsih mptbase libata i2c_core floppy fjes dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 1 PID: 2793 Comm: beam.smp Tainted: G             L ------------   3.10.0-514.26.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff880095ac8fb0 ti: ffff880095b00000 task.ti: ffff880095b00000
RIP: 0010:[<ffffffff810f98ea>]  [<ffffffff810f98ea>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff880095b03c80  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffff8800b8aedd00 RCX: 0000000000000001
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000286
RBP: ffff880095b03cd0 R08: ffffffff816b0a68 R09: 000000018040003d
R10: 00000000afa67f01 R11: ffffea0002be99c0 R12: 00000000fffffff5
R13: ffffffff8155a588 R14: ffff880095b03c10 R15: ffffffff8168ea3b
FS:  00007efe2333e700(0000) GS:ffff8800bb440000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efdfd58c068 CR3: 00000000ab787000 CR4: 00000000000007e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106c430 ffff880095b03d58
0000000000000003 00000000b110864f ffff8800917cbc80 0000000000000000
ffffffff8106c430 ffffffff8106c430 ffff880095b03d00 ffffffff810f99ef
Call Trace:
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff8106c430>] ? leave_mm+0x70/0x70
[<ffffffff810f99ef>] smp_call_function_single+0x5f/0xa0
[<ffffffff813183b5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f9f93>] smp_call_function_many+0x223/0x260
[<ffffffff8106c5f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106c6c9>] flush_tlb_mm_range+0x69/0x140
[<ffffffff811ac733>] tlb_flush_mmu.part.61+0x33/0xc0
[<ffffffff811ada25>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b4944>] unmap_region+0xf4/0x140
[<ffffffff811b4ec1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b6e75>] do_munmap+0x245/0x420
[<ffffffff811b7091>] vm_munmap+0x41/0x60
[<ffffffff811b8042>] SyS_munmap+0x22/0x30
[<ffffffff81697809>] system_call_fastpath+0x16/0x1b
Code: 80 77 01 00 48 89 de 48 03 14 c5 80 99 ad 81 48 89 df e8 9a 2c 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #4724
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb4516f72
Build id: 3.10.0-1127.19.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffffb447e730
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffffb447e730
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffffb447e730
Build id: 3.10.0-1127.19.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffffb451708f
Build id: 3.10.0-1127.19.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb4785005
Build id: 3.10.0-1127.19.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb451763b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb447e8f8
Build id: 3.10.0-1127.19.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb447e96b
Build id: 3.10.0-1127.19.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffb45ede36
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffb45ef8ff
Build id: 3.10.0-1127.19.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffb45efa23
Build id: 3.10.0-1127.19.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffb45f9234
Build id: 3.10.0-1127.19.1.el7.x86_64
0xf4
14
? xfs_iunlock
xfs 0xffffffffc034f344
Build id: 3.10.0-1127.19.1.el7.x86_64
0xe4
15
? __vma_rb_erase
vmlinux 0xffffffffb45f97f1
Build id: 3.10.0-1127.19.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffffb45fb825
Build id: 3.10.0-1127.19.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffffb45fba65
Build id: 3.10.0-1127.19.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffffb45fcd02
Build id: 3.10.0-1127.19.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffffb4b92ed2
Build id: 3.10.0-1127.19.1.el7.x86_64
0x25
20
? system_call_after_swapgs
vmlinux 0xffffffffb4b92e15
Build id: 3.10.0-1127.19.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [psvc:18101]
Modules linked in: vmfsrtx_export(POE) vmfsrtx_bridge(POE) vmfsrtx_bridge_helper(OE) vmfsrtx_dev(POE) vmfsrtx_dev_helper(POE) xt_CHECKSUM ipt_MASQUERADE fuse nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon sg pcspkr joydev parport_pc parport vmw_vmci i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx nfit libnvdimm crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm serio_raw ata_piix libata mptspi scsi_transport_spi mptscsih mptbase vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod [last unloaded: vmfsrtx_export]
CPU: 0 PID: 18101 Comm: psvc Kdump: loaded Tainted: P           OEL ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
task: ffff941cbbc262a0 ti: ffff941c8caec000 task.ti: ffff941c8caec000
RIP: 0010:[<ffffffffb4516f72>]  [<ffffffffb4516f72>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff941c8caefc20  EFLAGS: 00000202
RAX: 0000000000000020 RBX: 0000000000000b71 RCX: 0000000000000020
RDX: 0000000000000020 RSI: 0000000000000020 RDI: 0000000000000286
RBP: ffff941c8caefc70 R08: ffffffffb4c13940 R09: ffffffffb460280c
R10: ffff941d7ae1f0a0 R11: ffffc16401be4200 R12: ffff941cf67af428
R13: ffffffffc0365021 R14: ffff941c8caefbf0 R15: 0000000000000000
FS:  00007ff6ff964700(0000) GS:ffff941d7ae00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff75084f000 CR3: 000000007fdd6000 CR4: 00000000000007f0
Call Trace:
[<ffffffffb447e730>] ? leave_mm+0x120/0x120
[<ffffffffb447e730>] ? leave_mm+0x120/0x120
[<ffffffffb447e730>] ? leave_mm+0x120/0x120
[<ffffffffb451708f>] smp_call_function_single+0x5f/0xa0
[<ffffffffb4785005>] ? cpumask_next_and+0x35/0x50
[<ffffffffb451763b>] smp_call_function_many+0x22b/0x270
[<ffffffffb447e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb447e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffb45ede36>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffb45ef8ff>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffb45efa23>] tlb_finish_mmu+0x23/0x30
[<ffffffffb45f9234>] unmap_region+0xf4/0x140
[<ffffffffc034f344>] ? xfs_iunlock+0xe4/0x120 [xfs]
[<ffffffffb45f97f1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb45fb825>] do_munmap+0x2a5/0x480
[<ffffffffb45fba65>] vm_munmap+0x65/0xb0
[<ffffffffb45fcd02>] SyS_munmap+0x22/0x30
[<ffffffffb4b92ed2>] system_call_fastpath+0x25/0x2a
[<ffffffffb4b92e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 a0 10 15 b5 48 89 df e8 07 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #3939
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff85f111ba
Build id: 3.10.0-957.10.1.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff85e7a400
Build id: 3.10.0-957.10.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffff85e7a400
Build id: 3.10.0-957.10.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffff85e7a400
Build id: 3.10.0-957.10.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffff85f112cf
Build id: 3.10.0-957.10.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff86175b35
Build id: 3.10.0-957.10.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff85f1187b
Build id: 3.10.0-957.10.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff85e7a5c8
Build id: 3.10.0-957.10.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff85e7a638
Build id: 3.10.0-957.10.1.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffff85fe4507
Build id: 3.10.0-957.10.1.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffff85fe5e05
Build id: 3.10.0-957.10.1.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff85fef4a4
Build id: 3.10.0-957.10.1.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff85fefa61
Build id: 3.10.0-957.10.1.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff85ff1a95
Build id: 3.10.0-957.10.1.el7.x86_64
0x2a5
15
vm_munmap
vmlinux 0xffffffff85ff1cd5
Build id: 3.10.0-957.10.1.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff85ff2f62
Build id: 3.10.0-957.10.1.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff86575ddb
Build id: 3.10.0-957.10.1.el7.x86_64
0x22
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [systemd-journal:4394]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc i2c_piix4 parport vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw
 ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 4394 Comm: systemd-journal Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8a774cf9a080 ti: ffff8a774d6b0000 task.ti: ffff8a774d6b0000
RIP: 0010:[<ffffffff85f111ba>]  [<ffffffff85f111ba>] generic_exec_single+0xfa/0x1b0
RSP: 0018:ffff8a774d6b3c30  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8a774d6b3bc0 RCX: 000000000000003c
RDX: 000000000000000f RSI: 0000000000000004 RDI: 0000000000000286
RBP: ffff8a774d6b3c78 R08: ffffffff866132c0 R09: ffffffff85ff881c
R10: ffff8a77cfc1f0a0 R11: ffffb82f3984b400 R12: 0000000000000010
R13: ffffffff85e611c6 R14: ffffffffffffffc0 R15: 0000000000000286
FS:  00007f494534b880(0000) GS:ffff8a77cfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f494535c000 CR3: 00000001fa7ba000 CR4: 00000000003607f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 [<ffffffff85e7a400>] ? leave_mm+0x110/0x110
 [<ffffffff85e7a400>] ? leave_mm+0x110/0x110
 [<ffffffff85e7a400>] ? leave_mm+0x110/0x110
 [<ffffffff85f112cf>] smp_call_function_single+0x5f/0xa0
 [<ffffffff86175b35>] ? cpumask_next_and+0x35/0x50
 [<ffffffff85f1187b>] smp_call_function_many+0x22b/0x270
 [<ffffffff85e7a5c8>] native_flush_tlb_others+0xb8/0xc0
 [<ffffffff85e7a638>] flush_tlb_mm_range+0x68/0x140
 [<ffffffff85fe4507>] tlb_flush_mmu.part.76+0x37/0xe0
 [<ffffffff85fe5e05>] tlb_finish_mmu+0x55/0x60
 [<ffffffff85fef4a4>] unmap_region+0xf4/0x140
 [<ffffffff85fefa61>] ? __vma_rb_erase+0x121/0x220
 [<ffffffff85ff1a95>] do_munmap+0x2a5/0x480
 [<ffffffff85ff1cd5>] vm_munmap+0x65/0xb0
 [<ffffffff85ff2f62>] SyS_munmap+0x22/0x30
 [<ffffffff86575ddb>] system_call_fastpath+0x22/0x27
Code: 00 b7 01 00 48 89 de 48 03 14 c5 a0 b9 b4 86 48 89 df e8 ca b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #1603
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffb09113ee
Build id: 3.10.0-957.21.3.el7.x86_64
0xfe
2
? leave_mm
vmlinux 0xffffffffb087a4e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffb087a4e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffb087a4e0
Build id: 3.10.0-957.21.3.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffb09114ff
Build id: 3.10.0-957.21.3.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffb0b75cd5
Build id: 3.10.0-957.21.3.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffb0911aab
Build id: 3.10.0-957.21.3.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffb087a6a8
Build id: 3.10.0-957.21.3.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffb087a718
Build id: 3.10.0-957.21.3.el7.x86_64
0x68
10
tlb_flush_mmu.part.76
vmlinux 0xffffffffb09e4687
Build id: 3.10.0-957.21.3.el7.x86_64
0x37
11
tlb_finish_mmu
vmlinux 0xffffffffb09e5f85
Build id: 3.10.0-957.21.3.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffffb09ef624
Build id: 3.10.0-957.21.3.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffffb09efbe1
Build id: 3.10.0-957.21.3.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffffb09f1c15
Build id: 3.10.0-957.21.3.el7.x86_64
0x2a5
15
vm_munmap
vmlinux 0xffffffffb09f1e55
Build id: 3.10.0-957.21.3.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffffb09f30e2
Build id: 3.10.0-957.21.3.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffffb0f75ddb
Build id: 3.10.0-957.21.3.el7.x86_64
0x22
18
? system_call_after_swapgs
vmlinux 0xffffffffb0f75d21
Build id: 3.10.0-957.21.3.el7.x86_64
0xae
NMI watchdog: BUG: soft lockup - CPU#7 stuck for 67s! [postmaster:11792]
Modules linked in: tcp_diag inet_diag vmw_vsock_vmci_transport vsock coretemp crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd ppdev sg pcspkr joydev vmw_balloon parport_pc parport vmw_vmci i2c_piix4 nfsd nfs_acl lockd grace auth_rpcgss sunrpc ip_tables xfs libcrc32c vmwgfx drm_kms_helper sr_mod cdrom syscopyarea ata_generic sysfillrect pata_acpi sysimgblt fb_sys_fops ttm sd_mod crc_t10dif crct10dif_generic drm ata_piix crct10dif_pclmul libata crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 7 PID: 11792 Comm: postmaster Kdump: loaded Tainted: G             L ------------   3.10.0-957.21.3.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 10/22/2013
task: ffff8f50fb10d140 ti: ffff8f50f6674000 task.ti: ffff8f50f6674000
RIP: 0010:[<ffffffffb09113ee>]  [<ffffffffb09113ee>] generic_exec_single+0xfe/0x1b0
RSP: 0018:ffff8f50f6677c30  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffff8f50fb10d140 RCX: 0000000000000007
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000282
RBP: ffff8f50f6677c78 R08: ffffffffb1013dc8 R09: ffffffffb09f88a8
R10: ffff8f546e7df0a0 R11: ffffe318c532a240 R12: ffff8f50f6677cc8
R13: 0000000000000000 R14: ffff8f50fb10d140 R15: ffff8f547ffcf000
FS:  00007fba960ae8c0(0000) GS:ffff8f546e7c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fba960f5000 CR3: 00000000b815a000 CR4: 00000000000007e0
Call Trace:
[<ffffffffb087a4e0>] ? leave_mm+0x110/0x110
[<ffffffffb087a4e0>] ? leave_mm+0x110/0x110
[<ffffffffb087a4e0>] ? leave_mm+0x110/0x110
[<ffffffffb09114ff>] smp_call_function_single+0x5f/0xa0
[<ffffffffb0b75cd5>] ? cpumask_next_and+0x35/0x50
[<ffffffffb0911aab>] smp_call_function_many+0x22b/0x270
[<ffffffffb087a6a8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffb087a718>] flush_tlb_mm_range+0x68/0x140
[<ffffffffb09e4687>] tlb_flush_mmu.part.76+0x37/0xe0
[<ffffffffb09e5f85>] tlb_finish_mmu+0x55/0x60
[<ffffffffb09ef624>] unmap_region+0xf4/0x140
[<ffffffffb09efbe1>] ? __vma_rb_erase+0x121/0x220
[<ffffffffb09f1c15>] do_munmap+0x2a5/0x480
[<ffffffffb09f1e55>] vm_munmap+0x65/0xb0
[<ffffffffb09f30e2>] SyS_munmap+0x22/0x30
[<ffffffffb0f75ddb>] system_call_fastpath+0x22/0x27
[<ffffffffb0f75d21>] ? system_call_after_swapgs+0xae/0x146
Code: 48 89 de 48 03 14 c5 a0 bc 54 b1 48 89 df e8 4a b7 27 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 87 
Complete report #235
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffffabb16616
Build id: 3.10.0-1062.18.1.el7.x86_64
0x106
2
? leave_mm
vmlinux 0xffffffffaba7e440
Build id: 3.10.0-1062.18.1.el7.x86_64
0x110
3
? leave_mm
vmlinux 0xffffffffaba7e440
Build id: 3.10.0-1062.18.1.el7.x86_64
0x110
4
? leave_mm
vmlinux 0xffffffffaba7e440
Build id: 3.10.0-1062.18.1.el7.x86_64
0x110
5
smp_call_function_single
vmlinux 0xffffffffabb1672f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffffabd81845
Build id: 3.10.0-1062.18.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffffabb16cdb
Build id: 3.10.0-1062.18.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffffaba7e608
Build id: 3.10.0-1062.18.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffffaba7e67b
Build id: 3.10.0-1062.18.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffffabbebc86
Build id: 3.10.0-1062.18.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffffabbed74f
Build id: 3.10.0-1062.18.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffffabbed873
Build id: 3.10.0-1062.18.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffffabbf7084
Build id: 3.10.0-1062.18.1.el7.x86_64
0xf4
14
? __vma_rb_erase
vmlinux 0xffffffffabbf7641
Build id: 3.10.0-1062.18.1.el7.x86_64
0x121
15
do_munmap
vmlinux 0xffffffffabbf9655
Build id: 3.10.0-1062.18.1.el7.x86_64
0x2a5
16
vm_munmap
vmlinux 0xffffffffabbf9895
Build id: 3.10.0-1062.18.1.el7.x86_64
0x65
17
SyS_munmap
vmlinux 0xffffffffabbfab32
Build id: 3.10.0-1062.18.1.el7.x86_64
0x22
18
system_call_fastpath
vmlinux 0xffffffffac18dede
Build id: 3.10.0-1062.18.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [reader#1:2258]
Modules linked in: vsock_diag unix_diag af_packet_diag netlink_diag udp_diag tcp_diag inet_diag ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc i2c_piix4 vmw_vmci parport CyProtectDrv(OE) ip_tables ext4 mbcache jbd2 sr_mod cdrom
sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ata_piix drm libata mptspi scsi_transport_spi mptscsih crct10dif_pclmul crct10dif_common mptbase nfit crc32c_intel serio_raw libnvdimm vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 2258 Comm: reader#1 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1062.18.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff99bdda37a0e0 ti: ffff99bdc96c8000 task.ti: ffff99bdc96c8000
RIP: 0010:[<ffffffffabb16616>]  [<ffffffffabb16616>] generic_exec_single+0x106/0x1c0
RSP: 0018:ffff99bdc96cbc20  EFLAGS: 00000202
RAX: 0000000000000008 RBX: ffff99bdc96cbbf0 RCX: 0000000000000038
RDX: 00000000000000ff RSI: 0000000000000008 RDI: 0000000000000282
RBP: ffff99bdc96cbc70 R08: ffffffffac213f40 R09: ffffffffabc004c8
R10: ffff99bdffc9f0a0 R11: fffff361202a0340 R12: 00000000000280da
R13: ffffffffabc3c032 R14: ffff99bdc96cbbd8 R15: 0000000000000000
FS:  00007f3fbafa0700(0000) GS:ffff99bdffc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f3fc5686000 CR3: 00000008185d8000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaba7e440>] ? leave_mm+0x110/0x110
[<ffffffffaba7e440>] ? leave_mm+0x110/0x110
[<ffffffffaba7e440>] ? leave_mm+0x110/0x110
[<ffffffffabb1672f>] smp_call_function_single+0x5f/0xa0
[<ffffffffabd81845>] ? cpumask_next_and+0x35/0x50
[<ffffffffabb16cdb>] smp_call_function_many+0x22b/0x270
[<ffffffffaba7e608>] native_flush_tlb_others+0xb8/0xc0
[<ffffffffaba7e67b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffffabbebc86>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffffabbed74f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffffabbed873>] tlb_finish_mmu+0x23/0x30
[<ffffffffabbf7084>] unmap_region+0xf4/0x140
[<ffffffffabbf7641>] ? __vma_rb_erase+0x121/0x220
[<ffffffffabbf9655>] do_munmap+0x2a5/0x480
[<ffffffffabbf9895>] vm_munmap+0x65/0xb0
[<ffffffffabbfab32>] SyS_munmap+0x22/0x30
[<ffffffffac18dede>] system_call_fastpath+0x25/0x2a
Code: 14 c5 60 e8 74 ac 48 89 df e8 b7 22 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 f6 43 20 01 <75> f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 00 0f 85 8f 
Complete report #216
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff810f933a
Build id: 3.10.0-693.5.2.el7.x86_64
0xfa
2
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.5.2.el7.x86_64
0x70
3
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.5.2.el7.x86_64
0x70
4
? leave_mm
vmlinux 0xffffffff8106e4a0
Build id: 3.10.0-693.5.2.el7.x86_64
0x70
5
smp_call_function_single
vmlinux 0xffffffff810f943f
Build id: 3.10.0-693.5.2.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff813227c5
Build id: 3.10.0-693.5.2.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff810f99d3
Build id: 3.10.0-693.5.2.el7.x86_64
0x223
8
native_flush_tlb_others
vmlinux 0xffffffff8106e668
Build id: 3.10.0-693.5.2.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8106e738
Build id: 3.10.0-693.5.2.el7.x86_64
0x68
10
tlb_flush_mmu.part.63
vmlinux 0xffffffff811acfc3
Build id: 3.10.0-693.5.2.el7.x86_64
0x33
11
tlb_finish_mmu
vmlinux 0xffffffff811ae2e5
Build id: 3.10.0-693.5.2.el7.x86_64
0x55
12
unmap_region
vmlinux 0xffffffff811b53b4
Build id: 3.10.0-693.5.2.el7.x86_64
0xf4
13
? __vma_rb_erase
vmlinux 0xffffffff811b5971
Build id: 3.10.0-693.5.2.el7.x86_64
0x121
14
do_munmap
vmlinux 0xffffffff811b7972
Build id: 3.10.0-693.5.2.el7.x86_64
0x292
15
vm_munmap
vmlinux 0xffffffff811b7bb5
Build id: 3.10.0-693.5.2.el7.x86_64
0x65
16
SyS_munmap
vmlinux 0xffffffff811b8cc2
Build id: 3.10.0-693.5.2.el7.x86_64
0x22
17
system_call_fastpath
vmlinux 0xffffffff816b5089
Build id: 3.10.0-693.5.2.el7.x86_64
0x16
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [vmtoolsd:590]
Modules linked in: tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter xt_set ip_set nfnetlink ip_vs nf_conntrack vmw_vsock_vmci_transport vsock dm_mirror dm_region_hash dm_log dm_mod snd_seq_midi snd_seq_midi_event sb_edac edac_core iosf_mbi crc32_pclmul ghash_clmulni_intel ppdev aesni_intel snd_ens1371 lrw gf128mul glue_helper ablk_helper cryptd snd_rawmidi snd_ac97_codec vmw_balloon ac97_bus snd_seq snd_seq_device snd_pcm nfit snd_timer joydev pcspkr libnvdimm snd sg vmw_vmci soundcore i2c_piix4 shpchp parport_pc parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables
xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif crct10dif_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx drm_kms_helper serio_raw mptspi scsi_transport_spi mptscsih e1000 syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mptbase ata_piix libata i2c_core
CPU: 1 PID: 590 Comm: vmtoolsd Tainted: G             L ------------   3.10.0-693.5.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/13/2018
task: ffff8800b6d28fd0 ti: ffff880131fb0000 task.ti: ffff880131fb0000
RIP: 0010:[<ffffffff810f933a>]  [<ffffffff810f933a>] generic_exec_single+0xfa/0x1a0
RSP: 0018:ffff880131fb3c60  EFLAGS: 00000202
RAX: 0000000000000080 RBX: ffff880131fb3c30 RCX: 0000000000000001
RDX: ffffffff816ce470 RSI: 0000000000000080 RDI: 0000000000000286
RBP: ffff880131fb3cb0 R08: ffffffff816ce468 R09: ffffffff811bd6d8
R10: ffff880139659bc0 R11: ffffea0002786500 R12: ffffffff811f1a6e
R13: ffff880131fb3c20 R14: ffff880139076400 R15: 0000000000000001
FS:  00007f10c4e3c880(0000) GS:ffff880139640000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f10c4e60000 CR3: 00000000b6e93000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000000 ffffffff8106e4a0 ffff880131fb3d38
0000000000000003 000000001d3e7fa8 000000001d3e7fa8 0000000000000000
ffffffff8106e4a0 ffffffff8106e4a0 ffff880131fb3ce0 ffffffff810f943f
Call Trace:
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff8106e4a0>] ? leave_mm+0x70/0x70
[<ffffffff810f943f>] smp_call_function_single+0x5f/0xa0
[<ffffffff813227c5>] ? cpumask_next_and+0x35/0x50
[<ffffffff810f99d3>] smp_call_function_many+0x223/0x260
[<ffffffff8106e668>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8106e738>] flush_tlb_mm_range+0x68/0x140
[<ffffffff811acfc3>] tlb_flush_mmu.part.63+0x33/0xc0
[<ffffffff811ae2e5>] tlb_finish_mmu+0x55/0x60
[<ffffffff811b53b4>] unmap_region+0xf4/0x140
[<ffffffff811b5971>] ? __vma_rb_erase+0x121/0x220
[<ffffffff811b7972>] do_munmap+0x292/0x470
[<ffffffff811b7bb5>] vm_munmap+0x65/0xb0
[<ffffffff811b8cc2>] SyS_munmap+0x22/0x30
[<ffffffff816b5089>] system_call_fastpath+0x16/0x1b
Code: 00 78 01 00 48 89 de 48 03 14 c5 e0 fd b0 81 48 89 df e8 3a db 23 00 84 c0 75 46 45 85 ed 74 11 f6 43 20 01 74 0b 0f 1f 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #132
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff8db16f22
Build id: 3.10.0-1127.13.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8da7e730
Build id: 3.10.0-1127.13.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff8da7e730
Build id: 3.10.0-1127.13.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff8da7e730
Build id: 3.10.0-1127.13.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff8db1703f
Build id: 3.10.0-1127.13.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff8dd84fc5
Build id: 3.10.0-1127.13.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff8db175eb
Build id: 3.10.0-1127.13.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8da7e8f8
Build id: 3.10.0-1127.13.1.el7.x86_64
0xb8
9
flush_tlb_mm_range
vmlinux 0xffffffff8da7e96b
Build id: 3.10.0-1127.13.1.el7.x86_64
0x6b
10
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff8dbede16
Build id: 3.10.0-1127.13.1.el7.x86_64
0xa6
11
arch_tlb_finish_mmu
vmlinux 0xffffffff8dbef8df
Build id: 3.10.0-1127.13.1.el7.x86_64
0x3f
12
tlb_finish_mmu
vmlinux 0xffffffff8dbefa03
Build id: 3.10.0-1127.13.1.el7.x86_64
0x23
13
unmap_region
vmlinux 0xffffffff8dbf9214
Build id: 3.10.0-1127.13.1.el7.x86_64
0xf4
14
? xfs_iunlock
xfs 0xffffffffc04d3344
Build id: 3.10.0-1127.13.1.el7.x86_64
0xe4
15
? __vma_rb_erase
vmlinux 0xffffffff8dbf97d1
Build id: 3.10.0-1127.13.1.el7.x86_64
0x121
16
do_munmap
vmlinux 0xffffffff8dbfb805
Build id: 3.10.0-1127.13.1.el7.x86_64
0x2a5
17
vm_munmap
vmlinux 0xffffffff8dbfba45
Build id: 3.10.0-1127.13.1.el7.x86_64
0x65
18
SyS_munmap
vmlinux 0xffffffff8dbfcce2
Build id: 3.10.0-1127.13.1.el7.x86_64
0x22
19
system_call_fastpath
vmlinux 0xffffffff8e192ed2
Build id: 3.10.0-1127.13.1.el7.x86_64
0x25
20
? system_call_after_swapgs
vmlinux 0xffffffff8e192e15
Build id: 3.10.0-1127.13.1.el7.x86_64
0xa2
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [munin-graph:27473]
Modules linked in: nf_conntrack_netlink nfnetlink binfmt_misc vboxnetadp(OE) vboxnetflt(OE) xt_conntrack iptable_filter xt_REDIRECT nf_nat_redirect vboxdrv(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ppdev sg virtio_balloon parport_pc parport i2c_piix4 pcspkr joydev auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif crct10dif_generic cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_scsi net_failover virtio_console failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata crc32c_intel serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 27473 Comm: munin-graph Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
task: ffff8ed1b4ad1070 ti: ffff8ed034014000 task.ti: ffff8ed034014000
RIP: 0010:[<ffffffff8db16f22>]  [<ffffffff8db16f22>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff8ed034017c20  EFLAGS: 00000202
RAX: 0000000000000004 RBX: ffff8ed034017bf0 RCX: 0000000000000830
RDX: 0000000000000004 RSI: 0000000000000004 RDI: 0000000000000282
RBP: ffff8ed034017c70 R08: ffffffff8e213e40 R09: ffffffff8dc027dc
R10: ffff8ed1bfc1f0a0 R11: ffffe83742019380 R12: ffff8ecfbb5d14f8
R13: ffffffffc04e9021 R14: 0000000000000008 R15: 0000000000000150
FS:  00007f1285922740(0000) GS:ffff8ed1bfc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f12857d3000 CR3: 00000001af1cc000 CR4: 00000000000007f0
Call Trace:
[<ffffffff8da7e730>] ? leave_mm+0x120/0x120
[<ffffffff8da7e730>] ? leave_mm+0x120/0x120
[<ffffffff8da7e730>] ? leave_mm+0x120/0x120
[<ffffffff8db1703f>] smp_call_function_single+0x5f/0xa0
[<ffffffff8dd84fc5>] ? cpumask_next_and+0x35/0x50
[<ffffffff8db175eb>] smp_call_function_many+0x22b/0x270
[<ffffffff8da7e8f8>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff8da7e96b>] flush_tlb_mm_range+0x6b/0x140
[<ffffffff8dbede16>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff8dbef8df>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff8dbefa03>] tlb_finish_mmu+0x23/0x30
[<ffffffff8dbf9214>] unmap_region+0xf4/0x140
[<ffffffffc04d3344>] ? xfs_iunlock+0xe4/0x120 [xfs]
[<ffffffff8dbf97d1>] ? __vma_rb_erase+0x121/0x220
[<ffffffff8dbfb805>] do_munmap+0x2a5/0x480
[<ffffffff8dbfba45>] vm_munmap+0x65/0xb0
[<ffffffff8dbfcce2>] SyS_munmap+0x22/0x30
[<ffffffff8e192ed2>] system_call_fastpath+0x25/0x2a
[<ffffffff8e192e15>] ? system_call_after_swapgs+0xa2/0x13a
Code: 89 de 48 03 14 c5 60 10 75 8e 48 89 df e8 17 52 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00 
Complete report #840340
Frame # Function Binary Source or offset Line
1
generic_exec_single
vmlinux 0xffffffff84516ae2
Build id: 3.10.0-1160.76.1.el7.x86_64
0x102
2
? leave_mm
vmlinux 0xffffffff8447ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
3
? leave_mm
vmlinux 0xffffffff8447ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
4
? leave_mm
vmlinux 0xffffffff8447ddd0
Build id: 3.10.0-1160.76.1.el7.x86_64
0x120
5
smp_call_function_single
vmlinux 0xffffffff84516bff
Build id: 3.10.0-1160.76.1.el7.x86_64
0x5f
6
? cpumask_next_and
vmlinux 0xffffffff84786845
Build id: 3.10.0-1160.76.1.el7.x86_64
0x35
7
smp_call_function_many
vmlinux 0xffffffff845171ab
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22b
8
native_flush_tlb_others
vmlinux 0xffffffff8447e258
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb8
9
? cpumask_next_and
vmlinux 0xffffffff84786845
Build id: 3.10.0-1160.76.1.el7.x86_64
0x35
10
smp_call_function_many
vmlinux 0xffffffff845171ab
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22b
11
native_flush_tlb_others
vmlinux 0xffffffff8447e258
Build id: 3.10.0-1160.76.1.el7.x86_64
0xb8
12
tlb_flush_mmu_tlbonly
vmlinux 0xffffffff845ee6a6
Build id: 3.10.0-1160.76.1.el7.x86_64
0xa6
13
arch_tlb_finish_mmu
vmlinux 0xffffffff845f016f
Build id: 3.10.0-1160.76.1.el7.x86_64
0x3f
14
tlb_finish_mmu
vmlinux 0xffffffff845f0293
Build id: 3.10.0-1160.76.1.el7.x86_64
0x23
15
unmap_region
vmlinux 0xffffffff845f9a84
Build id: 3.10.0-1160.76.1.el7.x86_64
0xf4
16
? __wake_up_common_lock
vmlinux 0xffffffff844d3320
Build id: 3.10.0-1160.76.1.el7.x86_64
0x90
17
? __vma_rb_erase
vmlinux 0xffffffff845fa041
Build id: 3.10.0-1160.76.1.el7.x86_64
0x121
18
do_munmap
vmlinux 0xffffffff845fc025
Build id: 3.10.0-1160.76.1.el7.x86_64
0x2a5
19
vm_munmap
vmlinux 0xffffffff845fc265
Build id: 3.10.0-1160.76.1.el7.x86_64
0x65
20
SyS_munmap
vmlinux 0xffffffff845fd562
Build id: 3.10.0-1160.76.1.el7.x86_64
0x22
21
system_call_fastpath
vmlinux 0xffffffff84b99f92
Build id: 3.10.0-1160.76.1.el7.x86_64
0x25
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 112s! [grep:58785]
Modules linked in: nfsv3 rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop vmw_vsock_vmci_transport vsock xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun fuse devlink overlay ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_seq_midi snd_seq_midi_event snd_ens1371 coretemp snd_rawmidi ppdev snd_ac97_codec iosf_mbi
crc32_pclmul ac97_bus snd_seq ghash_clmulni_intel snd_seq_device aesni_intel snd_pcm snd_timer snd lrw gf128mul glue_helper soundcore ablk_helper sg cryptd joydev pcspkr vmw_balloon parport_pc parport vmw_vmci i2c_piix4 sch_fq_codel nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common crc32c_intel mptspi scsi_transport_spi mptscsih serio_raw mptbase e1000 ata_piix libata drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 58785 Comm: grep Kdump: loaded Tainted: G             L ------------   3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/31/2013
task: ffff9179b045a100 ti: ffff91786add8000 task.ti: ffff91786add8000
RIP: 0010:[<ffffffff84516ae2>]  [<ffffffff84516ae2>] generic_exec_single+0x102/0x1c0
RSP: 0018:ffff91786addbc20  EFLAGS: 00000202
RAX: 0000000000000040 RBX: ffffffff844d74c2 RCX: 0000000000000002
RDX: 0000000000000040 RSI: 0000000000000000 RDI: 0000000000000286
RBP: ffff91786addbc70 R08: ffffffff84c13a48 R09: 0000000100400021
R10: 0000000004f28201 R11: ffff917904f28b80 R12: 0000000000000000
R13: ffff9179f661acc0 R14: ffff9179f661acc0 R15: ffff91797fec6300
FS:  00007f7bb67fb740(0000) GS:ffff9179f6600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7bb605ab30 CR3: 0000000038b4e000 CR4: 00000000001607f0
Call Trace:
[<ffffffff8447ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8447ddd0>] ? leave_mm+0x120/0x120
[<ffffffff8447ddd0>] ? leave_mm+0x120/0x120
[<ffffffff84516bff>] smp_call_function_single+0x5f/0xa0
[<ffffffff84786845>] ? cpumask_next_and+0x35/0x50
[<ffffffff845171ab>] smp_call_function_many+0x22b/0x270
[<ffffffff8447e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff84786845>] ? cpumask_next_and+0x35/0x50
[<ffffffff845171ab>] smp_call_function_many+0x22b/0x270
[<ffffffff8447e258>] native_flush_tlb_others+0xb8/0xc0
[<ffffffff845ee6a6>] tlb_flush_mmu_tlbonly+0xa6/0xc0
[<ffffffff845f016f>] arch_tlb_finish_mmu+0x3f/0x80
[<ffffffff845f0293>] tlb_finish_mmu+0x23/0x30
[<ffffffff845f9a84>] unmap_region+0xf4/0x140
[<ffffffff844d3320>] ? __wake_up_common_lock+0x90/0xc0
[<ffffffff845fa041>] ? __vma_rb_erase+0x121/0x220
[<ffffffff845fc025>] do_munmap+0x2a5/0x480
[<ffffffff845fc265>] vm_munmap+0x65/0xb0
[<ffffffff845fd562>] SyS_munmap+0x22/0x30
[<ffffffff84b99f92>] system_call_fastpath+0x25/0x2a
Code: 89 de 48 03 14 c5 00 19 15 85 48 89 df e8 d7 6e 28 00 84 c0 75 53 45 85 ed 74 16 f6 43 20 01 74 10 0f 1f 84 00 00 00 00 00 f3 90 <f6> 43 20 01 75 f8 31 c0 48 8b 7c 24 28 65 48 33 3c 25 28 00 00