Info

Permalink
Function
retint_careful
First occurrence
2020-09-22
Last occurrence
2025-04-15
State
NEW
Components
kernel
Quality
-2

Statistics

Operating system Count
CentOS 7 4,637
Architecture Count
x86_64 4,637
Related packages Count
kernel 4,635
 0:3.10.0-1160.el7 581
 0:3.10.0-1127.el7 336
 0:3.10.0-957.el7 248
 0:3.10.0-862.el7 189
 0:3.10.0-1160.53.1.el7 150
 0:3.10.0-1160.118.1.el7 146
 0:3.10.0-1127.18.2.el7 139
 0:3.10.0-1160.71.1.el7 138
 0:3.10.0-1127.19.1.el7 129
 0:3.10.0-693.el7 129
 0:3.10.0-1160.45.1.el7 101
 0:3.10.0-1062.el7 100
 0:3.10.0-1160.49.1.el7 99
 0:3.10.0-1160.11.1.el7 91
 0:3.10.0-1160.15.2.el7 91
 0:3.10.0-1160.62.1.el7 86
 0:3.10.0-1160.76.1.el7 84
 0:3.10.0-1160.21.1.el7 67
 0:3.10.0-1160.119.1.el7 66
 0:3.10.0-1160.83.1.el7 65
 0:3.10.0-1062.1.2.el7 64
 0:3.10.0-1160.90.1.el7 63
 0:3.10.0-1160.25.1.el7 61
 0:3.10.0-1062.9.1.el7 60
 0:3.10.0-957.1.3.el7 58
 0:3.10.0-1160.108.1.el7 54
 0:3.10.0-1062.12.1.el7 52
 0:3.10.0-1062.18.1.el7 51
 0:3.10.0-1160.66.1.el7 51
 0:3.10.0-1160.81.1.el7 48
 0:3.10.0-1160.59.1.el7 47
 0:3.10.0-514.el7 47
 0:3.10.0-1160.80.1.el7 46
 0:3.10.0-957.27.2.el7 44
 0:3.10.0-957.21.3.el7 43
 0:3.10.0-1160.31.1.el7 42
 0:3.10.0-862.14.4.el7 42
 0:3.10.0-693.5.2.el7 38
 0:3.10.0-1160.102.1.el7 37
 0:3.10.0-1160.88.1.el7 37
 0:3.10.0-1160.42.2.el7 35
 0:3.10.0-957.12.1.el7 34
 0:3.10.0-1160.92.1.el7 32
 0:3.10.0-1160.41.1.el7 29
 0:3.10.0-1127.10.1.el7 28
 0:3.10.0-957.10.1.el7 28
 0:3.10.0-957.5.1.el7 28
 0:3.10.0-1160.105.1.el7 26
 0:3.10.0-327.el7 26
 0:3.10.0-1160.36.2.el7 24
 0:3.10.0-1127.13.1.el7 23
 0:3.10.0-1160.24.1.el7 23
 0:3.10.0-1160.6.1.el7 23
 0:3.10.0-1160.95.1.el7 22
 0:3.10.0-1062.1.1.el7 20
 0:3.10.0-862.2.3.el7 18
 0:3.10.0-957.12.2.el7 18
 0:3.10.0-693.21.1.el7 16
 0:3.10.0-693.2.2.el7 16
 0:3.10.0-1062.4.1.el7 11
 0:3.10.0-1160.99.1.el7 10
 0:3.10.0-514.10.2.el7 10
 0:3.10.0-1062.4.3.el7 9
 0:3.10.0-1127.8.2.el7 8
 0:3.10.0-1160.114.2.el7 8
 0:3.10.0-514.16.1.el7 8
 0:3.10.0-862.11.6.el7 8
 0:3.10.0-693.17.1.el7 8
 0:3.10.0-862.3.2.el7 6
 0:3.10.0-1160.2.2.el7 5
 0:3.10.0-229.el7 5
 0:3.10.0-693.11.1.el7 5
 0:3.10.0-862.9.1.el7 5
 0:3.10.0-514.21.2.el7 5
 0:3.10.0-514.21.1.el7 4
 0:3.10.0-693.11.6.el7 4
 0:3.10.0-862.6.3.el7 4
 0:3.10.0-327.18.2.el7 3
 0:3.10.0-514.26.2.el7 3
 0:3.10.0-957.21.2.el7 3
 0:3.10.0-123.20.1.el7 2
 0:3.10.0-327.36.2.el7 2
 0:3.10.0-327.36.3.el7 2
 0:3.10.0-862.3.3.el7 2
 0:3.10.0-327.10.1.el7 1
 0:3.10.0-514.2.2.el7 1
 0:3.10.0-514.6.1.el7 1
 0:3.10.0-1160.119.1.el7.tuxcare.els11 1
 0:3.10.0-327.22.2.el7 1
 0:3.10.0-327.28.2.el7 1
kernel-rt 2
 0:3.10.0-693.2.2.rt56.623.el7 2

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #3918
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff8656c4f0
Build id: 3.10.0-957.10.1.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [httpd:30150]
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 21s! [httpd:30145]
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc i2c_piix4 parport vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw
 ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 30145 Comm: httpd Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8a76bba26180 ti: ffff8a70b23f8000 task.ti: ffff8a70b23f8000
RIP: 0010:[<ffffffff8656c4f0>]  [<ffffffff8656c4f0>] retint_careful+0xe/0x32
RSP: 0000:ffff8a70b23fbf88  EFLAGS: 00000203
RAX: 0140000000000000 RBX: ffff8a70b23fbf48 RCX: ffff8a70b23f8000
RDX: 0000000000000008 RSI: ffff8a70b23fbf78 RDI: 000000000000fe0e
RBP: 00007f4cdc298230 R08: 0000000000000001 R09: ffff8a76bba26180
R10: 00007f4d45279ee0 R11: 00007f4d452bfb80 R12: ffff8a76abfde800
R13: 0000000000000003 R14: ffff8a76abfde800 R15: ffffffff860415d8
FS:  00007f4d23fff700(0000) GS:ffff8a77cfc80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f595079c000 CR3: 0000001870f14000 CR4: 00000000003607e0
Modules linked in: ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock ppdev vmw_balloon sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc i2c_piix4 parport vmw_vmci ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_pclmul crct10dif_common crc32c_intel serio_raw
 ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix mptspi scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 30150 Comm: httpd Kdump: loaded Tainted: G        W    L ------------   3.10.0-957.10.1.el7.x86_64 #1
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff8a774e5b6180 ti: ffff8a70aff54000 task.ti: ffff8a70aff54000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
Complete report #805671
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff815ea650
Build id: 3.10.0-123.20.1.el7.x86_64
0xe
BUG: soft lockup - CPU#2 stuck for 67s! [cc1plus:31756]
Modules linked in: nls_utf8 isofs fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT iptable_filter tun bridge stp llc binfmt_misc bwpcidrv(POF) vboxsf(OF) dm_mirror dm_region_hash dm_log dm_mod snd_intel8x0 snd_ac97_codec ac97_bus snd_seq crc32_pclmul snd_seq_device ghash_clmulni_intel snd_pcm snd_page_alloc aesni_intel lrw gf128mul snd_timer glue_helper ablk_helper cryptd vboxguest(OF) serio_raw snd pcspkr video soundcore sg i2c_piix4 i2c_core mperf nfsd auth_rpcgss nfs_acl lockd sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic sd_mod crc_t10dif pata_acpi ahci ata_piix libahci crct10dif_pclmul crct10dif_common crc32c_intel e1000 libata
CPU: 2 PID: 31756 Comm: cc1plus Tainted: PF          O--------------   3.10.0-123.20.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff880078264440 ti: ffff8800b9cb2000 task.ti: ffff8800b9cb2000
RIP: 0010:[<ffffffff815ea650>]  [<ffffffff815ea650>] retint_careful+0xe/0x32
RSP: 0000:ffff8800b9cb3f88  EFLAGS: 00010203
RAX: 0140000000000001 RBX: ffff8800b9cb3f60 RCX: ffff8800b9cb2000
RDX: 0000000000000008 RSI: ffff8800b9cb3f78 RDI: 000000000000fe2e
RBP: 000000000000030a R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000021 R11: 0000000000000021 R12: ffff880035f8ddc0
R13: ffff8800bfb135c0 R14: ffff880035f8ddc0 R15: ffff880035f8ddc0
FS:  00007f3386d2c840(0000) GS:ffff8800bfb00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe98de14000 CR3: 00000000add27000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000021 0000000000000021 0000000004648e40 0000000000000021
0000000000000074 00000000045e0bd8 00000000045ae188 0000000000018574
0000000000000086 ffffffffffffff02 00000000008cc1e6 0000000000000033
Call Trace:
Code: 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 aa ed ff ff 5f 65 48 8b 0c 25 b0 b7 00 00 48 81 e9 d8 1f 
Complete report #373
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffffa4589510
Build id: 3.10.0-1127.13.1.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [zmstat-allprocs:8867]
Modules linked in: nf_conntrack_netlink nfnetlink binfmt_misc vboxnetadp(OE) vboxnetflt(OE) xt_conntrack iptable_filter xt_REDIRECT nf_nat_redirect vboxdrv(OE) iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack sg ppdev joydev pcspkr parport_pc virtio_balloon parport i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi virtio_net virtio_console net_failover virtio_scsi failover bochs_drm drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata serio_raw virtio_pci virtio_ring virtio drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 8867 Comm: zmstat-allprocs Kdump: loaded Tainted: G           OEL ------------   3.10.0-1127.13.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.11.2-0-gf9626ccb91-prebuilt.qemu-project.org 04/01/2014
task: ffff8f9caba60000 ti: ffff8f9caba68000 task.ti: ffff8f9caba68000
RIP: 0010:[<ffffffffa4589510>]  [<ffffffffa4589510>] retint_careful+0xe/0x32
RSP: 0000:ffff8f9caba6bf88  EFLAGS: 00000203
RAX: 0000000000000001 RBX: ffffffffa45892ee RCX: ffff8f9caba68000
RDX: 0000000000000008 RSI: ffff8f9caba6bf78 RDI: 000000000000fe0e
RBP: 0000000001694cc8 R08: 0000000000000000 R09: ffff8f9d3fc03e08
R10: ffffffffa505846c R11: 0000000000007ffe R12: ffffffffa45892ee
R13: ffffffffa45892fa R14: ffffffffa45892ee R15: ffffffffa45892fa
FS:  00007f93b18da740(0000) GS:ffff8f9d3fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007ff5c96de000 CR3: 00000002340ce000 CR4: 00000000000006f0
Call Trace:
Code: 38 48 8b 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 90 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 ca d6 ff ff 5f 65 48 8b 0c 25 b8 0e 01 00 48 81 e9 d8 3f 
Complete report #610856
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff8163e4cd
Build id: 3.10.0-327.28.2.el7.x86_64
0xb
BUG: soft lockup - CPU#0 stuck for 25s! [osa-dispatcher:16857]
Modules linked in: zram(C) vmw_vsock_vmci_transport vsock ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_conntrack nf_conntrack ip6table_filter ip6_tables iptable_filter coretemp crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd vmw_balloon pcspkr sg vmxnet3 vmw_vmci i2c_piix4 shpchp parport_pc parport sch_fq_codel binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx crct10dif_pclmul crct10dif_common serio_raw drm_kms_helper ttm ahci libahci vmw_pvscsi drm i2c_core ata_piix libata floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 16857 Comm: osa-dispatcher Tainted: G         C   L ------------   3.10.0-327.28.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88002012b980 ti: ffff8800b3a38000 task.ti: ffff8800b3a38000
RIP: 0010:[<ffffffff8163e4cd>]  [<ffffffff8163e4cd>] retint_careful+0xb/0x32
RSP: 0000:ffff8800b3a3bf88  EFLAGS: 00000203
RAX: 0140000000000001 RBX: ffffffff8163b608 RCX: ffff8800b3a38000
RDX: 0000000000000008 RSI: ffff8800b3a3bf78 RDI: 000000000000fe2e
RBP: 0000000000b5dc90 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000072 R11: 0000000000b4e63f R12: 0000000000000000
R13: ffff880074638640 R14: ffff88017fc167c0 R15: ffff880086eeb840
FS:  00007f258a97c740(0000) GS:ffff88017fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9e506afca8 CR3: 00000000680a4000 CR4: 00000000000007f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
 0000000000b4e63f 0000000000000072 0000000000b4e63d 0000000000b54788
 0000000000b54788 000000000000000a 0000000000b53e10 00007f258a889bd0
 00007f258a889bd0 ffffffffffffff02 00007f258a4826ab 0000000000000033
Call Trace:
Code: 38 48 8b 7c 24 40 48 83 c4 50 48 cf 66 66 66 90 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 66 66 66 90 <66> 66 90 57 e8 7a e7 ff ff 5f 65 48 8b 0c 25 b8 b7 00 00 48 81 
Complete report #534335
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff8278b50d
Build id: 3.10.0-1160.15.2.el7.x86_64
0xb
NMI watchdog: BUG: soft lockup - CPU#58 stuck for 22s! [python:64590]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_14203(E) rpcsec_gss_krb5 nfsv4 dns_resolver nfs fscache uinput devlink binfmt_misc ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter falcon_lsm_pinned_14108(E) snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm qxl snd_timer ttm drm_kms_helper iTCO_wdt syscopyarea
crc32_pclmul ghash_clmulni_intel aesni_intel lrw sysfillrect gf128mul sysimgblt glue_helper ppdev fb_sys_fops ablk_helper snd iTCO_vendor_support joydev cryptd parport_pc drm virtio_rng pcspkr i2c_i801 lpc_ich soundcore virtio_balloon parport drm_panel_orientation_quirks nfsd nfs_acl lockd auth_rpcgss grace sunrpc ip_tables xfs libcrc32c virtio_net net_failover virtio_console virtio_blk failover crct10dif_pclmul ahci virtio_pci libahci crct10dif_common serio_raw crc32c_intel libata virtio_ring virtio dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: falcon_kal]
CPU: 58 PID: 64590 Comm: python Kdump: loaded Tainted: P        W   EL ------------   3.10.0-1160.15.2.el7.x86_64 #1
Hardware name: Red Hat KVM/RHEL-AV, BIOS 1.13.0-2.module+el8.4.0+534+4680a14e 04/01/2014
task: ffff8b11623cc200 ti: ffff8b10d5f5c000 task.ti: ffff8b10d5f5c000
RIP: 0010:[<ffffffff8278b50d>]  [<ffffffff8278b50d>] retint_careful+0xb/0x32
RSP: 0000:ffff8b10d5f5ff88  EFLAGS: 00000203
RAX: 0140000000000000 RBX: 000000000001acc0 RCX: ffff8b10d5f5c000
RDX: 0000000000000008 RSI: ffff8b10d5f5ff78 RDI: 000000000000fe0e
RBP: 00007f206f77e8e8 R08: 0000000000000001 R09: ffff8b11623cc200
R10: 000000000000003c R11: 0000000000000050 R12: ffff8b10d5f5ffd8
R13: ffff8b10d5f5ffd8 R14: ffffffff827878ff R15: ffff8b10d5f5ff58
FS:  00007f24ba814740(0000) GS:ffff8cfbac480000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000843f7008 CR3: 000000066df9a000 CR4: 0000000000760ee0
PKRU: 55555554
Call Trace:
Code: 8b 74 24 38 48 8b 7c 24 40 48 83 c4 50 48 cf 66 66 66 90 90 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 66 66 66 90 <66> 66 90 57 e8 fa d6 ff ff 5f 65 48 8b 0c 25 b8 0e 01 00 48 81 
Complete report #511751
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffffa7589510
Build id: 3.10.0-1127.19.1.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 188s! [opensearch[FI-P:12372]
Modules linked in: xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc devlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock sunrpc ext4 mbcache jbd2 sb_edac coretemp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw gf128mul glue_helper vmw_balloon ablk_helper cryptd joydev pcspkr sg parport_pc vmw_vmci parport i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix crct10dif_pclmul
crct10dif_common crc32c_intel libata mptspi serio_raw scsi_transport_spi mptscsih e1000 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 12372 Comm: opensearch[FI-P Kdump: loaded Tainted: G    B        L ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 06/22/2012
task: ffff91dc6cc7c1c0 ti: ffff91da9175c000 task.ti: ffff91da9175c000
RIP: 0010:[<ffffffffa7589510>]  [<ffffffffa7589510>] retint_careful+0xe/0x32
RSP: 0000:ffff91da9175ff88  EFLAGS: 00000203
RAX: 0140000000000000 RBX: ffffffffa75956ee RCX: ffff91da9175c000
RDX: 0000000000000008 RSI: ffff91da9175ff78 RDI: 000000000000fe0e
RBP: 00007efc6c000020 R08: ffffffffa761e620 R09: ffff91dc6cc7c1c0
R10: 0000000000000003 R11: 0000000000000246 R12: ffffffffa75956ee
R13: ffffffffa75956fa R14: ffffffffa75956ee R15: ffffffffa75956fa
FS:  00007efa8476b700(0000) GS:ffff91dc7fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007efc753b4008 CR3: 0000000347ef8000 CR4: 00000000000407f0
Call Trace:
Code: 38 48 8b 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 90 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 ca d6 ff ff 5f 65 48 8b 0c 25 b8 0e 01 00 48 81 e9 d8 3f 
Complete report #83
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff8168de0d
Build id: 3.10.0-514.el7.x86_64
0xb
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 25s! [3_scheduler:3036]
Modules linked in: fuse uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter snd_hda_codec_generic iosf_mbi crc32_pclmul snd_hda_intel ghash_clmulni_intel aesni_intel snd_hda_codec lrw gf128mul glue_helper ablk_helper snd_hda_core snd_hwdep snd_seq ppdev snd_seq_device cryptd snd_pcm snd_timer snd pcspkr parport_pc virtio_balloon parport sg i2c_piix4 soundcore nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi virtio_console virtio_blk virtio_net qxl drm_kms_helper crct10dif_pclmul syscopyarea crct10dif_common sysfillrect
crc32c_intel sysimgblt ata_piix fb_sys_fops serio_raw ttm drm virtio_pci virtio_ring virtio libata i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 3036 Comm: 3_scheduler Tainted: G             L ------------   3.10.0-514.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
task: ffff880221633ec0 ti: ffff8802216b8000 task.ti: ffff8802216b8000
RIP: 0010:[<ffffffff8168de0d>]  [<ffffffff8168de0d>] retint_careful+0xb/0x32
RSP: 0000:ffff8802216bbf88  EFLAGS: 00000203
RAX: 0000000000000000 RBX: ffffffff8168b070 RCX: ffff8802216b8000
RDX: 0000000000000008 RSI: 00000000a80ea80c RDI: 000000000000fe2e
RBP: 00007f7d35094388 R08: ffff8800029b6a40 R09: 0000000180400020
R10: 00000000029b6a01 R11: ffffea00000a6d80 R12: 0000000000000002
R13: ffff88022d315140 R14: ffff880239d16c40 R15: ffff88022d2dc4c0
FS:  00007f7d3effd700(0000) GS:ffff880239d00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000f5920002 CR3: 00000000b8f44000 CR4: 00000000000406e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000001 0000000000018f88 00007f7d35094388
00007f7d350c85b8 00007f7d3504b270 0000000000018f88 00007f7d35094388
00007f7d350400a0 ffffffffffffffff 00000000005cc405 0000000000000033
Call Trace:
Code: 38 48 8b 7c 24 40 48 83 c4 50 48 cf 66 66 66 90 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 66 66 66 90 <66> 66 90 57 e8 6a e8 ff ff 5f 65 48 8b 0c 25 b8 cd 00 00 48 81 
Complete report #23611
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffffa871c4f0
Build id: 3.10.0-862.14.4.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 31s! [node_exporter:1745]
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 31s! [prometheus:1627]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_lsm_pinned_10703(E) rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache vmw_vsock_vmci_transport vsock falcon_lsm_pinned_10404(E) sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd vmw_balloon nfit pcspkr joydev sg libnvdimm vmw_vmci shpchp i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sr_mod cdrom sd_mod crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm crct10dif_pclmul crct10dif_common ata_piix crc32c_intel mptspi serio_raw libata scsi_transport_spi mptscsih mptbase vmxnet3 i2c_core floppy dm_mirror dm_region_hash dm_log dm_mod [last unloaded: falcon_lsm_serviceable]
CPU: 4 PID: 1627 Comm: prometheus Kdump: loaded Tainted: P            EL ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9b7ce1333f40 ti: ffff9b7cd9e70000 task.ti: ffff9b7cd9e70000
RIP: 0010:[<ffffffffa871c4f0>]  [<ffffffffa871c4f0>] retint_careful+0xe/0x32
RSP: 0000:ffff9b7cd9e73f88  EFLAGS: 00000203
RAX: 0140000000000000 RBX: 0000000000000000 RCX: ffff9b7cd9e70000
RDX: 0000000000000008 RSI: ffff9b7cd9e73f78 RDI: 000000000000fe0e
RBP: 000000c00b599038 R08: 0000000000000001 R09: ffff9b7ce1333f40
R10: 0000000002b5a065 R11: 0000000000000001 R12: ffff9b7ce4f48860
R13: ffff9b7cd9e73f58 R14: 000000003121869f R15: ffff9b7ce1333f40
FS:  00007f5608dfa700(0000) GS:ffff9b7cffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f8d03838000 CR3: 0000000421b8e000 CR4: 00000000003607e0
Call Trace:
Code: 54 24 30 48 8b 74 24 38 48 8b 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 0f 1f 40 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 7a d8 ff ff 5f 65 48 8b 0c 25 78 0e 01 00 48 81 e9 d8 3f 
Complete report #17348
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffffb398b510
Build id: 3.10.0-1160.6.1.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 43s! [gmain:3018]
Modules linked in: vmfsrtx_export(POE) vmfsrtx_bridge(POE) vmfsrtx_bridge_helper(OE) vmfsrtx_dev(POE) vmfsrtx_dev_helper(POE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ppdev ablk_helper cryptd joydev pcspkr vmw_balloon sg parport_pc vmw_vmci
parport i2c_piix4 auth_rpcgss sunrpc ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm nfit crct10dif_pclmul crct10dif_common crc32c_intel libnvdimm ata_piix mptspi scsi_transport_spi mptscsih libata mptbase serio_raw vmxnet3 drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: vmfsrtx_bridge_helper]
CPU: 2 PID: 3018 Comm: gmain Kdump: loaded Tainted: P        W  OEL ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
task: ffff9e2f87fed280 ti: ffff9e2f8dfe0000 task.ti: ffff9e2f8dfe0000
RIP: 0010:[<ffffffffb398b510>]  [<ffffffffb398b510>] retint_careful+0xe/0x32
RSP: 0000:ffff9e2f8dfe3f88  EFLAGS: 00000203
RAX: 0140000000000000 RBX: ffffffffb39977ae RCX: ffff9e2f8dfe0000
RDX: 0000000000000008 RSI: ffff9e2f8dfe3f78 RDI: 000000000000fe0e
RBP: 0000000002302eb0 R08: ffffffffb3a1e640 R09: ffff9e2f87fed280
R10: 0000000000000000 R11: 0005676ffaed7581 R12: ffffffffb39977ae
R13: ffffffffb39977ba R14: ffffffffb39977ae R15: ffffffffb39977ba
FS:  00007f8bdc318700(0000) GS:ffff9e4d7ee80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f92c5b0f000 CR3: 00000001fdbe6000 CR4: 00000000000407e0
Call Trace:
Code: 38 48 8b 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 90 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 4a cb ff ff 5f 65 48 8b 0c 25 b8 0e 01 00 48 81 e9 d8 3f 
Complete report #14818
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff8160c190
Build id: 3.10.0-229.el7.x86_64
0xe
BUG: soft lockup - CPU#2 stuck for 25s! [vmtoolsd:939]
Modules linked in: tcp_lp bnep bluetooth rfkill fuse vmhgfs(OF) dm_mirror dm_region_hash dm_log ext4 mbcache jbd2 snd_ens1371 snd_rawmidi crc32_pclmul snd_ac97_codec ghash_clmulni_intel ac97_bus snd_seq snd_seq_device aesni_intel snd_pcm lrw gf128mul snd_timer glue_helper ablk_helper snd cryptd soundcore ppdev vmw_balloon sg pcspkr parport_pc parport i2c_piix4 vmw_vmci shpchp dm_multipath uinput dm_mod ip_tables xfs libcrc32c sr_mod sd_mod cdrom crc_t10dif ata_generic pata_acpi crct10dif_pclmul crct10dif_common crc32c_intel vmwgfx serio_raw ata_piix drm_kms_helper ttm libata mptspi scsi_transport_spi drm mptscsih mptbase e1000 i2c_core
CPU: 2 PID: 939 Comm: vmtoolsd Tainted: GF          O--------------   3.10.0-229.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 05/19/2017
task: ffff8802e6625b00 ti: ffff880035e7c000 task.ti: ffff880035e7c000
RIP: 0010:[<ffffffff8160c190>]  [<ffffffff8160c190>] retint_careful+0xe/0x32
RSP: 0000:ffff880035e7ff88  EFLAGS: 00000203
RAX: 0140000000000001 RBX: ffff8802e4007738 RCX: ffff880035e7c000
RDX: 0000000000000008 RSI: ffff880035e7ff78 RDI: 000000000000fe2e
RBP: 00007f7b3152d1c0 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000006 R11: 0000000000000000 R12: ffff880036abb5f0
R13: 00007f7b2f30a000 R14: ffffffff81188b37 R15: ffff880035e7ff28
FS:  00007f7b2f2eb740(0000) GS:ffff8802eda40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7b2f30a000 CR3: 00000002e70e9000 CR4: 00000000001407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000000 0000000000000006 0000000000000002 000000000000001e
0000000000000000 00007f7b00810000 00007f7b00015658 00000000c9d0a26a
00007fff307a2586 ffffffffffffff02 00007f7b2ee7eead 0000000000000033
Call Trace:
Code: 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 da ed ff ff 5f 65 48 8b 0c 25 b8 b7 00 00 48 81 e9 d8 3f 
Complete report #2440
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff8168e350
Build id: 3.10.0-514.10.2.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 38s! [vmtoolsd:998]
Modules linked in: nsrbbb(POE) vmw_vsock_vmci_transport vsock intel_powerclamp iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper sg cryptd ppdev pcspkr vmw_balloon i2c_piix4 vmw_vmci parport_pc parport shpchp nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables ext4 mbcache jbd2 sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common serio_raw vmwgfx crc32c_intel drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm vmxnet3 drm ata_piix vmw_pvscsi i2c_core libata floppy fjes dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 998 Comm: vmtoolsd Tainted: P           OEL ------------   3.10.0-514.10.2.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
task: ffff88042c373ec0 ti: ffff880035e98000 task.ti: ffff880035e98000
RIP: 0010:[<ffffffff8168e350>]  [<ffffffff8168e350>] retint_careful+0xe/0x32
RSP: 0000:ffff880035e9bf88  EFLAGS: 00000203
RAX: 0140000000000000 RBX: ffffffff8168b5b0 RCX: ffff880035e98000
RDX: 0000000000000008 RSI: ffff880035e9bf78 RDI: 000000000000fe2e
RBP: 00007ffe907f0a10 R08: 0000000000000001 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: ffff8803266b0c80 R14: ffff88043fc16c40 R15: ffff88042772b200
FS:  00007f6243a15740(0000) GS:ffff88043fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007fbc81ebe000 CR3: 0000000035c0e000 CR4: 00000000000407f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
0000000000000206 0000000000000000 00007f6241e4b0c8 0000000000c31678
0000000000000000 0000000000010000 0000000000015658 0000000027f3980c
00000000c9949994 ffffffffffffff02 00007f6241bf8f7d 0000000000000033
Call Trace:
Code: 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 6a e8 ff ff 5f 65 48 8b 0c 25 b8 cd 00 00 48 81 e9 d8 3f 
Complete report #5445
Frame # Function Binary Source or offset Line
1
retint_careful
vmlinux 0xffffffff96b6c4f0
Build id: 3.10.0-957.10.1.el7.x86_64
0xe
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 35s! [Capture0--dpdk-:9737]
Modules linked in: fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink igb_uio(OE) uio vboxsf(OE) sunrpc ppdev iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd vmwgfx joydev pcspkr ttm snd_intel8x0 snd_ac97_codec drm_kms_helper ac97_bus snd_seq i2c_piix4 snd_seq_device sg syscopyarea sysfillrect sysimgblt snd_pcm fb_sys_fops drm parport_pc parport snd_timer vboxguest(OE) video snd soundcore drm_panel_orientation_quirks ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic ata_generic pata_acpi crct10dif_pclmul
crct10dif_common crc32c_intel serio_raw ata_piix e1000 libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 9737 Comm: Capture0--dpdk- Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-957.10.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff9d635a25a080 ti: ffff9d6472690000 task.ti: ffff9d6472690000
RIP: 0010:[<ffffffff96b6c4f0>]  [<ffffffff96b6c4f0>] retint_careful+0xe/0x32
RSP: 0000:ffff9d6472693f88  EFLAGS: 00003203
RAX: 0140000000000000 RBX: ffffffff96b785eb RCX: ffff9d6472690000
RDX: 0000000000000008 RSI: ffff9d6472693f78 RDI: 000000000000fe0e
RBP: 0000000000000000 R08: 0000000000000000 R09: ffff9d635a25a080
R10: 0000000000000002 R11: 000a6e7f7c5777ee R12: ffffffff96b785eb
R13: ffffffff96b785f7 R14: ffffffff96b785eb R15: ffffffff96b785f7
FS:  00007fffe8aa1700(0000) GS:ffff9d655fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000000000000004c CR3: 00000001aae8e000 CR4: 00000000000606f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
Code: 54 24 30 48 8b 74 24 38 48 8b 7c 24 40 48 83 c4 50 48 cf 0f 1f 40 00 0f 1f 40 00 48 cf 0f ba e2 03 73 2c fb 0f 1f 80 00 00 00 00 <57> e8 8a d5 ff ff 5f 65 48 8b 0c 25 78 0e 01 00 48 81 e9 d8 3f