Info

Permalink
Function
find_get_pages_tag
First occurrence
2020-11-18
Last occurrence
2025-07-18
State
NEW
Components
kernel
Quality
-41

Statistics

Operating system Count
CentOS 7 3,609
Architecture Count
x86_64 3,609
Related packages Count
kernel 3,609
 0:3.10.0-1160.66.1.el7 988
 0:3.10.0-1127.19.1.el7 410
 0:3.10.0-1160.62.1.el7 331
 0:3.10.0-1160.45.1.el7 297
 0:3.10.0-1160.83.1.el7 276
 0:3.10.0-1160.25.1.el7 228
 0:3.10.0-862.14.4.el7 209
 0:3.10.0-1127.10.1.el7 178
 0:3.10.0-1160.6.1.el7 136
 0:3.10.0-1160.118.1.el7 132
 0:3.10.0-1160.31.1.el7 96
 0:3.10.0-1160.108.1.el7 69
 0:3.10.0-1160.119.1.el7 56
 0:3.10.0-1160.59.1.el7 53
 0:3.10.0-1160.24.1.el7 45
 0:3.10.0-1160.114.2.el7 44
 0:3.10.0-1160.15.2.el7 42
 0:3.10.0-1160.90.1.el7 3
 0:3.10.0-1160.76.1.el7 3
 0:3.10.0-1160.el7 3
 0:3.10.0-957.12.1.el7 2
 0:3.10.0-1160.36.2.el7 2
 0:3.10.0-693.el7 2
 0:3.10.0-1160.41.1.el7 1
 0:3.10.0-1160.80.1.el7 1
 0:3.10.0-957.el7 1
 0:3.10.0-1160.105.1.el7 1

Packages names with count less than the total count of reports are most probably not the cause of the problem.

History

Daily:

Weekly:

Monthly:

Report backtrace


Complete report #1267153
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffa4dbe872
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffffa4dccb51
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffa4dca87a
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc021f290
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffffa4f55c60
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc021f25b
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffffa4dcb9d1
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffffa4e7df40
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffffa4cc7045
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffffa4e7eac4
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffffa4e7edcf
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffffa4e7f2a3
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffffa4e6ac1c
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffffa4e7fe9b
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffffa4cbdfbf
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffffa4cbf0d6
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffffa4cbefb0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffffa4cc5f91
Build id: 3.10.0-1160.66.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffffa4cc5ec0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffffa5399df7
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffffa4cc5ec0
Build id: 3.10.0-1160.66.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kworker/u24:1:13424]
Modules linked in: nvidia_uvm(OE) xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 devlink tun ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep sunrpc nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) intel_pmc_core intel_powerclamp coretemp iTCO_wdt snd_hda_codec_hdmi iTCO_vendor_support intel_rapl iosf_mbi kvm_intel ath10k_pci kvm ath10k_core ath irqbypass mac80211 crc32_pclmul snd_hda_codec_realtek ghash_clmulni_intel
snd_hda_codec_generic aesni_intel lrw gf128mul cfg80211 glue_helper ablk_helper cryptd dell_wmi pcspkr snd_hda_intel sparse_keymap dell_smbios dcdbas snd_hda_codec dell_wmi_descriptor sg snd_hda_core i2c_i801 snd_seq snd_hwdep snd_seq_device drm_kms_helper btusb btrtl snd_pcm btbcm syscopyarea btintel mei_me sysfillrect bluetooth mei snd_timer sysimgblt fb_sys_fops snd drm soundcore idma64 i2c_designware_platform virt_dma drm_panel_orientation_quirks i2c_designware_core dell_rbtn rfkill acpi_pad ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic ahci libahci libata r8169 crct10dif_pclmul crct10dif_common crc32c_intel wmi i2c_hid video dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 4 PID: 13424 Comm: kworker/u24:1 Kdump: loaded Tainted: P    B      OE  ------------   3.10.0-1160.66.1.el7.x86_64 #1
Hardware name: Dell Inc. Inspiron 5680/0PXWHK, BIOS 1.1.1 02/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-8:16)
task: ffff94890ba0c200 ti: ffff9494b6e6c000 task.ti: ffff9494b6e6c000
RIP: 0010:[<ffffffffa4dbe872>]  [<ffffffffa4dbe872>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff9494b6e6f9b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff94972e50c000 RCX: 0000000000000031
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffdf8aed4d3a40
RBP: ffff9494b6e6fa18 R08: ffff9494b6e6fa98 R09: ffff9496936a8148
R10: 0000000000004000 R11: ffff94937485bf00 R12: ffff9494b6e6f968
R13: 0000000000000286 R14: ffff94972e50ca68 R15: 0000000000000286
FS:  0000000000000000(0000) GS:ffff949735500000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f9368778080 CR3: 0000000967a10000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffffa4dccb51>] pagevec_lookup_tag+0x21/0x30
[<ffffffffa4dca87a>] write_cache_pages+0x11a/0x470
[<ffffffffc021f290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffa4f55c60>] ? submit_bio+0x70/0x150
[<ffffffffc021f25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffa4dcb9d1>] do_writepages+0x21/0x50
[<ffffffffa4e7df40>] __writeback_single_inode+0x40/0x260
[<ffffffffa4cc7045>] ? wake_up_bit+0x25/0x30
[<ffffffffa4e7eac4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa4e7edcf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa4e7f2a3>] wb_writeback+0x263/0x2f0
[<ffffffffa4e6ac1c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffa4e7fe9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffa4cbdfbf>] process_one_work+0x17f/0x440
[<ffffffffa4cbf0d6>] worker_thread+0x126/0x3c0
[<ffffffffa4cbefb0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa4cc5f91>] kthread+0xd1/0xe0
[<ffffffffa4cc5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa5399df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa4cc5ec0>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #1171951
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffb37c9546
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x76
2
pagevec_lookup_tag
vmlinux 0xffffffffb37d7c61
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffb37d590a
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc04a76c0
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xb0
5
? put_prev_entity
vmlinux 0xffffffffb36ec6ab
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x8b
6
? pick_next_entity
vmlinux 0xffffffffb36e8699
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xa9
7
? set_next_entity
vmlinux 0xffffffffb36e918c
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x3c
8
xfs_vm_writepages
xfs 0xffffffffc04a767b
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x6b
9
do_writepages
vmlinux 0xffffffffb37d6ab1
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x21
10
__writeback_single_inode
vmlinux 0xffffffffb388ccf0
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x40
11
writeback_sb_inodes
vmlinux 0xffffffffb388d8bc
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x1cc
12
__writeback_inodes_wb
vmlinux 0xffffffffb388dbc7
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xa7
13
wb_writeback
vmlinux 0xffffffffb388e0a3
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x263
14
bdi_writeback_workfn
vmlinux 0xffffffffb388ecaf
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x2cf
15
process_one_work
vmlinux 0xffffffffb36c32ef
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffffb36c4436
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffffb36c4310
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x2b0
18
kthread
vmlinux 0xffffffffb36cb621
Build id: 3.10.0-1160.105.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffffb36cb550
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffffb3dc51f7
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffffb36cb550
Build id: 3.10.0-1160.105.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 21s! [kworker/u256:1:92009]
Modules linked in: xt_REDIRECT nf_nat_redirect ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop devlink overlay ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel
lrw gf128mul pcspkr glue_helper ablk_helper ppdev cryptd vmw_balloon joydev sg parport_pc parport vmw_vmci i2c_piix4 ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi sd_mod crc_t10dif vmwgfx crct10dif_generic drm_kms_helper crct10dif_pclmul crct10dif_common crc32c_intel syscopyarea sysfillrect sysimgblt fb_sys_fops ttm serio_raw ata_piix drm nfit vmxnet3 libata libnvdimm vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 92009 Comm: kworker/u256:1 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.105.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 12/12/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff94246524d280 ti: ffff9423da580000 task.ti: ffff9423da580000
RIP: 0010:[<ffffffffb37c9546>]  [<ffffffffb37c9546>] find_get_pages_tag+0x76/0x240
RSP: 0000:ffff9423da5839b8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff9423da583948 RCX: 0000000000000000
RDX: 0000000000000000 RSI: fffff0d10753a51c RDI: fffff0d11001ff00
RBP: ffff9423da583a18 R08: ffff9423da583a98 R09: 0000000000000000
R10: 0000000000000400 R11: 0000000000000000 R12: 0000000000000010
R13: ffffffffb39a2381 R14: ffffffffffffff10 R15: ffff94255e2c6b38
FS:  0000000000000000(0000) GS:ffff94272b6c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f32d2fa6000 CR3: 000000028eade000 CR4: 00000000003607e0
Call Trace:
[<ffffffffb37d7c61>] pagevec_lookup_tag+0x21/0x30
[<ffffffffb37d590a>] write_cache_pages+0x11a/0x480
[<ffffffffc04a76c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffffb36ec6ab>] ? put_prev_entity+0x8b/0x400
[<ffffffffb36e8699>] ? pick_next_entity+0xa9/0x190
[<ffffffffb36e918c>] ? set_next_entity+0x3c/0x100
[<ffffffffc04a767b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffffb37d6ab1>] do_writepages+0x21/0x50
[<ffffffffb388ccf0>] __writeback_single_inode+0x40/0x260
[<ffffffffb388d8bc>] writeback_sb_inodes+0x1cc/0x430
[<ffffffffb388dbc7>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffffb388e0a3>] wb_writeback+0x263/0x2f0
[<ffffffffb388ecaf>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffffb36c32ef>] process_one_work+0x17f/0x440
[<ffffffffb36c4436>] worker_thread+0x126/0x3c0
[<ffffffffb36c4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffffb36cb621>] kthread+0xd1/0xe0
[<ffffffffb36cb550>] ? insert_kthread_work+0x40/0x40
[<ffffffffb3dc51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb36cb550>] ? insert_kthread_work+0x40/0x40
Code: 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 4d 01 00 00 <8b> 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c
Complete report #530105
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff859be8a6
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x76
2
pagevec_lookup_tag
vmlinux 0xffffffff859ccb71
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff859ca89a
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0416290
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa0
5
? __enqueue_entity
vmlinux 0xffffffff858dfc28
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x78
6
? put_prev_entity
vmlinux 0xffffffff858e3f6b
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x8b
7
? pick_next_entity
vmlinux 0xffffffff858e2469
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xa9
8
? set_next_entity
vmlinux 0xffffffff858e283c
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x3c
9
xfs_vm_writepages
xfs 0xffffffffc041625b
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x6b
10
do_writepages
vmlinux 0xffffffff859cb9f1
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x21
11
__writeback_single_inode
vmlinux 0xffffffff85a7df50
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x40
12
writeback_sb_inodes
vmlinux 0xffffffff85a7ead4
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x1c4
13
__writeback_inodes_wb
vmlinux 0xffffffff85a7eddf
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x9f
14
wb_writeback
vmlinux 0xffffffff85a7f2b3
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x263
15
bdi_writeback_workfn
vmlinux 0xffffffff85a7feab
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x2cb
16
process_one_work
vmlinux 0xffffffff858bdfbf
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x17f
17
worker_thread
vmlinux 0xffffffff858bf0d6
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x126
18
? manage_workers.isra.26
vmlinux 0xffffffff858befb0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x2a0
19
kthread
vmlinux 0xffffffff858c5f91
Build id: 3.10.0-1160.76.1.el7.x86_64 		0xd1
20
? insert_kthread_work
vmlinux 0xffffffff858c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x40
21
ret_from_fork_nospec_begin
vmlinux 0xffffffff85f99df7
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x21
22
? insert_kthread_work
vmlinux 0xffffffff858c5ec0
Build id: 3.10.0-1160.76.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#5 stuck for 25s! [kworker/u16:2:343]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon coretemp pcspkr joydev sg i2c_piix4 vmw_vmci parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops
ttm ata_piix drm libata mptspi scsi_transport_spi serio_raw e1000 mptscsih mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 5 PID: 343 Comm: kworker/u16:2 Not tainted 3.10.0-1160.76.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 01/07/2011
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9b2b71e7b180 ti: ffff9b2b71e8c000 task.ti: ffff9b2b71e8c000
RIP: 0010:[<ffffffff859be8a6>]  [<ffffffff859be8a6>] find_get_pages_tag+0x76/0x240
RSP: 0000:ffff9b2b71e8f9b8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff9b2b71e02848 RCX: 0000000000000019
RDX: 0000000000000001 RSI: 0000000000000000 RDI: fffff48e82da9640
RBP: ffff9b2b71e8fa18 R08: ffff9b2b71e8fa98 R09: ffff9b2b73f01f48
R10: 0000000000002c00 R11: ffff9b2b76892870 R12: ffff9b2b7fd6e000
R13: ffff9b2b71e1e4c0 R14: ffff9b2b00000030 R15: 1400000000000001
FS:  0000000000000000(0000) GS:ffff9b2b7fd40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fc5ceb30968 CR3: 0000000234a2c000 CR4: 00000000000007e0
Call Trace:
[<ffffffff859ccb71>] pagevec_lookup_tag+0x21/0x30
[<ffffffff859ca89a>] write_cache_pages+0x11a/0x470
[<ffffffffc0416290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff858dfc28>] ? __enqueue_entity+0x78/0x80
[<ffffffff858e3f6b>] ? put_prev_entity+0x8b/0x400
[<ffffffff858e2469>] ? pick_next_entity+0xa9/0x190
[<ffffffff858e283c>] ? set_next_entity+0x3c/0xe0
[<ffffffffc041625b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff859cb9f1>] do_writepages+0x21/0x50
[<ffffffff85a7df50>] __writeback_single_inode+0x40/0x260
[<ffffffff85a7ead4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff85a7eddf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff85a7f2b3>] wb_writeback+0x263/0x2f0
[<ffffffff85a7feab>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff858bdfbf>] process_one_work+0x17f/0x440
[<ffffffff858bf0d6>] worker_thread+0x126/0x3c0
[<ffffffff858befb0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff858c5f91>] kthread+0xd1/0xe0
[<ffffffff858c5ec0>] ? insert_kthread_work+0x40/0x40
[<ffffffff85f99df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff858c5ec0>] ? insert_kthread_work+0x40/0x40
Code: 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 <8b> 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c
Complete report #583145
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffaadbe842
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffffaadccb51
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffaadca85a
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc062d290
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xa0
5
? queue_io
dm_mod 0xffffffffc0276680
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x80
6
? cpumask_next_and
vmlinux 0xffffffffaaf86a55
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x35
7
xfs_vm_writepages
xfs 0xffffffffc062d25b
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x6b
8
do_writepages
vmlinux 0xffffffffaadcb9b1
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x21
9
__writeback_single_inode
vmlinux 0xffffffffaae7df10
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x40
10
writeback_sb_inodes
vmlinux 0xffffffffaae7ea94
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffffaae7ed9f
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffffaae7f273
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffffaae6ae6c
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffffaae7fe6b
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffffaacbde8f
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffffaacbefa6
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffffaacbee80
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffffaacc5e61
Build id: 3.10.0-1160.59.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffffaacc5d90
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffffab399df7
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffffaacc5d90
Build id: 3.10.0-1160.59.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u16:1:2388]
Modules linked in: veth xt_nat xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc intel_powerclamp coretemp intel_rapl kvm_intel kvm snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi irqbypass snd_hda_intel snd_hda_codec snd_hda_core iTCO_wdt snd_hwdep iTCO_vendor_support
eeepc_wmi asus_wmi sparse_keymap crc32_pclmul ghash_clmulni_intel aesni_intel lrw rfkill snd_seq snd_seq_device snd_pcm mei_me i2c_i801 gf128mul tpm_infineon glue_helper ablk_helper cryptd mei mxm_wmi snd_timer sg snd soundcore pcspkr lpc_ich wmi sch_fq_codel ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic i915 i2c_algo_bit iosf_mbi drm_kms_helper syscopyarea sysfillrect sysimgblt ahci fb_sys_fops libahci drm libata crct10dif_pclmul crct10dif_common crc32c_intel r8169 serio_raw drm_panel_orientation_quirks video dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 2388 Comm: kworker/u16:1 Kdump: loaded Tainted: G    B        L ------------   3.10.0-1160.59.1.el7.x86_64 #1
Hardware name: ASUS All Series/Z87-A, BIOS 1602 10/29/2013
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff97d139d4d280 ti: ffff97ccc23a0000 task.ti: ffff97ccc23a0000
RIP: 0010:[<ffffffffaadbe842>]  [<ffffffffaadbe842>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff97ccc23a39b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff97cee72d4000 RCX: 000000000000001b
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffe0c4c7b7e9c0
RBP: ffff97ccc23a3a18 R08: ffff97ccc23a3a98 R09: ffff97d1ac028508
R10: 0000000000007000 R11: 000000000000049a R12: ffff97ccc23a3968
R13: 0000000000000286 R14: ffff97cee72d46c8 R15: 0000000000000286
FS:  0000000000000000(0000) GS:ffff97d1dfa80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000036df7f0 CR3: 0000000227010000 CR4: 00000000001607e0
Call Trace:
[<ffffffffaadccb51>] pagevec_lookup_tag+0x21/0x30
[<ffffffffaadca85a>] write_cache_pages+0x11a/0x470
[<ffffffffc062d290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc0276680>] ? queue_io+0x80/0x80 [dm_mod]
[<ffffffffaaf86a55>] ? cpumask_next_and+0x35/0x50
[<ffffffffc062d25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffaadcb9b1>] do_writepages+0x21/0x50
[<ffffffffaae7df10>] __writeback_single_inode+0x40/0x260
[<ffffffffaae7ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffaae7ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffaae7f273>] wb_writeback+0x263/0x2f0
[<ffffffffaae6ae6c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffaae7fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffaacbde8f>] process_one_work+0x17f/0x440
[<ffffffffaacbefa6>] worker_thread+0x126/0x3c0
[<ffffffffaacbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffaacc5e61>] kthread+0xd1/0xe0
[<ffffffffaacc5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffab399df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaacc5d90>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #973807
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff8d1974d2
Build id: 3.10.0-862.14.4.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffff8d1a4ce1
Build id: 3.10.0-862.14.4.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff8d1a270a
Build id: 3.10.0-862.14.4.el7.x86_64 		0x15a
4
? xfs_aops_discard_page
xfs 0xffffffffc057a850
Build id: 3.10.0-862.14.4.el7.x86_64 		0x150
5
? generic_make_request
vmlinux 0xffffffff8d31f1cb
Build id: 3.10.0-862.14.4.el7.x86_64 		0x10b
6
? submit_bio
vmlinux 0xffffffff8d31f450
Build id: 3.10.0-862.14.4.el7.x86_64 		0x70
7
xfs_vm_writepages
xfs 0xffffffffc057a65d
Build id: 3.10.0-862.14.4.el7.x86_64 		0xbd
8
do_writepages
vmlinux 0xffffffff8d1a3b81
Build id: 3.10.0-862.14.4.el7.x86_64 		0x21
9
__writeback_single_inode
vmlinux 0xffffffff8d24cfc0
Build id: 3.10.0-862.14.4.el7.x86_64 		0x40
10
writeback_sb_inodes
vmlinux 0xffffffff8d24da54
Build id: 3.10.0-862.14.4.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffff8d24ddbf
Build id: 3.10.0-862.14.4.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffff8d24e5f3
Build id: 3.10.0-862.14.4.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffff8d23ab0c
Build id: 3.10.0-862.14.4.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffff8d24ef7b
Build id: 3.10.0-862.14.4.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffff8d0b613f
Build id: 3.10.0-862.14.4.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffff8d0b71d6
Build id: 3.10.0-862.14.4.el7.x86_64 		0x126
17
? manage_workers.isra.24
vmlinux 0xffffffff8d0b70b0
Build id: 3.10.0-862.14.4.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffff8d0bdf21
Build id: 3.10.0-862.14.4.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffff8d0bde50
Build id: 3.10.0-862.14.4.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffff8d7255dd
Build id: 3.10.0-862.14.4.el7.x86_64 		0x7
21
? insert_kthread_work
vmlinux 0xffffffff8d0bde50
Build id: 3.10.0-862.14.4.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kworker/u224:1:15362]
Modules linked in: fuse rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache ip6t_rpfilter ipt_REJECT nf_reject_ipv4 ip6t_REJECT nf_reject_ipv6 bonding xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) sunrpc dm_mirror dm_region_hash dm_log dm_mod vfat fat intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel kvm irqbypass snd_hda_codec_hdmi snd_hda_codec_realtek crc32_pclmul ghash_clmulni_intel joydev snd_hda_codec_generic
aesni_intel snd_hda_intel lrw snd_hda_codec gf128mul glue_helper snd_hda_core ablk_helper eeepc_wmi cryptd snd_hwdep asus_wmi sparse_keymap snd_seq iTCO_wdt rfkill iTCO_vendor_support snd_seq_device snd_pcm sg pcspkr snd_timer snd soundcore mei_me shpchp i2c_i801 mei sch_fq_codel binfmt_misc ip_tables xfs libcrc32c uas usb_storage sr_mod sd_mod cdrom crc_t10dif crct10dif_generic nouveau video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm mxm_wmi ahci e1000e libahci crct10dif_pclmul crct10dif_common crc32c_intel libata i2c_core serio_raw r8169 ptp pps_core mii wmi
CPU: 4 PID: 15362 Comm: kworker/u224:1 Tainted: G    B   W  OE  ------------   3.10.0-862.14.4.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/PRIME X299-A, BIOS 1401 05/21/2018
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff9a9b6c82eeb0 ti: ffff9aa92e65c000 task.ti: ffff9aa92e65c000
RIP: 0010:[<ffffffff8d1974d2>]  [<ffffffff8d1974d2>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff9aa92e65f9a0  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9aa92e65f968 RCX: 0000000000000027
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffdf8dc1443500
RBP: ffff9aa92e65fa00 R08: ffff9aa92e65fa90 R09: ffff9aa99bcd5f20
R10: 0000000000003400 R11: 000008cbf344f5c0 R12: ffff9aa9970c1c88
R13: 0000000000000286 R14: ffff9aa9970c0000 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffff9aa99df00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002afc03e84000 CR3: 000000022c60e000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff8d1a4ce1>] pagevec_lookup_tag+0x21/0x30
[<ffffffff8d1a270a>] write_cache_pages+0x15a/0x4e0
[<ffffffffc057a850>] ? xfs_aops_discard_page+0x150/0x150 [xfs]
[<ffffffff8d31f1cb>] ? generic_make_request+0x10b/0x320
[<ffffffff8d31f450>] ? submit_bio+0x70/0x150
[<ffffffffc057a65d>] xfs_vm_writepages+0xbd/0xe0 [xfs]
[<ffffffff8d1a3b81>] do_writepages+0x21/0x50
[<ffffffff8d24cfc0>] __writeback_single_inode+0x40/0x260
[<ffffffff8d24da54>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8d24ddbf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8d24e5f3>] wb_writeback+0x263/0x2f0
[<ffffffff8d23ab0c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8d24ef7b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff8d0b613f>] process_one_work+0x17f/0x440
[<ffffffff8d0b71d6>] worker_thread+0x126/0x3c0
[<ffffffff8d0b70b0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff8d0bdf21>] kthread+0xd1/0xe0
[<ffffffff8d0bde50>] ? insert_kthread_work+0x40/0x40
[<ffffffff8d7255dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff8d0bde50>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #832197
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff8cdc94e6
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x86
2
pagevec_lookup_tag
vmlinux 0xffffffff8cdd7bf1
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff8cdd589a
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc05446c0
Build id: 3.10.0-1160.90.1.el7.x86_64 		0xb0
5
xfs_vm_writepages
xfs 0xffffffffc054467b
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x6b
6
do_writepages
vmlinux 0xffffffff8cdd6a41
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x21
7
__writeback_single_inode
vmlinux 0xffffffff8ce8cc80
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x40
8
writeback_sb_inodes
vmlinux 0xffffffff8ce8d84c
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x1cc
9
__writeback_inodes_wb
vmlinux 0xffffffff8ce8db57
Build id: 3.10.0-1160.90.1.el7.x86_64 		0xa7
10
wb_writeback
vmlinux 0xffffffff8ce8e033
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x263
11
? get_nr_inodes
vmlinux 0xffffffff8ce78ffc
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x4c
12
bdi_writeback_workfn
vmlinux 0xffffffff8ce8ec3f
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x2cf
13
process_one_work
vmlinux 0xffffffff8ccc32ef
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x17f
14
worker_thread
vmlinux 0xffffffff8ccc4436
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x126
15
? manage_workers.isra.26
vmlinux 0xffffffff8ccc4310
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x2b0
16
kthread
vmlinux 0xffffffff8cccb621
Build id: 3.10.0-1160.90.1.el7.x86_64 		0xd1
17
? insert_kthread_work
vmlinux 0xffffffff8cccb550
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x40
18
ret_from_fork_nospec_begin
vmlinux 0xffffffff8d3c51dd
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x7
19
? insert_kthread_work
vmlinux 0xffffffff8cccb550
Build id: 3.10.0-1160.90.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [kworker/u8:0:16348]
Modules linked in: falcon_lsm_serviceable(PE) falcon_nf_netcontain(PE) falcon_kal(E) falcon_lsm_pinned_15805(E) vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel ppdev lrw vmw_balloon gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg parport_pc i2c_piix4 vmw_vmci parport sch_fq_codel binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mptspi scsi_transport_spi ata_piix drm mptscsih nfit crct10dif_pclmul crct10dif_common vmxnet3 mptbase serio_raw crc32c_intel libata drm_panel_orientation_quirks libnvdimm floppy dm_mirror dm_region_hash dm_log dm_mod
CPU: 2 PID: 16348 Comm: kworker/u8:0 Kdump: loaded Tainted: P            E  ------------   3.10.0-1160.90.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 11/12/2020
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff931dc4a94200 ti: ffff931935f2c000 task.ti: ffff931935f2c000
RIP: 0010:[<ffffffff8cdc94e6>]  [<ffffffff8cdc94e6>] find_get_pages_tag+0x86/0x240
RSP: 0000:ffff931935f2f9b8  EFLAGS: 00010202
RAX: 0000000000000002 RBX: ffff931935f2f940 RCX: 0000000000000003
RDX: 0000000000000002 RSI: ffffefb254f0501c RDI: ffffefb254f05000
RBP: ffff931935f2fa18 R08: ffff931935f2fa98 R09: 0000000000000000
R10: 0000000000003400 R11: 0000000000000000 R12: 0000000000000000
R13: ffff931dffd9ad40 R14: ffff931dffd9ad40 R15: ffffffff8cce8cc4
FS:  0000000000000000(0000) GS:ffff931dffd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 00000000bab24000 CR4: 00000000007607e0
PKRU: 00000000
Call Trace:
[<ffffffff8cdd7bf1>] pagevec_lookup_tag+0x21/0x30
[<ffffffff8cdd589a>] write_cache_pages+0x11a/0x480
[<ffffffffc05446c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffffc054467b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffff8cdd6a41>] do_writepages+0x21/0x50
[<ffffffff8ce8cc80>] __writeback_single_inode+0x40/0x260
[<ffffffff8ce8d84c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffff8ce8db57>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffff8ce8e033>] wb_writeback+0x263/0x2f0
[<ffffffff8ce78ffc>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8ce8ec3f>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffff8ccc32ef>] process_one_work+0x17f/0x440
[<ffffffff8ccc4436>] worker_thread+0x126/0x3c0
[<ffffffff8ccc4310>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffff8cccb621>] kthread+0xd1/0xe0
[<ffffffff8cccb550>] ? insert_kthread_work+0x40/0x40
[<ffffffff8d3c51dd>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffff8cccb550>] ? insert_kthread_work+0x40/0x40
Code: db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 4d 01 00 00 8b 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 <f0> 0f b1 4f 1c 39 d0 89 c1 0f 85 42 01 00 00 48 39 3b 0f 85 95
Complete report #756214
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffbe1b6926
Build id: 3.10.0-957.el7.x86_64 		0x86
2
pagevec_lookup_tag
vmlinux 0xffffffffbe1c4331
Build id: 3.10.0-957.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffbe1c1f3a
Build id: 3.10.0-957.el7.x86_64 		0x15a
4
? xfs_aops_discard_page
xfs 0xffffffffc05cf5b0
Build id: 3.10.0-957.el7.x86_64 		0x150
5
xfs_vm_writepages
xfs 0xffffffffc05cf3ab
Build id: 3.10.0-957.el7.x86_64 		0x6b
6
do_writepages
vmlinux 0xffffffffbe1c3181
Build id: 3.10.0-957.el7.x86_64 		0x21
7
__writeback_single_inode
vmlinux 0xffffffffbe26f930
Build id: 3.10.0-957.el7.x86_64 		0x40
8
writeback_sb_inodes
vmlinux 0xffffffffbe2703c4
Build id: 3.10.0-957.el7.x86_64 		0x1c4
9
__writeback_inodes_wb
vmlinux 0xffffffffbe27072f
Build id: 3.10.0-957.el7.x86_64 		0x9f
10
wb_writeback
vmlinux 0xffffffffbe270f63
Build id: 3.10.0-957.el7.x86_64 		0x263
11
? get_nr_inodes
vmlinux 0xffffffffbe25cd7c
Build id: 3.10.0-957.el7.x86_64 		0x4c
12
bdi_writeback_workfn
vmlinux 0xffffffffbe2718eb
Build id: 3.10.0-957.el7.x86_64 		0x2cb
13
process_one_work
vmlinux 0xffffffffbe0b9d4f
Build id: 3.10.0-957.el7.x86_64 		0x17f
14
worker_thread
vmlinux 0xffffffffbe0bade6
Build id: 3.10.0-957.el7.x86_64 		0x126
15
? manage_workers.isra.25
vmlinux 0xffffffffbe0bacc0
Build id: 3.10.0-957.el7.x86_64 		0x2a0
16
kthread
vmlinux 0xffffffffbe0c1c31
Build id: 3.10.0-957.el7.x86_64 		0xd1
17
? insert_kthread_work
vmlinux 0xffffffffbe0c1b60
Build id: 3.10.0-957.el7.x86_64 		0x40
18
ret_from_fork_nospec_begin
vmlinux 0xffffffffbe774c1d
Build id: 3.10.0-957.el7.x86_64 		0x7
19
? insert_kthread_work
vmlinux 0xffffffffbe0c1b60
Build id: 3.10.0-957.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 24s! [kworker/u4:0:46795]
Modules linked in: ipt_rpfilter xt_multiport ip_set_hash_ip ip_set_hash_net ipip tunnel4 ip_tunnel veth xt_set ip_set_bitmap_port ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport nf_tables(T) ip_set ip_vs_sh ip_vs_wrr ip_vs_rr ip_vs dummy iptable_raw xt_CT ip6table_mangle ip6t_MASQUERADE nf_nat_masquerade_ipv6 ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 xt_comment xt_mark nf_conntrack_netlink nfnetlink xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat xt_conntrack ipt_REJECT nf_reject_ipv4 tun ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter vmw_vsock_vmci_transport vsock sunrpc iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd
vmw_balloon pcspkr joydev sg i2c_piix4 vmw_vmci parport_pc parport overlay(T) br_netfilter bridge stp llc nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack ip_tables xfs libcrc32c sr_mod cdrom sd_mod crc_t10dif crct10dif_generic crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ahci nfit libahci vmw_pvscsi libnvdimm drm_panel_orientation_quirks ata_generic pata_acpi ata_piix libata dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 46795 Comm: kworker/u4:0 Kdump: loaded Tainted: G             L ------------ T 3.10.0-957.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff910e6b886180 ti: ffff910f38580000 task.ti: ffff910f38580000
RIP: 0010:[<ffffffffbe1b6926>]  [<ffffffffbe1b6926>] find_get_pages_tag+0x86/0x240
RSP: 0000:ffff910f385839a8  EFLAGS: 00010206
RAX: 0000000000000003 RBX: ffff910f33c26470 RCX: 0000000000000004
RDX: 0000000000000003 RSI: ffffe2c304afda1c RDI: ffffe2c304afda00
RBP: ffff910f38583a08 R08: ffff910f38583a98 R09: 0000000000000000
R10: 74f3d1524d5c2dc0 R11: 0000000000000000 R12: 0000000000000000
R13: 000000005da2094d R14: ffffe2c3045c2dc0 R15: ffffffffbe37b6de
FS:  0000000000000000(0000) GS:ffff910f3fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f77ad057000 CR3: 0000000137530000 CR4: 00000000007607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 00000000
Call Trace:
[<ffffffffbe1c4331>] pagevec_lookup_tag+0x21/0x30
[<ffffffffbe1c1f3a>] write_cache_pages+0x15a/0x4e0
[<ffffffffc05cf5b0>] ? xfs_aops_discard_page+0x150/0x150 [xfs]
[<ffffffffc05cf3ab>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffbe1c3181>] do_writepages+0x21/0x50
[<ffffffffbe26f930>] __writeback_single_inode+0x40/0x260
[<ffffffffbe2703c4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffffbe27072f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffbe270f63>] wb_writeback+0x263/0x2f0
[<ffffffffbe25cd7c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffbe2718eb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffbe0b9d4f>] process_one_work+0x17f/0x440
[<ffffffffbe0bade6>] worker_thread+0x126/0x3c0
[<ffffffffbe0bacc0>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffffbe0c1c31>] kthread+0xd1/0xe0
[<ffffffffbe0c1b60>] ? insert_kthread_work+0x40/0x40
[<ffffffffbe774c1d>] ret_from_fork_nospec_begin+0x7/0x21
[<ffffffffbe0c1b60>] ? insert_kthread_work+0x40/0x40
Code: db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 <f0> 0f b1 4f 1c 39 d0 89 c1 0f 85 3b 01 00 00 48 39 3b 0f 85 8d
Complete report #648357
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffa6fc93b0
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x70
2
pagevec_lookup_tag
vmlinux 0xffffffffa6fd7ad1
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffa6fd577a
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc07a26c0
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xb0
5
? cpumask_next_and
vmlinux 0xffffffffa719bf95
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x35
6
xfs_vm_writepages
xfs 0xffffffffc07a267b
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffffa6fd6921
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffffa708c790
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40
9
writeback_sb_inodes
vmlinux 0xffffffffa708d35c
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x1cc
10
__writeback_inodes_wb
vmlinux 0xffffffffa708d667
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xa7
11
wb_writeback
vmlinux 0xffffffffa708db43
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x263
12
? bdi_dirty_limit
vmlinux 0xffffffffa6fd6060
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40
13
bdi_writeback_workfn
vmlinux 0xffffffffa708e64c
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x1cc
14
process_one_work
vmlinux 0xffffffffa6ec31df
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x17f
15
worker_thread
vmlinux 0xffffffffa6ec4326
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x126
16
? manage_workers.isra.26
vmlinux 0xffffffffa6ec4200
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x2b0
17
kthread
vmlinux 0xffffffffa6ecb511
Build id: 3.10.0-1160.83.1.el7.x86_64 		0xd1
18
? insert_kthread_work
vmlinux 0xffffffffa6ecb440
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40
19
ret_from_fork_nospec_begin
vmlinux 0xffffffffa75c51f7
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x21
20
? insert_kthread_work
vmlinux 0xffffffffa6ecb440
Build id: 3.10.0-1160.83.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u32:0:8727]
Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables binfmt_misc ip_set_hash_net bridge stp llc ip_set squashfs loop snd_hda_codec_hdmi snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq iTCO_wdt intel_powerclamp iTCO_vendor_support coretemp snd_seq_device kvm_intel snd_pcm kvm snd_timer sg snd irqbypass pcspkr soundcore i2c_i801 lpc_ich i7core_edac asus_atk0110 acpi_cpufreq tpm_infineon xfs libcrc32c sr_mod
cdrom ata_generic pata_acpi raid1 pata_marvell sd_mod crc_t10dif crct10dif_generic crct10dif_common nouveau video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm mxm_wmi libahci libata crc32c_intel serio_raw sky2 drm_panel_orientation_quirks wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: ip_tables]
CPU: 0 PID: 8727 Comm: kworker/u32:0 Kdump: loaded Tainted: G    B     I  L ------------   3.10.0-1160.83.1.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/P6T DELUXE, BIOS 2209    09/21/2010
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff8d56b5aeb180 ti: ffff8d56814c4000 task.ti: ffff8d56814c4000
RIP: 0010:[<ffffffffa6fc93b0>]  [<ffffffffa6fc93b0>] find_get_pages_tag+0x70/0x240
RSP: 0018:ffff8d56814c79b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffc03766f7 RCX: 0000000000000001
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffeb14053ccb40
RBP: ffff8d56814c7a18 R08: ffff8d56814c7a98 R09: ffff8d56dd5a3208
R10: 0000000000003400 R11: 0000000000000000 R12: ffff8d583b0c6718
R13: ffffffffc0721caa R14: ffff8d56814c79c8 R15: ffff8d56b5e06000
FS:  0000000000000000(0000) GS:ffff8d583f200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fe7e95c7d70 CR3: 00000001ba410000 CR4: 00000000000007f0
Call Trace:
[<ffffffffa6fd7ad1>] pagevec_lookup_tag+0x21/0x30
[<ffffffffa6fd577a>] write_cache_pages+0x11a/0x480
[<ffffffffc07a26c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffffa719bf95>] ? cpumask_next_and+0x35/0x50
[<ffffffffc07a267b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffffa6fd6921>] do_writepages+0x21/0x50
[<ffffffffa708c790>] __writeback_single_inode+0x40/0x260
[<ffffffffa708d35c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffffa708d667>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffffa708db43>] wb_writeback+0x263/0x2f0
[<ffffffffa6fd6060>] ? bdi_dirty_limit+0x40/0xe0
[<ffffffffa708e64c>] bdi_writeback_workfn+0x1cc/0x470
[<ffffffffa6ec31df>] process_one_work+0x17f/0x440
[<ffffffffa6ec4326>] worker_thread+0x126/0x3c0
[<ffffffffa6ec4200>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffffa6ecb511>] kthread+0xd1/0xe0
[<ffffffffa6ecb440>] ? insert_kthread_work+0x40/0x40
[<ffffffffa75c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa6ecb440>] ? insert_kthread_work+0x40/0x40
Code: 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 <0f> 85 4d 01 00 00 8b 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89
Complete report #622426
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffaedc9376
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x76
2
pagevec_lookup_tag
vmlinux 0xffffffffaedd7a91
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffaedd573a
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc06446c0
Build id: 3.10.0-1160.80.1.el7.x86_64 		0xb0
5
? submit_bio
vmlinux 0xffffffffaef6a120
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc064467b
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffffaedd68e1
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffffaee8c750
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffffaeccc5f5
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffffaee8d31c
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x1cc
11
__writeback_inodes_wb
vmlinux 0xffffffffaee8d627
Build id: 3.10.0-1160.80.1.el7.x86_64 		0xa7
12
wb_writeback
vmlinux 0xffffffffaee8db03
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffffaee78a7c
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffffaee8e70f
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x2cf
15
process_one_work
vmlinux 0xffffffffaecc319f
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffffaecc42e6
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffffaecc41c0
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x2b0
18
kthread
vmlinux 0xffffffffaeccb4d1
Build id: 3.10.0-1160.80.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffffaeccb400
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffffaf3c51f7
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffffaeccb400
Build id: 3.10.0-1160.80.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#6 stuck for 21s! [kworker/u16:0:26171]
Modules linked in: nf_log_ipv4 nf_log_common xt_CHECKSUM xt_DSCP bridge stp llc xt_conntrack ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter cls_u32 sch_sfq sch_htb tun n_hdlc ppp_synctty ppp_generic slhc iptable_nat nf_nat_ipv4 xt_mac xt_mark xt_limit xt_dscp iptable_mangle xt_multiport xt_state ipt_MASQUERADE nf_nat_masquerade_ipv4 ipt_REJECT nf_reject_ipv4 xt_LOG nf_nat_ftp nf_nat nf_conntrack_ftp nf_conntrack_ipv4 nf_defrag_ipv4 arc4 nf_conntrack md4 nls_utf8 cifs binfmt_misc ccm dns_resolver vmw_vsock_vmci_transport vsock sunrpc ppdev vmw_balloon joydev coretemp vmw_vmci pcspkr sg i2c_piix4 parport_pc parport ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect
sysimgblt fb_sys_fops ttm ata_piix libata drm mptspi scsi_transport_spi serio_raw mptscsih vmxnet3 mptbase drm_panel_orientation_quirks floppy dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 6 PID: 26171 Comm: kworker/u16:0 Kdump: loaded Tainted: G             L ------------   3.10.0-1160.80.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/21/2015
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8f06cfd13180 ti: ffff8f07e5c08000 task.ti: ffff8f07e5c08000
RIP: 0010:[<ffffffffaedc9376>]  [<ffffffffaedc9376>] find_get_pages_tag+0x76/0x240
RSP: 0000:ffff8f07e5c0b9b8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff8f08b97b9048 RCX: 0000000000000001
RDX: 0000000000000001 RSI: 0000000000000040 RDI: fffff42d44daeec0
RBP: ffff8f07e5c0ba18 R08: ffff8f07e5c0ba98 R09: ffff8f06be061f48
R10: 0000000000000400 R11: 0000000000000400 R12: ffff8f08bfd2c000
R13: ffff8f08765b8380 R14: ffff8f0700000030 R15: 1400000000000001
FS:  0000000000000000(0000) GS:ffff8f08bfd80000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000006ec0f8 CR3: 000000001fde8000 CR4: 00000000000007e0
Call Trace:
[<ffffffffaedd7a91>] pagevec_lookup_tag+0x21/0x30
[<ffffffffaedd573a>] write_cache_pages+0x11a/0x480
[<ffffffffc06446c0>] ? xfs_vm_writepages+0xb0/0xb0 [xfs]
[<ffffffffaef6a120>] ? submit_bio+0x70/0x150
[<ffffffffc064467b>] xfs_vm_writepages+0x6b/0xb0 [xfs]
[<ffffffffaedd68e1>] do_writepages+0x21/0x50
[<ffffffffaee8c750>] __writeback_single_inode+0x40/0x260
[<ffffffffaeccc5f5>] ? wake_up_bit+0x25/0x30
[<ffffffffaee8d31c>] writeback_sb_inodes+0x1cc/0x430
[<ffffffffaee8d627>] __writeback_inodes_wb+0xa7/0xe0
[<ffffffffaee8db03>] wb_writeback+0x263/0x2f0
[<ffffffffaee78a7c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffaee8e70f>] bdi_writeback_workfn+0x2cf/0x470
[<ffffffffaecc319f>] process_one_work+0x17f/0x440
[<ffffffffaecc42e6>] worker_thread+0x126/0x3c0
[<ffffffffaecc41c0>] ? manage_workers.isra.26+0x2b0/0x2b0
[<ffffffffaeccb4d1>] kthread+0xd1/0xe0
[<ffffffffaeccb400>] ? insert_kthread_work+0x40/0x40
[<ffffffffaf3c51f7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaeccb400>] ? insert_kthread_work+0x40/0x40
Code: 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 4d 01 00 00 <8b> 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c
Complete report #499794
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff9e7be866
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x86
2
pagevec_lookup_tag
vmlinux 0xffffffff9e7ccb51
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff9e7ca85a
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc02ec290
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xa0
5
xfs_vm_writepages
xfs 0xffffffffc02ec25b
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x6b
6
do_writepages
vmlinux 0xffffffff9e7cb9b1
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x21
7
__writeback_single_inode
vmlinux 0xffffffff9e87def0
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40
8
? wake_up_bit
vmlinux 0xffffffff9e6c6f15
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x25
9
writeback_sb_inodes
vmlinux 0xffffffff9e87ea74
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x1c4
10
__writeback_inodes_wb
vmlinux 0xffffffff9e87ed7f
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x9f
11
wb_writeback
vmlinux 0xffffffff9e87f253
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x263
12
bdi_writeback_workfn
vmlinux 0xffffffff9e87fe4b
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x2cb
13
process_one_work
vmlinux 0xffffffff9e6bde8f
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x17f
14
worker_thread
vmlinux 0xffffffff9e6befa6
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x126
15
? manage_workers.isra.26
vmlinux 0xffffffff9e6bee80
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x2a0
16
kthread
vmlinux 0xffffffff9e6c5e61
Build id: 3.10.0-1160.41.1.el7.x86_64 		0xd1
17
? SyS_futex
vmlinux 0xffffffff9e7156f0
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x80
18
? insert_kthread_work
vmlinux 0xffffffff9e6c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40
19
ret_from_fork_nospec_begin
vmlinux 0xffffffff9ed95df7
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x21
20
? insert_kthread_work
vmlinux 0xffffffff9e6c5d90
Build id: 3.10.0-1160.41.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 21s! [kworker/u20:1:23055]
Modules linked in: xt_nat veth nf_conntrack_netlink xt_addrtype br_netfilter xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink overlay(T) ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport vsock sunrpc sb_edac iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper ppdev cryptd sg vmw_balloon pcspkr joydev vmw_vmci parport_pc parport i2c_piix4 ip_tables
xfs libcrc32c sr_mod cdrom ata_generic pata_acpi vmwgfx drm_kms_helper ata_piix syscopyarea sysfillrect sysimgblt fb_sys_fops nfit ttm sd_mod libata libnvdimm crc_t10dif crct10dif_generic drm crct10dif_pclmul crct10dif_common crc32c_intel serio_raw vmxnet3 vmw_pvscsi drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 2 PID: 23055 Comm: kworker/u20:1 Kdump: loaded Tainted: G             L ------------ T 3.10.0-1160.41.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8c94d2be1080 ti: ffff8c8f07934000 task.ti: ffff8c8f07934000
RIP: 0010:[<ffffffff9e7be866>]  [<ffffffff9e7be866>] find_get_pages_tag+0x86/0x240
RSP: 0000:ffff8c8f079379b8  EFLAGS: 00010202
RAX: 0000000000000002 RBX: ffff8c9c6e54f6d8 RCX: 0000000000000003
RDX: 0000000000000002 RSI: fffff3fbcf0a281c RDI: fffff3fbcf0a2800
RBP: ffff8c8f07937a18 R08: ffff8c8f07937a98 R09: 0000000000000000
R10: 30118f146c39cb40 R11: 0000000000000000 R12: 0000000000000000
R13: 00000000140f024c R14: fffff3fb9939cb40 R15: ffffffff9e98ccbe
FS:  0000000000000000(0000) GS:ffff8ca31f100000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000036ae01162000 CR3: 00000001dc504000 CR4: 00000000003607e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff9e7ccb51>] pagevec_lookup_tag+0x21/0x30
[<ffffffff9e7ca85a>] write_cache_pages+0x11a/0x470
[<ffffffffc02ec290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc02ec25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff9e7cb9b1>] do_writepages+0x21/0x50
[<ffffffff9e87def0>] __writeback_single_inode+0x40/0x260
[<ffffffff9e6c6f15>] ? wake_up_bit+0x25/0x30
[<ffffffff9e87ea74>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9e87ed7f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9e87f253>] wb_writeback+0x263/0x2f0
[<ffffffff9e87fe4b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff9e6bde8f>] process_one_work+0x17f/0x440
[<ffffffff9e6befa6>] worker_thread+0x126/0x3c0
[<ffffffff9e6bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff9e6c5e61>] kthread+0xd1/0xe0
[<ffffffff9e7156f0>] ? SyS_futex+0x80/0x190
[<ffffffff9e6c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffff9ed95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff9e6c5d90>] ? insert_kthread_work+0x40/0x40
Code: db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 <f0> 0f b1 4f 1c 39 d0 89 c1 0f 85 3b 01 00 00 48 39 3b 0f 85 8d
Complete report #499391
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffaf5be622
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffffaf5cc1a1
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffaf5c9eaa
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0433290
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa0
5
xfs_vm_writepages
xfs 0xffffffffc043325b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x6b
6
do_writepages
vmlinux 0xffffffffaf5cb001
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
7
__writeback_single_inode
vmlinux 0xffffffffaf67d9a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
8
? wake_up_bit
vmlinux 0xffffffffaf4c6e55
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x25
9
writeback_sb_inodes
vmlinux 0xffffffffaf67e524
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x1c4
10
__writeback_inodes_wb
vmlinux 0xffffffffaf67e82f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x9f
11
wb_writeback
vmlinux 0xffffffffaf67ed03
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x263
12
bdi_writeback_workfn
vmlinux 0xffffffffaf67f8fb
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x2cb
13
process_one_work
vmlinux 0xffffffffaf4bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x17f
14
worker_thread
vmlinux 0xffffffffaf4beee6
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x126
15
? manage_workers.isra.26
vmlinux 0xffffffffaf4bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x2a0
16
kthread
vmlinux 0xffffffffaf4c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xd1
17
? insert_kthread_work
vmlinux 0xffffffffaf4c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
18
ret_from_fork_nospec_begin
vmlinux 0xffffffffafb95df7
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
19
? insert_kthread_work
vmlinux 0xffffffffaf4c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/u8:0:950]
Modules linked in: tcp_lp xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables devlink ip6table_filter ip6_tables iptable_filter vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) sunrpc snd_hda_codec_via snd_hda_codec_generic snd_hda_intel snd_hda_codec intel_powerclamp coretemp intel_rapl iTCO_wdt iTCO_vendor_support kvm_intel kvm snd_hda_core irqbypass snd_hwdep snd_seq crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper snd_seq_device snd_pcm i2c_i801 cryptd pcspkr snd_timer snd soundcore mei_me mei ie31200_edac lpc_ich sg ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic
i915 i2c_algo_bit iosf_mbi drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ahci drm libahci crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw atl1c drm_panel_orientation_quirks video dm_mirror dm_region_hash dm_log dm_mod fuse [last unloaded: ipmi_msghandler]
CPU: 1 PID: 950 Comm: kworker/u8:0 Kdump: loaded Tainted: G        W  OEL ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: PCWARE IPMH61R3/IPMH61R3, BIOS 4.6.5 03/07/2013
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff930f537da100 ti: ffff9311062a0000 task.ti: ffff9311062a0000
RIP: 0010:[<ffffffffaf5be622>]  [<ffffffffaf5be622>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff9311062a39b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9312754fc298 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffe4eecc81a5dc RDI: ffffe4eecc827600
RBP: ffff9311062a3a18 R08: ffff9311062a3a98 R09: 0000000000000000
R10: 6c44ccd2da06fb00 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000014553d99 R14: ffffe4eec606fb00 R15: ffffffffaf78c62e
FS:  0000000000000000(0000) GS:ffff93129f280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000b7fb3000 CR3: 0000000383010000 CR4: 00000000001627e0
Call Trace:
[<ffffffffaf5cc1a1>] pagevec_lookup_tag+0x21/0x30
[<ffffffffaf5c9eaa>] write_cache_pages+0x11a/0x470
[<ffffffffc0433290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc043325b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffaf5cb001>] do_writepages+0x21/0x50
[<ffffffffaf67d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffffaf4c6e55>] ? wake_up_bit+0x25/0x30
[<ffffffffaf67e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffaf67e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffaf67ed03>] wb_writeback+0x263/0x2f0
[<ffffffffaf67f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffaf4bddcf>] process_one_work+0x17f/0x440
[<ffffffffaf4beee6>] worker_thread+0x126/0x3c0
[<ffffffffaf4bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffaf4c5da1>] kthread+0xd1/0xe0
[<ffffffffaf4c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffafb95df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffaf4c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #448435
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff81182d26
Build id: 3.10.0-693.el7.x86_64 		0x86
2
pagevec_lookup_tag
vmlinux 0xffffffff81190111
Build id: 3.10.0-693.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff8118db9a
Build id: 3.10.0-693.el7.x86_64 		0x15a
4
? xfs_aops_discard_page
xfs 0xffffffffc0268980
Build id: 3.10.0-693.el7.x86_64 		0x150
5
xfs_vm_writepages
xfs 0xffffffffc026878d
Build id: 3.10.0-693.el7.x86_64 		0xbd
6
do_writepages
vmlinux 0xffffffff8118effe
Build id: 3.10.0-693.el7.x86_64 		0x1e
7
__writeback_single_inode
vmlinux 0xffffffff8122d7d0
Build id: 3.10.0-693.el7.x86_64 		0x40
8
writeback_sb_inodes
vmlinux 0xffffffff8122e414
Build id: 3.10.0-693.el7.x86_64 		0x1c4
9
__writeback_inodes_wb
vmlinux 0xffffffff8122e77f
Build id: 3.10.0-693.el7.x86_64 		0x9f
10
wb_writeback
vmlinux 0xffffffff8122efb3
Build id: 3.10.0-693.el7.x86_64 		0x263
11
bdi_writeback_workfn
vmlinux 0xffffffff8122f45b
Build id: 3.10.0-693.el7.x86_64 		0x2cb
12
process_one_work
vmlinux 0xffffffff810a881a
Build id: 3.10.0-693.el7.x86_64 		0x17a
13
worker_thread
vmlinux 0xffffffff810a94e6
Build id: 3.10.0-693.el7.x86_64 		0x126
14
? manage_workers.isra.24
vmlinux 0xffffffff810a93c0
Build id: 3.10.0-693.el7.x86_64 		0x2a0
15
kthread
vmlinux 0xffffffff810b098f
Build id: 3.10.0-693.el7.x86_64 		0xcf
16
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40
17
ret_from_fork
vmlinux 0xffffffff816b4f18
Build id: 3.10.0-693.el7.x86_64 		0x58
18
? insert_kthread_work
vmlinux 0xffffffff810b08c0
Build id: 3.10.0-693.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [kworker/u25:0:1210]
Modules linked in: vxlan ip6_udp_tunnel udp_tunnel iptable_mangle xt_mark veth xt_nat ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink nfnetlink iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter xt_conntrack nf_nat nf_conntrack br_netfilter bridge stp llc overlay(T) vmw_vsock_vmci_transport vsock vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) sb_edac edac_core coretemp iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev ppdev vmw_balloon vmw_vmci sg i2c_piix4 shpchp nfit parport_pc libnvdimm parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm sr_mod drm cdrom
ata_generic pata_acpi sd_mod ata_piix crc_t10dif crct10dif_generic libata crct10dif_pclmul vmxnet3 crct10dif_common floppy i2c_core crc32c_intel serio_raw vmw_pvscsi dm_mirror dm_region_hash dm_log dm_mod
CPU: 4 PID: 1210 Comm: kworker/u25:0 Tainted: G           OEL ------------ T 3.10.0-693.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 04/05/2016
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff880035ee9fa0 ti: ffff8802b2620000 task.ti: ffff8802b2620000
RIP: 0010:[<ffffffff81182d26>]  [<ffffffff81182d26>] find_get_pages_tag+0x86/0x1a0
RSP: 0018:ffff8802b26239b0  EFLAGS: 00010202
RAX: 0000000000000002 RBX: ffffea0021379b40 RCX: ffffea00264470dc
RDX: 0000000000000003 RSI: ffff880d0a334698 RDI: 00000000000003ff
RBP: ffff8802b2623a08 R08: ffff8802b2623a00 R09: ffffea00264470c0
R10: 0000000000000002 R11: 0000000000000220 R12: ffffea0021379b40
R13: 0000000000000000 R14: ffff880200035000 R15: 0000000001fcdf90
FS:  0000000000000000(0000) GS:ffff88081fd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f03e4365fb8 CR3: 0000001015a88000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Stack:
ffff8802b2623b08 ffff880da5f0a0a8 00000000000000bc 00000000000000c0
000000000000000f 000000001bfe5003 ffff8802b2623a88 ffff880da5f0a0a0
ffff8802b2623c40 0000000000000000 ffffea0021379b40 ffff8802b2623a20
Call Trace:
[<ffffffff81190111>] pagevec_lookup_tag+0x21/0x30
[<ffffffff8118db9a>] write_cache_pages+0x15a/0x4d0
[<ffffffffc0268980>] ? xfs_aops_discard_page+0x150/0x150 [xfs]
[<ffffffffc026878d>] xfs_vm_writepages+0xbd/0xe0 [xfs]
[<ffffffff8118effe>] do_writepages+0x1e/0x40
[<ffffffff8122d7d0>] __writeback_single_inode+0x40/0x220
[<ffffffff8122e414>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff8122e77f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8122efb3>] wb_writeback+0x263/0x2f0
[<ffffffff8122f45b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff810a881a>] process_one_work+0x17a/0x440
[<ffffffff810a94e6>] worker_thread+0x126/0x3c0
[<ffffffff810a93c0>] ? manage_workers.isra.24+0x2a0/0x2a0
[<ffffffff810b098f>] kthread+0xcf/0xe0
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff816b4f18>] ret_from_fork+0x58/0x90
[<ffffffff810b08c0>] ? insert_kthread_work+0x40/0x40
Code: 0f 1f 44 00 00 4c 8b 0e 4d 85 c9 74 48 41 f6 c1 03 0f 85 b6 00 00 00 45 8b 51 1c 45 85 d2 74 e5 41 8d 52 01 49 8d 49 1c 44 89 d0 <f0> 41 0f b1 51 1c 44 39 d0 89 c2 0f 85 9e 00 00 00 4c 39 0e 0f
Complete report #322265
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffb93be866
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x86
2
pagevec_lookup_tag
vmlinux 0xffffffffb93ccb51
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffb93ca85a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc040f290
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa0
5
xfs_vm_writepages
xfs 0xffffffffc040f25b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
6
do_writepages
vmlinux 0xffffffffb93cb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
7
__writeback_single_inode
vmlinux 0xffffffffb947df10
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
8
? wake_up_bit
vmlinux 0xffffffffb92c6f15
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x25
9
writeback_sb_inodes
vmlinux 0xffffffffb947ea94
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x1c4
10
__writeback_inodes_wb
vmlinux 0xffffffffb947ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x9f
11
wb_writeback
vmlinux 0xffffffffb947f273
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x263
12
? get_nr_inodes
vmlinux 0xffffffffb946ae6c
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x4c
13
bdi_writeback_workfn
vmlinux 0xffffffffb947fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2cb
14
process_one_work
vmlinux 0xffffffffb92bde8f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x17f
15
worker_thread
vmlinux 0xffffffffb92befa6
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x126
16
? manage_workers.isra.26
vmlinux 0xffffffffb92bee80
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2a0
17
kthread
vmlinux 0xffffffffb92c5e61
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xd1
18
? insert_kthread_work
vmlinux 0xffffffffb92c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
19
ret_from_fork_nospec_begin
vmlinux 0xffffffffb9995df7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
20
? insert_kthread_work
vmlinux 0xffffffffb92c5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [kworker/u256:1:76843]
CPU: 3 PID: 76843 Comm: kworker/u256:1 Tainted: G        W    L ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090007  05/18/2018
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9b562ee4d280 ti: ffff9b5727f50000 task.ti: ffff9b5727f50000
RIP: 0010:[<ffffffffb93be866>]  [<ffffffffb93be866>] find_get_pages_tag+0x86/0x240
RSP: 0000:ffff9b5727f539b8  EFLAGS: 00010202
RAX: 0000000000000002 RBX: ffff9b57396504f8 RCX: 0000000000000003
RDX: 0000000000000002 RSI: ffffe57e8c694edc RDI: ffffe57e8c694ec0
RBP: ffff9b5727f53a18 R08: ffff9b5727f53a98 R09: 0000000000000000
R10: 0000000000000400 R11: 0000000000000400 R12: ffffffffc02a4fc8
R13: ffff9b5727f53970 R14: ffff9b55f587f000 R15: ffff9b56c7f8ad80
FS:  0000000000000000(0000) GS:ffff9b56c16c0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007feb55176b80 CR3: 00000000f108c000 CR4: 00000000001406e0
Call Trace:
[<ffffffffb93ccb51>] pagevec_lookup_tag+0x21/0x30
[<ffffffffb93ca85a>] write_cache_pages+0x11a/0x470
[<ffffffffc040f290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc040f25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb93cb9b1>] do_writepages+0x21/0x50
[<ffffffffb947df10>] __writeback_single_inode+0x40/0x260
[<ffffffffb92c6f15>] ? wake_up_bit+0x25/0x30
[<ffffffffb947ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb947ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb947f273>] wb_writeback+0x263/0x2f0
[<ffffffffb946ae6c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb947fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb92bde8f>] process_one_work+0x17f/0x440
[<ffffffffb92befa6>] worker_thread+0x126/0x3c0
[<ffffffffb92bee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb92c5e61>] kthread+0xd1/0xe0
[<ffffffffb92c5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffb9995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb92c5d90>] ? insert_kthread_work+0x40/0x40
Code: db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 <f0> 0f b1 4f 1c 39 d0 89 c1 0f 85 3b 01 00 00 48 39 3b 0f 85 8d
Complete report #312047
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffb0fbe842
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffffb0fccb51
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffb0fca85a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc062a290
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffffb1155e70
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x70
6
? update_group_power
vmlinux 0xffffffffb0ee8264
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x174
7
? cpumask_next_and
vmlinux 0xffffffffb1186a45
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x35
8
xfs_vm_writepages
xfs 0xffffffffc062a25b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
9
do_writepages
vmlinux 0xffffffffb0fcb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
10
__writeback_single_inode
vmlinux 0xffffffffb107df10
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
11
writeback_sb_inodes
vmlinux 0xffffffffb107ea94
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x1c4
12
__writeback_inodes_wb
vmlinux 0xffffffffb107ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x9f
13
wb_writeback
vmlinux 0xffffffffb107f273
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x263
14
? get_nr_inodes
vmlinux 0xffffffffb106ae6c
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x4c
15
bdi_writeback_workfn
vmlinux 0xffffffffb107fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2cb
16
process_one_work
vmlinux 0xffffffffb0ebde8f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x17f
17
worker_thread
vmlinux 0xffffffffb0ebefa6
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x126
18
? manage_workers.isra.26
vmlinux 0xffffffffb0ebee80
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2a0
19
kthread
vmlinux 0xffffffffb0ec5e61
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xd1
20
? system_call_after_swapgs
vmlinux 0xffffffffb1595ec9
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x96
21
? insert_kthread_work
vmlinux 0xffffffffb0ec5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffb1595de4
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xe
23
? insert_kthread_work
vmlinux 0xffffffffb0ec5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [kworker/u12:3:1891]
Modules linked in: xt_addrtype xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat xt_conntrack br_netfilter nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink overlay(T) ppdev snd_intel8x0 squashfs snd_ac97_codec parport_pc ac97_bus loop snd_seq pcspkr parport sg virtio_balloon snd_seq_device snd_pcm joydev i2c_piix4 snd_timer snd soundcore auth_rpcgss binfmt_misc sunrpc ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm virtio_blk ata_piix drm virtio_console libata 8139too serio_raw 8139cp floppy virtio_pci drm_panel_orientation_quirks virtio_ring mii
virtio dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 1891 Comm: kworker/u12:3 Kdump: loaded Tainted: G    B          ------------ T 3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff8c2033c65280 ti: ffff8c1dc72f4000 task.ti: ffff8c1dc72f4000
RIP: 0010:[<ffffffffb0fbe842>]  [<ffffffffb0fbe842>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff8c1dc72f79b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8c20eab35800 RCX: 000000000000002b
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffe388ca1bd2c0
RBP: ffff8c1dc72f7a18 R08: ffff8c1dc72f7a98 R09: ffff8c1d4380ea88
R10: 0000000000001c00 R11: 0000000000000400 R12: ffff8c1dc72f7948
R13: ffffffffb1152f1b R14: ffff8c1dc72f7920 R15: ffff8c20eab35800
FS:  00007f0c7ac1a700(0000) GS:ffff8c20ffc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f0c74002028 CR3: 00000002a4610000 CR4: 00000000000007f0
Call Trace:
[<ffffffffb0fccb51>] pagevec_lookup_tag+0x21/0x30
[<ffffffffb0fca85a>] write_cache_pages+0x11a/0x470
[<ffffffffc062a290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb1155e70>] ? submit_bio+0x70/0x150
[<ffffffffb0ee8264>] ? update_group_power+0x174/0x280
[<ffffffffb1186a45>] ? cpumask_next_and+0x35/0x50
[<ffffffffc062a25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb0fcb9b1>] do_writepages+0x21/0x50
[<ffffffffb107df10>] __writeback_single_inode+0x40/0x260
[<ffffffffb107ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb107ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb107f273>] wb_writeback+0x263/0x2f0
[<ffffffffb106ae6c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb107fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb0ebde8f>] process_one_work+0x17f/0x440
[<ffffffffb0ebefa6>] worker_thread+0x126/0x3c0
[<ffffffffb0ebee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb0ec5e61>] kthread+0xd1/0xe0
[<ffffffffb1595ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb0ec5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffffb1595de4>] ret_from_fork_nospec_begin+0xe/0x21
[<ffffffffb0ec5d90>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #303571
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff83dbe859
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x79
2
pagevec_lookup_tag
vmlinux 0xffffffff83dccb51
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff83dca85a
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0751290
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffff83f55e70
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc075125b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffff83dcb9b1
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffff83e7df10
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffff83cc6f15
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffff83e7ea94
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffff83e7ed9f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffff83e7f273
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffff83e6ae6c
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffff83e7fe6b
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffff83cbde8f
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffff83cbefa6
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffff83cbee80
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffff83cc5e61
Build id: 3.10.0-1160.45.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffff83cc5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffff84395df7
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffff83cc5d90
Build id: 3.10.0-1160.45.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u32:2:22233]
Modules linked in: xt_limit ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport xt_conntrack nf_conntrack iptable_filter snd_hda_codec_analog snd_hda_codec_generic squashfs loop snd_hda_codec_hdmi intel_powerclamp coretemp kvm_intel iTCO_wdt snd_hda_intel iTCO_vendor_support kvm snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device irqbypass snd_pcm pcspkr sg i2c_i801 lpc_ich snd_timer snd soundcore i7core_edac asus_atk0110 tpm_infineon acpi_cpufreq ip_tables xfs libcrc32c raid1 sd_mod crc_t10dif crct10dif_generic sr_mod cdrom crct10dif_common ata_generic pata_acpi nouveau video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect pata_marvell sysimgblt fb_sys_fops ttm ahci mxm_wmi drm libahci libata firewire_ohci crc32c_intel
firewire_core serio_raw crc_itu_t sky2 floppy drm_panel_orientation_quirks wmi dm_mirror dm_region_hash dm_log dm_mod
CPU: 0 PID: 22233 Comm: kworker/u32:2 Kdump: loaded Tainted: G    B     I    ------------   3.10.0-1160.45.1.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/P6T DELUXE, BIOS 2209    09/21/2010
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff9e5fbb9c0000 ti: ffff9e5f4df10000 task.ti: ffff9e5f4df10000
RIP: 0010:[<ffffffff83dbe859>]  [<ffffffff83dbe859>] find_get_pages_tag+0x79/0x240
RSP: 0018:ffff9e5f4df139b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffc02ee577 RCX: 000000000000002b
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffe44e04fa1a40
RBP: ffff9e5f4df13a18 R08: ffff9e5f4df13a98 R09: ffff9e5f4bb2f588
R10: 0000000000003400 R11: 0000000000000005 R12: ffff9e5ff5016618
R13: ffffffffc0702c0a R14: ffff9e5f4df139c8 R15: ffff9e5ff577f800
FS:  0000000000000000(0000) GS:ffff9e5ff9200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f99ebeb7160 CR3: 0000000098c10000 CR4: 00000000000007f0
Call Trace:
[<ffffffff83dccb51>] pagevec_lookup_tag+0x21/0x30
[<ffffffff83dca85a>] write_cache_pages+0x11a/0x470
[<ffffffffc0751290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff83f55e70>] ? submit_bio+0x70/0x150
[<ffffffffc075125b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff83dcb9b1>] do_writepages+0x21/0x50
[<ffffffff83e7df10>] __writeback_single_inode+0x40/0x260
[<ffffffff83cc6f15>] ? wake_up_bit+0x25/0x30
[<ffffffff83e7ea94>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff83e7ed9f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff83e7f273>] wb_writeback+0x263/0x2f0
[<ffffffff83e6ae6c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff83e7fe6b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff83cbde8f>] process_one_work+0x17f/0x440
[<ffffffff83cbefa6>] worker_thread+0x126/0x3c0
[<ffffffff83cbee80>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff83cc5e61>] kthread+0xd1/0xe0
[<ffffffff83cc5d90>] ? insert_kthread_work+0x40/0x40
[<ffffffff84395df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff83cc5d90>] ? insert_kthread_work+0x40/0x40
Code: b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b 57 1c <85> d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c 39 d0 89
Complete report #231335
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff82fbe822
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffff82fccb31
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff82fca83a
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc04eb290
Build id: 3.10.0-1160.36.2.el7.x86_64 		0xa0
5
? update_curr
vmlinux 0xffffffff82ee480c
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x14c
6
? account_entity_dequeue
vmlinux 0xffffffff82ee0f3e
Build id: 3.10.0-1160.36.2.el7.x86_64 		0xae
7
? cpumask_next_and
vmlinux 0xffffffff831869f5
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x35
8
xfs_vm_writepages
xfs 0xffffffffc04eb25b
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x6b
9
do_writepages
vmlinux 0xffffffff82fcb991
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x21
10
__writeback_single_inode
vmlinux 0xffffffff8307dec0
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x40
11
? __free_pages
vmlinux 0xffffffff82fc7b55
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x25
12
writeback_sb_inodes
vmlinux 0xffffffff8307ea44
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x1c4
13
__writeback_inodes_wb
vmlinux 0xffffffff8307ed4f
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x9f
14
wb_writeback
vmlinux 0xffffffff8307f223
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x263
15
? get_nr_inodes
vmlinux 0xffffffff8306ae1c
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x4c
16
bdi_writeback_workfn
vmlinux 0xffffffff8307fe1b
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x2cb
17
process_one_work
vmlinux 0xffffffff82ebde6f
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x17f
18
worker_thread
vmlinux 0xffffffff82ebef86
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x126
19
? manage_workers.isra.26
vmlinux 0xffffffff82ebee60
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x2a0
20
kthread
vmlinux 0xffffffff82ec5e41
Build id: 3.10.0-1160.36.2.el7.x86_64 		0xd1
21
? insert_kthread_work
vmlinux 0xffffffff82ec5d70
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffff83595df7
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x21
23
? insert_kthread_work
vmlinux 0xffffffff82ec5d70
Build id: 3.10.0-1160.36.2.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u8:3:326]
Modules linked in: tcp_lp cmac arc4 md4 nls_utf8 cifs ccm dns_resolver xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter bnep sunrpc squashfs loop intel_powerclamp coretemp iTCO_wdt iTCO_vendor_support intel_rapl kvm_intel ppdev mei_wdt kvm irqbypass crc32_pclmul btusb ghash_clmulni_intel btrtl btbcm btintel bluetooth aesni_intel snd_hda_codec_realtek snd_hda_codec_generic
lrw snd_hda_codec_hdmi gf128mul glue_helper ablk_helper cryptd joydev pcspkr sg rfkill lpc_ich snd_hda_intel i2c_i801 snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device mei_me parport_pc snd_pcm parport mei wmi snd_timer snd soundcore ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic i915 i2c_algo_bit iosf_mbi ahci drm_kms_helper libahci syscopyarea sysfillrect sysimgblt fb_sys_fops drm e1000e crct10dif_pclmul crct10dif_common libata crc32c_intel serio_raw ptp pps_core drm_panel_orientation_quirks video dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 326 Comm: kworker/u8:3 Kdump: loaded Tainted: G    B          ------------   3.10.0-1160.36.2.el7.x86_64 #1
Hardware name: LENOVO 10A7S00U00/SHARKBAY, BIOS FBKT75AUS 04/01/2014
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff8b923dae9080 ti: ffff8b923ba04000 task.ti: ffff8b923ba04000
RIP: 0010:[<ffffffff82fbe822>]  [<ffffffff82fbe822>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff8b923ba079b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000036
RDX: 0000000000000000 RSI: 0000000000000040 RDI: fffff694937c1680
RBP: ffff8b923ba07a18 R08: ffff8b923ba07a98 R09: ffff8b90b7e51b88
R10: 0000000000003400 R11: 000002e7746ea980 R12: 0000000000000000
R13: ffff8b8cae7e6018 R14: 0000000000000000 R15: 000000000001acc0
FS:  0000000000000000(0000) GS:ffff8b925ea00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fd609b40000 CR3: 0000000506410000 CR4: 00000000001607f0
Call Trace:
[<ffffffff82fccb31>] pagevec_lookup_tag+0x21/0x30
[<ffffffff82fca83a>] write_cache_pages+0x11a/0x470
[<ffffffffc04eb290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff82ee480c>] ? update_curr+0x14c/0x1e0
[<ffffffff82ee0f3e>] ? account_entity_dequeue+0xae/0xd0
[<ffffffff831869f5>] ? cpumask_next_and+0x35/0x50
[<ffffffffc04eb25b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff82fcb991>] do_writepages+0x21/0x50
[<ffffffff8307dec0>] __writeback_single_inode+0x40/0x260
[<ffffffff82fc7b55>] ? __free_pages+0x25/0x30
[<ffffffff8307ea44>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8307ed4f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8307f223>] wb_writeback+0x263/0x2f0
[<ffffffff8306ae1c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8307fe1b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff82ebde6f>] process_one_work+0x17f/0x440
[<ffffffff82ebef86>] worker_thread+0x126/0x3c0
[<ffffffff82ebee60>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff82ec5e41>] kthread+0xd1/0xe0
[<ffffffff82ec5d70>] ? insert_kthread_work+0x40/0x40
[<ffffffff83595df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff82ec5d70>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #214064
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffb61be622
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffffb61cc1a1
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffb61c9eaa
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc06a1290
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa0
5
? update_group_power
vmlinux 0xffffffffb60e7fc8
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x28
6
? cpumask_next_and
vmlinux 0xffffffffb6386395
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x35
7
? find_busiest_group
vmlinux 0xffffffffb60e8364
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x144
8
xfs_vm_writepages
xfs 0xffffffffc06a125b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x6b
9
do_writepages
vmlinux 0xffffffffb61cb001
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
10
__writeback_single_inode
vmlinux 0xffffffffb627d9a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
11
writeback_sb_inodes
vmlinux 0xffffffffb627e524
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x1c4
12
__writeback_inodes_wb
vmlinux 0xffffffffb627e82f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x9f
13
wb_writeback
vmlinux 0xffffffffb627ed03
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x263
14
? get_nr_inodes
vmlinux 0xffffffffb626a92c
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x4c
15
bdi_writeback_workfn
vmlinux 0xffffffffb627f8fb
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x2cb
16
process_one_work
vmlinux 0xffffffffb60bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x17f
17
worker_thread
vmlinux 0xffffffffb60beee6
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x126
18
? manage_workers.isra.26
vmlinux 0xffffffffb60bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x2a0
19
kthread
vmlinux 0xffffffffb60c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xd1
20
? system_call_after_swapgs
vmlinux 0xffffffffb6795ec9
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x96
21
? insert_kthread_work
vmlinux 0xffffffffb60c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
22
ret_from_fork_nospec_begin
vmlinux 0xffffffffb6795df7
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
23
? insert_kthread_work
vmlinux 0xffffffffb60c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u32:1:13181]
Modules linked in: squashfs loop binfmt_misc xt_limit ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common xt_LOG nf_conntrack_ipv4 nf_defrag_ipv4 xt_multiport xt_conntrack nf_conntrack iptable_filter sunrpc snd_hda_codec_analog snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec joydev intel_powerclamp snd_hda_core coretemp kvm_intel snd_hwdep snd_seq kvm snd_seq_device irqbypass iTCO_wdt iTCO_vendor_support snd_pcm pcspkr sg snd_timer i2c_i801 snd asus_atk0110 tpm_infineon soundcore lpc_ich i7core_edac acpi_cpufreq ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi raid1 pata_marvell sd_mod crc_t10dif crct10dif_generic crct10dif_common nouveau video i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm mxm_wmi ahci drm libahci libata crc32c_intel
firewire_ohci serio_raw firewire_core sky2 crc_itu_t drm_panel_orientation_quirks floppy wmi dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 13181 Comm: kworker/u32:1 Kdump: loaded Tainted: G    B     I    ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: System manufacturer System Product Name/P6T DELUXE, BIOS 2209    09/21/2010
Workqueue: writeback bdi_writeback_workfn (flush-253:2)
task: ffff935af5ab5280 ti: ffff935ac6fc8000 task.ti: ffff935ac6fc8000
RIP: 0010:[<ffffffffb61be622>]  [<ffffffffb61be622>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff935ac6fcb9b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffffc024e577 RCX: 000000000000003a
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffdc1e01165bc0
RBP: ffff935ac6fcba18 R08: ffff935ac6fcba98 R09: ffff9359c9604888
R10: 0000000000003000 R11: 0000000000000005 R12: ffff935b62aadc18
R13: ffff935b62aadc00 R14: ffffffffc0603c0a R15: ffff935ac6fcb9c0
FS:  0000000000000000(0000) GS:ffff935b79200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7427b57000 CR3: 0000000022410000 CR4: 00000000000007f0
Call Trace:
[<ffffffffb61cc1a1>] pagevec_lookup_tag+0x21/0x30
[<ffffffffb61c9eaa>] write_cache_pages+0x11a/0x470
[<ffffffffc06a1290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb60e7fc8>] ? update_group_power+0x28/0x280
[<ffffffffb6386395>] ? cpumask_next_and+0x35/0x50
[<ffffffffb60e8364>] ? find_busiest_group+0x144/0x990
[<ffffffffc06a125b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb61cb001>] do_writepages+0x21/0x50
[<ffffffffb627d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffffb627e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb627e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb627ed03>] wb_writeback+0x263/0x2f0
[<ffffffffb626a92c>] ? get_nr_inodes+0x4c/0x70
[<ffffffffb627f8fb>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffffb60bddcf>] process_one_work+0x17f/0x440
[<ffffffffb60beee6>] worker_thread+0x126/0x3c0
[<ffffffffb60bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb60c5da1>] kthread+0xd1/0xe0
[<ffffffffb6795ec9>] ? system_call_after_swapgs+0x96/0x13a
[<ffffffffb60c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb6795df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb60c5cd0>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #189874
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff87dbe639
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x79
2
pagevec_lookup_tag
vmlinux 0xffffffff87dcc1a1
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff87dc9eaa
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0433290
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffff87f55870
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc043325b
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffff87dcb001
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffff87e7d9a0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffff87cc6e55
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffff87e7e524
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffff87e7e82f
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffff87e7ed03
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffff87e6a92c
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffff87e7f8fb
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffff87cbddcf
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffff87cbeee6
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffff87cbedc0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffff87cc5da1
Build id: 3.10.0-1160.24.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffff87cc5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffff88395ddd
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x7
21
? insert_kthread_work
vmlinux 0xffffffff87cc5cd0
Build id: 3.10.0-1160.24.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/u256:2:120843]
Modules linked in: tcp_lp rfcomm xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vmw_vsock_vmci_transport bnep vsock sunrpc dm_mirror dm_region_hash dm_log snd_seq_midi snd_seq_midi_event iosf_mbi crc32_pclmul dm_mod ghash_clmulni_intel snd_ens1371 ppdev snd_rawmidi snd_ac97_codec ac97_bus snd_seq vmw_balloon snd_seq_device aesni_intel btusb lrw gf128mul
 glue_helper ablk_helper cryptd btrtl snd_pcm btbcm btintel bluetooth joydev pcspkr rfkill snd_timer snd soundcore sg i2c_piix4 vmw_vmci parport_pc parport binfmt_misc ip_tables xfs libcrc32c sd_mod sr_mod crc_t10dif cdrom crct10dif_generic ata_generic pata_acpi vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel nfit drm libnvdimm mptspi serio_raw e1000 ata_piix scsi_transport_spi libata mptscsih mptbase drm_panel_orientation_quirks fuse
CPU: 2 PID: 120843 Comm: kworker/u256:2 Kdump: loaded Tainted: G    B   W    L ------------   3.10.0-1160.24.1.el7.x86_64 #1
Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/29/2019
Workqueue: writeback bdi_writeback_workfn (flush-8:0)
task: ffff9d6485f3a100 ti: ffff9d648a864000 task.ti: ffff9d648a864000
RIP: 0010:[<ffffffff87dbe639>]  [<ffffffff87dbe639>] find_get_pages_tag+0x79/0x240
RSP: 0018:ffff9d648a8679b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9d6547d49048 RCX: 0000000000000013
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffd6e6c4a7a940
RBP: ffff9d648a867a18 R08: ffff9d648a867a98 R09: ffff9d6431443988
R10: 0000000000005c00 R11: ffff9d650ccc2140 R12: ffff9d65b2efb000
R13: ffff9d650ccc2140 R14: ffff9d6400000030 R15: 1400000000000001
FS:  0000000000000000(0000) GS:ffff9d65b5680000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00002d8909439004 CR3: 000000011a954000 CR4: 00000000003607e0
Call Trace:
 [<ffffffff87dcc1a1>] pagevec_lookup_tag+0x21/0x30
 [<ffffffff87dc9eaa>] write_cache_pages+0x11a/0x470
 [<ffffffffc0433290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
 [<ffffffff87f55870>] ? submit_bio+0x70/0x150
 [<ffffffffc043325b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
 [<ffffffff87dcb001>] do_writepages+0x21/0x50
 [<ffffffff87e7d9a0>] __writeback_single_inode+0x40/0x260
 [<ffffffff87cc6e55>] ? wake_up_bit+0x25/0x30
 [<ffffffff87e7e524>] writeback_sb_inodes+0x1c4/0x430
 [<ffffffff87e7e82f>] __writeback_inodes_wb+0x9f/0xd0
 [<ffffffff87e7ed03>] wb_writeback+0x263/0x2f0
 [<ffffffff87e6a92c>] ? get_nr_inodes+0x4c/0x70
 [<ffffffff87e7f8fb>] bdi_writeback_workfn+0x2cb/0x460
 [<ffffffff87cbddcf>] process_one_work+0x17f/0x440
 [<ffffffff87cbeee6>] worker_thread+0x126/0x3c0
 [<ffffffff87cbedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
 [<ffffffff87cc5da1>] kthread+0xd1/0xe0
 [<ffffffff87cc5cd0>] ? insert_kthread_work+0x40/0x40
 [<ffffffff88395ddd>] ret_from_fork_nospec_begin+0x7/0x21
 [<ffffffff87cc5cd0>] ? insert_kthread_work+0x40/0x40
Code: b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b 57 1c <85> d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c 39 d0 89
Complete report #151789
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffa23be62a
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x6a
2
pagevec_lookup_tag
vmlinux 0xffffffffa23cc1a1
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffa23c9eaa
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0312290
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xa0
5
xfs_vm_writepages
xfs 0xffffffffc031225b
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x6b
6
do_writepages
vmlinux 0xffffffffa23cb001
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
7
__writeback_single_inode
vmlinux 0xffffffffa247d9a0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
8
? wake_up_bit
vmlinux 0xffffffffa22c6e55
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x25
9
writeback_sb_inodes
vmlinux 0xffffffffa247e524
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x1c4
10
__writeback_inodes_wb
vmlinux 0xffffffffa247e82f
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x9f
11
wb_writeback
vmlinux 0xffffffffa247ed03
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x263
12
? bdi_dirty_limit
vmlinux 0xffffffffa23ca770
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
13
bdi_writeback_workfn
vmlinux 0xffffffffa247f7fc
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x1cc
14
process_one_work
vmlinux 0xffffffffa22bddcf
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x17f
15
worker_thread
vmlinux 0xffffffffa22beee6
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x126
16
? manage_workers.isra.26
vmlinux 0xffffffffa22bedc0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x2a0
17
kthread
vmlinux 0xffffffffa22c5da1
Build id: 3.10.0-1160.25.1.el7.x86_64 		0xd1
18
? insert_kthread_work
vmlinux 0xffffffffa22c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40
19
ret_from_fork_nospec_begin
vmlinux 0xffffffffa2995df7
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x21
20
? insert_kthread_work
vmlinux 0xffffffffa22c5cd0
Build id: 3.10.0-1160.25.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u12:2:22922]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vfat fat sb_edac snd_intel8x0 snd_ac97_codec iosf_mbi crc32_pclmul joydev ac97_bus snd_seq iTCO_wdt ghash_clmulni_intel snd_seq_device iTCO_vendor_support ppdev snd_pcm aesni_intel sg lrw gf128mul glue_helper ablk_helper cryptd snd_timer pcspkr snd vboxguest(OE) lpc_ich parport_pc video
i2c_piix4 soundcore parport nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci drm libahci libata crct10dif_pclmul crct10dif_common e1000 crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 22922 Comm: kworker/u12:2 Kdump: loaded Tainted: G           OEL ------------   3.10.0-1160.25.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff9ed6359fe300 ti: ffff9ed634e64000 task.ti: ffff9ed634e64000
RIP: 0010:[<ffffffffa23be62a>]  [<ffffffffa23be62a>] find_get_pages_tag+0x6a/0x240
RSP: 0018:ffff9ed634e679b8  EFLAGS: 00000282
RAX: 0000000000000000 RBX: ffff9ed86fdf0000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffe8090287795c RDI: ffffe8090075f980
RBP: ffff9ed634e67a18 R08: ffff9ed634e67a98 R09: 0000000000000000
R10: 0000000000000800 R11: ffff9ed5ab7bf180 R12: ffff9ed634e67968
R13: 0000000000000018 R14: ffff9ed634e67920 R15: 0000000000000286
FS:  0000000000000000(0000) GS:ffff9ed883cc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f479e713003 CR3: 00000003e6410000 CR4: 00000000000406e0
Call Trace:
[<ffffffffa23cc1a1>] pagevec_lookup_tag+0x21/0x30
[<ffffffffa23c9eaa>] write_cache_pages+0x11a/0x470
[<ffffffffc0312290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffc031225b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffa23cb001>] do_writepages+0x21/0x50
[<ffffffffa247d9a0>] __writeback_single_inode+0x40/0x260
[<ffffffffa22c6e55>] ? wake_up_bit+0x25/0x30
[<ffffffffa247e524>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffa247e82f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffa247ed03>] wb_writeback+0x263/0x2f0
[<ffffffffa23ca770>] ? bdi_dirty_limit+0x40/0xe0
[<ffffffffa247f7fc>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffa22bddcf>] process_one_work+0x17f/0x440
[<ffffffffa22beee6>] worker_thread+0x126/0x3c0
[<ffffffffa22bedc0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffa22c5da1>] kthread+0xd1/0xe0
[<ffffffffa22c5cd0>] ? insert_kthread_work+0x40/0x40
[<ffffffffa2995df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffa22c5cd0>] ? insert_kthread_work+0x40/0x40
Code: ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 <48> 89 f8 83 e0 03 0f 85 49 01 00 00 8b 57 1c 85 d2 74 e5 8d 4a
Complete report #118316
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff90fb6cd6
Build id: 3.10.0-957.12.1.el7.x86_64 		0x76
2
pagevec_lookup_tag
vmlinux 0xffffffff90fc4681
Build id: 3.10.0-957.12.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff90fc22ba
Build id: 3.10.0-957.12.1.el7.x86_64 		0x11a
4
? xfs_aops_discard_page
xfs 0xffffffffc055d5b0
Build id: 3.10.0-957.12.1.el7.x86_64 		0x150
5
? cpumask_next_and
vmlinux 0xffffffff91175af5
Build id: 3.10.0-957.12.1.el7.x86_64 		0x35
6
xfs_vm_writepages
xfs 0xffffffffc055d3ab
Build id: 3.10.0-957.12.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffff90fc34d1
Build id: 3.10.0-957.12.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffff9106ff60
Build id: 3.10.0-957.12.1.el7.x86_64 		0x40
9
writeback_sb_inodes
vmlinux 0xffffffff910709f4
Build id: 3.10.0-957.12.1.el7.x86_64 		0x1c4
10
__writeback_inodes_wb
vmlinux 0xffffffff91070d5f
Build id: 3.10.0-957.12.1.el7.x86_64 		0x9f
11
wb_writeback
vmlinux 0xffffffff91071593
Build id: 3.10.0-957.12.1.el7.x86_64 		0x263
12
? get_nr_inodes
vmlinux 0xffffffff9105d3ac
Build id: 3.10.0-957.12.1.el7.x86_64 		0x4c
13
bdi_writeback_workfn
vmlinux 0xffffffff91071f1b
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2cb
14
process_one_work
vmlinux 0xffffffff90eb9d8f
Build id: 3.10.0-957.12.1.el7.x86_64 		0x17f
15
worker_thread
vmlinux 0xffffffff90ebae26
Build id: 3.10.0-957.12.1.el7.x86_64 		0x126
16
? manage_workers.isra.25
vmlinux 0xffffffff90ebad00
Build id: 3.10.0-957.12.1.el7.x86_64 		0x2a0
17
kthread
vmlinux 0xffffffff90ec1c71
Build id: 3.10.0-957.12.1.el7.x86_64 		0xd1
18
? insert_kthread_work
vmlinux 0xffffffff90ec1ba0
Build id: 3.10.0-957.12.1.el7.x86_64 		0x40
19
ret_from_fork_nospec_begin
vmlinux 0xffffffff91575bf7
Build id: 3.10.0-957.12.1.el7.x86_64 		0x21
20
? insert_kthread_work
vmlinux 0xffffffff90ec1ba0
Build id: 3.10.0-957.12.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [kworker/u32:3:10644]
Modules linked in: veth xt_nat nf_conntrack_netlink nfnetlink xt_addrtype br_netfilter dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop overlay tcp_lp fuse xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables devlink iptable_filter sunrpc dm_mirror dm_region_hash dm_log dm_mod ppdev iosf_mbi kvm_intel kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr joydev parport_pc virtio_balloon parport i2c_piix4 binfmt_misc ip_tables xfs libcrc32c ata_generic pata_acpi virtio_console virtio_net virtio_blk bochs_drm drm_kms_helper syscopyarea
sysfillrect sysimgblt fb_sys_fops ttm floppy drm ata_piix libata crct10dif_pclmul crct10dif_common crc32c_intel serio_raw virtio_pci virtio_ring drm_panel_orientation_quirks virtio
CPU: 2 PID: 10644 Comm: kworker/u32:3 Kdump: loaded Tainted: G             L ------------   3.10.0-957.12.1.el7.x86_64 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-252:15)
task: ffff89873f785140 ti: ffff8991962c4000 task.ti: ffff8991962c4000
RIP: 0010:[<ffffffff90fb6cd6>]  [<ffffffff90fb6cd6>] find_get_pages_tag+0x76/0x240
RSP: 0018:ffff8991962c79b8  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff899133348040 RCX: 0000000000000000
RDX: 0000000000000000 RSI: fffff5139e8a8e9c RDI: fffff513848c3bc0
RBP: ffff8991962c7a18 R08: ffff8991962c7a98 R09: 0000000000000000
R10: 0000000000000400 R11: 0000000000000000 R12: ffff8991962c7a80
R13: ffff8991962c7a98 R14: 000000000000000e R15: 0000000000000003
FS:  0000000000000000(0000) GS:ffff8992ff280000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00000000b833d002 CR3: 0000000feab30000 CR4: 00000000003606e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
[<ffffffff90fc4681>] pagevec_lookup_tag+0x21/0x30
[<ffffffff90fc22ba>] write_cache_pages+0x11a/0x470
[<ffffffffc055d5b0>] ? xfs_aops_discard_page+0x150/0x150 [xfs]
[<ffffffff91175af5>] ? cpumask_next_and+0x35/0x50
[<ffffffffc055d3ab>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff90fc34d1>] do_writepages+0x21/0x50
[<ffffffff9106ff60>] __writeback_single_inode+0x40/0x260
[<ffffffff910709f4>] writeback_sb_inodes+0x1c4/0x490
[<ffffffff91070d5f>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff91071593>] wb_writeback+0x263/0x2f0
[<ffffffff9105d3ac>] ? get_nr_inodes+0x4c/0x70
[<ffffffff91071f1b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff90eb9d8f>] process_one_work+0x17f/0x440
[<ffffffff90ebae26>] worker_thread+0x126/0x3c0
[<ffffffff90ebad00>] ? manage_workers.isra.25+0x2a0/0x2a0
[<ffffffff90ec1c71>] kthread+0xd1/0xe0
[<ffffffff90ec1ba0>] ? insert_kthread_work+0x40/0x40
[<ffffffff91575bf7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff90ec1ba0>] ? insert_kthread_work+0x40/0x40
Code: 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 <8b> 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c
Complete report #98818
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffffb69beb72
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffffb69cc3e1
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffffb69ca0ba
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0583310
Build id: 3.10.0-1127.10.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffffb6b546a0
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc05832db
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffffb69cb211
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffffb6a7cc30
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffffb68c7745
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffffb6a7d7c4
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffffb6a7dacf
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffffb6a7dfb3
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x263
13
? bdi_dirty_limit
vmlinux 0xffffffffb69ca980
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x40
14
bdi_writeback_workfn
vmlinux 0xffffffffb6a7eaac
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x1cc
15
process_one_work
vmlinux 0xffffffffb68be6bf
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffffb68bf7d6
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffffb68bf6b0
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffffb68c6691
Build id: 3.10.0-1127.10.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffffb68c65c0
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffffb6f92d37
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffffb68c65c0
Build id: 3.10.0-1127.10.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kworker/u8:2:1270]
Modules linked in: tcp_lp xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ppdev parport_pc parport vmw_vsock_vmci_transport vsock iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi vmw_vmci nf_conntrack_netbios_ns nf_conntrack_broadcast ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter iTCO_wdt iTCO_vendor_support dcdbas pcspkr joydev sg i2c_i801 ipmi_ssif lpc_ich ipmi_si ipmi_devintf i3000_edac
ipmi_msghandler nfsd auth_rpcgss nfs_acl lockd grace sunrpc binfmt_misc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic crct10dif_common sr_mod cdrom ata_generic pata_acpi radeon i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata tg3 8139too 8139cp serio_raw drm_panel_orientation_quirks mii ptp pps_core dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 0 PID: 1270 Comm: kworker/u8:2 Kdump: loaded Tainted: G    B        L ------------   3.10.0-1127.10.1.el7.x86_64 #1
Hardware name: Dell Computer Corporation PowerEdge 840/0XM091, BIOS A04 07/24/2007
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff943f143e62a0 ti: ffff943e42a98000 task.ti: ffff943e42a98000
RIP: 0010:[<ffffffffb69beb72>]  [<ffffffffb69beb72>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff943e42a9b9b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff943e42a9b940 RCX: 0000000000000018
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffef334106f5c0
RBP: ffff943e42a9ba18 R08: ffff943e42a9ba98 R09: ffff943e5d830540
R10: 0000000000000400 R11: ffffef33408c8800 R12: ffff943e42a9b988
R13: ffff943e748fd618 R14: ffff943e748fd600 R15: ffff943e42a9b9c0
FS:  0000000000000000(0000) GS:ffff943f1fc00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f4948db68c0 CR3: 00000000626e6000 CR4: 00000000000007f0
Call Trace:
[<ffffffffb69cc3e1>] pagevec_lookup_tag+0x21/0x30
[<ffffffffb69ca0ba>] write_cache_pages+0x11a/0x470
[<ffffffffc0583310>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffffb6b546a0>] ? submit_bio+0x70/0x150
[<ffffffffc05832db>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffffb69cb211>] do_writepages+0x21/0x50
[<ffffffffb6a7cc30>] __writeback_single_inode+0x40/0x260
[<ffffffffb68c7745>] ? wake_up_bit+0x25/0x30
[<ffffffffb6a7d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffffb6a7dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffffb6a7dfb3>] wb_writeback+0x263/0x2f0
[<ffffffffb69ca980>] ? bdi_dirty_limit+0x40/0xe0
[<ffffffffb6a7eaac>] bdi_writeback_workfn+0x1cc/0x460
[<ffffffffb68be6bf>] process_one_work+0x17f/0x440
[<ffffffffb68bf7d6>] worker_thread+0x126/0x3c0
[<ffffffffb68bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffffb68c6691>] kthread+0xd1/0xe0
[<ffffffffb68c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffffb6f92d37>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffffb68c65c0>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #19398
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff905be3c2
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x62
2
pagevec_lookup_tag
vmlinux 0xffffffff905cbfc1
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff905c9cca
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc0367290
Build id: 3.10.0-1160.6.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffff90755450
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc036725b
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffff905cae21
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffff9067d730
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffff904c6cd5
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffff9067e2b4
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffff9067e5bf
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffff9067ea93
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffff9066a6bc
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffff9067f68b
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffff904bdc4f
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffff904bed66
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffff904bec40
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffff904c5c21
Build id: 3.10.0-1160.6.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffff904c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffff90b94df7
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x21
21
? insert_kthread_work
vmlinux 0xffffffff904c5b50
Build id: 3.10.0-1160.6.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [kworker/u12:1:13070]
Modules linked in: xt_CHECKSUM ipt_MASQUERADE nf_nat_masquerade_ipv4 tun devlink ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat iptable_mangle iptable_security iptable_raw nf_conntrack ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vfat fat sb_edac snd_intel8x0 snd_ac97_codec iosf_mbi crc32_pclmul ac97_bus snd_seq ghash_clmulni_intel snd_seq_device iTCO_wdt snd_pcm joydev iTCO_vendor_support ppdev aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr snd_timer sg snd vboxguest(OE) lpc_ich parport_pc soundcore
video parport i2c_piix4 nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod sr_mod cdrom crc_t10dif crct10dif_generic vmwgfx drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm ahci libahci drm libata crct10dif_pclmul crct10dif_common e1000 crc32c_intel serio_raw drm_panel_orientation_quirks dm_mirror dm_region_hash dm_log dm_mod fuse
CPU: 3 PID: 13070 Comm: kworker/u12:1 Kdump: loaded Tainted: G    B      OEL ------------   3.10.0-1160.6.1.el7.x86_64 #1
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
Workqueue: writeback bdi_writeback_workfn (flush-253:1)
task: ffff8d2092e0b180 ti: ffff8d206319c000 task.ti: ffff8d206319c000
RIP: 0010:[<ffffffff905be3c2>]  [<ffffffff905be3c2>] find_get_pages_tag+0x62/0x240
RSP: 0018:ffff8d206319f9b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff8d21ef9ec000 RCX: 0000000000000028
RDX: 0000000000000000 RSI: 0000000000000040 RDI: fffffd354ce1e680
RBP: ffff8d206319fa18 R08: ffff8d206319fa98 R09: ffff8d21badafd48
R10: 0000000000001400 R11: 7fffffffffffffff R12: ffff8d206319f968
R13: 0000000000000286 R14: ffff8d206319f920 R15: ffff8d21f0f7d018
FS:  0000000000000000(0000) GS:ffff8d2203cc0000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f6df0344000 CR3: 000000037e810000 CR4: 00000000000406e0
Call Trace:
[<ffffffff905cbfc1>] pagevec_lookup_tag+0x21/0x30
[<ffffffff905c9cca>] write_cache_pages+0x11a/0x470
[<ffffffffc0367290>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff90755450>] ? submit_bio+0x70/0x150
[<ffffffffc036725b>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff905cae21>] do_writepages+0x21/0x50
[<ffffffff9067d730>] __writeback_single_inode+0x40/0x260
[<ffffffff904c6cd5>] ? wake_up_bit+0x25/0x30
[<ffffffff9067e2b4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff9067e5bf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff9067ea93>] wb_writeback+0x263/0x2f0
[<ffffffff9066a6bc>] ? get_nr_inodes+0x4c/0x70
[<ffffffff9067f68b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff904bdc4f>] process_one_work+0x17f/0x440
[<ffffffff904bed66>] worker_thread+0x126/0x3c0
[<ffffffff904bec40>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff904c5c21>] kthread+0xd1/0xe0
[<ffffffff904c5b50>] ? insert_kthread_work+0x40/0x40
[<ffffffff90b94df7>] ret_from_fork_nospec_begin+0x21/0x21
[<ffffffff904c5b50>] ? insert_kthread_work+0x40/0x40
Code: 8d 47 08 80 ce 01 45 31 ff 89 55 ac 48 89 45 a0 49 8b 04 24 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 <48> 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 8b
Complete report #16576
Frame # Function Binary Source or offset Line
1
find_get_pages_tag
vmlinux 0xffffffff893bebd6
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x76
2
pagevec_lookup_tag
vmlinux 0xffffffff893cc401
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
3
write_cache_pages
vmlinux 0xffffffff893ca10a
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x11a
4
? xfs_vm_writepages
xfs 0xffffffffc05ad310
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xa0
5
? submit_bio
vmlinux 0xffffffff895546e0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x70
6
xfs_vm_writepages
xfs 0xffffffffc05ad2db
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x6b
7
do_writepages
vmlinux 0xffffffff893cb261
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x21
8
__writeback_single_inode
vmlinux 0xffffffff8947cc40
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
9
? wake_up_bit
vmlinux 0xffffffff892c7745
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x25
10
writeback_sb_inodes
vmlinux 0xffffffff8947d7c4
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x1c4
11
__writeback_inodes_wb
vmlinux 0xffffffff8947dacf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x9f
12
wb_writeback
vmlinux 0xffffffff8947dfa3
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x263
13
? get_nr_inodes
vmlinux 0xffffffff89469c5c
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x4c
14
bdi_writeback_workfn
vmlinux 0xffffffff8947eb9b
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2cb
15
process_one_work
vmlinux 0xffffffff892be6bf
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x17f
16
worker_thread
vmlinux 0xffffffff892bf7d6
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x126
17
? manage_workers.isra.26
vmlinux 0xffffffff892bf6b0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x2a0
18
kthread
vmlinux 0xffffffff892c6691
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xd1
19
? insert_kthread_work
vmlinux 0xffffffff892c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40
20
ret_from_fork_nospec_begin
vmlinux 0xffffffff89992d24
Build id: 3.10.0-1127.19.1.el7.x86_64 		0xe
21
? insert_kthread_work
vmlinux 0xffffffff892c65c0
Build id: 3.10.0-1127.19.1.el7.x86_64 		0x40 
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [kworker/u12:2:355]
Modules linked in: btrfs raid6_pq xor vfat msdos fat ext4 mbcache jbd2 uinput xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT nf_reject_ipv4 tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter devlink sunrpc snd_hda_codec_generic joydev snd_hda_intel snd_hda_codec snd_hda_core snd_hwdep snd_seq snd_seq_device snd_pcm ppdev snd_timer snd soundcore pcspkr sg virtio_balloon parport_pc parport i2c_piix4 binfmt_misc fuse ip_tables xfs libcrc32c sr_mod cdrom ata_generic pata_acpi cirrus drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm ata_piix libata 8139too virtio_blk virtio_console serio_raw 8139cp virtio_pci virtio_ring
drm_panel_orientation_quirks floppy virtio mii dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 355 Comm: kworker/u12:2 Kdump: loaded Tainted: G    B          ------------   3.10.0-1127.19.1.el7.x86_64 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Workqueue: writeback bdi_writeback_workfn (flush-253:0)
task: ffff997e08b120e0 ti: ffff997fdb93c000 task.ti: ffff997fdb93c000
RIP: 0010:[<ffffffff893bebd6>]  [<ffffffff893bebd6>] find_get_pages_tag+0x76/0x240
RSP: 0018:ffff997fdb93f9b8  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffff9980576ca000 RCX: 000000000000000d
RDX: 0000000000000000 RSI: 0000000000000040 RDI: ffffe2e78edc9dc0
RBP: ffff997fdb93fa18 R08: ffff997fdb93fa98 R09: ffff9981b06548c0
R10: 0000000000003400 R11: 0000000000000005 R12: ffff997fdb93f948
R13: ffffffff8955178b R14: ffff997fdb93f920 R15: ffff9980576ca000
FS:  00007f2c92778700(0000) GS:ffff9981ffc40000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 000055749b1d0228 CR3: 0000000297410000 CR4: 00000000000007e0
Call Trace:
[<ffffffff893cc401>] pagevec_lookup_tag+0x21/0x30
[<ffffffff893ca10a>] write_cache_pages+0x11a/0x470
[<ffffffffc05ad310>] ? xfs_vm_writepages+0xa0/0xa0 [xfs]
[<ffffffff895546e0>] ? submit_bio+0x70/0x150
[<ffffffffc05ad2db>] xfs_vm_writepages+0x6b/0xa0 [xfs]
[<ffffffff893cb261>] do_writepages+0x21/0x50
[<ffffffff8947cc40>] __writeback_single_inode+0x40/0x260
[<ffffffff892c7745>] ? wake_up_bit+0x25/0x30
[<ffffffff8947d7c4>] writeback_sb_inodes+0x1c4/0x430
[<ffffffff8947dacf>] __writeback_inodes_wb+0x9f/0xd0
[<ffffffff8947dfa3>] wb_writeback+0x263/0x2f0
[<ffffffff89469c5c>] ? get_nr_inodes+0x4c/0x70
[<ffffffff8947eb9b>] bdi_writeback_workfn+0x2cb/0x460
[<ffffffff892be6bf>] process_one_work+0x17f/0x440
[<ffffffff892bf7d6>] worker_thread+0x126/0x3c0
[<ffffffff892bf6b0>] ? manage_workers.isra.26+0x2a0/0x2a0
[<ffffffff892c6691>] kthread+0xd1/0xe0
[<ffffffff892c65c0>] ? insert_kthread_work+0x40/0x40
[<ffffffff89992d24>] ret_from_fork_nospec_begin+0xe/0x21
[<ffffffff892c65c0>] ? insert_kthread_work+0x40/0x40
Code: 48 c7 45 b0 00 00 00 00 31 db 48 89 45 b8 48 85 db 0f 84 8b 00 00 00 48 8b 3b 48 85 ff 74 45 48 89 f8 83 e0 03 0f 85 49 01 00 00 <8b> 57 1c 85 d2 74 e5 8d 4a 01 48 8d 77 1c 89 d0 f0 0f b1 4f 1c