QEMU/KVM and GPU Passthrough Troubleshooting

From WikiMLT
Vir­tu­al Ma­chines
Prox­mox

Guest HD­MI Au­dio Crack­ling and IRQ xx: No­body Cared

Win­dows

The so­lu­tion of the is­sue Guest GPU HD­MI Au­dio Crack­ling, Bro­ken or Los­ing is well ex­plained by Jonp at UN­RAID Fo­rums. In short we must try to en­able the MSI – Mes­sage Sig­naled In­ter­rupts op­tion if the de­vice sup­port it. Here is a de­tailed step-by-step in­struc­tion how to do that.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 1: Open Device Manager and from the View menu switch to Resources by type.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 1: Open De­vice Man­ag­er and from the View menu switch to Re­sources by type.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 2: Expand the section Interrupt request (IRQ).

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 2: Ex­pand the sec­tion In­ter­rupt re­quest (IRQ).

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 3: Find the problematic audio device and by the context menu choice Properties.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 3: Find the prob­lem­at­ic au­dio de­vice and by the con­text menu choice Prop­er­ties.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 4: Within the Details tab witch to Property Device instance path.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 4: With­in the De­tails tab witch to Prop­er­ty De­vice in­stance path.

  • QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 5: In this par­tic­u­lar case the Val­ue is PCI­\­VEN­_­10­DE­&­DEV­_­0E­08­&­SUB­SYS­_­102­F10­DE­&­REV­_­A1­\­4&­29­fcb­422­&0­&00­16. This is a re­la­ti­ve path to Com­pu­ter­\­HKEY­_­LO­CAL­_­MA­CHI­NE­\­SYS­TEM­\­Cur­rent­Con­trol­Set­\­Enum in the Win­dows 10's regis­try.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 5: In this par­tic­u­lar case the Val­ue is PCI­\­VEN­_­10­DE­&­DEV­_­0E­08­&­SUB­SYS­_­102­F10­DE­&­REV­_­A1­\­4&­29­fcb­422­&0­&00­16. This is a re­la­ti­ve path to Com­pu­ter­\­HKEY­_­LO­CAL­_­MA­CHI­NE­\­SYS­TEM­\­Cur­rent­Con­trol­Set­\­Enum in the Win­dows 10's regis­try.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 6: Run Registry Editor as Administrator.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 6: Run Reg­istry Ed­i­tor as Ad­min­is­tra­tor.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 7: In the Registry Editor tool, paste Com­pu­ter­\­HKEY­_­LO­CAL­_­MA­CHI­NE­\­SYS­TEM­\­Cur­rent­Con­trol­Set­\­Enum in the path line and hit Enter.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 7: In the Reg­istry Ed­i­tor tool, paste Com­pu­ter­\­HKEY­_­LO­CAL­_­MA­CHI­NE­\­SYS­TEM­\­Cur­rent­Con­trol­Set­\­Enum in the path line and hit En­ter.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 8: In Device Manager, use the context menu and Copy the Value of the target Device instance path.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 8: In De­vice Man­ag­er, use the con­text menu and Copy the Val­ue of the tar­get De­vice in­stance path.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 9: In Registry Editor append one backslash to the path and Paste the Value.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 9: In Reg­istry Ed­i­tor ap­pend one back­slash to the path and Paste the Val­ue.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 10: In Registry Editor, navigate to \Device Parameters\Interrupt Ma­na­ge­ment\Me­ssa­ge­Sig­na­led­In­ter­rupt­Pro­per­ties.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 10: In Reg­istry Ed­i­tor, nav­i­gate to \De­vice Parameters\Interrupt Ma­na­ge­ment\Me­ssa­ge­Sig­na­led­In­ter­rupt­Pro­per­ties.

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 11: Then choose Modify from the context menu the Value of the DWORD Key MSISupported (MSI – Message Signaled Interrupts).

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 11: Then choose Mod­i­fy from the con­text menu the Val­ue of the DWORD Key MSISup­port­ed (MSI – Mes­sage Sig­naled In­ter­rupts).

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 12: Change the Value of the DWORD Key MSISupported to 1 [0x0..01].

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 12: Change the Val­ue of the DWORD Key MSISup­port­ed to 1 [0x0..01].

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 13: Confirm the change of the DWORD Key MSISupported (MSI – Message Signaled Interrupts).

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 13: Con­firm the change of the DWORD Key MSISup­port­ed (MSI – Mes­sage Sig­naled In­ter­rupts).

  • QEMU KVM Windows Guest Fix NVIDIA NVS 315 HDMI Audio Step 14: Reboot the guest Operating system.

    QE­MU KVM Win­dows Guest Fix NVIDIA NVS 315 HD­MI Au­dio Step 14: Re­boot the guest Op­er­at­ing sys­tem.

I was men­tioned the ker­nel com­mand irqpool as so­lu­tion for this is­sue, but in my opin­ion and ac­cord­ing to some oth­er posts, pro­vid­ed in the ref­er­ences sec­tion, this set­ting is an ac­tu­al so­lu­tion for the prob­lem irq 44: no­body cared (try boot­ing with the "irqpool" op­tion); han­dlers: vfio_​​​intx_​​​handler; Dis­abling IRQ #44, that ap­pears at my sys­tem when boot­ing a VM with GPU passthrough with a dis­play at­tached to it, which was caus­ing host sys­tem re­boot.

Lin­ux

Lin­ux guests usu­al­ly en­able MSI by them­selves. To force use of MSI (Mes­sage Sig­naled In­ter­rupts) for GPU au­dio de­vices, use the fol­low­ing com­mand and re­boot. 

echo "options snd-hda-intel enable_msi=1" | sudo tee /etc/modprobe.d/snd-hda-intel.conf

sudo update-initramfs -u -k all

Use lsp­ci ‑vv and check for the fol­low­ing line on your de­vice to see if MSI is en­abled.

sudo lspci -nnv -s 09:00
#Out­put
09:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117GL [T600] [10de:1fb1] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: NVIDIA Corporation TU117GL [T600] [10de:1488]
	Flags: bus master, fast devsel, latency 0, IRQ 103, IOMMU group 23
	Memory at fb000000 (32-bit, non-prefetchable) [size=16M]
	Memory at 7fd0000000 (64-bit, prefetchable) [size=256M]
	Memory at 7fe0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at f000 [size=128]
	Expansion ROM at fc000000 [virtual] [disabled] [size=512K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Legacy Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Capabilities: [250] Latency Tolerance Reporting
	Capabilities: [258] L1 PM Substates
	Capabilities: [128] Power Budgeting <?>
	Capabilities: [420] Advanced Error Reporting
	Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900] Secondary PCI Express
	Capabilities: [bb0] Physical Resizable BAR
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia

09:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)
	Subsystem: NVIDIA Corporation Device [10de:1488]
	Flags: bus master, fast devsel, latency 0, IRQ 99, IOMMU group 23
	Memory at fc080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [60] Power Management version 3
	Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Express Endpoint, MSI 00
	Capabilities: [100] Advanced Error Reporting
	Kernel driver in use: snd_hda_intel
	Kernel modules: snd_hda_intel

If it says En­able+, MSI is work­ing, En­able- means it is sup­port­ed but dis­abled, and if the line is miss­ing, MSI is not sup­port­ed by the PCIe hard­ware.

This can po­ten­tial­ly al­so im­prove per­for­mance for oth­er passthrough de­vices, in­clud­ing GPUs, but that de­pends on the hard­ware be­ing used.

Ref­er­ences

Passthrough GPU: Win­dows and SPICE/VNC

Don't use em­u­lat­ed Video and Graph­ic while you are us­ing GPU passthrough. It leads per­for­mance loss.

Spice may give trou­ble when pass­ing through a GPU as it presents a "vir­tu­al" PCI graph­ic card to the guest and some dri­vers have prob­lems with that, even when both cards show up. It's al­ways worth a try to dis­able SPICE and check again if some­thing fails.

Ref­er­ences

Passthrough GPU: Win­dows and 6ch/​​​9ch vs ac97 au­dio

Do not use 6ch/​​​9ch au­dio de­vices in the vir­tu­al ma­chine. It cre­ates aw­ful stut­ter­ing and per­for­mance loss. Use ac97 au­dio de­vices in­stead. This in­for­ma­tion is out­dat­ed – prob­a­bly the dri­vers have been up­dat­ed.

Ref­er­ences

Passthrough GPU: Win­dows and NVIDIA GPU Er­ror 43

I've ex­pe­ri­enc­ing this is­sue with the de­fault OVMF im­age (OVMF_CODE.ms.fd -> OVMF_CODE.secboot.fd). With the im­age OVMF_VARS-with-csm.fd the things were well. How­ev­er here is a gen­er­al so­lu­tion. Ap­ply the fol­low­ing changes in the Win­dows' vir­tu­al ma­chine XML file. You can use ei­ther virsh ed­it {your-win­dows-vm-name} or the XML tab in virt-man­ag­er. 

  ...
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  ...

Ref­er­ences

Hide the Warn­ings KVM: vcpu0 ig­nored rdm­sr

sudo nano /etc/modprobe.d/kvm.conf

options kvm report_ignored_msrs=0 # Or: GRUB_CMDLINE_LINUX_DEFAULT="... kvm.report_ignored_msrs=0 ..."

sudo update-initramfs -u -k all
  • Note, ac­cord­ing to my ex­pe­ri­ence, this op­tion may cause crash­es of the host sys­tem!?

Ref­er­ences

KVM Mod­e­in­fo

modinfo kvm

filename:       /lib/modules/5.4.0-77-generic/kernel/arch/x86/kvm/kvm.ko
license:        GPL
author:         Qumranet
srcversion:     20C68083F39E14AB616D0B8
depends:
retpoline:      Y
intree:         Y
name:           kvm
vermagic:       5.4.0-77-generic SMP mod_unload modversions
sig_id:         PKCS#7
signer:         Build time autogenerated kernel key
sig_key:        67:66:2F:F8:26:8F:56:E9:37:7F:B7:AD:33:FA:97:31:CA:7F:FF:50
sig_hashalgo:   sha512
signature:      6C:0B:65:F9:46:AC:D8:B7:94:E8:B9:9D:A0:4B:97:E6:63:52:5A:FF:
                ...
parm:           nx_huge_pages:bool
parm:           nx_huge_pages_recovery_ratio:uint
parm:           ignore_msrs:bool
parm:           report_ignored_msrs:bool
parm:           min_timer_period_us:uint
parm:           kvmclock_periodic_sync:bool
parm:           tsc_tolerance_ppm:uint
parm:           lapic_timer_advance_ns:int
parm:           vector_hashing:bool
parm:           enable_vmware_backdoor:bool
parm:           force_emulation_prefix:bool
parm:           pi_inject_timer:bint
parm:           halt_poll_ns:uint
parm:           halt_poll_ns_grow:uint
parm:           halt_poll_ns_grow_start:uint
parm:           halt_poll_ns_shrink:uint

ThinkServ­er TD350 Is­sues

With­in the TD350's doc­u­men­ta­tion is writ­ten it doesn't sup­port any video card. How­ev­er NVS 315 works with mi­nor is­sues as these men­tioned here.

  • ThinkServer RD350 generates warnings while a monitor is attached to one of the outputs of (not supported PCI GPU) NVS 315. At all everything else works as expected.

    ThinkServ­er RD350 gen­er­ates warn­ings while a mon­i­tor is at­tached to one of the out­puts of (not sup­port­ed PCI GPU) NVS 315. At all every­thing else works as ex­pect­ed.

  • ThinkServer RD350 gives some kind of priority to NVS 315 when there is attached monitor at boot and when GRUB_CMD_LINE contains vfio-pci.ids=10de:107c,10de:0e08 the system halts at this stage.

    ThinkServ­er RD350 gives some kind of pri­or­i­ty to NVS 315 when there is at­tached mon­i­tor at boot and when GRUB_CMD_LINE con­tains vfio-pci.ids=10de:107c,10de:0e08 the sys­tem halts at this stage.

Retrieved from "https://wiki.metalevel.tech/index.php?title=QEMU/KVM_and_GPU_Passthrough_Troubleshooting&oldid=31932"