OpenBSD in Stereo with Linux VFIO
I use a Huawei Matebook X as my primary OpenBSD laptop and one aspect of its hardware support has always been lacking: audio never played out of the right-side speaker. The speaker did actually work, but only in Windows and only after the Realtek Dolby Atmos audio driver from Huawei was installed. Under OpenBSD and Linux, and even Windows with the default Intel sound driver, audio only ever played out of the left speaker.
Now, after some extensive reverse engineering and debugging with the help of VFIO on Linux, I finally have audio playing out of both speakers on OpenBSD.
Table of Contents
The Linux kernel has functionality called VFIO which enables direct access to a physical device (like a PCI card) from userspace, usually passing it to an emulator like QEMU.
To my surprise, it seems to be primarily used these days by gamers who boot Linux, then use QEMU to run a game in Windows and use VFIO to pass the computer's GPU device through to Windows.
By using Linux and VFIO, I was able to boot Windows 10 inside of QEMU and pass my laptop's PCI audio device through to Windows, allowing the Realtek audio drivers to natively control the audio device. Combined with QEMU's tracing functionality, I was able to get a log of all PCI I/O between Windows and the PCI audio device.
To use VFIO to pass-through a PCI device, it first needs to be stubbed out so the
Linux kernel's default drivers don't attach to it.
GRUB can be configured to instruct the kernel to ignore the PCI audio device
8086:9d71) and explicitly enable the Intel IOMMU driver by adding the following
/etc/default/grub and running
GRUB_CMDLINE_LINUX_DEFAULT="text pci-stub.ids=8086:9d71 iommu=pt intel_iommu=on"
With the audio device stubbed out, a new VFIO device can be created from it:
$ sudo modprobe pci-stub $ sudo modprobe vfio-pci $ echo 0000:00:1f.3 | sudo tee /sys/bus/pci/devices/0000:00:1f.3/driver/unbind $ echo 0x8086 0x9d71 | sudo tee /sys/bus/pci/drivers/vfio-pci/new_id
Then the VFIO device (
00:1f.3) can be passed to QEMU:
$ sudo ../qemu/x86_64-softmmu/qemu-system-x86_64 \ -M q35 -m 2G -cpu host,kvm=off \ -enable-kvm \ -device vfio-pci,host=00:1f.3,multifunction=on,x-no-mmap \ -hda win10-tmp.img \ -trace events=events.txt 2>&1 | tee debug-output
I was using my own build of QEMU for this, due to some custom logging I needed (more on that later), but the default QEMU package should work fine.
events.txt was a file of all VFIO events I wanted logged (which was all of
Since I was frequently killing QEMU and restarting it, Windows 10 wanted to go
through its unexpected shutdown routine each time (and would sometimes just fail
to boot again).
To avoid this and to get a consistent set of logs each time, I used
to take a snapshot of a base image first, then boot QEMU with that snapshot.
The snapshot just gets thrown away the next time
qemu-img is run and Windows
always starts from a consistent state.
$ qemu-img create -f qcow2 -b win10.img win10-tmp.img
QEMU will now log each VFIO event which gets saved to a
[...] firstname.lastname@example.org:vfio_pci_read_config (0000:00:1f.3, @0x2e, len=0x2) 0x3200 email@example.com:vfio_region_read (0000:00:1f.3:region0+0xc, 2) = 0x0 firstname.lastname@example.org:vfio_region_read (0000:00:1f.3:region0+0xe, 2) = 0x1 email@example.com:vfio_region_write (0000:00:1f.3:region0+0xc, 0x0, 2) [...]
With a full log of all PCI I/O activity from Windows, I compared it to the output
from OpenBSD and tried to find the magic register writes that enabled the second
After days of combing through the logs and annotating them by looking up hex
values in the documentation,
diffing runtime register values, and even
brute-forcing it by mechanically duplicating all PCI I/O activity in the OpenBSD
driver, nothing would activate the right speaker.
One strange thing that I noticed was if I booted Windows 10 in QEMU and it activated the speaker, then booted OpenBSD in QEMU without resetting the PCI device's power in-between (as a normal system reboot would do), both speakers worked in OpenBSD and the configuration that the HDA controller presented was different, even without any changes in OpenBSD.
A Primer on Intel HDA
Most modern computers with integrated sound chips use an Intel High Definition Audio (HDA) Controller device, with one or more codecs (like the Realtek ALC269) hanging off of it. These codecs do the actual audio processing and communicate with DACs and ADCs to send digital audio to the connected speakers, or read analog audio from a microphone and convert it to a digital input stream. In my Huawei Matebook X, this is done through a Realtek ALC298 codec.
On OpenBSD, these HDA controllers are supported by the
driver, with all of the per-codec details in the lengthy
This file has grown quite large with lots of codec- and machine-specific quirks
to route things properly, toggle various GPIO pins, and unmute speakers that are
for some reason muted by default.
azalia0 at pci0 dev 31 function 3 "Intel 200 Series HD Audio" rev 0x21: msi azalia0: host: High Definition Audio rev. 1.0 azalia0: host: 9 output, 7 input, and 0 bidi streams azalia0: found a codec at #0 azalia0: found a codec at #2 azalia_init_corb: CORB allocation succeeded. azalia_init_corb: CORBWP=0; size=256 azalia_init_rirb: RIRB allocation succeeded. azalia_init_rirb: RIRBRP=0, size=256 azalia0: codec vid 0x10ec0298, subid 0x320019e5, rev. 1.3, HDA version 1.0 azalia_codec_init: There are 36 widgets in the audio function. [...] azalia0: codecs: Realtek ALC298, Intel/0x280b, using Realtek ALC298
azalia driver talks to the HDA controller and sets up various buffers and
then walks the list of codecs.
Each codec supports a number of widget nodes which can be interconnected in
Some of these nodes can be
on the fly to do things like turning a microphone port into a headphone port.
The newer Huawei Matebook X Pro released a few months ago is also plagued with this speaker problem, although it has four speakers and only two work by default. A fix is being proposed for the Linux kernel which just reconfigures those widget pins in the Intel HDA driver. Unfortunately no pin reconfiguration is enough to fix my Matebook X with its two speakers.
While reading more documentation on the HDA, I realized there was a lot more activity going on than I was able to see through the PCI tracing.
For speed and efficiency, HDA controllers use a DMA engine to transfer audio
streams as well as the commands from the OS driver to the codecs.
In the output above, the
CORBWP=0; size=256 and
RIRBRP=0, size=256 indicate
the setup of the CORB (Command Output Ring Buffer) and RIRB (Response Input Ring
Buffer) each with 256 entries.
The HDA driver allocates a DMA address and then writes it to the two
CORBUBASE registers, and again for the RIRB.
When the driver wants to send a command to a codec, such as
CORB_GET_PARAMETER with a parameter of
the codec address, the node index, the command verb, and the parameter, and then
writes that value to the CORB ring at the address it set up with the controller
at initialization time (
CORBUBASE) plus the offset of the ring
Once the command is on the ring, it does a PCI write to the
advancing it by one.
This lets the controller know a new command is queued, which it then acts on
and writes the response value on the RIRB ring at the same position as the
command (but at the RIRB's DMA address).
It then generates an interrupt, telling the driver to read the new
and process the new results.
Side note: if you've ever had a kernel panic or blue screen while playing music and had your audio device repeat the same second of audio over and over until you power it off, this is why. The audio driver put the last samples of audio onto the ring buffer of the HDA device and the hardware played them as it walked the ring buffer, but because the kernel panicked and can't put new samples onto the ring nor tell the hardware to stop consuming it, the HDA device just keeps walking around the ring playing that last sample of audio.
Since the actual command contents and responses are handled through DMA writes and reads, these important values weren't showing up in the VFIO PCI trace output that I had gathered. Time to hack QEMU.
Logging DMA Memory Values in QEMU
Since DMA activity wouldn't show up through QEMU's VFIO tracing and I obviously
couldn't get Windows to dump these values like I could in OpenBSD, I could make
QEMU recognize the PCI write to the
CORBWP register as an indication that a
command has just been written to the CORB ring.
in QEMU adds some HDA awareness to remember the CORB and RIRB DMA addresses as
they get programmed in the controller.
Then any time a PCI write to the
CORBWP register is done, QEMU fetches the new
CORB command from DMA memory, decodes it into the codec address, node address,
command, and parameter, and prints it out.
When a PCI read of the
RIRBWP register is requested, QEMU reads the response and
prints the corresponding CORB command that it stored earlier.
With this hack in place, I now had a full log of all CORB commands and RIRB responses sent to and read from the codec:
firstname.lastname@example.org:vfio_region_read (0000:00:1f.3:region0+0x48, 2) = 0xdb CORBWP advance to 220, last WP 219 CORB = 0x21f0800 (caddr:0x0 nid:0x21 control:0xf08 param:0x0) email@example.com:vfio_region_write (0000:00:1f.3:region0+0x48, 0xdc, 2) [...] firstname.lastname@example.org:vfio_region_write (0000:00:1f.3:region0+0x5d, 0x1, 1) RIRBWP advance to 220, last WP 219 CORB caddr:0x0 nid:0x21 control:0xf08 param:0x0 response:0x82 (ex 0x0) email@example.com:vfio_region_read (0000:00:1f.3:region0+0x58, 2) = 0xdc [...]
An early version of this patch left me stumped for a few days because, even after
submitting all of the same CORB commands in OpenBSD, the second speaker still
It wasn't until re-reading the HDA spec that I realized the Windows driver was
submitting more than one command at a time, writing multiple CORB entries and
CORBWP value that was advanced by two.
This required turning my CORB/RIRB reading into a
for loop, reading each new
command and response between the new
RIRBWP value and the one
Sure enough, the magic commands to enable the second speaker were sent in these periods where it submitted more than one command at a time.
Minimizing the Magic
The full log of VFIO PCI activity from the Windows driver was over 65,000 lines and contained 3,150 CORB commands, which was a lot to sort through. It took me a couple more days to reduce that down to a small subset that was actually required to activate the second speaker, and that could only be done through trial and error:
- Boot OpenBSD with the full list of CORB commands in the
- Comment out a group of them
- Compile kernel and install it, halt the QEMU guest
- Suspend and wake the laptop, resetting PCI power to the audio device to reset the speaker/Dolby initialization and ensure the previous run isn't influencing the current test (I'm guessing there is an easier to way to reset PCI power than suspending the laptop, but oh well)
- Start QEMU, boot OpenBSD with the new kernel
- Play an MP3 with
mpg123which has alternating left- and right-channel audio and listen for both channels to play
This required a dozen or so iterations because sometimes I'd comment out too many commands and the right speaker would stop working. Other times the combination of commands would hang the controller and it wouldn't process any further commands. At one point the combination of commands actually flipped the channels around so the right channel audio was playing through the left speaker.
After about a week of this routine, I ended up with a list of 662 CORB commands that are needed to get the second speaker working.
The stereo sound from OpenBSD is wonderful now and I can finally stop downmixing
everything to mono to play from the left speaker.
In case one ever needs to do this,
sndiod can be run with
-c 0:0 to reduce
the channels to one.
Update (2019-03-24): This list of CORB commands was
by Thomas Espeleta for inclusion in the Linux kernel and then Stefan Sperling
(stsp@) implemented that logic back in the OpenBSD
Coming full circle, I
Stefan's implementation and OpenBSD now fully supports the audio on the Huawei
Thanks to rjc for proofreading and feedback.