Sunday, January 23, 2022

What FreeBSD type of NIC to use in VMware Virtual Machine?

Today, I have received a question from one of my readers based in Germany. Hellmuth has the following question ...

Hi,

i just stumbled across your blog and read that you use FreeBSD.

For a long time, I wondered what to choose as the „best“ guest driver for FreeBSD: em, the vmx in the FreeBSD source, or the kld which comes with the open VMware Tools ?

Do you have an idea ? What do you use ?

Best regards,

Hellmuth

So here is the answer to Hellmuth and I believe, the answer can help somebody else in VMware and FreeBSD communities.

Hi Hellmuth,

thanks for reaching out. 

Interesting question, but it seems to me there must be some confusion.

VMware supports two production-ready types of Ethernet NIC cards
  1. Emulation of Intel NIC (em) - https://www.freebsd.org/cgi/man.cgi?em(4)
  2. Paravirtual NIC (vmx, vmxnet3) - https://www.freebsd.org/cgi/man.cgi?vmx(4)
KLD is not a NIC type but a dynamic kernel linker facility - https://www.freebsd.org/cgi/man.cgi?query=kld

FreeBSD supports Intel NICs (em) for ages and VMware virtualization emulates these NICs, so FreeBSD believes it is really a physical NIC from Intel. NIC driver for Intel NIC is in generic FreeBSD kernel, so it works by default. 

For paravirtualized NIC (VMXNET3), you need VMware driver for vmx network device as such NIC device does not have any physical equivalent. 

In the past, such driver was part of VMware Tools, however, nowadays, vmx driver is part of the generic kernel so it works by default after standard FreeBSD installation in the same way how em driver works for Intel-based NIC.

So let's answer Hellmuth's two questions:

Q1: Do you have an idea?
A1: The idea is described in this blog post. 

Q2: What do you use?
A2: Nowadays, I use VMXNET3 driver, thus vmx ethernet interfaces. The reason is to leverage para-virtualized driver which is more efficient as it does not need to emulate real Intel physical network adapter and instead work more optimally within a virtual environment. This can save some CPU cycles, therefore it has a positive impact on overall system performance.

To be absolutely sure with my statements, I have just quickly tested such behavior with VM hardware version 14 (vSphere 6.7) and FreeBSD 13 and I can confirm it works as described above. 

What kind of driver is used is driven by the way how the Virtual Machine is configured. 
  • If you select VM Compatibility with ESXi 6.7 and later, your FreeBSD VM will be preconfigured with paravirtualized (VMXNET3) NIC. 
  • If you choose older VM Compatibilities, in such case your FreeBSD VM will use Intel E1000 NIC.   
Below are screenshots from the FreeBSD VM deployment workflow when choosing VM Compatibility with ESXi 6.7 and later.
Screenshot: New Virtual Machine for FreeBSD Guest OS compatible with ESXi 6.7 and later

Screenshot: New Virtual Machine for FreeBSD Guest OS compatible with ESXi 6.7 and later

If I missed or forgot something important, do not hesitate to write your comment below this blog post.

3 comments:

Anonymous said...

Hi David,
thanks for the insights. To get this to an end i had a look at the source of the open-vm-tools in the ports and the vmx driver in the FreeBSD source tree. Until now i was under the impression that you get two versions of the vmx driver when you install the open-vm-tools - one from the kernel source and one as a kld from open-vm-tools. This is wrong. If open-vm-tools is installed, still - if configured - the vmx driver from the kernel is used in any case.

The last few months i had severe problems with APD conditions in my own setup, FreeBSD is very sensible to this. In debugging this i played with the em and vmx drivers. In an APD condition, if the guest is restarted, does still not find its disk and is using the em driver, it seems the em driver tries to switch to PXE to boot. There seems to be little information available about that and about e1000e driver configuration in vmx files.

Anyway, i'll stay with the vmx3 driver now that i know there is only one of it :-).
hellmuth

David Pasek said...

Hi Hellmuth,
well, in the past, VMware Tools were available only from VMware, however, with Vmtools 10.x and above, Vmware open sources VMware Tools.

For further details about VMware Tools read the documents below ...

https://docs.vmware.com/en/VMware-Tools/10.1.0/com.vmware.vsphere.vmwaretools.doc/GUID-8B6EA5B7-453B-48AA-92E5-DB7F061341D1.html

https://kb.vmware.com/s/article/2073803

https://www.vcdx200.com/2015/11/vmware-tools-10-and-shared-productlocker.html

https://www.vcdx200.com/2017/02/how-to-install-vmware-tools-on-freebsd.html

David Pasek said...

I'm very interested in your problem with APD (All Path Down) as I was dealing with such problems during my VMware consulting career relatively often and did some research in this area. Can you share some further details by email? Or better, would you like to schedule a video conf call (zoom or other) to discuss your problem interactively?

David.