Simulating an Openstack environment that consists of 12 Openstack nodes and 4 switches. When I first started, performance was horrible. Here are some the things I did to improve performance. Still learning, so if anyone has any tips, please share.
Tweak 1: Convert a simple dotviz file to a complicated messy looking Vagrantfile
A Vagrantfile is not a simple document to generate. Really! It is the kind of file you want a computer to create. Thank goodness, there is a cool opensource tool - topology-generator - to do that for you.
Tweak 2: Increase UDP Buffers
Tweak 3: Use the virtio_net driver where you can.
The performance of the virtio_net driver is just miles ahead of the
e1000 VM NIC drivers.
The virtionet driver is not perfect. By default, the virtionet NIC speed to
-1. But it can be
changed! I would love to know how to set it to a particular speed by default using a kernel cmdline argument.
Here is how I set the virtio_net enabled NIC speed. I need a better way. If someone knows a better way, please share.
auto ens6 iface ens6 inet manual description bond member bond-primary ens6 bond-master bond0 post-up ethtool -s $IFACE speed 10000 duplex full auto ens7 iface ens7 inet manual description bond member bond-primary ens6 bond-master bond0 post-up ethtool -s $IFACE speed 10000 duplex full
This config is not perfect. Ethtool does not kick in all the time, resulting in inconsistent bonds or connectivity issues. For all nodes
that need stable network bonding, I stick with the
e1000 NIC driver.
Tweak 4: Use kernel 4.x and higher
Nested Virtualization works so much better on a 4.x kernel than a 3.x Linux kernel. I am not sure why. Here is a great video to teach you more about nested virtualization.
That is all I have for now.
If I find more ways to improve vagrant-libvirt performance I will share it on this post.
Results so far:
A Level 2 Windows Server 2012 VM running in a Level 1 Linux VM ping performance
$ ping 192.168.8.76 PING 192.168.8.76 (192.168.8.76) 56(84) bytes of data. 64 bytes from 192.168.8.76: icmp_seq=1 ttl=127 time=32.0 ms 64 bytes from 192.168.8.76: icmp_seq=2 ttl=127 time=13.4 ms 64 bytes from 192.168.8.76: icmp_seq=3 ttl=127 time=2.75 ms 64 bytes from 192.168.8.76: icmp_seq=4 ttl=127 time=2.57 ms 64 bytes from 192.168.8.76: icmp_seq=5 ttl=127 time=3.02 ms 64 bytes from 192.168.8.76: icmp_seq=6 ttl=127 time=7.05 ms 64 bytes from 192.168.8.76: icmp_seq=7 ttl=127 time=3.10 ms 64 bytes from 192.168.8.76: icmp_seq=8 ttl=127 time=2.88 ms
Not bad I think. Originally the latency averaged 30ms