The following Overlay Engine functionality is deployed to VNS3 as a plugin using the container system. These instructions cover customization of the container image that will be used so that customer keys and rule sets can be employed.
Please be familiar with the VNS3 Plug-In Configuration Guide.
Simply stated Overlay Engines make your Overlay Network faster.
With Overlay Engines running on a VNS3 controller in a jumbo frame enabled cloud environment like AWS VPC, total throughput of 12-20 Gbps is achievable (tested on c4.2xl) depending on a tradeoff for the use-case between latency and throughput.
Overlay Engine Plugins add another Overlay Network process to a VNS3 Controller allowing more connections at higher speeds. The number of Overlay Engines allocated to a particular VNS3 controller depends on the number of plugin slots licensed. It should be noted that launching more Overlay Engines than your instance has CPU cores will not produce any benefit.
Overlay Engines are run as a sealed plugin (no user access via SSH) which gets access to relevant Overlay Network keys and mechanisms in the host VNS3 controller. The Overlay Engine then runs all the processes necessary to handle Overlay Network client connections. This allows the Overlay Network to utilize multithreaded capabilities of the VNS3 instance resources.
Overlay Engine plugins need two things:
For access to this plugin please contact Cohesive Sales at email@example.com.
The Overlay Engine Plugin is accessible at the following restricted URLs; be sure to select the appropriate version for your version of VNS3:
These are in a read-only Amazon S3 storage location. Only Cohesive Networks can update or modify files stored in this location.
Cohesive Sales/Support can provide URLs to be used directly in a VNS3 Controller via the Web UI or API to import the container for use into that controller. (General screenshot walkthrough and help available in the plug-in configuration document.)
From the Container —> Images menu item, choose Upload Image.
Name the Container Image overlayengine.
This is a requirement to get the allocated plugins to receive access to the required directories on the VNS3 host.
To use the pre-configured plugin paste the URL into the Image File URL box.
When the Image has imported it will say Ready in the Status Column.
To launch an Overlay Engine container, choose Allocate from the Action menu.
After selecting Allocate from the Actions menu, you will be prompted for a container name, a description, and the command to be used to start the container.
The name should be UNIQUE among all Overlay Engine containers allocated to a VNS3 controller.
For the Overlay Engine Plugin, the start command is: /usr/bin/supervisord
After executing the Allocate operation, you will be taken to the Container Display page.
You should see your Overlay Engine container running with the name you specified. It should have been given an IP address on your internal plug-in subnet (in this case ‘188.8.131.52’).
Overlay Engines will need two types of firewall rules added to the VNS3 controller:
DNAT rules to redirect incoming clients’ traffic to an Overlay Engine.
Depending on the number of CPU cores your instance offers, you may be able to increase your total overlay throughput further by using additional overlay engine containers. The ideal situation is having one fewer Overlay Engine containers than CPU cores. The format for these rules is as follows:
PREROUTING_CUST -i eth0 -p udp --dport 1194 -j HMARK --hmark-tuple src,sport --hmark-mod [N+1] --hmark-rnd 0xf1e2d3c4 --hmark-offset 0x1000
PREROUTING_CUST -p udp --dport 1194 -m mark --mark 0x1000 -j ACCEPT
PREROUTING_CUST -p udp --dport 1194 -m mark --mark 0x1001 -j DNAT --to-destination 198.51.100.2:1194
PREROUTING_CUST -p udp --dport 1194 -m mark --mark 0x[1000+N] -j DNAT --to-destination 198.51.100.[2+N-1]:1194
Where “N” is the number of Overlay Engine containers you have.
Repeat the pattern of “PREROUTING_CUST … -j DNAT” rules as appropriate. The first “PREROUTING_CUST … -j ACCEPT” rule is for traffic which will connect to VNS3’s default internal OpenVPN server; there should then follow one DNAT rule for each Overlay Engine container.
Remember if you are using more than nine Overlay Engines that you are working in hexadecimal with regards to the “–mark”; the tenth packet mark would be 0x100A in the example above.
FORWARD rules to allow OpenVPN packets to move across the VNS3 host to/from the Overlay Engine Plugin(s):
FORWARD_CUST -i plugin0 -p udp --sport 1194 -j ACCEPT
FORWARD_CUST -o plugin0 -p udp --dport 1194 -j ACCEPT
Keep in mind that you will now have overlay clients’ traffic arriving on VNS3’s plugin0 interface as well as tun0. You may need to update any firewall rules which reference the tun0 interface. You may specify interfaces as a comma separated list, such as “-i tun0,plugin0”.
Updated on 16 Aug 2019