VNS3 BGP Configuration Guide
This guide demonstrates how to set up a High-available(HA) site-to-site BGP over IPSec connection between 2 VNS3 Controller topologies.
Prerequisites: You should have basic knowledge of VNS3 IPSec configurations and how to create a VNS3 peered topology. For this example we will base our topology off the diagram below. The controller on the Left (Oregon) is acting as our Customer’s VNS3 controller. In the same subnet as the Customer VNS3 instance they have an EC2 instance that needs redundant connectivity to the DB server on the right side of the diagram(cohesive). The 2 controllers on the right will be referred to the Primary and Secondary California Controllers. They are in the same Region, different Availability Zones (AZ), and are peered via VNS3 peering. This example will demonstrate how to configure automated IPSec failover in the event of an entire AZ outage. For more information on VNS3 peering, please review: https://docs.cohesive.net/docs/vns3/peering/
For this guide we are utilizing the VNS3 Encrypted Overlay Network for the California Side of the topology. Click HERE for information about the VNS3 Overlay Network
You will need to have HA enabled on the Primary and Secondary controllers in order to create a VNS3 peering mesh. Make sure your peered controllers and the connecting controller (Oregon in our case) does not have overlapping ASN’s. You must manually edit the ASN during the initial licensing of your VNS3 controllers. For this example, in the Oregon VPC we brought up a Lite edition VNS3 which has a default ASN of 65001. In the California VPC, we brought up 2 BYOL edition controllers and manually edited their ASN’s during the licensing process (see screenshot below). Please email support@cohiesive.net if you have any questions.
Border Gateway Protocol (BGP)
BGP is a routing protocol that allows BGP “peers” (connected devices) to exchange routing and reachability information autonomously. Each peer manages a routing table with all the routing information it knows about, then advertises this information to its BGP peers. This way, if a system goes down, the BGP peers’ routing tables will get updated automatically, allowing traffic to get to its desired destination in the case of a system down or failover event.
A BGP configuration is done by bringing up 2 separate Route-based IPsec connections in VNS3. Once those are configured, you will define your BGP peers and specify weighted routes. The purpose of this is to create an active-active configuration where in the case that the primary endpoint goes down, the traffic will automatically get routed through the secondary VPN connection, providing automatic failover.
For more information about BGP, please review THIS GUIDE
Primary BGP Configuration (Primary Controller)
First, navigate to the IPSec page in the VNS3 Web UI of your primary controller (Primary Cali in our case). Select New endpoint.
Fill out the endpoint configuration for the Oregon Controller.
- Name accordingly
- Public IP address of the remote (Customer) VNS3
- Enable NAT-T
- IKEv1 radio button
- Define a PSK
- Enable Route-based VPN (via VTI)
- Specify a VTI interface Address as a /30 CIDR. For route-based IPSec connections in VNS3, a Virtual Tunnel Interface (VTI) is brought up and used to route traffic over the IPSec connection. The VTI interface address can be any non-overlapping address, defined as a /30 CIDR. Addresses from the 169.254.x.y space are commonly used.
- Set the local and remote CIDR’s to 0.0.0.0/0
- Create Endpoint
Define a New eBGP Peer
Once the endpoint is created you will now create the BGP peer configuration. In the Actions dropdown menu for the endpoint you just created, select New eBGP Peer.
On the “Define a New eBGP Peer” page, fill out the information of the connecting VNS3 controller. In this case the “Customer/Oregon” Controller.
- The Peer’s IP address will be the other remote VTI address that lies within that same /30 CIDR of the VTI address that you defined in the previous step. (i.e. we defined our VTI address as 169.254.2.2/30, so the peer IP address will be 169.254.2.3)
- Peer ASN (65001)
- Define the peer’s access list. The “peer’s access list” will be where you define the local and remote CIDRs. Use the terms ‘in permit’ and ‘out permit’ to define what traffic will traverse this BGP connection. The “out permit” addresses will be advertised to the BGP peer(s), and “in permit” addresses will accept specific advertisements from the Remote(Customer) VNS3 BGP peer. Do not configure network distance for the primary endpoint.
- Do not a Network Distance for this peer
- Create
Secondary BGP Configuration (Secondary Controller)
Navigate to the IPSec page in the VNS3 Web UI of your Secondary controller (Secondary Cali). Select New endpoint.
Fill out all the same information as you did with the Primary controller but change the 3rd octet in the VTI interface address. Create.
Once the Endpoint is created, Select “New eBGP Peer” From the Actions menu.
Fill out all the BGP Peer information and do set an inbound Network Distance of 10. This network distance will be advertised to its BGP peers. The network distance specified makes this connection appear to be farther away than the primary BGP peer, therefore traffic will only get routed over this tunnel if the Primary connection goes down.
The tunnels and BGP peers will not show connected until we configure the connecting (Customer) device, in this use-case another VNS3 controller.
BGP on Customer Controller
BGP from Customer VNS3 to Primary
The Customer device could be any IPSec device that supports active-active BGP. For this example we are connecting to another VNS3 controller hosted in the Customer’s AWS account. Navigate to the IPSec page in the Customer(Oregon) VNS3 Web UI and create the Primary endpoint configuration. Create a new endpoint, name it accordingly(Primary), and fill out the rest of the parameters. The VTI address must fall within the /30 VTI interface CIDR defined on on the Primary Controller. Once created you should see the tunnel show “Connected” in green text.
Create the BGP peer as you did in the previous steps and be sure to not specify a network Distance.
You should see the tunnel and BGP peers come up shortly. We will now configure the secondary connection.
BGP from Customer VNS3 to Secondary VNS3
Create a Secondary endpoint and be sure to use a different VTI interface address that’s in the same /30 CIDR as your California Secondary endpoint VTI address.(i.e. 169.254.5.3/30)
For the Secondary BGP peer configuration, be sure to add an Network distance of 10. It doesn’t matter whether this is inbound or outbound network distance. This network distance will get advertise to the peers, making this connection only routable if the Primary connection goes down.
Routing
If your VNS3 device is sitting in a subnet with other VM instances that you would like to use the BGP connection(s), you will have to add the necessary cloud VPC/subnet routes as well as a route advertisement in the VNS3 routes page to you local VPC subnet CIDR(s).
You have now created a success active-active Highly Available BGP configuration. If you have any questions please contact Cohesive support at support.cohesive.net or by email at support@cohesive.net.