Table of Contents
CloudWatch Logging Container Detail
The CloudWatch Logging container was created for VNS3 version 4.8.3+. If you are using an VNS3 version prior to 4.8.3, please scroll down to the Manual Configuration section of this document.
Getting Started with VNS3 Plugin System
The AWS CloudWatch Logs agent is deployed to VNS3 (version 4.8.3+) as a plugin using the container system.
Please be familiar with the VNS3 Plug-In Configuration Guide.
CloudWatch Logging Container - What does it do?
The CloudWatch Logging Container pulls live logs from a host VNS3 controller via the AWS CloudWatch Logs agent and displays these logs in AWS CloudWatch. A CloudFormation template is used to automatically create all the necessary AWS components.
The CloudFormation template creates metrics filters based on potential log messages that notify/alarm an SNS topic when triggered. (text message, email, etc.)
The goal is to minimize outages by monitoring the VNS3 controller’s IPsec, overlay, and/or system logs by sending automatic alerts when unfavorable conditions arise.
CloudWatch Logging Container - What does it need?
User input to deploy the CloudWatch Logs Agent:
- Create CloudFormation Stack via the template URL
- Define a log Groupname and SNS topic (email or phone number)
- Attach Role to VNS3 Instance
- Add the appropriate VNS3 firewall rules
- Upload the CloudWatch Logging Container image
- Provide configuration options (Groupname and Logs) to the container via environment variables
Deploying the CloudWatch Logging Container
Deploying the CloudFormation Template
Create a new CloudFormation Stack with the following URL:
This is a read-only Amazon S3 storage location. Only Cohesive Networks can update or modify files stored in this location.
The CloudFormation template will automatically create all the AWS components (IAM Role, SNS topic, CloudWatch metric filters, alarms, and a log group). You will need to define a unique log Groupname and provide an email address that will be used for the SNS topic.
NOTE: Once the CloudFormation template is deployed, you will receive a SNS subscription confirmation email. Confirm the subscription BEFORE deploying the container.
The CloudFormation template creates an IAM Role, attach this Role to your existing VNS3 controller (the Role name will correlate to the CloudFormation Stack name). From the EC2 Console, select the VNS3 controller, Select Actions > Instance Settings > Attach/Replace IAM Role
If you already have an IAMs Role attached to the instance, you will need to apply the two new policies to the existing Role. (see Developers section below)
Uploading the Container Image to the VNS3 Plugin System
From the Containers page in the VNS3 Web UI, select Container —> Images, select Upload Image.
The name can be upper, lower, or mixed case. The word “logger” is required to provide containers allocated from this image with the appropriate log files from VNS3.
Copy and paste the CloudWatch Logs Container Image file URL:
CloudWatch Container Firewall Rules
The CloudWatch Container requires the following firewall rules to be added to the VNS3 controller BEFORE allocating the container:
Anywhere it appears, replace
<container_ip> with the IP you will use for your CloudWatch container.
#Port forward tcp44 to CloudWatch Logging container to SSH for post setup configuration PREROUTING_CUST -i eth0 -p tcp --dport 44 -j DNAT --to <container_ip>:22 #SNAT for container network MACRO_CUST -o eth0 -s <container_ip> -j MASQUERADE
Allocating the Container from the Image
When the Image has imported successfully, it will say Ready in the Status Column.
To launch a container from the image, choose Allocate from the Action menu.
Launching the CloudWatch Logging Container
After selecting “Allocate”, provide a name for your container,
/usr/bin/supervisord as the Command, a description if you’d like, and the required environment variables (groupname and logs, as seen in the picture below).
You can allow VNS3 to auto-assign a container network IP, but we recommend that you choose one manually. The environment variables section is where you define what system logs will be sent to CloudWatch.
Environment variables format:
groupname="<log group name>", logs="<log1> <log2> <log3>"
If you are using a version of VNS3 older than 4.8.3, you must define the groupname and log streams by SSHing into the container and editing the awslogs.conf configuration file. (See the Developers section below.)
Here are the system logs which can be offloaded to CloudWatch for monitoring and alerting:
- kern.log - VNS3 kernel log
- syslog - VNS3 system log
- vnscubed_api_background/completed/ - Directory showing completed API calls
- vnscubed_connection_logs/ipsec.log - IPsec system log file
- vnscubed_connection_logs/tun0.log - Overlay Network server connection log files
If you would like to send multiple logs to CloudWatch, use a space-separated list in quotes as seen in the picture below:
Manual Configuration (for developers)
When using a VNS3 version prior to 4.8.3
After allocating the container and applying the necessary firewall rules to VNS3, you can SSH into the container on port 44.
The username is
container_admin, and the default password is
We recommend that you change this password immediately:
~$ sudo passwd container_admin
If you are using a version of VNS3 prior to 4.8.3, SSH into the container, and cd into the
/opt/awslogs/ directory. You will need to know what AWS region the VNS3 is deployed in for the next step.
curl http://169.254.169.254/latest/meta-data/placement/availability-zone to get the region. Do not include the letter at the end of the region. (i.e. us-east-1 instead of us-east-1b )
Once you have the region, from the
/opt/awslogs/ directory run the following command:
sudo python ./awslogs-agent-setup.py --region us-east-1 (change region accordingly)
This will launch the aws-logs agent in interactive mode. You do not have to specify the Access Key ID or Secret Access Key when prompted. Skip through till it asks for “Path of log to upload [var/log/syslog]:". Here, you will copy and paste the desired log path that will be sent to CloudWatch from the list below:
- /mnt/logs/kern.log - VNS3 kernel log
- /mnt/logs/syslog - VNS3 system log
- /mnt/logs/vnscubed_api_background/completed/ - Directory showing completed API calls
- /mnt/logs/vnscubed_connection_logs/ipsec.log - IPsec system log file
- /mnt/logs/vnscubed_connection_logs/tun0.log - Overlay Network server connection log files
Only select one log, you can add more later.
The next input is “Destination of Log Group name”. This must be identical to what you defined as the Log Groupname when you deployed the CloudFormation template. If you forgot the Log Groupname, simply go to the CloudWatch console and select Logs in the left column. You will see the Log Groupname under Log Groups.
For the Log Stream name, you must use the log path minus the /mnt/logs/ path.
If you are defined
/mnt/logs/vnscubed_connection_logs/ipsec.logas the “Path to log to upload”, you will need to define the logstream name as just
We keep the default “timestamp format” and “initial position” but feel free to change these settings at our own discretion. You will then have the option to add other log streams now, otherwise type n to complete the configuration.
The configuration files will be made although the logging agent will not run successfully until you add the following lines to the top of the /var/awslogs/etc/awslogs.conf file:
[general] state_file = /var/awslogs/state/agent-state
Once the awslogs.conf file is edited, run:
sudo service awslogs restart to pull in the completed configuration.
Editing the Configuration (post setup)
SSH into the container with the steps above. If you would like to add additional log streams or edit any log stream data, you can edit the configuration file found at
Variables that can be edited: log group name, log stream name, initial position, log file, date format, and buffer duration (default is the minimum, =5000ms).
To add additional log streams, copy the format that is in awslogs.conf and change the highlighted sections:
Once you have finished editing this file, you need need to start and stop the container in order for the new configuration to take effect.
Attaching IAMs Policies to Pre-existing Role
Two policies are made by the CloudFormation Template: CloudWatch List and Write, IAM Security Credential Pass Policy
If you have a pre-existing IAM Role attached to your VNS3 instance, you will need to manually add the policies created by the CloudFormation template to the pre-existing role.
The Role name will correlate to The CloudFormation Stack name. Copy and paste the two policies to the Role that is already attached to the instance.
Creating Additional CloudWatch Metric Filters and Alarms
Navigate to the CloudWatch Console and select Logs. Select the VNS3 Logs and Create Metric Filter.
Define a Filter Pattern and Assign the Metric.
Once the Metric Filter is created, you can create an Alarm based on the Filter. Select Create Alarm and define the appropriate parameters.
The container uses the AWS-logs agent to send system logs to AWS CloudWatch.
When the container is initially allocated, a startup script runs which creates a file with the groupname and log(s) streams you chose to send to CloudWatch (via the environment variables). This startup script also adds a file into the supervisord configuration /etc/supervisor/conf.d to make the AWS-logs agent run on boot.
Export a Container Image
In the event that your VNS3 controller needs to be replaced or upgraded, you will need a copy of your configured CloudWatch Agent Container. We recommend downloading an image of your configured container as part of the deployment process:
From the Containers page in the VNS3 Web UI, select Action > Save as Image for your CloudWatch Container. Once that process is complete, you’ll be brought to the Images page. Select Action > Export on the new image, and provide a name.
Once Exporting is complete, you will have the option to download the image locally.