CloudWatch Logs
CloudWatch Logging Plugin Detail
The CloudWatch Logging Plugin was created for VNS3 version 4.8.3+. If you are using a VNS3 version prior to 4.8.3, please contact Cohesive Support via email, support@cohesive.net.
Getting Started with VNS3 Plugin System
The AWS CloudWatch Logs agent is deployed to VNS3 (version 4.8.3+) as a plugin using the Plugin Catalog via the VNS3 Web UI. Please contact Cohesive Support if you would like to install and confirgure the plugin via the RESTful API.
Please be familiar with the VNS3 Plug-In Configuration Guide.
CloudWatch Logging Plugin - What does it do?
The CloudWatch Logging Plugin pulls live logs from a host VNS3 Controller via the AWS CloudWatch Logs agent and displays these logs in AWS CloudWatch for further processing/visibility. A CloudFormation template is used to create all the necessary AWS components automatically (IAMS Role, Cloudwatch Log Group, alarms, etc.). The CloudFormation template creates metrics filters based on potential log messages that notify an SNS topic when triggered, via text message or email.
In addition to the default IPsec log alerts our CloudFormtaion Template creates, you can create custom Alarms in CloudWatch to send you alerts based on specific log messages or patterns.
The goal is to minimize outages by monitoring the VNS3 controller’s IPsec, overlay, and/or system logs by sending automatic alerts when unfavorable conditions arise.
CloudWatch Logging PLugin - What does it need?
User input to deploy the CloudWatch Logs Agent:
- Create CloudFormation Stack via the template URL
- Define a log Groupname and SNS topic (email or phone number)
- Attach Role to VNS3 Instance
- Install the CW Logging Pugin from the VNS3 PLugin Catalog
- Start a Plugin Instance of the CW Logging image
- Add the appropriate VNS3 firewall rules
- Edit configuration values (log_group_name & log_stream_name) in the awslogs.json configuration file
Deploying the CloudWatch Logging Plugin
Deploying the CloudFormation Template
Create a new CloudFormation Stack with the following URL:
https://vns3-containers-read-all.s3.amazonaws.com/CloudWatch_Logger/CWloggerCFtemplate_20231211.yaml
This is a read-only Amazon S3 storage location. Only Cohesive Networks can update or modify files stored in this location.
The CloudFormation template will automatically create all the AWS components (IAM Role, SNS topic, CloudWatch metric filters, alarms, and log group). You will need to define a unique log Groupname and provide an email address that will be used for the SNS topic.
NOTE: Once the CloudFormation template is deployed, you will receive an SNS subscription confirmation email. Confirm the subscription BEFORE deploying the plugin.
The CloudFormation template creates an IAM Role; attach this IAM Role to your existing VNS3 controller (the Role name will correlate to the CloudFormation Stack name). From the EC2 Console, select the VNS3 controller, Select Actions > Instance Settings > Attach/Replace IAM Role 
If you already have an IAM role attached to the instance, you must apply the two new policies to the existing Role. (see Developers section below)
Launching the CloudWatch Logging Plugin
From the VNS3 Web UI, Navigate to the Catalog page under the Plugins tab. Scroll down to the CloudWatch Plugin and select Install.
NOTE: If your controller is in a private subnet and it is unable to fetch the image from the Plugin Catalog, use this URL to manually fetch the image from a public S3 bucket"
Navigate to the Plugins Dashboard. When the Image has been imported successfully, it will say Ready in the Status Column.
In the Actions menu, select Start Instance. Name the plugin, select a plugin IP, then click Start. Once the instance is started, go to the Instances Tab.
In the Actions dropdown menu, select Manage.
CloudWatch Plugin Firewall Rules
The CloudWatch Plugin requires the following firewall rules to be added to the VNS3 controller BEFORE running the UPDATE Executable.
Navigate tothe VNS3 Firewall page and paste in the following three rules, one at a time. Replace <plugin_ip> with the IP that is assigned to your CloudWatch Plugin. If these rules are not in place, you will see errors in the CW Agent Log.
#SNAT for Plugin network
POSTROUTING -o eth0 -s <plugin_ip> -j MASQUERADE
FORWARD -s <plugin_ip> -j ACCEPT
FORWARD -d <plugin_ip> -j ACCEPT
Configuring the CloudWatch Logging Agent
In the awslogs.json configuration file, deselect the “Wrap Text” checkbox at the bottom of the page, select “edit”, then change the log_group_name parameter to match what you defined in your CloudFormation Template. The default configuration sends the syslog, ipsec.log, and tun0.log to CloudWatch [SEE awslogs.json]. If you would like to add or change what logs are being sent to CloudWatch, you will need to change the file_path and log_stream_name parameters as well. If you are sending logs not generated by VNS3 itself, make sure the timestamp_format parameter matches the log date/time format. If it is incorrect, you will see errors in the CW Agent Log. Once your awslogs.json file is saved, RUN the UPDATE Executable to initialize the agent to CloudWatch. It takes 10-15 minutes before you will start seeing the logs in your AWS CloudWatch console.
Additional Configuration Information
Here are the VNS3 system logs which can be offloaded to CloudWatch for monitoring and alerting:
- auth.log
- boot.log
- dmesg
- kern.log - VNS3 kernel log
- syslog - VNS3 system log
- vnscubed_api_background.log
- vnscubed_api_server.log
- vnscubed_bgpmon.log
- vnscubed_monitor.log
- vnscubed_webui.log
- webserver_access.log
- webserver_error.log
- vnscubed_api_background/completed/ - Directory showing completed API calls
- vnscubed_connection_logs/ipsec.log - IPsec system log file
- vnscubed_connection_logs/tun0.log - Overlay Network server connection log files
- vnscubed_connection_logs/unauthorized.log
If you would like to send multiple logs to CloudWatch, use a space-separated list that is entirely in quotations, as seen in the picture below:
Attaching IAMs Policies to Pre-existing Role
Two policies are made by the CloudFormation Template: CloudWatch List and Write IAM Security Credential Pass Policy.
If you have a pre-existing IAM Role attached to your VNS3 instance, you must manually add the policies created by the CloudFormation template to the pre-existing role.
The Role name will correlate to The CloudFormation Stack name. Copy and paste the two policies to the Role that is already attached to the instance.
Creating Additional CloudWatch Metric Filters and Alarms
Navigate to the CloudWatch Console and select Logs. Select the VNS3 Logs and Create Metric Filter.
Define a Filter Pattern and Assign the Metric.
Once the Metric Filter is created, you can create an Alarm based on the Filter. Select Create Alarm and define the appropriate parameters.
For Developers
The plugin uses the AWS-logs agent to send system logs to AWS CloudWatch. Logging Agent reference found HERE
When the plugin is initially allocated, a startup script downloads the latest AWS-logs agent. Once you edit the awslogs.json configuration file as needed, run the UPDATE executable command to push your configuration to CloudWatch. It normally takes 10 minutes until you start seeing the log streams in your CloudWatch Console. If you want to make edits to the confiruation, edit the awslogs.json file, save a new version, then run the UPDATE executable again. If you would like to stop the CW Logging agent, select STOP from the Process manager OR stop the entire plugin from the Plugin’s Actions dropdown menu.
Export a Plugin Image
In the event that your VNS3 controller needs to be replaced or upgraded, you will need a copy of your configured CloudWatch Agent Plugin. If you are using version 6.x or newer, your CW plugin configuration will come along if you upgrade via a VNS3 Snapshot. We recommend downloading an image of your configured plugin as part of the deployment process:
From the Plugins page in the VNS3 Web UI, select Action > Save as Image for your CloudWatch PLugin. Once that process is complete, you’ll be brought to the Images page. Select Action > Export on the new image, and provide a name.
Once Exporting is complete, you will have the option to download the image locally.