In order to get a complete view of your network traffic flows between applications inside Network Insight, it should know things about your application stacks. This post will go into how to create an integration with vRealize Automation in order to get all new application stacks into Network Insight. This will happen on the moment that an application is deployed from a blueprint, instantly transferring info so every greenfield application will give context in Network Insight.
Disclaimer: This is something I built, it is not officially supported by VMware.
As you may know by now, Network Insight listens to everything going on inside your network and creates intelligible information out of it. You get a crystal clear donut-shaped diagram that tells you, in one view, exactly which applications and workloads communicate with each other. Furthermore, it gives you a set of recommended firewall rules that are applicable to your applications using an advanced learning engine. These recommended firewall rules can be used to create a micro-segmented environment and accelerating the security of your applications significantly.
vRealize Automation
Assuming you already know about vRealize Automation (if not, go read up first ?), I’ll dive straight in. As an example, I’ll be using a 3 tiered application blueprint that contains a web server, application server, and database server tier. Of the web and application tiers, there can be multiple VMs deployed but there’s just one database server. They are linked to vSphere templates which will be cloned when a deployment is requested.
We can use this blueprint design to extract the application stack structure and automatically insert that information into Network Insight using its open API. The Blueprint name will be used for the application name and the VM tier names will be used as the tier names inside Network Insight.
Below is a demonstration video on how this process works:
* The first 8,5 minute is around applications within Network Insight. If you want to skip to vRA, skip to 8:23
vRealize Orchestrator
Doing anything that takes information from vRA and pushes it to another system, usually uses vRealize Orchestrator (vRO) as the orchestration/execution engine. This case is no different. There’s currently no vRO plugin that presents you with standard actions that can be used for Network Insight, so I’ll show you the code that can be used in a scripted task. Roughly, this is what happens from a deployment to talking to the Network Insight API:
vRO Workflow
Create a new workflow inside vRO and give it a name that you can remember. This workflow consists only of scripted tasks and will first retrieve information from vRA about the new deployment: name and VMs with tier names. Here’s how it’ll look:
First scriptable task:
- IN Parameter: payload (Properties type)
- OUT Parameters: deploymentName (String type), tierInfo (Array type)
- Script can be found here.
Second scriptable task:
- IN Parameters: deploymentName (String type), tierInfo (Array type)
- Script can be found here.
vRealize Automation has to be added into the Orchestrator instance for it to connect to it and get information about the deployments. Use the workflow vRealize Automation -> Configuration -> Add a vRA host to do so.
If you want to make it easier, you can also import it into vRO from this file:
vRA-New-Application-to-NetworkInsight.workflow
(md5 checksum = 41759bbe4952200a514e2d2601a5c6ff)
Workflow Step-by-Step
I’ll explain exactly what I’m doing here, going through the scriptable tasks step by step.
Scriptable Task 1
// IMPORTANT: This needs to be your vRA host. In my case it happened to the third server configured.
// there is probably a better way to handle this.
var vraHost = Server.findAllForType("vCACCAFE:VCACHost")[2];
// save the catalogRequestId from the payload given by vRA
var requestId = payload.get("catalogRequestId");
// create a REST client on the selected vRA host, which we can use to do API calls against
var client = vraHost.createCatalogClient();
// get the resources (read: VMs) that are a part of the catalogRequestId
var response = client.get("/consumer/requests/" + requestId + "/resources").getBodyAsJson();
This uses the built-in functions to connect to vRA and retrieve information on the deployment that is given to the workflow. The payload variable contains a key named catalogRequestId, which contains the ID of the deployment. The response variable will contain the deployment info, including the VMs.
// placeholder variables which we can fill; a list of VMs included in this deployment and the name of the deployment
var vm_list = new Array();
// go through the response of the API call, which is too large to give an example here
for(var x in response.content)
{
var resource = response.content[x];
// if the resourceTypeRef { "id": "Infrastructure.Virtual" } -> it's a virtual machine
if(resource.resourceTypeRef.id == "Infrastructure.Virtual")
{
// placeholder var for the VM tier
var tierName = "";
// go find the tier name in the key/value array, the key is called "Component"
for(var k in resource.resourceData.entries) {
var property = resource.resourceData.entries[k];
if(property.key == "Component") tierName = property.value.value;
}
// construct an array to hold the VM info that we want to store and save it to vm_list for later use
var vm_info = new Array();
vm_info['name'] = resource.name;
vm_info['tier'] = tierName;
vm_info['id'] = resource.id;
vm_list.push(vm_info);
}
// if the resourceTypeRef { "id": "Infrastructure.Virtual" } -> it's the deployment itself and we want its name
if(resource.resourceTypeRef.id == "composition.resource.type.deployment") {
deploymentName = resource.name;
}
}
Here we parse the response variable and look for VM components and the deployment name and store these in variables.
tierInfo = new Array();
// Do things with the deployment name and list of VMs we've just queried vRA for.
System.log("Deployment name: " + deploymentName);
// List VMs and sort out the tier names and put them in the OUT parameter 'tierInfo'
for(var r in vm_list)
{
var tierName = vm_list[r].tier;
var vmName = vm_list[r].name;
if(tierInfo[tierName] == null) {
tierInfo[tierName] = new Array()
}
tierInfo[tierName].push(vmName);
}
We’ve stored info VM specific so far, but we want to be tier specific. The Network Insight API calls are going to be per tier. After structuring the right variables, deploymentName and tierInfo will be passed onto the second scriptable task.
Scriptable Task 2
// update the first 2 with the information of your Network Insight instance and credentials
var baseUrl = 'https://network-insight.platform.local';
var content = '{ "password": "VMware1!", "username": "admin@local", "domain": { "value": "local", "domain_type": "LOCAL" } }';
// leave these ones
var queryString = '/api/ni/auth/token';
var httpMethod = 'POST';
// create REST object to perform API calls with
var restHost = RESTHostManager.createHost("DynamicRequest");
var transientHost = RESTHostManager.createTransientHostFrom(restHost);
transientHost.url = baseUrl;
// put together the call to get an authtoken for Network Insight
var requestUrl = baseUrl + queryString;
var request = transientHost.createRequest(httpMethod, queryString, content);
request.contentType = 'application/json';
// energize!
var response;
response = request.execute();
// determine auth token from JSON output
var jsonObj = JSON.parse(response.contentAsString);
var authToken = jsonObj.token;
This first part creates a few objects to manage REST calls and then requests a authentication from Network Insight. Don’t forgot to modify the first 2 lines and point them towards your own Network Insight instance.
/////// create application
// put together the API call params
queryString = '/api/ni/groups/applications';
content = '{ "name": "' + deploymentName + '" }';
requestUrl = baseUrl + queryString;
// update to new params
System.log("Request full URL: " + requestUrl);
request = transientHost.createRequest(httpMethod, queryString, content);
request.setHeader("Authorization", "NetworkInsight " + authToken);
request.contentType = 'application/json';
// energize!
var response;
response = request.execute();
// get application entity ID
var jsonObj = JSON.parse(response.contentAsString);
var appEntityID = jsonObj.entity_id;
We’ve now created the application container inside Network Insight and saved the appEntityID which is returned on a successfull creation.
/////// create application tiers
queryString = "/api/ni/groups/applications/" + appEntityID + "/tiers";
// go through each tier that's in tierInfo and put together an API call for each tier
for(var tier in tierInfo)
{
var vmFilter = '';
var vm_list = tierInfo[tier];
// find VMs in this tier and put their names in the filter format
for(var vmName in vm_list)
{
if(vmFilter != "") {
vmFilter += " or ";
}
vmFilter += "name = '" + vm_list[vmName] + "'";
}
// put together the API call params
content = '{ "name": "' + tier + '", "group_membership_criteria": [ { "membership_type": "SearchMembershipCriteria", "search_membership_criteria": { "entity_type": "BaseVirtualMachine", "filter": "' + vmFilter + '" } } ] }';
requestUrl = baseUrl + queryString;
// update to new params
request = transientHost.createRequest(httpMethod, queryString, content);
request.setHeader("Authorization", "NetworkInsight " + authToken);
request.contentType = 'application/json';
// energize!
var response;
response = request.execute();
}
The final piece creates the tiers using the appEntityID to create them under. It received the wanted tier names and the included VMs and builds a VM name filter to use in this new tier, referencing the VMs that were created.
vRealize Automation Event Broker
vRO is called when vRA finishes the deployment of the VMs and the infrastructure (networking, storage, compute) via the Event Broker. This is a model inside vRA where you can subscribe to certain events and start vRO workflows when that event happens. In our case, we need the Blueprint request completed event, which will execute when a deployment has finished and all VMs are running.
To create this subscription, go to Administration – Events and then Subscriptions. Create a subscription there with the above mentioned event and select your workflow.
Watch it in action!
Once you’ve set up the workflow and subscribed it to the Event Broker, go ahead and do a deployment to see if it works. In the end, you should see something like this in both vRA and Network Insight:
|
|
| App in vRA | App in Network Insight |
Conclusion
If you’ve come this far; thanks for sticking with me! Network Insight integrations are an important part of gaining control of your application landscape and help keep it secure. This (long) post was based on customer demand which involved vRealize Automation. However, because the Network Insight API can be called from anything that supports HTTP calls; you can integrate any automation/orchestration product.
Let me know if you looking at other integrations, always good to see what you’re doing with the Network Insight API! 🙂
Leave a Reply