Tag Archives: vrops

vRealize Operations – Monitoring OpenShift Container Platform environments

The latest release of  vRealize Operations (the “manager” part of the product name has now been dropped), brings the ability to manage your Kubernetes environments from the vSphere infrastructure up.

The Kubernetes integration in vRealize Operations 8.1;

  • vSphere with Kubernetes integration:
    • Ability to discover vSphere with Kubernetes objects as part of the vCenter Server inventory.
    • New summary pages for Supervisor Cluster, Namespaces, Tanzu Kubernetes cluster, and vSphere Pods.
    • ​Out-of-the-box dashboards, alerts, reports, and views for vSphere with Kubernetes.
  • The VMware Management Packs that are new and those that are updated for vRealize Operations Manager 8.1 are:
    • VMware vRealize Operations Management Pack for Container Monitoring 1.4.3

Where does OpenShift Container Platform fit in?

All though the above highlighted release notes point towards vSphere with Kubernetes (aka project pacific), the Container monitoring management pack has been available for a while and has received a number of updates.

This management pack can be used with any of your Kubernetes setups. Bringing components into your infrastructure monitoring view;

  • Kubernetes;
    • Clusters
    • Nodes
    • Pods
    • Containers
    • Services

So this means you can add in your OCP environment for monitoring.

Configuring vRealize Operations to monitor your OpenShift Clusters

Grab the latest Container monitoring management pack to be installed in your vRealize Operations environment.

  1. Log in to the vRealize Operations Manager with administrator privileges.
  2. In the menu, select Administration and in the left pane select Solutions > Repository.
  3. On the Repository tab, click Add/Upgrade.
  4. Browse to locate the temporary folder and select the PAK file.
  5. Click Upload. The upload might take several minutes.
  6. Read and accept the EULA,and click Next.
  7. When the vRealize Operations Management Pack for Container Monitoring is installed, click Finish.

To link any Kubernetes to your environment for monitoring, you need to install the cAdvisor Daemon.  For OCP I used the cAdvisor YAML Definition on HostPort, secondly you need to create some credentials to authenticate to your cluster from your connection in vROPs.

Below is my token created from following the KB above, ensure you copy just the token itself, and when pasted there are no line breaks etc.

Through testing, I’ve found that token based authentication works well, and I followed this KB 75169, which gives you a sample YAML, creating this secret against Kube-System was fine in my lab environment, but for production usage, like all access and authorization creation you must understand the configuration and document.

If you want to review all your Kubernetes authentication options, see here.

Finally configure your Kubernetes Adapter in your vRealize Operations interface;

  1. From the main menu of vRealize Operations Manager, click Administration, and then in the left pane, click Solutions.
  2. From the Solutions list, select VMware vRealize Operations Management Pack for Container Monitoring.
  3. Click the Configure icon to edit an object.
  4. Enter the display name of the adapter.
  5. Enter the http URL of the Kubernetes master node in the Master URL text box.
  6. Select DaemonSet as the cAdvisor Service.
  7. Enter the port number of cAdvisor (Default is 31194)
  8. Enter the Credential details of the Master URL.
  9. Under advanced settings if the OCP cluster is running on vCenter Server which is monitored by vRealize Operations, you can view a link from the Kubernetes node to the vSphere Virtual Machine. To view the link, enter the IP address of the vCenter Server instance.

New Environment Views

You will find this management pack will create a lot of new objects in vRealize operations, one of them is a new K8S-World, which will hold the data of all the Kubernetes adapters configured in your environment, this grouping sits under the new Containers World.

In the below environment view, I am monitoring the memory utilisation of the three linked components.

  • Kubernetes Cluster Node > The Virtual Machine in vSphere > The VMware ESXi host the VM is running on.

Looking at the Dashboards

By default, you only get the Kubernetes overview dashboard, which is built upon some of the imported views.

However I also highly recommend you try out the Kubernetes Namespace dashboard created by my colleague Simon Conyard.

The Overview Dashboard is split into three main areas;

  • Kubernetes Cluster Summary showing cluster alerts figures
  • Performance of Cluster nodes
  • Performance of Pods & Containers

Select your Kubernetes cluster, which will relate to the name provided when configuring your Kubernetes adapter earlier in the configuration. As you can see here, this dashboard is Kubernetes platform agnostic, and I took this screenshot just as a VMware Tanzu Kubernetes Grid platform was added to vRealize Operations.

By selecting the active alerts, I will get a summary of which objects have triggered in the cluster, which I can select and view in the environment section.

Continuing down the dashboard, we then focus on the nodes, where I’ve selected my lowest health node.

We can see the node properties, pods relationship, key metrics and even pick our own metrics and properties to view for that node as well.

Below we can see my node health is reduced due to high memory consumption.

Finally, the last part of the dashboard looks at pods and containers. In the below we can see my API server pod availability has been all over the place. So I really need to look at this if I want to continue giving demos with this environment.

Finally I just want to highlight my favourite piece of the dashboard, which is the associated components, as below it shows you everything that is brought together as part of an app deployment in Kubernetes, but as a VI admin it can really help you visualise how everything is brought together, even if you’ve never used Kubernetes before.

Summary

Although this blog post points towards OpenShift Container Platform. There is nothing which is exclusive to OCP. Which for me shows the power of the vRealize Operations platform, the ability to be agnostic of the Kubernetes platform which it monitors. We could easily do a Find all + replace on this blog post and change OpenShift/OCP to Azure Kubernetes Service, and everything would remain the same. Alternatively you could read this blog post.

Regards

vROPs Dashboard – Display port group with VLAN ID and map to virtual machines

My customer asked me how to display the VLAN ID for a virtual machine in vROPs.

The short answer is, due to the VLAN ID only being available from the Distributed switch adapter, its not simple to display this in a single list view.

So after some thinking around this, I came up with the following simple dashboard which was edited from the default “Distributed Switch Configuration” dashboard.

The dashboard is simple, and shows a selectable list of port groups, which then will refresh the virtual machine list, so you can see which virtual machines are attached to that port group.

You can download the dashboard and edited view here:

 

Regards

vRSLCM 8.0 – vROPs 7.5 upgrade fails due to Admin password expiry

When the vRealize 8 products dropped, I was like a kid in a sweet shop, upgrading everything as quick as possible before my customers tried to, so I could encounter any issues first, but also the new features, so I could show them off.

The issue

During the upgrade of vROPs, I hit an issue that my Local Admin account in vROPs had expired, but I received no warning when using the vROPs 7.5 instance and logged into the interface using the Admin account.

Before I found the issue;

During the upgrade in vRSLCM, my upgrade task failed with “vROPS upgrade failure”, Error Code: LCMVROPSYSTEM25008, Upgrade.pak_pre_apply_validate_failed.

Continue reading vRSLCM 8.0 – vROPs 7.5 upgrade fails due to Admin password expiry

vROPs – What is the “IsGreenForPlacement” Metric + Dashboard

A customer of mine queried the details of a metric available in vROPs “IsGreenForPlacement”

You can find this by selecting a cluster in vROPs, go to All Metrics, and just search placement.

And here is a screenshot of the Metric in a sparkline.

The customer uses this metric to give a signal (Green/Red Button on a vROPs Dashboard) if a vSphere cluster can be used for on-going deployments.

Unfortunately there’s not much documented information publicly. And we ran into an  issue where the metric stayed positive (yes you can deploy), but the Storage datastore had run out of space. So I went off to dig out what this metric actually does.

IsGreenForPlacement – details

After speaking to the internal teams on vROPs I found the answer;

“Regarding IsGreenForPlacement metric, only CPU and Memory participates on calculation of this metric, by default if CPU and MEM workload is less than 80% it is green.”

So storage is not taken into account.

Creating a Dashboard

This one will be really simple. Continue reading vROPs – What is the “IsGreenForPlacement” Metric + Dashboard

Using vROPs to track adoption – Such as VMware Tools upgrades

I had an interesting question from a customer recently;

Can we track the adoption/upgrades of VMware tools to the latest version in a vROPs dashboard

At first, I thought sure this is easy. But then looking at the various different default options, nothing would give me a graph which shows the increasing uptake or upgrades of VMware Tools to the last version.

I consulted internally, and between myself and my customer we came up with the following solution. Which was quite simple once we thought it out.

Solution

  1. Create a custom group based on the information you want to capture

In the below example, I’ve done this to target VMs with VMware Tools 10.3.10, you could leave it as “10.3” to target any 10.3.X release. Continue reading Using vROPs to track adoption – Such as VMware Tools upgrades