Tag Archives: 8.0

vRealize Automation 8.0 – Wildcard SSL certificate support and deployment issues – LCMVRAVACONFIG590003

Ok, so I’m just going to call it out straight away, when using wildcard SSL certificates with vRealize Automation 8.0, read the release notes.

I did not, and caused myself quite a few headaches with the deployment, which you can read about further in this post.

Cannot set wildcard certs for certain domain names, specifically those not using a Public Suffix.

vRealize Automation 8.0 supports setting a wildcard certificate only for DNS names that match the content of the Public Suffix List ([https://publicsuffix.org/]) 

For example, a valid wildcard certificate: you can use a wildcard certificate with DNS name like "*.myorg.com". This is supported because "com" is part of the Public Suffix List. 

An invalid wildcard certificate example: you cannot use a wildcard certificate with DNS name like "*.myorg.local".This is not supported because "local" is not part of Public Suffix List. 

Workaround: Only use domain names in the Public Suffix List.

The issues caused by using an unsupported wildcard SSL

When deploying vRA 8.0 via vRSLCM, either as part of the easy installer or as part of an existing vRSLCM setup, you will asked to provide an SSL certificate.

This does not validate your certificate is supported for use with the vRA 8.0 deployment. vRSLCM will do some checking on the SSL selected, but is only to ensure the SSL certificate is not about to expire, you will see a Green tick and “healthy” status as below.

Once you hit deploy, you will find your vRA appliance finally stood up, however the initialization tasks will stall.

Error Code: LCMVRAVACONFIG590003
Cluster Initialization failed on VRA.

vRA Initialize Cluster failed on vRA VA - ***Hostname***. Please login to the vRA and check /var/log/deploy.log file for more information on failure.

We know the new architecture is based on Kubernetes, you can read about this here.

So we log into the appliance to see what exactly is going on.

We can check the health of vRA appliance with the following commands;

vracli status

vracli status deploy

In the deploy command we are pointed to an issue (which we already know is the case) (I also forgot to get a screenshot of this).

By using the ‘kubectl’ command line, we are able to see that the namespace ‘prelude’ is having service issues, when we look into the services that make up this namespace we see that the offending service is “user-profile-service-app-XXXX”, and also the “cgs-service-XXXX” is affected too.

I restarted the pods by deleting the existing pods and letting kubernetes restore the state, however the services still did not run.

Note: It is not supported by VMware for the kubernetes environment running inside of the vRA Components to be altered without strict guidance via a VMware Support ticket and with assistance from our GSS Team!

Basically, if you do what I did, you are on your own!

You can get the support bundle from the vRA appliance by using SSH to connect to the appliance and running the following command;

vracli log-bundle

By doing this on my environment and reviewing the logs, we found;

logs from user-profile-service-app are as follows,

Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://sc-dc1-vra001.simon.local/csp/gateway/am/api/auth/token-public-key": Certificate for <sc-dc1-vra001.simon.local> doesn't match any of the subject alternative names: [*.simon.local]; nested exception is javax.net.ssl.SSLPeerUnverifiedException: Certificate for <sc-dc1-vra001.simon.local> doesn't match any of the subject alternative names: [*.simon.local]

So there we have it, after troubleshooting the issue, I confirmed the issue, then found out its part of the release notes.

There is an open action to allow for Wildcard support for non public suffixes in the future of vRA 8.0.

Regards

Dean

vRSLCM 8.0 – vROPs 7.5 upgrade fails due to Admin password expiry

When the vRealize 8 products dropped, I was like a kid in a sweet shop, upgrading everything as quick as possible before my customers tried to, so I could encounter any issues first, but also the new features, so I could show them off.

The issue

During the upgrade of vROPs, I hit an issue that my Local Admin account in vROPs had expired, but I received no warning when using the vROPs 7.5 instance and logged into the interface using the Admin account.

Before I found the issue;

During the upgrade in vRSLCM, my upgrade task failed with “vROPS upgrade failure”, Error Code: LCMVROPSYSTEM25008, Upgrade.pak_pre_apply_validate_failed.

I downloaded the vRLCM support bundle to see if I could find more information on why the upgrade failed but just found the same information.

See this documentation link for the logs location on the appliance.

The log file you are interested in is;

/vlcmsupport-XXX/var/log/vrlcm/vmware_vrlcm.log

And below you can see the same error information in the log.

I then proceeded to troubleshoot the vROPs environment directly, as we know the upgrade was underway before the failure.

  • You can follow the instructions to create a support bundle here KB 2074601.

For upgrades, we need to look at the following file;

/Log Bundle/Logs/pakManager/vcopsPakManager.root.query.log

And it is here in this log file, I could see the  Query status result;

“Admin_account_check”

With another upgrade validation log to check.

"resource_arguments": [
"admin_account_check",
"/var/log/vmware/vcops/vcops_upgrade_validation_20191020-103109.log"
],

I then used Putty to connect to my vROPs instance, and low and behold, I was prompted to change the password of my Admin account.

 

The Fix

Well it’s simple, change the admin password as above, and then update the password stored in vRSLCM, re-run the upgrade and this time if the stars align, it will be successful.

Or follow this guide if you are locked out;

Regards

Dean