Tag Archives: VCF

vRA and Tanzu Header

Walk through – Using vRA to deploy vSphere with Tanzu Namespaces & Guest Clusters

This walk through will detail the technical configurations for using vRA Code Stream to deploy vSphere with Tanzu supervisor namespaces and guest clusters.

Requirement

For a recent customer proof-of-concept, we wanted to show the full automation capabilities and combine this with the consumption of vSphere with Tanzu.

The end goal was to use Cloud Assembly and Code Stream to cover several automation tasks, and then offer them as self-service capability via a catalog item for an end-user to consume.

High Level Steps

To achieve our requirements, we’ll be configuring the following:

  • Cloud Assembly
    • VCF SDDC Manager Integration
    • Kubernetes Cloud Zone – Tanzu Supervisor Cluster
    • Cloud Template to deploy a new Tanzu Supervisor Namespace
  • Code Stream
    • Tasks to provision a new Supervisor Namespace using the Cloud Assembly Template
    • Tasks to provision a new Tanzu Guest Cluster inside of the Supervisor namespace using CI Tasks and the kubectl command line tool
    • Tasks to create a service account inside of the Tanzu Guest Cluster
    • Tasks to create Kubernetes endpoint for the new Tanzu Guest Cluster in both Cloud Assembly and Code Stream
  • Service Broker
    • Catalog Item to allow End-Users to provision a brand new Tanzu Guest Cluster in its own Supervisor Namespace
Pre-Requisites

In my Lab environment I have the following deployed:

  • VMware Cloud Foundation 4.2
    • With Workload Management enabled (vSphere with Tanzu)
  • vRealize Automation 8.3
  • A Docker host to be used by Code Stream

For the various bits of code, I have placed them in my GitHub repository here.

Configuring Cloud Assembly to deploy Tanzu supervisor namespaces

This configuration is detailed in this blog post, I’ll just cover the high-level configuration below.

  • Configure an integration for SDDC manager under Infrastructure Tab > Integrations

Continue reading Walk through – Using vRA to deploy vSphere with Tanzu Namespaces & Guest Clusters

VMware Cloud Foundation VCF Header

VCF – SDDC Manager – How to delete bundles

Just a quick blog post on deleting unnecessary or unneeded bundles from VCF – SDDC Manager.

There is two parts to this.

  • Getting your Bundle ID you want to delete from the API
  • Deleting the Bundle using a script on the SDDC Manager appliance.

In your SDDC Manager:

  1. Click Development Center
  2. Click API Explorer
  3. Expand “APIs for managing bundles”
  4. Expand the first “GET” command

VCF SDDC Manager - API - Get Bundles

  • Click Execute, no need to fill anything in

VCF SDDC Manager - API - Get Bundles - Execute

  • Download or Copy the response output.

VCF SDDC Manager - API - Get Bundles - Response

  • Find your Bundle ID within your output, you need to look for the top level ID of the JSON block, and ensure that this bundle says successfully downloaded.

VCF SDDC Manager API Get Bundles Response JSON Find Bundle ID

SSH to your SDDC Manager and elevate to root.

# su
{provide password to elevate to root}
# /opt/vmware/vcf/lcm/lcm-app/bin/bundle_cleanup.py {Bundle_id}

Example below
# /opt/vmware/vcf/lcm/lcm-app/bin/bundle_cleanup.py f004390e-26be-4690-9d7b-d447860e3169

VCF SDDC Manager bundle cleanup.py script

You will see the following output when the script has run.

-----------------------------------------------------
LOG FILE : /var/log/vmware/vcf/lcm/bundle_cleanup.log
-----------------------------------------------------
2021-03-08 12:18:31,809 [INFO] root: Performing cleanup for bundle with IDs : [' f004390e-26be-4690-9d7b-d447860e3169']
2021-03-08 12:18:31,809 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select count(*) from upgrade where upgrade_status in ('INPROGRES S','CANCELLING');"
2021-03-08 12:18:31,848 [INFO] root: b' 0\n'
2021-03-08 12:18:31,848 [INFO] root: b'\n'
2021-03-08 12:18:31,848 [INFO] root: RC: 0
2021-03-08 12:18:31,849 [INFO] root: Out: 0

2021-03-08 12:18:31,849 [INFO] root: Stopping LCM service.
2021-03-08 12:18:31,849 [INFO] root: Execute cmd: systemctl stop lcm
2021-03-08 12:18:32,290 [INFO] root: RC: 0
2021-03-08 12:18:32,290 [INFO] root: Out:
2021-03-08 12:18:32,291 [INFO] root: Removing LCM NFS mount.
2021-03-08 12:18:32,291 [INFO] root: Execute cmd: rm -rf /nfs/vmware/vcf/nfs-mou nt/bundle/f004390e-26be-4690-9d7b-d447860e3169
2021-03-08 12:18:32,683 [INFO] root: RC: 0
2021-03-08 12:18:32,684 [INFO] root: Out:
2021-03-08 12:18:32,684 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select upload_id from bundle_upload where bundle_id = 'f004390e- 26be-4690-9d7b-d447860e3169';"
2021-03-08 12:18:32,704 [INFO] root: b'\n'
2021-03-08 12:18:32,705 [INFO] root: RC: 0
2021-03-08 12:18:32,705 [INFO] root: Out:

2021-03-08 12:18:32,705 [INFO] root: Bundle with ID : f004390e-26be-4690-9d7b-d4 47860e3169 not found in bundle upload table
2021-03-08 12:18:32,706 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select download_id from bundledownload_by_id where bundle_id = ' f004390e-26be-4690-9d7b-d447860e3169';"
2021-03-08 12:18:32,724 [INFO] root: b' 0fb2e30e-d991-4b63-8686-42fab98a1c9e\n'
2021-03-08 12:18:32,724 [INFO] root: b'\n'
2021-03-08 12:18:32,725 [INFO] root: RC: 0
2021-03-08 12:18:32,725 [INFO] root: Out: 0fb2e30e-d991-4b63-8686-42fab98a1c9e

2021-03-08 12:18:32,725 [INFO] root: Execute cmd: curl -s -X DELETE localhost/ta sks/registrations/0fb2e30e-d991-4b63-8686-42fab98a1c9e
2021-03-08 12:18:32,830 [INFO] root: RC: 0
2021-03-08 12:18:32,830 [INFO] root: Out:
2021-03-08 12:18:32,830 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select upgrade_id from upgrade where bundle_id = 'f004390e-26be- 4690-9d7b-d447860e3169';"
2021-03-08 12:18:32,852 [INFO] root: b'\n'
2021-03-08 12:18:32,853 [INFO] root: RC: 0
2021-03-08 12:18:32,853 [INFO] root: Out:

2021-03-08 12:18:32,853 [INFO] root: Bundle with ID : f004390e-26be-4690-9d7b-d4 47860e3169 not found in upgrade table
2021-03-08 12:18:32,854 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select upgrade_id from upgrade where bundle_id = 'f004390e-26be- 4690-9d7b-d447860e3169';"
2021-03-08 12:18:32,873 [INFO] root: b'\n'
2021-03-08 12:18:32,874 [INFO] root: RC: 0
2021-03-08 12:18:32,874 [INFO] root: Out:

2021-03-08 12:18:32,874 [INFO] root: Bundle with ID : f004390e-26be-4690-9d7b-d4 47860e3169 not found in upgrade table
2021-03-08 12:18:32,875 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select count(*) from bundle where bundle_id = 'f004390e-26be-469 0-9d7b-d447860e3169';"
2021-03-08 12:18:32,894 [INFO] root: b' 1\n'
2021-03-08 12:18:32,895 [INFO] root: b'\n'
2021-03-08 12:18:32,895 [INFO] root: RC: 0
2021-03-08 12:18:32,895 [INFO] root: Out: 1

2021-03-08 12:18:32,896 [INFO] root: Deleting bundle & upgrade info for bundle I D : f004390e-26be-4690-9d7b-d447860e3169
2021-03-08 12:18:32,896 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -c "delete from bundle where bundle_id = 'f004390e-26be-4690-9d7b-d44 7860e3169';"
2021-03-08 12:18:32,923 [INFO] root: b'DELETE 1\n'
2021-03-08 12:18:32,924 [INFO] root: RC: 0
2021-03-08 12:18:32,924 [INFO] root: Out: DELETE 1

2021-03-08 12:18:32,924 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select count(*) from image where bundle_id = 'f004390e-26be-4690 -9d7b-d447860e3169';"
2021-03-08 12:18:32,943 [INFO] root: b' 1\n'
2021-03-08 12:18:32,943 [INFO] root: b'\n'
2021-03-08 12:18:32,943 [INFO] root: RC: 0
2021-03-08 12:18:32,944 [INFO] root: Out: 1

2021-03-08 12:18:32,944 [INFO] root: Deleting bundle f004390e-26be-4690-9d7b-d44 7860e3169 in image table
2021-03-08 12:18:32,944 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -c "delete from image where bundle_id = 'f004390e-26be-4690-9d7b-d447 860e3169';"
2021-03-08 12:18:32,967 [INFO] root: b'DELETE 1\n'
2021-03-08 12:18:32,967 [INFO] root: RC: 0
2021-03-08 12:18:32,967 [INFO] root: Out: DELETE 1

2021-03-08 12:18:32,968 [INFO] root: Execute cmd: psql --host=localhost -U postg res -d lcm -tc "select count(*) from partner_bundle_metadata where bundle_id = ' f004390e-26be-4690-9d7b-d447860e3169';"
2021-03-08 12:18:32,990 [INFO] root: b' 0\n'
2021-03-08 12:18:32,990 [INFO] root: b'\n'
2021-03-08 12:18:32,990 [INFO] root: RC: 0
2021-03-08 12:18:32,990 [INFO] root: Out: 0

2021-03-08 12:18:32,990 [INFO] root: Bundle with ID : f004390e-26be-4690-9d7b-d4 47860e3169 not found in partner_bundle_metadata table
2021-03-08 12:18:32,991 [INFO] root: Starting LCM service.
2021-03-08 12:18:32,991 [INFO] root: Execute cmd: systemctl start lcm
2021-03-08 12:18:33,135 [INFO] root: RC: 0
2021-03-08 12:18:33,136 [INFO] root: Out:

Going back into your SDDC Manager UI, and clicking the Bundle Management page, you will see your bundle has now been deleted.

It will take a few minutes for the Bundle services to restart, and you may see the message “Depot still initializing”.

Regards

 

VMware Cloud Foundation Header

Nested VCF Lab – Error while creating NFS datastore

Whilst deploying my nested VCF environment for my home lab, I kept hitting the same issue over and over again, even when I rolled the environment back and redeployed it.

VCF bring up error while creating NFS datastore

Error while creating NFS Datastore for host XXX.XXX.XXX.XXX

Looking into the debug log files on the Cloud Builder appliance found in the below location;

vcf-bringup-debug.log /var/log/vmware/vcf/bringup/

You can see basically the same error message, and not much help.

ERROR [c.v.e.s.o.model.error.ErrorFactory,pool-3-thread-7] [TP9EK1] VCF_HOST_CREATE_NFS_DATASTORE_FAILED

And the log ends with the below comments, I’ve left my Task ID numbers in, but obviously these are unique to my bring up;

DEBUG [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-3-thread-7] Collected the following errors for task with name CreateNFSDatastoreOnHostsAction and ID 7f000001-6ed0-12cd-816e-d1f7a33f006f: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.vcf.common.fsm.plugins.action.hostmessages)]]

DEBUG [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-3-thread-19] Invoking task CreateNFSDatastoreOnHostsAction.UNDO Description: Mount Repository NFS Datastore on ESXi Hosts, Plugin: HostPlugin, ParamBuilder null, Input map: {hosts=SDDCManagerConfiguration____13__hosts, nasDatastoreName=SDDCManagerConfiguration____13__nasDatastoreName, nfsRepoDirPath=SDDCManagerConfiguration____13__nfsRepoDirPath, repoVMIp=SDDCManagerConfiguration____13__repoVMIp}, Id: 7f000001-6ed0-12cd-816e-d1f7a33f006e ...

DEBUG [c.v.e.s.o.c.c.ContractParamBuilder,pool-3-thread-19] Contract task Mount Repository NFS Datastore on ESXi Hosts input: {"hosts":[{"address":"172.18.30.10","username":"root","password":"*****"},{"address":"172.18.30.11","username":"root","password":"*****"},{"address":"172.18.30.12","username":"root","password":"*****"},{"address":"172.18.30.13","username":"root","password":"*****"}],"nasDatastoreName":"lcm-bundle-repo","nfsRepoDirPath":"/nfs/vmware/vcf/nfs-mount","repoVMIp":"172.18.30.50"}

DEBUG [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-3-thread-19] Collected the following errors for task with name CreateNFSDatastoreOnHostsAction and ID 7f000001-6ed0-12cd-816e-d1f7a33f006f: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.vcf.common.fsm.plugins.action.hostmessages)]]

WARN  [c.v.e.s.o.c.ProcessingOrchestratorImpl,pool-3-thread-10] Processing State completed with failure

INFO  [c.v.e.s.o.core.OrchestratorImpl,pool-3-thread-15] End of Orchestration with FAILURE for Execution ID 8c9c5ab1-e48a-414e-9c4d-8936e6f12c91

The Fix

I struggled with this one for a while, at first I considered maybe an IP address conflict with the SDDC manager appliance, but it wasn’t that, I had the same issue after trying again with a different IP address.

I discussed this with our internal support, and I was pointed to the direction of KB 1005948.

When I followed the article, I noticed that the default vmkernel used to access my subnet and the subnet of my SDDC manager was VMK2, which is assigned for VSAN traffic; Continue reading Nested VCF Lab – Error while creating NFS datastore

VMware Cloud Foundation

VMware Cloud Foundation – SDDC Manager License key is not valid

Quick blog post, there has been known issue in VMware Cloud Foundation 3.5, 3.7 & 3.8 that reports your SDDC manager license key as invalid as the below screenshot shows.

This has been listed in the “known issues” section of the release notes for each version, but there is no KB article for this.

Validation fails on SDDC Manager license.During the bringup process, the validation for the SDDC Manager license fails.

Workaround: Enter a blank license and proceed. You can enter the correct license value later in the process.

 

SDDC Manager License Key Not Valid

Regards

Dean