How to Escape Strings in Terraform with a Dollar Sign ($)

July 14, 2021GeneralCloud Templates (blueprints), dollar sign, escape, interpolation syntax, Invalid character, invalid character error, Terraform, vRA ProviderDean

The Issue

When using Terraform to perform an action, and the input is using a $, you can end up with an output such as the below.

│ Error: Invalid character
│ 
│  on main.tf line 104, in resource "vra_blueprint" "this":
│ 104:      network: '${resource.Cloud_Network_1.id}'
│ 
│ This character is not used within the language.

This happened to me when I was using the Terraform vRA Provider to create Cloud Templates (blueprints) in my vRA environment. The vRA cloud templates use a syntax such as ${input.something}, which clashes with the syntax used by Terraform to identify inputs.

The Cause

Terraform implements a interpolations syntax. These interpolations are wrapped in ${}, such as ${var.foo}.

The interpolation syntax is powerful and allows you to reference variables, attributes of resources, call functions, etc.

The Fix

You can escape interpolation with double dollar signs: $${foo} will be rendered as a literal ${foo}.

Regards

Follow @Saintdle

Dean Lewis

Authenticate Docker to Harbor Image Registry with a Robot Account

July 13, 2021VMware, Kubernetesdocker, Harbor, Image Registry, LoginDean

Just a quick blog post on how to authenticate Docker to a Harbor Image Registry, using a Robot Account, which is good for programmatically access to push/pull images from your registry.

Harbor introduced the capability for administrators to create system robot accounts you can use you run automated actions in your Harbor instances. System robot accounts allow you to use a robot account to perform maintenance or repeated task across all or a subset of projects in your Harbor instance.

In Harbor, to create your robot account, within your project:

Click the Robot Accounts tab
Click New Robot Account
Enter your details as necessary and click save
Select export to file and save the JSON file.

This file will be outputted with a “$” in the name. You can either change the name of the file, or escape the character in the terminal, which I what I’ll be doing below.

Within your terminal run the following commands:

username=$(cat file.json | jq -r .name)
password=$(cat file.json | jq -r .token) <<< See below update
echo "$password" | docker login https://URL --username "$username" --password-stdin

Example
username=$(cat "robot\$veducate.json" | jq -r .name)
password=$(cat robot\$veducate.json | jq -r .token)
echo "$password" | docker login https://harbor-repo.veducate.com --username "$username" --password-stdin

Update October 2021
Since Harbor 2.2 minor release, and I found that within the JSON the key name has changed to secret, so this is the updated example

username=$(cat robot-veducate.json | jq -r .name)
password=$(cat robot-veducate.json | jq -r .secret)
echo "$password" | docker login https://harbor-repo.veducate.com --username "$username" --password-stdin

Now you can pull/push images to your repository using this authentication details.

Regards

Follow @Saintdle

Dean Lewis

Tanzu Mission Control – Using custom policies to ensure Kasten protects a deployed application

July 12, 2021VMware, KubernetesCustom Policies, Kasten, Labels, OPA, Rego, Tanzu Mission ControlDean

The Issue

A while ago I was chatting to Michael Cade, and we pondered the question “How do we ensure Kasten is protecting a newly deployed application in our Kubernetes environment”.

We chatted about how one of the best ways to make your Kasten protection policy flexible is by using metadata labels.

We came up with the simple idea: “What if something forces a known label on the metadata of any applications deployed by our developers in the future?”

This blog post covers this use case using Tanzu Mission Control with custom policies.

The Solution

One of the products we can use to enforce labels on a Kubernetes resource is Open Policy Agent Gatekeeper. Which is an external admission controller which allows you to create policies for the admission of resource creation/changes/updates based on a criteria.

OPA policies are expressed in a high-level declarative language called Rego. (Pronounced “ray-go”.)

Tanzu Mission Control, the fleet management SaaS tool for managing your Kubernetes platforms, provides you the ability to create policies of various types to manage the operation and security posture of your Kubernetes clusters and other organizational objects, implemented by using the OPA Gatekeeper.

Implementing The Solution

For this solution “art of the possible” blog post, we are going to keep it really simple, and implement a policy which covers the following: Continue reading Tanzu Mission Control – Using custom policies to ensure Kasten protects a deployed application →

Deploying OpenShift clusters (IPI) using vRA Code Stream

June 30, 2021VMware, KubernetesCode Stream, Deploy, openshift, Tanzu Mission Control, vRADean

This walk-through will detail the technical configurations for using vRA Code Stream to deploy Red Hat OpenShift Clusters, register them as Kubernetes endpoints in vRA Cloud Assembly and Code Stream, and finally register the newly created cluster in Tanzu Mission Control.

The deployment uses the Installer Provisioned Infrastructure method for deploying OpenShift to vSphere. Which means the installation tool “openshift-install” provisions the virtual machines and configures them for you, with the cluster using internal load balancing for it’s API interfaces.

This post mirrors my original blog post on using vRA to deploy AWS EKS clusters.

Pre-reqs

Red Hat Cloud Account
- With the ability to download and use a Pull Secret for creating OpenShift Clusters
vRA access to create Code Stream Pipelines and associated objects inside the pipeline when it runs.
- Get CSP API access token for vRA Cloud or on-premises edition.
Tanzu Mission Control access with ability to attach new clusters
- Get an CSP API access token for TMC
vRA Code Stream configured with an available Docker Host that can connect to the network you will deploy the OpenShift clusters to.
- This Docker container is used for the pipeline
- You can find the Dockerfile here, and alter per your needs, including which versions of OpenShift you want to deploy.
SSH Key for a bastion host access to your OpenShift nodes.
vCenter account with appropriate permissions to deploy OpenShift
- You can use this PowerCLI script to create the role permissions.
DNS records created for OpenShift Cluster
- api.{cluster_id}.{base_domain}
- *.apps.{cluster_id}.{base_domain}
Files to create the pipeline are stored in either of these locations:
- https://github.com/saintdle/vRA-Deploy-OpenShift-Clusters
- https://code.vmware.com/samples?id=7635

High Level Steps of this Pipeline

Create an OpenShift Cluster
- Build a install-config.yaml file to be used by the OpenShift-Install command line tool
- Create cluster based on number of user provided inputs and vRA Variables
Register OpenShift Cluster with vRA
- Create a service account on the cluster
- collect details of the cluster
- Register cluster as Kubernetes endpoint for Cloud Assembly and Code Stream using the vRA API
Register OpenShift Cluster with Tanzu Mission Control
- Using the API

Creating a Code Stream Pipeline to deploy a OpenShift Cluster and register the endpoints with vRA and Tanzu Mission Control

Create the variables to be used

First, we will create several variables in Code Stream, you could change the pipeline tasks to use inputs instead if you wanted. Continue reading Deploying OpenShift clusters (IPI) using vRA Code Stream →

Passing JSON into vRA Code Stream CI Task – MalformedJsonException

June 30, 2021GeneralCode Stream, JSON, Variable, vRADean

The Issue

Whilst working with a vRA Code Stream CI Task, I needed to build a YAML file in my container, which I would use to provide the values to my CLI Tool I was running. Within this YAML File, there is a section of JSON input (yep I know, it’s a Red Hat thing!!!).

I wanted to pass in this JSON section as a vRA variable, as it contains my authentication details to the Red Hat Cloud Website.

So my vRA variable would be as below:

{"auths":{"cloud.openshift.com":{"auth":"token-key","email":"[email protected]"},"registry.connect.redhat.com":{"auth":"token-key","email":"[email protected]"},"registry.redhat.io":{"auth":"token-key","email":"[email protected]"}}}

So my CI Task looked something like this:

cat << EOF > install-config.yaml
apiVersion: v1
baseDomain: simon.local
compute: 
- hyperthreading: Enabled 
  name: worker
  replicas: 1
  platform:
    vsphere: 
      cpus: 4
      coresPerSocket: 1
      memoryMB: 8192
      osDisk:
        diskSizeGB: 120
PullSecret: '${var.pullSecret}'
EOF

When running the Pipeline, I kept hitting an issue where the task would fail with a message similar to the below.

com.google.gson.stream.MalformedJsonException: Unterminated array at line 1 column 895 path $[39]

The Cause

This, I believe is because the tasks are passed to the Docker Host running the container via the Docker API using JSON format. The payload then contains my outer wrapping of YAML and within that more JSON. So the system gets confused with the various bits of JSON.

The Fix

To get around this issue, I encoded my JSON data in Base64. Saved this Base64 code to the variable. Then in my CI task I added an additional line before creating the file which creates a environment variable which decodes my base64 provided from a vRA variable.

Below is my new CI Task code.

export pullSecret=$(echo ${var.pullSecret} | base64 -d)

cat << EOF > install-config.yaml
apiVersion: v1
baseDomain: simon.local
compute: 
- hyperthreading: Enabled 
  name: worker
  replicas: 1
  platform:
    vsphere: 
      cpus: 4
      coresPerSocket: 1
      memoryMB: 8192
      osDisk:
        diskSizeGB: 120
PullSecret: '$pullSecret'
EOF

Regards

Follow @Saintdle

Dean Lewis

vEducate.co.uk

Fixing issues and blogging

Tanzu Mission Control – Using custom policies to ensure Kasten protects a deployed application

The Issue

The Solution

Implementing The Solution

Deploying OpenShift clusters (IPI) using vRA Code Stream

Pre-reqs

High Level Steps of this Pipeline

Creating a Code Stream Pipeline to deploy a OpenShift Cluster and register the endpoints with vRA and Tanzu Mission Control

Create the variables to be used

Passing JSON into vRA Code Stream CI Task – MalformedJsonException

The Issue

The Cause

The Fix