MongoDB + Kubernetes Header

MongoDB Container data loss issue – A Journey

Kubernetes, , , , , ,

Over the past month or so I noticed an issue with my Pac-Man Kubernetes application, which I use for demonstrations as a basic app front-end that writes to a database back end, running in Kubernetes.

  • When I restored my instances using Kasten, my Pac-Man high scores were missing.
  • This issue happened when I made some changes to my deployment files to configure authentication to the MongoDB using environment variables in my deployment file.

This blog post is a detail walk-through of the steps I took to troubleshoot the issue, and then rectify it!

Summary if you don’t want to read the post

If you are not looking to read through this blog post, here is the summary:

  • I changed MongoDB images, I needed to configure a new mount point location to match the MongoDB configuration
  • New MongoDB image is non-root, so had to use an Init container to configure the permissions on the PV first
Overview of the application

The application is made up of the following components:

  • Namespace
  • Deployment
    • MongoDB Pod
      • DB Authentication configured
      • Attached to a PVC
    • Pac-Man Pod
      • Nodejs web front end that connects back to the MongoDB Pod by looking for the Pod DNS address internally.
  • RBAC Configuration for Pod Security and Service Account
  • Secret which holds the data for the MongoDB Usernames and Passwords to be configured
  • Service
    • Type: LoadBalancer
      • Used to balance traffic to the Pac-Man Pods

Pac-Man Kubernetes Diagram

Confirming the behaviour

The behaviour I was seeing when my application was deployed:

  • Pac-Man web page – I could save a high score, and it would show in the high scores list
    • This showed the connectivity to the database was working, as the app would hang if it could not write to the database.
  • I would protect my application using Kasten. When I deleted the namespace, and restored everything, my application would be running, but there was no high scores to show.
  • This was apparent from deploying the branch version v0.5.0 and v0.5.1 from my GitHub.
  • Deploying the branch v0.2.0 would not product the same behaviour
    • This configuration did not have any database authentication setup, meaning MongoDB was open to the world if they could connect without a UN/Password.
Testing the Behaviour

Continue reading MongoDB Container data loss issue – A Journey →

vRealize Operations Header

vRealize Operations – Creating interactions between separate dashboards

VMware, , ,

Whilst reading some of the older vRealize Operations documentation, I stumbled on something I didn’t think was possible.

  • The ability to create interactions between separate dashboards.

At first, I thought could not be correct? I don’t remember seeing this option. But sure enough it’s there. So, I thought I’d write a quick blog about it and share to the world.

  • You can apply sections or context from one dashboard to another. You can connect widgets and views to widgets and views in the same dashboard or to other dashboards to investigate problems or better analyze the provided information.
Configuring Interactions between Dashboards

First, I’ve created two dashboards, which are based on the old troubleshooting dashboards. Both dashboards have an Object Picker List to filter the various related objects on each dashboard.

  • Dashboard-1 – Troubleshoot Cluster
  • Dashboard-2 – Troubleshoot VM

The premise is simple, when I select a Cluster object from Dashboard-1, I want the list of VMs to be filtered in Dashboard-2, to those only in the selected Cluster.

vROPs - Dashboard Interaction - example dashboards Continue reading vRealize Operations – Creating interactions between separate dashboards →

Tanzu Blog Logo Header

Data Management For VMware Tanzu API Postman Collection

VMware, , , ,

Whilst working with the “Data Management for VMware Tanzu” product, I decided to have a stab at working with the API as well.

Using my favourite API tool, Postman, I created an easy-to-use collection.

I’ve posted this here:

The commands in this collection are built from the Swagger API, which can be accessed from the following URL:

The official documentation for the API can be found here.

So far, I’ve created the following areas/use cases:

Operations Covered in this Collection:

    Login
    Get Provider Health
    Get Organsation
    Users
        Get Users
        Create Users
        Change Password for Current User
    Databases
        Get Databases
            Specific Database
            Get Root & Admin User Passwords
            Get Database Alerts
            Get Database Version
        Create Database
        Delete Database
        Backup
            Create & Restore (inc. Point in Time Restore)
        Logs
            Create, Download, Delete
    Get Environment
    Get Templates
    Instance Plans
        Get All, Create and Delete
    Get Tasks, Events and Alarms
    Get All Backups
    Get Audit Logs
Variables inside the collection

The main variables you will need to set before use are:

  • baseUrl
  • userName
  • Password

Data Management for Tanzu - API Postman Collection - Variables

Getting Started

Under the login folder, run the “Get Access Token” request. This has a test configured, so that from the returned data, it will copy the Bearer Token and OrgId for the User Account to the Environment Variable collection that is active.

Data Management for Tanzu - API Postman Collection - Get Access Token

Some API requests are only valid for certain Authenticated role types, such as creating Databases can only be successful with a user role of Org_Admin or Org_User.

How did I create this?

Continue reading Data Management For VMware Tanzu API Postman Collection →

Tanzu Blog Logo Header

Data Management for VMware Tanzu – Self-Service DBaaS

VMware, , , , , ,

In my first blog post, I covered the prerequisites and how to deploy the components for the Data Management for VMware Tanzu platform.

In this blog post we will cover using this new infrastructure for Self-Service Database-as-a-Service deployment and configuration.

Configure Database Templates and Instance Plans
  • Log into your Provider Appliance as the Provider administrator account.

The last actions to configure are publishing the database templates and configuring Instance Plans. (If you have setup your Organisation to use instance plans).

  • Click Templates from the left-hand navigation pane
  • Select your Template which has been sync’d from the Tanzu Net (Or repo as air-gap configuration).
  • Publish the Template

Data Management for Tanzu - Publish Template

  • Click Instance Plans from the left-hand navigation pane
  • Click “Add New Plan”
  • Configure as necessary

Data Management for Tanzu Instance Plans

Configure additional Org Users

Continue reading Data Management for VMware Tanzu – Self-Service DBaaS →

Tanzu Blog Logo Header

Data Management for VMware Tanzu – Getting Started

VMware, , , , ,

This blog post will cover deploying the infrastructure and components for Data Management for VMware Tanzu.

My second blog post will cover using this infrastructure for Self-Service Database-as-a-Service.

What is Data Management for VMware Tanzu?

Data Management for VMware Tanzu (DMS) is a newly released solution from VMware (July 2021) providing data-as-a-service toolkit for on-demand provisioning & automated management of MySQL and PostreSQL databases on vSphere platforms.

DMS is accessible as both a Graphical UI and via REST API, to meet the needs of administrators and developers and their consumption needs.

With DMS, it provides the ability to create and manage data services through a centralized platform in a self-service fashion, with the following features:

  • Simplified management for admins, acting as a Database fleet management tool; presenting a view of the organization’s database instances running on multi-cloud infrastructure.
  • Database users have the ability to consume self-service capabilities to create new database instances, or to operate on existing instances safely and securely, without requiring infrastructure or database expertise.
  • DMS also provides full automation for provisioning data service instances, backups, security patches, and periodic updates of the data service engine.

Data Management for Tanzu Provider Home Page

Data Management for Tanzu Provider Create Database Page

Understanding the components

DMS is made up of the following architectural components:

  • Provider – this is the core appliance you will deploy, which offers the central UI and API for all users to interact with the Data services and functions. It acts as the control plane to the other components.
  • Agent – These appliances are deployed to extend the control plan into the various vSphere environments, providing a point of presence for provisioning and management operations of the Services deployed.
  • Service – These are photon appliances which host the deployed instance of the data service (database). They communicate with the Agent that deployed them, via a private API. DMS supports the deployment of MySQL and PostgreSQL currently.
  • Template Repo – publishes a set of Data Management for VMware Tanzu Database Templates on Tanzu Network. The provider will poll the Tanzu Network periodically for new templates. There is also a method to handle air-gap environments.

S3 storage is required to be used for several items such as location to store the templates, database configurations and database backups.

Full deployment models for the components can be found here.

Data Management for Tanzu Architecture

Understanding Organisations and User Access

DMS implements the concept of Organisations to provide a logical grouping of users. There are two types:

  • Provider Org – A type of organization to which one or more Provider Administrator user belongs.
    • One provider org can exist in a single DMS installation.
    • This is automatically created during the deployment of the Provider Appliance
    • The Provider Org name is the company name specified at deployment.
  • Agent Org – A type of organization with one or more Organization Administrator or Organization User members.
    • These orgs are created via the DMS UI/API once the Provider appliance has been deployed and can be created at any time.

DMS pre-defines these three user roles:

  • Provider Administrator
    • This is the single Provider Role in the installation
    • Among other tasks, users in this role can import additional Provider Administrator users, create organizations, and create and import organization users
  • Organization Administrator
  • Organization User

The Provider Administrator user will assign a role to each DMS user that they create or import in an organization.

A user that is assigned the Organization Administrator role can manage all services in the organization to which they belong. A user assigned the Organization User can manage only the services that they provision.

More detailed information on the User roles and responsibilities can be found here.

Getting Started

Now first and foremost, I’ll point you towards the official documentation to use as a reference to review alongside this blog post.

Prerequisites

There are always several things to get sorted before you ever dive right in! The official requirements are detailed here, I’m going to call out some of the more finicky pieces you need to be aware of. Continue reading Data Management for VMware Tanzu – Getting Started →