Tag Archives: DevOps

helm header image

Helm upgrade –reuse-values Fails with Nil Pointer Error After a Chart Version Bump

Kubernetes, , , , , , , , , , , , ,
If you have been running a Helm chart for a while and using --reuse-values to carry your previous configuration forward on upgrades, you may have hit an error like the one below after bumping to a new chart version: 
Error: UPGRADE FAILED: template: acme-web-proxy/templates/deployment.yaml:22:15: executing "acme-web-proxy/templates/deployment.yaml" at <include "acme-web-proxy.podAnnotations" .>: error calling include: template: acme-web-proxy/templates/_helpers.tpl:41:71: executing "acme-web-proxy.podAnnotations" at <include (print $.Template.BasePath "/configmap.yaml") .>: error calling include: template: acme-web-proxy/templates/configmap.yaml:18:6: executing "acme-web-proxy/templates/configmap.yaml" at <include "acme-web-proxy.metrics.config" .>: error calling include: template: acme-web-proxy/templates/configmap.yaml:34:25: executing "acme-web-proxy/templates/configmap.yaml" at <.Values.server.metrics.enabled>: nil pointer evaluating interface {}.enabled
Running the same upgrade with an explicit values file works without issue: 
helm upgrade -n my-namespace acme-web-proxy acme/web-proxy \
  --version 1.5.0 \
  -f helm/acme-web-proxy-values.yaml
Release "acme-web-proxy" has been upgraded. Happy Helming!
Read on to understand why these two commands behave differently and what you can do about it.

The Issue

When upgrading a Helm chart using --reuse-values, the upgrade fails with a nil pointer error. The error traces back to a template trying to access a values key that does not exist in the stored release values, in this case .Values.server.metrics.enabled. The same upgrade succeeds when you pass a values file explicitly using -f.

The Cause

The difference comes down to how Helm builds the values set that gets rendered into your chart templates. With helm upgrade --reuse-values, Helm takes only the user-supplied values stored from the previous release and uses those as the complete set of overrides. It does not start from the new chart version’s values.yaml defaults. Any key introduced in the new chart version is simply missing. With helm upgrade -f values.yaml, Helm starts from the new chart’s values.yaml defaults and merges your file on top. Keys added in the new chart version are populated with their default values before your overrides are applied. In the example above, chart version 1.5.0 added a new server.metrics.enabled key. The chart template accesses it directly without a nil guard: 
{{- if .Values.server.metrics.enabled }}
  # metrics configuration block
{{- end }}
When you upgrade with --reuse-values, the server.metrics map does not exist in the stored values at all. Go’s template engine cannot evaluate .enabled on a nil pointer and the render fails immediately. This is expected behaviour. The Helm documentation states that --reuse-values reuses the last release’s values and merges in any overrides from --set. Merging in new chart defaults is not part of what it does.

The Fix

There are three approaches depending on your workflow.

Option 1: Always upgrade with an explicit values file

In my opinion, this is the most reliable approach. Keep a values file that captures every override you need and pass it on every upgrade: 
helm upgrade -n my-namespace acme-web-proxy acme/web-proxy \
  --version 1.5.0 \
  -f helm/acme-web-proxy-values.yaml
Helm loads the new chart’s values.yaml defaults first and then applies your file on top. New keys get their defaults and your existing overrides stay intact.

Option 2: Supply the missing key with –set

If you want to keep using --reuse-values, you can backfill the missing key on the command line. Check the new chart’s values.yaml for the expected default and pass it in: 
helm upgrade -n my-namespace acme-web-proxy acme/web-proxy \
  --version 1.5.0 \
  --reuse-values \
  --set server.metrics.enabled=false
This resolves the immediate error, but it is a fragile approach for ongoing upgrades. Each time a new chart version introduces a key that a template does not nil-guard, you will hit the same problem again.

Option 3: Use –reset-then-reuse-values (Helm 3.14+)

Helm 3.14 added the --reset-then-reuse-values flag. It resets to the new chart’s defaults first and then re-applies your previously stored overrides on top: 
helm upgrade -n my-namespace acme-web-proxy acme/web-proxy \
  --version 1.5.0 \
  --reset-then-reuse-values
If you are on Helm 3.14 or later, this flag handles the new defaults problem without requiring you to maintain a full values file. You can check your Helm version with helm version.

Why –reuse-values is risky across chart version bumps

--reuse-values was designed for cases where you want to re-apply the same set of overrides without listing them again. It works well when upgrading within the same chart version or when a new version does not introduce any new required template values. Once a chart adds a new key and the template author accesses it without a nil guard such as {{- if .Values.server.metrics }}, any upgrade using --reuse-values will break for anyone who does not have that key in their stored values. It is partly a chart authoring problem, but you will encounter it regardless and need to know how to unblock yourself. The most consistent approach is to treat your values file as the source of truth and always pass -f on every upgrade. Your intent is explicit, the file is reviewable in source control, and you will not get caught out when a chart adds new keys. Regards Follow me on Bluesky Dean Lewis
Kubernetes Header Image

Fixing “Kubernetes configuration file is group-readable or world-readable” warnings

Kubernetes, , , , , , ,

The Issue

When using kubectl or oc you may see warnings that your Kubernetes configuration file is readable by group or by everyone.

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/user/cluster/admin-kubeconfig
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/user/cluster/admin-kubeconfig

The Cause

The kubeconfig file has permissions that allow access for group or others. The tools expect your kubeconfig to be readable and writable only by your user.

You can confirm this with a long listing. If you see read permission for group or others, the file is too open.

ls -l /home/user/cluster/admin-kubeconfig
-rw-r--r--  1 user  staff   12345  Sep  3 14:05 /home/user/cluster/admin-kubeconfig
# ^ group and others have read access

The Fix

  1. Restrict the file permissions so only your user can read and write it. 
    chmod 600 /home/user/cluster/admin-kubeconfig
  2. Optionally restrict the directory that holds the file. 
    chmod 700 /home/user/cluster
  3. Verify the new permissions. The output should show owner read and write only. 
    ls -l /home/user/cluster/admin-kubeconfig
-rw-------  1 user  staff   12345  Sep  3 14:05 /home/user/cluster/admin-kubeconfig
  4. Consider moving the kubeconfig into your home configuration folder for easier use, then point your tools at it. 
    mkdir -p ~/.kube
mv /home/user/cluster/admin-kubeconfig ~/.kube/admin-kubeconfig
export KUBECONFIG=~/.kube/admin-kubeconfig

    If you work with several kubeconfigs, you can join them in an environment variable.

    export KUBECONFIG=~/.kube/admin-kubeconfig:~/.kube/other.kubeconfig
  5. Keep your kubeconfig private. Do not share it, and do not commit it to a source control system.

Regards


Bluesky Icon
Follow me on Bluesky

Dean Lewis

Kubernetes

Quick Tip: Supercharge Kubernetes Resource Retrieval with ‘kubectl get -f’

Kubernetes, , , , , , , , ,

Did you know you can use the -f argument with kubectl get? Yep me either.

It’s pretty handy actually, as it will provide the status for all your Kubernetes resources deployed using that file or even file from hyperlink!

Below is a screenshot example using a file.

kubectl get -f

You can also specify multiple files by adding -f {file}for each file you want to check (this also works when deploying resources too!).

kubect get -f multiple files

And another example, using a hyperlink as the file location.

kubectl get -f from url

Hope this little tip helps someone!

Just for fun, here’s the ChatGPT write-up!

The command “kubectl get -f” is used in Kubernetes to retrieve information about Kubernetes resources defined in a file or multiple files. Let’s break down the command and its components:

  • “kubectl”: It is the command-line interface (CLI) tool for interacting with Kubernetes clusters. It allows you to manage and control Kubernetes resources.
  • “get”: It is the action or subcommand used to retrieve information about Kubernetes resources.
  • “-f”: It is a flag that specifies that the input will be provided from a file or multiple files rather than directly on the command line. It is followed by the path to the file(s) containing the Kubernetes resource definitions.

When you use “kubectl get -f <file>”, Kubernetes reads the file(s) provided and retrieves the information about the resources defined within those files. The information can include the names, statuses, and other details of the resources.

For example, if you have a file named “deployment.yaml” that defines a Kubernetes Deployment resource, you can use the command “kubectl get -f deployment.yaml” to retrieve information about that specific Deployment resource.

You can also provide multiple files by separating them with commas or specifying a directory containing multiple resource files. For instance, “kubectl get -f file1.yaml,file2.yaml” or “kubectl get -f /path/to/files” (where /path/to/files is the directory path).

By using this command, you can quickly retrieve information about Kubernetes resources defined in files without needing to manually create or modify resources using the command line.

Regards

Dean Lewis

o WOMAN JOB INTERVIEW facebook

Interview with Daniel Bryant, Ambassador Labs – Kubernetes, PaaS, Err what’s next?

Kubernetes, , , , ,

I’m really excited to get this interview out of the door. I missed Daniel’s session at KubeCon, “From Kubernetes to PaaS to … Err, What’s Next?”. The room was packed, I wasn’t able to sit in, so instead I watched it from the KubeCon live stream, sat on the beanbags in the hallway.

The session was fantastic, but I couldn’t ask any questions afterwards. So I dropped Daniel a message on twitter, and he agreed to chat, and be recorded for an interview.

Originally, we parked 25 minutes for the interview, but had so much fun we ended up at 47 minutes or so. Rather than cut everything down back to the 25 minutes mark. I decided to split the interview in two halves, so you can listen during your coffee breaks.

We break down Daniel’s KubeCon session in more depth, but importantly for me, give it a platform/infrastructure operations spin, as this is my background in IT as I build my knowledge in the Cloud Native world and learn knew technology and software.

I hope you enjoy it as much as I did recording it! (YouTube Playlist).

Part 1

Part 2

Regards

Dean Lewis