Update of Kubernetes secrets

Kubernetes secret is a way of storing sensitive data in Kubernetes, which can be used by various components in the cluster. Secrets are usually encrypted before being stored in order to prevent unauthorized access. Encryption makes update of Kubernetes secrets difficult to deal with. In this blog post I'd try to propose few options how to easily modify these resources.

Input

At begging let's create secret and x-ray it in many different ways.

# create Namespace
$ kubectl create namespace test-ns
namespace/test-ns created

# Create secret
$ kubectl -n test-ns create secret generic credentials \
>   --from-literal=username=username1 \
>   --from-literal=password=password2
secret/credentials created

Ok, so secret exists in the namespace test-ns, let's look it:

# listing
$ kubectl -n test-ns get secrets credentials 
NAME          TYPE     DATA   AGE
credentials   Opaque   2      17s


# description
$ kubectl -n test-ns describe secrets credentials 
Name:         credentials
Namespace:    test-ns
Labels:       <none>
Annotations:  <none>

Type:  Opaque

Data
====
password:  9 bytes
username:  9 bytes


# Encrypted content
$ kubectl -n test-ns get secrets credentials -ojson | jq .data
{
  "password": "cGFzc3dvcmQy",
  "username": "dXNlcm5hbWUx"
}

# Decrypted fields
$ kubectl -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d
username1

$ kubectl -n test-ns get secrets credentials -ojson | jq -r .data.password | base64 -d
password2

As you can see I'm using here jq command but you can use other JSON parser.

Update

Now let's move to funny part - modification. So, How to update the secrets in Kubernetes? The list I present for sure is not completed but I guess it contains the most common examples.

Edit secret

First option is kind of open-heart surgery. Basically we are editing the secret in terminal. Kubernetes secrets aren't the real secrets. Those are configuration items encrypted using base64 algorithm which easy do decrypt by running base64 -d

# First create hash of the value 
$ echo username2 | base64 
dXNlcm5hbWUyCg==
#
$ kubectl -n test-ns edit secrets credentials


# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
data:
  password: cGFzc3dvcmQ0cG9zdGZpeAo=
  username: dXNlcm5hbWU0bmV3cG9zdGZpeAo=
kind: Secret
metadata:
  creationTimestamp: "2022-01-18T05:17:25Z"
  name: credentials
  namespace: test-ns
  resourceVersion: "5526322"
  selfLink: /api/v1/namespaces/test-ns/secrets/credentials
  uid: 87a0f786-9c6d-4a96-b2d6-7553de7b3d40
type: Opaque


# save it and check

$ kubectl -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d
username2

If you made mistake like you will be turned back to the editor!

# secrets "credentials" was not valid:
# * patch: Invalid value: "map[data:map[username:dXNdGZpeAo=]]": error decoding from json: illegal base64 data at input byte 11
#

Thus as soon as you not provide correct base64 value you wan't be able to apply the change.

Apply Kubernetes manifest

Another approach I would describe is a standard one for Kubernetes as a tool which fills in love with immutable components.

$ kubectl -n test-ns get secrets credentials -oyaml > credential.yaml
$ cat credential.yaml 
apiVersion: v1
data:
  password: cGFzc3dvcmQ0cG9zdGZpeAo=
  username: dXNlcm5hbWU0bmV3cG9zdGZpeAo=
kind: Secret
metadata:
  creationTimestamp: "2022-01-18T05:17:25Z"
  name: credentials
  namespace: test-ns
  resourceVersion: "5526322"
  selfLink: /api/v1/namespaces/test-ns/secrets/credentials
  uid: 87a0f786-9c6d-4a96-b2d6-7553de7b3d40
type: Opaque

# modify credential.yaml accordingly and apply (using encrypted values)
$ kubectl apply -f credential.yaml

Update secret by patching it

The next approach is a kind of hack. Patching is no common way to play with k8s. It's like a on the fly editing small piece of manifest. As always we are working with encrypted (base64) values. Additionally it's worth to mention here what JSON is not only option YAML is eligible as well. Using content as string will work too.

$ echo username3 | base64 
dXNlcm5hbWUzCg==

$ cat patch.json 
[
    {
        "op" : "replace" ,
        "path" : "/data/username" ,
        "value" : "dXNlcm5hbWUzCg=="
    }
]


$ kubectl -n test-ns patch secret credentials --type json --patch "$(cat patch.json)"
secret/credentials patched


$ kubectl -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d
username3

Patch many

This is extended version of previous approach to the many fields.

echo username4 | base64 
dXNlcm5hbWU0Cg==
$ echo password4 | base64 
cGFzc3dvcmQ0Cg==

# patch2.json content
[
    {
        "op" : "replace" ,
        "path" : "/data/username" ,
        "value" : "dXNlcm5hbWU0Cg=="
    },
    {
        "op" : "replace" ,
        "path" : "/data/password" ,
        "value" : "cGFzc3dvcmQ0Cg=="
    }    
]

$ kubectl -n test-ns patch secret credentials --type json --patch "$(cat patch2.json)"
secret/credentials patched


$ kubectl  -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d
username4
$ kubectl  -n test-ns get secrets credentials -ojson | jq -r .data.password | base64 -d
password4

One-liner (quasi)

If you need to change secret in the pipepine/shell script take one of below one-liners.

add 'new' to the secret

TMP=$(k -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d); TMP2=$(echo ${TMP}new);HASH=$(echo ${TMP2} | base64);kubectl -n test-ns patch secret credentials --type json --patch "[{"op":"replace","path":"/data/username","value":${HASH}}]"

$ TMP=$(k -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d); TMP2=$(echo ${TMP}new);HASH=$(echo ${TMP2} | base64);kubectl -n test-ns patch secret credentials --type json --patch '[{"op":"replace","path":"/data/username","value":"'${HASH}'"}]'
secret/credentials patched

$ k -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d
username4new

Add postfix to all fields of the secret

This is the last option to pick up. If you have to add some string (postfix) to all fields in the secret just run similar script. It can be useful in testing stuff .

#!/bin/bash

POSTFIX="postfix"
NAMESPACE="test-ns"
SECRET="credentials"

for field in $(kubectl -n ${NAMESPACE} get secrets ${SECRET} -ojson | jq -r .data | jq 'keys' | jq -r .[])
do
    echo $field
    TMP=$(kubectl -n test-ns get secrets ${SECRET} -ojson | jq -r .data.${field} | base64 -d)
    TMP2=$(echo ${TMP}${POSTFIX})
    HASH=$(echo ${TMP2} | base64)
    kubectl -n ${NAMESPACE} patch secret ${SECRET} --type json --patch '[{"op":"replace","path":"/data/'${field}'","value":"'${HASH}'"}]'
done

One last Just to check

run fooor.sh

$ k -n test-ns get secrets credentials -ojson | jq -r .data.username | base64 -d
username4newpostfix

Conslusion

There are a lot of possibilities to update the secret resources in the Kubernetes. You can do it interactively or in batch mode. You can do it one by one or at one go.