Skip to content

Configuring applications

So far, we've been running the applications without setting any configurations.

However, in the real-world there's typically configurations you want to tune per environment you deploy to.

Let's have a look how we can configure our apps in Kubernetes.

Environment variables

The most common way to configure applications is through environment variables.

Let's create a YAML manifest named redis.yaml with the following contents:

---
apiVersion: v1
kind: Service
metadata:
  name: redis
  labels:
    app: redis
spec:
  selector:
    app: redis
  ports:
  - port: 6379
    targetPort: 6379
    protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  labels:
    app: redis
spec:
  replicas: 1
  progressDeadlineSeconds: 20
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - image: bitnami/redis
        name: redis
        ports:
        - containerPort: 6379
        env:
        - name: REDIS_PORT
          value: "6379"
        - name: REDIS_PASSWORD
          value: password123

Warning

Never write passwords directly to manifests outside this training!

This will create a Redis deployment that uses environment variables to set the port and the password for the Redis server, and a Service that points to the Redis Pods.

Multiple resources can be created from a single YAML file when using three dashes (---) as the separator.

We encountered a new resource here, the Service. We can now use it as the address for accessing Redis. More on that in the next exercise!

Also, we introduced one new section to the YAML: env. In this section, we can define arbitrary environment variables for the application container.

The Redis container will read the port from the REDIS_PORT environment variable and the password from REDIS_PASSWORD.

Note

The port value must be wrapped with quotes so that the value is passed as a string.

This is because the environment variables can only be strings.

Let's create the Redis Deployment and Service:

$ kubectl apply -f redis.yaml
service/redis created
deployment.apps/redis created

Let's test that the environment variable worked as intended by writing and reading values to the database using a Redis CLI.

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a wrongpassword -h redis incr demo
...
Warning: AUTH failed

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis incr demo
...
(integer) 1

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis get demo
...
"1"

Great! We can write and read values to Redis, but only if we supply the right password.

Warning

This is perhaps not the suitable way to deploy Redis in production.

However, it should be enough for demonstration purposes.

Environment variables from ConfigMaps

Kubernetes also supports loading environment variables from external sources such as ConfigMaps. A ConfigMap is a collection of key-value pairs, and they're typically used for storing application configurations.

Let's create a YAML file named redis-config.yaml with the following contents:

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
data:
  port: "6379"
  password: password123

In this YAML, we define a ConfigMap where we use the data field to define our key-value pairs. We define a port and a password that we want to use for Redis.

Let's create the ConfigMap resource:

$ kubectl apply -f redis-config.yaml
configmap/redis-config created

Next, let's edit Redis' Deployment YAML manifest (redis.yaml) we created above to read the environment variables from the ConfigMap.

...
env:
- name: REDIS_PORT
  valueFrom:
    configMapKeyRef:
      name: redis-config
      key: port
- name: REDIS_PASSWORD
  valueFrom:
    configMapKeyRef:
      name: redis-config
      key: password
...

Let's update the Deployment, and wait for the change to roll out:

$ kubectl apply -f redis.yaml
service/redis unchanged
deployment.apps/redis configured

$ kubectl rollout status deployment redis
deployment "redis" successfully rolled out

If you forgot to create the ConfigMap or misspelled the ConfigMap key names, the deployment will fail because the Pods can't load the configurations. Let's verify that Redis still works as expected.

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a wrongpassword -h redis incr demo
...
Warning: AUTH failed

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis incr demo
...
(integer) 1

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis get demo
...
"1"

Note that we didn't configure Redis with persistence so all the previous data was lost when we rolled out the update.

Alternative way to load environment variables from ConfigMaps

It can get pretty tedious to link all the ConfigMap values individually to environment variables. Fortunately, Kubernetes allows loading all the ConfigMap values directly to environment variables as well. Let's try it out!

First, we need to edit the ConfigMap a bit. The ConfigMap keys have to reference the environment variable names. Let's edit the redis-config.yaml file to look like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
data:
  REDIS_PORT: "6379"
  REDIS_PASSWORD: password123

Then, we'll need to update the ConfigMap in Kubernetes.

$ kubectl apply -f redis-config.yaml
configmap/redis-config configured

Next, we can update our manifest for Redis (redis.yaml) to pull the environment variables directly from the ConfigMap:

...
env: []
envFrom:
- configMapRef:
    name: redis-config
...

This time, we'll use envFrom instead of env to load the ConfigMap. We can delete the env contents because all the environment variables are loaded from the ConfigMap.

Let's update the Deployment and wait for the change to roll out:

$ kubectl apply -f redis.yaml
service/redis unchanged
deployment.apps/redis configured

$ kubectl rollout status deployment redis
deployment "redis" successfully rolled out

We can again verify that the config was loaded successfully using the Redis CLI like we did in the earlier sections.

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a wrongpassword -h redis incr demo
$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis incr demo
$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis get demo

Files from ConfigMaps

Sometimes it's too cumbersome to read configurations from environment variables.

For example, the configuration is too long for environment variables or it's too complex contain it. In these cases it's better to be able to load configurations as files.

Fortunately, Kubernetes supports loading ConfigMaps as files in the application containers.

Let's create some custom content for the NGINX web server from the earlier examples.

Similar to Redis before, we want a Service for our NGINX Pods, so they are a bit simpler to access.

You can edit your demo-deployment.yaml to add the Service, and then apply.

apiVersion: v1
kind: Service
metadata:
  name: demo
  labels:
    app: demo
spec:
  selector:
    app: demo-pod
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP

We can start by creating ConfigMap with the custom content.

Let's create a YAML manifest named demo-content.yaml with the following contents:

apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-content
data:
  hello.txt: Hello world!
  filler.txt: |
    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
    tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
    veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea
    commodo consequat. Duis aute irure dolor in reprehenderit in voluptate
    velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat
    cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id
    est laborum.

The ConfigMap contains two entries for our files where the entry key is the filename. Using the pipe character, we can create an entry that contains multiple lines.

Let's create the ConfigMap:

$ kubectl apply -f demo-content.yaml
configmap/demo-content configured

ConfigMaps can also be created directly from files using kubectl. For example, if we had the hello.txt and filler.txt contents as files, we could also create them like this: kubectl create configmap demo-content --from-file=hello.txt --from-file=filler.txt

We can now load the content from the ConfigMap as a volume in the demo web server Deployment. Let's edit the demo-deployment.yaml to add more details:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  labels:
    app: demo
spec:
  replicas: 3
  progressDeadlineSeconds: 20
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: demo-pod
  template:
    metadata:
      labels:
        app: demo-pod
    spec:
      containers:
      - image: nginxinc/nginx-unprivileged
        name: nginx
        ports:
        - containerPort: 8080
        readinessProbe:
          periodSeconds: 5
          successThreshold: 1
          httpGet:
            path: /
            port: 8080
        volumeMounts:
        - name: content
          mountPath: /usr/share/nginx/html/content
      volumes:
      - name: content
        configMap:
          name: demo-content

We added two new things to the YAML:

  • Volumes section (spec.template.spec.volumes): This describes file system volumes that are going to be mounted to the Pods. Specifically, it tells what are used as data sources for the volumes.
  • Volume mount section (spec.template.spec.containers.volumeMounts): This defines where the volumes from the volumes section are mounted to in the container file system. The name of the mount refers to the name of the volume in the volumes section.

In this example, we mount a single ConfigMap and place it's contents to the directory where NGINX serves its content. Let's deploy and test it:

$ kubectl apply -f demo-deployment.yaml
deployment/demo created

$ kubectl rollout status deployment demo
deployment "demo" successfully rolled out

$ kubectl run --restart=Never --rm -it --image=curlimages/curl call-demo -- http://demo/content/hello.txt
Hello world!

$ kubectl run --restart=Never --rm -it --image=curlimages/curl call-demo -- http://demo/content/filler.txt
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
...

Warning

Even though we're using this for loading web assets, it's generally not a good idea to use ConfigMap for storing huge amounts of data.

It's generally meant for small files like configurations.

Syncing ConfigMap updates to the application

When you update a ConfigMap, the changes may or may not be automatically synced to your application Pods. These are the rules that apply:

  • If you mount the ConfigMap as a volume, the ConfigMap updates will automatically flow to the Pods within about a minute. Note that this will only update files in the container. If an application reads those files, they still need to re-read them.
  • If you load the ConfigMap values as environment variables, the updates will NOT automatically flow to the Pod. Instead, you need to restart or recreate the Pods in order to reload the ConfigMap.

Let's see this in action. First, let's try this by updating the content for the demo web server. To do this, let's edit one of the key-value pairs in the demo-content.yaml file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-content
data:
  hello.txt: Yo!

We need to remember to update the ConfigMap in Kubernetes:

$ kubectl apply -f demo-content.yaml
configmap/demo-content configured

If we now repeatedly query the content for our demo web server, we should see the content switch from the old value to the new:

$ kubectl run --restart=Never --rm -it --image=curlimages/curl call-demo -- http://demo/content/hello.txt
Hello world!

(wait about a minute)
$ kubectl run --restart=Never --rm -it --image=curlimages/curl call-demo -- http://demo/content/hello.txt
Yo!

Next, let's try to reload Redis configuration ConfigMap to the Redis server. To do this, let's edit the password in the redis-config.yaml file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
data:
  REDIS_PORT: "6379"
  # !! DANGER! BAD PRACTICE: Never write passwords directly to
  # ConfigMaps manifests outside this training!
  REDIS_PASSWORD: newpassword

Let's update it in Kubernetes:

$ kubectl apply -f redis-config.yaml
configmap/redis-config configured

Redis hasn't reloaded the ConfigMap yet, so we should be able to contact it with the old password.

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis ping
...
PONG

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a newpassword -h redis ping
...
Warning: AUTH failed

We can use the kubectl sub-command rollout restart to recreate the Redis Pod:

$ kubectl rollout restart deployment redis
deployment.apps/redis restarted

The new Pod should now use the updated password.

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a newpassword -h redis ping
...
PONG

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis ping
...
Warning: AUTH failed

Secrets

In the previous section, we used ConfigMap to store application configurations. Along the configuration, we also stored sensitive information such as the Redis password. As you might have noticed from the comments, this is not how you should store passwords.

In Kubernetes, you should never write sensitive information directly to Deployments, ConfigMaps or other resources. It's actually recommended practice to use Secrets for storing sensitive information.

Let's take the sensitive information out of the Redis ConfigMap redis-config.yaml.

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
data:
  REDIS_PORT: "6379"

Next, let's create a new Secret YAML file named redis-secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: redis-secret
stringData:
  REDIS_PASSWORD: password123

In the YAML, we'll use the Secret kind to tell Kubernetes that we want to create a secret.

Like in ConfigMap, Secret can store arbitrary key-value pairs. However, unlike ConfigMaps, Secrets stores the values as binary data in base64 format (i.e. encoded but not encrypted). When a Secret is loaded into a Pod as a volume or as an environment variable, the data is automatically decoded.

The data can be provided using either the field data or stringData:

  • data accepts values in base64 format
  • stringData accepts values in string format

In the secret file above, we provide the data in string format, which Kubernetes automatically converts to base64.

Let's create update the ConfigMap and create the Secret in Kubernetes:

$ kubectl apply -f redis-config.yaml -f redis-secret.yaml
configmap/redis-config configured
secret/redis-secret created

The secret could also be created directly from files stored on disk. Run kubectl create secret generic -h to see a few examples.

Also similar to ConfigMaps, we can load Secrets as environment variables. Let's update redis.yaml again to read the environment variables from the Secret:

...
env: []
envFrom:
- configMapRef:
    name: redis-config
- secretRef:
    name: redis-secret
...

Let's update the Redis Deployment and test that it still works as expected:

$ kubectl apply -f redis.yaml
service/redis unchanged
deployment.apps/redis configured

$ kubectl rollout status deployment redis
deployment "redis" successfully rolled out

$ kubectl run --rm -it --restart=Never --image=bitnami/redis redis-cli -- redis-cli -a password123 -h redis ping
...
PONG

Perfect! Kubernetes also supports loading entries from Secrets as individual environment variables and as volumes. These work very similar to how they work in ConfigMaps. Feel free to experiment with them!

About handling sensitive information

Before you start handling sensitive information such as passwords in Kubernetes, make you consider these.

Check if it's OK to store secrets in Kubernetes

Kubernetes clusters are often shared with multiple people so make sure you're aware who has access to the secrets. When you store secrets in a namespace, anyone who can launch Pods in the same namespace can read those secrets.

How to store them in Kubernetes

If you need to make secrets accessible to containers in Kubernetes, always use Secrets to store the data. It's generally not a good idea to store the secrets to other resource manifests such Deployments or ConfigMaps. Note that using Secrets doesn't imply that they're encrypted.

How to store them outside Kubernetes

If you need to store secrets outside the Kubernetes cluster, it's recommended to use a solution specifically for that. Here's a couple of examples:

It's generally not a good idea that you store secrets in the following places:

  • Git: It may be tempting to store secrets in Git. It's generally not a good idea as the secrets are visible to anyone with the access to the repository. However, if you absolutely need to do it, make sure you encrypt the secrets before you commit them to Git to limit access to the people with the decryption key.
  • Docker images: It's never a good idea to store secrets in Docker images themselves. Anyone with the access to the Docker image will then have access to the secrets as well.

Guidelines

Here's a few guidelines and tips for using ConfigMaps and Secrets in your application:

  1. If you have other configurations besides the secrets for your app, you can either store them in a separate ConfigMap or the same Secret.
  2. Loading the Secrets as environment variables is more straightforward than loading them invidividually.
  3. Larger secrets and configs can be loaded as files.
  4. You can also "hardcode" non-secret configurations directly as environment variables in the Deployment manifest, if you wish to.

Next

In the next section, we'll learn how to make our applications available outside the Kubernetes cluster.