CKAD-exercises Documentation

File: README.md

CKAD Exercises

A set of exercises that helped me prepare for the Certified Kubernetes Application Developer exam, offered by the Cloud Native Computing Foundation, organized by curriculum domain.
They may as well serve as learning and practicing with Kubernetes.

Make a mental note of the breadcrumb at the start of the excercise section, to quickly locate the relevant document in kubernetes.io.
It is recommended that you read the official documents before attempting exercises below it.
During the exam, you are only allowed to refer to official documentation from a browser window within the exam VM.
A Quick Reference box will contain helpful links for each exam exercise as well.

Can I PR? There is an error/an alternative way/an extra question/solution I can offer

Absolutely! Feel free to PR and edit/add questions and solutions, but please stick to the existing format.

If this repo has helped you in any way, feel free to post on discussions or buy me a coffee!

File: a.core_concepts.md

Core Concepts (13%)

kubernetes.io > Documentation > Reference > kubectl CLI > kubectl Cheat Sheet

kubernetes.io > Documentation > Tasks > Monitoring, Logging, and Debugging > Get a Shell to a Running Container

kubernetes.io > Documentation > Tasks > Access Applications in a Cluster > Configure Access to Multiple Clusters

kubernetes.io > Documentation > Tasks > Access Applications in a Cluster > Accessing Clusters using API

kubernetes.io > Documentation > Tasks > Access Applications in a Cluster > Use Port Forwarding to Access Applications in a Cluster

Create a namespace called 'mynamespace' and a pod with image nginx called nginx on this namespace

show

bash

kubectl create namespace mynamespace
kubectl run nginx --image=nginx --restart=Never -n mynamespace

Create the pod that was just described using YAML

show

Easily generate YAML with:

bash

kubectl run nginx --image=nginx --restart=Never --dry-run=client -n mynamespace -o yaml > pod.yaml

bash

cat pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: mynamespace
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml

Alternatively, you can run in one line

bash

kubectl run nginx --image=nginx --restart=Never --dry-run=client -o yaml | kubectl create -n mynamespace -f -

Create a busybox pod (using kubectl command) that runs the command "env". Run it and see the output

show

bash

kubectl run busybox --image=busybox --command --restart=Never -it --rm -- env # -it will help in seeing the output, --rm will immediately delete the pod after it exits
# or, just run it without -it
kubectl run busybox --image=busybox --command --restart=Never -- env
# and then, check its logs
kubectl logs busybox

Create a busybox pod (using YAML) that runs the command "env". Run it and see the output

show

bash

# create a  YAML template with this command
kubectl run busybox --image=busybox --restart=Never --dry-run=client -o yaml --command -- env > envpod.yaml
# see it
cat envpod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  containers:
  - command:
    - env
    image: busybox
    name: busybox
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

# apply it and then see the logs
kubectl apply -f envpod.yaml
kubectl logs busybox

Get the YAML for a new namespace called 'myns' without creating it

show

bash

kubectl create namespace myns -o yaml --dry-run=client

Create the YAML for a new ResourceQuota called 'myrq' with hard limits of 1 CPU, 1G memory and 2 pods without creating it

show

bash

kubectl create quota myrq --hard=cpu=1,memory=1G,pods=2 --dry-run=client -o yaml

Get pods on all namespaces

show

bash

kubectl get po --all-namespaces

Alternatively

bash

kubectl get po -A

Create a pod with image nginx called nginx and expose traffic on port 80

show

bash

kubectl run nginx --image=nginx --restart=Never --port=80

Change pod's image to nginx:1.24.0. Observe that the container will be restarted as soon as the image gets pulled

show

Note: The RESTARTS column should contain 0 initially (ideally - it could be any number)

bash

# kubectl set image POD/POD_NAME CONTAINER_NAME=IMAGE_NAME:TAG
kubectl set image pod/nginx nginx=nginx:1.24.0
kubectl describe po nginx # you will see an event 'Container will be killed and recreated'
kubectl get po nginx -w # watch it

Note: some time after changing the image, you should see that the value in the RESTARTS column has been increased by 1, because the container has been restarted, as stated in the events shown at the bottom of the kubectl describe pod command:

text

Events:
  Type    Reason     Age                  From               Message
  ----    ------     ----                 ----               -------
[...]
  Normal  Killing    100s                 kubelet, node3     Container pod1 definition changed, will be restarted
  Normal  Pulling    100s                 kubelet, node3     Pulling image "nginx:1.24.0"
  Normal  Pulled     41s                  kubelet, node3     Successfully pulled image "nginx:1.24.0"
  Normal  Created    36s (x2 over 9m43s)  kubelet, node3     Created container pod1
  Normal  Started    36s (x2 over 9m43s)  kubelet, node3     Started container pod1

Note: you can check pod's image by running

bash

kubectl get po nginx -o jsonpath='{.spec.containers[].image}{"\n"}'

Get nginx pod's ip created in previous step, use a temp busybox image to wget its '/'

show

bash

kubectl get po -o wide # get the IP, will be something like '10.1.1.131'
# create a temp busybox pod
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- 10.1.1.131:80

Alternatively you can also try a more advanced option:

bash

# Get IP of the nginx pod
NGINX_IP=$(kubectl get pod nginx -o jsonpath='{.status.podIP}')
# create a temp busybox pod
kubectl run busybox --image=busybox --env="NGINX_IP=$NGINX_IP" --rm -it --restart=Never -- sh -c 'wget -O- $NGINX_IP:80'

Or just in one line:

bash

kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- $(kubectl get pod nginx -o jsonpath='{.status.podIP}:{.spec.containers[0].ports[0].containerPort}')

Get pod's YAML

show

bash

kubectl get po nginx -o yaml
# or
kubectl get po nginx -oyaml
# or
kubectl get po nginx --output yaml
# or
kubectl get po nginx --output=yaml

Get information about the pod, including details about potential issues (e.g. pod hasn't started)

show

bash

kubectl describe po nginx

Get pod logs

show

bash

kubectl logs nginx

If pod crashed and restarted, get logs about the previous instance

show

bash

kubectl logs nginx -p
# or
kubectl logs nginx --previous

Execute a simple shell on the nginx pod

show

bash

kubectl exec -it nginx -- /bin/sh

Create a busybox pod that echoes 'hello world' and then exits

show

bash

kubectl run busybox --image=busybox -it --restart=Never -- echo 'hello world'
# or
kubectl run busybox --image=busybox -it --restart=Never -- /bin/sh -c 'echo hello world'

Do the same, but have the pod deleted automatically when it's completed

show

bash

kubectl run busybox --image=busybox -it --rm --restart=Never -- /bin/sh -c 'echo hello world'
kubectl get po # nowhere to be found :)

Create an nginx pod and set an env value as 'var1=val1'. Check the env value existence within the pod

show

bash

kubectl run nginx --image=nginx --restart=Never --env=var1=val1
# then
kubectl exec -it nginx -- env
# or
kubectl exec -it nginx -- sh -c 'echo $var1'
# or
kubectl describe po nginx | grep val1
# or
kubectl run nginx --restart=Never --image=nginx --env=var1=val1 -it --rm -- env
# or
kubectl run nginx --image nginx --restart=Never --env=var1=val1 -it --rm -- sh -c 'echo $var1'

File: b.multi_container_pods.md

Multi-container Pods (10%)

Create a Pod with two containers, both with image busybox and command "echo hello; sleep 3600". Connect to the second container and run 'ls'

show

The easiest way to do it is create a pod with a single container and save its definition in a YAML file:

bash

kubectl run busybox --image=busybox --restart=Never -o yaml --dry-run=client -- /bin/sh -c 'echo hello;sleep 3600' > pod.yaml
vi pod.yaml

Copy/paste the container related values, so your final YAML should contain the following two containers (make sure those containers have a different name):

yaml

containers:
  - args:
    - /bin/sh
    - -c
    - echo hello;sleep 3600
    image: busybox
    imagePullPolicy: IfNotPresent
    name: busybox
    resources: {}
  - args:
    - /bin/sh
    - -c
    - echo hello;sleep 3600
    image: busybox
    name: busybox2

bash

kubectl create -f pod.yaml
# Connect to the busybox2 container within the pod
kubectl exec -it busybox -c busybox2 -- /bin/sh
ls
exit

# or you can do the above with just a one-liner
kubectl exec -it busybox -c busybox2 -- ls

# you can do some cleanup
kubectl delete po busybox

show

The easiest way to do it is create a pod with a single container and save its definition in a YAML file:

bash

kubectl run box --image=nginx --restart=Never --port=80 --dry-run=client -o yaml > pod-init.yaml

Copy/paste the container related values, so your final YAML should contain the volume and the initContainer:

Volume:

yaml

containers:
- image: nginx
...
  volumeMounts:
  - name: vol
    mountPath: /usr/share/nginx/html
volumes:
- name: vol
  emptyDir: {}

initContainer:

yaml

...
initContainers:
- args:
  - /bin/sh
  - -c
  - echo "Test" > /work-dir/index.html
  image: busybox
  name: box
  volumeMounts:
  - name: vol
    mountPath: /work-dir

In total you get:

yaml

apiVersion: v1
kind: Pod
metadata:
  labels:
    run: box
  name: box
spec:
  initContainers: 
  - args: 
    - /bin/sh 
    - -c 
    - echo "Test" > /work-dir/index.html
    image: busybox 
    name: box 
    volumeMounts: 
    - name: vol 
      mountPath: /work-dir 
  containers:
  - image: nginx
    name: nginx
    ports:
    - containerPort: 80
    volumeMounts: 
    - name: vol 
      mountPath: /usr/share/nginx/html 
  volumes: 
  - name: vol 
    emptyDir: {}

bash

# Apply pod
kubectl apply -f pod-init.yaml

# Get IP
kubectl get po -o wide

# Execute wget
kubectl run box-test --image=busybox --restart=Never -it --rm -- /bin/sh -c "wget -O- $(kubectl get pod box -o jsonpath='{.status.podIP}')"

# you can do some cleanup
kubectl delete po box

File: c.pod_design.md

Pod design (20%)

Labels And Annotations

Deployments

Jobs

Cron Jobs

Labels and Annotations

kubernetes.io > Documentation > Concepts > Overview > Working with Kubernetes Objects > Labels and Selectors

Create 3 pods with names nginx1,nginx2,nginx3. All of them should have the label app=v1

show

bash

kubectl run nginx1 --image=nginx --restart=Never --labels=app=v1
kubectl run nginx2 --image=nginx --restart=Never --labels=app=v1
kubectl run nginx3 --image=nginx --restart=Never --labels=app=v1
# or
for i in `seq 1 3`; do kubectl run nginx$i --image=nginx -l app=v1 ; done

Show all labels of the pods

show

bash

kubectl get po --show-labels

Change the labels of pod 'nginx2' to be app=v2

show

bash

kubectl label po nginx2 app=v2 --overwrite
# or edit the pod yaml
kubectl edit po nginx2

Get the label 'app' for the pods (show a column with APP labels)

show

bash

kubectl get po -L app
# or
kubectl get po --label-columns=app

Get only the 'app=v2' pods

show

bash

kubectl get po -l app=v2
# or
kubectl get po -l 'app in (v2)'
# or
kubectl get po --selector=app=v2

Get 'app=v2' and not 'tier=frontend' pods

show

bash

kubectl get po -l app=v2,tier!=frontend
# or
kubectl get po -l 'app in (v2), tier notin (frontend)'
# or
kubectl get po --selector=app=v2,tier!=frontend

Add a new label tier=web to all pods having 'app=v2' or 'app=v1' labels

show

bash

kubectl label po -l "app in(v1,v2)" tier=web

Add an annotation 'owner: marketing' to all pods having 'app=v2' label

show

bash

kubectl annotate po -l "app=v2" owner=marketing

Remove the 'app' label from the pods we created before

show

bash

kubectl label po nginx1 nginx2 nginx3 app-
# or
kubectl label po nginx{1..3} app-
# or
kubectl label po -l app app-

Annotate pods nginx1, nginx2, nginx3 with "description='my description'" value

show

bash

kubectl annotate po nginx1 nginx2 nginx3 description='my description'

#or

kubectl annotate po nginx{1..3} description='my description'

Check the annotations for pod nginx1

show

bash

kubectl annotate pod nginx1 --list

# or

kubectl describe po nginx1 | grep -i 'annotations'

# or

kubectl get po nginx1 -o custom-columns=Name:metadata.name,ANNOTATIONS:metadata.annotations.description

As an alternative to using | grep you can use jsonPath like kubectl get po nginx1 -o jsonpath='{.metadata.annotations}{"\n"}'

Remove the annotations for these three pods

show

bash

kubectl annotate po nginx{1..3} description- owner-

Remove these pods to have a clean state in your cluster

show

bash

kubectl delete po nginx{1..3}

Pod Placement

Create a pod that will be deployed to a Node that has the label 'accelerator=nvidia-tesla-p100'

show

Add the label to a node:

bash

kubectl label nodes <your-node-name> accelerator=nvidia-tesla-p100
kubectl get nodes --show-labels

We can use the 'nodeSelector' property on the Pod YAML:

yaml

apiVersion: v1
kind: Pod
metadata:
  name: cuda-test
spec:
  containers:
    - name: cuda-test
      image: "k8s.gcr.io/cuda-vector-add:v0.1"
  nodeSelector: # add this
    accelerator: nvidia-tesla-p100 # the selection label

You can easily find out where in the YAML it should be placed by:

bash

kubectl explain po.spec

OR:
Use node affinity (https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/#schedule-a-pod-using-required-node-affinity)

yaml

apiVersion: v1
kind: Pod
metadata:
  name: affinity-pod
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: accelerator
            operator: In
            values:
            - nvidia-tesla-p100
  containers:
    ...

Create a pod that will be placed on node `node01` using `nodeName`

show

nodeName forces the Pod to be bound to a specific node (bypassing the scheduler). For more details, see the official docs: https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodename

yaml

apiVersion: v1
kind: Pod
metadata:
  name: nodename-pod
spec:
  nodeName: node01
  containers:
  - name: nodename-con
    image: nginx

Verify which node it landed on:

bash

kubectl get pod nodename-pod -o wide

Taint a node with key `tier` and value `frontend` with the effect `NoSchedule`. Then, create a pod that tolerates this taint.

show

Taint a node:

bash

kubectl taint node node1 tier=frontend:NoSchedule # key=value:Effect
kubectl describe node node1 # view the taints on a node

And to tolerate the taint:

yaml

apiVersion: v1
kind: Pod
metadata:
  name: frontend
spec:
  containers:
  - name: nginx
    image: nginx
  tolerations:
  - key: "tier"
    operator: "Equal"
    value: "frontend"
    effect: "NoSchedule"

Create a pod that will be placed on node `controlplane`. Use nodeSelector and tolerations.

show

bash

vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  name: frontend
spec:
  containers:
  - name: nginx
    image: nginx
  nodeSelector:
    kubernetes.io/hostname: controlplane
  tolerations:
  - key: "node-role.kubernetes.io/control-plane"
    operator: "Exists"
    effect: "NoSchedule"

bash

kubectl create -f pod.yaml

Deployments

kubernetes.io > Documentation > Concepts > Workloads > Workload Resources > Deployments

Create a deployment with image nginx:1.18.0, called nginx, having 2 replicas, defining port 80 as the port that this container exposes (don't create a service for this deployment)

show

bash

kubectl create deployment nginx  --image=nginx:1.18.0  --dry-run=client -o yaml > deploy.yaml
vi deploy.yaml
# change the replicas field from 1 to 2
# add this section to the container spec and save the deploy.yaml file
# ports:
#   - containerPort: 80
kubectl apply -f deploy.yaml

or, do something like:

bash

kubectl create deployment nginx  --image=nginx:1.18.0  --dry-run=client -o yaml | sed 's/replicas: 1/replicas: 2/g'  | sed 's/image: nginx:1.18.0/image: nginx:1.18.0\n        ports:\n        - containerPort: 80/g' | kubectl apply -f -

or,

bash

kubectl create deploy nginx --image=nginx:1.18.0 --replicas=2 --port=80

View the YAML of this deployment

show

bash

kubectl get deploy nginx -o yaml

View the YAML of the replica set that was created by this deployment

show

bash

kubectl describe deploy nginx # you'll see the name of the replica set on the Events section and in the 'NewReplicaSet' property
# OR you can find rs directly by:
kubectl get rs -l app=nginx
# you could also just do kubectl get rs
kubectl get rs nginx-7bf7478b77 -o yaml

Get the YAML for one of the pods

show

bash

kubectl get po # get all the pods
# OR you can find pods directly by:
kubectl get po -l run=nginx # if you created deployment by 'run' command
kubectl get po -l app=nginx # if you created deployment by 'create' command
kubectl get po nginx-7bf7478b77-gjzp8 -o yaml

Check how the deployment rollout is going

show

bash

kubectl rollout status deploy nginx

Update the nginx image to nginx:1.19.8

show

bash

kubectl set image deploy nginx nginx=nginx:1.19.8
# alternatively...
kubectl edit deploy nginx # change the .spec.template.spec.containers[0].image

The syntax of the 'kubectl set image' command is kubectl set image (-f FILENAME | TYPE NAME) CONTAINER_NAME_1=CONTAINER_IMAGE_1 ... CONTAINER_NAME_N=CONTAINER_IMAGE_N [options]

Check the rollout history and confirm that the replicas are OK

show

bash

kubectl rollout history deploy nginx
kubectl get deploy nginx
kubectl get rs # check that a new replica set has been created
kubectl get po

Undo the latest rollout and verify that new pods have the old image (nginx:1.18.0)

show

bash

kubectl rollout undo deploy nginx
# wait a bit
kubectl get po # select one 'Running' Pod
kubectl describe po nginx-5ff4457d65-nslcl | grep -i image # should be nginx:1.18.0

Do an on-purpose update of the deployment with a wrong image nginx:1.91

show

bash

kubectl set image deploy nginx nginx=nginx:1.91
# or
kubectl edit deploy nginx
# change the image to nginx:1.91
# vim tip: type (without quotes) '/image' and press Enter, to navigate quickly

Verify that something's wrong with the rollout

show

bash

kubectl rollout status deploy nginx
# or
kubectl get po # you'll see 'ErrImagePull' or 'ImagePullBackOff'

Return the deployment to the second revision (number 2) and verify the image is nginx:1.19.8

show

bash

kubectl rollout undo deploy nginx --to-revision=2
kubectl describe deploy nginx | grep Image:
kubectl rollout status deploy nginx # Everything should be OK

Check the details of the fourth revision (number 4)

show

bash

kubectl rollout history deploy nginx --revision=4 # You'll also see the wrong image displayed here

Scale the deployment to 5 replicas

show

bash

kubectl scale deploy nginx --replicas=5
kubectl get po
kubectl describe deploy nginx

Autoscale the deployment, pods between 5 and 10, targeting CPU utilization at 80%

show

bash

kubectl autoscale deploy nginx --min=5 --max=10 --cpu-percent=80
# view the horizontalpodautoscalers.autoscaling for nginx
kubectl get hpa nginx

Pause the rollout of the deployment

show

bash

kubectl rollout pause deploy nginx

Update the image to nginx:1.19.9 and check that there's nothing going on, since we paused the rollout

show

bash

kubectl set image deploy nginx nginx=nginx:1.19.9
# or
kubectl edit deploy nginx
# change the image to nginx:1.19.9
kubectl rollout history deploy nginx # no new revision

Resume the rollout and check that the nginx:1.19.9 image has been applied

show

bash

kubectl rollout resume deploy nginx
kubectl rollout history deploy nginx
kubectl rollout history deploy nginx --revision=6 # insert the number of your latest revision

Delete the deployment and the horizontal pod autoscaler you created

show

bash

kubectl delete deploy nginx
kubectl delete hpa nginx

# or
kubectl delete deploy/nginx hpa/nginx

Implement canary deployment by running two instances of nginx marked as version=v1 and version=v2 so that the load is balanced at 75%-25% ratio

show

Deploy 3 replicas of v1:

text

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-v1
  labels:
    app: my-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
      version: v1
  template:
    metadata:
      labels:
        app: my-app
        version: v1
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: workdir
          mountPath: /usr/share/nginx/html
      initContainers:
      - name: install
        image: busybox:1.28
        command:
        - /bin/sh
        - -c
        - "echo version-1 > /work-dir/index.html"
        volumeMounts:
        - name: workdir
          mountPath: "/work-dir"
      volumes:
      - name: workdir
        emptyDir: {}

Create the service:

text

apiVersion: v1
kind: Service
metadata:
  name: my-app-svc
  labels:
    app: my-app
spec:
  type: ClusterIP
  ports:
  - name: http
    port: 80
    targetPort: 80
  selector:
    app: my-app

Test if the deployment was successful from within a Pod:

text

# run a wget to the Service my-app-svc
kubectl run -it --rm --restart=Never busybox --image=gcr.io/google-containers/busybox --command -- wget -qO- my-app-svc

version-1

Deploy 1 replica of v2:

text

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-v2
  labels:
    app: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
      version: v2
  template:
    metadata:
      labels:
        app: my-app
        version: v2
    spec:
      containers:
      - name: nginx
        image: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: workdir
          mountPath: /usr/share/nginx/html
      initContainers:
      - name: install
        image: busybox:1.28
        command:
        - /bin/sh
        - -c
        - "echo version-2 > /work-dir/index.html"
        volumeMounts:
        - name: workdir
          mountPath: "/work-dir"
      volumes:
      - name: workdir
        emptyDir: {}

Observe that calling the ip exposed by the service the requests are load balanced across the two versions:

text

# run a busyBox pod that will make a wget call to the service my-app-svc and print out the version of the pod it reached.
kubectl run -it --rm --restart=Never busybox --image=gcr.io/google-containers/busybox -- /bin/sh -c 'while sleep 1; do wget -qO- my-app-svc; done'

version-1
version-1
version-1
version-2
version-2
version-1

If the v2 is stable, scale it up to 4 replicas and shutdown the v1:

text

kubectl scale --replicas=4 deploy my-app-v2
kubectl delete deploy my-app-v1
while sleep 0.1; do curl $(kubectl get svc my-app-svc -o jsonpath="{.spec.clusterIP}"); done
version-2
version-2
version-2
version-2
version-2
version-2

Jobs

Create a job named pi with image perl:5.34 that runs the command with arguments "perl -Mbignum=bpi -wle 'print bpi(2000)'"

show

bash

kubectl create job pi  --image=perl:5.34 -- perl -Mbignum=bpi -wle 'print bpi(2000)'

Wait till it's done, get the output

show

bash

kubectl get jobs -w # wait till 'SUCCESSFUL' is 1 (will take some time, perl image might be big)
kubectl get po # get the pod name
kubectl logs pi-**** # get the pi numbers
kubectl delete job pi

bash

kubectl get jobs -w # wait till 'SUCCESSFUL' is 1 (will take some time, perl image might be big)
kubectl logs job/pi
kubectl delete job pi

bash

kubectl wait --for=condition=complete --timeout=300s job pi
kubectl logs job/pi
kubectl delete job pi

Create a job with the image busybox that executes the command 'echo hello;sleep 30;echo world'

show

bash

kubectl create job busybox --image=busybox -- /bin/sh -c 'echo hello;sleep 30;echo world'

Follow the logs for the pod (you'll wait for 30 seconds)

show

bash

kubectl get po # find the job pod
kubectl logs busybox-ptx58 -f # follow the logs

See the status of the job, describe it and see the logs

show

bash

kubectl get jobs
kubectl describe jobs busybox
kubectl logs job/busybox

Delete the job

show

bash

kubectl delete job busybox

Create the same job, make it run 5 times, one after the other. Verify its status and delete it

show

bash

kubectl create job busybox --image=busybox --dry-run=client -o yaml -- /bin/sh -c 'echo hello;sleep 30;echo world' > job.yaml
vi job.yaml

Add job.spec.completions=5 and job.spec.completionMode=Indexed

yaml

apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  completions: 5 # add this line
  completionMode: Indexed # add this line
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: busybox
    spec:
      containers:
      - args:
        - /bin/sh
        - -c
        - echo hello;sleep 30;echo world
        image: busybox
        name: busybox
        resources: {}
      restartPolicy: OnFailure
status: {}

bash

kubectl create -f job.yaml

Verify that it has been completed:

bash

kubectl get job busybox -w # will take two and a half minutes
kubectl delete jobs busybox

Create the same job, but make it run 5 parallel times

show

bash

vi job.yaml

Add job.spec.parallelism=5

yaml

apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  parallelism: 5 # add this line
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: busybox
    spec:
      containers:
      - args:
        - /bin/sh
        - -c
        - echo hello;sleep 30;echo world
        image: busybox
        name: busybox
        resources: {}
      restartPolicy: OnFailure
status: {}

bash

kubectl create -f job.yaml
kubectl get jobs

It will take some time for the parallel jobs to finish (>= 30 seconds)

bash

kubectl delete job busybox

Create a job but ensure that it will be automatically terminated by kubernetes if it takes more than 30 seconds to execute

show

bash

kubectl create job busybox --image=busybox --dry-run=client -o yaml -- /bin/sh -c 'while true; do echo hello; sleep 10;done' > job.yaml
vi job.yaml

Add job.spec.activeDeadlineSeconds=30

bash

apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  activeDeadlineSeconds: 30 # add this line
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: busybox
    spec:
      containers:
      - args:
        - /bin/sh
        - -c
        - while true; do echo hello; sleep 10;done
        image: busybox
        name: busybox
        resources: {}
      restartPolicy: OnFailure
status: {}

Cron jobs

kubernetes.io > Documentation > Tasks > Run Jobs > Running Automated Tasks with a CronJob

Create a cron job with image busybox that runs on a schedule of "/1 * * *" and writes 'date; echo Hello from the Kubernetes cluster' to standard output

show

bash

kubectl create cronjob busybox --image=busybox --schedule="*/1 * * * *" -- /bin/sh -c 'date; echo Hello from the Kubernetes cluster'

See its logs and delete it

show

bash

kubectl get po # copy the ID of the pod whose container was just created
kubectl logs <busybox-***> # you will see the date and message 
kubectl delete cj busybox # cj stands for cronjob

Create the same cron job again, and watch the status. Once it ran, check which job ran by the created cron job. Check the log, and delete the cron job

show

bash

kubectl get cj
kubectl get jobs --watch
kubectl get po --show-labels # observe that the pods have a label that mentions their 'parent' job
kubectl logs busybox-1529745840-m867r
# Bear in mind that Kubernetes will run a new job/pod for each new cron job
kubectl delete cj busybox

Create a cron job with image busybox that runs every minute and writes 'date; echo Hello from the Kubernetes cluster' to standard output. The cron job should be terminated if it takes more than 17 seconds to start execution after its scheduled time (i.e. the job missed its scheduled time).

show

bash

kubectl create cronjob time-limited-job --image=busybox --restart=Never --dry-run=client --schedule="* * * * *" -o yaml -- /bin/sh -c 'date; echo Hello from the Kubernetes cluster' > time-limited-job.yaml
vi time-limited-job.yaml

Add cronjob.spec.startingDeadlineSeconds=17

bash

apiVersion: batch/v1
kind: CronJob
metadata:
  creationTimestamp: null
  name: time-limited-job
spec:
  startingDeadlineSeconds: 17 # add this line
  jobTemplate:
    metadata:
      creationTimestamp: null
      name: time-limited-job
    spec:
      template:
        metadata:
          creationTimestamp: null
        spec:
          containers:
          - args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
            image: busybox
            name: time-limited-job
            resources: {}
          restartPolicy: Never
  schedule: '* * * * *'
status: {}

Create a cron job with image busybox that runs every minute and writes 'date; echo Hello from the Kubernetes cluster' to standard output. The cron job should be terminated if it successfully starts but takes more than 12 seconds to complete execution.

show

bash

kubectl create cronjob time-limited-job --image=busybox --restart=Never --dry-run=client --schedule="* * * * *" -o yaml -- /bin/sh -c 'date; echo Hello from the Kubernetes cluster' > time-limited-job.yaml
vi time-limited-job.yaml

Add cronjob.spec.jobTemplate.spec.activeDeadlineSeconds=12

bash

apiVersion: batch/v1
kind: CronJob
metadata:
  creationTimestamp: null
  name: time-limited-job
spec:
  jobTemplate:
    metadata:
      creationTimestamp: null
      name: time-limited-job
    spec:
      activeDeadlineSeconds: 12 # add this line
      template:
        metadata:
          creationTimestamp: null
        spec:
          containers:
          - args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
            image: busybox
            name: time-limited-job
            resources: {}
          restartPolicy: Never
  schedule: '* * * * *'
status: {}

Keep only the last 2 successful and 1 failed runs of a CronJob

show

Create a CronJob with history limits configured:

bash

kubectl create cronjob history-demo \
  --image=busybox \
  --schedule="*/1 * * * *" \
  --dry-run=client -o yaml \
  -- /bin/sh -c 'date; echo Hello from history demo' > history-demo.yaml

Edit the file:

bash

vi history-demo.yaml

Add the following fields under spec:

bash

apiVersion: batch/v1
kind: CronJob
metadata:
  name: history-demo
spec:
  schedule: "*/1 * * * *"
  successfulJobsHistoryLimit: 2   # keep last 2 successful jobs
  failedJobsHistoryLimit: 1        # keep last 1 failed job
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: history-demo
            image: busybox
            args:
            - /bin/sh
            - -c
            - date; echo Hello from history demo
          restartPolicy: Never

Apply the CronJob:

bash

kubectl apply -f history-demo.yaml

Verify job history behavior:

bash

kubectl get cj history-demo
kubectl get jobs --watch

After several runs, confirm that:

Only 2 successful Jobs are kept
Only 1 failed Job is kept (if failures occur)

Clean up:

bash

kubectl delete cj history-demo

Create a job from cronjob.

show

bash

kubectl create job --from=cronjob/sample-cron-job sample-job

File: CODE_OF_CONDUCT.md

Contributor Covenant Code of Conduct

Our Pledge

In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.

Our Standards

Examples of behavior that contributes to creating a positive environment
include:

Using welcoming and inclusive language
Being respectful of differing viewpoints and experiences
Gracefully accepting constructive criticism
Focusing on what is best for the community
Showing empathy towards other community members

Examples of unacceptable behavior by participants include:

The use of sexualized language or imagery and unwelcome sexual attention or
advances
Trolling, insulting/derogatory comments, and personal or political attacks
Public or private harassment
Publishing others' private information, such as a physical or electronic
address, without explicit permission
Other conduct which could reasonably be considered inappropriate in a
professional setting

Our Responsibilities

Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.

Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.

Scope

This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.

Enforcement

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at [email protected]. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.

Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.

Attribution

This Code of Conduct is adapted from the Contributor Covenant, version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html

For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

File: d.configuration.md

Configuration (18%)

ConfigMaps

SecurityContext

Resource Requests and Limits

#Tips, export to variable
export ns="-n secret-ops"
export do="--dry-run=client -oyaml"

ConfigMaps

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure a Pod to Use a ConfigMap

Create a configmap named config with values foo=lala,foo2=lolo

show

bash

kubectl create configmap config --from-literal=foo=lala --from-literal=foo2=lolo

Display its values

show

bash

kubectl get cm config -o yaml
# or
kubectl describe cm config

Create and display a configmap from a file

Create the file with

bash

echo -e "foo3=lili\nfoo4=lele" > config.txt

show

bash

kubectl create cm configmap2 --from-file=config.txt
kubectl get cm configmap2 -o yaml

Create and display a configmap from a .env file

Create the file with the command

bash

echo -e "var1=val1\n# this is a comment\n\nvar2=val2\n#anothercomment" > config.env

show

bash

kubectl create cm configmap3 --from-env-file=config.env
kubectl get cm configmap3 -o yaml

Create and display a configmap from a file, giving the key 'special'

Create the file with

bash

echo -e "var3=val3\nvar4=val4" > config4.txt

show

bash

kubectl create cm configmap4 --from-file=special=config4.txt
kubectl describe cm configmap4
kubectl get cm configmap4 -o yaml

Create a configMap called 'options' with the value var5=val5. Create a new nginx pod that loads the value from variable 'var5' in an env variable called 'option'

show

bash

kubectl create cm options --from-literal=var5=val5
kubectl run nginx --image=nginx --restart=Never --dry-run=client -o yaml > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    env:
    - name: option # name of the env variable
      valueFrom:
        configMapKeyRef:
          name: options # name of config map
          key: var5 # name of the entity in config map
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl exec -it nginx -- env | grep option # will show 'option=val5'

Create a configMap 'anotherone' with values 'var6=val6', 'var7=val7'. Load this configMap as env variables into a new nginx pod

show

bash

kubectl create configmap anotherone --from-literal=var6=val6 --from-literal=var7=val7
kubectl run --restart=Never nginx --image=nginx -o yaml --dry-run=client > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    envFrom: # different than previous one, that was 'env'
    - configMapRef: # different from the previous one, was 'configMapKeyRef'
        name: anotherone # the name of the config map
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl exec -it nginx -- env

Create a configMap 'cmvolume' with values 'var8=val8', 'var9=val9'. Load this as a volume inside an nginx pod on path '/etc/lala'. Create the pod and 'ls' into the '/etc/lala' directory.

show

bash

kubectl create configmap cmvolume --from-literal=var8=val8 --from-literal=var9=val9
kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  volumes: # add a volumes list
  - name: myvolume # just a name, you'll reference this in the pods
    configMap:
      name: cmvolume # name of your configmap
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    volumeMounts: # your volume mounts are listed here
    - name: myvolume # the name that you specified in pod.spec.volumes.name
      mountPath: /etc/lala # the path inside your container
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl exec -it nginx -- /bin/sh
cd /etc/lala
ls # will show var8 var9
cat var8 # will show val8

SecurityContext

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure a Security Context for a Pod or Container

Create the YAML for an nginx pod that runs with the user ID 101. No need to create the pod

show

bash

kubectl run nginx --image=nginx --restart=Never --dry-run=client -o yaml > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  securityContext: # insert this line
    runAsUser: 101 # UID for the user
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

Create the YAML for an nginx pod that has the capabilities "NET_ADMIN", "SYS_TIME" added to its single container

show

bash

kubectl run nginx --image=nginx --restart=Never --dry-run=client -o yaml > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    securityContext: # insert this line
      capabilities: # and this
        add: ["NET_ADMIN", "SYS_TIME"] # this as well
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

Resource requests and limits

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Assign CPU Resources to Containers and Pods

Create an nginx pod with requests cpu=100m,memory=256Mi and limits cpu=200m,memory=512Mi

show

bash

kubectl run nginx --image=nginx --dry-run=client -o yaml > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "256Mi"
        cpu: "100m"
      limits:    
        memory: "512Mi"
        cpu: "200m"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

Limit Ranges

kubernetes.io > Documentation > Concepts > Policies > Limit Ranges (https://kubernetes.io/docs/concepts/policy/limit-range/)

Create a namespace named limitrange with a LimitRange that limits pod memory to a max of 500Mi and min of 100Mi

show

bash

kubectl create ns limitrange

vi 1.yaml

yaml

apiVersion: v1
kind: LimitRange
metadata:
  name: ns-memory-limit
  namespace: limitrange
spec:
  limits:
  - max: # max and min define the limit range
      memory: "500Mi"
    min:
      memory: "100Mi"
    type: Pod

bash

kubectl apply -f 1.yaml

Describe the namespace limitrange

show

bash

kubectl describe limitrange ns-memory-limit -n limitrange

Create an nginx pod that requests 250Mi of memory in the limitrange namespace

show

vi 2.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: limitrange
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "250Mi"
      limits:
        memory: "500Mi" # limit has to be specified and be <= limitrange
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

bash

kubectl apply -f 2.yaml

Resource Quotas

kubernetes.io > Documentation > Concepts > Policies > Resource Quotas (https://kubernetes.io/docs/concepts/policy/resource-quotas/)

Create ResourceQuota in namespace `one` with hard requests `cpu=1`, `memory=1Gi` and hard limits `cpu=2`, `memory=2Gi`.

show

Create the namespace:

bash

kubectl create ns one

Create the ResourceQuota

bash

vi rq-one.yaml

yaml

apiVersion: v1
kind: ResourceQuota
metadata:
  name: my-rq
  namespace: one
spec:
  hard:
    requests.cpu: "1"
    requests.memory: 1Gi
    limits.cpu: "2"
    limits.memory: 2Gi

bash

kubectl apply -f rq-one.yaml

bash

kubectl create quota my-rq --namespace=one --hard=requests.cpu=1,requests.memory=1Gi,limits.cpu=2,limits.memory=2Gi

Attempt to create a pod with resource requests `cpu=2`, `memory=3Gi` and limits `cpu=3`, `memory=4Gi` in namespace `one`

show

bash

vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: one
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "3Gi"
        cpu: "2"
      limits:
        memory: "4Gi"
        cpu: "3"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

bash

kubectl create -f pod.yaml

Expected error message:

bash

Error from server (Forbidden): error when creating "pod.yaml": pods "nginx" is forbidden: exceeded quota: my-rq, requested: limits.cpu=3,limits.memory=4Gi,requests.cpu=2,requests.memory=3Gi, used: limits.cpu=0,limits.memory=0,requests.cpu=0,requests.memory=0, limited: limits.cpu=2,limits.memory=2Gi,requests.cpu=1,requests.memory=1Gi

Create a pod with resource requests `cpu=0.5`, `memory=1Gi` and limits `cpu=1`, `memory=2Gi` in namespace `one`

show

bash

vi pod2.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
  namespace: one
spec:
  containers:
  - image: nginx
    name: nginx
    resources:
      requests:
        memory: "1Gi"
        cpu: "0.5"
      limits:
        memory: "2Gi"
        cpu: "1"
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

bash

kubectl create -f pod2.yaml

Show the ResourceQuota usage in namespace one

bash

kubectl get resourcequota -n one

text

NAME    AGE   REQUEST                                          LIMIT
my-rq   10m   requests.cpu: 500m/1, requests.memory: 1Gi/1Gi   limits.cpu: 1/2, limits.memory: 2Gi/2Gi

Secrets

kubernetes.io > Documentation > Concepts > Configuration > Secrets

kubernetes.io > Documentation > Tasks > Inject Data Into Applications > Distribute Credentials Securely Using Secrets

Create a secret called mysecret with the values password=mypass

show

bash

kubectl create secret generic mysecret --from-literal=password=mypass

Create a secret called mysecret2 that gets key/value from a file

Create a file called username with the value admin:

bash

echo -n admin > username

show

bash

kubectl create secret generic mysecret2 --from-file=username

Get the value of mysecret2

show

bash

kubectl get secret mysecret2 -o yaml
echo -n YWRtaW4= | base64 -d # on MAC it is -D, which decodes the value and shows 'admin'

Alternative using --jsonpath:

bash

kubectl get secret mysecret2 -o jsonpath='{.data.username}' | base64 -d  # on MAC it is -D

Alternative using --template:

bash

kubectl get secret mysecret2 --template '{{.data.username}}' | base64 -d  # on MAC it is -D

Alternative using jq:

bash

kubectl get secret mysecret2 -o json | jq -r .data.username | base64 -d  # on MAC it is -D

Create an nginx pod that mounts the secret mysecret2 in a volume on path /etc/foo

show

bash

kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  volumes: # specify the volumes
  - name: foo # this name will be used for reference inside the container
    secret: # we want a secret
      secretName: mysecret2 # name of the secret - this must already exist on pod creation
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    volumeMounts: # our volume mounts
    - name: foo # name on pod.spec.volumes
      mountPath: /etc/foo #our mount path
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl exec -it nginx -- /bin/bash
ls /etc/foo  # shows username
cat /etc/foo/username # shows admin

Delete the pod you just created and mount the variable 'username' from secret mysecret2 onto a new nginx pod in env variable called 'USERNAME'

show

bash

kubectl delete po nginx
kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    env: # our env variables
    - name: USERNAME # asked name
      valueFrom:
        secretKeyRef: # secret reference
          name: mysecret2 # our secret's name
          key: username # the key of the data in the secret
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl exec -it nginx -- env | grep USERNAME | cut -d '=' -f 2 # will show 'admin'

Create a Secret named 'ext-service-secret' in the namespace 'secret-ops'. Then, provide the key-value pair API_KEY=LmLHbYhsgWZwNifiqaRorH8T as literal.

show

bash

export ns="-n secret-ops"
export do="--dry-run=client -oyaml"
k create secret generic ext-service-secret --from-literal=API_KEY=LmLHbYhsgWZwNifiqaRorH8T $ns $do > sc.yaml
k apply -f sc.yaml

Consuming the Secret. Create a Pod named 'consumer' with the image 'nginx' in the namespace 'secret-ops' and consume the Secret as an environment variable. Then, open an interactive shell to the Pod, and print all environment variables.

show

bash

export ns="-n secret-ops"
export do="--dry-run=client -oyaml"
k run consumer --image=nginx $ns $do > nginx.yaml
vi nginx.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: consumer
  name: consumer
  namespace: secret-ops
spec:
  containers:
  - image: nginx
    name: consumer
    resources: {}
    env:
    - name: API_KEY
      valueFrom:
        secretKeyRef:
          name: ext-service-secret
          key: API_KEY
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

bash

k create $ns -f nginx.yaml
k exec -it $ns consumer -- /bin/sh
#env

Create a Secret named 'my-secret' of type 'kubernetes.io/ssh-auth' in the namespace 'secret-ops'. Define a single key named 'ssh-privatekey', and point it to the file 'id_rsa' in this directory.

show

bash

#Tips, export to variable
export do="--dry-run=client -oyaml"
export ns="-n secret-ops"

#if id_rsa file didn't exist.
ssh-keygen

k create secret generic my-secret $ns --type="kubernetes.io/ssh-auth" --from-file=ssh-privatekey=id_rsa $do > sc.yaml
k apply -f sc.yaml

Create a Pod named 'consumer' with the image 'nginx' in the namespace 'secret-ops', and consume the Secret as Volume. Mount the Secret as Volume to the path /var/app with read-only access. Open an interactive shell to the Pod, and render the contents of the file.

show

bash

#Tips, export to variable
export ns="-n secret-ops"
export do="--dry-run=client -oyaml"
k run consumer --image=nginx $ns $do > nginx.yaml
vi nginx.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: consumer
  name: consumer
  namespace: secret-ops
spec:
  containers:
    - image: nginx
      name: consumer
      resources: {}
      volumeMounts:
        - name: foo
          mountPath: "/var/app"
          readOnly: true
  volumes:
    - name: foo
      secret:
        secretName: my-secret
        optional: true
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {}

bash

k exec -it $ns consumer -- /bin/sh
# cat /var/app/ssh-privatekey
# exit

ServiceAccounts

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure Service Accounts for Pods

See all the service accounts of the cluster in all namespaces

show

bash

kubectl get sa --all-namespaces

Alternatively

bash

kubectl get sa -A

Create a new serviceaccount called 'myuser'

show

bash

kubectl create sa myuser

Alternatively:

bash

# let's get a template easily
kubectl get sa default -o yaml > sa.yaml
vim sa.yaml

yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: myuser

bash

kubectl create -f sa.yaml

Create an nginx pod that uses 'myuser' as a service account

show

bash

kubectl run nginx --image=nginx --restart=Never -o yaml --dry-run=client > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  serviceAccountName: myuser # we use pod.spec.serviceAccountName
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  serviceAccount: myuser # we use pod.spec.serviceAccount
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl get pod nginx -o jsonpath='{.spec.serviceAccountName}' # output: myuser

kubectl exec -it nginx -- cat /var/run/secrets/kubernetes.io/serviceaccount/token # to check if the ServiceAccount token is mounted inside the Pod. Starting from Kubernetes 1.24+, the token is dynamically projected into the Pod

Generate an API token for the service account 'myuser'

show

bash

kubectl create token myuser

File: e.observability.md

Observability (18%)

Liveness, readiness and startup probes

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure Liveness, Readiness and Startup Probes

Create an nginx pod with a liveness probe that just runs the command 'ls'. Save its YAML in pod.yaml. Run it, check its probe status, delete it.

show

bash

kubectl run nginx --image=nginx --restart=Never --dry-run=client -o yaml > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    livenessProbe: # our probe
      exec: # add this line
        command: # command definition
        - ls # ls command
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl describe pod nginx | grep -i liveness # run this to see that liveness probe works
kubectl delete -f pod.yaml

Modify the pod.yaml file so that liveness probe starts kicking in after 5 seconds whereas the interval between probes would be 5 seconds. Run it, check the probe, delete it.

show

bash

kubectl explain pod.spec.containers.livenessProbe # get the exact names

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    livenessProbe:
      initialDelaySeconds: 5 # add this line
      periodSeconds: 5 # add this line as well
      exec:
        command:
        - ls
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl describe po nginx | grep -i liveness
kubectl delete -f pod.yaml

Create an nginx pod (that includes port 80) with an HTTP readinessProbe on path '/' on port 80. Again, run it, check the readinessProbe, delete it.

show

bash

kubectl run nginx --image=nginx --dry-run=client -o yaml --restart=Never --port=80 > pod.yaml
vi pod.yaml

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    imagePullPolicy: IfNotPresent
    name: nginx
    resources: {}
    ports:
    - containerPort: 80 # Note: Readiness probes runs on the container during its whole lifecycle. Since nginx exposes 80, containerPort: 80 is not required for readiness to work.
    readinessProbe: # declare the readiness probe
      httpGet: # add this line
        path: / #
        port: 80 #
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}

bash

kubectl create -f pod.yaml
kubectl describe pod nginx | grep -i readiness # to see the pod readiness details
kubectl delete -f pod.yaml

Lots of pods are running in `qa`,`alan`,`test`,`production` namespaces. All of these pods are configured with liveness probe. Please list all pods whose liveness probe are failed in the format of `/` per line.

show

A typical liveness probe failure event

text

LAST SEEN   TYPE      REASON      OBJECT              MESSAGE
22m         Warning   Unhealthy   pod/liveness-exec   Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory

collect failed pods namespace by namespace

kubectl get events -o json | jq -r '.items[] | select(.message | contains("Liveness probe failed")).involvedObject | .namespace + "/" + .name'

Logging

Create a busybox pod that runs `i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done`. Check its logs

show

bash

kubectl run busybox --image=busybox --restart=Never -- /bin/sh -c 'i=0; while true; do echo "$i: $(date)"; i=$((i+1)); sleep 1; done'
kubectl logs busybox -f # follow the logs

Debugging

Create a busybox pod that runs 'ls /notexist'. Determine if there's an error (of course there is), see it. In the end, delete the pod

show

bash

kubectl run busybox --restart=Never --image=busybox -- /bin/sh -c 'ls /notexist'
# show that there's an error
kubectl logs busybox
kubectl describe po busybox
kubectl delete po busybox

Create a busybox pod that runs 'notexist'. Determine if there's an error (of course there is), see it. In the end, delete the pod forcefully with a 0 grace period

show

bash

kubectl run busybox --restart=Never --image=busybox -- notexist
kubectl logs busybox # will bring nothing! container never started
kubectl describe po busybox # in the events section, you'll see the error
# also...
kubectl get events | grep -i error # you'll see the error here as well
kubectl delete po busybox --force --grace-period=0

Get CPU/memory utilization for nodes ([metrics-server](https://github.com/kubernetes-incubator/metrics-server) must be running)

show

bash

kubectl top nodes

File: f.services.md

Services and Networking (13%)

Create a pod with image nginx called nginx and expose its port 80

show

bash

kubectl run nginx --image=nginx --restart=Never --port=80 --expose
# observe that a pod as well as a service are created

Confirm that ClusterIP has been created. Also check endpoints

show

bash

kubectl get svc nginx # services
kubectl get ep # endpoints

Get service's ClusterIP, create a temp busybox pod and 'hit' that IP with wget

show

bash

kubectl get svc nginx # get the IP (something like 10.108.93.130)
kubectl run busybox --rm --image=busybox -it --restart=Never --
wget -O- [PUT THE POD'S IP ADDRESS HERE]:80
exit

bash

IP=$(kubectl get svc nginx --template={{.spec.clusterIP}}) # get the IP (something like 10.108.93.130)
kubectl run busybox --rm --image=busybox -it --restart=Never --env="IP=$IP" -- wget -O- $IP:80 --timeout 2
# Tip: --timeout is optional, but it helps to get answer more quickly when connection fails (in seconds vs minutes)

Convert the ClusterIP to NodePort for the same service and find the NodePort port. Hit service using Node's IP. Delete the service and the pod at the end.

show

bash

kubectl edit svc nginx

yaml

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2018-06-25T07:55:16Z
  name: nginx
  namespace: default
  resourceVersion: "93442"
  selfLink: /api/v1/namespaces/default/services/nginx
  uid: 191e3dac-784d-11e8-86b1-00155d9f663c
spec:
  clusterIP: 10.97.242.220
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: nginx
  sessionAffinity: None
  type: NodePort # change cluster IP to nodeport
status:
  loadBalancer: {}

bash

kubectl patch svc nginx -p '{"spec":{"type":"NodePort"}}'

bash

kubectl get svc

text

# result:
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        1d
nginx        NodePort    10.107.253.138   <none>        80:31931/TCP   3m

bash

wget -O- NODE_IP:31931 # if you're using Kubernetes with Docker for Windows/Mac, try 127.0.0.1
#if you're using minikube, try minikube ip, then get the node ip such as 192.168.99.117

bash

kubectl delete svc nginx # Deletes the service
kubectl delete pod nginx # Deletes the pod

Create a deployment called foo using image 'dgkanatsios/simpleapp' (a simple server that returns hostname) and 3 replicas. Label it as 'app=foo'. Declare that containers in this pod will accept traffic on port 8080 (do NOT create a service yet)

show

bash

kubectl create deploy foo --image=dgkanatsios/simpleapp --port=8080 --replicas=3
kubectl label deployment foo --overwrite app=foo #This is optional since kubectl create deploy foo will create label app=foo by default

Get the pod IPs. Create a temp busybox pod and try hitting them on port 8080

show

bash

kubectl get pods -l app=foo -o wide # 'wide' will show pod IPs
kubectl run busybox --image=busybox --restart=Never -it --rm -- sh
wget -O- <POD_IP>:8080 # do not try with pod name, will not work
# try hitting all IPs generated after running 1st command to confirm that hostname is different
exit
# or
kubectl get po -o wide -l app=foo | awk '{print $6}' | grep -v IP | xargs -L1 -I '{}' kubectl run --rm -ti tmp --restart=Never --image=busybox -- wget -O- http://\{\}:8080
# or
kubectl get po -l app=foo -o jsonpath='{range .items[*]}{.status.podIP}{"\n"}{end}' | xargs -L1 -I '{}' kubectl run --rm -ti tmp --restart=Never --image=busybox -- wget -O- http://\{\}:8080

Create a service that exposes the deployment on port 6262. Verify its existence, check the endpoints

show

bash

kubectl expose deploy foo --port=6262 --target-port=8080
kubectl get service foo # you will see ClusterIP as well as port 6262
kubectl get endpoints foo # you will see the IPs of the three replica pods, listening on port 8080

Create a temp busybox pod and connect via wget to foo service. Verify that each time there's a different hostname returned. Delete deployment and services to cleanup the cluster

show

bash

kubectl get svc # get the foo service ClusterIP
kubectl run busybox --image=busybox -it --rm --restart=Never -- sh
wget -O- foo:6262 # DNS works! run it many times, you'll see different pods responding
wget -O- <SERVICE_CLUSTER_IP>:6262 # ClusterIP works as well
# you can also kubectl logs on deployment pods to see the container logs
kubectl delete svc foo
kubectl delete deploy foo

Create an nginx deployment of 2 replicas, expose it via a ClusterIP service on port 80. Create a NetworkPolicy so that only pods with labels 'access: granted' can access the pods in this deployment and apply it

kubernetes.io > Documentation > Concepts > Services, Load Balancing, and Networking > Network Policies

Note that network policies may not be enforced by default, depending on your k8s implementation. E.g. Azure AKS by default won't have policy enforcement, the cluster must be created with an explicit support for netpol https://docs.microsoft.com/en-us/azure/aks/use-network-policies#overview-of-network-policy

show

bash

kubectl create deployment nginx --image=nginx --replicas=2
kubectl expose deployment nginx --port=80

kubectl describe svc nginx # see the 'app=nginx' selector for the pods
# or
kubectl get svc nginx -o yaml

vi policy.yaml

yaml

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: access-nginx # pick a name
spec:
  podSelector:
    matchLabels:
      app: nginx # selector for the pods
  ingress: # allow ingress traffic
  - from:
    - podSelector: # from pods
        matchLabels: # with this label
          access: granted

bash

# Create the NetworkPolicy
kubectl create -f policy.yaml

# Check if the Network Policy has been created correctly
# make sure that your cluster's network provider supports Network Policy (https://kubernetes.io/docs/tasks/administer-cluster/declare-network-policy/#before-you-begin)
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://nginx:80 --timeout 2                          # This should not work. --timeout is optional here. But it helps to get answer more quickly (in seconds vs minutes)
kubectl run busybox --image=busybox --rm -it --restart=Never --labels=access=granted -- wget -O- http://nginx:80 --timeout 2  # This should be fine

Create an Ingress resource to expose an existing service using HTTP path routing

You already have:

a Deployment nginx
a Service nginx exposing port 80

You want to expose it externally using an Ingress rule.

Note that in CKAD, you are not required to install an Ingress Controller, but you must know how to define and troubleshoot an Ingress resource and understand how it connects to a Service.

show
Verify the service exists:

bash

kubectl get svc nginx

Expected:

Type: ClusterIP
Port: 80

If the Deployment and Service do not already exist in your practice environment, you can create them using the following commands:

bash

kubectl create deployment nginx --image=nginx --port=80
kubectl expose deployment nginx --port=80 --name=nginx

Create an Ingress resource:

bash

vi ingress.yaml

bash

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
spec:
  rules:
  - host: nginx.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx
            port:
              number: 80

Apply it:

bash

kubectl apply -f ingress.yaml

Verify the Ingress:

bash

kubectl get ingress
kubectl describe ingress nginx-ingress

File: g.state.md

State Persistence (8%)

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure a Pod to Use a Volume for Storage

kubernetes.io > Documentation > Tasks > Configure Pods and Containers > Configure a Pod to Use a PersistentVolume for Storage

Define volumes

Create busybox pod with two containers, each one will have the image busybox and will run the 'sleep 3600' command. Make both containers mount an emptyDir at '/etc/foo'. Connect to the second busybox, write the first column of '/etc/passwd' file to '/etc/foo/passwd'. Connect to the first busybox and write '/etc/foo/passwd' file to standard output. Delete pod.

show

This question is probably a better fit for the 'Multi-container-pods' section but I'm keeping it here as it will help you get acquainted with state

The easiest way to do this is to create a template pod with:

bash

kubectl run busybox --image=busybox --restart=Never -o yaml --dry-run=client -- /bin/sh -c 'sleep 3600' > pod.yaml
vi pod.yaml

Copy paste the container definition and type the lines that have a comment in the end:

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  dnsPolicy: ClusterFirst
  restartPolicy: Never
  containers:
  - args:
    - /bin/sh
    - -c
    - sleep 3600
    image: busybox
    imagePullPolicy: IfNotPresent
    name: busybox
    resources: {}
    volumeMounts: #
    - name: myvolume #
      mountPath: /etc/foo #
  - args:
    - /bin/sh
    - -c
    - sleep 3600
    image: busybox
    name: busybox2 # don't forget to change the name during copy paste, must be different from the first container's name!
    volumeMounts: #
    - name: myvolume #
      mountPath: /etc/foo #
  volumes: #
  - name: myvolume #
    emptyDir: {} #

In case you forget to add bash -- /bin/sh -c 'sleep 3600' in template pod create command, you can include command field in config file

yaml

spec:
  containers:
  - image: busybox
    name: busybox
    command: ["/bin/sh", "-c", "sleep 3600"]

Connect to the second container:

bash

kubectl exec -it busybox -c busybox2 -- /bin/sh
cat /etc/passwd | cut -f 1 -d ':' > /etc/foo/passwd # instead of cut command you can use awk -F ":" '{print $1}'
cat /etc/foo/passwd # confirm that stuff has been written successfully
exit

Connect to the first container:

bash

kubectl exec -it busybox -c busybox -- /bin/sh
mount | grep foo # confirm the mounting
cat /etc/foo/passwd
exit
kubectl delete po busybox

Create a PersistentVolume of 10Gi, called 'myvolume'. Make it have accessMode of 'ReadWriteOnce' and 'ReadWriteMany', storageClassName 'normal', mounted on hostPath '/etc/foo'. Save it on pv.yaml, add it to the cluster. Show the PersistentVolumes that exist on the cluster

show

bash

vi pv.yaml

yaml

kind: PersistentVolume
apiVersion: v1
metadata:
  name: myvolume
spec:
  storageClassName: normal
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
    - ReadWriteMany
  hostPath:
    path: /etc/foo

Show the PersistentVolumes:

bash

kubectl create -f pv.yaml
# will have status 'Available'
kubectl get pv

Create a PersistentVolumeClaim for this PersistentVolume, called 'mypvc', a request of 4Gi and an accessMode of ReadWriteOnce, with the storageClassName of normal, and save it on pvc.yaml. Create it on the cluster. Show the PersistentVolumeClaims of the cluster. Show the PersistentVolumes of the cluster

show

bash

vi pvc.yaml

yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: mypvc
spec:
  storageClassName: normal
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 4Gi

Create it on the cluster:

bash

kubectl create -f pvc.yaml

Show the PersistentVolumeClaims and PersistentVolumes:

bash

kubectl get pvc # will show as 'Bound'
kubectl get pv # will show as 'Bound' as well

Create a busybox pod with command 'sleep 3600', save it on pod.yaml. Mount the PersistentVolumeClaim to '/etc/foo'. Connect to the 'busybox' pod, and copy the '/etc/passwd' file to '/etc/foo/passwd'

show

Create a skeleton pod:

bash

kubectl run busybox --image=busybox --restart=Never -o yaml --dry-run=client -- /bin/sh -c 'sleep 3600' > pod.yaml
vi pod.yaml

Add the lines that finish with a comment:

yaml

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: busybox
  name: busybox
spec:
  containers:
  - args:
    - /bin/sh
    - -c
    - sleep 3600
    image: busybox
    imagePullPolicy: IfNotPresent
    name: busybox
    resources: {}
    volumeMounts: #
    - name: myvolume #
      mountPath: /etc/foo #
  dnsPolicy: ClusterFirst
  restartPolicy: Never
  volumes: #
  - name: myvolume #
    persistentVolumeClaim: #
      claimName: mypvc #
status: {}

Create the pod:

bash

kubectl create -f pod.yaml

Connect to the pod and copy '/etc/passwd' to '/etc/foo/passwd':

bash

kubectl exec busybox -it -- cp /etc/passwd /etc/foo/passwd

Create a second pod which is identical with the one you just created (you can easily do it by changing the 'name' property on pod.yaml). Connect to it and verify that '/etc/foo' contains the 'passwd' file. Delete pods to cleanup. Note: If you can't see the file from the second pod, can you figure out why? What would you do to fix that?

show

Create the second pod, called busybox2:

bash

vim pod.yaml
# change 'metadata.name: busybox' to 'metadata.name: busybox2'
kubectl create -f pod.yaml
kubectl exec busybox2 -- ls /etc/foo # will show 'passwd'
# cleanup
kubectl delete po busybox busybox2
kubectl delete pvc mypvc
kubectl delete pv myvolume

If the file doesn't show on the second pod but it shows on the first, it has most likely been scheduled on a different node.

bash

# check which nodes the pods are on
kubectl get po busybox -o wide
kubectl get po busybox2 -o wide

If they are on different nodes, you won't see the file, because we used the hostPath volume type.
If you need to access the same files in a multi-node cluster, you need a volume type that is independent of a specific node.
There are lots of different types per cloud provider (see here), a general solution could be to use NFS.

Create a busybox pod with 'sleep 3600' as arguments. Copy '/etc/passwd' from the pod to your local folder

show

bash

kubectl run busybox --image=busybox --restart=Never -- sleep 3600
kubectl cp busybox:/etc/passwd ./passwd # kubectl cp command
# previous command might report an error, feel free to ignore it since copy command works
cat passwd

File: h.helm.md

Managing Kubernetes with Helm

Note: Helm is part of the new CKAD syllabus. Here are a few examples of using Helm to manage Kubernetes.

Helm in K8s

Creating a basic Helm chart

show

bash

helm create chart-test ## this would create a helm

Running a Helm chart

show

bash

helm install -f myvalues.yaml myredis ./redis

Find pending Helm deployments on all namespaces

show

bash

helm list --pending -A

Uninstall a Helm release

show

bash

helm uninstall -n namespace release_name

Upgrading a Helm chart

show

bash

helm upgrade -f myvalues.yaml -f override.yaml redis ./redis

Using Helm repo

show

Add, list, remove, update and index chart repos

bash

helm repo add [NAME] [URL]  [flags]

helm repo list / helm repo ls

helm repo remove [REPO1] [flags]

helm repo update / helm repo up

helm repo update [REPO1] [flags]

helm repo index [DIR] [flags]

Download a Helm chart from a repository

show

bash

helm pull [chart URL | repo/chartname] [...] [flags] ## this would download a helm, not install 
helm pull --untar [rep/chartname] # untar the chart after downloading it

Add the Bitnami repo at https://charts.bitnami.com/bitnami to Helm

show

bash

helm repo add bitnami https://charts.bitnami.com/bitnami

Write the contents of the values.yaml file of the `bitnami/node` chart to standard output

show

bash

helm show values bitnami/node

Install the `bitnami/node` chart setting the number of replicas to 5

show

To achieve this, we need two key pieces of information:

The name of the attribute in values.yaml which controls replica count
A simple way to set the value of this attribute during installation

To identify the name of the attribute in the values.yaml file, we could get all the values, as in the previous task, and then grep to find attributes matching the pattern replica

bash

helm show values bitnami/node | grep -i replica

which returns

bash

## @param replicaCount Specify the number of replicas for the application
replicaCount: 1

We can use the --set argument during installation to override attribute values. Hence, to set the replica count to 5, we need to run

bash

helm install mynode bitnami/node --set replicaCount=5

File: i.crd.md

Extend the Kubernetes API with CRD (CustomResourceDefinition)

Note: CRD is part of the new CKAD syllabus. Here are a few examples of installing custom resource into the Kubernetes API by creating a CRD.

CRD in K8s

Create a CustomResourceDefinition manifest file for an Operator with the following specifications :

Name : operators.stable.example.com
Group : stable.example.com
Schema: <name: string><age: integer>
Scope: Namespaced
Names: <singular: operator><shortNames: op>
Kind: Operator

show

yaml

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must match the spec fields below, and be in the form: <plural>.<group>
  name: operators.stable.example.com
spec:
  group: stable.example.com
  versions:
    - name: v1
      served: true
      # One and only one version must be marked as the storage version.
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                email:
                  type: string
                name:
                  type: string
                age:
                  type: integer
  scope: Namespaced
  names:
    plural: operators
    singular: operator
    # kind is normally the CamelCased singular type. Your resource manifests use this.
    kind: Operator
    shortNames:
    - op

Create the CRD resource in the K8S API

show

bash

kubectl apply -f operator-crd.yml

Create custom object from the CRD

Name : operator-sample
Kind: Operator
Spec:
- email: [email protected]
- name: operator sample
- age: 30

show

yaml

apiVersion: stable.example.com/v1
kind: Operator
metadata:
  name: operator-sample
spec:
  email: [email protected]
  name: "operator sample"
  age: 30

bash

kubectl apply -f operator.yml

Listing operator

show

Use singular, plural and short forms

bash

kubectl get operators
or
kubectl get operator
or
kubectl get op

File: j.podman.md

Define, build and modify container images

Note: The topic is part of the new CKAD syllabus. Here are a few examples of using podman to manage the life cycle of container images. The use of docker had been the industry standard for many years, but now large companies like Red Hat are moving to a new suite of open source tools: podman, skopeo and buildah. Also Kubernetes has moved in this direction. In particular, podman is meant to be the replacement of the docker command: so it makes sense to get familiar with it, although they are quite interchangeable considering that they use the same syntax.

Podman basics

Create a Dockerfile to deploy an Apache HTTP Server which hosts a custom main page

show

dockerfile

FROM docker.io/httpd:2.4
RUN echo "Hello, World!" > /usr/local/apache2/htdocs/index.html

Build and see how many layers the image consists of

show

bash

:~$ podman build -t simpleapp .
STEP 1/2: FROM httpd:2.4
STEP 2/2: RUN echo "Hello, World!" > /usr/local/apache2/htdocs/index.html
COMMIT simpleapp
--> ef4b14a72d0
Successfully tagged localhost/simpleapp:latest
ef4b14a72d02ae0577eb0632d084c057777725c279e12ccf5b0c6e4ff5fd598b

:~$ podman images
REPOSITORY               TAG         IMAGE ID      CREATED        SIZE
localhost/simpleapp      latest      ef4b14a72d02  8 seconds ago  148 MB
docker.io/library/httpd  2.4         98f93cd0ec3b  7 days ago     148 MB

:~$ podman image tree localhost/simpleapp:latest
Image ID: ef4b14a72d02
Tags:     [localhost/simpleapp:latest]
Size:     147.8MB
Image Layers
├── ID: ad6562704f37 Size:  83.9MB
├── ID: c234616e1912 Size: 3.072kB
├── ID: c23a797b2d04 Size: 2.721MB
├── ID: ede2e092faf0 Size: 61.11MB
├── ID: 971c2cdf3872 Size: 3.584kB Top Layer of: [docker.io/library/httpd:2.4]
└── ID: 61644e82ef1f Size: 6.144kB Top Layer of: [localhost/simpleapp:latest]

Run the image locally, inspect its status and logs, finally test that it responds as expected

show

bash

:~$ podman run -d --name test -p 8080:80 localhost/simpleapp
2f3d7d613ea6ba19703811d30704d4025123c7302ff6fa295affc9bd30e532f8

:~$ podman ps
CONTAINER ID  IMAGE                       COMMAND           CREATED        STATUS            PORTS                 NAMES
2f3d7d613ea6  localhost/simpleapp:latest  httpd-foreground  5 seconds ago  Up 6 seconds ago  0.0.0.0:8080->80/tcp  test

:~$ podman logs test
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.0.2.100. Set the 'ServerName' directive globally to suppress this message
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.0.2.100. Set the 'ServerName' directive globally to suppress this message
[Sat Jun 04 16:15:38.071377 2022] [mpm_event:notice] [pid 1:tid 139756978220352] AH00489: Apache/2.4.53 (Unix) configured -- resuming normal operations
[Sat Jun 04 16:15:38.073570 2022] [core:notice] [pid 1:tid 139756978220352] AH00094: Command line: 'httpd -D FOREGROUND'

:~$ curl 0.0.0.0:8080
Hello, World!

Run a command inside the pod to print out the index.html file

show

bash

:~$ podman exec -it test cat /usr/local/apache2/htdocs/index.html
Hello, World!

Tag the image with ip and port of a private local registry and then push the image to this registry

show

Note: Some small distributions of Kubernetes (such as microk8s) have a built-in registry you can use for this exercise. If this is not your case, you'll have to setup it on your own.

bash

:~$ podman tag localhost/simpleapp $registry_ip:5000/simpleapp

:~$ podman push $registry_ip:5000/simpleapp

Verify that the registry contains the pushed image and that you can pull it

show

bash

:~$ curl http://$registry_ip:5000/v2/_catalog
{"repositories":["simpleapp"]}

# remove the image already present
:~$ podman rmi $registry_ip:5000/simpleapp

:~$ podman pull $registry_ip:5000/simpleapp
Trying to pull 10.152.183.13:5000/simpleapp:latest...
Getting image source signatures
Copying blob 643ea8c2c185 skipped: already exists
Copying blob 972107ece720 skipped: already exists
Copying blob 9857eeea6120 skipped: already exists
Copying blob 93859aa62dbd skipped: already exists
Copying blob 8e47efbf2b7e skipped: already exists
Copying blob 42e0f5a91e40 skipped: already exists
Copying config ef4b14a72d done
Writing manifest to image destination
Storing signatures
ef4b14a72d02ae0577eb0632d084c057777725c279e12ccf5b0c6e4ff5fd598b

Create a container without running/starting it

show

bash

:~$ podman create busybox # create
Resolved "busybox" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/busybox:latest...
Getting image source signatures
Copying blob sha256:213a27df5921cd9ae24732504c590bb6408911c20fb50a597f2a40896d554a8f
Copying config sha256:3fba0c87fcc8ba126bf99e4ee205b43c91ffc6b15bb052315312e71bc6296551
Writing manifest to image destination
51b613406e8889213c176523e1c430e4bd00047965b0c22cff5b1c9badfbc452

:~$ podman container ls -a
CONTAINER ID  IMAGE                             COMMAND     CREATED        STATUS      PORTS       NAMES
51b613406e88  docker.io/library/busybox:latest  sh          2 minutes ago  Created                 adoring_almeida

Export a container to output.tar file

show

bash

:~$ podman container ls -a # pick the container id
CONTAINER ID  IMAGE                             COMMAND     CREATED        STATUS      PORTS       NAMES
51b613406e88  docker.io/library/busybox:latest  sh          2 minutes ago  Created                 adoring_almeida

:~$ podman export <container id> --output=output.tar

:~$ ls -al output.tar
-rw-r--r--@ 1 limistah  wheel  4272640 28 Aug 13:48 output.tar

Run a pod with the image pushed to the registry

show

bash

:~$ kubectl run simpleapp --image=$registry_ip:5000/simpleapp --port=80

:~$ curl ${kubectl get pods simpleapp -o jsonpath='{.status.podIP}'}
Hello, World!

Log into a remote registry server and then read the credentials from the default file

show

Note: The two most used container registry servers with a free plan are DockerHub and Quay.io.

bash

:~$ podman login --username $YOUR_USER --password $YOUR_PWD docker.io
:~$ cat ${XDG_RUNTIME_DIR}/containers/auth.json
{
        "auths": {
                "docker.io": {
                        "auth": "Z2l1bGl0JLSGtvbkxCcX1xb617251xh0x3zaUd4QW45Q3JuV3RDOTc="
                }
        }
}

show

bash

:~$ kubectl create secret generic mysecret --from-file=.dockerconfigjson=${XDG_RUNTIME_DIR}/containers/auth.json --type=kubernetes.io/dockeconfigjson
secret/mysecret created
:~$ kubectl create secret docker-registry mysecret2 --docker-server=https://index.docker.io/v1/ --docker-username=$YOUR_USR --docker-password=$YOUR_PWD
secret/mysecret2 created

Create the manifest for a Pod that uses one of the two secrets just created to pull an image hosted on the relative private remote registry

show

yaml

apiVersion: v1
kind: Pod
metadata:
  name: private-reg
spec:
  containers:
  - name: private-reg-container
    image: $YOUR_PRIVATE_IMAGE
  imagePullSecrets:
  - name: mysecret

Clean up all the images and containers

show

bash

:~$ podman rm --all --force
:~$ podman rmi --all
:~$ kubectl delete pod simpleapp

File: README.md Copy Chapter

CKAD Exercises

Contents Copy Chapter

Can I PR? There is an error/an alternative way/an extra question/solution I can offer Copy Section

File: a.core_concepts.md Copy Chapter

Core Concepts (13%)

Create a namespace called 'mynamespace' and a pod with image nginx called nginx on this namespace Copy Section

Create the pod that was just described using YAML Copy Section

Create a busybox pod (using kubectl command) that runs the command "env". Run it and see the output Copy Section

Create a busybox pod (using YAML) that runs the command "env". Run it and see the output Copy Section

Get the YAML for a new namespace called 'myns' without creating it Copy Section

Create the YAML for a new ResourceQuota called 'myrq' with hard limits of 1 CPU, 1G memory and 2 pods without creating it Copy Section

Get pods on all namespaces Copy Section

Create a pod with image nginx called nginx and expose traffic on port 80 Copy Section

Change pod's image to nginx:1.24.0. Observe that the container will be restarted as soon as the image gets pulled Copy Section

Get nginx pod's ip created in previous step, use a temp busybox image to wget its '/' Copy Section

Get pod's YAML Copy Section

Get information about the pod, including details about potential issues (e.g. pod hasn't started) Copy Section

Get pod logs Copy Section

If pod crashed and restarted, get logs about the previous instance Copy Section

Execute a simple shell on the nginx pod Copy Section

Create a busybox pod that echoes 'hello world' and then exits Copy Section

Do the same, but have the pod deleted automatically when it's completed Copy Section

Create an nginx pod and set an env value as 'var1=val1'. Check the env value existence within the pod Copy Section

File: b.multi_container_pods.md Copy Chapter

Multi-container Pods (10%)

Create a Pod with two containers, both with image busybox and command "echo hello; sleep 3600". Connect to the second container and run 'ls' Copy Section

File: c.pod_design.md Copy Chapter

Pod design (20%)

Labels and Annotations Copy Chapter

Create 3 pods with names nginx1,nginx2,nginx3. All of them should have the label app=v1 Copy Section

Show all labels of the pods Copy Section

Change the labels of pod 'nginx2' to be app=v2 Copy Section

Get the label 'app' for the pods (show a column with APP labels) Copy Section

Get only the 'app=v2' pods Copy Section

Get 'app=v2' and not 'tier=frontend' pods Copy Section

Add a new label tier=web to all pods having 'app=v2' or 'app=v1' labels Copy Section

Add an annotation 'owner: marketing' to all pods having 'app=v2' label Copy Section

Remove the 'app' label from the pods we created before Copy Section

Annotate pods nginx1, nginx2, nginx3 with "description='my description'" value Copy Section

Check the annotations for pod nginx1 Copy Section

Remove the annotations for these three pods Copy Section

Remove these pods to have a clean state in your cluster Copy Section

Pod Placement Copy Chapter

Create a pod that will be deployed to a Node that has the label 'accelerator=nvidia-tesla-p100' Copy Section

Create a pod that will be placed on node `node01` using `nodeName` Copy Section

Taint a node with key `tier` and value `frontend` with the effect `NoSchedule`. Then, create a pod that tolerates this taint. Copy Section

Create a pod that will be placed on node `controlplane`. Use nodeSelector and tolerations. Copy Section

Deployments Copy Chapter

Create a deployment with image nginx:1.18.0, called nginx, having 2 replicas, defining port 80 as the port that this container exposes (don't create a service for this deployment) Copy Section

View the YAML of this deployment Copy Section

View the YAML of the replica set that was created by this deployment Copy Section

Get the YAML for one of the pods Copy Section

Check how the deployment rollout is going Copy Section

Update the nginx image to nginx:1.19.8 Copy Section

Check the rollout history and confirm that the replicas are OK Copy Section

Undo the latest rollout and verify that new pods have the old image (nginx:1.18.0) Copy Section

Do an on-purpose update of the deployment with a wrong image nginx:1.91 Copy Section

Verify that something's wrong with the rollout Copy Section

Return the deployment to the second revision (number 2) and verify the image is nginx:1.19.8 Copy Section

Check the details of the fourth revision (number 4) Copy Section

Scale the deployment to 5 replicas Copy Section

Autoscale the deployment, pods between 5 and 10, targeting CPU utilization at 80% Copy Section

Pause the rollout of the deployment Copy Section

Update the image to nginx:1.19.9 and check that there's nothing going on, since we paused the rollout Copy Section

Resume the rollout and check that the nginx:1.19.9 image has been applied Copy Section

Delete the deployment and the horizontal pod autoscaler you created Copy Section

Implement canary deployment by running two instances of nginx marked as version=v1 and version=v2 so that the load is balanced at 75%-25% ratio Copy Section

Jobs Copy Chapter

Create a job named pi with image perl:5.34 that runs the command with arguments "perl -Mbignum=bpi -wle 'print bpi(2000)'" Copy Section

Wait till it's done, get the output Copy Section

Create a job with the image busybox that executes the command 'echo hello;sleep 30;echo world' Copy Section

Follow the logs for the pod (you'll wait for 30 seconds) Copy Section

See the status of the job, describe it and see the logs Copy Section

Delete the job Copy Section

Create the same job, make it run 5 times, one after the other. Verify its status and delete it Copy Section

Create the same job, but make it run 5 parallel times Copy Section

Create a job but ensure that it will be automatically terminated by kubernetes if it takes more than 30 seconds to execute Copy Section

Cron jobs Copy Chapter

Create a cron job with image busybox that runs on a schedule of "*/1 * * * *" and writes 'date; echo Hello from the Kubernetes cluster' to standard output Copy Section

File: README.md

Contents

Can I PR? There is an error/an alternative way/an extra question/solution I can offer

File: a.core_concepts.md

Create a namespace called 'mynamespace' and a pod with image nginx called nginx on this namespace

Create the pod that was just described using YAML

Create a busybox pod (using kubectl command) that runs the command "env". Run it and see the output

Create a busybox pod (using YAML) that runs the command "env". Run it and see the output

Get the YAML for a new namespace called 'myns' without creating it

Create the YAML for a new ResourceQuota called 'myrq' with hard limits of 1 CPU, 1G memory and 2 pods without creating it

Get pods on all namespaces

Create a pod with image nginx called nginx and expose traffic on port 80

Change pod's image to nginx:1.24.0. Observe that the container will be restarted as soon as the image gets pulled

Get nginx pod's ip created in previous step, use a temp busybox image to wget its '/'

Get pod's YAML

Get information about the pod, including details about potential issues (e.g. pod hasn't started)

Get pod logs

If pod crashed and restarted, get logs about the previous instance

Execute a simple shell on the nginx pod

Create a busybox pod that echoes 'hello world' and then exits

Do the same, but have the pod deleted automatically when it's completed

Create an nginx pod and set an env value as 'var1=val1'. Check the env value existence within the pod

File: b.multi_container_pods.md

Create a Pod with two containers, both with image busybox and command "echo hello; sleep 3600". Connect to the second container and run 'ls'

File: c.pod_design.md

Labels and Annotations

Create 3 pods with names nginx1,nginx2,nginx3. All of them should have the label app=v1

Show all labels of the pods

Change the labels of pod 'nginx2' to be app=v2

Get the label 'app' for the pods (show a column with APP labels)

Get only the 'app=v2' pods

Get 'app=v2' and not 'tier=frontend' pods

Add a new label tier=web to all pods having 'app=v2' or 'app=v1' labels

Add an annotation 'owner: marketing' to all pods having 'app=v2' label

Remove the 'app' label from the pods we created before

Annotate pods nginx1, nginx2, nginx3 with "description='my description'" value

Check the annotations for pod nginx1

Remove the annotations for these three pods

Remove these pods to have a clean state in your cluster

Pod Placement

Create a pod that will be deployed to a Node that has the label 'accelerator=nvidia-tesla-p100'

Create a pod that will be placed on node `node01` using `nodeName`

Taint a node with key `tier` and value `frontend` with the effect `NoSchedule`. Then, create a pod that tolerates this taint.

Create a pod that will be placed on node `controlplane`. Use nodeSelector and tolerations.

Deployments

Create a deployment with image nginx:1.18.0, called nginx, having 2 replicas, defining port 80 as the port that this container exposes (don't create a service for this deployment)

View the YAML of this deployment

View the YAML of the replica set that was created by this deployment

Get the YAML for one of the pods

Check how the deployment rollout is going

Update the nginx image to nginx:1.19.8

Check the rollout history and confirm that the replicas are OK

Undo the latest rollout and verify that new pods have the old image (nginx:1.18.0)

Do an on-purpose update of the deployment with a wrong image nginx:1.91

Verify that something's wrong with the rollout

Return the deployment to the second revision (number 2) and verify the image is nginx:1.19.8

Check the details of the fourth revision (number 4)

Scale the deployment to 5 replicas

Autoscale the deployment, pods between 5 and 10, targeting CPU utilization at 80%

Pause the rollout of the deployment

Update the image to nginx:1.19.9 and check that there's nothing going on, since we paused the rollout

Resume the rollout and check that the nginx:1.19.9 image has been applied

Delete the deployment and the horizontal pod autoscaler you created

Implement canary deployment by running two instances of nginx marked as version=v1 and version=v2 so that the load is balanced at 75%-25% ratio

Jobs

Create a job named pi with image perl:5.34 that runs the command with arguments "perl -Mbignum=bpi -wle 'print bpi(2000)'"

Wait till it's done, get the output

Create a job with the image busybox that executes the command 'echo hello;sleep 30;echo world'

Follow the logs for the pod (you'll wait for 30 seconds)

See the status of the job, describe it and see the logs

Delete the job

Create the same job, make it run 5 times, one after the other. Verify its status and delete it

Create the same job, but make it run 5 parallel times

Create a job but ensure that it will be automatically terminated by kubernetes if it takes more than 30 seconds to execute

Cron jobs

Create a cron job with image busybox that runs on a schedule of "/1 * * *" and writes 'date; echo Hello from the Kubernetes cluster' to standard output

See its logs and delete it

Create the same cron job again, and watch the status. Once it ran, check which job ran by the created cron job. Check the log, and delete the cron job

Create a cron job with image busybox that runs every minute and writes 'date; echo Hello from the Kubernetes cluster' to standard output. The cron job should be terminated if it successfully starts but takes more than 12 seconds to complete execution.

Keep only the last 2 successful and 1 failed runs of a CronJob