• Copy file in/out of pod to root

kubectl cp examplefile.zip xxpod-2133rfsdf:/examplefile.zip
  • Ext4 Folder is not empty

    • Err

[ERROR] --initialize specified but the data directory has files in it. Aborting.
    • Ans

  - "--ignore-db-dir=lost+found"
  • Get pod event

# work on kubectl v1.14 against a v1.11 API
kubectl get event --namespace abc-namespace --field-selector involvedObject.name=my-pod-zl6m6
kubectl describe event [POD_NAME] --namespace [POD's_NAMESPACE]
  • Create self-signed CA

kubectl create secret tls daas-tls --key daas.trendmicro.com.key --cert daas.trendmicro.com.crt
  • Install kubectl

curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl

chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl  # For user
sudo mv ./kubectl /usr/bin/kubectl  # For root
kubectl version
  • Apply kubeconfig

  • Operate current context token

kubectl config use-context {contextName}    # set current context
kubectl config current-context    # get current context
  • Switch current namespace(context)

kubectl config set-context --current --namespace={my namespace}
  • Probe (liveness/readiness)

ref: https://andrewlock.net/deploying-asp-net-core-applications-to-kubernetes-part-6-adding-health-checks-with-liveness-readiness-and-startup-probes/

  • Keep running pod

command: ["ping", "-t", "google.com"]
command: ["/bin/sh", "-ec", "while true; do echo 'test'; sleep 5 ; done"]
  • Add command to deployment

apiVersion: v1
kind: Pod
  name: command-demo
    purpose: demonstrate-command
  - name: command-demo-container
    image: debian
    command: ["printenv"]
  restartPolicy: OnFailure
  • Https (k8s ingress TLS) “default backend 404” error

    • Ans: tls secret not correct, TLS not 

  • Create TLS secret by file: .crt .key (Letsencrypt)

kubectl -n XX create secret tls tls-XXX \
  --cert=signed.crt \
  • Create an base64 string for k8s Secret Opaque

echo -n 'password' | base64
# Or Notepad++ => MINE tool > Base64 Encode with Unix EOL

<--- sample file
apiVersion: v1
kind: Secret
  name: your-secrets
type: Opaque
  root-password: XXXXXXX
  • Clean key by patch

$ kubectl patch configmap myconfigmap --type=json -p='[{"op": "remove", "path": "/data/mykey"}]'
  • Rollback version

# List old
kubectl rollout history deployment/app
# Rollback to
kubectl rollout undo deployment/app --to-revision=2
  • Release pv (Persistent Volume) to be avaliable again.  unbind, unbound PV

kubectl edit pv PV_NAME
# Remove spec.claimRef
# Or command:
kubectl patch pv {{PV_NAME}} --type=json -p='[{"op": "remove", "path": "/spec/claimRef"}]'
  • Search and get pod name 

kubectl get pods -l app=my-app -o custom-columns=:metadata.name
  • kubectl Copy file into pod: error directory not exists or not found. 

kubectl --kubeconfig=xxx cp {{filename}} {{namespace}}/{{pod}}:/{{filename}}
# {{filename}} is needed!!
  • CronJob sample

apiVersion: batch/v1beta1
kind: CronJob
  name: jj-triggerjob
  schedule: "* 6 * * *"
      backoffLimit: 4
          - name: jj-triggerjob
            image: dockerhub/repo:latest
                cpu: 400m
                memory: 512Mi
                cpu: 400m
                memory: 512Mi
                - /bin/sh
                - -c
                - " echo \"Running trigger job\";
                    status_code=$(curl -o /dev/null -s -w \"%{http_code}\" -X PUT jjgoodapi/api/v1/person -H \"accept: application/octet-stream\" -H \"my-trace-id: $(uuidgen)\" -H \"my-application-name: biapi-triggerjob\" -H \"my-api-key: XXXXX\");
                    echo $exit_status;
                    if ! echo $status_code | grep -e \"202\" -e \"406\" ;
                        echo \"Failed without status code: 202 or 406\";
                        echo \"Done trigger job\";
                        exit 1;
                    echo \"Passed with status code: 202 or 406\";
                    echo \"Done trigger job\";
          # imagepullsecrets:
          # - name: XXAccount
          restartPolicy: OnFailure
            beta.kubernetes.io/os: linux
    • Warning:

      • always has ;

      • can’t use square brackets [ ] around with if echo $xxx | grep

      • Use “curl -o /dev/null -s -w \”%{http_code}\” ” to get status code

  • Unknown object type “nil”

    • error msg

error: error validating "/home/jj/deployment.yaml": error validating data: [ValidationError(Ingress.spec.tls[0].hosts): unknown object type "nil" in Ingress.spec.tls[0].hosts[0],
    • Solve: Fix wrong yaml format.

  • Apply private registry credential

kubectl create secret generic regcred \
    --from-file=.dockerconfigjson={{path/to/.docker/config.json}} \

# Add to deployment (pod)
  - name: xxx
    image: xxx
  - name: regcred
  • Deployment not updating after image updated, Force deployment rolling-update

      - image: xxx
        imagePullPolicy: Always
  • Deployment not updating after configmap updated

    • Update label to trigger deployment rolling update

    configmap-version: 1

K8s dashboard

  • Paste string to EXEC UI

    • ctrl-shift-v

  • Login issue: namespace change to default

    • Solution: type it back at uri

  • Login issue: exec into pod via Firefox will redirect back to k8s portal.

    • Sol: Use other browsers.


  • Always get handshake fail when `docker pull`  

    • Solution:

      Login DockerHub with the account which has no email address.

  • Download image fail: Authentication fail 401

  • Check the files permission in docker image

    docker run –rm -ti –entrypoint sh jj/docker-stacks -c “ls -alF /usr/local/bin/” 


  • Switch user

FROM tw.registry.trendmicro.com/ik8s/win-dotnetcore-runtime:3.1-nanoserver-1809
USER "ContainerAdministrator"
  • Force delete pod

kubectl -n yyy delete pods xxxx --force --grace-period 0
  • Docker with GrayLog

  • Unable to start container by docker-compose

    • Msg: “UnixHTTPConnectionPool(host=’localhost’, port=None): Read timed out. (read timeout=60)”

    • Ans: ` sudo service docker restart`

  • [Character in Dockerfile]: ” will be split by space 

in echo " xxx string " > file.txt

# result: file.txt
# xxx
# string
  • [Character in Dockerfile]: ” will be remove inside ‘ “xxx” ‘

echo ' "xxx string" ' > file.txt

# result: file.txt
#  xxx string
  • [Cronjob] – Clean container&image daily at mid-night

# Clean container
0 0 * * * docker rm -f $(docker ps -aq)
# Clean image without baseImage
0 5 * * * docker image prune -f; docker rmi -f $(docker images | awk '/^[^m][^c][^r]*/{ print $3 }')
0 5 * * * docker rmi -f $(docker images | awk '$1 !~/ik8s/{ print $3 }')
0 5 * * * docker image prune -f --filter="dangling=true"; docker image prune -f --all --filter until=168h

# Clean all unused build cache
docker builder prune -a
# Clean all
docker system prune -a
# Clean image older than 48h
docker image prune -f --all --filter until=48h
# Clean dangling images
docker rmi $(sudo docker images -f "dangling=true" -q)
  • Not enough memory to start Docker on Windows

    • Modify `C:\Program Files\Docker\Docker\resources\MobyLinux.ps1` and change `$Memory = 512`  MB as you want

  • Install with `sudo` but `docker run` without it,  got error: “docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.35/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See ‘docker run –help’.“.

sudo groupadd docker
sudo usermod -aG docker $USER  # Add user into group

Ref: https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user
docker login  # Yes, login first
  • Windows 10: Change docker images and any stuff to another folder, notice that the slash and case of character.

-- C:\ProgramData\docker\config\daemon.json --
  "registry-mirrors": [],
  "insecure-registries": [],
  "debug": true,
  "experimental": false,
  • docker: Error response from daemon: driver failed programming external connectivity on endpoint

    • Restart docker

  • The SPA default page middleware could not return the default page ‘/index.html’ because it was not found, and no other middleware handled the request.

  • mkdir /host_mnt/c: file exists.

    • Re-apply Shared Drive in docker-Desktop.

Docker compose

  • Setting up network mode (avoid IP not found)

version: '3.1'

    image: zookeeper:3.4.13
    container_name: zk
    network_mode: bridge
  • Docker IP not match

# Check docker container IP
docker network inspect XXX

# Rebuild network (restart not working)
docker-compose down
docker-compose up

PS. docker-compose restart <- won't rebuild
  • ERROR: client version 1.22 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version

--- docker-compose.yml ---
version: '2.1'