K8s FAQ
-
Copy file in/out of pod to root
kubectl cp examplefile.zip xxpod-2133rfsdf:/examplefile.zip
-
Ext4 Folder is not empty
-
Err
[ERROR] --initialize specified but the data directory has files in it. Aborting.
-
Ans
args:
- "--ignore-db-dir=lost+found"
-
Get pod event
# work on kubectl v1.14 against a v1.11 API
kubectl get event --namespace abc-namespace --field-selector involvedObject.name=my-pod-zl6m6
kubectl describe event [POD_NAME] --namespace [POD's_NAMESPACE]
-
Create self-signed CA
kubectl create secret tls daas-tls --key daas.trendmicro.com.key --cert daas.trendmicro.com.crt
-
Install kubectl
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl # For user
sudo mv ./kubectl /usr/bin/kubectl # For root
kubectl version
-
Apply kubeconfig
-
Operate current context token
kubectl config use-context {contextName} # set current context
kubectl config current-context # get current context
-
Switch current namespace(context)
kubectl config set-context --current --namespace={my namespace}
-
Probe (liveness/readiness)
-
Keep running pod
command: ["ping", "-t", "google.com"]
command: ["/bin/sh", "-ec", "while true; do echo 'test'; sleep 5 ; done"]
-
Add command to deployment
apiVersion: v1
kind: Pod
metadata:
name: command-demo
labels:
purpose: demonstrate-command
spec:
containers:
- name: command-demo-container
image: debian
command: ["printenv"]
args: ["HOSTNAME", "KUBERNETES_PORT"]
restartPolicy: OnFailure
-
Https (k8s ingress TLS) “default backend 404” error
-
Ans: tls secret not correct, TLS not
-
Create TLS secret by file: .crt .key (Letsencrypt)
kubectl -n XX create secret tls tls-XXX \
--cert=signed.crt \
--key=domain.key
-
Create an base64 string for k8s Secret Opaque
echo -n 'password' | base64
# Or Notepad++ => MINE tool > Base64 Encode with Unix EOL
<--- sample file
apiVersion: v1
kind: Secret
metadata:
name: your-secrets
type: Opaque
data:
root-password: XXXXXXX
-
Clean key by patch
$ kubectl patch configmap myconfigmap --type=json -p='[{"op": "remove", "path": "/data/mykey"}]'
-
Rollback version
# List old
kubectl rollout history deployment/app
# Rollback to
kubectl rollout undo deployment/app --to-revision=2
-
Release pv (Persistent Volume) to be avaliable again. unbind, unbound PV
kubectl edit pv PV_NAME
# Remove spec.claimRef
# Or command:
kubectl patch pv {{PV_NAME}} --type=json -p='[{"op": "remove", "path": "/spec/claimRef"}]'
-
Search and get pod name
kubectl get pods -l app=my-app -o custom-columns=:metadata.name
-
kubectl Copy file into pod: error directory not exists or not found.
kubectl --kubeconfig=xxx cp {{filename}} {{namespace}}/{{pod}}:/{{filename}}
# {{filename}} is needed!!
-
CronJob sample
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: jj-triggerjob
spec:
schedule: "* 6 * * *"
jobTemplate:
spec:
backoffLimit: 4
template:
spec:
containers:
- name: jj-triggerjob
image: dockerhub/repo:latest
resources:
limits:
cpu: 400m
memory: 512Mi
requests:
cpu: 400m
memory: 512Mi
args:
- /bin/sh
- -c
- " echo \"Running trigger job\";
status_code=$(curl -o /dev/null -s -w \"%{http_code}\" -X PUT jjgoodapi/api/v1/person -H \"accept: application/octet-stream\" -H \"my-trace-id: $(uuidgen)\" -H \"my-application-name: biapi-triggerjob\" -H \"my-api-key: XXXXX\");
echo $exit_status;
if ! echo $status_code | grep -e \"202\" -e \"406\" ;
then
echo \"Failed without status code: 202 or 406\";
echo \"Done trigger job\";
exit 1;
fi;
echo \"Passed with status code: 202 or 406\";
echo \"Done trigger job\";
"
# imagepullsecrets:
# - name: XXAccount
restartPolicy: OnFailure
nodeSelector:
beta.kubernetes.io/os: linux
-
Warning:
-
always has ;
-
can’t use square brackets [ ] around with if echo $xxx | grep
-
Use “curl -o /dev/null -s -w \”%{http_code}\” ” to get status code
-
Unknown object type “nil”
-
error msg
error: error validating "/home/jj/deployment.yaml": error validating data: [ValidationError(Ingress.spec.tls[0].hosts): unknown object type "nil" in Ingress.spec.tls[0].hosts[0],
-
Solve: Fix wrong yaml format.
-
Apply private registry credential
kubectl create secret generic regcred \
--from-file=.dockerconfigjson={{path/to/.docker/config.json}} \
--type=kubernetes.io/dockerconfigjson
# Add to deployment (pod)
spec:
containers:
- name: xxx
image: xxx
imagePullSecrets:
- name: regcred
-
Deployment not updating after image updated, Force deployment rolling-update
spec:
template:
spec:
containers:
- image: xxx
imagePullPolicy: Always
-
Deployment not updating after configmap updated
-
Update label to trigger deployment rolling update
metadata:
labels:
configmap-version: 1
-
ConfigMap has a size limit of 1Mb, exceed error:
-
Msg: The ConfigMap “XXXXXXX” is invalid: metadata.annotations: Too long: must have at most 262144 characters
-
Use kubectl replace
-
Ref: https://github.com/coreos/prometheus-operator/issues/535#issuecomment-319936366
K8s dashboard
-
Paste string to EXEC UI
-
ctrl-shift-v
-
Login issue: namespace change to default
-
Solution: type it back at uri
-
Login issue: exec into pod via Firefox will redirect back to k8s portal.
-
Sol: Use other browsers.
DockerHub
-
Always get handshake fail when `docker pull`
-
Solution:
Login DockerHub with the account which has no email address.
-
Download image fail: Authentication fail 401
-
Message:
(HTTP code 401) unexpected – {“message”:”Get https://registry-1.docker.io/v2/library/jenkins/manifests/2.46.3: unauthorized: incorrect username or password”}
Solution:
logging into Kitematic with your username instead of email.
-
Check the files permission in docker image
docker run –rm -ti –entrypoint sh jj/docker-stacks -c “ls -alF /usr/local/bin/”
Docker
-
Switch user
Dockerfile
---
FROM tw.registry.trendmicro.com/ik8s/win-dotnetcore-runtime:3.1-nanoserver-1809
USER "ContainerAdministrator"
-
Force delete pod
kubectl -n yyy delete pods xxxx --force --grace-period 0
-
Docker with GrayLog
-
Unable to start container by docker-compose
-
Msg: “UnixHTTPConnectionPool(host=’localhost’, port=None): Read timed out. (read timeout=60)”
-
Ans: ` sudo service docker restart`
-
[Character in Dockerfile]: ” will be split by space
in echo " xxx string " > file.txt
# result: file.txt
# xxx
# string
-
[Character in Dockerfile]: ” will be remove inside ‘ “xxx” ‘
echo ' "xxx string" ' > file.txt
# result: file.txt
# xxx string
-
[Cronjob] – Clean container&image daily at mid-night
# Clean container
0 0 * * * docker rm -f $(docker ps -aq)
# Clean image without baseImage
0 5 * * * docker image prune -f; docker rmi -f $(docker images | awk '/^[^m][^c][^r]*/{ print $3 }')
0 5 * * * docker rmi -f $(docker images | awk '$1 !~/ik8s/{ print $3 }')
0 5 * * * docker image prune -f --filter="dangling=true"; docker image prune -f --all --filter until=168h
# Clean all unused build cache
docker builder prune -a
# Clean all
docker system prune -a
# Clean image older than 48h
docker image prune -f --all --filter until=48h
# Clean dangling images
docker rmi $(sudo docker images -f "dangling=true" -q)
-
Not enough memory to start Docker on Windows
-
Modify `C:\Program Files\Docker\Docker\resources\MobyLinux.ps1` and change `$Memory = 512` MB as you want
-
Install with `sudo` but `docker run` without it, got error: “docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.35/containers/create: dial unix /var/run/docker.sock: connect: permission denied.See ‘docker run –help’.“.
sudo groupadd docker
sudo usermod -aG docker $USER # Add user into group
Ref: https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user
-
ERROR: Get https://registry-1.docker.io/v2/: remote error: tls: handshake failure
docker login # Yes, login first
-
Windows 10: Change docker images and any stuff to another folder, notice that the slash and case of character.
-- C:\ProgramData\docker\config\daemon.json --
{
"registry-mirrors": [],
"insecure-registries": [],
"debug": true,
"experimental": false,
"graph":"D:\\ProgramData\\docker"
}
-
docker: Error response from daemon: driver failed programming external connectivity on endpoint
-
Restart docker
-
The SPA default page middleware could not return the default page ‘/index.html’ because it was not found, and no other middleware handled the request.
-
mkdir /host_mnt/c: file exists.
-
Re-apply Shared Drive in docker-Desktop.
Docker compose
-
Setting up network mode (avoid IP not found)
version: '3.1'
services:
zookeeper-1:
image: zookeeper:3.4.13
container_name: zk
network_mode: bridge
-
Docker IP not match
# Check docker container IP
docker network inspect XXX
# Rebuild network (restart not working)
docker-compose down
docker-compose up
PS. docker-compose restart <- won't rebuild
-
ERROR: client version 1.22 is too old. Minimum supported API version is 1.24, please upgrade your client to a newer version
--- docker-compose.yml ---
version: '2.1'
...