Openshift - hqzhang/cloudtestbed GitHub Wiki
1. Deployment Datamodel
apiVersion: apps/v1
kind: Deployment
metadata:
name: myfirst-loki-distributed-distributor
namespace: hongqi-zhang-dev
labels:
helm.sh/chart: loki-distributed-0.79.4
spec:
replicas: 1
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/name: loki-distributed
app.kubernetes.io/instance: myfirst
template:
metadata:
annotations:
checksum/config: d26a821026d5ca3140f7ad12cabcf400dffbb336b12206e20af4469f0a056867
labels:
app.kubernetes.io/name: loki-distributed
app.kubernetes.io/instance: myfirst
spec:
serviceAccountName: myfirst-loki-distributed
securityContext:
fsGroup: 10001
runAsGroup: 10001
runAsNonRoot: true
runAsUser: 10001
terminationGracePeriodSeconds: 30
containers:
- name: distributor
image: docker.io/grafana/loki:2.9.10
imagePullPolicy: IfNotPresent
args:
- -config.file=/etc/loki/config/config.yaml
- -target=distributor
ports:
- name: http
containerPort: 3100
protocol: TCP
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
readOnlyRootFilesystem: true
readinessProbe:
httpGet:
path: /ready
port: http
initialDelaySeconds: 30
timeoutSeconds: 1
livenessProbe:
httpGet:
path: /ready
port: http
initialDelaySeconds: 300
volumeMounts:
- name: config
mountPath: /etc/loki/config
volumes:
- name: config
configMap:
name: myfirst-loki-distributed
1. ServiceAccount
Role/ClusterRole-->RoleBinding/ClusterRoleBinding
2. securityContext:
fsGroup: 10001
runAsGroup: 10001
runAsNonRoot: true
runAsUser: 10001
3. Volumes:
- name: config
configMap:
name: myfirst-loki-distributed
- name: mypd
persistentVolumeClaim:
claimName: myclaim
volumeMounts:
- name: config
mountPath: /etc/loki/config
- name: mypd
mountPath: "/var/www/html"
4. StatefulSet vs Deployment
Both create pod by replicas setting
But Deploy pods no difference while
Statefulset pod has order xxx-0(primary, others standby),xxx-1,xxx-2
5. DeamonSet vs Deployment
Both create pods, while DeamonSet create pod on every node.
6. Security Context Constraints
SCC limit a pod's access to a host environment by controlling pod permissions
oc adm policy add-scc-to-user <scc_name> -z <serviceaccount_name>
NAME PRIV CAPS SELINUX RUNASUSER FSGROUP SUPGROUP PRIORITY READONLYROOTFS VOLUMES
anyuid false [] MustRunAs RunAsAny RunAsAny RunAsAny 10 false [configMap
hostaccess false [] MustRunAs MustRunAsRange MustRunAs RunAsAny <none> false [configMap
hostmount-anyuid false [] MustRunAs RunAsAny RunAsAny RunAsAny <none> false [configMap
hostnetwork false [] MustRunAs MustRunAsRange MustRunAs MustRunAs <none> false [configMap
nonroot false [] MustRunAs MustRunAsNonRoot RunAsAny RunAsAny <none> false [configMap
privileged true [*] RunAsAny RunAsAny RunAsAny RunAsAny <none> false [*]
restricted false [] MustRunAs MustRunAsRange MustRunAs RunAsAny <none> false [configMap
kind: SecurityContextConstraints
apiVersion: security.openshift.io/v1
metadata:
name: scc-admin
allowPrivilegedContainer: true
runAsUser:
type: RunAsAny
seLinuxContext:
type: RunAsAny
fsGroup:
type: RunAsAny
supplementalGroups:
type: RunAsAny
users:
- my-admin-user
groups:
- my-admin-group ///How to use it add user into users
7. tolerations: using node taints
kubectl taint nodes node1 key1=value1:NoSchedule
How to use:
tolerations:
- key: "key1"
operator: "Equal"
value: "value1"
effect: "NoSchedule"
8. A PersistentVolumeClaim (PVC)
kind: PersistentVolume
metadata:
name: pv0003
spec:
capacity:
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: slow
mountOptions:
- hard
- nfsvers=4.1
nfs:
path: /tmp
server: 172.17.0.2
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: myclaim
spec:
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 8Gi
storageClassName: slow
selector:
matchLabels:
release: "stable"
matchExpressions:
How to use
oc apply -f myfile.yaml. # create or update myfile.yaml dont use create one.
oc create -f myfile.yaml. # just create new myfile.yaml
oc expose svc/loki-grafana # create route to expose, use oc create route is more flexible.
oc create route
oc annotate route <route_name> \
--overwrite haproxy.router.openshift.io/timeout=<timeout><time_unit>
kc expose svc/loki-grafana #
oc port-forward svc/loki-grafana # this can forward svc/pod/contain etc to local space.
Basic Commands:
login Log in to a server
new-project Request a new project
new-app Create a new application
status Show an overview of the current project
project Switch to another project
projects Display existing projects
explain Get documentation for a resource
Build and Deploy Commands:
rollout Manage a Kubernetes deployment or OpenShift deployment config
rollback Revert part of an application back to a previous deployment
new-build Create a new build configuration
start-build Start a new build
cancel-build Cancel running, pending, or new builds
import-image Import images from a container image registry
tag Tag existing images into image streams
Application Management Commands:
create Create a resource from a file or from stdin
apply Apply a configuration to a resource by file name or stdin
get Display one or many resources
describe Show details of a specific resource or group of resources
edit Edit a resource on the server
set Commands that help set specific features on objects
label Update the labels on a resource
annotate Update the annotations on a resource
expose Expose a replicated application as a service or route
delete Delete resources by file names, stdin, resources and names, or by resources and
label selector
scale Set a new size for a deployment, replica set, or replication controller
autoscale Autoscale a deployment config, deployment, replica set, stateful set, or
replication controller
secrets Manage secrets
serviceaccounts Manage service accounts in your project
Troubleshooting and Debugging Commands:
logs Print the logs for a container in a pod
rsh Start a shell session in a container
rsync Copy files between a local file system and a pod
port-forward Forward one or more local ports to a pod
debug Launch a new instance of a pod for debugging
exec Execute a command in a container
proxy Run a proxy to the Kubernetes API server
attach Attach to a running container
run Run a particular image on the cluster
cp Copy files and directories to and from containers
wait Experimental: Wait for a specific condition on one or many resources
Advanced Commands:
adm Tools for managing a cluster
replace Replace a resource by file name or stdin
patch Update fields of a resource
process Process a template into list of resources
extract Extract secrets or config maps to disk
observe Observe changes to resources and react to them (experimental)
policy Manage authorization policy
auth Inspect authorization
image Useful commands for managing images
registry Commands for working with the registry
idle Idle scalable resources
api-versions Print the supported API versions on the server, in the form of "group/version"
api-resources Print the supported API resources on the server
cluster-info Display cluster information
diff Diff the live version against a would-be applied version
kustomize Build a kustomization target from a directory or URL.
Settings Commands:
logout End the current server session
config Modify kubeconfig files
whoami Return information about the current session
completion Output shell completion code for the specified shell (bash, zsh or fish)
Other Commands:
help Help about any command
plugin Provides utilities for interacting with plugins
version Print the client and server version information
ns=my-project
region=east
sa=loki
echo "login cluster"
ibmcloud oc cluster config -q -c logs-dev-us-${region}-cluster --admin
echo "set scc"
oc adm policy remove-scc-from-user -z $sa -n my-project
oc adm policy remove-scc-from-user -z $sa-grafana -n my-project
oc adm policy add-scc-to-user anyuid -z ${sa} -n ${ns}
oc adm policy add-scc-to-user anyuid -z ${sa}-grafana -n ${ns}
echo "view scc"
oc adm policy who-can use scc anyuid
echo "cleanup project"
helm uninstall loki
helm uninstall loki-grafana
#helm uninstall loki-promtail
echo "wait 20 seconds"
sleep 20
echo "install loki.."
helm upgrade --install loki --namespace=loki grafana/loki
echo "instakk grafana.."
helm install loki-grafana grafana/grafana --set grafana.securityContext.fsGroup=65533,grafana.securityContext.runAsUser=1000740001
#echo "install promtail.."
#helm install loki-promtail grafana/promtail
#install stack=FLG
#helm install loki-stack grafana/loki-stack \
# --values config.yaml -n my-project \
# --set grafana.securityContext.fsGroup=65533,grafana.securityContext.runAsUser=1000740001
echo "expose grafana"
oc delete route/loki-grafana
oc export svc/loki-grafana
echo "get admin pass.."
kubectl get secret --namespace $ns loki-grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
echo "port-forward if expose doesnot work"
#kubectl port-forward --namespace $ns service/loki-grafana 3000:80
#Statfulset Usage
How to use statefulset in database etc.
#PVC USage
How to use pvc in oc , we will try that for example jenkins
1.create a PVC without pv
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nginx-hq
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
2. Use PVC
apiVersion: v1
kind: Pod
metadata:
name: mypod1
spec:
containers:
- name: myfrontend
image: nginx
volumeMounts:
- mountPath: "/var/www/html"
name: mypd
volumes:
- name: mypd
persistentVolumeClaim:
claimName: nginx-hq
3. Create NFS pv
apiVersion: v1
kind: PersistentVolume
metadata:
name: mypv0011
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
nfs:
path: /Users/hongqizhang/data
server: 192.168.0.11
persistentVolumeReclaimPolicy: Recycle
#Openshift(oc) Installation
brew update
brew install openshift-cli
oc version
#config
docker->preference->daemon set 172.30.0.0/16 and restart
oc cluster up
#brawser
http://127.0.0.1:8443
1. create a dockerfile
FROM nginx:stable
# support running as arbitrary user which belogs to the root group
RUN chmod g+rwx /var/cache/nginx /var/run /var/log/nginx
# users are not allowed to listen on priviliged ports
RUN sed -i.bak -e 's/listen\( *\)80;/listen 8081;/' -e 's/listen\(.*\)80;//' /etc/nginx/conf.d/default.conf
EXPOSE 8081
# comment user directive as master process is run as user in OpenShift anyhow
RUN sed -i.bak 's/^user/#user/' /etc/nginx/nginx.conf
2. build and deploy
myapp=wavecloud/nginx-openshift
docker build -f Dockerfile -t $myapp .
docker login -uzhanghongqi -p
docker push $myapp
oc new-app $myapp --name nginx-oc -l name=dcnginx
oc expose svc $myapp --port=$myport #for http
#for https:
oc create route edge --service=$myapp --port=$myport \
--cert=wavecloud.crt \
--key=wavecloud.key \
--ca-cert=wavecloud.crt \
--hostname=nginx-myproject.127.0.0.1.nip.io
3.verification
myroute=$(oc get route --selector app=$myapp |awk 'FNR == 2 {print $2}')
echo $myroute
res="Welcome to nginx"
result=$(curl -k https://$myroute |grep "Welcome to nginx!")
4. export template and kubemodel
oc export dc,is,svc,route --as-template=nginx-templete -l name=dcnginx > mytemplate
oc export svc,route,is,dc -l name=dcnginx. >mykubemodel
5. clean all
oc delete all -l name=dcnginx
6. deploy with kubemode and template
oc create -f projectkube.yaml
or
oc process -f projecttemp.yaml | oc create -f -
7. verify again
oute=$(oc get route --selector app=$myapp |awk 'FNR == 2 {print $2}')
curl http://$route
set -x
set -x
echo "login and project"
oc login -u=hongqi.zhang -p=Passw0rd1
oc project hq-project;
IMAGE=mysecondtest
echo "clean old stuff"
oc delete all -l name=builddrup;
oc delete all -l name=deploydrup;
oc delete all -l name=buildsql;
oc delete all -l name=deploydrup;
echo "build mysql image"
rm -rf Dockerfile;
cp sqlDockerfile Dockerfile
oc new-build --strategy docker --binary --name testsql -l name=buildsql;
oc start-build testsql --from-dir . --follow;
echo "build mydrupal image"
rm -rf Dockerfile;
cp drupDockerfile Dockerfile
oc new-build --strategy docker --binary --name mydrupimg -l name=builddrup;
oc start-build mydrupimg --from-dir . --follow;
echo "deploy mysql and drupal"
oc new-app testsql -l name=deploysql \
-e MYSQL_ROOT_PASSWORD=pass123456 \
-e MYSQL_USER=drupaluser \
-e MYSQL_PASSWORD=pass123456 \
-e MYSQL_DATABASE=drupaldb
oc new-app mydrupimg -l name=deploydrup;
oc expose svc/testsql;
oc expose svc/mydrupimg;
echo "verification and test";
myroute=$(oc get route |grep secontest|awk '{print $2}')
curl http://${myroute}/drupal