Traceback (most recent call last): File "test.py", line 11, in <module> import requests ImportError: No module named requests
https://pypi.org/project/requests/
https://stackoverflow.com/questions/18363022/importerror-no-module-named-pip
Security Researcher, DevOps, SRE
Traceback (most recent call last): File "test.py", line 11, in <module> import requests ImportError: No module named requests
https://pypi.org/project/requests/
https://stackoverflow.com/questions/18363022/importerror-no-module-named-pip
$ fdisk -l $ growpart /dev/xvda $ growpart /dev/xvda 1 $ lsblk $ xfs_growfs -d / $ df -h
Scrot is a quite popular command line tool and one of the swiss army tools sysadmin need to have. When I’m not be able to install it via the repository, this is what I did to make it work :
$ wget http://mirror.ghettoforge.org/distributions/gf/el/7/gf/x86_64/giblib-1.2.4-27.gf.el7.x86_64.rpm $ wget http://packages.psychotic.ninja/7/base/x86_64/RPMS/scrot-0.8-12.el7.psychotic.x86_64.rpm $ sudo rpm -i giblib-1.2.4-27.gf.el7.x86_64.rpm $ sudo rpm -i scrot-0.8-12.el7.psychotic.x86_64.rpm
This article will give you the a simple way how to deploy a kubernetes cluster and it’s components on DigitalOcean Managed Kubernetes (DOKS).
Along with my learning journey with kubernetes, I started to get my hands on trying kubernetes on DigitalOcean. It’s actually one of my favorite hosting platform that also offers Kubernetes managed service (DOKS). I have some of my small projects running on k8s on DO, since it’s very easy to deploy and I can run completely manged k8s cluster with only 15 dollars per month!
So now I’m going to deploy a new kubernetes cluster and try to run a simple service along with it. Not only that, I want my service to be internal and to have the ingress controller and a Load balancer in front of the cluster to serve the traffic.
Let’s take a look at the picture below, if you familiar with k8s, this diagram must be quite straightforward. It starts with Load balancer and with ingress controller, the traffic goes through the internal service and eventually ended up to the pods.
There are things you want to prepare before deploying kubernetes cluster on DO :
If everything set, let’s continue with the implementation. So as I mentioned before, deploying managed kubernetes cluster on DigitalOcean is quite straightforward.
First step, let’s deploy a new cluster on DOKS. We can do that simply by using doctl command line.
$ doctl kubernetes cluster create my-cluster --node-pool "name=my-cluster-node;size=s-1vcpu-2gb;count=1" --region sgp1
The command execution will result to create a new kubernetes cluster along with the node pool with only one node and minimum spec. You might want to change the parameters depends on your preference and also to change the region. In this testing, I’m deploying to Singapore region (sgp1).
and we just need to wait until it finished. The output will look like this:
Notice: Cluster is provisioning, waiting for cluster to be running .................................................................. Notice: Cluster created, fetching credentials Notice: Adding cluster credentials to kubeconfig file found in "/home/.kube/config" Notice: Setting current-context to do-sgp1-my-cluster ID Name Region Version Auto Upgrade Status Node Pools d570cdaa-c985-495c-b6e7-d005aa1ef5dd my-cluster sgp1 1.20.2-do.0 false running my-cluster-node
Once the provisioning finished, check the current-context to make sure we’re on the exact cluster.
$ kubectl config current-context do-sgp1-my-cluster
If everything looks good, then we have a new k8s cluster provisioned. Let’s continue to setup other stuff.
Once we have deployed the cluster, if we take a look at the console (https://cloud.digitalocean.com/kubernetes/clusters), it will showing my new cluster :
And let’s create a YAML file to define our internal service.
# service.yml apiVersion: v1 kind: Service metadata: name: test-backend spec: type: ClusterIP selector: app: test-app ports: - port: 80 targetPort: 80 --- apiVersion: apps/v1 kind: Deployment metadata: name: test-app spec: replicas: 2 selector: matchLabels: app: test-app template: metadata: labels: app: test-app spec: containers: - image: tutum/hello-world:latest name: test-app ports: - containerPort: 80 protocol: TCP
Apply the YAML file :
$ kubectl apply -f service.yml service/test-backend created deployment.apps/test-app created
Check the service and if the pods are already running:
$ kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 129m test-backend ClusterIP 10.245.93.249 <none> 80/TCP 28m $ kubectl get pods NAME READY STATUS RESTARTS AGE test-app-65f85568c4-4t685 1/1 Running 0 29m test-app-65f85568c4-6q49w 1/1 Running 0 29m
It looks like everything set and we already deployed our first internal service on kubernetes DigitalOcean. Let’s move on!
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/do/deploy.yaml namespace/ingress-nginx created serviceaccount/ingress-nginx created configmap/ingress-nginx-controller created clusterrole.rbac.authorization.k8s.io/ingress-nginx created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created role.rbac.authorization.k8s.io/ingress-nginx created rolebinding.rbac.authorization.k8s.io/ingress-nginx created service/ingress-nginx-controller-admission created service/ingress-nginx-controller created deployment.apps/ingress-nginx-controller created validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created serviceaccount/ingress-nginx-admission created clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created role.rbac.authorization.k8s.io/ingress-nginx-admission created rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created job.batch/ingress-nginx-admission-create created job.batch/ingress-nginx-admission-patch created
Note that after applied the ingress controller, it also deployed a new load balancer automatically:
And let’s create a new YAML file for the ingress definition:
# ingress.yml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: my-ingress annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: app1.example.com http: paths: - path: / backend: serviceName: test-backend servicePort: 80
As usual, apply the YAML file :
$ kubectl apply -f ingress.yml ingress.networking.k8s.io/my-ingress created
View the ingress :
$ kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE my-ingress <none> app1.devopsid.com 139.59.195.196 80 5m34s
As you can see, after we have deployed the ingress, we will get the external IP address which is came from the external Load Balancer DigitalOcean. And as I mentioned before, I have created a new record on my DNS provider, and point the record app1.devopsid.com to the IP address of Load Balancer. so I can access the app via app1.devopsid.com.
Finally, let’s check our new app on browser. If it’s working fine, we should be able to see this web page :
If you got any questions, let me know in the comments!
In this lab, I use Ubuntu 20.04 LTS for my k8s cluster.
To remove the k8s node from the cluster, first drain the node :
$ kubectl get nodes $ kubectl drain node
If required, you also need to ignore daemonsets :
$ kubectl drain node --ignore-daemonsets
And delete the node :
$ kubectl delete node <node-name>
If no errors, proceed to remove or terminate the instance/server for the k8s node.
Hello again, this article is a walk through how to setup your own kubernetes cluster with Ubuntu 20.04 LTS. Some steps are very straightforward, and you can directly follow along while you try to setup yourself.
So before get started, I tried this using 2 ubuntu servers :
I believe this is the cheapest kubernetes cluster specs that you can get. The purpose of this is only to try to init the cluster from the get-go and do the simple deployment. So here it goes :
$ sudo apt update $ sudo apt install -y docker.io $ sudo systemctl start docker $ sudo systemctl enable docker $ sudo sed -i '/ swap / s/^\(.*\)$/#/g' /etc/fstab $ sudo swapoff -a
To enable the ip forwarding permanently, edit the file “/etc/sysctl.conf” and look for line “net.ipv4.ip_forward=1″ and un-comment it. After making the changes in the file, execute the following command :
$ sudo sysctl -p net.ipv4.ip_forward = 1
Execute the following command on all nodes :
$ sudo apt install -y apt-transport-https curl $ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add $ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main" $ sudo apt update $ sudo apt install -y kubelet kubeadm kubectl
On k8s master, now let’s init the cluster :
$ kubeadm init
This command will give you the output something like this :
After ini the cluster, I encountered error that prevent me to run kubectl command :
The connection to the server localhost:8080 was refused – did you specify the right host or port?
If you also face the same issue, the solution is simply to run this command :
$ mkdir -p $HOME/.kube $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config $ sudo chown $(id -u):$(id -g) $HOME/.kube/config
In this tutorial, I use Calico (https://www.projectcalico.org/)
$ kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml
This following command is optional, but recommended. It is to enable the bash completion, when you executing kubectl sub commands. Do this on k8s master :
$ echo 'source <(kubectl completion bash)' >>~/.bashrc $ source .bashrc
Enable ingress with nginx on k8s master :
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/deploy.yaml
Once the k8s master is ready, then we need to connect the k8s node to the master. We can simply do that by SSH to the k8s node, and execute the join command that we got after cluster creation completed.
$ kubeadm join 178.128.103.123:6443 --token htsn3w.juidt9j3t4zbgu3t --discovery-token-ca-cert-hash sha256:ea2e5654fb6e8bc31be463f60177f3b5d31b1da5019a20fd7a2336435b970a77
Check on the k8s master whether the nodes are ready :
$ kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master Ready control-plane,master 24h v1.20.1 k8s-node-0 Ready <none> 24h v1.20.1
if you get to see the nodes ready and we’re set. Now we can continue with the deployment.
Now, we come to the fun stuff. After cluster is ready, and let’s deploy something on it. Let’s create deployment for nginx, the easy one.
From k8s master, save this file below as nginx-deployment.yml (or whatever you can call it).
apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: replicas: 1 selector: matchLabels: run: nginx-deployment template: metadata: labels: run: nginx-deployment spec: containers: - image: nginx name: nginx-webserver ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: nginx-service spec: type: NodePort selector: run: nginx-deployment ports: - port: 80
Then create deployment from this file :
$ kubectl create -f nginx-deployment.yml deployment.apps/nginx-deployment created service/nginx-service created
Check the deployment, whether it has succeed :
$ kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE nginx-deployment 1/1 1 1 110s
Now, you see the nginx deployment has started the replica, and it’s now running fine.
Next, you can check whether the service has deployed :
$ kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 34h nginx-service NodePort 10.111.139.39 <none> 80:30992/TCP 5m27s
We can see the nginx service is already in place, and since the deployment already succeed, let’s also check whether nginx is really running by testing the cluster IP. So we can do something like :
$ curl 10.111.139.39
and the output is :
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href=”http://nginx.org/”>nginx.org</a>.<br/>
Commercial support is available at
<a href=”http://nginx.com/”>nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Yes, the nginx now running successfully!
Next, lets try to increase the replica of the existing deployment. We want to increase the replica from 1 to 4. By doing that, we just need to update the yml file we just deployed with.
$ vim nginx-deployment.yml
I set the font to bold to indicates that line that I altered in the file. Change the number with the desired number.
apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment spec: replicas: 4 selector: matchLabels: run: nginx-deployment template: metadata: labels: run: nginx-deployment spec: containers: - image: nginx name: nginx-webserver ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: nginx-service spec: type: NodePort selector: run: nginx-deployment ports: - port: 80
Save the file again, and run the command to update the deployment :
$ kubectl apply -f nginx-deployment.yml deployment.apps/nginx-deployment unchanged service/nginx-service unchanged
And also check whether the number of replicas have increased :
$ kubectl get deployments nginx-deployment NAME READY UP-TO-DATE AVAILABLE AGE nginx-deployment 4/4 4 4 14h
so if the number of replicas already equal with desired count, then we have successfully scaled up the service.
Another related k8s articles :
aws s3 ls --summarize --human-readable --recursive s3://bucket_name/go/to/path | grep 'Total'
Change bucket_name and go/to/path to yours
Check the remote file whether it exists
ssh -q ubuntu@10.10.10.10 [[ -f /etc/nginx/conf.d/test.conf ]] && echo "true" || echo "false"
aws codebuild create-webhook --project-name --filter-groups "[[{\"type\":\"EVENT\",\"pattern\":\"PULL_REQUEST_MERGED\"}]]"
Easy steps to install spotify with Snap on Fedora 28, Enjoy!
sudo dnf install snapd sudo ln -s /var/lib/snapd/snap /snap sudo snap install spotify sudo ln -s /var/lib/snapd/snap/bin/spotify /usr/bin/spotify