Deploying Kubernetes Cluster With Ingress and Load Balancer on DigitalOcean

This article will give you the a simple way how to deploy a kubernetes cluster and it’s components on DigitalOcean Managed Kubernetes (DOKS).

Along with my learning journey with kubernetes, I started to get my hands on trying kubernetes on DigitalOcean. It’s actually one of my favorite hosting platform that also offers Kubernetes managed service (DOKS). I have some of my small projects running on k8s on DO, since it’s very easy to deploy and I can run completely manged k8s cluster with only 15 dollars per month!

Getting started

So now I’m going to deploy a new kubernetes cluster and try to run a simple service along with it. Not only that, I want my service to be internal and to have the ingress controller and a Load balancer in front of the cluster to serve the traffic.

Let’s take a look at the picture below, if you familiar with k8s, this diagram must be quite straightforward. It starts with Load balancer and with ingress controller, the traffic goes through the internal service and eventually ended up to the pods.

Image by Devopsid.com
Image by Devopsid.com

Prerequisites

There are things you want to prepare before deploying kubernetes cluster on DO :

Implementation

If everything set, let’s continue with the implementation. So as I mentioned before, deploying managed kubernetes cluster on DigitalOcean is quite straightforward.

Deploy cluster

First step, let’s deploy a new cluster on DOKS. We can do that simply by using doctl command line.

$ doctl kubernetes cluster create my-cluster --node-pool "name=my-cluster-node;size=s-1vcpu-2gb;count=1" --region sgp1

The command execution will result to create a new kubernetes cluster along with the node pool with only one node and minimum spec. You might want to change the parameters depends on your preference and also to change the region. In this testing, I’m deploying to Singapore region (sgp1).

and we just need to wait until it finished. The output will look like this:

Notice: Cluster is provisioning, waiting for cluster to be running
..................................................................
Notice: Cluster created, fetching credentials
Notice: Adding cluster credentials to kubeconfig file found in "/home/.kube/config"
Notice: Setting current-context to do-sgp1-my-cluster
ID Name Region Version Auto Upgrade Status Node Pools
d570cdaa-c985-495c-b6e7-d005aa1ef5dd my-cluster sgp1 1.20.2-do.0 false running my-cluster-node

Once the provisioning finished, check the current-context to make sure we’re on the exact cluster.

$ kubectl config current-context
do-sgp1-my-cluster

If everything looks good, then we have a new k8s cluster provisioned. Let’s continue to setup other stuff.

Deploy the internal service

Once we have deployed the cluster, if we take a look at the console (https://cloud.digitalocean.com/kubernetes/clusters), it will showing my new cluster :

And let’s create a YAML file to define our internal service.

# service.yml
apiVersion: v1
kind: Service
metadata:
  name: test-backend
spec:
  type: ClusterIP
  selector:
    app: test-app
  ports:
    - port: 80
      targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: test-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: test-app
  template:
    metadata:
      labels:
        app: test-app
    spec:
      containers:
      - image: tutum/hello-world:latest
        name: test-app
        ports:
        - containerPort: 80
          protocol: TCP

Apply the YAML file :

$ kubectl apply -f service.yml
service/test-backend created
deployment.apps/test-app created

Check the service and if the pods are already running:

$ kubectl get service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.245.0.1 <none> 443/TCP 129m
test-backend ClusterIP 10.245.93.249 <none> 80/TCP 28m
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
test-app-65f85568c4-4t685 1/1 Running 0 29m
test-app-65f85568c4-6q49w 1/1 Running 0 29m

It looks like everything set and we already deployed our first internal service on kubernetes DigitalOcean. Let’s move on!

Deploy the ingress controller and Load Balancer

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/do/deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

Note that after applied the ingress controller, it also deployed a new load balancer automatically:

And let’s create a new YAML file for the ingress definition:

# ingress.yml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: app1.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: test-backend
          servicePort: 80

As usual, apply the YAML file :

$ kubectl apply -f ingress.yml
ingress.networking.k8s.io/my-ingress created

View the ingress :

$ kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
my-ingress <none> app1.devopsid.com 139.59.195.196 80 5m34s

As you can see, after we have deployed the ingress, we will get the external IP address which is came from the external Load Balancer DigitalOcean. And as I mentioned before, I have created a new record on my DNS provider, and point the record app1.devopsid.com to the IP address of Load Balancer. so I can access the app via app1.devopsid.com.

Finally, let’s check our new app on browser. If it’s working fine, we should be able to see this web page :

If you got any questions, let me know in the comments!

Setup kubernetes cluster in Ubuntu 20.04 from scratch

Hello again, this article is a walk through how to setup your own kubernetes cluster with Ubuntu 20.04 LTS. Some steps are very straightforward, and you can directly follow along while you try to setup yourself.

So before get started, I tried this using 2 ubuntu servers :

    • ks8-master : 2gb memory, 2vCPUs
    • k8s-node-0 : 1gb memory, 1vCPU

I believe this is the cheapest kubernetes cluster specs that you can get. The purpose of this is only to try to init the cluster from the get-go and do the simple deployment. So here it goes :

Install the docker and disable the swap on all k8s nodes :

$ sudo apt update
$ sudo apt install -y docker.io
$ sudo systemctl start docker
$ sudo systemctl enable docker
$ sudo sed -i '/ swap / s/^\(.*\)$/#/g' /etc/fstab
$ sudo swapoff -a

Enable the port forwarding on all k8s nodes:

To enable the ip forwarding permanently, edit the file “/etc/sysctl.conf” and look for line “net.ipv4.ip_forward=1″ and un-comment it. After making the changes in the file, execute the following command :

$ sudo sysctl -p
net.ipv4.ip_forward = 1

Install k8s packages on all k8s nodes :

Execute the following command on all nodes :

$ sudo apt install -y apt-transport-https curl
$ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
$ sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
$ sudo apt update
$ sudo apt install -y kubelet kubeadm kubectl

Init the cluster on k8s master :

On k8s master, now let’s init the cluster :

$ kubeadm init

This command will give you the output something like this :

Error when running kubectl

After ini the cluster, I encountered error that prevent me to run kubectl command :

The connection to the server localhost:8080 was refused – did you specify the right host or port?

If you also face the same issue, the solution is simply to run this command :

$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install network plugin on k8s master :

In this tutorial, I use Calico (https://www.projectcalico.org/)

$ kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml

Enable bash completion for kubectl commands

This following command is optional, but recommended. It is to enable the bash completion, when you executing kubectl sub commands. Do this on k8s master :

$ echo 'source <(kubectl completion bash)' >>~/.bashrc
$ source .bashrc

Enable nginx ingress

Enable ingress with nginx on k8s master :

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/deploy.yaml

How to join the k8s node to k8s master :

Once the k8s master is ready, then we need to connect the k8s node to the master. We can simply do that by SSH to the k8s node, and execute the join command that we got after cluster creation completed.

$ kubeadm join 178.128.103.123:6443 --token htsn3w.juidt9j3t4zbgu3t --discovery-token-ca-cert-hash sha256:ea2e5654fb6e8bc31be463f60177f3b5d31b1da5019a20fd7a2336435b970a77

Check on the k8s master whether the nodes are ready :

$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master Ready control-plane,master 24h v1.20.1
k8s-node-0 Ready <none> 24h v1.20.1

if you get to see the nodes ready and we’re set. Now we can continue with the deployment.

Deploy nginx on k8s cluster

Now, we come to the fun stuff.  After cluster is ready, and let’s deploy something on it. Let’s create deployment for nginx, the easy one.

From k8s master, save this file below as nginx-deployment.yml (or whatever you can call it).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      run: nginx-deployment
  template:
    metadata:
      labels:
        run: nginx-deployment
    spec:
      containers:
      - image: nginx
        name: nginx-webserver
        ports:
        - containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: NodePort
  selector:
    run: nginx-deployment
  ports:
    - port: 80

Then create deployment from this file :

$ kubectl create -f nginx-deployment.yml
deployment.apps/nginx-deployment created
service/nginx-service created

Check the deployment, whether it has succeed :

$ kubectl get deployments
NAME             READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 1/1   1          1         110s

Now, you see the nginx deployment has started the replica, and it’s now running fine.

Next, you can check whether the service has deployed :

$ kubectl get services
NAME          TYPE      CLUSTER-IP    EXTERNAL-IP PORT(S)      AGE
kubernetes    ClusterIP 10.96.0.1     <none>      443/TCP      34h
nginx-service NodePort  10.111.139.39 <none>      80:30992/TCP 5m27s

We can see the nginx service is already in place, and since the deployment already succeed, let’s also check whether nginx is really running by testing the cluster IP. So we can do something like :

$ curl 10.111.139.39

and the output is :

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href=”http://nginx.org/”>nginx.org</a>.<br/>
Commercial support is available at
<a href=”http://nginx.com/”>nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Yes, the nginx now running successfully!

Increase replica

Next, lets try to increase the replica of the existing deployment. We want to increase the replica from 1 to 4. By doing that, we just need to update the yml file we just deployed with.

$ vim nginx-deployment.yml

I set the font to bold to indicates that line that I altered in the file. Change the number with the desired number.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 4
  selector:
    matchLabels:
      run: nginx-deployment
  template:
    metadata:
      labels:
        run: nginx-deployment
    spec:
      containers:
      - image: nginx
        name: nginx-webserver
        ports:
        - containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: NodePort
  selector:
    run: nginx-deployment
  ports:
    - port: 80

Save the file again, and run the command to update the deployment :

$ kubectl apply -f nginx-deployment.yml
deployment.apps/nginx-deployment unchanged
service/nginx-service unchanged

And also check whether the number of replicas have increased :

$ kubectl get deployments nginx-deployment
NAME             READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 4/4   4          4         14h

so if the number of replicas already equal with desired count, then we have successfully scaled up the service.

Another related k8s articles :