In this post we’ll walk through setting up a load-balanced service using MicroK8s Kubernetes. This post assumes you already have a working MicroK8s cluster, for more information on setting up a cluster please see my post here.
The high level flow looks like this:
MicroK8s does not include a load balancer by default, this would normally be implemented by the cloud service provider and integrated with load balancing hardware. To add support for load balancing, we can install the MetalLB add-on as a network load balancer solution. The add-on is installed by running the following command from any of the cluster nodes. The command does not need to be run on each node, just once from any node.
microk8s enable metallbNext enable the ingress add-on. This installs the NGINX ingress controller and provides an API for creating ingress rules.
microk8s enable ingressNext we need to create the external ingress service. This will specificy the ports and IPs which will handle external traffic balancing across replica set instances. Specifying an IP is not required, but is supported and can be helpful for setting up localized DNS solutions (more on this in a future post).
apiVersion: v1
kind: Service
metadata:
name: ingress
namespace: ingress
spec:
selector:
name: nginx-ingress-microk8s
type: LoadBalancer
# loadBalancerIP is optional. MetalLB will automatically allocate an IP
# from its pool if not specified. You can also specify one manually.
# loadBalancerIP: x.y.z.a
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80
We’ll keep things simple and use the helloworld docker image to create 2 pod replicas in our deployment specification. This produces an http server that serves up “It Works!” on port 80.
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld-deployment
labels:
app: helloworld
spec:
replicas: 2
selector:
matchLabels:
app: helloworld
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: helloworld
image: httpd:2.4
ports:
- containerPort: 80Next create the service, which sits between the pod instances and the ingress layer. This will allow traffic to be routed from the ingress layer to the chosen pod instance.
apiVersion: v1
kind: Service
metadata:
name: helloworld
spec:
type: NodePort
selector:
app: helloworld
ports:
- name: http
protocol: TCP
port: 80
targetPort: 80The last thing we need to do is setup the ingress definition for our application. Each load balancer service can support N number of application hosts. The host should reflect the name that will be accessed from the browser.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: helloworld-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: helloworld.local
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: helloworld
port:
number: 80Everything should be working at this point, we can verify with the following procedure. First locate the IP of your external ingress controller:
microk8s.kubectl get svc -n ingressFind the EXTERNAL-IP next to your ingres service, it should look something like this:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
service/ingress LoadBalancer 10.15.18.19 192.168.1.4 80:30574/TCPNow insert your external IP into the following curl command to verify the load balancer is routing traffic correctly.
curl --header "Host: helloworld.local" http://YOUR_IP_HERE/If it works you should see a curl output that looks like:
<html><body><h1>It works!</h1></body></html>
Hey Matt, thanks for this write up… I’ve done all this and in the end when I curl the url, it works from only one of my hosts in the cluster out of a four node cluster. One of the nodes says it works, the others all eventually get:
curl: (28) Failed to connect to 10.0.0.229 port 80 after 129506 ms: Connection timed out
also get this from my workstation on the same network.
I don’t believe there is anything special about the host it works from, it isn’t even the host that I configured everything from. I worked with kube in my last job, quite a while back, I am now retired and just doing this for fun as I want to do home automation and stick everything in containers and seems to make sense to run it under kube and microk8s is pretty awesome on raspberry pi cluster… so just wondering where I should start to look for problems? get pods and get svc returns the same on all 4 nodes… any ideas on what I need to stat poking at would be appreciated if you have the time, thanks again.
Hi Bill, Are you running a firewall on your host nodes? That might explain why it works from one by not the others.
Your story resonates with me! I got into Raspberry Pi to build some home automation and I have experience working with Kubernetes/Docker so I thought I’d give it a try. So far I’m pretty happy with the solution.
Hello Mat, I have made a microk8s cluster using raspberry pi 4 devices.
I have tried example shown but I get nginx error 404 not found. I have looked for solutions on the internet but I couldn’t figure it out. If you happen to have some tips I would be more than grateful to hear from you!
Hi Marko, are you getting this when running the example curl command? It could be due to a mismatch between the host value in the curl command and the host defined in the ingress rule.
Hello Mat, thank you for your time.
Unfortunately, I still haven’t managed to make service avaliable. When I do curl command with the host specified and external IP, I get 504 Gateway Timeout error code.
Also, when I try to curl helloworld NodePort service, using it’s cluster IP address, everything stops and I don’t get any message.
I have also tried accessing individual nodes from the outside of the cluster(since It’s nodeport service, this should give me helloworld message), using node local IP address and I get 503 service unavailable webpage.
Any help would be appreciated.
Thanks in advance,
Marko