MetalLB , Loadbalancer for baremetal kubernetes cluster
--
1. Introduction
Kubernetes does not offer an implementation of network load-balancers for bare metal clusters. In that case, bare metal cluster operators are left with two solution: “NodePort” and “externalIPs” services.
Both of these options has significant limitation for production workload. Kubernetes does ship with all glue code that calls out to various cloud providers (AWS, Azure, GCP, Openstack etc). If we are not running on a supported cloudprovider, LoadBalancer services will remain in the “pending” state indefinitely. Considering this, kubernetes has come up with metallb to support for bare metal.
2. Without MetalLB / LoadBalancer?
Install kubernetes cluster or use existing bare metal kubernetes cluster. Use can use kubeadm installation link to setup cluster https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
Alternatively use the existing k8s cluster, if you already have.
Create a deployment and service with type “LoadBalancer”. Deployment below has three replica set for load balancing the service requests.
Create simple loadbalancer service
kubectl create –f simple_deployment.yaml
kubectl create –f simple_service.yaml
k8suser@machine1:~/MetalLb$ kubectl get svc -A | grep simple
default simple-service LoadBalancer 10.106.232.35 <pending> 80:30126/TCP
k8suser@machine1:~/MetalLb$ kubectl get pods -A | grep simple
default simple-deployment-9f5578997-4xz6m 1/1 Running 0 43m
default simple-deployment-9f5578997-55fm2 1/1 Running 0 43m
default simple-deployment-9f5578997-9rtl2 1/1 Running 0 43m
Since, No LoadBalancer is installed on kubernetes cluster, “simple-service” will be in pending state for getting external IP from LoadBalancer.
3. Installation for MetalLB
Executing the below links from yaml file or create file locally and copy content from link to create k8s deployments. The below yaml files are also available here https://github.com/ykumar-rb/POC/blob/master/MetalLB#L40
Metallb Installation steps
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
# On first install only
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
Above steps will create
- “metallb-system” namespace
- service accounts for controller and speaker
- RBAC for controller and speaker
- Speaker daemonset
- Controller deployment
MetalLB k8s resources
MetalLB remains idle until configured. This is accomplished by creating and deploying a configmap into the same namespace (metallb-system) as the deployment.
4. Configuration for MetalLB
After installing MetalLB, next step is to configure it. MetalLB’s configuration can be done with standard Kubernetes ConfigMap created under the metallb-system namespace.
It will have details like
- IP address pool
- Protocol to be used
Protocol can be layer2 or BGP. For our use-case, L2 configuration is sufficient.
Layer 2 mode does not require the IPs to be bound to the network interfaces of your worker nodes. It works by responding to ARP requests on your local network directly, to give the machine’s MAC address to clients.
For example, the following configuration gives MetalLB control over IPs from 192.168.1.240 to 192.168.1.250 and configures Layer 2 mode.
create L2 config for MetalLB
kubectl apply -f config.yaml
ARP at Layer-2 helps in advertising IPs on a LAN segment. In most cases, this is sufficient, but if there are use cases, where service IPs needs to be directly addressable from Internet, Layer-3 support will be needed. This is True with any cloud IPs. MetalLB does a neat job in using Layer-2 ARP protocol for responding to Service IPs (++). More details are available here https://metallb.universe.tf/configuration/
config.yaml
Note: If we edit config for address pool , then do delete metalLB manifest and create them again to pick new configmap. No need to delete LoadBalancer services, as services will be assigned new IP if address pool got updated.
5. LoadBalancer Services after MetalLB installed
We can see that LoadBalancer services now get an IP from pool configured for MetalLB after installation of MetalLB. The curl on assigned ip “192.168.1.240” will get proper HTTP response.
Execution Output
k8suser@kdss-wjhdl-0:~$ kubectl get svc -A | grep simple
default simple-service LoadBalancer 10.107.105.178 192.168.1.240 80:32483/TCP 7m26s
Ensure IP 192.168.1.240 / the address pool must be set as no_proxy and NO_PROXY environment.
The LoadBalancer services/ workloads are accessible with the assigned IP via LoadBalancer(MetalLB). The services are reachable over browser.
The LoadBalancer services/ workloads are accessible with the assigned IP via LoadBalancer(MetalLB). The service are reachable over browser http://192.168.1.240