Deploy NetScaler CPXs as Local DNS Caches for Kubernetes Nodes

Application pods in a Kubernetes cluster rely on DNS to communicate with other application pods. DNS requests from applications inside a Kubernetes cluster are handled by Kubernetes DNS (kube-dns). Due to wider adoption of microservices architectures, DNS request rates inside a Kubernetes cluster are increasing. As a result, Kubernetes DNS (kube-dns) is overburdened. Now you can deploy NetScaler CPX as a local DNS cache on each Kubernetes node and forward DNS requests from application pods in the node to NetScaler CPX. Hence, you can resolve DNS requests faster and significantly reduce the load on Kubernetes DNS.

To deploy NetScaler CPXs, a Kubernetes DaemonSet entity is used to schedule NetScaler CPX pods on each node in the Kubernetes cluster. A Kubernetes DaemonSet ensures that there is an instance of NetScaler CPX on each Kubernetes node in the cluster. To make application pods direct traffic to CPX DNS pods, you need to create a Kubernetes service with endpoints as NetScaler CPX pods. Cluster IP of this service is used as the DNS endpoint for the application pods. To make sure that the application pods use NetScaler CPX service cluster IP address for DNS resolution, you need to update the kubelet configuration file on each node with NetScaler CPX service cluster IP.

The following environment variables are introduced to support the deployment of NetScaler CPX as NodeLocal DNS cache:

• KUBE_DNS_SVC_IP: Specifies the cluster IP address of the kube-dns service which is a mandatory argument to trigger the configuration on a NetScaler CPX pod. The NetScaler CPX pod directs DNS queries to this IP address when the DNS query response is not available in the NetScaler CPX cache.

• CPX_DNS_SVC_IP: Specifies the cluster IP address of the NetScaler CPX service. The CPX_DNS_SVC_IP environment variable is used to configure local DNS on nodes. When you configure this variable, an iptables rule is added to direct the DNS requests originating from application pods to the local NetScaler CPX pod inside the node.

• NS_DNS_FORCE_TCP: This environment variable forces using TCP for DNS requests even if the queries are received over UDP.

• NS_DNS_EXT_RESLV_IP: Specifies the IP address of the external name server to direct the DNS requests for a specific domain.

• NS_DNS_MATCH_DOMAIN: Specifies the external domain string to be matched against to direct the queries to the external name server.

Deploy NetScaler CPXs as DNS Caches on Nodes

Deploying NetScaler CPX as local DNS cache for a Kubernetes cluster includes the following tasks:

On the master node:

• Create a Kubernetes service with endpoints as NetScaler CPX pods

• Create a ConfigMap for defining environment variables for NetScaler CPX pods

• Schedule NetScaler CPX pods on each node in the Kubernetes cluster using a Kubernetes DaemonSet.

On worker nodes:

• Modify the kubelet configuration file with the cluster IP address of NetScaler CPX service to forward DNS requests to NetScaler CPXs.

Configuration on the Kubernetes Master Node

Perform the following steps on the Kubernetes master node to deploy NetScaler CPX as the local DNS cache for nodes:

1. Create a service with NetScaler CPX pods as endpoints using the cpx_dns_svc.yaml file.

   kubectl apply -f cpx_dns_svc.yaml
   

   The cpx_dns_svc.yaml file is provided as follows:

           apiVersion: v1
           kind: Service
           metadata:
             name: cpx-dns-svc
             labels:
               app: cpxd
           spec:
             ports:
             - protocol: UDP
               port: 53
               name: dns
             - protocol: TCP
               port: 53
               name: dns-tcp
             selector:
               app: cpx-daemon
   

2. Get the IP address of the NetScaler CPX service.

   kubectl get svc cpx-dns-svc
   

3. Get the IP address of the Kube DNS service.

   kubectl get svc -n kube-system
   

4. Create a ConfigMap for defining environment variables for NetScaler CPX pods. These environment variables are used to pass IP addresses of NetScaler CPX service and Kube DNS service. In this step, a sample ConfigMap cpx-dns-cache is created using the environment variables specified as data (key-value pairs) in a file.

   kubectl create configmap cpx-dns-cache --from-file <path-to-file>
   

   The following is a sample file with the environment variables as key-value pairs.

     CPX_DNS_SVC_IP: 10.111.95.145
     EULA: "yes"
     KUBE_DNS_SVC_IP: 10.96.0.10
     NS_CPX_LITE: "1"
     NS_DNS_EXT_RESLV_IP: 10.102.217.142
     NS_DNS_MATCH_DOMAIN: citrix.com
     PLATFORM: CP1000
   

   The following is a sample ConfigMap:

   apiVersion: v1
   data:
     CPX_DNS_SVC_IP: 10.111.95.145
     EULA: "yes"
     KUBE_DNS_SVC_IP: 10.96.0.10
     NS_CPX_LITE: "1"
     NS_DNS_EXT_RESLV_IP: 10.102.217.142
     NS_DNS_MATCH_DOMAIN: citrix.com
     PLATFORM: CP1000
   kind: ConfigMap
   metadata:
     creationTimestamp: "2019-10-15T07:45:54Z"
     name: cpx-dns-cache
     namespace: default
     resourceVersion: "8026537"
     selfLink: /api/v1/namespaces/default/configmaps/cpx-dns-cache
     uid: 8d06f6ee-133b-4e1a-913c-9963cbf4f48
   

5. Create a Kubernetes DaemonSet for NetScaler CPX on the master node.

   kubectl apply -f cpx_daemonset.yaml
   

   The cpx_daemonset.yaml file is provided as follows:

   apiVersion: apps/v1
   kind: DaemonSet
   metadata:
     name: cpx-daemon
     labels:
           app: cpxd
   spec:
      selector:
           matchLabels:
             app: cpx-daemon
   template:
      metadata:
           labels:
             app: cpx-daemon
      spec:
          containers:
          - name: cpxd
           imagePullPolicy: IfNotPresent
           image: localhost:5000/dev/cpx
           volumeMounts:
            - mountPath: /netns/default/
           name: test-vol
           ports:
           - containerPort: 53
      envFrom:
          - configMapRef:
          name: cpx-dns-cache
      securityContext:
      privileged: true
      allowPrivilegeEscalation: true
      capabilities:
       add: ["NET_ADMIN"]
      volumes:
         -  name: test-vol
         hostPath:
         path: /proc/1/ns
         type: Directory
   

Configuration on Worker Nodes in the Kubernetes Cluster

Once you complete configuration on master node, perform the following step on worker nodes:

1. Modify the kubelet configuration file so that application pods can use NetScaler CPX service cluster IP for DNS resolution using one of the following steps:

   • Follow the steps in reconfigure a Node’s kubelet and modify the --cluster-dns argument value in the following format.

          --cluster-dns=<CPX_DNS_SVC_IP>,<KUBE_DNS_SVC_IP>
     

     or

   • Edit the /etc/systemd/system/kubelet.service.d/10-kubeadm.conf file and modify the --cluster-dns argument using the following steps.

     1. Edit the kubelet configuration and specify the cluster IP address of NetScaler CPX service and kube-dns service IP address for the --cluster-dns argument.

         root@node:~# cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf | grep KUBELET\_DNS\_ARGS
        
         Environment="KUBELET_DNS_ARGS=--cluster-dns=10.111.95.145,10.96.0.10 --cluster-domain=cluster.local"
         ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_DNS_ARGS
        

     2. Reload the kubelet of nodes using the following commands:

         # systemctl daemon-reload
         # service kubelet restart