Configure NodeLocal DNSCache
TOC
OverviewKey FeaturesImportant NotesInstallationInstall via MarketplaceHow It WorksArchitectureConfigurationDefault Health Check PortDNS Availability RiskOptional: Configure Multiple DNS ServersNetwork Policy ConfigurationOverview
NodeLocal DNSCache is a cluster plugin that improves cluster DNS performance by running a DNS caching proxy on cluster nodes. This plugin reduces DNS query latency and improves cluster stability by caching DNS responses locally on each node, minimizing the load on the central DNS service.
Key Features
- Local DNS Caching: Caches DNS responses locally on each node to reduce query latency
- Improved Performance: Significantly reduces DNS lookup times for applications
Important Notes
Production usage risk
NodeLocal DNSCache changes the DNS path of newly created Pods on each node. Before enabling it in a production cluster, evaluate the following risks:
- The
node-cachePod runs withhostNetwork: trueand exposes its health check endpoint on the node loopback127.0.0.1:8080. This can conflict with otherhostNetworkPods or node-host processes that also bind127.0.0.1:8080or0.0.0.0:8080. Regular Pods are not affected. - If the node-local DNS Pod on a node is unavailable, DNS resolution for Pods on that node can fail. The traffic does not automatically fail over to CoreDNS unless the Pod resolver is configured with an additional DNS server and the resolver implementation retries the next server.
Use this plugin in test or pre-production environments first. For production environments, confirm the port usage, monitoring, alerting, and emergency recovery plan before enabling the plugin.
Deployment Considerations:
-
Kube-OVN Underlay Mode: The plugin does not support deployment in Kube-OVN Underlay mode. If deployed, it may cause DNS query failures.
-
Kubelet Restart: Deploying this plugin will cause the kubelet to restart.
-
Pod Restart Required: After the plugin is successfully deployed, it will not affect running Pods, but will only take effect on newly created Pods. When the CNI is Kube-OVN, you need to manually add the parameter "--node-local-dns-ip=(IP address of the local DNS cache server)" to the kube-ovn-controller.
-
NetworkPolicy Configuration: If NetworkPolicy is configured in the cluster, you need to additionally allow both from and to directions for the node CIDR and nodeLocalDNSIP in the networkPolicy to ensure proper communication.
-
Cluster Upgrade via Rebuilding: If the cluster is upgraded by rebuilding nodes (re-provisioning), kubelet configuration changes will be lost. To make the NodeLocal DNS configuration persistent across upgrades, you need to add the
--cluster-dnsparameter tokubeletExtraArgsin the following three places of the cluster template:KubeadmControlPlane→initConfiguration→nodeRegistration→kubeletExtraArgsKubeadmControlPlane→joinConfiguration→nodeRegistration→kubeletExtraArgsKubeadmConfigTemplate→template→spec→joinConfiguration→nodeRegistration→kubeletExtraArgs
Add the following parameter to each of the above
kubeletExtraArgssections:
4.2.x Upgrade Notes
When upgrading this plugin from versions below 4.2.0 (excluding 4.2.0 itself) to 4.2.x, the following steps are required due to ResourcePatch compatibility issues:
Before Upgrade:
- Record the
--node-local-dns-ipparameter value from the kube-ovn-controller ResourcePatch configuration - Delete the ResourcePatch for the
deploy/kube-ovn-controllerresource
After Upgrade:
- Manually add the recorded
--node-local-dns-ipparameter back to the kube-ovn-controller configuration
Note: This compatibility issue has been resolved in version 4.3 and above, so manual intervention is not required for upgrades to 4.3+.
Installation
Install via Marketplace
-
Navigate to Administrator > Marketplace > Cluster Plugins.
-
Search for "Alauda Build of NodeLocal DNSCache" in the plugin list.
-
Click Install to open the installation configuration page.
-
Configure the required parameters:
-
Review the deployment notes and ensure your environment meets the requirements.
-
Click Install to complete the installation.
-
Wait for the plugin status to change to "Ready".
How It Works
Architecture
Configuration
Default Health Check Port
The node-cache Pod runs with hostNetwork: true and exposes its health check endpoint on the node loopback 127.0.0.1:8080. The current plugin package does not provide an installation parameter to change this health check port.
Because node-cache uses the node network namespace, the 127.0.0.1:8080 it occupies is the node's loopback, not a pod's. Conflict only happens with workloads that share the node network namespace or run directly on the node host:
- A
hostNetwork: truePod that binds127.0.0.1:8080or0.0.0.0:8080. - A process running directly on the node host that binds
127.0.0.1:8080or0.0.0.0:8080.
Regular Pods (each in its own network namespace) are not affected, even if they listen on 8080.
Before installing the plugin, check whether 127.0.0.1:8080 is already used on each node:
If the port is already used by a hostNetwork Pod or a node-host process, choose one of the following approaches:
- Move the conflicting workload to another port before installing NodeLocal DNSCache.
- Run the workload as a regular Pod (not
hostNetwork), or bind it to a specific node IP other than127.0.0.1or0.0.0.0. - If the port must be changed for NodeLocal DNSCache, contact Alauda support to confirm whether your target version provides a supported configuration method.
Avoid manually editing the generated ConfigMap or DaemonSet as a long-term solution. Manual changes can be overwritten when the plugin is upgraded, reinstalled, or reconciled by the platform.
DNS Availability Risk
After NodeLocal DNSCache takes effect, newly created Pods use the node-local DNS address as their DNS server. If the node-cache Pod on a node is unavailable, for example during a crash, eviction, image pull failure, or plugin upgrade, Pods on that node can fail to resolve DNS names.
To reduce the operational risk:
-
Monitor the NodeLocal DNSCache DaemonSet and alert when any
node-cachePod is not Ready. -
Plan a maintenance window for plugin upgrades because DNS resolution on a node can be affected while the node-local DNS Pod is unavailable.
-
Emergency recovery: If DNS resolution on a node fails because the
node-cachePod is unavailable, restore kubeletcluster-dnsto the CoreDNS ClusterIP and recreate the affected Pods so that their/etc/resolv.confis regenerated. The CoreDNS ClusterIP is theClusterIPof the DNS Service in thekube-systemnamespace. To find it:Look for the DNS Service (commonly named
kube-dnsorcoredns) and use itsClusterIP.
Optional: Configure Multiple DNS Servers
Kubelet supports multiple cluster-dns addresses. You can configure the CoreDNS ClusterIP as a secondary DNS server so that new Pods receive both DNS servers in /etc/resolv.conf.
Example:
In this example, 169.254.20.10 is the NodeLocal DNSCache IP address and 10.96.0.10 is the CoreDNS ClusterIP. Replace both values with the addresses used by your cluster. To find the CoreDNS ClusterIP, run kubectl get svc -n kube-system and use the ClusterIP of the DNS Service (commonly named kube-dns or coredns).
This configuration can reduce the impact of a node-local DNS failure, but it is not a no-impact failover mechanism:
- The failover behavior depends on the resolver implementation in the workload image. For example, the glibc resolver typically tries the next nameserver only after the current one times out, so DNS resolution can become slower during a failure. musl libc, which is used by Alpine Linux, can query multiple nameservers in parallel and may fail over faster.
- Resolver options such as
timeoutandattemptsalso affect the failure delay. - If NetworkPolicy is enabled, allow Pods to access both the NodeLocal DNSCache IP address and the CoreDNS ClusterIP on TCP and UDP port
53. - When the CNI is Kube-OVN, the
--node-local-dns-ipparameter onkube-ovn-controllerdoes not conflict with a multi-addresscluster-dnson kubelet.--node-local-dns-iplets Kube-OVN deliver traffic to the node-local DNS IP, while the secondary CoreDNS ClusterIP still reaches CoreDNS through the normal Service path.
Validation in an IPv4 cluster showed the following behavior when the node-local DNS process on the workload node was stopped:
- A Pod with only the NodeLocal DNSCache IP address in
/etc/resolv.conffailed to resolve DNS names. - A Pod with both the NodeLocal DNSCache IP address and the CoreDNS ClusterIP in
/etc/resolv.confcould resolve DNS names through CoreDNS. - With the default glibc resolver behavior, failover to CoreDNS can take several seconds because the resolver waits for the first nameserver to time out.
- With musl libc, such as in Alpine Linux images, DNS queries can be sent to multiple nameservers in parallel, so failover can be faster.
If your cluster is upgraded by rebuilding nodes, add the multi-address cluster-dns value to each kubeletExtraArgs location described in the Important Notes section.
Network Policy Configuration
Important: If your cluster has NetworkPolicy enabled, you must configure proper rules to allow DNS traffic to the NodeLocal DNSCache. Without these rules, pods may not be able to resolve DNS queries.
When using NetworkPolicy, ensure the following DNS traffic is allowed: