Troubleshoot Kubernetes service not being resolved

2 min read | by Jordi Prats

When you create a Kubernetes Service, pods from within the same namespace should be able to resolve it's IP by name. For example, if we create a service named ampa-votacions; any pod from the same namespace should be able to resolve it's IP. But sometimes it can't be resolved:

$ kubectl exec -it ampa-install-ws7cw -- sh / $ nslookup ampa-votacions Server: 172.20.0.10 Address: 172.20.0.10:53 ** server can't find ampa-votacions.us-west-2.compute.internal: NXDOMAIN ** server can't find ampa-votacions.ampa.svc.cluster.local: NXDOMAIN ** server can't find ampa-votacions.svc.cluster.local: NXDOMAIN ** server can't find ampa-votacions.ampa.svc.cluster.local: NXDOMAIN ** server can't find ampa-votacions.cluster.local: NXDOMAIN ** server can't find ampa-votacions.svc.cluster.local: NXDOMAIN ** server can't find ampa-votacions.cluster.local: NXDOMAIN ** server can't find ampa-votacions.us-west-2.compute.internal: NXDOMAIN

One common issue is that there are no endpoints available, we can check it using kubectl describe:

$ kubectl describe svc ampa-votacions Name: ampa-votacions Namespace: ampa Labels: app.kubernetes.io/instance=ampa  app.kubernetes.io/managed-by=Helm  app.kubernetes.io/name=ampa  app.kubernetes.io/version=1.19.4  component=votacions  helm.sh/chart=ampa-0.1.0 Annotations: meta.helm.sh/release-name: ampa  meta.helm.sh/release-namespace: ampa Selector: app=ampa,component=votacions Type: ClusterIP IP Families: <none> IP: None IPs: <none> Port: votacions 8080/TCP TargetPort: 8080/TCP Endpoints: <none> Session Affinity: None Events: <none>

We'll need to check:

  • What's going on with the pods / deployments what are supposed to be behind this service
  • Check that the pods / deployments are exposing the service
  • Check that the selectorLabels are matching the correct pods

Once we find and resolve the issue, we can check again the service using kubectl describe svc. If the issues is really fixed, we will be able to see the presence of endpoints:

$ kubectl describe svc ampa-votacions Name: ampa-votacions Namespace: ampa Labels: app.kubernetes.io/instance=ampa  app.kubernetes.io/managed-by=Helm  app.kubernetes.io/name=ampa  app.kubernetes.io/version=1.19.14  component=votacions  helm.sh/chart=ampa-0.1.0 Annotations: meta.helm.sh/release-name: ampa  meta.helm.sh/release-namespace: ampa Selector: app=ampa,component=votacions Type: ClusterIP IP Families: <none> IP: None IPs: <none> Port: votacions 8080/TCP TargetPort: 8080/TCP Endpoints: 10.103.196.193:8080 Session Affinity: None Events: <none>

Once we have endpoints available, we will be able to start resolving the service from the pods

Posted on 19/04/2021

Categories