Internal Loadbalancers with Application Gateway (AKS)


By default, the Loadbalancer Kubernetes service (in Azure) is set up as an external facing Loadbalancer with a Public IP that makes it publicly accessible, making it vulnerable to attacks or other exploits. (See Fig. 1.)

Fig. 1. External Load Balancer


To provide our application with higher security (Web Application Firewall, SSL, etc.) we need to manage requests to the Service with additional services like for e.g. the Application Gateway service. (See Fig. 2.)

Fig. 2. Internal LB and Application Gateway


Info: Services can support SSL themselves (i.e. we can configure Nginx application server to use certificates), though doing so with the Application Gateway will offload this task from the service.


Internal LoadBalancer Configuration

For a Loadbalancer to be only internally accessible we need to change the Kubernetes configuration to request an internal LoadBalancer, by applying the following annotation: "true". For the full configuration see Figure 3.

Fig. 3. Configuration for Internal LB


To deploy this service execute the command: kubectl create -f deployment-frontend-internal.yaml, which delegates to Kubernetes to request from Azure Resource Manager an Internal Loadbalancer, with a private IP for our service.

To get the IP we need to execute the command kubectl get svc --watch. The watch flag will keep you updated when Azure provides your service with an IP. We need this IP for forwarding requests from Application Gateway.

Configuring the Application Gateway

Get a 🍵 as this requires more steps:

  1. Adding a Subnet for our Application Gateway in the Virtual Network of the Kubernetes Cluster.
    1. Go to your Kubernetes resources and select the Virtual Network.
    2. Under Settings > Subnets > + Subnet 
    3. Fill in the information as desired. (Or as shown in the Fig. 3. Careful! If you select a Network Security Group you need to allow access to GW ports.)

      Fig. 4. Subnet Configuration
  2. Adding an Application Gateway:
    1. Fill all information in the first blade Basic.
    2. In the Settings blade (shown in Fig. 4.):
      1. Open Virtual Network.
      2. Select the Virtual Network of the Kubernetes cluster.
      3. Select the Subnet created in step 4.
    3. Complete the setup.

      Fig. 5. Steps for Subnet selection
  3. Configuring the Application Gateway:
    1. Navigate to Backend Pools:
      1. Select existing pool appGatewayBackendPool.
      2. Add target [Private IP of the Service]. Save
    2. Navigate to HTTP Settings, which defines the incoming port of the request. We use the existing Settings, as we want the direct call (port 80) to be forwarded to the front end service.
    3. Navigate to Listeners and define a Listener for port 80. As a visualization HTTP Setting defines a port for an incoming call and the Listener is the connection to the rule.
    4. Navigate to Rules. Ensure that your rule is connecting the Listener with the Backend pool(By default the rule is set up to use the default listener and the default backend pool.)

Conclusion: The defaults work just fine. But remember to do the steps when adding additional backend pool, http setting, listener and the rule that ties everything together.

Info: When creating an HTTP Setting and a listener usually it takes up to 5 minutes for the configuration to kick in. (You won’t be able to select the Settings you create)


Now our service is accessible only through the Application Gateway. Typing the IP of the Application Gateway on a browser will start this process:

  1. The HTTP Listener (port 80 by default) gets activated:
  2. The rule associated with this HTTP Listener will forward the call to:
  3. Backend pool associated with this rule, whose Target is the internal IP of the Loadbalancer.

Now we are left with Securing our Gateway with a certificate and with a Web Application Firewall. That will be in an upcoming article. 😉

If you enjoyed the article, please share and comment below!
  • Badal

    can you provide detailed steps on configuring the app gateway given a private IP address for internal load balancer? So in the backend pool there will be only 1 IP and that will be of internal load balancer. But this isn’t working for me so not sure what is missing in the configuration

    • Nico Mommaerts

      Doesn’t work for me either, I get a 502 from the Application Gateway. Did you manage to solve it?

      • Rinor Maloku

        Please verify that your Health Checks are passing, you can find those in the Application Gateway. If the Health checks fail the application will not forward calls to the backend.

        • Nico Mommaerts

          No the health checks are not passing but I don’t know why. There is not much to debug or verify. The subnet you added, was it a normal subnet or a gateway subnet? I tried with both btw. I also opened the nsg on the cluster, and added an extra rule to the internal load balancer ip.

          • Rinor Maloku

            You need to configure custom health probes. i.e. the App GW needs an endpoint to request to get a successful status code

          • Nico Mommaerts

            Shouldn’t the default probe be enough? I have a simple example pod running that listens on port 80 on /

          • Rinor Maloku

            If your application returns a status code from 200 to 399 then yes. But apparently it is not, it’s the only way how the AppGw deems a backend as unhealthy.

          • Nico Mommaerts

            It does return 200 though. Misconfigured nsg might also be the problem:
            You got it working with the default probe? The default probe tries to listen on, I wonder how that works when the backend endpoint is an IP address and not a VM.

          • Nico Mommaerts

            Ok got it working with the default probe. Turns out I took the ip address of the internal lb as endpoint, instead of the ip address of the services. I presumed they were the same..
            Thank you for the rubberducking 🙂

          • Rinor Maloku

            Great and thanks for the valuable addition, now we cover one more occasion that could go wrong!