Internal Loadbalancers with Application Gateway (AKS)


By default, the Loadbalancer Kubernetes service (in Azure) is set up as an external facing Loadbalancer with a Public IP that makes it publicly accessible, making it vulnerable to attacks or other exploits. (See Fig. 1.)

Fig. 1. External Load Balancer


To provide our application with higher security (Web Application Firewall, SSL, etc.) we need to manage requests to the Service with additional services like for e.g. the Application Gateway service. (See Fig. 2.)

Fig. 2. Internal LB and Application Gateway


Info: Services can support SSL themselves (i.e. we can configure Nginx application server to use certificates), though doing so with the Application Gateway will offload this task from the service.


Update as of 07 July 2019: A better solution now is using the controller provided by Azure, for more information check out the following GitHub repository for Application Gateway Kubernetes Ingress.

Internal LoadBalancer Configuration

For a load balancer to be only internally accessible we need to change the Kubernetes configuration to request an internal LoadBalancer, by applying the following annotation: "true". For the full configuration see Figure 3.

Fig. 3. Configuration for Internal LB


To deploy this service execute the command: kubectl create -f deployment-frontend-internal.yaml, which delegates to Kubernetes to request from Azure Resource Manager an Internal Loadbalancer, with a private IP for our service.

To get the IP we need to execute the command kubectl get svc --watch. The watch flag will keep you updated when Azure provides your service with an IP. We need this IP for forwarding requests from Application Gateway.

Configuring the Application Gateway

Get a 🍵 as this requires more steps:

  1. Adding a Subnet for our Application Gateway in the Virtual Network of the Kubernetes Cluster.
    1. Go to your Kubernetes resources and select the Virtual Network.
    2. Under Settings > Subnets > + Subnet 
    3. Fill in the information as desired. (Or as shown in the Fig. 3. Careful! If you select a Network Security Group you need to allow access to GW ports.)

      Fig. 4. Subnet Configuration
  2. Adding an Application Gateway:
    1. Fill all information in the first blade Basic.
    2. In the Settings blade (shown in Fig. 4.):
      1. Open Virtual Network.
      2. Select the Virtual Network of the Kubernetes cluster.
      3. Select the Subnet created in step 4.
    3. Complete the setup.

      Fig. 5. Steps for Subnet selection
  3. Configuring the Application Gateway:
    1. Navigate to Backend Pools:
      1. Select existing pool appGatewayBackendPool.
      2. Add target [Private IP of the Service]. Save
    2. Navigate to HTTP Settings, which defines the incoming port of the request. We use the existing Settings, as we want the direct call (port 80) to be forwarded to the front end service.
    3. Navigate to Listeners and define a Listener for port 80. As a visualization HTTP Setting defines a port for an incoming call and the Listener is the connection to the rule.
    4. Navigate to Rules. Ensure that your rule is connecting the Listener with the Backend pool(By default the rule is set up to use the default listener and the default backend pool.)

Conclusion: The defaults work just fine. But remember to do the steps when adding additional backend pool, http setting, listener and the rule that ties everything together.

Info: When creating an HTTP Setting and a listener usually it takes up to 5 minutes for the configuration to kick in. (You won’t be able to select the Settings you create)


Now our service is accessible only through the Application Gateway. Typing the IP of the Application Gateway on a browser will start this process:

  1. The HTTP Listener (port 80 by default) gets activated:
  2. The rule associated with this HTTP Listener will forward the call to:
  3. Backend pool associated with this rule, whose Target is the internal IP of the Loadbalancer.

Now we are left with Securing our Gateway with a certificate and with a Web Application Firewall. That will be in an upcoming article. 😉

If you enjoyed the article, please share and comment below!
  • Badal

    can you provide detailed steps on configuring the app gateway given a private IP address for internal load balancer? So in the backend pool there will be only 1 IP and that will be of internal load balancer. But this isn’t working for me so not sure what is missing in the configuration

    • Nico Mommaerts

      Doesn’t work for me either, I get a 502 from the Application Gateway. Did you manage to solve it?

      • Rinor Maloku

        Please verify that your Health Checks are passing, you can find those in the Application Gateway. If the Health checks fail the application will not forward calls to the backend.

        • Nico Mommaerts

          No the health checks are not passing but I don’t know why. There is not much to debug or verify. The subnet you added, was it a normal subnet or a gateway subnet? I tried with both btw. I also opened the nsg on the cluster, and added an extra rule to the internal load balancer ip.

          • Rinor Maloku

            You need to configure custom health probes. i.e. the App GW needs an endpoint to request to get a successful status code

          • Nico Mommaerts

            Shouldn’t the default probe be enough? I have a simple example pod running that listens on port 80 on /

          • Rinor Maloku

            If your application returns a status code from 200 to 399 then yes. But apparently it is not, it’s the only way how the AppGw deems a backend as unhealthy.

          • Nico Mommaerts

            It does return 200 though. Misconfigured nsg might also be the problem:
            You got it working with the default probe? The default probe tries to listen on, I wonder how that works when the backend endpoint is an IP address and not a VM.

          • Nico Mommaerts

            Ok got it working with the default probe. Turns out I took the ip address of the internal lb as endpoint, instead of the ip address of the services. I presumed they were the same..
            Thank you for the rubberducking 🙂

          • Rinor Maloku

            Great and thanks for the valuable addition, now we cover one more occasion that could go wrong!

      • Badal

        yes. The ping health endpoint should return some result for backend pool configuration to work correctly.

  • Chetan Sarma

    Hello, How can I use ingress for the same purpose? I am able to assign private ip to ingress controller but I am not sure how to assign dns to it. any help will be awesome.

    • Rinor Maloku

      Hey Chetan Sarma,
      If you make it private, you cannot assign a DNS, as it won’t be visible to public DNS Servers (besides if you set up an Internal Private Networking DNS)

      • Chetan Sarma

        What about azure dns zones? Are they only public?

        • Rinor Maloku

          Yes the DNS Zones are. If you want to have it public, why not use public IP for the ingress controller?

          • Chetan Sarma

            Okay, I don’t want then public, I am trying to setup internal to my vnet, so it is not accessible to internet. I can get the private ip assigned to ingress controller but I am not able to assign dns to it. I see azure private dns zone is in preview.

  • javy

    @rinor_maloku:disqus , A nice and to the point article. Thanks for this. I was wondering when would you be posting Securing the Gateway with a certificate and with a Web Application Firewall?

    • Rinor Maloku

      Hi @disqus_irJmGypwBX:disqus,

      I doubt that I will get a lot of time to tackle that article. But I can point out some directions of what we used to do:
      – To secure the Application Gateway with a Certificate I uploaded it in a KeyVault and used it from ARM Templates, in a infrastructure as code solution.
      – For having the WAF please check some ARM templates like this one:

  • Kanika Gambhir

    I am using ISTIO within AKS cluster in my current project. And this project has front end as Azure Application gateway. However, Istio uses Istio Ingress Controller as front end.

    I am looking for a way through which I can get traffic from App Gateway to ISTIO Ingress Controller using a particular dns name(internal dns) like routed to the ip address of istio ingress controller.This traffic should be secured using TLS.

    Any help would be appreciated!
    Thank you,

    • Hi Kanika,

      Sorry I just saw this, I would be very interested on how you solved it?

      Best, Rinor

  • Dina Helmy

    hi Rinor did you manage to write a post on how to secure gateway with a certificate and how can I interact with keyvault