istio, gateways, and ingress gateways

I've been using Istio ¹ a fair bit recently and have started hanging out in the community Slack. I don't really feel like I understand something until I can explain it to someone else, so as time allows I've been trying to answer some of the questions I find.

One thing I've seen come up several times is folks asking about best practices for Istio Gateways. Specifically, should you have one Gateway or many? What about Ingress Gateways?

In keeping with tradition, a standard issue "it depends" is the correct answer to these questions, but if you just need a quick suggestion to get you started here you go.

• Gateways: you should probably have as few as you can get away with, but also it probably doesn't matter.
• Ingress Gateways: you should probably have one.

Ingress Gateways

An Istio Ingress Gateway is a Kubernetes Deployment that consists of just an Envoy proxy. Unlike the other proxies in the mesh which are bolted on to your workloads in sidecar fashion, the Ingress Gateway proxies sit on their own at the edge of the mesh accepting outside traffic and directing it inwards, like a load balancer in your cluster.

It may seem at first that having multiple Ingress Gateways can be a strategy for high availability, but a single Ingress Gateway Deployment can be as highly available as any other Deployment by way of multiple replicas across multiple zones and by scaling in response to traffic by way of its Horizontal Pod Autoscaler.

If your traffic scale is truly large or your isolation needs are extreme, then like with traditional load balancers those could be reasons to have more than one Ingress Gateway, but if you can't think of any specific reasons right now then start with one.

Gateways

Unlike an Ingress Gateway an Istio Gateway does not map to a Kubernetes Deployment. It is purely an Istio configuration object which Istio consumes and uses to program the lower level Envoy config on your behalf.

Specifically, a Gateway will inform the Envoy Listener configuration. But unlike Istio Virtual Services which also control Envoy Listeners throughout the mesh, a Gateway object will only apply to those Envoy Listeners on the Ingress Gateway Deployment.

Ugh, this stuff is terminology word salad.

Basically you can think of an Istio Gateway object kind of like an nginx server block with fewer options. It tells the Ingress Gateway Deployment:

• which port(s) to listen on
• which protocol(s) to listen for
• which hostname(s) to handle requests for
• which TLS certificate(s) to use

Where I once configured this blog with an nginx server block like this.

server {  
  listen 443 ssl;
  server_name rob.salmond.ca;

  ssl_certificate /etc/letsencrypt/live/rob.salmond.ca/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/rob.salmond.ca/privkey.key;

  location / {
    proxy_pass http://localhost:2370;
    proxy_redirect off;
  }
}

I now use an Istio Gateway config object like this.

apiVersion: networking.istio.io/v1beta1  
kind: Gateway  
metadata:  
  name: my-gateway
spec:  
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - 'rob.salmond.ca'
    port:
      name: http
      number: 80
      protocol: HTTP
  - hosts:
    - 'rob.salmond.ca'
    port:
      name: https-default
      number: 443
      protocol: HTTPS
    tls:
      credentialName: ingress-cert
      mode: SIMPLE

Note that routing traffic to the NodeJS app (localhost:2370) is not handled here. Istio uses Virtual Services to configure the matching of requests to their corresponding apps. Also rather than a full path to a file the TLS certificate is loaded from a Kubernetes Secret called ingress-cert.

Great but how many Gateways do I need?

If you're just serving traffic for a single domain, or even a wildcard domain, you only need one Gateway. By separating the routing configuration into Virtual Services Istio just needs your Gateway to indicate that it is hosting *.sweetdomain.com while you use Virtual Services to route traffic for www.sweetdomain.com, api.sweetdomain.com, signup.sweetdomain.com, etc.

If you're serving traffic for a lot of different domains you can either pack them all onto a single Gateway object by appending more hosts entries onto the servers list, or create a separate (and more succinct) Gateway object for each and every domain. Which way you go will be a combination of taste and the limitations of whatever god awful YAML templating tool you find yourself chained to.

Why is it a matter of taste? (or why does it "probably not matter"?)

Because with both of these approaches, under the hood Istio is going to turn these configurations into Envoy Listeners for you, and there will only be one Listener for each unique host:port pair. For your average web app this means that no matter how many Gateways you create, or hosts you add to one Gateway, you will end up with just one 0.0.0.0:8080 Listener and one 0.0.0.0:8443 Listener (these port numbers in Istio are += 8000 to avoid running the proxy as root).

For every host you add Istio will append its configuration ² to the corresponding Listener. When requests come in on that Listener, Envoy will either look at the Host or Authority headers or use SNI inspection to figure out which domain the connection is destined for then pass it off to the corresponding Envoy Route (where Istio has sent your Virtual Service configurations to handle request matching) for forwarding to the correct place.

I suppose that regardless of which approach you use to add all your different domains you could wind up with a Listener config so large it blows Envoy out in some interesting way, but if you get to that point then I'm afraid you will need more insight than this blog post can offer.