In the fifth and final part of this series, we will look at exposing Apache Kafka in Strimzi using Kubernetes Ingress. This article will explain how to use Ingress controllers on Kubernetes, how Ingress compares with Red Hat OpenShift routes, and how it can be used with Strimzi and Kafka. Off-cluster access using Kubernetes Ingress is available only from Strimzi 0.12.0. (Links to previous articles in the series can be found at the end.)
Note: Productized and supported versions of the Strimzi and Apache Kafka projects are available as part of the Red Hat AMQ product.
Kubernetes Ingress
Ingress is a Kubernetes API for managing external access to HTTP/HTTPS services, which was added in Kubernetes 1.1. Ingress is the Kubernetes counterpart to Red Hat OpenShift routes, which we discussed previously. It acts as a Layer 7 load balancer for HTTP or HTTPS traffic. The Ingress resources will define the rules for routing the traffic to different services and pods. An Ingress controller takes care of the actual routing. For more information about Ingress, check out the Kubernetes website.
Ingress is a sort of strange part of the Kubernetes API. The Ingress API itself is part of every Kubernetes cluster, but the Ingress controller that would do the routing is not part of core Kubernetes. So, while you may be able to create the Ingress resources, there may be nothing to actually route the traffic.
For the Ingress resources to actually do something, you need to make sure an Ingress controller is installed. There are many different Ingress controllers. The Kubernetes project itself has two controllers:
- NGINX controller
- GCE controller for Google Cloud
Many additional controllers are created and maintained by different communities and companies. A list of different controllers can be found on the Kubernetes website.
Most of the controllers rely on a load balancer or node port service, which will get the external traffic to the controller. Once the traffic reaches the controller, the controller will route it based on rules specified in the Ingress resource to the different services and pods. The controller itself usually also runs as yet another application inside the Kubernetes cluster.
Some of the controllers are tailored for a specific public cloud. For example, the AWS ALB Ingress Controller provisions the AWS Application Load Balancer to do the routing instead of doing it inside a pod in your Kubernetes cluster.
Ingress offers a lot of functionality for HTTP applications such as:
- TLS termination
- Redirecting from HTTP to HTTPS
- Routing based on HTTP request path
Some of the controllers, such as the NGINX controller, also offer TLS passthrough, which is a feature we use in Strimzi.
Using Ingress in Strimzi
Ingress support in Strimzi has been added in Strimzi 0.12.0. It uses TLS passthrough and was tested with the NGINX Ingress Controller. Before using it, please make sure that the TLS passthrough is enabled in the controller. Note that Ingress support in Strimzi 0.12.0 is experimental. If you have any feedback or want to help make it work with different Ingress controllers, you can get in touch with us through Slack, our mailing list, or GitHub.
Although some Ingress controllers also support working directly with TCP connections, TLS passthrough seems to be more widely supported. Therefore, we decided to prefer TLS passthrough in Strimzi. That also means that when using Ingress, TLS encryption will always be enabled.
One of the main differences of Ingress, as compared with OpenShift routes, is that for Ingress you must specify the host address in your Kafka custom resource. The router in OpenShift will automatically assign a host address based on the route name and the project. In Ingress, however, the host address must be specified in the Ingress resource. You also have to take care that DNS resolves the host address to the Ingress controller. Strimzi cannot generate it for you, because it does not know which DNS addresses are configured for the Ingress controller.
If you want to try it, for example, on Minikube or in other environments where you don't have any managed DNS service to add the hosts for the Kafka cluster, you can use a wildcard DNS service, such as nip.io or xip.io, and set it to point to the IP address of your Ingress controller. For example, you can do: broker-0.<minikube-ip-address>.nip.io.
The way Strimzi uses Ingress to expose Apache Kafka should be familiar to you from the previous articles. We create one service as a bootstrap service and additional services for individual access to each of the Kafka brokers in the cluster. For each of these services, we will also create one Ingress resource with the corresponding TLS passthrough rule.
When configuring Strimzi to use Ingress, you must specify the type of the external listener as ingress and specify the Ingress hosts used for the different brokers, as well as for bootstrap, in the configuration field:
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: my-cluster
spec:
kafka:
# ...
listeners:
# ...
external:
type: ingress
configuration:
bootstrap:
host: bootstrap.192.168.64.46.nip.io
brokers:
- broker: 0
host: broker-0.192.168.64.46.nip.io
- broker: 1
host: broker-1.192.168.64.46.nip.io
- broker: 2
host: broker-2.192.168.64.46.nip.io
# ...
Using Ingress in your clients is very similar to OpenShift routes. Because it always uses TLS encryption, you have to first download the server certificate (replace my-cluster with the name of your cluster):
kubectl get secret cluster-name-cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
keytool -import -trustcacerts -alias root -file ca.crt -keystore truststore.jks -storepass password -noprompt
Once you have the TLS certificate, you can use the bootstrap host you specified in the Kafka custom resource and connect to the Kafka cluster. Because Ingress uses TLS passthrough, you always have to connect on port 443. The following example uses the kafka-console-producer.sh utility, which is part of Apache Kafka:
bin/kafka-console-producer.sh --broker-list :443 --producer-property security.protocol=SSL --producer-property ssl.truststore.password=password --producer-property ssl.truststore.location=./truststore.jks --topic
For example:
bin/kafka-console-producer.sh --broker-list bootstrap.192.168.64.46.nip.io:443 --producer-property security.protocol=SSL --producer-property ssl.truststore.password=password --producer-property ssl.truststore.location=./truststore.jks --topic
For more details, see the Strimzi documentation.
Customizations
DNS annotations
Many users employ additional tools, such as ExternalDNS, to automatically manage DNS records for their load balancers. ExternalDNS uses annotations on Ingress resources to manage their DNS names. It also supports many different DNS services, such as Amazon AWS Route53, Google Cloud DNS, Azure DNS, etc.
Strimzi lets you assign these annotations through the Kafka custom resource using a field called dnsAnnotations. Using the DNS annotations is simple:
# ...
listeners:
external:
type: ingress
configuration:
bootstrap:
host: kafka-bootstrap.mydomain.com
brokers:
- broker: 0
host: kafka-broker-0.mydomain.com
- broker: 1
host: kafka-broker-1.mydomain.com
- broker: 2
host: kafka-broker-2.mydomain.com
overrides:
bootstrap:
dnsAnnotations:
external-dns.alpha.kubernetes.io/hostname: kafka-bootstrap.mydomain.com.
external-dns.alpha.kubernetes.io/ttl: "60"
brokers:
- broker: 0
dnsAnnotations:
external-dns.alpha.kubernetes.io/hostname: kafka-broker-0.mydomain.com.
external-dns.alpha.kubernetes.io/ttl: "60"
- broker: 1
dnsAnnotations:
external-dns.alpha.kubernetes.io/hostname: kafka-broker-1.mydomain.com.
external-dns.alpha.kubernetes.io/ttl: "60"
- broker: 2
dnsAnnotations:
external-dns.alpha.kubernetes.io/hostname: kafka-broker-2.mydomain.com.
external-dns.alpha.kubernetes.io/ttl: "60"
# ...
Again, Strimzi lets you configure the annotations directly. That gives you more freedom and makes this feature usable even when you use a tool other than ExternalDNS. It also lets you configure other options than just the DNS names, such as the time-to-live of the DNS records, and so on.
Pros and cons
Kubernetes Ingress is not always easy to use because you have to install the Ingress controller, and you have to configure the hosts. It is also available only with TLS encryption because of the TLS passthrough functionality that Strimzi uses. However, it can offer an interesting option for clusters where node ports are not an option, for example, for security reasons and cases in which using load balancers would be too expensive.
When using Strimzi Kafka operator with Ingress, you must also consider performance. The Ingress controller usually runs inside your cluster as yet another deployment and adds an additional step through which your data has to flow between your clients and the brokers. You need to scale it properly to ensure it will not be a bottleneck for your clients.
Thus, Ingress might not be the best option when most of your applications using Kafka are outside of your Kubernetes cluster and you need to handle tens or hundreds of megabytes of throughput per second. However, especially in situations when most of your applications are inside your cluster and only a minority are outside and when the throughput you need is not so high, Ingress might be a convenient option.
The Ingress API and the Ingress controllers can usually be installed on OpenShift clusters as well, but they do not offer any advantages over the OpenShift routes. So, on OpenShift, you will likely want to use OpenShift routes instead.
What's next?
This was, for now, the last post in this series about accessing Strimzi Kafka clusters. In the five articles, we covered all the supported mechanisms that the Strimzi operator supports for accessing Apach Kafka from both inside and outside of your Kubernetes or Red Hat OpenShift cluster. We will, of course, keep posting articles on other topics and if something about accessing Kafka changes in the future, we will add to this series.
If you liked this series, star us on GitHub and follow Strimzi on Twitter to stay up to date with the project.
