Kubernetes, Istio and Knative (2020)

Exploratory approach with minikube

Clean the lab (lab work)

The iptables NAT chain gets flushed separately here.

The following may be a got fit for the lab’s ~/.bash_aliases:

coredns versus systemd

Systemd likes to reinvent how Linux handles things. This does not exclude DNS resolvers:

Without doing this, you may get problems when you are starting the minikube env. I had many issues with coredns.

Snap installs

This is quick and dirty. In production: never use Snapd.

Start minikube

You may be tempted to think, that the --memory and --cpus parameters can be omitted because this installs the services natively. I found out that these variables are used to calculate quotas and limits. If you do not specify them, you have to edit the ConfigMaps and Namespaces later. If you know where they are…

Now to the computing backbone: our simple single-node Kubernetes “cluster” is ready. Please be patient with that little 4 core system:

Basics: what is all this stuff? Recap on Kubernetes

The idea to placing the theory here is to allow a pragmatic and simplified reflection of the Kubernetes related concepts. At this point, it can be combined with a systematic exploration.

Open

Basic overview about the Kube components that matter (in my experience)

If you are a seasoned Kubernetes administrator, you may find the illustration to be inaccurate. That is because this is just teaching material.

• Pods are a core concept of Kubernetes

• Within a Deployment, you specify that you want a certain number of instances to be available (or let an Operator manage the scaling). In our case, we will deploy an application that gets managed within the cluster by a Serverless operator.

• A Service is the result of a Deployment. This way it becomes available. In our case, we will create an API server endpoint that clients can connect to.

• If persistent storage is required, it’s usually better to use network file systems and to mount these external resources into the transient instances.

• Labels and Selectors are important if you have many Services. You may have different settings for different environments (test, prod, staging, … lab)

On-premise Serverless stack - Istio and Knative

Install Istio with the observability tools

Istio is like an extension to Kubernetes. It adds a Service Mesh. In my opinion, this is the future of how we are going to run and think about services in Cloud-Native environments.

From a security perspective, I like having an mTLS (mutual Transport Layer Security) enabled Service Mesh network between Pods. If you make a Threat Model of your env you may find out that the Micro Services exchange confidential data containing PII (Personally Identifiable Information).

Encryption can help to develop means within the Micro Service architecture that treat the trust boundaries within third-party networks. Like between AWS EC2 instances.

If you are only interested in in-transit encryption for Kubernetes, you may also like to learn more about Cilium. I only used CNI and for a single-node minikube lab that is fine.

Other use-cases of Istio are Blue-Green deployments, Canary deployments or Dark Deployments. These happen with the Mixer component.

Keep these variables (or the commands). We are going to need them to test the routes and reachability of our Egress endpoints of the Service Mesh (with curl).

Connect to Kiali with the browser from your Dev system without a tunnel

Kiali is a wonderful way to gain instant familiarity with a Micro Service Mesh Network. It uses Prometheus (a metrics collector) and Jaeger (a tracer).

With both systems combined, you can gain insights into the auto-scaling behavior of Knative services. This relates to Istio in so far, that its Sidecar containers get injected into the Knative service Pods, and that the metrics are provided this way.
– But let’s focus on one thing at a time. Before we are ready to observe all of this, we need to continue with the setup. Keep in mind: you don’t just want the front-row seats for this. You want to be a player!

Although minikube has tunneling and kubectl has got proxyfication features, it may be more comfortable to map Kiali to the VM’s IP. This could also be done with Ingress or MetalLB, but the simplest form of mapping out the service is this:

Consider this specific step replaceable, but keep the one-liner in mind unless you prefer Ingress or MetalLB.

Add the gateway - only in a Dev env

Knative needs this because its routing (by default) will assume a real Kube cluster.

Patch the Istio Ingress-gateway

Again, this is just for the lab. Naturally, if you were to use OpenShift this would be a little different.

Recap: what are Operators, ConfigMaps and Routes?

At this point (Q2 2020), there are different kinds of Operators that can extend the Kubernetes architecture. The topic is vast and involves Custom Resource Definitions (CRDs) that can extend the set of functions of the Kube API. Knative brings an operator, that will initialize an “Autoscaler”. Keep in mind that in production you may not want your API endpoints to scale down to 0 instances, and that you can configure all of that. For demonstration purposes, I refrained from doing this here. It’s super easy to do, via a ConfigMap object.

ConfigMaps simply hold the configuration similar to Linux’s /etc/ directory. They can be edited, and the respective Operators observe them. So in case you want to change Knative’s domain or the autoscaling behavior, you will find documented variables in the ConfigMap.

Routes are how Services are bridged out, either via custom LoadBalancers, Ingress, or other Service objects. Knative changes the Routes for the autoscaling to distribute the requests. This is visualized in a video in this essay.

The good news is, that you’ll pick this up in great detail over time. I find very little use for encyclopedic summaries about Kubernetes. This is an area of rapid progress and in a couple of months, everything here will be different again. If you are a Linux user, you will many opportunities to harness your command-line skills to pick things up on the fly.

Time for Hello World - Knative Go

Here we build, tag and push the local image. You can use Docker or Podman to do so.

Time for 1, 2, 3, many curls

Let’s make a few HTTP requests to the service in parallel:

Development of a Kotlin SpringBoot REST server into your Knative cluster

Let’s deliver Chuck Norris jokes at scale, via a REST service endpoint developed with Spring Boot, written in Kotlin.
The advantage of this is that the code is easy to read and that the service behavior will be compliant to the standards so that we can do HTTP load-testing. This way, we can see auto-scaling Serverless workloads in action.

I use a Maven package that provides the joke texts via Spring Boot Actuator package. This way I do not need a DBMS.

The application is rather primitive. It offers two endpoints:

• / - returns HTML, via a Thymeleaf template

• /joke - returns JSON, via a library and Spring

I published the image to DockerHub. You can also build it yourself and push it into your own registry.

I would like to highlight that no code changes are required to run this application in Knative (or Docker). The code remains untouched.

Java 11 (versions higher than 8) is aware of Linux’s cgroups limits within container environments. This way, the container environment's memory limit is respected by the Java runtime.

– And you do not need to add mystical /dev/urandom mounts anymore. Just as a side-note.

Now get the endpoint:

Initially, it will be slow because it’s instantiating the application on-demand. You can use this to warm up the cluster

I tried to get an Ubuntu VM to make more than 200 curl requests using GNU parallel. Although I unset the limits, I reached up to 162 requests per second, which is not enough to put the respective cluster under enough stress (the default requests per second to trigger the Knative autoscaler is 200:

Instead, I used wrk, also because its invocation is similar to curl.

On the right, you can see the Pods serving Chuck Norris jokes. On the left, you can see the curl command to bring up the initial instance. This is followed by the wrk command to put the Service under enough load.

Open

Live demonstration of the Knative autoscaler, even autoscaling Chuck Norris jokes