Managing microservice traffic using Istio

duration 30 minutes

Explore how to manage microservice traffic using Istio.

What you’ll learn

You will learn how to deploy an application to a Kubernetes cluster and enable Istio on it. You will also learn how to configure Istio to shift traffic to implement blue-green deployments for microservices.

What is Istio?

Istio is a service mesh, meaning that it’s a platform for managing how microservices interact with each other and the outside world. Istio consists of a control plane and sidecars injected into application pods. The sidecars contain the Envoy proxy. You can think of Envoy as a sidecar that intercepts and controls all the HTTP and TCP traffic to and from your container.

While Istio runs on top of Kubernetes and that will be the focus of this guide, you can also use Istio with other environments such as Docker Compose. Istio has many features such as traffic shifting, request routing, access control, and distributed tracing, but the focus of this guide will be on traffic shifting.

Why Istio?

Istio provides a collection of features that allows you to manage several aspects of your services. One example is Istio’s routing features. You can route HTTP requests based on several factors such as HTTP headers or cookies. Another use case for Istio is telemetry, which you can use to enable distributed tracing. Distributed tracing allows you to visualize how HTTP requests travel between different services in your cluster by using a tool such as Jaeger. Additionally, as part of its collection of security features, Istio allows you to enable mutual TLS between pods in your cluster. Enabling TLS between pods secures communication between microservices internally.

You’ll use Istio to implement blue-green deployments. The traffic shifting feature allows you to allocate a percentage of traffic to certain versions of services. You can use this feature to shift 100 percent of live traffic to blue deployments and 100 percent of test traffic to green deployments. Then, you can shift the traffic to point to the opposite deployments as necessary to perform blue-green deployments.

The microservice you’ll deploy is called hello. It responds with a JSON object that has a version number and a greeting. The version number will be automatically incremented when you update the pom.xml version. This number will allow you to view which version of the microservice is running in your production or test environments.

What are blue-green deployments?

Blue-green deployments are a way of deploying your applications such that you have two environments where your application runs. In this scenario, you will have a production environment and a test environment. At any point in time, the blue deployment can accept production traffic and the green deployment can accept test traffic, or vice-versa. When you want to deploy a new version of your application, you will deploy to the color that is currently acting as your test environment. After the new version is verified on the test environment, the traffic will be shifted over. Thus, your live traffic is now being handled by what used to be the test site.

Prerequisites

Before you begin, have the following tools installed:

First, you will need a containerization software for building containers. Kubernetes supports a variety of container types. You will use Docker in this guide. For installation instructions, refer to the official Docker documentation.

WINDOWS | MAC

Use Docker Desktop, where a local Kubernetes environment is pre-installed and enabled. If you do not see the Kubernetes tab then you have an older version of Docker Desktop; upgrade to the latest version.

Complete the setup for your operating system:

  • Set up Docker for Windows.

  • Set up Docker for Mac.

  • After following one of the sets of instructions, ensure that Kubernetes (not Swarm) is selected as the orchestrator in Docker Preferences.

LINUX

You will use Minikube as a single-node Kubernetes cluster that runs locally in a virtual machine. For Minikube installation instructions see the minikube installation instructions. Make sure to read the Requirements section as different operating systems require different prerequisites to get Minikube running.

Getting started

The fastest way to work through this guide is to clone the Git repository and use the projects that are provided inside:

git clone https://github.com/openliberty/guide-istio-intro.git
cd guide-istio-intro

The start directory contains the starting project that you will build upon.

The finish directory contains the finished project that you will build.

Starting and preparing your cluster for deployment

Start your Kubernetes cluster.

WINDOWS | MAC

Start your Docker Desktop environment.

LINUX

Run the following command from a command line:

minikube start --memory=8192 --cpus=4 --kubernetes-version=v1.10.0

Next, validate that you have a healthy Kubernetes environment by running the following command from the command line.

kubectl get nodes

This command should return a Ready status for the master node.

WINDOWS | MAC

You do not need to do any other step.

LINUX

Run the following command to configure the Docker CLI to use Minikube’s Docker daemon. After you run this command, you will be able to interact with Minikube’s Docker daemon and build new images directly to it from your host machine:

eval $(minikube docker-env)

Deploying Istio

First, go to the Istio release page and download the latest stable release. Extract the archive and navigate to the directory with the extracted files.

Next, deploy the Istio custom resource definitions. Custom resource definitions allow Istio to define custom Kubernetes resources that you can use in your resource definition files.

kubectl apply -f install/kubernetes/helm/istio/templates/crds.yaml

Next, deploy Istio resources to your cluster by running the kubectl apply command, which creates or updates Kubernetes resources defined in a yaml file. This command will deploy Istio.

kubectl apply -f install/kubernetes/istio-demo.yaml

Verify that Istio was successfully deployed. All the values in the AVAILABLE column will have a value of 1 after the deployment is complete.

kubectl get deployments -n istio-system

Ensure that the Istio deployments are all available before you continue. The deployments might take a few minutes to become available. If the deployments aren’t available after a few minutes, then increase the amount of memory available to your Kubernetes cluster. On Docker Desktop, you can increase the memory from your Docker preferences. On Minikube, you can increase the memory using the --memory flag.

NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
grafana                  1         1         1            1           44s
istio-citadel            1         1         1            1           44s
istio-egressgateway      1         1         1            1           44s
istio-galley             1         1         1            1           44s
istio-ingressgateway     1         1         1            1           44s
istio-pilot              1         1         1            1           44s
istio-policy             1         1         1            1           44s
istio-sidecar-injector   1         1         1            1           44s
istio-telemetry          1         1         1            1           44s
istio-tracing            1         1         1            1           43s
prometheus               1         1         1            1           44s
servicegraph             1         1         1            1           44s

Finally, create the istio-injection label and set its value to enabled.

kubectl label namespace default istio-injection=enabled

Adding this label enables automatic Istio sidecar injection. Automatic injection means that sidecars will automatically be injected into your pods when you deploy your application. You don’t need to perform any additional steps for the sidecars to be injected.

Deploying version 1 of the hello microservice

Navigate to the start directory and run the following command. It may take a few minutes to complete. It will build the application and then package it into a Docker image. To build the Docker image, it uses a Maven plug-in called dockerfile-maven-plugin.

mvn clean package

The command builds a Docker image for the hello microservice. You can verify that this image was created by running the following command.

docker images

You’ll see an image called hello:1.0-SNAPSHOT listed in a table similar to the output.

REPOSITORY                 TAG            IMAGE ID       CREATED          SIZE
hello                      1.0-SNAPSHOT   d316c2c2c6ba   9 seconds ago    501MB
istio/galley               1.0.1          7ac6c7be3d3e   5 days ago       65.8MB
istio/citadel              1.0.1          abcc721c2454   5 days ago       51.7MB
istio/mixer                1.0.1          0d97b4000ed5   5 days ago       64.5MB
istio/sidecar_injector     1.0.1          a122adc160b7   5 days ago       45.3MB
istio/proxyv2              1.0.1          f1bf7b920fe1   5 days ago       352MB
istio/pilot                1.0.1          46d3b4e95fc3   5 days ago       290MB
open-liberty               latest         ed1ca62c4bd5   7 days ago       501MB
prom/prometheus            v2.3.1         b82ef1f3aa07   2 months ago     119MB

To deploy the hello microservice to the Kubernetes cluster, use the following command to deploy the microservice.

kubectl apply -f hello.yaml

You can see that your resources are created:

gateway.networking.istio.io/hello-gateway created
service/hello-service created
deployment.apps/hello-deployment-blue created
deployment.apps/hello-deployment-green created
destinationrule.networking.istio.io/hello-destination-rule created

View the hello.yaml file. It contains two deployments, a service, a gateway, and a destination rule. One of the deployments is labeled blue and the second deployment is labeled green. The service points to both of these deployments. The Istio gateway is the entry point for HTTP requests to the cluster. A destination rule is used to apply policies post-routing, in this situation it is used to define service subsets that can be specifically routed to.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: hello-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "example.com"
    - "test.example.com"
---
apiVersion: v1
kind: Service
metadata:
  name: hello-service
  labels:
    app: hello
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: hello
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-deployment-blue
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
      version: blue
  template:
    metadata:
      labels:
        app: hello
        version: blue
    spec:
      containers:
      - name: hello-container
        image: hello:1.0-SNAPSHOT
        ports:
        - containerPort: 9080
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-deployment-green
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
      version: green
  template:
    metadata:
      labels:
        app: hello
        version: green
    spec:
      containers:
      - name: hello-container
        image: hello:1.0-SNAPSHOT
        ports:
        - containerPort: 9080
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: hello-destination-rule
spec:
  host: hello-service
  subsets:
  - name: blue
    labels:
      version: blue
  - name: green
    labels:
      version: green

View the traffic.yaml file. It contains two virtual services. A virtual service defines how requests are routed to your applications. In the virtual services, you can configure the weight, which controls the amount of traffic going to each deployment. In this case, the weights should be 100 or 0, which corresponds to which deployment is live.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-virtual-service
spec:
  hosts:
  - "example.com"
  gateways:
  - hello-gateway
  http:
  - route:
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: blue
      weight: 100
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: green
      weight: 0
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-test-virtual-service
spec:
  hosts:
  - "test.example.com"
  gateways:
  - hello-gateway
  http:
  - route:
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: blue
      weight: 0
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: green
      weight: 100

Deploy the resources defined in the traffic.yaml file.

kubectl apply -f traffic.yaml

You can see that the virtual services have been created.

virtualservice.networking.istio.io/hello-virtual-service created
virtualservice.networking.istio.io/hello-test-virtual-service created

You can check that all of the deployments are available by running the following command.

kubectl get deployments

The command produces a list of deployments for your microservices that is similar to the following output.

NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
hello-deployment-blue    1         1         1            1           1m
hello-deployment-green   1         1         1            1           1m

After all the deployments are available, you will make a request to v1 of the deployed application. As defined in the hello.yaml file the gateway is expecting the host to be example.com. However, requests to example.com won’t be routed to the appropriate IP address. To ensure that the gateway routes your requests appropriately, ensure that the Host header is set to example.com. For instance, you can set the Host header with the -H option of the curl command.

WINDOWS | MAC

Make a request to the service by running the following curl command.

curl -HHost:example.com http://localhost/hello

If the curl command is unavailable, then use Postman. Postman enables you to make requests using a graphical interface. To make a request with Postman, enter http://localhost/hello into the URL bar. Next, switch to the Headers tab and add a header with key of Host and value of example.com. Finally, click the blue Send button to make the request.

LINUX

Make a request to the service by using curl.

curl -HHost:example.com http://`minikube ip`:31380/hello

You’ll see a greeting message along with a corresponding version.

{"greeting":"hello","version":"1.0-SNAPSHOT"}

Deploying version 2 of the hello microservice

The hello microservice is set up to respond with the version that is set in the pom.xml file. The tag for the Docker image is also dependent on the version specified in the pom.xml file. Use the Maven command to bump the version of the microservice to 2.0-SNAPSHOT.

mvn versions:set -DnewVersion=2.0-SNAPSHOT

Build the new version of the Docker container.

mvn clean package

Deploy the new image to the green deployment.

kubectl set image deployment/hello-deployment-green hello-container=hello:2.0-SNAPSHOT

You will work with two environments. One of the environments is a test site located at test.example.com. The other environment is your production environment located at example.com. To start with, the production environment is tied to the blue deployment and the test environment is tied to the green deployment.

Test the updated microservice by making requests to the test site. The version field in the response JSON is now 2.0-SNAPSHOT on the test site and is still 1.0-SNAPSHOT on the live site.

WINDOWS | MAC

Make a request to the service by running the following curl command.

curl -HHost:test.example.com http://localhost/hello

If the curl command is unavailable, then use Postman.

LINUX

Make a request to the service by using curl.

curl -HHost:test.example.com http://`minikube ip`:31380/hello

You’ll see a greeting message along with a corresponding version.

{"greeting":"hello","version":"2.0-SNAPSHOT"}

After you see that the microservice is working on the test site, modify the weights in the traffic.yaml file to shift 100 percent of the live traffic to the green deployment.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-virtual-service
spec:
  hosts:
  - "example.com"
  gateways:
  - hello-gateway
  http:
  - route:
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: blue
      weight: 0
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: green
      weight: 100
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: hello-test-virtual-service
spec:
  hosts:
  - "test.example.com"
  gateways:
  - hello-gateway
  http:
  - route:
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: blue
      weight: 100
    - destination:
        port:
          number: 9080
        host: hello-service
        subset: green
      weight: 0

Deploy the updated traffic.yaml file.

kubectl apply -f traffic.yaml

Ensure that the live traffic is now being routed to version 2 of the microservice.

WINDOWS | MAC

Make a request to the service by running the following curl command.

curl -HHost:example.com http://localhost/hello

If the curl command is unavailable, then use Postman.

LINUX

Make a request to the service by using curl.

curl -HHost:example.com http://`minikube ip`:31380/hello

You’ll see a greeting message along with a corresponding version.

{"greeting":"hello","version":"2.0-SNAPSHOT"}

Testing microservices that are running on Kubernetes

Next, you will create a test to verify that the correct version of your microservice is running.

Create the endpoint test class in the src/test/java/it/io/openliberty/guides/rest/EndpointTest.java file:

package it.io.openliberty.guides.rest;

import static org.junit.Assert.assertEquals;

import org.junit.Test;

import javax.json.JsonObject;

import javax.ws.rs.client.Client;
import javax.ws.rs.client.ClientBuilder;
import javax.ws.rs.client.WebTarget;
import javax.ws.rs.core.Response;

import org.apache.cxf.jaxrs.provider.jsrjsonp.JsrJsonpProvider;

public class EndpointTest {

    @Test
    public void testGetGreeting() {
        // Allows for overriding the "Host" http header
        System.setProperty("sun.net.http.allowRestrictedHeaders", "true");

        String hostname = System.getProperty("cluster.ip");
        String port = System.getProperty("port");
        String url = String.format("http://%s:%s/hello", hostname, port);

        Client client = ClientBuilder.newClient();
        client.register(JsrJsonpProvider.class);

        WebTarget target = client.target(url);
        Response response = target
            .request()
            .header("Host", System.getProperty("host-header"))
            .get();

        assertEquals("Incorrect response code from " + url,
                     200,
                     response.getStatus());

        JsonObject obj = response.readEntity(JsonObject.class);
        assertEquals("The greeting property must have message \"hello\"",
                     "hello",
                     obj.getString("greeting"));

        assertEquals("The version must match the pom.xml file",
                     System.getProperty("app.name"),
                     obj.getString("version"));

        response.close();
    }
}

The testGetGreeting test case verifies that the correct greeting and version number are returned from the rest service.

WINDOWS | MAC

Run the command to start the tests:

mvn verify -Ddockerfile.skip=true

LINUX

Run the command to start the tests:

mvn verify -Ddockerfile.skip=true -Dcluster.ip=`minikube ip` -Dport=31380

The cluster.ip and port parameters refer to the IP address and port for the Istio gateway.

The dockerfile.skip=true flag skips rebuilding the Docker images.

If the tests pass, then you should see output similar to the following example:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.rest.EndpointTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.648 s - in it.io.openliberty.guides.rest.EndpointTest

Results:

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Tearing down your environment

You might want to teardown all the deployed resources as a cleanup step.

Delete your resources from the cluster.

kubectl delete -f hello.yaml
kubectl delete -f traffic.yaml

Delete the istio-injection label from the default namespace. The hyphen immediately after the label name indicates that the label should be deleted.

kubectl label namespace default istio-injection-

Navigate to the directory where you extracted Istio and delete the Istio resources from the cluster.

kubectl delete -f install/kubernetes/istio-demo.yaml
kubectl delete -f install/kubernetes/helm/istio/templates/crds.yaml

WINDOWS | MAC

Nothing more needs to be done for Docker Desktop.

LINUX

Perform the following steps to return your environment to a clean state.

  1. Point the Docker daemon back to your local machine:

    eval $(minikube docker-env -u)
  2. Stop your Minikube cluster:

    minikube stop
  3. Delete your cluster:

    minikube delete

Great work! You’re done!

You have deployed a microservice that runs on Open Liberty to a Kubernetes cluster and used Istio to implement a blue-green deployment scheme.

Contribute to this guide

Is something missing or broken? Raise an issue, or send us a pull request.