30 minutes
Explore how to manage the impact of failures by using MicroProfile and Istio Fault Tolerance to add retry and fallback behaviours to microservices.
You will learn how to combine MicroProfile Retry and Fallback policies with Istio Retry to make your microservices more resilient to common failures, such as network problems.
Microservices that are created using Eclipse MicroProfile can be freely deployed in a service mesh to reduce the complexity associated with managing microservices. Istio is a service mesh, meaning that it’s a platform for managing how microservices interact with each other and the outside world. Istio consists of a control plane and sidecars that are injected into application pods. The sidecars contain the Envoy proxy. You can think of Envoy as a sidecar that intercepts and controls all the HTTP and TCP traffic to and from your container. If you would like to learn more about Istio, check out the Managing microservice traffic using Istio guide.
MicroProfile and Istio both provide simple and flexible solutions to build fault-tolerant microservices. Fault tolerance provides different strategies for building robust behaviour to cope with unexpected failures. A few fault tolerance policies that MicroProfile can offer include Retry, Timeout, Circuit Breaker, Bulkhead, and Fallback. There is some overlap that exists between MicroProfile and Istio Fault Tolerance, such as the Retry policy. However, Istio does not offer any fallback capabilities. To view the available fault tolerance policies in MicroProfile and Istio, refer to the comparison between MicroProfile and Istio fault handling.
Use retry policies to fail quickly and recover from brief intermittent issues. An application might experience these transient failures when a microservice is undeployed, a database is overloaded by queries, the network connection becomes unstable, or the site host has a brief downtime. In these cases, rather than failing quickly on these transient failures, a retry policy provides another chance for the request to succeed. Simply retrying the request might be all you need to do to make it succeed.
Fallback offers an alternative execution path when an execution does not complete successfully.
You will use the @Fallback annotation from the MicroProfile Fault Tolerance specification to define criteria
for when to provide an alternative solution for a failed execution.
You will develop microservices that demonstrate MicroProfile Fault Tolerance with Istio fault handling. Both MicroProfile and Istio can be used when you want your microservices to have a service mesh architecture with Istio, and use MicroProfile to provide the extra fault tolerance policies that do not exist within Istio.
The application that you will be working with is an inventory service, which collects, stores, and returns the system properties.
It uses the system service to retrieve the system properties for a particular host.
You will add fault tolerance to the inventory service so that it reacts accordingly when the system service is unavailable.
Before you begin, you need a containerization software for building containers. Kubernetes supports various container runtimes. You will use Docker in this guide. For Docker installation instructions, refer to the official Docker documentation.
WINDOWS
MAC
LINUX
Use Docker Desktop, where a local Kubernetes environment is pre-installed and enabled. If you do not see the Kubernetes tab, then upgrade to the latest version of Docker Desktop.
Complete the setup for your operating system:
Set up Docker for Windows.
After you complete the Docker setup instructions for your operating system, ensure that Kubernetes (not Swarm) is selected as the orchestrator in Docker Preferences.
The fastest way to work through this guide is to clone the Git repository and use the projects that are provided inside:
git clone https://github.com/openliberty/guide-microprofile-istio-retry-fallback.git
cd guide-microprofile-istio-retry-fallbackThe start directory contains the starting project that you will build upon.
The finish directory contains the finished project that you will build.
Before you begin, make sure you have all the necessary prerequisites.
Start your Kubernetes cluster.
WINDOWS
MAC
LINUX
Start your Docker Desktop environment.
Ensure that you have enough memory allocated to your Docker Desktop enviornment. 8GB is recommended but 4GB should be adequate if you don’t have enough RAM.
Ensure that Kubernetes is running on Docker Desktop and that the context is set to docker-desktop.
Next, validate that you have a healthy Kubernetes environment by running the following command from the active command-line session.
kubectl get nodesThis command should return a Ready status for the master node.
WINDOWS
MAC
LINUX
You do not need to do any other step.
Install Istio by following the instructions in the official Istio Getting started documentation.
Run the following command to verify that the istioctl path was set successfully:
istioctl versionThe output will be similar to the following example:
no running Istio pods in "istio-system"
1.24.2Run the following command to configure the Istio profile on Kubernetes:
istioctl install --set profile=demoThe following output appears when the installation is complete:
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation completeVerify that Istio was successfully deployed by running the following command:
kubectl get deployments -n istio-systemAll the values in the AVAILABLE column will have a value of 1 after
the deployment is complete.
NAME READY UP-TO-DATE AVAILABLE AGE
istio-egressgateway 1/1 1 1 2m48s
istio-ingressgateway 1/1 1 1 2m48s
istiod 1/1 1 1 2m48sEnsure that the Istio deployments are all available before you continue. The deployments might take a few minutes to become available. If the deployments aren’t available after a few minutes, then increase the amount of memory available to your Kubernetes cluster. On Docker Desktop, you can increase the memory from your Docker preferences. On Minikube, you can increase the memory by using the --memory flag.
Finally, create the istio-injection label and set its value to enabled:
kubectl label namespace default istio-injection=enabledAdding this label enables automatic Istio sidecar injection. Automatic injection means that sidecars are automatically injected into your pods when you deploy your application.
Navigate to the guide-microprofile-istio-retry-fallback/start directory to begin.
The MicroProfile Fault Tolerance API is included in the MicroProfile dependency that is specified in your pom.xml file.
Look for the dependency with the microprofile artifact ID.
This dependency provides a library that allows you to use the fault tolerance policies in your microservices.
The InventoryResource.java file
makes a request to the system service through the MicroProfile Rest Client API.
If you want to learn more about MicroProfile Rest Client,
you can follow the Consuming RESTful services with template interfaces guide.
To simulate that your system service is temporarily down due to brief intermittent issues, you will pause the pod that is associated
with your system service, then try to send requests to the service. When the system pod is paused, requests to the service return a 503 status code,
and the systemClient.getProperties() in InventoryResource.java throws a WebApplicationException.
To retry the requests to your system service after a WebApplicationException has occurred, add the @Retry annotation.
inventory/src/main/java/io/openliberty/guides/inventory/InventoryResource.java
To retry the service request a maximum of 3 times, only when a WebApplicationException occurs,
add the @Retry annotation before the getPropertiesForHost() method.
A request to a service might fail for many different reasons. The default Retry policy initiates a retry for every java.lang.Exception.
However, you can base a Retry policy on a specific exception by using the retryOn parameter. You can identify more than one exception
as an array of values. For example, @Retry(retryOn = {RuntimeException.class, TimeoutException.class}).
You can set limits on the number of retry attempts to avoid overloading a busy service with retry requests.
The @Retry annotation has the maxRetries parameter to limit the number of retry attempts. The default number for maxRetries is 3 requests.
The integer value must be greater than or equal to -1. A value of -1 indicates to continue retrying indefinitely.
Navigate to the guide-microprofile-istio-retry-fallback/start directory and run the following command to build your application and integrate the Retry policy into your microservices:
mvn packageNext, run the docker build commands to build container images for your application:
docker build -t system:1.0-SNAPSHOT system/.
docker build -t inventory:1.0-SNAPSHOT inventory/.The -t flag in the docker build command allows the Docker image to be labeled (tagged) in the name[:tag] format.
The tag for an image describes the specific image version. If the optional [:tag] tag is not specified, the latest tag is created by default.
To verify that the images for your system and inventory microservices are built, run the docker images command to list all local Docker images.
docker imagesYour two images system and inventory should appear in the list of all Docker images:
REPOSITORY TAG
inventory 1.0-SNAPSHOT
system 1.0-SNAPSHOTTo deploy your microservices to the Kubernetes cluster, use the following command:
kubectl apply -f services.yamlYou will see an output similar to the following:
gateway.networking.istio.io/sys-app-gateway created service/system-service created service/inventory-service created deployment.apps/system-deployment created deployment.apps/inventory-deployment created
The traffic.yaml file contains two virtual services. A virtual service defines how requests are routed to your applications.
Deploy the resources defined in the traffic.yaml file:
kubectl apply -f traffic.yamlRun the following command to check the status of your pods:
kubectl get podsIf all the pods are healthy and running, you will see an output similar to the following:
NAME READY STATUS RESTARTS AGE
inventory-deployment-645767664f-nbtd9 2/2 Running 0 30s
system-deployment-6bd97d9bf6-4ccds 2/2 Running 0 30sCheck that all of the deployments are available. You need to wait until all of your deployments are ready and available before making requests to your microservices.
kubectl get deploymentsNAME READY UP-TO-DATE AVAILABLE AGE
inventory-deployment 1/1 1 1 1m
system-deployment 1/1 1 1 1mYou will make a request to the system service from the inventory service to access the
JVM system properties of your running container. The Istio gateway is the entry point for HTTP requests to the cluster.
As defined in the services.yaml file, the gateway is expecting the
Host header of your system service and inventory service to be system.example.com and inventory.example.com, respectively.
However, requests to system.example.com and inventory.example.com won’t be routed to the appropriate IP address.
To ensure that the gateway routes your requests appropriately, ensure that the Host header is set appropriately.
You can set the Host header with the -H option of the curl command.
Make a request to the service by using curl:
WINDOWS
MAC
LINUX
curl -H Host:inventory.example.com http://localhost/inventory/systems/system-service -IIf the curl command is unavailable, then use Postman. Postman enables you
to make requests using a graphical interface. To make a request with Postman, enter http://localhost/inventory/systems/system-service
into the URL bar. Next, switch to the Headers tab and add a header with key of Host and value of inventory.example.com.
Finally, click the blue Send button to make the request.
You will see the following output:
HTTP/1.1 200 OK
x-powered-by: Servlet/4.0
content-type: application/json
date: Mon, 19 Aug 2019 19:49:47 GMT
content-language: en-US
x-envoy-upstream-service-time: 4242
server: istio-envoy
transfer-encoding: chunkedBecause the system service is available, the request to the service is successful and returns a 200 response code.
To see the number of times that the system service was called, check the logs of the system pod by using the kubectl logs command.
Replace [system-pod-name] with the pod name associated with
your system service, which you previously saw when running the kubectl get pods command.
WINDOWS
MAC
LINUX
kubectl logs [system-pod-name] -c istio-proxy | find /C "system-service:9090"You will see that the kubectl logs command returns a value of 1, meaning that 1 request is made to the system service:
1Now you will make the system service unavailable and observe that MicroProfile’s Retry policy will take effect.
Pause the system service pod to simulate that the service is unavailable.
Remember to replace [system-pod-name] with the pod name that is associated with your system service.
kubectl exec -it [system-pod-name] -- /opt/ol/wlp/bin/server pauseYou will see the following output:
Pausing the defaultServer server. Pausing the defaultServer server completed.
Make a request to the service by using curl:
WINDOWS
MAC
LINUX
curl -H Host:inventory.example.com http://localhost/inventory/systems/system-service -IIf the curl command is unavailable, then use Postman.
You will see the following output:
HTTP/1.1 503 Service Unavailable
x-powered-by: Servlet/4.0
content-length: 91
content-type: text/plain
date: Thu, 15 Aug 2019 13:21:57 GMT
server: istio-envoy
x-envoy-upstream-service-time: 2929
content-language: en-USBecause the the system service is unavailable, the request returns a 503 response code.
However, the request retried several times, as specified by the MicroProfile @Retry annotation.
See the number of times that the service was retried:
WINDOWS
MAC
LINUX
kubectl logs [system-pod-name] -c istio-proxy | find /C "system-service:9090"You will see the following output:
5This command returns 5 because you made a total of 5 requests to the system service: the initial request, 3 MicroProfile retry requests, and the successful request that you made before you paused the system service.
Previously, you implemented the Retry policy to retry requests to your system service by using MicroProfile Fault Tolerance.
This Retry policy can also be implemented with Istio Fault Tolerance.
traffic.yaml
Add the retries field under the route specification in the traffic.yaml file.
This tells Istio to retry requests a maximum of 4 times when the request returns any 5xx response code.
The attempts field is required in the configuration of the
Istio Retry policy. This field specifies the
maximum number of retries that will be attempted for a given request. To retry a request on specific conditions, use the
retryOn field. Because your paused system service responds with a 503 response code,
you set retryOn to be 5xx.
Other retry conditions
can also be specified in retryOn. Optionally, the perTryTimeout field can be added to Istio’s Retry policy
to specify the amount of time that is allocated to each retry attempt.
After you configure the number of retries that Istio performs, deploy your microservices again:
kubectl replace --force -f services.yaml
kubectl replace --force -f traffic.yamlWait until all of your deployments are ready and available.
The [system-pod-name] will be regenerated and is different than the one you used previously.
Run the kubectl get pods command to get the new [system-pod-name].
Pause the system service pod to simulate that the service is unavailable.
kubectl exec -it [system-pod-name] -- /opt/ol/wlp/bin/server pauseMake a request to the service by using curl:
WINDOWS
MAC
LINUX
curl -H Host:inventory.example.com http://localhost/inventory/systems/system-service -IIf the curl command is unavailable, then use Postman.
Because the system service is unavailable, the request still returns a 503 response code.
This time, however, Istio retried the request several more times before failing.
See the number of times that the service was retried:
WINDOWS
MAC
LINUX
kubectl logs [system-pod-name] -c istio-proxy | find /C "system-service:9090"You will see the following output:
20This command returns a value of 20, indicating that a total of 20 requests were made to the system service, including:
5 Istio requests: the initial request and 4 retry requests that you enabled in the traffic.yaml file
For each of the 5 Istio requests, MicroProfile sent 4 requests: the initial request and 3 retry requests
Next, you will disable some MicroProfile Fault Tolerance capabilities, so that your system service retries with only Istio’s Retry policy.
When both MicroProfile and Istio Fault Tolerance capabilities are enabled, there is a compounding effect that may be unexpected. If both MicroProfile and Istio set their own Retry policies on a service, the maximum number of retries that occur is not equivalent to either of the number of retries specified in MicroProfile or Istio. The number of retries set by MicroProfile and Istio are actually multiplied.
If you want to use Istio as your service mesh and only its fault tolerance capabilities, you can turn off MicroProfile Fault Tolerance by adding a property. This configuration avoids any overlap in behaviours.
MicroProfile Fault Tolerance offers a config property MP_Fault_Tolerance_NonFallback_Enabled that disables all
MicroProfile Fault Tolerance capabilities except fallback. If MP_Fault_Tolerance_NonFallback_Enabled is set to false, only
the @Fallback behaviour is enabled. The other behaviours specified by the MicroProfile Fault Tolerance annotations,
including @Retry, won’t take effect.
You will define the MP_Fault_Tolerance_NonFallback_Enabled config property in a ConfigMap.
ConfigMaps store configuration settings about a Kubernetes pod. This configuration is loaded into the pod as an environment variable that is
used by the pod’s containers. The environment variables are defined in the pod’s specification by using the envFrom field. To learn more about ConfigMaps,
check out the Configuring microservices running in Kubernetes guide.
Use the MP_Fault_Tolerance_NonFallback_Enabled config property to disable the retries performed by MicroProfile,
so that only Istio performs retries.
services.yaml
Add a ConfigMap into the services.yaml file, and
set the MP_Fault_Tolerance_NonFallback_Enabled config property to false.
Add the envFrom field
to inject the ConfigMap with the MP_Fault_Tolerance_NonFallback_Enabled property into your pods.
The name of the ConfigMap, which is inventory-config,
becomes the environment variable name that is specified in the envFrom field.
Deploy your microservices again to turn off all MicroProfile Fault Tolerance capabilities, except fallback:
kubectl replace --force -f services.yamlWait until all of your deployments are ready and available.
Run the kubectl get pods command to get the new [system-pod-name].
Pause the system service pod to simulate that the service is unavailable:
kubectl exec -it [system-pod-name] -- /opt/ol/wlp/bin/server pauseMake a request to the service by using curl:
WINDOWS
MAC
LINUX
curl -H Host:inventory.example.com http://localhost/inventory/systems/system-service -IIf the curl command is unavailable, then use Postman.
Because the system service is unavailable, the request still returns a 503 response code.
This time, however, the request was retried several times with Istio, without any retries from MicroProfile.
See the number of times that the service was retried:
WINDOWS
MAC
LINUX
kubectl logs [system-pod-name] -c istio-proxy | find /C "system-service:9090"You will see the following output:
5This command returns 5, indicating that a total of 5 requests were made to the system service.
Because MicroProfile’s Retry policy is disabled, only Istio’s retries are performed.
Since retrying the requests to the system service still does not succeed, you need a "fall back" plan.
You will create a fallback method as an alternative solution for when retry requests to the system service have failed.
Although you disabled MicroProfile @Retry and other MicroProfile Fault Tolerance policies using the
MP_Fault_Tolerance_NonFallback_Enabled config property, the fallback policy is still available.
As mentioned before, Istio does not offer any fallback capabilities, so the MicroProfile Fallback capability can be used to complement it.
The @Fallback annotation dictates a method to call when the original method encounters a failed execution.
If your microservices have a Retry policy specified, then the fallback occurs after all of the retries have failed.
inventory/src/main/java/io/openliberty/guides/inventory/InventoryResource.java
Create the getPropertiesFallback() method.
Add the @Fallback annotation before the getPropertiesForHost() method,
to call the getPropertiesFallback() method when a failure occurs.
The getPropertiesFallback() method, which is the designated fallback method for the original
getPropertiesForHost() method, prints out a warning message in the browser that says
the system service may not be running.
Rebuild your application to add fallback behaviour to your microservices:
mvn packageNext, run the docker build commands to rebuild the container images for your application:
docker build -t inventory:1.0-SNAPSHOT inventory/.
docker build -t system:1.0-SNAPSHOT system/.Deploy your microservices again to turn off all MicroProfile Fault Tolerance capabilities, except fallback:
kubectl replace --force -f services.yaml
kubectl replace --force -f traffic.yamlWait until all of your deployments are ready and available.
Run the kubectl get pods command to get the new [system-pod-name].
Pause the system service pod to simulate that the service is unavailable:
kubectl exec -it [system-pod-name] -- /opt/ol/wlp/bin/server pauseMake a request to the service by using curl:
WINDOWS
MAC
LINUX
curl -H Host:inventory.example.com http://localhost/inventory/systems/system-service -IIf the curl command is unavailable, then use Postman.
You will see the following output:
HTTP/1.1 200 OK
x-powered-by: Servlet/4.0
x-from-fallback: yes
content-type: application/json
date: Mon, 19 Aug 2019 19:49:47 GMT
content-language: en-US
x-envoy-upstream-service-time: 4242
server: istio-envoy
transfer-encoding: chunkedYou can see that the request is now successful and returns a 200 response code, with a header called
x-from-fallback, indicating that the fallback method is called when the system service is not available.
See the number of times that the service is retried before the fallback method is called:
WINDOWS
MAC
LINUX
kubectl logs [system-pod-name] -c istio-proxy | find /C "system-service:9090"You will see the following output:
1This command returns 1, indicating that 1 request was made to the system service.
The Istio retries that you enabled on the inventory service are not performed.
Although the getPropertiesForHost() method failed, the getPropertiesFallback() fallback method was called.
When you are done checking out the MicroProfile and Istio Fault Tolerance features, you might want to tear down all the deployed resources as a cleanup step.
Delete your resources from the cluster:
kubectl delete -f services.yaml
kubectl delete -f traffic.yamlDelete the istio-injection label from the default namespace. The hyphen immediately
after the label name indicates that the label should be deleted.
kubectl label namespace default istio-injection-Delete all Istio resources from the cluster:
WINDOWS
MAC
LINUX
istioctl uninstall --purge
1<?xml version="1.0" encoding="UTF-8"?>
2<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
3
4 <modelVersion>4.0.0</modelVersion>
5
6 <groupId>io.openliberty.guides</groupId>
7 <artifactId>guide-microprofile-istio-retry-fallback-inventory</artifactId>
8 <version>1.0-SNAPSHOT</version>
9 <packaging>war</packaging>
10
11 <properties>
12 <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
13 <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
14 <maven.compiler.source>11</maven.compiler.source>
15 <maven.compiler.target>11</maven.compiler.target>
16 <!-- Liberty configuration -->
17 <liberty.var.http.port>9091</liberty.var.http.port>
18 <liberty.var.https.port>9454</liberty.var.https.port>
19 <liberty.var.system.http.port>9090</liberty.var.system.http.port>
20 <liberty.var.system.https.port>9453</liberty.var.system.https.port>
21 </properties>
22
23 <dependencies>
24 <!-- Provided dependencies -->
25 <dependency>
26 <groupId>jakarta.platform</groupId>
27 <artifactId>jakarta.jakartaee-api</artifactId>
28 <version>10.0.0</version>
29 <scope>provided</scope>
30 </dependency>
32 <dependency>
33 <groupId>org.eclipse.microprofile</groupId>
34 <artifactId>microprofile</artifactId>
35 <version>7.0</version>
36 <type>pom</type>
37 <scope>provided</scope>
38 </dependency>
40 </dependencies>
41
42 <build>
43 <finalName>${project.artifactId}</finalName>
44 <plugins>
45 <plugin>
46 <groupId>org.apache.maven.plugins</groupId>
47 <artifactId>maven-war-plugin</artifactId>
48 <version>3.4.0</version>
49 </plugin>
50 <!-- Enable liberty-maven plugin -->
51 <plugin>
52 <groupId>io.openliberty.tools</groupId>
53 <artifactId>liberty-maven-plugin</artifactId>
54 <version>3.11.3</version>
55 </plugin>
56 <!-- Plugin to run unit tests -->
57 <plugin>
58 <groupId>org.apache.maven.plugins</groupId>
59 <artifactId>maven-surefire-plugin</artifactId>
60 <version>3.5.3</version>
61 </plugin>
62 <!-- Plugin to run functional tests -->
63 <plugin>
64 <groupId>org.apache.maven.plugins</groupId>
65 <artifactId>maven-failsafe-plugin</artifactId>
66 <version>3.5.3</version>
67 </plugin>
68 </plugins>
69 </build>
70</project>71Nice work! You learned how to build resilient microservices by using Istio Retry and MicroProfile Fallback. You also observed how MicroProfile Fault Tolerance integrates with and complements Istio Fault Tolerance.
Developing fault-tolerant microservices with Istio Retry and MicroProfile Fallback by Open Liberty is licensed under CC BY-ND 4.0
Thank you for your feedback!
Raise an issue to share feedback
Create a pull request to contribute to this guide
Ask a question on Stack Overflow
