Configuring microservices running in Kubernetes

duration 15 minutes

Prerequisites:

Explore how to externalize configuration using MicroProfile Config and configure your microservices using Kubernetes ConfigMaps and Secrets.

What you’ll learn

You will learn how and why to externalize your microservice’s configuration. Externalized configuration is useful because configuration usually changes depending on your environment. You will also learn how to configure the environment by providing required values to your application using Kubernetes. Using environment variables allows for easier deployment to different environments.

MicroProfile Config provides useful annotations that you can use to inject configured values into your code. These values can come from any config sources, such as environment variables. To learn more about MicroProfile Config, read the Configuring microservices guide.

Furthermore, you’ll learn how to set these environment variables with ConfigMaps and Secrets. These resources are provided by Kubernetes and act as a data source for your environment variables. You can use a ConfigMap or Secret to set environment variables for any number of containers.

Additional prerequisites

Before you begin, you need a containerization software for building containers. Kubernetes supports various container runtimes. You will use Docker in this guide. For Docker installation instructions, refer to the official Docker documentation.

Use Docker Desktop, where a local Kubernetes environment is pre-installed and enabled. If you do not see the Kubernetes tab, then upgrade to the latest version of Docker Desktop.

Complete the setup for your operating system:

After you complete the Docker setup instructions for your operating system, ensure that Kubernetes (not Swarm) is selected as the orchestrator in Docker Preferences.

Use Docker Desktop, where a local Kubernetes environment is pre-installed and enabled. If you do not see the Kubernetes tab, then upgrade to the latest version of Docker Desktop.

Complete the setup for your operating system:

After you complete the Docker setup instructions for your operating system, ensure that Kubernetes (not Swarm) is selected as the orchestrator in Docker Preferences.

You will use Minikube as a single-node Kubernetes cluster that runs locally in a virtual machine. Make sure you have kubectl installed. If you need to install kubectl, see the kubectl installation instructions. For Minikube installation instructions, see the Minikube documentation.

Getting started

The fastest way to work through this guide is to clone the Git repository and use the projects that are provided inside:

git clone https://github.com/openliberty/guide-kubernetes-microprofile-config.git
cd guide-kubernetes-microprofile-config

The start directory contains the starting project that you will build upon.

The finish directory contains the finished project that you will build.

Starting and preparing your cluster for deployment

Start your Kubernetes cluster.

Start your Docker Desktop environment.

Ensure that Kubernetes is running on Docker Desktop and that the context is set to docker-desktop.

Run the following command from a command-line session:

minikube start

Next, validate that you have a healthy Kubernetes environment by running the following command from the active command-line session.

kubectl get nodes

This command should return a Ready status for the master node.

You do not need to do any other step.

Run the following command to configure the Docker CLI to use Minikube’s Docker daemon. After you run this command, you will be able to interact with Minikube’s Docker daemon and build new images directly to it from your host machine:

eval $(minikube docker-env)

Deploying the microservices

The two microservices you will deploy are called system and inventory. The system microservice returns the JVM system properties of the running container and it returns the pod’s name in the HTTP header making replicas easy to distinguish from each other. The inventory microservice adds the properties from the system microservice to the inventory. This demonstrates how communication can be established between pods inside a cluster. To build these applications, navigate to the start directory and run the following command.

cd start
mvn clean package

Run the following command to download or update to the latest Open Liberty Docker image:

docker pull openliberty/open-liberty:kernel-java8-openj9-ubi

Next, run the docker build commands to build container images for your application:

docker build -t system:1.0-SNAPSHOT system/.
docker build -t inventory:1.0-SNAPSHOT inventory/.

The -t flag in the docker build command allows the Docker image to be labeled (tagged) in the name[:tag] format. The tag for an image describes the specific image version. If the optional [:tag] tag is not specified, the latest tag is created by default.

Run the following command to deploy the necessary Kubernetes resources to serve the applications.

kubectl apply -f kubernetes.yaml

When this command finishes, wait for the pods to be in the Ready state. Run the following command to view the status of the pods.

kubectl get pods

When the pods are ready, the output shows 1/1 for READY and Running for STATUS.

NAME                                   READY     STATUS    RESTARTS   AGE
system-deployment-6bd97d9bf6-6d2cj     1/1       Running   0          34s
inventory-deployment-645767664f-7gnxf  1/1       Running   0          34s

After the pods are ready, you will make requests to your services.

The default host name for Docker Desktop is localhost.

The default host name for minikube is 192.168.99.100. Otherwise it can be found using the minikube ip command.

Navigate to http://[hostname]:31000/system/properties and use the user name bob and the password bobpwd to authenticate. Replace [hostname] with the IP address or host name of your Kubernetes cluster. Open your browser’s developer console and examine the response headers.

You can also run the curl command to make requests to your microservices. To view the header, you may use the -I option in the curl command when making a request to http://[hostname]:31000/system/properties. Use the -u option to pass in the user name bob and the password bobpwd.

curl http://localhost:31000/system/properties -u bob:bobpwd -I
curl http://`minikube ip`:31000/system/properties -u bob:bobpwd -I

Observe that the X-App-Name header currently has the value system:

X-App-Name: system

Similarly, navigate to http://[hostname]:32000/inventory/systems/system-service, or use the following curl command, to add the system to your inventory.

curl http://localhost:32000/inventory/systems/system-service
curl http://`minikube ip`:32000/inventory/systems/system-service

Modifying system microservice

The system service is hardcoded to have system as the app name. You’ll make this name configurable by adding the appName member and X-App-Name header.

Replace the SystemResource class.
system/src/main/java/io/openliberty/guides/system/SystemResource.java

SystemResource.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2017, 2019 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License v1.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-v10.html
 8 *
 9 * Contributors:
10 *     IBM Corporation - Initial implementation
11 *******************************************************************************/
12// end::copyright[]
13package io.openliberty.guides.system;
14
15// CDI
16import javax.enterprise.context.RequestScoped;
17import javax.inject.Inject;
18import javax.ws.rs.GET;
19// JAX-RS
20import javax.ws.rs.Path;
21import javax.ws.rs.Produces;
22import javax.ws.rs.core.MediaType;
23import javax.ws.rs.core.Response;
24
25import org.eclipse.microprofile.config.inject.ConfigProperty;
26
27@RequestScoped
28@Path("/properties")
29public class SystemResource {
30
31  @Inject
32  // tag::ConfigProperty[]
33  @ConfigProperty(name = "APP_NAME")
34  // end::ConfigProperty[]
35  // tag::appName[]
36  private String appName;
37  // end::appName[]
38
39  @Inject
40  @ConfigProperty(name = "HOSTNAME")
41  private String hostname;
42
43  @GET
44  @Produces(MediaType.APPLICATION_JSON)
45  // tag::getProperties[]
46  public Response getProperties() {
47    return Response.ok(System.getProperties())
48      .header("X-Pod-Name", hostname)
49      // tag::header-appName[]
50      .header("X-App-Name", appName)
51      // end::header-appName[]
52      .build();
53  }
54  // end::getProperties[]
55}

The appName member was added to the header value in getProperties().

These changes use MicroProfile Config and CDI to inject the value of an environment variable called APP_NAME into the appName member of the SystemResource class.

Modifying inventory microservice

The inventory service is hardcoded to use bob and bobpwd as the credentials to authenticate against the system service. You’ll make these credentials configurable.

Replace the SystemClient class.
inventory/src/main/java/io/openliberty/guides/inventory/client/SystemClient.java

SystemClient.java

  1// tag::copyright[]
  2/*******************************************************************************
  3 * Copyright (c) 2017, 2019 IBM Corporation and others.
  4 * All rights reserved. This program and the accompanying materials
  5 * are made available under the terms of the Eclipse Public License v1.0
  6 * which accompanies this distribution, and is available at
  7 * http://www.eclipse.org/legal/epl-v10.html
  8 *
  9 * Contributors:
 10 *     IBM Corporation - Initial implementation
 11 *******************************************************************************/
 12// end::copyright[]
 13package io.openliberty.guides.inventory.client;
 14
 15import java.net.URI;
 16import java.util.Base64;
 17import java.util.Properties;
 18
 19import javax.enterprise.context.RequestScoped;
 20import javax.inject.Inject;
 21import javax.ws.rs.client.Client;
 22import javax.ws.rs.client.ClientBuilder;
 23import javax.ws.rs.client.Invocation.Builder;
 24import javax.ws.rs.core.HttpHeaders;
 25import javax.ws.rs.core.MediaType;
 26import javax.ws.rs.core.Response;
 27import javax.ws.rs.core.Response.Status;
 28
 29import org.eclipse.microprofile.config.inject.ConfigProperty;
 30
 31@RequestScoped
 32public class SystemClient {
 33
 34  // Constants for building URI to the system service.
 35  private final String SYSTEM_PROPERTIES = "/system/properties";
 36  private final String PROTOCOL = "http";
 37
 38  @Inject
 39  @ConfigProperty(name = "default.http.port")
 40  String DEFAULT_PORT;
 41
 42  // Basic Auth Credentials
 43  // tag::credentials[]
 44  // tag::system-app-username[]
 45  @Inject
 46  @ConfigProperty(name = "SYSTEM_APP_USERNAME")
 47  private String username;
 48  // end::system-app-username[]
 49
 50  // tag::system-app-password[]
 51  @Inject
 52  @ConfigProperty(name = "SYSTEM_APP_PASSWORD")
 53  private String password;
 54  // end::system-app-password[]
 55  // end::credentials[]
 56
 57  // Wrapper function that gets properties
 58  public Properties getProperties(String hostname) {
 59    String url = buildUrl(PROTOCOL, hostname, Integer.valueOf(DEFAULT_PORT), SYSTEM_PROPERTIES);
 60    Builder clientBuilder = buildClientBuilder(url);
 61    return getPropertiesHelper(clientBuilder);
 62  }
 63
 64  // tag::doc[]
 65  /**
 66   * Builds the URI string to the system service for a particular host.
 67   * @param protocol
 68   *          - http or https.
 69   * @param host
 70   *          - name of host.
 71   * @param port
 72   *          - port number.
 73   * @param path
 74   *          - Note that the path needs to start with a slash!!!
 75   * @return String representation of the URI to the system properties service.
 76   */
 77  // end::doc[]
 78  protected String buildUrl(String protocol, String host, int port, String path) {
 79    try {
 80      URI uri = new URI(protocol, null, host, port, path, null, null);
 81      return uri.toString();
 82    } catch (Exception e) {
 83      System.err.println("Exception thrown while building the URL: " + e.getMessage());
 84      return null;
 85    }
 86  }
 87
 88  // Method that creates the client builder
 89  protected Builder buildClientBuilder(String urlString) {
 90    try {
 91      Client client = ClientBuilder.newClient();
 92      Builder builder = client.target(urlString).request();
 93      return builder
 94        .header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON)
 95        .header(HttpHeaders.AUTHORIZATION, getAuthHeader());
 96    } catch (Exception e) {
 97      System.err.println("Exception thrown while building the client: " + e.getMessage());
 98      return null;
 99    }
100  }
101
102  // Helper method that processes the request
103  protected Properties getPropertiesHelper(Builder builder) {
104    try {
105      Response response = builder.get();
106      if (response.getStatus() == Status.OK.getStatusCode()) {
107        return response.readEntity(Properties.class);
108      } else {
109        System.err.println("Response Status is not OK.");
110      }
111    } catch (RuntimeException e) {
112      System.err.println("Runtime exception: " + e.getMessage());
113    } catch (Exception e) {
114      System.err.println("Exception thrown while invoking the request: " + e.getMessage());
115    }
116    return null;
117  }
118
119  private String getAuthHeader() {
120    String usernamePassword = username + ":" + password;
121    String encoded = Base64.getEncoder().encodeToString(usernamePassword.getBytes());
122    return "Basic " + encoded;
123  }
124}

The changes introduced here use MicroProfile Config and CDI to inject the value of the environment variables SYSTEM_APP_USERNAME and SYSTEM_APP_PASSWORD into the SystemClient class.

Creating a ConfigMap and Secret

There are several ways to configure an environment variable in a Docker container. You can set it directly in the Dockerfile with the ENV command. You can also set it in your kubernetes.yaml file by specifying a name and a value for the environment variable you want to set for a specific container. With these options in mind, you’re going to use a ConfigMap and Secret to set these values. These are resources provided by Kubernetes that are used as a way to provide configuration values to your containers. A benefit is that they can be reused across many different containers, even if they all require different environment variables to be set with the same value.

Create a ConfigMap to configure the app name with the following kubectl command.

kubectl create configmap sys-app-name --from-literal name=my-system

This command deploys a ConfigMap named sys-app-name to your cluster. It has a key called name with a value of my-system. The --from-literal flag allows you to specify individual key-value pairs to store in this ConfigMap. Other available options, such as --from-file and --from-env-file, provide more versatility as to what you want to configure. Details about these options can be found in the Kubernetes CLI documentation.

Create a Secret to configure the credentials that inventory will use to authenticate against system with the following kubectl command.

kubectl create secret generic sys-app-credentials --from-literal username=bob --from-literal password=bobpwd

This command looks very similar to the command to create a ConfigMap, one difference is the word generic. It means you’re creating a Secret that is generic, in other words it stores information that isn’t specialized in any way. There are different types of secrets, such as secrets to store Docker credentials and secrets to store public and private key pairs.

A Secret is similar to a ConfigMap. A key difference is that a Secret is used for confidential information such as credentials. One of the main differences is that you must explicitly tell kubectl to show you the contents of a Secret. Additionally, when it does show you the information, it only shows you a Base64 encoded version so that a casual onlooker doesn’t accidentally see any sensitive data. Secrets don’t provide any encryption by default, that is something you’ll either need to do yourself or find an alternate option to configure.

kubernetes.yaml

 1apiVersion: apps/v1
 2kind: Deployment
 3metadata:
 4  name: system-deployment
 5  labels:
 6    app: system
 7spec:
 8  selector:
 9    matchLabels:
10      app: system
11  template:
12    metadata:
13      labels:
14        app: system
15    spec:
16      containers:
17      - name: system-container
18        image: system:1.0-SNAPSHOT
19        ports:
20        - containerPort: 9080
21        # Set the APP_NAME environment variable
22        env:
23        - name: APP_NAME
24          valueFrom:
25            configMapKeyRef:
26              name: sys-app-name
27              key: name
28---
29apiVersion: apps/v1
30kind: Deployment
31metadata:
32  name: inventory-deployment
33  labels:
34    app: inventory
35spec:
36  selector:
37    matchLabels:
38      app: inventory
39  template:
40    metadata:
41      labels:
42        app: inventory
43    spec:
44      containers:
45      - name: inventory-container
46        image: inventory:1.0-SNAPSHOT
47        ports:
48        - containerPort: 9080
49        # Set the SYSTEM_APP_USERNAME and SYSTEM_APP_PASSWORD environment variables
50        env:
51        - name: SYSTEM_APP_USERNAME
52          valueFrom:
53            secretKeyRef:
54              name: sys-app-credentials
55              key: username
56        - name: SYSTEM_APP_PASSWORD
57          valueFrom:
58            secretKeyRef:
59              name: sys-app-credentials
60              key: password
61---
62apiVersion: v1
63kind: Service
64metadata:
65  name: system-service
66spec:
67  type: NodePort
68  selector:
69    app: system
70  ports:
71  - protocol: TCP
72    port: 9080
73    targetPort: 9080
74    nodePort: 31000
75---
76apiVersion: v1
77kind: Service
78metadata:
79  name: inventory-service
80spec:
81  type: NodePort
82  selector:
83    app: inventory
84  ports:
85  - protocol: TCP
86    port: 9080
87    targetPort: 9080
88    nodePort: 32000

Dockerfile

 1FROM openliberty/open-liberty:kernel-java8-openj9-ubi
 2
 3ARG VERSION=1.0
 4ARG REVISION=SNAPSHOT
 5LABEL \
 6  org.opencontainers.image.authors="My Name" \
 7  org.opencontainers.image.vendor="Open Liberty" \
 8  org.opencontainers.image.url="local" \
 9  org.opencontainers.image.source="https://github.com/OpenLiberty/guide-kubernetes-microprofile-config" \
10  org.opencontainers.image.version="$VERSION" \
11  org.opencontainers.image.revision="$REVISION" \
12  vendor="Open Liberty" \
13  name="system" \
14  version="$VERSION-$REVISION" \
15  summary="The system microservice from the Configuring microservices running in Kubernetes guide" \
16  description="This image contains the system microservice running with the Open Liberty runtime."
17COPY --chown=1001:0 src/main/liberty/config /config/
18COPY --chown=1001:0 target/system.war /config/apps
19
20RUN configure.sh

Updating Kubernetes resources

Next, you will update your Kubernetes deployments to set the environment variables in your containers based on the values configured in the ConfigMap and Secret created previously. The env section under the system-container is where the APP_NAME environment variable will be set. The env section under the inventory-container is where the SYSTEM_APP_USERNAME and SYSTEM_APP_PASSWORD environment variables will be set.

Replace the kubernetes file.
kubernetes.yaml

kubernetes.yaml

 1apiVersion: apps/v1
 2kind: Deployment
 3metadata:
 4  name: system-deployment
 5  labels:
 6    app: system
 7spec:
 8  selector:
 9    matchLabels:
10      app: system
11  template:
12    metadata:
13      labels:
14        app: system
15    spec:
16      containers:
17      - name: system-container
18        image: system:1.0-SNAPSHOT
19        ports:
20        - containerPort: 9080
21        # Set the APP_NAME environment variable
22        env:
23        - name: APP_NAME
24          valueFrom:
25            configMapKeyRef:
26              name: sys-app-name
27              key: name
28---
29apiVersion: apps/v1
30kind: Deployment
31metadata:
32  name: inventory-deployment
33  labels:
34    app: inventory
35spec:
36  selector:
37    matchLabels:
38      app: inventory
39  template:
40    metadata:
41      labels:
42        app: inventory
43    spec:
44      containers:
45      - name: inventory-container
46        image: inventory:1.0-SNAPSHOT
47        ports:
48        - containerPort: 9080
49        # Set the SYSTEM_APP_USERNAME and SYSTEM_APP_PASSWORD environment variables
50        env:
51        - name: SYSTEM_APP_USERNAME
52          valueFrom:
53            secretKeyRef:
54              name: sys-app-credentials
55              key: username
56        - name: SYSTEM_APP_PASSWORD
57          valueFrom:
58            secretKeyRef:
59              name: sys-app-credentials
60              key: password
61---
62apiVersion: v1
63kind: Service
64metadata:
65  name: system-service
66spec:
67  type: NodePort
68  selector:
69    app: system
70  ports:
71  - protocol: TCP
72    port: 9080
73    targetPort: 9080
74    nodePort: 31000
75---
76apiVersion: v1
77kind: Service
78metadata:
79  name: inventory-service
80spec:
81  type: NodePort
82  selector:
83    app: inventory
84  ports:
85  - protocol: TCP
86    port: 9080
87    targetPort: 9080
88    nodePort: 32000

In the kubernetes.yaml file where the containers are defined, you can see the valueFrom field that allows you to specify the value of an environment variable from various sources. These sources include a ConfigMap, a Secret, and information about the cluster. In this example configMapKeyRef gets the value name from the ConfigMap sys-app-name. Similarly, secretKeyRef gets the values username and password from the Secret sys-app-credentials.

Deploying your changes

Rebuild the application using mvn clean package.

mvn clean package

Run the docker build commands to rebuild container images for your application:

docker build -t system:1.0-SNAPSHOT system/.
docker build -t inventory:1.0-SNAPSHOT inventory/.

Run the following command to deploy your changes to the Kubernetes cluster.

kubectl replace --force -f kubernetes.yaml

Navigate to http://[hostname]:31000/system/properties and examine the response headers in your developer console, or run the following curl command:

curl http://localhost:31000/system/properties -u bob:bobpwd -I
curl http://`minikube ip`:31000/system/properties -u bob:bobpwd -I

You will see that the app name has changed from system to my-system.

X-App-Name: my-system

Verify that http://[hostname]:32000/inventory/systems/system-service is working as intended. If it is not, then check the configuration of the credentials.

Testing the microservices

Run the integration tests against a cluster running with appName my-system:

mvn failsafe:integration-test -Dsystem.appName=my-system

Run the integration tests against a cluster running at Minikube’s IP address and with appName my-system:

mvn failsafe:integration-test -Dcluster.ip=`minikube ip` -Dsystem.appName=my-system

The tests check that the system service responds with a header containing the configured name. The tests for inventory verify that the service may communicate with system using the configured credentials. If the credentials are misconfigured, then the inventory test will fail, so the inventory test indirectly verifies the credentials are correctly configured.

After the tests succeed, you should see output similar to the following in your console.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.system.SystemEndpointIT
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.709 s - in it.io.openliberty.guides.system.SystemEndpointIT

Results:

Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running it.io.openliberty.guides.inventory.InventoryEndpointIT
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.696 s - in it.io.openliberty.guides.inventory.InventoryEndpointIT

Results:

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

Tearing down the environment

Run the following commands to delete all the resources that you created.

kubectl delete -f kubernetes.yaml
kubectl delete configmap sys-app-name
kubectl delete secret sys-app-credentials

Nothing more needs to be done for Docker Desktop.

Perform the following steps to return your environment to a clean state.

  1. Point the Docker daemon back to your local machine:

    eval $(minikube docker-env -u)
  2. Stop your Minikube cluster:

    minikube stop
  3. Delete your cluster:

    minikube delete

Great work! You’re done!

You have used MicroProfile Config to externalize the configuration of two microservices, and then you configured them by creating a ConfigMap and Secret in your Kubernetes cluster.

Guide Attribution

Configuring microservices running in Kubernetes by Open Liberty is licensed under CC BY-ND 4.0

Copied to clipboard
Copy code block
Copy file contents

Prerequisites:

Nice work! Where to next?

What did you think of this guide?

Extreme Dislike Dislike Like Extreme Like

What could make this guide better?

Raise an issue to share feedback

Create a pull request to contribute to this guide

Need help?

Ask a question on Stack Overflow

Like Open Liberty? Star our repo on GitHub.

Star