Enabling distributed tracing in microservices with Zipkin

20 minutes

Prerequisites:

Table of contents

Explore how to enable and customize tracing of JAX-RS and non-JAX-RS methods by using MicroProfile OpenTracing and the Zipkin tracing system.

What you’ll learn

You’ll learn how to enable automatic tracing for JAX-RS methods and how to create custom tracers for non-JAX-RS methods by using MicroProfile OpenTracing.

OpenTracing is a standard API for instrumenting microservices for distributed tracing. Distributed tracing helps troubleshoot microservices by examining and logging requests as they propagate through a distributed system. Distributed tracing allows developers to tackle the otherwise difficult task of debugging these requests. Without a distributed tracing system in place, analyzing the workflows of operations becomes difficult. Pinpointing when and where a request is received and when responses are sent becomes difficult.

MicroProfile OpenTracing enables distributed tracing in microservices without adding any explicit distributed tracing code to the application. Note that the MicroProfile OpenTracing specification does not address the problem of defining, implementing, or configuring the underlying distributed tracing system. Rather, the specification makes it easier to instrument services with distributed tracing given an existing distributed tracing system.

You’ll configure the provided inventory and system services to use distributed tracing with MicroProfile OpenTracing. You’ll run these services in two separate JVMs made of two server instances to demonstrate tracing in a distributed environment. If all the components were to run on a single server, then any logging software would do the trick.

Getting started

The fastest way to work through this guide is to clone the Git repository and use the projects that are provided inside:

git clone https://github.com/openliberty/guide-microprofile-opentracing.git
cd guide-microprofile-opentracing

The start directory contains the starting project that you will build upon.

The finish directory contains the finished project that you will build.

Before you begin, make sure you have all the necessary prerequisites.

For this guide, Zipkin is used as the distributed tracing system. You can find the installation instructions for Zipkin at the Zipkin quickstart page. You’re not required to use Zipkin. You may choose to use another tracing system. However, this guide is written using Zipkin. If you use a different tracing system, the required instructions may differ.

Before you continue, confirm your Zipkin server is up and running. By default, Zipkin can be found at the http://localhost:9411 URL.

Try what you’ll build

The finish directory in the root directory of this guide contains two services that are configured to use MicroProfile OpenTracing. Give them a try before you continue.

To try out the services, navigate to the finish directory and run the Maven install phase to build the services

cd finish
mvn install

Then, run the Maven liberty:start-server goal to start them in two Open Liberty servers:

mvn liberty:start-server

Make sure your Zipkin server is running and point your browser to the http://localhost:9081/inventory/systems/localhost URL. When you visit this URL, you make two HTTP GET requests, one to the system service and one to the inventory service. Both of these requests are configured to be traced, so a new trace will be recorded in Zipkin. Visit the http://localhost:9411 URL or another location where you configured Zipkin to run. Run an empty query and sort the traces by latest start time first.

Verify that the new trace contains three spans with the following names:

get:io.openliberty.guides.inventory.inventoryresource.getpropertiesforhost
get:io.openliberty.guides.system.systemresource.getproperties
add() span

You can inspect each span by clicking it to reveal more detailed information, such as the time at which the request was received and the time at which a response was sent back.

If you examine the other traces, you might notice a red trace entry, which indicates the span caught an error. In this case, one of the tests accesses the /inventory/systems/badhostname endpoint, which is invalid, so an error is thrown. This behavior is expected.

When you’re done checking out the services, stop both Open Liberty servers using the Maven liberty:stop-server goal:

mvn liberty:stop-server

Running the services

Navigate to the start directory to begin.

You’ll need to start the services to see basic traces appear in Zipkin. So, before you proceed, build and start the provided system and inventory services in the starting project by running the Maven install goal:

mvn install

Then, run the liberty:start-server goal:

mvn liberty:start-server

When the servers start, you can find the system and inventory services at the following URLs:

Existing Tracer implementation

To collect traces across your systems, you need to implement the OpenTracing Tracer interface. For this guide, you can access a bare-bones Tracer implementation for the Zipkin server in the form of a user feature for Open Liberty.

This feature is already configured for you in your pom.xml and server.xml files. It’s automatically downloaded and installed into each service when you run a Maven build. You can find the opentracingZipkin feature enabled in your server.xml file.

The download-maven-plugin Maven plug-in in your pom.xml downloads and installs the opentracingZipkin feature.

If you want to install this feature yourself, see Enabling distributed tracing in IBM Documentation.

server.xml

 1<server description="Sample Liberty server">
 2
 3  <featureManager>
 4    <feature>jaxrs-2.1</feature>
 5    <feature>jsonp-1.1</feature>
 6    <feature>cdi-2.0</feature>
 7    <!-- tag::mpOpenTracing[] -->
 8    <feature>mpOpenTracing-2.0</feature>
 9    <!-- end::mpOpenTracing[] -->
10    <!-- tag::zipkinUsr[] -->
11    <feature>usr:opentracingZipkin-0.33</feature>
12    <!-- end::zipkinUsr[] -->
13  </featureManager>
14
15  <opentracingZipkin host="localhost"/>
16
17  <httpEndpoint httpPort="${default.http.port}" httpsPort="${default.https.port}"
18      id="defaultHttpEndpoint" host="*" />
19      
20  <webApplication location="inventory-service.war" contextRoot="/">
21    <!-- enable visibility to third party apis -->
22    <classloader apiTypeVisibility="api,ibm-api,spec,stable,third-party"/>
23  </webApplication>
24
25</server>

pom.xml

  1<?xml version='1.0' encoding='utf-8'?>
  2<project xmlns="http://maven.apache.org/POM/4.0.0" 
  3xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  4xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
  5http://maven.apache.org/xsd/maven-4.0.0.xsd">
  6    <modelVersion>4.0.0</modelVersion>
  7
  8    <parent>
  9        <groupId>io.openliberty.guides</groupId>
 10        <artifactId>guide-microprofile-opentracing</artifactId>
 11        <version>1.0-SNAPSHOT</version>
 12    </parent>
 13
 14    <artifactId>inventory-service</artifactId>
 15    <packaging>war</packaging>
 16
 17    <properties>
 18        <!-- Plugins -->
 19        <version.exec-maven-plugin>1.6.0</version.exec-maven-plugin>
 20    </properties>
 21
 22    <dependencies>
 23        <!-- Open Liberty features -->
 24        <dependency>
 25            <groupId>io.openliberty.features</groupId>
 26            <artifactId>jaxrs-2.1</artifactId>
 27            <type>esa</type>
 28        </dependency>
 29        <dependency>
 30            <groupId>io.openliberty.features</groupId>
 31            <artifactId>jsonp-1.1</artifactId>
 32            <type>esa</type>
 33        </dependency>
 34        <dependency>
 35            <groupId>io.openliberty.features</groupId>
 36            <artifactId>cdi-2.0</artifactId>
 37            <type>esa</type>
 38        </dependency>
 39        <dependency>
 40            <groupId>io.openliberty.features</groupId>
 41            <artifactId>mpOpenTracing-2.0</artifactId>
 42            <type>esa</type>
 43        </dependency>
 44        <!-- For tests -->
 45        <dependency>
 46            <groupId>junit</groupId>
 47            <artifactId>junit</artifactId>
 48        </dependency>
 49        <dependency>
 50            <groupId>org.apache.cxf</groupId>
 51            <artifactId>cxf-rt-rs-client</artifactId>
 52        </dependency>
 53        <dependency>
 54            <groupId>org.apache.cxf</groupId>
 55            <artifactId>cxf-rt-rs-extension-providers</artifactId>
 56        </dependency>
 57        <dependency>
 58            <groupId>org.glassfish</groupId>
 59            <artifactId>javax.json</artifactId>
 60        </dependency>
 61        <dependency>
 62            <groupId>org.apache.commons</groupId>
 63            <artifactId>commons-lang3</artifactId>
 64        </dependency>
 65    </dependencies>
 66
 67    <profiles>
 68        <!-- Start system service before running tests -->
 69        <profile>
 70            <id>setup-test</id>
 71            <activation>
 72                <activeByDefault>true</activeByDefault>
 73            </activation>
 74            <build>
 75                <plugins>
 76                    <plugin>
 77                        <groupId>org.codehaus.mojo</groupId>
 78                        <artifactId>exec-maven-plugin</artifactId>
 79                        <version>${version.exec-maven-plugin}</version>
 80                        <executions>
 81                            <execution>
 82                                <id>test-start-system-service</id>
 83                                <goals>
 84                                    <goal>exec</goal>
 85                                </goals>
 86                                <phase>pre-integration-test</phase>
 87                                <configuration>
 88                                    <workingDirectory>../system</workingDirectory>
 89                                    <executable>mvn${mvn.extension}</executable>
 90                                    <arguments>
 91                                        <argument>liberty:start-server</argument>
 92                                    </arguments>
 93                                </configuration>
 94                            </execution>
 95                        </executions>
 96                    </plugin>
 97                </plugins>
 98            </build>
 99        </profile>
100        <!-- Stop system service after running tests -->
101        <profile>
102            <id>teardown-test</id>
103            <activation>
104                <activeByDefault>false</activeByDefault>
105            </activation>
106            <build>
107                <plugins>
108                    <plugin>
109                        <groupId>org.codehaus.mojo</groupId>
110                        <artifactId>exec-maven-plugin</artifactId>
111                        <version>${version.exec-maven-plugin}</version>
112                        <executions>
113                            <execution>
114                                <id>test-stop-system-service</id>
115                                <goals>
116                                    <goal>exec</goal>
117                                </goals>
118                                <phase>post-integration-test</phase>
119                                <configuration>
120                                    <workingDirectory>../system</workingDirectory>
121                                    <executable>mvn${mvn.extension}</executable>
122                                    <arguments>
123                                        <argument>liberty:stop-server</argument>
124                                    </arguments>
125                                </configuration>
126                            </execution>
127                        </executions>
128                    </plugin>
129                </plugins>
130            </build>
131        </profile>
132    </profiles>
133
134    <build>
135        <plugins>
136            <plugin>
137                <groupId>org.apache.maven.plugins</groupId>
138                <artifactId>maven-war-plugin</artifactId>
139            </plugin>
140
141            <!-- Liberty plugin -->
142            <plugin>
143                <groupId>net.wasdev.wlp.maven.plugins</groupId>
144                <artifactId>liberty-maven-plugin</artifactId>
145                <configuration>
146                    <configFile>src/main/liberty/config/server.xml</configFile>
147                    <include>usr</include>
148                    <bootstrapProperties>
149                        <default.http.port>${inv.service.http.port}</default.http.port>
150                        <default.https.port>${inv.service.https.port}</default.https.port>
151                        <system.http.port>${sys.service.http.port}</system.http.port>
152                    </bootstrapProperties>
153                </configuration>
154                <executions>
155                    <execution>
156                        <id>install-apps</id>
157                        <configuration>
158                            <appsDirectory>apps</appsDirectory>
159                            <stripVersion>true</stripVersion>
160                            <installAppPackages>project</installAppPackages>
161                            <looseApplication>true</looseApplication>
162                        </configuration>
163                    </execution>
164                    <execution>
165                        <id>install-feature</id>
166                        <configuration>
167                            <skip>true</skip>
168                        </configuration>
169                    </execution>
170                </executions>
171            </plugin>
172
173            <!-- For downloading files -->
174            <!-- tag::download[] -->
175            <plugin>
176                <groupId>com.googlecode.maven-download-plugin</groupId>
177                <artifactId>download-maven-plugin</artifactId>
178                <version>${version.download-maven-plugin}</version>
179                <executions>
180                    <execution>
181                        <id>install-tracer</id>
182                        <phase>prepare-package</phase>
183                        <goals>
184                            <goal>wget</goal>
185                        </goals>
186                        <configuration>
187                            <url>${zipkin.usr.feature}</url>
188                            <unpack>true</unpack>
189                            <outputDirectory>
190                                ${project.build.directory}/liberty/wlp/usr
191                            </outputDirectory>
192                        </configuration>
193                    </execution>
194                </executions>
195            </plugin>
196            <!-- end::download[] -->
197
198            <!-- Plugin to run unit tests -->
199            <plugin>
200                <groupId>org.apache.maven.plugins</groupId>
201                <artifactId>maven-surefire-plugin</artifactId>
202            </plugin>
203
204            <!-- Plugin to run functional tests -->
205            <plugin>
206                <groupId>org.apache.maven.plugins</groupId>
207                <artifactId>maven-failsafe-plugin</artifactId>
208            </plugin>
209        </plugins>
210    </build>
211</project>

Enabling distributed tracing

The MicroProfile OpenTracing feature enables tracing of all JAX-RS methods by default. To further control and customize these traces, use the @Traced annotation to enable and disable tracing of particular methods. You can also inject a custom Tracer object to create and customize spans.

Enabling distributed tracing without code instrumentation

Because tracing is enabled by default for all JAX-RS methods, you need to enable only the mpOpenTracing feature and the usr:opentracingZipkin user feature in the server.xml file to see some basic traces in Zipkin.

Both of these features are already enabled in the inventory and system configuration files.

Make sure your services are running. Then, point your browser to any of their endpoints and check your Zipkin server for traces.

server.xml

 1<server description="Sample Liberty server">
 2
 3  <featureManager>
 4    <feature>jaxrs-2.1</feature>
 5    <feature>jsonp-1.1</feature>
 6    <feature>cdi-2.0</feature>
 7    <!-- tag::mpOpenTracing[] -->
 8    <feature>mpOpenTracing-2.0</feature>
 9    <!-- end::mpOpenTracing[] -->
10    <!-- tag::zipkinUsr[] -->
11    <feature>usr:opentracingZipkin-0.33</feature>
12    <!-- end::zipkinUsr[] -->
13  </featureManager>
14
15  <opentracingZipkin host="localhost"/>
16
17  <httpEndpoint httpPort="${default.http.port}" httpsPort="${default.https.port}"
18      id="defaultHttpEndpoint" host="*" />
19      
20  <webApplication location="inventory-service.war" contextRoot="/">
21    <!-- enable visibility to third party apis -->
22    <classloader apiTypeVisibility="api,ibm-api,spec,stable,third-party"/>
23  </webApplication>
24
25</server>

Enabling explicit distributed tracing

The @Traced annotation defines explicit span creation for specific classes and methods. If you place the annotation on a class, then it’s automatically applied to all methods within that class. If you place the annotation on a method, then it overrides the class annotation if one exists.

Enable tracing of the list() non-JAX-RS method by adding the @Traced annotation to the method.

Replace the InventoryManager class.
inventory/src/main/java/io/openliberty/guides/inventory/InventoryManager.java

InventoryManager.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2017, 2021 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License v1.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-v10.html
 8 *
 9 * Contributors:
10 *     IBM Corporation - Initial implementation
11 *******************************************************************************/
12// end::copyright[]
13package io.openliberty.guides.inventory;
14
15import java.util.ArrayList;
16import java.util.Collections;
17import java.util.List;
18import java.util.Properties;
19import javax.enterprise.context.ApplicationScoped;
20import javax.inject.Inject;
21
22import io.openliberty.guides.inventory.client.SystemClient;
23import io.openliberty.guides.inventory.model.InventoryList;
24import io.openliberty.guides.inventory.model.SystemData;
25import io.opentracing.Scope;
26import io.opentracing.Span;
27import io.opentracing.Tracer;
28
29import org.eclipse.microprofile.opentracing.Traced;
30
31@ApplicationScoped
32// tag::InventoryManager[]
33public class InventoryManager {
34
35    private List<SystemData> systems = Collections.synchronizedList(new ArrayList<>());
36    private SystemClient systemClient = new SystemClient();
37    // tag::customTracer[]
38    @Inject Tracer tracer;
39    // end::customTracer[]
40
41    public Properties get(String hostname) {
42        systemClient.init(hostname, 9080);
43        Properties properties = systemClient.getProperties();
44        return properties;
45    }
46
47    public void add(String hostname, Properties systemProps) {
48        Properties props = new Properties();
49        props.setProperty("os.name", systemProps.getProperty("os.name"));
50        props.setProperty("user.name", systemProps.getProperty("user.name"));
51
52        SystemData system = new SystemData(hostname, props);
53        // tag::Add[]
54        if (!systems.contains(system)) {
55            // tag::span[]
56            Span span = tracer.buildSpan("add() Span").start();
57            // end::span[]
58            // tag::Try[]
59            // tag::scope[]
60            try (Scope childScope = tracer.scopeManager()
61                                          .activate(span)
62                ) {
63            // end::scope[]
64                // tag::addToInvList[]
65                systems.add(system);
66                // end::addToInvList[]
67            } finally {
68                // tag::spanFinish[]
69                span.finish();
70                // end::spanFinish[]
71            }
72            // end::Try[]
73        }
74        // end::Add[]
75    }
76
77    // tag::Traced[]
78    @Traced(value = true, operationName = "InventoryManager.list")
79    // end::Traced[]
80    // tag::list[]
81    public InventoryList list() {
82        return new InventoryList(systems);
83    }
84    // end::list[]
85}
86// end::InventoryManager[]

The @Traced annotation can be configured with the following two parameters:

The value=[true|false] parameter indicates whether a particular class or method is traced. For example, while all JAX-RS methods are traced by default, you can disable their tracing by using the @Traced(false) annotation. This parameter is set to true by default.
The operationName=<Span name> parameter indicates the name of the span that is assigned to the particular method that is traced. If you omit this parameter, the span will be named with the following form: <package name>.<class name>.<method name>. If you use this parameter at a class level, then all methods within that class will have the same span name unless they’re explicitly overridden by another @Traced annotation.

Next, run the following command from the start directory to recompile your services.

mvn compile

Visit the http://localhost:9081/inventory/systems URL, check your Zipkin server, and sort the traces by newest first.

Look for a new trace record that is two spans long with one span for the listContents() JAX-RS method in the InventoryResource class and another span for the list() method in the InventoryManager class. Verify that these spans have the following names:

get:io.openliberty.guides.inventory.inventoryresource.listcontents
inventorymanager.list

Now, disable tracing on the InventoryResource class by setting @Traced(false) on the listContents() JAX-RS method.

Replace the InventoryResource class.
inventory/src/main/java/io/openliberty/guides/inventory/InventoryResource.java

InventoryResource.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2017, 2020 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License v1.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-v10.html
 8 *
 9 * Contributors:
10 *     IBM Corporation - Initial implementation
11 *******************************************************************************/
12// end::copyright[]
13package io.openliberty.guides.inventory;
14
15import java.util.Properties;
16import javax.enterprise.context.RequestScoped;
17import javax.inject.Inject;
18import javax.ws.rs.GET;
19import javax.ws.rs.Path;
20import javax.ws.rs.PathParam;
21import javax.ws.rs.Produces;
22import javax.ws.rs.core.MediaType;
23import javax.ws.rs.core.Response;
24
25import org.eclipse.microprofile.opentracing.Traced;
26
27import io.openliberty.guides.inventory.model.InventoryList;
28
29@RequestScoped
30@Path("/systems")
31// tag::InventoryResource[]
32public class InventoryResource {
33
34    @Inject InventoryManager manager;
35
36    @GET
37    @Path("/{hostname}")
38    @Produces(MediaType.APPLICATION_JSON)
39    // tag::getPropertiesForHost[]
40    public Response getPropertiesForHost(@PathParam("hostname") String hostname) {
41        Properties props = manager.get(hostname);
42        if (props == null) {
43            return Response.status(Response.Status.NOT_FOUND)
44                           .entity("{ \"error\" : \"Unknown hostname or the system service " 
45                           + "may not be running on " + hostname + "\" }")
46                           .build();
47        }
48        manager.add(hostname, props);
49        return Response.ok(props).build();
50    }
51    // end::getPropertiesForHost[]
52    
53    @GET
54    // tag::Traced-false[]
55    @Traced(false)
56    // end::Traced-false[]
57    @Produces(MediaType.APPLICATION_JSON)
58    // tag::listContents[]
59    public InventoryList listContents() {
60        return manager.list();
61    }
62    // end::listContents[]
63}
64// end::InventoryResource[]

Again, run the mvn compile command from the start directory to recompile your services:

mvn compile

Visit the http://localhost:9081/inventory/systems URL again, check your Zipkin server, and sort the traces by newest first.

Look for a new trace record that is just one span long for the remaining list() method in the InventoryManager class. Verify that this span has the following name:

inventorymanager.list

Injecting a custom Tracer object

The MicroProfile OpenTracing specification also makes the underlying OpenTracing Tracer instance available. The configured Tracer is accessed by injecting it into a bean by using the @Inject annotation from the Contexts and Dependency Injections API.

After injecting it, the Tracer will be used to build a Span. The Span will be activated and used in a Scope.

Replace the InventoryManager class.
inventory/src/main/java/io/openliberty/guides/inventory/InventoryManager.java

InventoryManager.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2017, 2021 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License v1.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-v10.html
 8 *
 9 * Contributors:
10 *     IBM Corporation - Initial implementation
11 *******************************************************************************/
12// end::copyright[]
13package io.openliberty.guides.inventory;
14
15import java.util.ArrayList;
16import java.util.Collections;
17import java.util.List;
18import java.util.Properties;
19import javax.enterprise.context.ApplicationScoped;
20import javax.inject.Inject;
21
22import io.openliberty.guides.inventory.client.SystemClient;
23import io.openliberty.guides.inventory.model.InventoryList;
24import io.openliberty.guides.inventory.model.SystemData;
25import io.opentracing.Scope;
26import io.opentracing.Span;
27import io.opentracing.Tracer;
28
29import org.eclipse.microprofile.opentracing.Traced;
30
31@ApplicationScoped
32// tag::InventoryManager[]
33public class InventoryManager {
34
35    private List<SystemData> systems = Collections.synchronizedList(new ArrayList<>());
36    private SystemClient systemClient = new SystemClient();
37    // tag::customTracer[]
38    @Inject Tracer tracer;
39    // end::customTracer[]
40
41    public Properties get(String hostname) {
42        systemClient.init(hostname, 9080);
43        Properties properties = systemClient.getProperties();
44        return properties;
45    }
46
47    public void add(String hostname, Properties systemProps) {
48        Properties props = new Properties();
49        props.setProperty("os.name", systemProps.getProperty("os.name"));
50        props.setProperty("user.name", systemProps.getProperty("user.name"));
51
52        SystemData system = new SystemData(hostname, props);
53        // tag::Add[]
54        if (!systems.contains(system)) {
55            // tag::span[]
56            Span span = tracer.buildSpan("add() Span").start();
57            // end::span[]
58            // tag::Try[]
59            // tag::scope[]
60            try (Scope childScope = tracer.scopeManager()
61                                          .activate(span)
62                ) {
63            // end::scope[]
64                // tag::addToInvList[]
65                systems.add(system);
66                // end::addToInvList[]
67            } finally {
68                // tag::spanFinish[]
69                span.finish();
70                // end::spanFinish[]
71            }
72            // end::Try[]
73        }
74        // end::Add[]
75    }
76
77    // tag::Traced[]
78    @Traced(value = true, operationName = "InventoryManager.list")
79    // end::Traced[]
80    // tag::list[]
81    public InventoryList list() {
82        return new InventoryList(systems);
83    }
84    // end::list[]
85}
86// end::InventoryManager[]

The Scope is used in a try block. The try block that you see here is called a try-with-resources statement, meaning that the Scope object is closed at the end of the statement. Defining custom spans inside such statements is a good practice. Otherwise, any exceptions that are thrown before the span is closed will leak the active span. The finish() method sets the ending timestamp and records the span.

Next, run the following command from the start directory to recompile your services.

mvn compile

Visit the http://localhost:9081/inventory/systems/localhost URL, check your Zipkin server, and sort the traces by newest first.

Look for two new trace records, one for the system service and one for the inventory service. The system trace contains one span for the getProperties() method in the SystemResource class. The inventory trace contains two spans. The first span is for the getPropertiesForHost() method in the InventoryResource class. The second span is the custom span that you created around the add() call. Verify that all of these spans have the following names:

The system trace:

get:io.openliberty.guides.system.systemresource.getproperties

The inventory trace:

get:io.openliberty.guides.inventory.inventoryresource.getpropertiesforhost
add() span

This simple example shows what you can do with the injected Tracer object. More configuration options are available, including setting a timestamp for when a span was created and destroyed. However, these options require an implementation of their own, which does not come as a part of the Zipkin user feature that is provided. In a real-world scenario, implement all the OpenTracing interfaces that you consider necessary, which might include the SpanBuilder interface. You can use this interface for span creation and customization, including setting timestamps.

SystemResource.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2017, 2020 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License v1.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-v10.html
 8 *
 9 * Contributors:
10 *     IBM Corporation - Initial implementation
11 *******************************************************************************/
12// end::copyright[]
13package io.openliberty.guides.system;
14
15import java.util.Properties;
16
17// CDI
18import javax.enterprise.context.RequestScoped;
19import javax.ws.rs.GET;
20// JAX-RS
21import javax.ws.rs.Path;
22import javax.ws.rs.Produces;
23import javax.ws.rs.core.MediaType;
24
25@RequestScoped
26@Path("properties")
27public class SystemResource {
28
29  @GET
30  @Produces(MediaType.APPLICATION_JSON)
31  // tag::Properties[]
32  public Properties getProperties() {
33    return System.getProperties();
34  }
35  // end::Properties[]
36}

InventoryResource.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2017, 2020 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License v1.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-v10.html
 8 *
 9 * Contributors:
10 *     IBM Corporation - Initial implementation
11 *******************************************************************************/
12// end::copyright[]
13package io.openliberty.guides.inventory;
14
15import java.util.Properties;
16import javax.enterprise.context.RequestScoped;
17import javax.inject.Inject;
18import javax.ws.rs.GET;
19import javax.ws.rs.Path;
20import javax.ws.rs.PathParam;
21import javax.ws.rs.Produces;
22import javax.ws.rs.core.MediaType;
23import javax.ws.rs.core.Response;
24
25import org.eclipse.microprofile.opentracing.Traced;
26
27import io.openliberty.guides.inventory.model.InventoryList;
28
29@RequestScoped
30@Path("/systems")
31// tag::InventoryResource[]
32public class InventoryResource {
33
34    @Inject InventoryManager manager;
35
36    @GET
37    @Path("/{hostname}")
38    @Produces(MediaType.APPLICATION_JSON)
39    // tag::getPropertiesForHost[]
40    public Response getPropertiesForHost(@PathParam("hostname") String hostname) {
41        Properties props = manager.get(hostname);
42        if (props == null) {
43            return Response.status(Response.Status.NOT_FOUND)
44                           .entity("{ \"error\" : \"Unknown hostname or the system service " 
45                           + "may not be running on " + hostname + "\" }")
46                           .build();
47        }
48        manager.add(hostname, props);
49        return Response.ok(props).build();
50    }
51    // end::getPropertiesForHost[]
52    
53    @GET
54    // tag::Traced-false[]
55    @Traced(false)
56    // end::Traced-false[]
57    @Produces(MediaType.APPLICATION_JSON)
58    // tag::listContents[]
59    public InventoryList listContents() {
60        return manager.list();
61    }
62    // end::listContents[]
63}
64// end::InventoryResource[]

Testing the services

No automated tests are provided to verify the correctness of the traces. Manually verify these traces by viewing them on the Zipkin server.

A few tests are included for you to test the basic functionality of the services. If a test failure occurs, then you might have introduced a bug into the code. These tests will run automatically as a part of the Maven build process when you run the mvn install command. You can also run these tests separately from the build by using the mvn verify command, but first make sure the servers are stopped.

When you’re done checking out the services, stop the server by using the Maven liberty:stop-server goal:

mvn liberty:stop-server

Great work! You’re done!

You have just used MicroProfile OpenTracing in Open Liberty to customize how and which traces are delivered to Zipkin.

Feel free to try one of the related MicroProfile guides. They demonstrate additional technologies that you can learn to expand on top of what you built here.