Enabling distributed tracing in microservices with OpenTelemetry and Jaeger

duration 20 minutes
New

Prerequisites:

Distributed tracing helps teams keep track of requests between microservices. MicroProfile Telemetry adopts OpenTelemetry tracing, so you can observe requests across your distributed systems.

What you’ll learn

The complexity of microservices architecture can make it more difficult to understand how services depend on or affect each other and to identify sources of latency or inaccuracies.

One way to increase the observability of an application is by emitting traces. OpenTelemetry is a set of APIs, SDKs, tooling, and integrations designed to create and manage telemetry data such as traces, metrics, and logs. MicroProfile Telemetry adopts OpenTelemetry so your Java applications can benefit from both manual and automatic traces.

Traces represent requests, which can contain multiple operations or spans. Each span comprises a name, time-related data, log messages, and metadata that describe what occurred during a transaction. Spans are associated with a context, which identifies the request within which the span occurred. Developers can then follow a single request between services through a potentially complex distributed system. Exporters send the data that MicroProfile Telemetry collects to Jaeger so you can visualize and monitor the generated spans.

The diagram shows multiple services, which is where distributed tracing is valuable. However, for simplicity, in this guide, you’ll configure only the system and inventory services to use Jaeger for distributed tracing with MicroProfile Telemetry. You’ll run these services in two separate JVMs made of two Open Liberty instances to demonstrate tracing in a distributed environment.

Application architecture

Additional prerequisites

Before you begin, deploy the Jaeger all-in-one executable file to start the Jaeger tracing system. The Jaeger all-in-one executable file is configured for quick local testing. Jaeger 1.46 or later is recommended. You can find information about the Jaeger server and instructions for starting the all-in-one executable file in the Jaeger documentation.

Before you proceed, make sure that your Jaeger server is up and running by visiting the http://localhost:16686 URL.

Getting started

The fastest way to work through this guide is to clone the Git repository and use the projects that are provided inside:

git clone https://github.com/openliberty/guide-microprofile-telemetry-jaeger.git
cd guide-microprofile-telemetry-jaeger

The start directory contains the starting project that you will build upon.

The finish directory contains the finished project that you will build.

Before you begin, make sure you have all the necessary prerequisites.

Try what you’ll build

The finish directory in the root of this guide contains the finished application. Give it a try before you proceed.

Open a command-line session and navigate to the finish/system directory. Run the following Maven goal to build the system service and deploy it to Open Liberty:

mvn liberty:run

Open another command-line session and navigate to the finish/inventory directory. Run the following Maven goal to build the inventory service and deploy it to Open Liberty:

mvn liberty:run

After you see the following message in both command-line sessions, both of your services are ready:

The defaultServer server is ready to run a smarter planet.

Make sure that your Jaeger server is running and navigate your browser to the http://localhost:9081/inventory/systems/localhost URL.

When you visit this endpoint, you make two GET HTTP requests, one to the system service and another to the inventory service. Both of these requests are configured to be traced, so a new trace is recorded in Jaeger. To view the traces, go to the http://localhost:16686 URL.

You can view the traces for the system or inventory services under the Search tab. If you see only the jaeger-query option in the drop-down menu, wait a little longer and refresh the page to see the application services.

Select the services in the Select A Service menu and click the Find Traces button at the end of the section. You will see the following result:

Get traces for the inventory service

The trace has five spans, four from the inventory service and one from the system service. Click the trace to view its details. Under Service & Operation, you see the spans in this trace. You can inspect each span by clicking it to reveal more detailed information, such as the times that a request was received and a response was sent.

Inventory details spans

After you’re finished reviewing the application, stop the Open Liberty instances by pressing CTRL+C in the command-line sessions where you ran the system and inventory services. Alternatively, you can run the following goals from the finish directory in another command-line session:

 mvn -pl system liberty:stop
 mvn -pl inventory liberty:stop

Building the application

You need to start the services to see basic traces appear in Jaeger.

When you run Open Liberty in dev mode, dev mode listens for file changes and automatically recompiles and deploys your updates whenever you save a new change.

Open a command-line session and navigate to the start/system directory. Run the following Maven goal to start the system service in dev mode:

mvn liberty:dev

Open a command-line session and navigate to the start/inventory directory. Run the following Maven goal to start the inventory service in dev mode:

mvn liberty:dev

After you see the following message, your Liberty instance is ready in dev mode:

**************************************************************
*    Liberty is running in dev mode.

Dev mode holds your command-line session to listen for file changes. Open another command-line session to continue, or open the project in your editor.

When the runtime instances start, you can find the system and inventory services at the following URLs:

Enabling Telemetry implementation

Navigate to the start directory to begin.

MicroProfile Telemetry allows you to observe traces without modifying the source code in your Jakarta RESTful applications. You can enable the mpTelemetry feature in the server.xml configuration file.

Replace the server.xml file of the system service:
system/src/main/liberty/config/server.xml

system/server.xml

 1<server description="system service">
 2
 3    <featureManager>
 4        <feature>cdi-4.0</feature>
 5        <feature>jsonb-3.0</feature>
 6        <feature>jsonp-2.1</feature>
 7        <feature>restfulWS-3.1</feature>
 8        <!-- tag::mpTelemetry[] -->
 9        <feature>mpTelemetry-1.1</feature>
10        <!-- end::mpTelemetry[] -->
11    </featureManager>
12
13    <httpEndpoint httpPort="${http.port}"
14                  httpsPort="${https.port}"
15                  id="defaultHttpEndpoint" host="*" />
16
17    <webApplication location="guide-microprofile-telemetry-system.war"
18                    contextRoot="/" />
19
20</server>

The mpTelemetry feature is now enabled in the server.xml of the system service.

Replace the server.xml file of the inventory service:
inventory/src/main/liberty/config/server.xml

inventory/server.xml

 1<server description="inventory service">
 2
 3    <featureManager>
 4        <feature>cdi-4.0</feature>
 5        <feature>jsonb-3.0</feature>
 6        <feature>jsonp-2.1</feature>
 7        <feature>restfulWS-3.1</feature>
 8        <feature>mpConfig-3.1</feature>
 9        <!-- tag::mpTelemetry[] -->
10        <feature>mpTelemetry-1.1</feature>
11        <!-- end::mpTelemetry[] -->
12    </featureManager>
13
14    <httpEndpoint httpPort="${http.port}"
15                  httpsPort="${https.port}"
16                  id="defaultHttpEndpoint" host="*" />
17
18    <webApplication location="guide-microprofile-telemetry-inventory.war"
19                    contextRoot="/">
20        <!-- tag::thirdPartyComment[] -->
21        <!-- enable visibility to third party apis -->
22        <!-- end::thirdPartyComment[] -->
23        <!-- tag::thirdParty[] -->
24        <classloader apiTypeVisibility="+third-party"/>
25        <!-- end::thirdParty[] -->
26    </webApplication>
27
28</server>

The mpTelemetry feature is now enabled in the server.xml of the inventory service.

By default, MicroProfile Telemetry tracing is off. To enable any tracing aspects, specify the otel properties in the MicroProfile configuration file.

Create the microprofile-config.properties file of the system service:
system/src/main/resources/META-INF/microprofile-config.properties

system/microprofile-config.properties

1# tag::service[]
2otel.service.name=system
3# end::service[]
4# tag::disabled[]
5otel.sdk.disabled=false
6# end::disabled[]

The MicroProfile properties file sets the otel.service.name property with the system service name and sets the otel.sdk.disabled property to false to enable tracing.

Create the microprofile-config.properties file of the inventory service:
inventory/src/main/resources/META-INF/microprofile-config.properties

inventory/microprofile-config.properties

1# tag::otel[]
2otel.service.name=inventory
3otel.sdk.disabled=false
4# end::otel[]

Similarly, specify the otel properties for the inventory service.

For more information about these and other Telemetry properties, see the MicroProfile Config properties for MicroProfile Telemetry documentation.

To run the system and inventory services, simply navigate your browser to the http://localhost:9081/inventory/systems/localhost URL. To view the traces, go to the http://localhost:16686 URL.

You can view the traces for the system or inventory services under the Search tab. Select the services in the Select A Service menu and click the Find Traces button at the end of the section. You’ll see the result as:

Default spans

Verify that there are two spans from the inventory service and one span from the system service. Click the trace to view its details.

Details default spans

Enabling explicit distributed tracing

Automatic instrumentation only instruments Jakarta RESTful web services and MicroProfile REST clients. To get further spans on other operations, such as database calls, you can add manual instrumentation to the source code.

Enabling OpenTelemetry APIs

The MicroProfile Telemetry feature has been enabled to trace all REST endpoints by default in the previous section. To further control and customize traces, use the @WithSpan annotation to enable particular methods. You can also inject a Tracer object to create and customize spans.

Replace the pom.xml Maven project file of the inventory service:
inventory/pom.xml

pom.xml

  1<?xml version='1.0' encoding='utf-8'?>
  2<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
  3http://maven.apache.org/xsd/maven-4.0.0.xsd">
  4    <modelVersion>4.0.0</modelVersion>
  5
  6    <groupId>io.openliberty.guides</groupId>
  7    <artifactId>guide-microprofile-telemetry-inventory</artifactId>
  8    <version>1.0-SNAPSHOT</version>
  9    <packaging>war</packaging>
 10
 11    <properties>
 12        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
 13        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
 14        <maven.compiler.source>11</maven.compiler.source>
 15        <maven.compiler.target>11</maven.compiler.target>
 16
 17        <!-- OpenLiberty runtime -->
 18        <liberty.var.system.http.port>9080</liberty.var.system.http.port>
 19        <liberty.var.http.port>9081</liberty.var.http.port>
 20        <liberty.var.https.port>9444</liberty.var.https.port>
 21    </properties>
 22
 23    <dependencies>
 24        <!-- Provided dependencies -->
 25        <dependency>
 26            <groupId>jakarta.platform</groupId>
 27            <artifactId>jakarta.jakartaee-api</artifactId>
 28            <version>10.0.0</version>
 29            <scope>provided</scope>
 30        </dependency>
 31        <dependency>
 32            <groupId>org.eclipse.microprofile</groupId>
 33            <artifactId>microprofile</artifactId>
 34            <version>6.1</version>
 35            <type>pom</type>
 36            <scope>provided</scope>
 37        </dependency>
 38        <!-- tag::opentelemetry[] -->
 39        <dependency>
 40            <groupId>io.opentelemetry</groupId>
 41            <artifactId>opentelemetry-api</artifactId>
 42            <version>1.36.0</version>
 43            <scope>provided</scope>
 44        </dependency>
 45        <dependency>
 46            <groupId>io.opentelemetry.instrumentation</groupId>
 47            <artifactId>opentelemetry-instrumentation-annotations</artifactId>
 48            <version>2.2.0</version>
 49            <scope>provided</scope>
 50        </dependency>
 51        <!-- end::opentelemetry[] -->
 52        <!-- For tests -->
 53        <dependency>
 54            <groupId>org.junit.jupiter</groupId>
 55            <artifactId>junit-jupiter</artifactId>
 56            <version>5.10.2</version>
 57            <scope>test</scope>
 58        </dependency>
 59        <dependency>
 60            <groupId>org.jboss.resteasy</groupId>
 61            <artifactId>resteasy-client</artifactId>
 62            <version>6.2.8.Final</version>
 63            <scope>test</scope>
 64        </dependency>
 65        <dependency>
 66            <groupId>org.jboss.resteasy</groupId>
 67            <artifactId>resteasy-json-binding-provider</artifactId>
 68            <version>6.2.8.Final</version>
 69            <scope>test</scope>
 70        </dependency>
 71        <dependency>
 72            <groupId>org.glassfish</groupId>
 73            <artifactId>jakarta.json</artifactId>
 74            <version>2.0.1</version>
 75            <scope>test</scope>
 76        </dependency>
 77        <dependency>
 78            <groupId>org.eclipse</groupId>
 79            <artifactId>yasson</artifactId>
 80            <version>3.0.3</version>
 81            <scope>test</scope>
 82        </dependency>
 83    </dependencies>
 84
 85    <build>
 86        <finalName>${project.artifactId}</finalName>
 87        <plugins>
 88            <plugin>
 89                <groupId>org.apache.maven.plugins</groupId>
 90                <artifactId>maven-war-plugin</artifactId>
 91                <version>3.4.0</version>
 92            </plugin>
 93
 94            <!-- Liberty plugin -->
 95            <plugin>
 96                <groupId>io.openliberty.tools</groupId>
 97                <artifactId>liberty-maven-plugin</artifactId>
 98                <version>3.10.2</version>
 99            </plugin>
100
101            <plugin>
102                <groupId>org.apache.maven.plugins</groupId>
103                <artifactId>maven-failsafe-plugin</artifactId>
104                <version>3.2.5</version>
105                <configuration>
106                    <systemPropertyVariables>
107                        <sys.http.port>${liberty.var.system.http.port}</sys.http.port>
108                        <inv.http.port>${liberty.var.http.port}</inv.http.port>
109                    </systemPropertyVariables>
110                </configuration>
111            </plugin>
112        </plugins>
113    </build>
114</project>

The OpenTelemetry API and OpenTelemetry Instrumentation Annotations must be provided as dependencies to your build path. The pom.xml now includes two io.opentelemetry dependencies.

Replace the server.xml file of the inventory service:
inventory/src/main/liberty/config/server.xml

inventory/server.xml

 1<server description="inventory service">
 2
 3    <featureManager>
 4        <feature>cdi-4.0</feature>
 5        <feature>jsonb-3.0</feature>
 6        <feature>jsonp-2.1</feature>
 7        <feature>restfulWS-3.1</feature>
 8        <feature>mpConfig-3.1</feature>
 9        <!-- tag::mpTelemetry[] -->
10        <feature>mpTelemetry-1.1</feature>
11        <!-- end::mpTelemetry[] -->
12    </featureManager>
13
14    <httpEndpoint httpPort="${http.port}"
15                  httpsPort="${https.port}"
16                  id="defaultHttpEndpoint" host="*" />
17
18    <webApplication location="guide-microprofile-telemetry-inventory.war"
19                    contextRoot="/">
20        <!-- tag::thirdPartyComment[] -->
21        <!-- enable visibility to third party apis -->
22        <!-- end::thirdPartyComment[] -->
23        <!-- tag::thirdParty[] -->
24        <classloader apiTypeVisibility="+third-party"/>
25        <!-- end::thirdParty[] -->
26    </webApplication>
27
28</server>

The OpenTelemetry APIs are exposed as third-party APIs in Open Liberty. To add the visibility of OpenTelemetry APIs to the application, add third-party to the types of API packages that this class loader supports. Instead of explicitly configuring a list of API packages that includes third-party, set the +third-party value to the apiTypeVisibility attribute in the classLoader configuration. This configuration adds third-party to the default list of API package types that are supported.

Enabling tracing in Jakarta CDI beans

You can trace your Jakarta CDI beans by annotating their methods with a @WithSpan annotation.

Replace the InventoryManager class:
inventory/src/main/java/io/openliberty/guides/inventory/InventoryManager.java

InventoryManager.java

 1// tag::copyright[]
 2/*******************************************************************************
 3 * Copyright (c) 2023 IBM Corporation and others.
 4 * All rights reserved. This program and the accompanying materials
 5 * are made available under the terms of the Eclipse Public License 2.0
 6 * which accompanies this distribution, and is available at
 7 * http://www.eclipse.org/legal/epl-2.0/
 8 *
 9 * SPDX-License-Identifier: EPL-2.0
10 *******************************************************************************/
11// end::copyright[]
12
13package io.openliberty.guides.inventory;
14
15import java.util.ArrayList;
16import java.util.Properties;
17
18import io.opentelemetry.instrumentation.annotations.WithSpan;
19import io.opentelemetry.instrumentation.annotations.SpanAttribute;
20
21import io.openliberty.guides.inventory.client.SystemClient;
22import io.openliberty.guides.inventory.model.InventoryList;
23import io.openliberty.guides.inventory.model.SystemData;
24import jakarta.enterprise.context.ApplicationScoped;
25import jakarta.inject.Inject;
26import java.util.List;
27import java.util.Collections;
28
29import org.eclipse.microprofile.config.inject.ConfigProperty;
30
31@ApplicationScoped
32public class InventoryManager {
33
34    @Inject
35    @ConfigProperty(name = "system.http.port")
36    private int SYSTEM_PORT;
37
38    private List<SystemData> systems = Collections.synchronizedList(new ArrayList<>());
39    private SystemClient systemClient = new SystemClient();
40
41    public Properties get(String hostname) {
42        systemClient.init(hostname, SYSTEM_PORT);
43        Properties properties = systemClient.getProperties();
44        return properties;
45    }
46
47    // tag::listWithSpan[]
48    @WithSpan
49    // end::listWithSpan[]
50    // tag::listMethod[]
51    public InventoryList list() {
52        return new InventoryList(systems);
53    }
54    // end::listMethod[]
55
56    // tag::addWithSpan[]
57    @WithSpan("Inventory Manager Add")
58    // end::addWithSpan[]
59    // tag::addMethod[]
60    // tag::spanAttribute[]
61    public void add(@SpanAttribute("hostname") String host,
62    // end::spanAttribute[]
63                    Properties systemProps) {
64        Properties props = new Properties();
65        props.setProperty("os.name", systemProps.getProperty("os.name"));
66        props.setProperty("user.name", systemProps.getProperty("user.name"));
67        SystemData system = new SystemData(host, props);
68        if (!systems.contains(system)) {
69            systems.add(system);
70        }
71    }
72    // end::addMethod[]
73
74    int clear() {
75        int propertiesClearedCount = systems.size();
76        systems.clear();
77        return propertiesClearedCount;
78    }
79}

The list() and add() methods are annotated with the @WithSpan annotation, which can accept an optional parameter that functions as the span name. In this example, the default span name assigned to the list() method is automatically generated through the instrumentation. You can also specify a custom span name. For example, Inventory Manager Add is specified as the span name for the add() method. The OpenTelemetry instrumentation provides a new span for each method. You can now collect and trace the spans across different services.

Optionally, you can include parameters and their values in the span by using the @SpanAttribute annotation. For example, the @SpanAttribute annotation specifies hostname as the attribute name for the host parameter , which helps trace the parameter within the add span.

To learn more about how to use OpenTelemetry annotations to instrument code, see the OpenTelemetry Annotations documentation.

Now, you can check out the traces that are generated by the @WithSpan annotation. Visit the http://localhost:9081/inventory/systems URL and then your Jaeger server at the http://localhost:16686 URL.

Select the inventory service and click the Find Traces button at the end of the section. You’ll see the result as:

Inventory Manager span

Verify that there are two spans from the inventory service. Click the trace to view its details. You’ll see the InventoryManager.list span that is created by the @WithSpan annotation.

Inventory Manager list span


To check out the information generated by the @SpanAttribute annotation, visit the http://localhost:9081/inventory/systems/localhost URL and then your Jaeger server at the http://localhost:16686 URL.

Select the inventory service and click the Find Traces button at the end of the section. You will see the following result:

Get traces for the inventory service

Verify that there are three spans from the inventory service and one span from the system service. Click the trace to view its details.

Inventory details spans

Click the Inventory Manager Add span and its Tags. You can see the hostname tag with the localhost value that is created by the @SpanAttribute annotation.

Inventory Manager add span


Injecting a custom Tracer object

The MicroProfile Telemetry specification makes the underlying OpenTelemetry Tracer instance available. The configured Tracer is accessed by injecting it into a bean. You can use it to instrument your code to create traces.

Replace the InventoryResource class:
inventory/src/main/java/io/openliberty/guides/inventory/InventoryResource.java

InventoryResource.java

  1// tag::copyright[]
  2/*******************************************************************************
  3 * Copyright (c) 2023 IBM Corporation and others.
  4 * All rights reserved. This program and the accompanying materials
  5 * are made available under the terms of the Eclipse Public License 2.0
  6 * which accompanies this distribution, and is available at
  7 * http://www.eclipse.org/legal/epl-2.0/
  8 *
  9 * SPDX-License-Identifier: EPL-2.0
 10 *******************************************************************************/
 11// end::copyright[]
 12package io.openliberty.guides.inventory;
 13
 14import java.util.Properties;
 15
 16import io.opentelemetry.api.trace.Tracer;
 17import io.opentelemetry.api.trace.Span;
 18import io.opentelemetry.context.Scope;
 19
 20import io.openliberty.guides.inventory.model.InventoryList;
 21import jakarta.enterprise.context.RequestScoped;
 22import jakarta.inject.Inject;
 23import jakarta.ws.rs.DELETE;
 24import jakarta.ws.rs.GET;
 25import jakarta.ws.rs.Path;
 26import jakarta.ws.rs.PathParam;
 27import jakarta.ws.rs.Produces;
 28import jakarta.ws.rs.core.MediaType;
 29import jakarta.ws.rs.core.Response;
 30
 31@RequestScoped
 32@Path("/systems")
 33public class InventoryResource {
 34
 35    // tag::manager[]
 36    @Inject
 37    private InventoryManager manager;
 38    // end::manager[]
 39
 40    // tag::inject[]
 41    @Inject
 42    private Tracer tracer;
 43    // end::inject[]
 44
 45    @GET
 46    @Path("/{hostname}")
 47    @Produces(MediaType.APPLICATION_JSON)
 48    public Response getPropertiesForHost(@PathParam("hostname") String hostname) {
 49        // tag::getPropertiesSpan[]
 50        Span getPropertiesSpan = tracer.spanBuilder("GettingProperties").startSpan();
 51        // end::getPropertiesSpan[]
 52        // tag::try[]
 53        Properties props = null;
 54        // tag::scope[]
 55        try (Scope scope = getPropertiesSpan.makeCurrent()) {
 56        // end::scope[]
 57            // tag::getSystem[]
 58            props = manager.get(hostname);
 59            // end::getSystem[]
 60            if (props == null) {
 61                // tag::addEvent1[]
 62                getPropertiesSpan.addEvent("Cannot get properties");
 63                // end::addEvent1[]
 64                return Response.status(Response.Status.NOT_FOUND)
 65                         .entity("{ \"error\" : \"Unknown hostname or the system "
 66                               + "service may not be running on " + hostname + "\" }")
 67                         .build();
 68            }
 69            // tag::addEvent2[]
 70            getPropertiesSpan.addEvent("Received properties");
 71            // end::addEvent2[]
 72            manager.add(hostname, props);
 73        } finally {
 74            // tag::end[]
 75            getPropertiesSpan.end();
 76            // end::end[]
 77        }
 78        // end::try[]
 79        return Response.ok(props).build();
 80
 81    }
 82
 83    @GET
 84    @Produces(MediaType.APPLICATION_JSON)
 85    public InventoryList listContents() {
 86        return manager.list();
 87    }
 88
 89    @DELETE
 90    @Produces(MediaType.APPLICATION_JSON)
 91    public Response clearContents() {
 92        int cleared = manager.clear();
 93
 94        if (cleared == 0) {
 95            return Response.status(Response.Status.NOT_MODIFIED)
 96                           .build();
 97        }
 98        return Response.status(Response.Status.OK)
 99                       .build();
100    }
101}

To access the Tracer, the @Inject annotation from the Contexts and Dependency Injections API injects the Tracer into a bean.

Before the InventoryManager calls the system service, it creates and starts a span called the GettingProperties by using the spanBuilder() and startSpan() Tracer APIs.

When you start a span, you must also end it by calling end() on the span. If you don’t end a span, it won’t be recorded at all and won’t show up in Jaeger. This code ensures that end() is always called by including it in a finally block.

After you start the span, make it current with the makeCurrent() call. Making a span current means that any new spans created in the same thread, either automatically by Open Liberty or manually by calling the API, will use this span as their parent span.

The makeCurrent() call returns a Scope. Make sure to always close the Scope, which stops the span from being current and makes the previous span current again. Use a try-with-resources block, which automatically closes the Scope at the end of the block.

Use the addEvent() Span API to create an event when the properties are received and an event when it fails to get the properties from the system service. Use the end() Span API to mark the GettingProperties span as completed.

To check out the traces that contain the GettingProperties span, visit the http://localhost:9081/inventory/systems/localhost URL and then your Jaeger server at the http://localhost:16686 URL.

Select the inventory service and click the Find Traces button at the end of the section. You’ll see the result:

Get traces for the inventory service

Verify that there are four spans from the inventory service and one span from the system service. Click the trace to view its details. You’ll see the GettingProperties span.

Inventory details spans


To check out the event adding to the GettingProperties span, visit the http://localhost:9081/inventory/systems/unknown URL and then your Jaeger server at the http://localhost:16686 URL.

Select the inventory service and click the Find Traces button at the end of the section. You will see the following result:

Get traces for unknown hostname

There are two spans from the inventory service. Click the trace to view its details. You’ll see the GettingProperties span. Click the GettingProperties span and its Logs. You can see the Cannot get properties message.

Logs at GettingProperties span


To learn more about how to use OpenTelemetry APIs to instrument code, see the OpenTelemetry Manual Instrumentation documentation.

Testing the application

Manually verify the traces by inspecting them on the Jaeger server. You will find some tests included to test the basic functionality of the services. If any of the tests fail, you might have introduced a bug into the code.

Running the tests

Since you started Open Liberty in dev mode, run the tests for the system and inventory services by pressing the enter/return key in the command-line sessions where you started the services.

When you are done checking out the services, exit dev mode by pressing CTRL+C in the shell sessions where you ran the system and inventory services.

Finally, stop the Jaeger service that you started in the Additional prerequisites section.

Great work! You’re done!

You just used MicroProfile Telemetry in Open Liberty to customize how and which traces are delivered to Jaeger.

Try out one of the related MicroProfile guides. These guides demonstrate more technologies that you can learn to expand on what you built in this guide.

Guide Attribution

Enabling distributed tracing in microservices with OpenTelemetry and Jaeger by Open Liberty is licensed under CC BY-ND 4.0

Copy file contents
Copied to clipboard

Prerequisites:

Nice work! Where to next?

What did you think of this guide?

Extreme Dislike Dislike Like Extreme Like

What could make this guide better?

Raise an issue to share feedback

Create a pull request to contribute to this guide

Need help?

Ask a question on Stack Overflow

Like Open Liberty? Star our repo on GitHub.

Star

Where to next?