Code instrumentation for MicroProfile Telemetry tracing

After you enable MicroProfile Telemetry and configure a trace server, you can instrument tracing in your application code. You can instrument your code automatically, manually, or by using the Java agent.

Automatic instrumentation

With automatic instrumentation, you can observe traces without modifying the source code in your applications. All that you need to do is configure runtime as described in Collect logs, metrics, and traces with OpenTelemetry. However, automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications.

In Open Liberty version 23.0.0.11 and later, spans are automatically generated for incoming HTTP requests, including static files, servlets, and JSPs.

Manual instrumentation

Automatic instrumentation is available only for JAX-RS and Jakarta RESTful web service applications. To create spans for other operations, such as database calls, you can add manual instrumentation to the source code for those operations by using the OpenTelemetry API.

However, before you manually instrument your code, you must make 3rd party APIs visible in your runtime environment by editing your runtime configuration. For more information, see Customize your application telemetry with the OpenTelemetry API.

The following examples show configuration options with the OpenTelemetry API.

  • Add extra information, such as the user ID, to the current span. Any information that you add to a span is visible when you look at the traces on your trace server.

    private static final AttributeKey<String> USER_ID_ATTR = AttributeKey.stringKey("userId");

@Inject private Span currentSpan;

@GET
public String myMethod() {
    ...
    currentSpan.setAttribute(USER_ID_ATTR, getUserId());
    ...
}

  • Annotate methods in any Jakarta CDI beans by using the @WithSpan annotation. This annotation creates a span and establishes any required relationships with the current trace context. You can annotate method parameters with the @SpanAttribute annotation to indicate which method parameters are part of the trace, as shown in the following example.

    @ApplicationScoped
class SpanBean {

    @WithSpan("name")
    void spanName() {
       ...
    }

    @WithSpan
    void spanArgs(@SpanAttribute(value = "arg") String arg) {
       ...
    }
}

    However, you cannot create a child span with the @WithSpan annotation by calling an internal method in the same class as the parent span. Instead, you can manually create a subspan in the child method, as shown in the next example.

  • Create a subspan around a particular operation, such as querying a database. This subspan shows you how long it took and the order in which it occurred relative to other spans.

    @Inject private Tracer tracer;

@GET
public String myMethod() {
    ...
    Span newSpan = tracer.spanBuilder("QueryDatabase").startSpan();
    try (Scope s = newSpan.makeCurrent()) {
        queryDatabase();
    } finally {
        newSpan.end();
    }
    ...
}

This example creates a span that is named QueryDatabase that can be used to track the execution time and other attributes of the queryDatabase() method.

Considerations for manual instrumentation

The following important considerations apply to manual instrumentation.

  • Call the .end() method on any span that you create, otherwise the span is not recorded.

  • The current span is used as the parent for any new spans that are created. Therefore, when you create a span, in most cases you also want to make it current. However, you must close the Scope instance that is returned by the Span.makeCurrent() method. You can close a Scope instance by specifying a try-with-resources block, as shown in the previous example for creating a subspan.

  • If you set any properties by using environment variables, including those in the server.env file, specify the keys in uppercase and replace all punctuation with an underscore. Write values normally.

For more information, see the OpenTelemetry manual instrumentation documentation. However, remember when you use the MicroProfile Telemetry feature in Open Liberty, you must obtain the OpenTelemetry and Tracer objects by injecting them, not by creating your own.

Agent instrumentation

The OpenTelemetry Instrumentation for Java project provides a Java agent JAR file that can be attached to any Java 8+ application. This file dynamically injects bytecode that adds telemetry support to popular open source libraries and frameworks. If you are using any of the supported libraries in your application, you can use this agent with Open Liberty to instrument them.

To enable the Java agent on your Open Liberty runtime, download the latest agent version from OpenTelemetry and add the following line to your jvm.options file.

-javaagent: path/to/opentelemetry-javaagent.jar

You can configure the agent with environment variables and system properties. You can find a list of supported libraries and frameworks in the OpenTelemetry Java instrumentation documentation.

Limitations of agent instrumentation

The OpenTelemetry Java agent is a tool that is provided by the OpenTelemetry project. Although it is compatible with Open Liberty, it is a separate project and is subject to the following limitations.

  • Configuration works differently when you use the agent. Configuration of the agent is well documented, but the following aspects are different from configuration without the agent:

    • Configuration is shared between all applications that are deployed to the server.

    • Configuration properties are only read from system properties and environment variables. They are not read from MicroProfile Config configuration sources.

    • Because the agent reads its configuration early in the startup process, system properties are not read from the bootstrap.properties file. Alternatively, you can set system properties in the jvm.options file by using the following syntax: -Dname=value

    • Implementations of SPI extensions within applications are ignored. For more information, see the agent documentation for providing SPI extensions.

  • When you use the agent, it takes over the instrumentation of REST calls and methods that are annotated with the @WithSpan annotation. As a result, the created spans might be slightly different.

  • The agent is not compatible with Java 2 security.

  • Open Liberty uses many open source libraries internally. Some of these libraries might be automatically instrumented by the agent.