Log and trace configuration

Open Liberty has a unified logging component that handles messages that are written by applications and the runtime, and provides First Failure Data Capture (FFDC) capability. Logging data that is written by applications by using the System.out, System.err, or java.util.logging.Logger streams is combined into the server logs.

A server has three primary log files:

  • console.log - This file is created by the server start command. It contains the redirected standard output and standard error streams from the JVM process. This console output is formatted for human readability and it lacks some information that is useful for automated log analysis.

  • messages.log - This file contains all messages that are written or captured by the logging component. All messages that are written to this file contain additional information such as the message timestamp and the ID of the thread that wrote the message. This file is suitable for automated log analysis. This file does not contain messages that are written directly by the JVM process.

  • trace.log - This file contains all messages that are written or captured by the logging component and any enabled trace. This file is created only if you enable trace. This file does not contain messages that are written directly by the JVM process.

The following sections provide more information about configuring your Open Liberty logs:

Logging configuration

The logging component can be controlled through the server configuration. The logging component can be fully configured in your server.xml file by the logging element. However, logging is initialized before the server.xml file is processed so configuring logging through the server.xml file can result in early log entries that use a different log configuration from later ones. To avoid this problem, you can provide much of the logging configuration in the boostrap.properties file and in some cases by using environment variables.

Log file storage management

The console.log file is created by redirecting the process stdout and stderr streams to a file. As a result, Open Liberty is unable to offer the same level of management, like log rollover, as it offers for the messages.log file. If you are concerned about the size of the console.log file, you can disable the console.log file and use the messages.log file instead. All the log messages that are sent to the console.log file are written to the messages.log file, and you can configure file rollover in the messages log.

To disable the console log, and configure the messages.log file to roll over three times at 100 Mb, you can use the following configuration:

com.ibm.ws.logging.max.file.size=100
com.ibm.ws.logging.max.files=3
com.ibm.ws.logging.console.log.level=OFF
com.ibm.ws.logging.copy.system.streams=false

JSON logging

You can simplify log parsing by producing your logs in JSON format. JSON is a self-describing format that many log analysis tools can consume without requiring format-specific parsing instructions. You can configure Open Liberty logs to produce logs in JSON format either by editing the bootstrap.properties file or by specifying an environment variable. The following two examples show the configuration for each of these options:

  • Configure JSON logging in the bootstrap.properties file:

    com.ibm.ws.logging.message.format=json
    com.ibm.ws.logging.message.source=message,trace,accessLog,ffdc,audit
  • Configure JSON logging with environment variables in the server.env file:

    WLP_LOGGING_MESSAGE_FORMAT=json
    WLP_LOGGING_MESSAGE_SOURCE=message,trace,accessLog,ffdc,audit

Configurable JSON field names

When logs are in JSON format, you can use the jsonFieldMappings attribute to replace default field names with new field names. Replacing the default field names might be necessary if other servers in the same logging configuration use different field names than the Open Liberty default names. For example, an Open Liberty message is referred to by the message field name, but the message in another container might be in a field called log. In this case, two different visualizations of the messages show in the logs on a dashboard. If you modify the Open Liberty output field name so that it matches the other log, you can view them in the same visualization. The following examples show sample configurations for renaming a JSON field.

  • To configure a new field name, you can include the following environmental variable in the server.env file:

    WLP_LOGGING_JSON_FIELD_MAPPINGS=loglevel:level

    In this example, the loglevel field name is replaced by the level field name.

  • To configure a field name for a specific source, you can include the following environmental variable in the server.env file:

    WLP_LOGGING_JSON_FIELD_MAPPINGS=message:message:log

    In this example, the message field name is replaced by the log field name in the message log.

To omit a field from the logs, specify the field name without a replacement, as shown in the following example:

WLP_LOGGING_JSON_FIELD_MAPPINGS=defaultFieldName:

To rename or omit multiple fields, specify a comma-separated list of field name mappings.

For a full list of the default JSON field names, see the JSON log events reference list.

For more information, see Configurable JSON log field names.

Custom message and trace JSON fields

You can add custom fields to your JSON-formatted message and trace output to gather information about a particular issue or incident. For example, if you want to check the requests from a specific user, you can add a custom field to filter application logs by that user’s ID. You can add another field for the session ID so that you can analyze and filter application logs for a specific session.

The Open Liberty LogRecordContext API can add custom fields to your log and trace records. This API adds a field by specifying a value for an extension. To use the LogRecordContext API, first import the com.ibm.websphere.logging.hpel.LogRecordContext class. The following examples show how to add different kinds of custom fields to your JSON logs.

To add a string-valued field to your application logs, you can include the following line in your application:

LogRecordContext.addExtension("userName","bob");

The newly specified field is added to log and trace entries that are created on the same thread that executes the addExtension method. In this example, a custom field that is called userName is added for the bob user ID.

To add fields with boolean, float, int, or long values, the extension name must include the suffixes _bool, _float, _int, or _long, as shown in the following examples:

LogRecordContext.addExtension("extensionName_bool","true");
LogRecordContext.addExtension("extensionName_int","112233");
LogRecordContext.addExtension("extensionName_float","1.2");
LogRecordContext.addExtension("extensionName_long","132");

When you specify these suffixes to add non-string values, the resulting JSON field values are not enclosed in quotes in the logs. Only string-valued JSON field values are enclosed in quotes in the logs.

To remove custom fields from the logs, use the following method:

LogRecordContext.removeExtension(extensionName);

After you remove an extension, JSON output for subsequent logs and trace that are made on the same thread do not include that field.

Selectable HTTP access log JSON fields

When logs are in JSON format and the accessLog source is specified, you can replace the default HTTP access log JSON fields with a different set of fields. You can use the jsonAccessLogFields attribute to specify whether your access logs use the default fields or a set of fields that is specified by the logFormat attribute. You specify the replacement fields in the logFormat attribute of the accessLogging element within the httpEndpoint element.

With this configuration, you can receive information that is otherwise not available in JSON logs, such as the remote user ID, request headers, and more. These logs can be used by log analysis tools, such as the Elastic stack, to monitor your server. For more information about HTTP access log format options, see HTTP access logging.

The following example shows a configuration in the server.xml file to replace the default HTTP access log fields with fields that are specified by the logFormat attribute. HTTP access logging must be enabled to receive JSON access logs.

<httpEndpoint httpPort="9080" httpsPort="9443" id="defaultHttpEndpoint">
    <accessLogging logFormat='%R{W} %u %{my_cookie}C %s'/>
</httpEndpoint>
<logging jsonAccessLogFields="logFormat"/>

Docker image logging configuration

In Docker environments, you can disable the messages log and format the console output as JSON by using environment variables, as shown in the following example:

WLP_LOGGING_MESSAGE_FORMAT=json
WLP_LOGGING_MESSAGE_SOURCE=
WLP_LOGGING_CONSOLE_FORMAT=json
WLP_LOGGING_CONSOLE_LOGLEVEL=info
WLP_LOGGING_CONSOLE_SOURCE=message,trace,accessLog,ffdc,audit

You can specify this configuration when you run the docker run command by using the -e flag to set the environment variables:

docker run -e "WLP_LOGGING_CONSOLE_SOURCE=message,trace,accessLog,ffdc"
           -e "WLP_LOGGING_CONSOLE_FORMAT=json"
           -e "WLP_LOGGING_CONSOLE_LOGLEVEL=info"
           -e "WLP_LOGGING_MESSAGE_FORMAT=json"
           -e "WLP_LOGGING_MESSAGE_SOURCE=" open-liberty

Binary logging

Liberty has a high-performance binary log format option that reduces the resources that are needed to write trace files. Generally, when you configure binary logging, the console.log is disabled for best performance. You can configure binary logging in the bootstrap.properties file, as shown in the following example:

websphere.log.provider=binaryLogging-1.0
com.ibm.ws.logging.console.log.level=OFF
com.ibm.ws.logging.copy.system.streams=false

The binaryLog command-line tool can be used to convert the binary log to a text file:

binaryLog view defaultServer

Configuration settings by source

The following table shows the equivalent server.xml file, bootstrap.properties file, and environment variable configurations along with brief descriptions. For more information, see the logging element.

Logging configuration settings
Server XML Attribute bootstrap property Env var Description

hideMessage

com.ibm.ws.logging.hideMessage

You can use this setting to configure the messages keys that you want to hide from the console.log and messages.log files. When the messages are hidden, they are redirected to the trace.log file.

jsonFieldMappings

com.ibm.ws.logging.json.field.mappings

WLP_LOGGING_JSON_FIELD_MAPPINGS

When logs are in JSON format, use this setting to replace default field names with new field names or to omit fields from the logs. For more information, see Configurable JSON field names

logDirectory

com.ibm.ws.logging.log.directory

LOG_DIR

You can use this setting to set a directory for all log files, excluding the console.log file, but including FFDC. The default is WLP_OUTPUT_DIR/serverName/logs. It is not recommended to set the logDirectory in the server.xml file since it can result in some log data being written to the default location prior to when the server.xml file is read.

consoleFormat

com.ibm.ws.logging.console.format

WLP_LOGGING_CONSOLE_FORMAT

This setting specifies the required format for the console. Valid values are dev, simple, or json format. By default, consoleFormat is set to dev.

consoleLogLevel

com.ibm.ws.logging.console.log.level

WLP_LOGGING_CONSOLE_LOGLEVEL

This setting controls the granularity of messages that go to the console. The valid values are INFO, AUDIT, WARNING, ERROR, and OFF. The default is AUDIT. If using with the Eclipse developer tools this must be set to the default.

consoleSource

com.ibm.ws.logging.console.source

WLP_LOGGING_CONSOLE_SOURCE

This setting specifies a comma-separated list of sources that route to the console. It applies only when the console format is set to json. The valid values are message, trace, accessLog, ffdc, and audit. By default, consoleSource is set to message. To use the audit source, enable the Liberty audit-1.0 feature. To use the accessLog source you need to have configured httpAccessLogging.

copySystemStreams

com.ibm.ws.logging.copy.system.streams

If this setting is set to true, messages that are written to the System.out and System.err streams are copied to process stdout and stderr streams and so appear in the console.log file. If this setting is set to false, those messages are written to configured logs such as the messages.log file or trace.log file, but they are not copied to stdout and stderr and do not appear in console.log. The default value is true.

com.ibm.ws.logging.newLogsOnStart

If this setting is set to true when Open Liberty starts, any existing messages.log or trace.log files are rolled over and logging writes to a new messages.log or trace.log file. If this setting is set to false, messages.log or trace.log files only refresh when they hit the size that is specified by the maxFileSize attribute. The default value is true. This setting cannot be provided using the logging element in the server.xml file because it is only processed during server bootstrap.

isoDateFormat

com.ibm.ws.logging.isoDateFormat

This setting specifies whether to use ISO-8601 formatted dates in log files. The default value is false.

If this setting is set to true, the ISO-8601 format is used in the messages.log file, the trace.log file, and the FFDC logs. The format is yyyy-MM-dd’T’HH:mm:ss.SSSZ.

If you specify a value of false, the date and time are formatted according to the default locale set in the system. If the default locale is not found, the format is dd/MMM/yyyy HH:mm:ss:SSS z.

maxFiles

com.ibm.ws.logging.max.files

This setting specifies how many of each of the logs files are kept. This setting also applies to the number of exception summary logs for FFDC. So if this number is 10, you might have 10 message logs, 10 trace logs, and 10 exception summaries in the ffdc/ directory. By default, the value is 2. The console log does not roll so this setting does not apply to the console.log file.

maxFileSize

com.ibm.ws.logging.max.file.size

This setting specifies the maximum size (in MB) that a log file can reach before it is rolled. Setting the value to 0 disables log rolling. The default value is 20. The console.log does not roll so this setting does not apply.

messageFileName

com.ibm.ws.logging.message.file.name

This setting sp[ecifies the name of the message log file. The message log file has a default name of messages.log. This file always exists, and contains INFO and other (AUDIT, WARNING, ERROR, FAILURE) messages in addition to the System.out and System.err streams . This log also contains time stamps and the issuing thread ID. If the log file is rolled over, the names of earlier log files have the format messages_timestamp.log

messageFormat

com.ibm.ws.logging.message.format

WLP_LOGGING_MESSAGE_FORMAT

This setting specifies the required format for the messages.log file. Valid values are simple or json format. By default, messageFormat is set to simple.

messageSource

com.ibm.ws.logging.message.source

WLP_LOGGING_MESSAGE_SOURCE

This setting specifies a list of comma-separated sources that route to the messages.log file. This setting applies only when the message format is set to json`. The valid values are message, trace, accessLog, ffdc, and audit. By default, messageSource is set to message. To use the audit source, enable the Liberty audit-1.0 feature. To use the accessLog source you need to have configured httpAccessLogging.

suppressSensitiveTrace

This attribute, when set to true, prevents potentially sensitive information from being exposed in log and trace files. The server trace can expose sensitive data when it traces untyped data, such as bytes received over a network connection. The default value is false.

traceFileName

com.ibm.ws.logging.trace.file.name

This setting specifies the name of the trace log file. The trace.log file is created only if additional or detailed trace is enabled. stdout is recognized as a special value, and causes trace to be directed to the original standard out stream.

traceFormat

com.ibm.ws.logging.trace.format

This setting controls the format of the trace log. The default format for Liberty is ENHANCED. You can also use BASIC and ADVANCED formats.

traceSpecification

com.ibm.ws.logging.trace.specification

This setting is used to selectively enable trace. The log detail level specification is in the following format:

component = level

The component specifies what log sources the level is set for. A component can be a logger name, trace group, or class name. The level specifies what level of trace to output for that component by using one of the following levels:

off, fatal, severe, warning, audit, info, config, detail, fine, finer, finest, all.

You can provide multiple log detail level specifications that are separated by colons.

An asterisk * acts as a wildcard to match multiple components based on a prefix. For example:

- * Specifies all traceable code that is running in the application server, including the product system code and customer code.

- com.ibm.ws.* Specifies all classes with the package name beginning with com.ibm.ws.

- com.ibm.ws.classloading.AppClassLoader Specifies the AppClassLoader class only.

appsWriteJson

com.ibm.ws.logging.apps.write.json

WLP_LOGGING_APPS_WRITE_JSON

When the message log or console is in JSON format, this setting allows applications to write JSON-formatted messages to those destinations, without modification.

jsonAccessLogFields

com.ibm.ws.json.access.log.fields

WLP_LOGGING_JSON_ACCESS_LOG_FIELDS

When logs are in JSON format, you can use this setting to replace the default HTTP access log JSON fields with fields that are specified by the logFormat attribute of the accesLogging element.