Run FIPS-compliant applications on Open Liberty

The Federal Information Processing Standard (FIPS) 140-2 is a US government security standard for cryptographic modules. Although FIPS compliance is determined by your underlying Java virtual machine (JVM), you can enable Open Liberty to run on a FIPS-compliant JVM.

FIPS enablement is important for many users, particularly if you work for or with US government agencies. Running your Open Liberty servers on a FIPS-compliant JVM ensures that only FIPS certified cryptography is used when an application uses Java security libraries or APIs. FIPS-compliant JVM options for Open Liberty are included in both IBM SDK, Java Technology Edition and IBM Semeru Runtimes.

For more information about enabling FIPS for Liberty with the IBM SDK, Java Technology Edition, see Setting up Liberty for FIPS compliance in the WebSphere Liberty documentation. The configuration is the same for both WebSphere Liberty and Open Liberty.

Enable FIPS for Open Liberty on IBM Semeru Runtimes

You can enable either IBM Semeru Runtime Certified Edition or Open Edition in FIPS mode in version 11.0.16 and later for Java 11 and version 17.0.4 and later for Java 17. Java 11 and 17 support for FIPS with Semeru Runtimes is available only on Red Hat Enterprise Linux (RHEL) 8 on x86 platforms. The RHEL 8 operating system must be running in FIPS mode because the IBM Semeru Runtimes rely on the operating system’s underlying Network Security Services (NSS) FIPS 140-2 certification. To run Open Liberty on IBM Semeru Runtimes in FIPS mode, Open Liberty version 22.0.0.8 or later is recommended. In FIPS mode, Semeru Runtimes does not support file-based keystores like JKS and PKCS#12. Certificates in your file-based keystores must be imported into the NSS database. Open Liberty does not create certificates in the NSS database.

Complete the following steps to configure your Open Liberty server to run on Semeru Runtimes in FIPS mode and to add your keys and certificates to the NSS database.

Confirm that your RHEL operating system is installed in FIPS mode.
If your RHEL operating system was not installed in FIPS mode, you must switch it to FIPS mode. For more information about how to enable or check the FIPS status for your RHEL operating system, see Switching the system to FIPS mode in the RHEL documentation.
Specify system properties to enable FIPS mode for the JVM and, optionally, to enable debug tracing.
The -Dsemeru.fips=true property specifies that the JVM uses only FIPS certified cryptography, and ensures that the TLS and SSL protocols use only FIPS certified algorithms. The optional -Djava.security.debug=semerufips property enables debug tracing. Add these properties to the jvm.options file in your Open Liberty server configuration directory, one property per line, as shown in the following example.
```
-Dsemeru.fips=true
-Djava.security.debug=semerufips
```
Create a Liberty configuration file that contains the NSS library that is required for reading a PKCS#11 keystore.
The file must be in a location that is accessible to Liberty and must contain the following information.
```
name = NSS-FIPS
library = /usr/lib64/libsoftokn3.so
slot = 3
showInfo = true
```
This file is referenced by the keystore location configuration attribute in step 5 as tmp/pkcs11cfg.cfg.
Import your keys and certificates to the NSS database.
In FIPS mode, Semeru Runtimes does not support file-based keystores like JKS and PKCS#12. Certificates in your file-based keystores must be imported into the NSS database.
You can import and manage your keys and certificates in the NSS database by using the NSS pk12util and certutil commands.
- To import keys from your keystore to the NSS database, use the pk12util command. In the following example, key.p12 is the keystore file and Liberty is the keystore password.
  pk12util -i key.p12 -W Liberty -d /etc/pki/nssdb
- You can import trusted certificates in the same way, as shown in the following example, where trust.p12 is the file that contains the certificate entries.
  pk12util -i trust.p12 -W Liberty -d /etc/pki/nssdb
- Trusted certificates must be marked as a trusted certificate authority (CA), with complete trust for both client and server certificates. You apply the CA by running the certutil command, as shown in the following example, where the -t argument specifies complete trust with the CT value.
  certutil -M -n trustCert -t “CT,CT,CT” -d /etc/pki/nssdb
- You can also use the certutil command to look at the contents of the NSS database, as shown in the following example.
  certutil -L -d /etc/pki/nssdb
  In FIPS mode, Semeru Runtimes does not support the usage of the NSS database with a password. If the pk12util tool prompts you to set up a password when you import the PKCS#12 file-based keystore, press Enter twice to set up an empty password.
If a password is entered as prompted instead of an empty password, then the user encounters a CKR_PIN_INCORRECT error in the liberty logs.
Create a keystore entry in your server.xml file that references the NSS database where you imported your keys and certificates.
The following server.xml example shows the keystore configuration to run Open Liberty in FIPS mode on Semeru Runtimes. In this example, location=“/tmp/pkcs11cfg.cfg” specifies the path to the Liberty configuration file that you created in step 3.
```
<featureManager>
    <feature>transportSecurity-1.0</feature>
</featureManager>

    <ssl id=“defaultSSLConfig"
         keyStoreRef="defaultKeyStore"
         sslProtocol=“TLSv1.2” />

    <keyStore id="defaultKeyStore" password="Liberty"
         location=“/tmp/pkcs11cfg.cfg” type=“PKCS11”
         fileBased=“false” provider=“SunPKCS11-NSS-FIPS” />
```
In this example, the keystore type attribute is set to PKCS11, but PKCS11-NSS-FIPS is also a valid value. This configuration instructs Open Liberty to use the NSS PKCS#11-based keystore instead of a file-based keystore.

You can now start your Open Liberty server in FIPS mode.

For more information about Semeru Runtimes in FIPS mode, see FIPS certified cryptography in IBM Semeru Runtimes. For more information about Open Liberty TLS configuration, see Secure communication with TLS and the Transport Security feature.