Run FIPS-compliant applications on Open Liberty

The Federal Information Processing Standard (FIPS) is a US government security standard for cryptographic modules. Although FIPS compliance is determined by your underlying Java virtual machine (JVM), you can enable Open Liberty to run on a FIPS-compliant JVM.

FIPS enablement is important for many users, particularly if you work for or with US government agencies. Running your Open Liberty servers on a FIPS-compliant JVM helps ensure that only FIPS-certified cryptography is used when an application uses Java security libraries or APIs. FIPS-compliant JVM options for Open Liberty are IBM SDK, Java Technology Edition or IBM Semeru Runtimes.

Enable FIPS 140-3 for Open Liberty on the IBM SDK or IBM Semeru Runtimes

To enable FIPS 140-3 for Liberty with the IBM SDK, Java Technology Edition or IBM Semeru Runtimes, complete the following steps.

If your Liberty server is running, stop the Liberty server.
Configure FIPS 140-3 by using one of the following methods.
- On 25.0.0.12 and later, use the securityUtility configureFIPS command to configure FIPS 140-3 across all servers, clients, and tools.
  - To configure FIPS 140-3 at a specific server, run the following command.+
    securityUtility configureFIPS --server=<server name>
  - To configure FIPS 140-3 at a specific client, run the following command.+
    securityUtility configureFIPS --client=<client name>
    For IBM Semeru Runtimes, The securityUtility configureFIPS command also creates a custom profile file.+
    For more information about the configureFIPS task, see configureFIPS.+
- For FIPS 140-3 set on Java SE 8, set the following properties in the jvm.options file.+
  -Dcom.ibm.jsse2.usefipsprovider=true -Dcom.ibm.jsse2.usefipsProviderName=IBMJCEPlusFIPS -Xenablefips140-3
  In this example, the com.ibm.jsse2.usefipsProviderName property specifies IBMJCEPlusFIPS as the FIPS provider.+
  For IBM Semeru Runtimes, set the following properties in the jvm.options file.+
  -Dsemeru.fips=true -Dsemeru.customprofile=OpenJCEPlusFIPS.FIPS140-3-Liberty -Djava.security.propertiesList=<path to wlp>/lib/security/fips140_3/FIPS140-3-Liberty.properties
  For IBM Semeru Runtimes, if you have a custom profile file, set the following properties in the jvm.options file.+
  -Dsemeru.fips=true -Dsemeru.customprofile=<your custom profile name> -Djava.security.propertiesList=<path to wlp>/lib/security/fips140_3/FIPS140-3-Liberty.properties:<path to your custom profile file>
  To enable FIPS 140-3 for tools, the properties must be set by using the JVM_ARGS environment variable instead of in the jvm.options file. The following example is for Java SE 8.+
  export JVM_ARGS="-Xenablefips140-3 -Dcom.ibm.jsse2.usefipsprovider=true -Dcom.ibm.jsse2.usefipsProviderName=IBMJCEPlusFIPS"
If you have LTPA validation keys, delete the LTPA validation keys. Do not delete LTPA keys that are not LTPA validation keys.
After you configure FIPS 140-3, restart the Liberty server to enable FIPS 140-3. A new ltpa.keys file is created using FIPS 140-3 approved algorithms when the Liberty server restarts.+ Any existing ltpa.keys files created before FIPS 140-3 was enabled are backed up to ltpa.keys.nofips. If you need to create new LTPA keys and LTPA validation keys that use FIPS 140-3 approved algorithms, you can create the manually by running the following createLTPAKeys command.+
```
securityUtility createLTPAKeys --password=mypassword --passwordEncoding=aes
```
For more information about the createLTPAKeys command, see createLTPAKeys.

Alternatively, to enable the outmoded FIPS 140-2 for Open Liberty with IBM Semeru Runtimes, complete the following steps. Be certain that you want to proceed; FIPS 140-2 validations are scheduled to move to the Historical List.

Enable FIPS 140-2 for Open Liberty on IBM Semeru Runtimes

You can enable either IBM Semeru Runtime Certified Edition or Open Edition in FIPS mode in version 11.0.16 and later for Java 11 and version 17.0.4 and later for Java 17. Java 11 and 17 support for FIPS with Semeru Runtimes is available only on Red Hat Enterprise Linux (RHEL) 8 on x86 platforms. The RHEL 8 operating system must be running in FIPS mode because the IBM Semeru Runtimes rely on the operating system’s underlying Network Security Services (NSS) FIPS 140-2 certification. To run Open Liberty on IBM Semeru Runtimes in FIPS mode, Open Liberty version 22.0.0.8 or later is recommended. In FIPS mode, Semeru Runtimes does not support file-based keystores like JKS and PKCS#12. Certificates in your file-based keystores must be imported into the NSS database. Open Liberty does not create certificates in the NSS database.

Complete the following steps to configure your Open Liberty server to run on Semeru Runtimes in FIPS mode and to add your keys and certificates to the NSS database.

Confirm that your RHEL operating system is installed in FIPS mode.
Run the following command:
```
fips-mode-setup --check
```
If FIPS mode is enabled, the command output is FIPS mode is enabled.
If your RHEL operating system was not installed in FIPS mode, you must switch it to FIPS mode. For more information about how to enable or check the FIPS status for your RHEL operating system, see Switching RHEL to FIPS mode in the RHEL documentation.
Specify system properties to enable FIPS mode for the JVM and, optionally, to enable debug tracing.
The -Dsemeru.fips=true property specifies that the JVM uses only FIPS certified cryptography, and ensures that the TLS and SSL protocols use only FIPS certified algorithms. The optional -Djava.security.debug=semerufips property enables debug tracing. Add these properties to the jvm.options file in your Open Liberty server configuration directory, one property per line, as shown in the following example.
```
-Dsemeru.fips=true
-Djava.security.debug=semerufips
```
Create a Liberty configuration file that contains the NSS library that is required for reading a PKCS#11 keystore.
The file must be in a location that is accessible to Liberty and must contain the following information.
```
name = NSS-FIPS
library = /usr/lib64/libsoftokn3.so
slot = 3
showInfo = true
```
This file is referenced by the keystore location configuration attribute in step 5 as tmp/pkcs11cfg.cfg.
Import your keys and certificates to the NSS database.
In FIPS mode, Semeru Runtimes does not support file-based keystores like JKS and PKCS#12. Certificates in your file-based keystores must be imported into the NSS database.
You can import and manage your keys and certificates in the NSS database by using the NSS pk12util and certutil commands.
- To import keys from your keystore to the NSS database, use the pk12util command. In the following example, key.p12 is the keystore file and Liberty is the keystore password.
  pk12util -i key.p12 -W Liberty -d /etc/pki/nssdb
- You can import trusted certificates in the same way, as shown in the following example, where trust.p12 is the file that contains the certificate entries.
  pk12util -i trust.p12 -W Liberty -d /etc/pki/nssdb
- Trusted certificates must be marked as a trusted certificate authority (CA), with complete trust for both client and server certificates. You apply the CA by running the certutil command, as shown in the following example, where the -t argument specifies complete trust with the CT value.
  certutil -M -n trustCert -t “CT,CT,CT” -d /etc/pki/nssdb
- You can also use the certutil command to look at the contents of the NSS database, as shown in the following example.
  certutil -L -d /etc/pki/nssdb
  In FIPS mode, Semeru Runtimes does not support the usage of the NSS database with a password. If the pk12util tool prompts you to set up a password when you import the PKCS#12 file-based keystore, press Enter twice to set up an empty password.
  If a password is entered as prompted instead of an empty password, then the user encounters a CKR_PIN_INCORRECT error in the liberty logs.
Create a keystore entry in your server.xml file that references the NSS database where you imported your keys and certificates.
The following server.xml example shows the keystore configuration to run Open Liberty in FIPS mode on Semeru Runtimes. In this example, location=“/tmp/pkcs11cfg.cfg” specifies the path to the Liberty configuration file that you created in step 3.
```
<featureManager>
    <feature>transportSecurity-1.0</feature>
</featureManager>

    <ssl id=“defaultSSLConfig"
         keyStoreRef="defaultKeyStore"
         sslProtocol=“TLSv1.2” />

    <keyStore id="defaultKeyStore" password="Liberty"
         location=“/tmp/pkcs11cfg.cfg” type=“PKCS11”
         fileBased=“false” provider=“SunPKCS11-NSS-FIPS” />
```
In this example, the keystore type attribute is set to PKCS11, but PKCS11-NSS-FIPS is also a valid value. This configuration instructs Open Liberty to use the NSS PKCS#11-based keystore instead of a file-based keystore.

You can now start your Open Liberty server in FIPS mode.

For more information about Semeru Runtimes in FIPS mode, see FIPS certified cryptography in IBM Semeru Runtimes. For more information about Open Liberty TLS configuration, see Secure communication with TLS and the Transport Security feature.