Navigate to the C:\NetIQ\Common\JRE\conf\security folder.
Take a backup of the java.security file.
Edit the java.security file by executing the following steps:
Search for the List of providers and their preference orders string, comment all the existing providers, and then add the following entries:
security.provider.1=BCFIPS security.provider.2=BCJSSE security.provider.3=SunJSSE security.provider.4=SUN security.provider.5=SunJCE
Search for the ssl.KeyManagerFactory.algorithm=SunX509 string, comment it, and add:
ssl.KeyManagerFactory.algorithm=PKIX
Search for the jdk.tls.disabledAlgorithms string add TLSv1, TLSv1.1 to the list.
For example, jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, DH keySize < 1024, \EC keySize < 224, 3DES_EDE_CBC, anon, NULL
Search for the keystore.type string, comment it, and then add:
keystore.type=bcfks
Navigate to the C:\NetIQ\Common\JRE\lib\security folder.
Take a backup of the cacerts file.
To convert the keystore file from jks to bcfks, run the following command:
keytool -importkeystore -srckeystore C:\NetIQ\IDM\FanoutAgent\config\keystore.jks -srcstorepass netiq123 -srcstoretype PKCS12 -destkeystore C:\NetIQ\IDM\FanoutAgent\config\keystore.jks -deststorepass netiq123 -deststoretype BCFKS -providerclass org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider -providerpath $CLASSPATH
Navigate to the C:\NetIQ\IDM\FanoutAgent\bin\ folder and update the following changes in the startAgent.bat file:
"%JAVA%" --add-opens=java.base/sun.security.internal.spec=ALL-UNNAMED -Djavax.net.ssl.trustStore=C:\NetIQ\Common\JRE\lib\security\cacerts -Djavax.net.ssl.trustStorePassword=changeit -Dkeystore.type=BCFKS -Dorg.bouncycastle.fips.approved_only=true -Dorg.bouncycastle.rsa.allow_multi_use=true -cp %CLASSPATH% com.netiq.idm.fanoutagent.FanoutAgent !FILENAME! !SERVERONLY!.
Start the Fanout Agent and check logs.