Hi Team,

Thank yo for your support.

Occurrence scenario
1.In the runtimeconfig management console, specify the license server as follows.
https://[IP of EPM]:8443/WebLM/LicenseServer

2. When I press update, the following error is output.
License Url is invalid, or server is not up, or license version is too low.

We deployed to use AAOD 8.1.2 with WebSphere v9.0.2 and Standalone webLM server 10.1.2.
In this configuration, AAOD 8.1.2 runtime config with WebSphere unable to reach WebLM server 10.1.2 for the runtime licenses with https.
But we confirmed AAOD 8.1.2 runtime config with WebSphere able to reach WebLM server 10.1.2 for the runtime licenses with http.
Furthermore we confirmed AAOD 8.1.2 runtime config with Tomcat able to reach WebLM server 10.1.2 for the runtime licenses with https.

We got below message in SystemErr.log when AAOD 8.1.2 runtime config with WebSphere unable to reach WebLM server 10.1.2 for the runtime licenses with https.
(java.security.NoSuchAlgorithmException: SunX509 TrustManagerFactory not available)

And we also got below message in License status for platform on Validation Page of OD.
Error :com.avaya.weblm.LicenseException: : Unable to create SSLContext

If my understanding is correct, the Java system is able to look into the trust store file trusted_weblm_certs.jks in the lib directory for the certificate on Tomcat.
Therefore we truyed to import “trusted_weblm_certs.jks” using IBM console on WebSphere.
And we also enable TLS setting using IBM console as well with below.
SSL certificate and key management > SSL configurations > NodeDefaultSSLSettings > Quality of protection (QoP) settings.

How can I resolve above error message? Please advise.

We are using this system with below products.
[[ EPM ]]
<swversion>
Avaya Connector Server
Version: RH8.4.64-AV04EP8
Offer Type: bundled
Operating System Version
Linux 4.18.0-372.26.1.el8_6.x86_64
<iaversion.php>
installagent 8.1.2.0.0202 full
docs 8.1.2.0.0202 full
vpms 8.1.2.0.0202 full
vpms 8.1.2.0.0341 patch
docs 8.1.2.0.0341 patch

<OVA for deploy>
ExperiencePortal-Primary-EPM-8.1.2.0.0202.ova
<Patches>
epavl-8.x.x.0.2210.tar.gz
EPM_8.1.2.0.0328.tar.gz
EPM_8.1.2.0.0341.tar.gz

[[ Web Application Server ]]
Orchestration Designer?08.12.11.01
JAVA?IBM Java V1.8.0
WAS?WebSphere Application Server V9.0.2.x
OS?RHEL8.1

Best regards, K.Yamahara

Hi.
With IBM, the JVM should be "IbmX509". "SunX509" is the default value used by JBoss. Did you explicitly configure that name ?
Not clear which configuration of the WS faces this issue.

Hi Massimo-san,

Thank you for your quick reply.

>With IBM, the JVM should be "IbmX509". "SunX509" is the default value used by JBoss. Did you explicitly configure that name ?
[Kenji]
I believe answer is "No".
So, please advise how to configure the Name for "IbmX509"?

If there are some recommended procedure in Dev guide or other Devconn thread, please advise.

Best regards, Kenji

Take a look at this article: https://kb.avaya.com/kb/index?page=content&id=SOLN335073

Hi Massimo-san.

Thank you for your support.

>So the Tomcat is working with the same OD same WebLM, the issue rises up when using the WS , right ?
[Kenji]
You are right. Only Websphere case is NOT working.

>I see WebSphere 9 with Java 8.
>I faced a similar issue in Java 1.8 that was trying to connect to the WebLM using SSL v3 when WebSphere was configured to only accept TLS v1.2.
>Might you confirm the WS configuration is fine ?
[Kenji]
Yes. We are using WebSphere 9 with Java 1.8.
And we configured WebSphere to use "TLS v1.2".

As you said, I find below AVAYA knowledge base. It sees it is fit for our configuration. We will try to configure this solution.

Orchestrational Designer: Unable to get license from WebLM on WebSphere
https://support.avaya.com/ext/index?page=content&id=SOLN335073
Doc ID : SOLN335073

Details :
OD runtime cannot get valid license under WebSphere 9.0.0.6 with Java 8

trace.out log file:
17/12/2018 14:13:37:785 DEBUG - F1KBXv93DfMULQ8sD4bgpRJ:/PruMVP : Remaining grace period 2236206022
17/12/2018 14:13:37:785 DEBUG - F1KBXv93DfMULQ8sD4bgpRJ:/PruMVP : License is bad, allowing grace period
17/12/2018 14:13:37:793 DEBUG - F1KBXv93DfMULQ8sD4bgpRJ:/PruMVP : License refresh timer started
vpAppLogClient.log
2018-12-17 14:13:37,776 9485 [WebContainer : 0] DEBUG - ReportWriter:report() - level: Warning ||type=In Progress ||transactionName=- ||reason=Runtime License Error :com.avaya.weblm.LicenseException: Problem with connection to server: Received fatal alert: handshake_failure ||variableName=- ||variableData=- ||userLogInfo=- ||moduleIdNodeId=PruMVP:Start
SystemOut_date.log
[12/20/18 15:30:48:724 EST] 00000096 SystemOut O WebContainer : 0, READ: TLSv1 Alert, length = 2
[12/20/18 15:30:48:724 EST] 00000096 SystemOut O WebContainer : 0, RECV TLSv1.2 ALERT: fatal, handshake_failure
[12/20/18 15:30:48:725 EST] 00000096 SystemOut O WebContainer : 0, called closeSocket()
[12/20/18 15:30:48:725 EST] 00000096 SystemOut O WebContainer : 0, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Problem Clarification :
Issue with OD, WebSphere 9.x, and Java 8. OD is unable to obtain license file from the WebLM when it is hosted on a Websphere server.

Cause :
Issue in Java that is trying to connect to the WebLM using SSLv3 when WebSphere is configured to only accept TLSv1.2

Solution :
You may be able to fix this problem by adding the following Generic JVM arguments
-Dhttps.protocols=TLSv1.2 -Djdk.tls.client.protocols=TLSv1.2 -Dcom.ibm.jsse2.overrideDefaultTLS=true

Best regards, K.Yamahara

Hi Massimo-san.

Thank you for your support.

We confirmed there is no similar logs in our system.
trace.out log file:
17/12/2018 14:13:37:785 DEBUG - F1KBXv93DfMULQ8sD4bgpRJ:/PruMVP : Remaining grace period 2236206022
17/12/2018 14:13:37:785 DEBUG - F1KBXv93DfMULQ8sD4bgpRJ:/PruMVP : License is bad, allowing grace period
17/12/2018 14:13:37:793 DEBUG - F1KBXv93DfMULQ8sD4bgpRJ:/PruMVP : License refresh timer started
vpAppLogClient.log:
2018-12-17 14:13:37,776 9485 [WebContainer : 0] DEBUG - ReportWriter:report() - level: Warning ||type=In Progress ||transactionName=- ||reason=Runtime License Error :com.avaya.weblm.LicenseException: Problem with connection to server: Received fatal alert: handshake_failure ||variableName=- ||variableData=- ||userLogInfo=- ||moduleIdNodeId=PruMVP:Start
SystemOut_date.log:
[12/20/18 15:30:48:724 EST] 00000096 SystemOut O WebContainer : 0, READ: TLSv1 Alert, length = 2
[12/20/18 15:30:48:724 EST] 00000096 SystemOut O WebContainer : 0, RECV TLSv1.2 ALERT: fatal, handshake_failure
[12/20/18 15:30:48:725 EST] 00000096 SystemOut O WebContainer : 0, called closeSocket()
[12/20/18 15:30:48:725 EST] 00000096 SystemOut O WebContainer : 0, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

If my understanding correct, this issue was resolve java1.8 which used our site.
By the way , please advise which file or parameter should we change if we need to add below Generic JVM arguments.
-Dhttps.protocols=TLSv1.2 -Djdk.tls.client.protocols=TLSv1.2 -Dcom.ibm.jsse2.overrideDefaultTLS=true

Again, please let you advise below my question.
>With IBM, the JVM should be "IbmX509". "SunX509" is the default value used by JBoss. Did you explicitly configure that name ?
[Kenji]
I believe answer is "No".
So, please advise how to configure the Name for "IbmX509"?
If there are some recommended procedure in Dev guide or other Devconn thread, please advise.

Or if we need to configure some of setting related secure TLS communication with WebLM server, please advise what setting should we perform.
Best regards, Kenji

Hi Massimo-san,

Thank you for your support.

As you said below, we need to configure name for our configuration to use "IbmX509" to comminicate with Weblm with secure link.
So, please advise how to configure the Name for "IbmX509"?
If there are some recommended procedure in Dev guide or other Devconn thread, please advise.
Futhermore if we need to configure some of setting related secure TLS communication with WebLM server except Name, please advise what setting should we perform.

> with IBM, the JVM should be "IbmX509". "SunX509" is the default value used by JBoss. Did you explicitly configure that name ?

If those configuration defficults to change by ourselves, could you please release other runtimeconfig for secure link with Websphere.

Best regards, K.Yamahara

Hi.

- IbmX509

Evidence you need IbmX509: https://docs.jboss.org/jbossweb/2.1.x/ssl-howto.html;

To setup "IbmX509" on the WS, useful docs from IBM should be

https://www.ibm.com/docs/en/was-nd/9.0.5?topic=configurations-key-manager-control-x509-certificate-identities

https://www.ibm.com/docs/en/was-nd/9.0.5?topic=configuration-creating-custom-key-manager-ssl

anyway feel free to search on the IBM online docs;




- In case you see a handshake failure on logs, try to apply the solution on the online doc on previous posts.



- SSL/TLS

FAQ: https://www.ibm.com/support/pages/websphere-ssl-faq-learning-more-about-websphere-ssl

SSL configurations on the WS: https://www.ibm.com/docs/en/was/9.0.5?topic=ssl-configurations

Hi Massimo-san

I am faced with same situation.

With enabling JVM-SSL tracing enabled, and replacing type of trustmanager from websphere side,
Runtimeconfig web app always try to use "SunX509" algorithm instead of using "ibmX509" algorithm
when get instance of TrustManagerFactory.

Error log from websphere shown below.

[] 000002e0 SystemErr R java.security.NoSuchAlgorithmException: SunX509 TrustManagerFactory not available
[] 000002e0 SystemErr R at sun.security.jca.GetInstance.getInstance(GetInstance.java:171)
[] 000002e0 SystemErr R at javax.net.ssl.TrustManagerFactory.getInstance(TrustManagerFactory.java:11)
[] 000002e0 SystemErr R at com.avaya.weblm.HttpConnection.createSecureSocketContext(HttpConnection.java:302)
[] 000002e0 SystemErr R at com.avaya.weblm.HttpConnection.writeProperties(HttpConnection.java:121)
[] 000002e0 SystemErr R at com.avaya.weblm.RequestMsg.send(RequestMsg.java:102)
[] 000002e0 SystemErr R at com.avaya.weblm.LicenseManager.commonGetFeature(LicenseManager.java:551)
[] 000002e0 SystemErr R at com.avaya.weblm.LicenseManager.getFeature(LicenseManager.java:491)
[] 000002e0 SystemErr R at com.ibm._jsp._validate._jspService(_validate.java:858)
[] 000002e0 SystemErr R at com.ibm.ws.jsp.runtime.HttpJspBase.service(HttpJspBase.java:99)
[] 000002e0 SystemErr R at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)

As you noticed, IBM JVM has no "SunX509" algorithm capability.
I sarched for runtimeconfig configuration files for variables changing TrustManagerFactory algorithm,
but no paremeters configures that.

IBM Support team doubt that runtimeconfig application may have fixed(hardcorded) values for calling trustManagerfactory,
so please advicese me the way to change runtimeconfig to trustManagerfactory algorithm from "SunX509" to "ibmX509" when RuntimeConfig web app getting TrustManagerFactory?

Thank you in advance.
Yusuke

Hi Massimo-san,

Thank you for your reply.

I understood what did you said in your last comments.
We need to modify/configure the OD runtimeconfig for Websphere with some config commands to use secure communication with IBMX509.
Then we are trying to do that. However it is very difficult to complete it.
It is almost same as new development for OD runtimeconfig with Websphere.
So, could you completely provide how to configure the OD runtimeconfig for Websphere with secure communication(IBMX509) again?

Anyway, I believe AVAYA should provide the developer guide or application note how to modify/configure the OD runtimeconfig for Websphere. And It should be guaranteed as an official configuration method by AVAYA.
Because AVAYA said OD is supporting websphere.

Best regards, Kenji

Hi Massimo-san,

Thank you for your support.

I tried to made the config guide for OD Runtime License Server Secure Access Configuration for Websphere.
Could you review and update it?

And could you accept to handle this ticket by escalated SR?

Best regards, K.Yamahara

It seems this is a known issue.
Kenji is currently investigating on the resolution.
This thread will be updated as we clarify how to fix.

Hi Massimo,

We are also facing same issue with OD 8.1.2 with IBM WAS 9.0.5 connecting to fetch the WebLM OD license. As you mentioned, the thread will be updated. Could you please let us know as to how to resolve the issue with SUNX509.

Description of the issue:

In the EP, the OD Runtime 8.1.2 is unable to get license from WebLM on the IBM WebSphere (IBM WAS) 9 using the JRE (Java) 1.8



Error messages:

- On MPP logs
Error initializing socket factory SSL context: SunX509 TrustManagerFactory not available

- On SystemErr.log
[] 000002e0 SystemErr R java.security.NoSuchAlgorithmException: SunX509 TrustManagerFactory not available

- On License status for platform on Validation Page of OD
Error :com.avaya.weblm.LicenseException: : Unable to create SSLContext




Root cause:

The OD-runtime (OD runtime files), deployed on the IBM WAS, cannot establish a secure HTTPS connection to the WebLM client so there is no connection to the WebLM server in order to catch the license.
The reason is that the IBM JVM (on top of which the IBM WAS is running) does support IbmX509 but does not support SunX509 hardcoded in the weblm.jar file in the class com.avaya.weblm.HttpConnection (liable to the HTTPS connection).
See picture below.
This problem does not affect neither Apache Tomcat nor JBoss/WildFly cause the JVM they are running on supports SunX509.



Solution:

You need the specific version of the weblm.jar file hardcoding IbmX509.




How to get the specific jar file fixing this issue:

- Using your Avaya SSO Login, open a Service Request (SR) ticket through the ACS (Support Request) here: https://support.avaya.com/support/en/secure/service-requests
In case the engineer needs additional details concerning this file, feel free to contact me at croci1@avaya.com providing the SR number (compulsory), I'll reply there.

- You can wait for it to become part of the next GA build (no ETA or tentative ETA available at this moment).



Thanks Kenji for your help !