Using SAP PI 7.40, we were able to successfully connect to an Active MQ queue using a JMS adapter over a secure SSL connection by installing the AMQ cert within the Java Keystore (JKS) and then updating the additional parameters (namely setTrustStore and setTrustStorePassword) appropriately.
Furthermore, since SSLv3 is vulnerable to POODLE attacks, we have since then moved to a secure TLS connection. To move from SSL to TSL, you need to update JMS.QueueConnectionFactoryImpl.constructor by specifying the enabled protocols (TLSv1, TLSv1.1, or TLSv1.2).
FYI – TLS is enabled from the broker side (AMQ) by using the following parameter: ssl://localhost:61616?transport.enabledProtocols=TLSv1,TLSv1.1,TLSv1.2
Once the AMQ cert is installed in the JKS, and both the broker (AMQ) and client (SAP PI) are configured for either SSL or TLS the channel should connect successfully.
Hope you find this useful!