Hi Team,

I have a customer who has reported the following issue:

• SCESession object is retaining a significant portion of the heap
• A growth of 50MB is observed over a period of 4 days in the memory retained by SCESession object - Comparison of baseline (13th Sep) and primary heapdump (17th Sep).
• In a non-stop run of 15days, SCESession object occupied 43% (200MB) of the heap

• com.avaya.sce.runtimecommon.SCESession – Interestingly this class seems to be retaining more of the heap size if you look at the utilization over a period of a week. Need further analysis with the developers to understand what is retaining memory

Number of Objects Used Heap Size Retained Heap Size
1,001 32,032 60,593,960
(A week later)
2,621 83,872 145,792,304


They checked through the devconnect site for any known issues with memory leak and found out that DD6 has a similar memory leak issue.
I have not been able to see any patches or workitems in the DD/AAOD-patches site related to the SCESession object!

Based on information shared they would like to know why SCESession class is leaking more memory overtime , is this a known issue?
Is there any patch that can fix it ? on present DD 5.1 version?

Also the customer would like to know the below:
What happens to SCEsession object when the IVR app does not respond within the specified time frame. Does the object get destroyed? I don’t think so because the object is destroyed only when the Voice portal makes a call to AvayaSessionTerminationServlet telling Dialog Designer framework to remove the session. So if we get a bad fetch error on MPPs because of the IVR app, will the voice portal call the AvayaSessionTerminationServlet to instruct the DD framework to remove the SCESession object which was acquired for the call?

Could you please let me know if you need any further information or heap dumps to be able to advice on the issue.

Note:


Customer is requesting inputs on priority as it is impacting their business. So appreciate if the inputs are shared at the earliest.

Thanks in advance.

Regards,
Sandeep.

I did not see any specific patch for this. Note 5.1 is past end of support. I suggest you upgrade to 6.0 or 7.0

Hi Ross,

Thanks for the update. Could you please share your inputs on the below query of the customer:

What happens to SCEsession object when the IVR app does not respond within the specified time frame. Does the object get destroyed? I don’t think so because the object is destroyed only when the Voice portal makes a call to AvayaSessionTerminationServlet telling Dialog Designer framework to remove the session. So if we get a bad fetch error on MPPs because of the IVR app, will the voice portal call the AvayaSessionTerminationServlet to instruct the DD framework to remove the SCESession object which was acquired for the call?


Thnaks,
Sandeep.

The SCESession is bound to the http session between the VXI and OD runtime, so worse case when the http session times out (default 30 minutes) the SCESession would be destroyed.

You may also want to look at the GC settings the customer is using.

Ross,

Pardon me, but what is GC setting? and where is the location of it?

Garbage Collection

From the release notes:

27. To optimize garbage collection performance on the Application Server, we recommend adding the
following:
For Windows: Create a system level environment variable:
export JAVA_OPTS = “-server -Xmx1024m -XX:MaxNewSize=30m -XX:+UseParNewGC -
XX:+UseConcMarkSweepGC-XX:CMSInitiatingOccupancyFraction=60 –
XX:ThreadStackSize=512”
For Linux: when Tomcat is started as a service, add the flowing to the beginning of your
/etc/init.d/tomcat file:
JAVA_OPTS="-server -Xmx1024M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -
XX:ThreadStackSize=512"
ulimit -n 8192
NOTE: The Xmx1024M parameter we try to set to somewhere around half the size of the RAM
configured on the box.
Restart the machine after setting the JAVA_OPTS, and verify that the new setting are in effect by
doing an: ps -ef | grep java

------------

Note this would ONLY be a starting point. Each application is different and ultimately the settings will need to be altered for the specific applications deployed.

Aha Thanks.

ross

Is the setting for Linux and AIX the same?

Customer has the below query.
As confirmed by the devconnect engineer,

“The SCESession is bound to the http session between the VXI and OD runtime, so worse case when the http session times out (default 30 minutes) the SCESession would be destroyed. ”

What if it’s not happening like in our case. We can see SCESession object being retained in the heap. That will only happen if it’s not being destroyed.
What would be the expected behavior on a very busy system like ours where we are churning sessions at a rate of knots. We do see a lot of timeouts and bad fetch errors on the MPPs.
Are there any logs which we can collect to confirm what could be happening in the background, on the IVR application end?

It is a "java" setting so should be. You could reduce the session timeout a bit but not too low. We are not seeing this issue else where. Again you should probably move to a more current release.