Author Message
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi All, I have an issue right now on my production environment. When my IVR receives plenty of calls consecutively. Tomcat is shutting down by itself, saying that it is out of memory. MY Application server is running of virtual environment with 8GB RAM Each. Please I need help to utilize it. I cannot find the tomcat 7 manager on its path.
See images of my temporary fix but it is not sufficient enough.
  • [Thumb - temp_fix.PNG]
[Disk] Download
  • [Thumb - issue.PNG]
[Disk] Download
WilsonYu
Joined: Nov 6, 2013
Messages: 3950
Offline
You may want to understand what some of the general but important JVM options mean first. The following link is a good start. It may be also beneficial to look more on the web about Tomcat performance tuning. The look of the error you posted seems you are running out of thread memory. This corresponds to the many concurrent calls. You may want to try reduce the stack size for each thread through xss.

http://www.tomcatexpert.com/blog/2011/11/22/performance-tuning-jvm-running-tomcat
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi, have read the link but I don't get it. I need to know how to fix it on the server itself. hoping for more replies on this issue. I want to maximize the 8GB Ram of the server and dedicate half of it for java heap or memory size of tomcat
WilsonYu
Joined: Nov 6, 2013
Messages: 3950
Offline
Tomcat/JVM tuning is kind of an art. It depends on the size of the apps on the system and the traffic happens at different time. I can't tell you exactly what to do and fix the problem. You sort of have to understand what the options are and run the JVM monitor to see where all the resources are allocated at peak time in order to do adjustment. Give what you described, you are running out of thread memory. It's not the heap memory so you may want to look at making Xss stack size smaller and adjusting garbage collection using the gc option. You may be some IT expert to help you.
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi All, still can't understand a thing about this issue. I have tried to just increase the memory allocation on catalina (based from the screenshot I gave). -Xsm512 and -XmX 1536. Still having same issues. I have a fail-over, but both servers have the same issue. Hope anyone can help based from their experience.
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi All, this is bugging me and our client right now. I need further assistance on this. I noticed that my tomcat and my eclipse environment is pointed to JDK, so what I did is to point it on JRE7. Restarted eclipse, restarted tomcat, deleted the -Xmx and -Xms in catalina.ini. IVR went live again, I runs smoothly for the first 6hours based on my assessment, then just this morning, it has the <b>same issue</b>. So I restarted it, then after around 20 to 30mins or so, it experienced the same issue again! I don't have any idea what might cause this, I used the same configuration already of our old environment (physical server) to this new server(VM SERVER). Any ideas please? Any inputs would help, I attached the logs please scroll down on the logs and you will see that there are calls then the issue came-up.
Filename logs.txt [Disk] Download
RossYakulis
Joined: Nov 6, 2013
Messages: 2652
Offline
Check the setting of "runtime-platform" in your web.xml file. My guess is it is set to "desktop" and not "Experience Portal"
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi Ross, as of the moment there is an Avaya SR ticket going through and we haven't seen it yet. May I know what web.xml file are you referring to and where can I possibly find it. Hoping for your response :) Thanks!
RossYakulis
Joined: Nov 6, 2013
Messages: 2652
Offline
If you open an SR then please do not post here, pick one or the other so as to not consume too many folks time.

The web.xml file is in YOUR APPLICATION. It is in the WEB-INF directory. That is where the servlet mappings are and other information are stored.

When you export an application you need to select the target platform (MPS, EP, VP ...) as well as the target application server. Based on the stack trace, your application is set for desktop and that WILL NOT EVER WORK in production. In desktop mode (or simulation mode) we start up threads and open sockets and etc to support the call flow highlighting and debugging that consume a lot of resources that you would not want to do in a production environment.
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi ross, apologies for that. I' somehow frustrated with the issue. I will check this one. I'll try to re-export the callflow.
Would you mind checking my attachment? I just want to confirm if this is the one you are pertaining to.
*Edited: just added the web.xml file.
Filename web.xml [Disk] Download
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi Ross, Have reviewed what you've said and you we're right, it was set to desktop, so I changed it to EPMS (Experience Portal).
Since I'm using Experience Portal 7.0, I assume EPMS will be also my runtime platform. Also, after I changed it, i noticed a tons of logs on the console when the IVR gets a call. Please see attached image. Thank you much again, i will monitor the server.
  • [Thumb - EPMS.PNG]
[Disk] Download
RossYakulis
Joined: Nov 6, 2013
Messages: 2652
Offline
Note the web.xml has


<context-param>
<param-name>runtime-Platform</param-name>
<param-value>Desktop</param-value>
<description>Runtime Platform {Desktop|Voice Portal|Experience Portal|Other|IR|MPS}</description>
</context-param>


for a deployed application it should NOT be Desktop.
  • [Thumb - Capture.PNG]
[Disk] Download
RossYakulis
Joined: Nov 6, 2013
Messages: 2652
Offline
You do not want to set that using project properties. You set the platform to "Experience Portal" when you export the application to a war file.
JimStvn
Joined: Jun 4, 2015
Messages: 111
Offline
Hi Ross, just to clarify, I won't be deploying my project as a war file. So my question is, what would be the project properties for the runtime platform of my callflows? Thanks
  • [Thumb - PProperties.PNG]
[Disk] Download
RossYakulis
Joined: Nov 6, 2013
Messages: 2652
Offline
REALLY!! ??!! You are going to run a production environment from your Eclipse development environment? The design and architecture of OD is you develop in Eclipse with Tomcat then export the application as a war to your choice of application server, Tomcat, Websphere, Weblogic.
Go to:   
Mobile view