Author Message
MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Question in regards to how the load balancer works for session affinity parameter. In another post you had suggested we use the following format which *should* direct calls to the appropriate server.

https://SERVER/services/TestService/ws/ucid?affinity=10.0.0.1&interactionID=UCID

This however doesn't work, and I need to do it several times before I hit the appropriate server.

Can you either elaborate on how to get this to direct to the appropriate server, or elaborate on this piece of redirection code from the document detailing how to redirect to the appropriate node from code?

if (!CallProperties.isHostedOnLocalNode(myCall)

{ String remoteNodeIp = CallProperties.getNodeThatHostsThisCall(myCall)
/* Redirect HTTP request to the correct node. */ }
JoelEzell
Joined: Nov 15, 2013
Messages: 780
Offline
Is session affinity enabled for your cluster? You can check this by going to the Cluster Administration dashboard, selecting your cluster, then selecting edit. You'll see a cluster attribute called "Is session affinity enabled?" If you need to enable it, you'll need to first put your cluster into "Deny New Service" mode.
MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Yes it is. We enabled it last week.
PatrickPeng
Joined: Feb 12, 2007
Messages: 9
Offline
Can you verify the affinity flag was defined to upstream in the lb_https_443.conf (or lb_https_443.conf.dis) similar as following?
upstream lb_http_backend_443 {
avaya.edp.csa affinity round_robin ;
keepalive 20000 ;
server 10.18.25.11:443 weight=1 max_fails=2 fail_timeout=15s ;
server 10.18.25.12:443 weight=1 max_fails=2 fail_timeout=15s ;
server 10.18.25.13:443 weight=1 max_fails=2 fail_timeout=15s ;
}

Next, we can collect more information by enabling the Nginx debug logging
(1) change the logging level to "debug" for error_log in /etc/nginx/nginx.conf (the default is "warn")
(2) run "service nginx reload"
(3) debug logging should be found in /var/log/Avaya/asset/nginx_error.log

Hopefully we can locate what went wrong of the "affinity" support from the log.

MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
it looks a little different, in that it has the ip_hash instead of round_robin, I'll attempt the logging option--

upstream lb_http_backend_443 {
avaya.edp.csa affinity ip_hash ;
keepalive 20000 ;
server IP:443 weight=1 max_fails=2 fail_timeout=15s ;
server IP:443 weight=1 max_fails=2 fail_timeout=15s ;
server IP:443 weight=1 max_fails=2 fail_timeout=15s ;
server IP:443 weight=1 max_fails=2 fail_timeout=15s ;
server IP:443 weight=1 max_fails=2 fail_timeout=15s ;
}
MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Looks like I don't have permissions to modify that file.

"nginx.conf" E212: Can't open file for writing
PatrickPeng
Joined: Feb 12, 2007
Messages: 9
Offline
Okay, let's try to collect more information in a different way
(1) run "traceHTTP" that should enable nginx http trace in /var/log/Avaya/trace/nginx_http.log
(2) collect the following files after a test request
- /var/log/Avaya/trace/nginx_http.log
- /var/log/Avaya/asset/nginx_error.log
- /var/log/messages
- tar all files under /etc/nginx
(3) the timestamp on execution of the test request

MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Ok, and to be clear, what server should I be collecting the logs from? So if I have a call pinned up to SERVER A, and I issue a web request to the load balancer address. Are you wanting them from LB or ServerA? I'm not familiar with how to collect logs from the load balancer IP, just the Breeze servers that have been taking the calls.

JoelEzell
Joined: Nov 15, 2013
Messages: 780
Offline
Hi Michael, if you go to the Breeze server administration page in System Manager, there will be an icon indicating which server is the active load balancer. That's the one you'll want to collect logs from.
MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Thought so, but wanted to be sure. I was unable to collect /var/log/messages, var/log/Avaya/asset/nginx_error.log, and some files in the nginx directory due to permissions. I've updated everything I was able to grab.

Filename nginx_http.log [Disk] Download
Filename nginx.zip [Disk] Download
MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Any feedback on this yet?
JoelEzell
Joined: Nov 15, 2013
Messages: 780
Offline
Hi Michael, apologies for the delayed response. Patrick has been out of the office over the last several days.

Since this one is proving to be a bit of a tough nut to crack, and because we've made some changes in this area in the last 18+ months since 3.2 was released, Breeze product leadership is directing us to not invest further effort in this particular investigation. Even if we found a bug, we would not be able to provide you a patch for 3.2. We really need to get your systems upgraded as soon as we can.

Sorry about that.

Joel
PatrickPeng
Joined: Feb 12, 2007
Messages: 9
Offline
I can see there was only one https request with affinity in the collected nginx_http logs
"https" "/services/TestService/ws/ucid?affinity=10.0.0.1&interactionID=12000151571525977562"

To support server affinity, the "affinity" has to be assigned to one of your upstream server IPs,
upstream lb_http_backend_443 {
avaya.edp.csa affinity ip_hash ;
keepalive 20000 ;
server 165.137.49.29:443 weight=1 max_fails=2 fail_timeout=15s ;
server 165.137.49.23:443 weight=1 max_fails=2 fail_timeout=15s ;
server 165.137.49.27:443 weight=1 max_fails=2 fail_timeout=15s ;
server 165.137.49.21:443 weight=1 max_fails=2 fail_timeout=15s ;
server 165.137.49.25:443 weight=1 max_fails=2 fail_timeout=15s ;
}

Since that didn't match to any of the upstream servers, the LB would then route the request to the upstream server based on the client IP hash value. In your case it always ends up to the same upstream server if the https request generated from a same client. You can disable the LB affinity (ip_hash) on cluster page via SMGR if you prefer to fall back to round robin.

MichaelNorman
Joined: Jun 3, 2015
Messages: 448
Offline
Thanks for the reply. I had 10.0.0.1 there going off some information earlier in the thread or maybe from another thread. Let me restate what we are trying to do to be clear.

Call is sequenced through Server A. In this environment there are several servers that could process the call. Call lands at the agent desktop and they handle the call. At some point in the agent/customer interaction, the agent will call a web service that we would want to direct to the same server that the call is sequenced through. We want to hit the same server because we are going to ingest audio to the call and collect digits.

How can we direct the call to the appropriate server if the agent desktop has no fore knowledge of which server the call was sequenced through? Or if this cannot be done directly through the load balancer, can we do it from code instead? The manual hints that this can be done but doesn't elaborate on the mechanism to do so:

if (!CallProperties.isHostedOnLocalNode(myCall)
{ String remoteNodeIp = CallProperties.getNodeThatHostsThisCall(myCall)
/* Redirect HTTP request to the correct node. */ }
JoelEzell
Joined: Nov 15, 2013
Messages: 780
Offline
Hi Michael, I'm checking with Patrick to make sure we have a common understanding of how the algorithm works. Hopefully I wasn't giving you the runaround. We'll be back with you tomorrow.
Go to:   
Mobile view