[vwnc] [Opentalk] Debugging RST frames
Locked
[vwnc] [Opentalk] Debugging RST frames
 
|
I was wondering if anyone had experienced seemingly random RST frames
coming from Opentalk-Seaside sockets? In our production environment we are seeing occasional failing requests where the load balancer is getting a RST from the backend server as per attached. This capture is seen from the inside interface of the LB. Looking at line 1323, where the client is initiating a new connection to the VIP, which is being redirected to 10.220.38.157:9001 which is where Opentalk broker is bound. The three way handshake completes, a PSH packet is sent, and then on 1330 and 1331 the server sends back a couple of RSTs. Now, it is certainly possible that other parties might be at fault, namely TCP stack on the OS and such, but I'd like to see if there's a good way of putting a logging hook somewhere close to the socket in the Opentalk framework to help me determine whether those requests ever actually make it that far? There's definitely no mention of them in WADispatcher>>handlerForRequest: so either the OS is interfering, or something is happening earlier on when transport is processing the request... Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
image001.png (327K)
Re: [vwnc] [Opentalk] Debugging RST frames
|
|
Boris,
What is the OS? What is the VM version you're using? Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Boris Popov Sent: Tuesday, March 03, 2009 3:29 PM To: [hidden email] Subject: [vwnc] [Opentalk] Debugging RST frames I was wondering if anyone had experienced seemingly random RST frames coming from Opentalk-Seaside sockets? In our production environment we are seeing occasional failing requests where the load balancer is getting a RST from the backend server as per attached. This capture is seen from the inside interface of the LB. Looking at line 1323, where the client is initiating a new connection to the VIP, which is being redirected to 10.220.38.157:9001 which is where Opentalk broker is bound. The three way handshake completes, a PSH packet is sent, and then on 1330 and 1331 the server sends back a couple of RSTs. Now, it is certainly possible that other parties might be at fault, namely TCP stack on the OS and such, but I'd like to see if there's a good way of putting a logging hook somewhere close to the socket in the Opentalk framework to help me determine whether those requests ever actually make it that far? There's definitely no mention of them in WADispatcher>>handlerForRequest: so either the OS is interfering, or something is happening earlier on when transport is processing the request... Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] [Opentalk] Debugging RST frames
|
|
Andres,
Sorry, my bad. OS Name: Microsoft(R) Windows(R) Server 2003, Standard Edition OS Version: 5.2.3790 Service Pack 2 Build 3790 VisualWorks(R) 7.6b Jan 13 2009 Copyright (C) 1999-2008 Cincom Systems, Inc. Release Identification: VM version: 64 platform: 41 release: 64.16 flags: 0 VI version: 0 minor: 0 release: 0 variant: 0 From version: 0 platform: 0 release: 0.1 flags: 0 This was also happening with the stock 7.6 VM. Hope this helps, -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Tuesday, March 03, 2009 4:42 PM To: VW NC Subject: Re: [vwnc] [Opentalk] Debugging RST frames Boris, What is the OS? What is the VM version you're using? Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Boris Popov Sent: Tuesday, March 03, 2009 3:29 PM To: [hidden email] Subject: [vwnc] [Opentalk] Debugging RST frames I was wondering if anyone had experienced seemingly random RST frames coming from Opentalk-Seaside sockets? In our production environment we are seeing occasional failing requests where the load balancer is getting a RST from the backend server as per attached. This capture is seen from the inside interface of the LB. Looking at line 1323, where the client is initiating a new connection to the VIP, which is being redirected to 10.220.38.157:9001 which is where Opentalk broker is bound. The three way handshake completes, a PSH packet is sent, and then on 1330 and 1331 the server sends back a couple of RSTs. Now, it is certainly possible that other parties might be at fault, namely TCP stack on the OS and such, but I'd like to see if there's a good way of putting a logging hook somewhere close to the socket in the Opentalk framework to help me determine whether those requests ever actually make it that far? There's definitely no mention of them in WADispatcher>>handlerForRequest: so either the OS is interfering, or something is happening earlier on when transport is processing the request... Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] [Opentalk] Debugging RST frames
|
|
Hmmm, ok. Several image level fixes that take advantage of the 7.6b VM
improvements for Windows are not yet integrated in our 7.7 builds. I would say that, from what I have seen in the past, it is likely that these issues could be addressed by the image level changes. I'd recommend contacting support about this issue. Andres. -----Original Message----- From: Boris Popov [mailto:[hidden email]] Sent: Tuesday, March 03, 2009 4:49 PM To: Valloud, Andres; VW NC Subject: RE: [vwnc] [Opentalk] Debugging RST frames Andres, Sorry, my bad. OS Name: Microsoft(R) Windows(R) Server 2003, Standard Edition OS Version: 5.2.3790 Service Pack 2 Build 3790 VisualWorks(R) 7.6b Jan 13 2009 Copyright (C) 1999-2008 Cincom Systems, Inc. Release Identification: VM version: 64 platform: 41 release: 64.16 flags: 0 VI version: 0 minor: 0 release: 0 variant: 0 From version: 0 platform: 0 release: 0.1 flags: 0 This was also happening with the stock 7.6 VM. Hope this helps, -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Tuesday, March 03, 2009 4:42 PM To: VW NC Subject: Re: [vwnc] [Opentalk] Debugging RST frames Boris, What is the OS? What is the VM version you're using? Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Boris Popov Sent: Tuesday, March 03, 2009 3:29 PM To: [hidden email] Subject: [vwnc] [Opentalk] Debugging RST frames I was wondering if anyone had experienced seemingly random RST frames coming from Opentalk-Seaside sockets? In our production environment we are seeing occasional failing requests where the load balancer is getting a RST from the backend server as per attached. This capture is seen from the inside interface of the LB. Looking at line 1323, where the client is initiating a new connection to the VIP, which is being redirected to 10.220.38.157:9001 which is where Opentalk broker is bound. The three way handshake completes, a PSH packet is sent, and then on 1330 and 1331 the server sends back a couple of RSTs. Now, it is certainly possible that other parties might be at fault, namely TCP stack on the OS and such, but I'd like to see if there's a good way of putting a logging hook somewhere close to the socket in the Opentalk framework to help me determine whether those requests ever actually make it that far? There's definitely no mention of them in WADispatcher>>handlerForRequest: so either the OS is interfering, or something is happening earlier on when transport is processing the request... Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] [Opentalk] Debugging RST frames
|
|
Andres,
Do you think I should still try the latest dev build? Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Tuesday, March 03, 2009 4:57 PM To: VW NC Subject: Re: [vwnc] [Opentalk] Debugging RST frames Hmmm, ok. Several image level fixes that take advantage of the 7.6b VM improvements for Windows are not yet integrated in our 7.7 builds. I would say that, from what I have seen in the past, it is likely that these issues could be addressed by the image level changes. I'd recommend contacting support about this issue. Andres. -----Original Message----- From: Boris Popov [mailto:[hidden email]] Sent: Tuesday, March 03, 2009 4:49 PM To: Valloud, Andres; VW NC Subject: RE: [vwnc] [Opentalk] Debugging RST frames Andres, Sorry, my bad. OS Name: Microsoft(R) Windows(R) Server 2003, Standard Edition OS Version: 5.2.3790 Service Pack 2 Build 3790 VisualWorks(R) 7.6b Jan 13 2009 Copyright (C) 1999-2008 Cincom Systems, Inc. Release Identification: VM version: 64 platform: 41 release: 64.16 flags: 0 VI version: 0 minor: 0 release: 0 variant: 0 From version: 0 platform: 0 release: 0.1 flags: 0 This was also happening with the stock 7.6 VM. Hope this helps, -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Tuesday, March 03, 2009 4:42 PM To: VW NC Subject: Re: [vwnc] [Opentalk] Debugging RST frames Boris, What is the OS? What is the VM version you're using? Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Boris Popov Sent: Tuesday, March 03, 2009 3:29 PM To: [hidden email] Subject: [vwnc] [Opentalk] Debugging RST frames I was wondering if anyone had experienced seemingly random RST frames coming from Opentalk-Seaside sockets? In our production environment we are seeing occasional failing requests where the load balancer is getting a RST from the backend server as per attached. This capture is seen from the inside interface of the LB. Looking at line 1323, where the client is initiating a new connection to the VIP, which is being redirected to 10.220.38.157:9001 which is where Opentalk broker is bound. The three way handshake completes, a PSH packet is sent, and then on 1330 and 1331 the server sends back a couple of RSTs. Now, it is certainly possible that other parties might be at fault, namely TCP stack on the OS and such, but I'd like to see if there's a good way of putting a logging hook somewhere close to the socket in the Opentalk framework to help me determine whether those requests ever actually make it that far? There's definitely no mention of them in WADispatcher>>handlerForRequest: so either the OS is interfering, or something is happening earlier on when transport is processing the request... Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] [Opentalk] Debugging RST frames
|
|
I think those changes are not in there yet.
-----Original Message----- From: Boris Popov [mailto:[hidden email]] Sent: Tuesday, March 03, 2009 5:02 PM To: Valloud, Andres; VW NC Subject: RE: [vwnc] [Opentalk] Debugging RST frames Andres, Do you think I should still try the latest dev build? Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Tuesday, March 03, 2009 4:57 PM To: VW NC Subject: Re: [vwnc] [Opentalk] Debugging RST frames Hmmm, ok. Several image level fixes that take advantage of the 7.6b VM improvements for Windows are not yet integrated in our 7.7 builds. I would say that, from what I have seen in the past, it is likely that these issues could be addressed by the image level changes. I'd recommend contacting support about this issue. Andres. -----Original Message----- From: Boris Popov [mailto:[hidden email]] Sent: Tuesday, March 03, 2009 4:49 PM To: Valloud, Andres; VW NC Subject: RE: [vwnc] [Opentalk] Debugging RST frames Andres, Sorry, my bad. OS Name: Microsoft(R) Windows(R) Server 2003, Standard Edition OS Version: 5.2.3790 Service Pack 2 Build 3790 VisualWorks(R) 7.6b Jan 13 2009 Copyright (C) 1999-2008 Cincom Systems, Inc. Release Identification: VM version: 64 platform: 41 release: 64.16 flags: 0 VI version: 0 minor: 0 release: 0 variant: 0 From version: 0 platform: 0 release: 0.1 flags: 0 This was also happening with the stock 7.6 VM. Hope this helps, -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Tuesday, March 03, 2009 4:42 PM To: VW NC Subject: Re: [vwnc] [Opentalk] Debugging RST frames Boris, What is the OS? What is the VM version you're using? Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Boris Popov Sent: Tuesday, March 03, 2009 3:29 PM To: [hidden email] Subject: [vwnc] [Opentalk] Debugging RST frames I was wondering if anyone had experienced seemingly random RST frames coming from Opentalk-Seaside sockets? In our production environment we are seeing occasional failing requests where the load balancer is getting a RST from the backend server as per attached. This capture is seen from the inside interface of the LB. Looking at line 1323, where the client is initiating a new connection to the VIP, which is being redirected to 10.220.38.157:9001 which is where Opentalk broker is bound. The three way handshake completes, a PSH packet is sent, and then on 1330 and 1331 the server sends back a couple of RSTs. Now, it is certainly possible that other parties might be at fault, namely TCP stack on the OS and such, but I'd like to see if there's a good way of putting a logging hook somewhere close to the socket in the Opentalk framework to help me determine whether those requests ever actually make it that far? There's definitely no mention of them in WADispatcher>>handlerForRequest: so either the OS is interfering, or something is happening earlier on when transport is processing the request... Thanks! -Boris -- +1.604.689.0322 DeepCove Labs Ltd. 4th floor 595 Howe Street Vancouver, Canada V6C 2T5 http://tinyurl.com/r7uw4 [hidden email] CONFIDENTIALITY NOTICE This email is intended only for the persons named in the message header. Unless otherwise indicated, it contains information that is private and confidential. If you have received it in error, please notify the sender and delete the entire message including any attachments. Thank you. _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] [Opentalk] Debugging RST frames
|
|
In reply to this post by Boris Popov, DeepCove Labs (SNN)
Ill get in touch with support, thanks! _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
«
Return to VisualWorks
|
1 view|%1 views
| Free forum by Nabble | Edit this page |