Workaround to browser maximum connection limit ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Workaround to browser maximum connection limit ?

GLASS mailing list
Hi guys,

In my app I have one scenario where we render huge reports. These reports could have say... 20/30 large tables as well as quite some charts and some other report elements. 

Previously we were using a single ajax request to generate the whole report html but that was a pain because client machine would have a really large TTFB. So I was wasting CPU and network at client machine while waiting. 

What we did now is that each report element renders a title + spinning while and does an ajax request which, on success does a #replaceWith:  with the real contents. The idea is to show the report as soon as possible and start replacing spinning wheels with real contents as soon as content is ready. That way I maximize CPU and network on client side. 

The second thing is that making this on AJAX calls, that would end up on different Gems on my GemStone which was very performant. I have 10 Seaside gems on a 8 cores CPU so all those AJAX request were load balanced via nginx over the 10 seaside gems, which on the other hand were split across all cores. Previously, with a single request, only one Gem took care and hence only one CPU core was used. 

This change was nice and improved performance. However, when I analyze requests, I see that I have many that are "Stalled". And yeah, on Chrome, they are all stalled when there is more than 6 requests to the same location. 

To conclude, it looks like i am doing what is called "Loading page content with many Ajax requests" [1]. But I still don't find an easy workaround. I would like to be able to use my 10 Gems over the 8 CPU cores....

Any idea?

Thanks in advance,



_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
On 03/20/2017 05:01 PM, Mariano Martinez Peck via Glass wrote:

> This change was nice and improved performance. However, when I analyze
> requests, I see that I have many that are "Stalled". And yeah, on
> Chrome, they are all stalled when there is more than 6 requests to the
> same location.
>
> To conclude, it looks like i am doing what is called "Loading page
> content with many Ajax requests" [1]. But I still don't find an easy
> workaround. I would like to be able to use my 10 Gems over the 8 CPU
> cores....
>
It's not clear to me whether the max parallel connections limit is
really "same domain" or "same host." Same host (IP address) makes more
sense, possibly. Either way, you could double the limit by giving your
machine two IP addresses (or map it to two domains, if that's what the
browser cares about) and spreading the requests around.

A hack and a nuisance, yes, but I don't immediately see any better option.

Regards,

-Martin

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
In reply to this post by GLASS mailing list
Hey Mariano,

On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass <[hidden email]> wrote:

The second thing is that making this on AJAX calls, that would end up on different Gems on my GemStone which was very performant. I have 10 Seaside gems on a 8 cores CPU so all those AJAX request were load balanced via nginx over the 10 seaside gems, which on the other hand were split across all cores. Previously, with a single request, only one Gem took care and hence only one CPU core was used. 

Quick question: does this mean these requests are "Seaside session-less”?
Because processing seaside requests in parallel is not possible (the session object is locked).

Johan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list


On Tue, Mar 21, 2017 at 3:20 AM, Johan Brichau via Glass <[hidden email]> wrote:
Hey Mariano,

On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass <[hidden email]> wrote:

The second thing is that making this on AJAX calls, that would end up on different Gems on my GemStone which was very performant. I have 10 Seaside gems on a 8 cores CPU so all those AJAX request were load balanced via nginx over the 10 seaside gems, which on the other hand were split across all cores. Previously, with a single request, only one Gem took care and hence only one CPU core was used. 

Quick question: does this mean these requests are "Seaside session-less”?

Uhhhh ... great question. I think they are indeed "quite session-less" ... but.. is there an easy way to check this? Of course, something easier than following code to see if I access session. Maybe simply putting some logging in session accessing code ? 

 
Because processing seaside requests in parallel is not possible (the session object is locked).


Yes, maybe this is the case for my scenario. But at least, the report can still be rendered ASAP rather than having to wait until fully done, even if all requests would not go parelel even if they are on multiple gems. 
But...honestly, I would love they go paralel....  do you have some reference or link explaining the locking? I remember some old Dale posts as well as some code but any hint would be appreciated. 
Have someone improved or make any progress on this locking / multi-gem issue?

Thanks in advance!
 

Johan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
Hi,

We use a background process to generate big reports. This is a topaz
session that logs in on its own (nothing to do with seaside as such)
and picks up the report to generate. It commits the report results in
GemStone and the UI aborts to pick up the progress and results.

There is not much sophistication here on the UI because we do not
display results until the report is done (only progress, which shows a
simple line of relative progress). We could display results I suppose,
but then you can sort them yet, and things like that.

Maybe you can spawn multiple background process jobs that each take a
bit of the work and commit the results as they do it. I suppose you've
already looked at the concurrency issues to do something like that?

If the rendering is heavy weight, you could perhaps generate some of
your html in the results of the report in stead of some report result
object.

Cheers
Otto

On Tue, Mar 21, 2017 at 12:41 PM, Mariano Martinez Peck via Glass
<[hidden email]> wrote:

>
>
> On Tue, Mar 21, 2017 at 3:20 AM, Johan Brichau via Glass
> <[hidden email]> wrote:
>>
>> Hey Mariano,
>>
>> On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass
>> <[hidden email]> wrote:
>>
>> The second thing is that making this on AJAX calls, that would end up on
>> different Gems on my GemStone which was very performant. I have 10 Seaside
>> gems on a 8 cores CPU so all those AJAX request were load balanced via nginx
>> over the 10 seaside gems, which on the other hand were split across all
>> cores. Previously, with a single request, only one Gem took care and hence
>> only one CPU core was used.
>>
>>
>> Quick question: does this mean these requests are "Seaside session-less”?
>
>
> Uhhhh ... great question. I think they are indeed "quite session-less" ...
> but.. is there an easy way to check this? Of course, something easier than
> following code to see if I access session. Maybe simply putting some logging
> in session accessing code ?
>
>
>>
>> Because processing seaside requests in parallel is not possible (the
>> session object is locked).
>
>
>
> Yes, maybe this is the case for my scenario. But at least, the report can
> still be rendered ASAP rather than having to wait until fully done, even if
> all requests would not go parelel even if they are on multiple gems.
> But...honestly, I would love they go paralel....  do you have some reference
> or link explaining the locking? I remember some old Dale posts as well as
> some code but any hint would be appreciated.
> Have someone improved or make any progress on this locking / multi-gem
> issue?
>
> Thanks in advance!
>
>>
>>
>> Johan
>>
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
Increase shared page cache size. Use ssd drives to cut seek time. Cluster data to pages so more data that you know will be needed occupies fewer pages. Cluster static data to pages.

Distributing work into separate gems is a great start, but it sounds like you are data access limited and so now more gems now wait for data. Even if you've done all mentioned above, there is still a cost of moving data from spc to temp obj cache of each of those gems. Because of that cost, you can achieve incredible performance (like 45 seconds cut to milliseconds) by using a work queue serviced by a single gem that has warm caches because it does the same kind of work (same data) repeatedly.

Code tuning techniques would also have incredible gains, but I'd have to tune that code myself because there are just too many tricks to list.

Paul Baumann


On Mar 21, 2017 6:52 AM, "Otto Behrens via Glass" <[hidden email]> wrote:
Hi,

We use a background process to generate big reports. This is a topaz
session that logs in on its own (nothing to do with seaside as such)
and picks up the report to generate. It commits the report results in
GemStone and the UI aborts to pick up the progress and results.

There is not much sophistication here on the UI because we do not
display results until the report is done (only progress, which shows a
simple line of relative progress). We could display results I suppose,
but then you can sort them yet, and things like that.

Maybe you can spawn multiple background process jobs that each take a
bit of the work and commit the results as they do it. I suppose you've
already looked at the concurrency issues to do something like that?

If the rendering is heavy weight, you could perhaps generate some of
your html in the results of the report in stead of some report result
object.

Cheers
Otto

On Tue, Mar 21, 2017 at 12:41 PM, Mariano Martinez Peck via Glass
<[hidden email]> wrote:
>
>
> On Tue, Mar 21, 2017 at 3:20 AM, Johan Brichau via Glass
> <[hidden email]> wrote:
>>
>> Hey Mariano,
>>
>> On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass
>> <[hidden email]> wrote:
>>
>> The second thing is that making this on AJAX calls, that would end up on
>> different Gems on my GemStone which was very performant. I have 10 Seaside
>> gems on a 8 cores CPU so all those AJAX request were load balanced via nginx
>> over the 10 seaside gems, which on the other hand were split across all
>> cores. Previously, with a single request, only one Gem took care and hence
>> only one CPU core was used.
>>
>>
>> Quick question: does this mean these requests are "Seaside session-less”?
>
>
> Uhhhh ... great question. I think they are indeed "quite session-less" ...
> but.. is there an easy way to check this? Of course, something easier than
> following code to see if I access session. Maybe simply putting some logging
> in session accessing code ?
>
>
>>
>> Because processing seaside requests in parallel is not possible (the
>> session object is locked).
>
>
>
> Yes, maybe this is the case for my scenario. But at least, the report can
> still be rendered ASAP rather than having to wait until fully done, even if
> all requests would not go parelel even if they are on multiple gems.
> But...honestly, I would love they go paralel....  do you have some reference
> or link explaining the locking? I remember some old Dale posts as well as
> some code but any hint would be appreciated.
> Have someone improved or make any progress on this locking / multi-gem
> issue?
>
> Thanks in advance!
>
>>
>>
>> Johan
>>
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
In reply to this post by GLASS mailing list

On Tue, Mar 21, 2017 at 7:52 AM, Otto Behrens <[hidden email]> wrote:
Hi,

We use a background process to generate big reports. This is a topaz
session that logs in on its own (nothing to do with seaside as such)
and picks up the report to generate. It commits the report results in
GemStone and the UI aborts to pick up the progress and results.

Yes, I thought about this too (as I have an extended version of your BackgroundProcess working nicely) but reports are Seaside and quite a bit of the effort happens on rendering rather than the "callback". 
 

There is not much sophistication here on the UI because we do not
display results until the report is done (only progress, which shows a
simple line of relative progress). We could display results I suppose,
but then you can sort them yet, and things like that.

Maybe you can spawn multiple background process jobs that each take a
bit of the work and commit the results as they do it. I suppose you've
already looked at the concurrency issues to do something like that?

Yes! And it's still a possibility but of course, not trivial. 
 

If the rendering is heavy weight, you could perhaps generate some of
your html in the results of the report in stead of some report result
object.


Yeah, this is the thing the rendering is heavy weight. 
mmmm I didn't understand your point here. Could you explain better? 

 
Cheers
Otto

On Tue, Mar 21, 2017 at 12:41 PM, Mariano Martinez Peck via Glass
<[hidden email]> wrote:
>
>
> On Tue, Mar 21, 2017 at 3:20 AM, Johan Brichau via Glass
> <[hidden email]> wrote:
>>
>> Hey Mariano,
>>
>> On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass
>> <[hidden email]> wrote:
>>
>> The second thing is that making this on AJAX calls, that would end up on
>> different Gems on my GemStone which was very performant. I have 10 Seaside
>> gems on a 8 cores CPU so all those AJAX request were load balanced via nginx
>> over the 10 seaside gems, which on the other hand were split across all
>> cores. Previously, with a single request, only one Gem took care and hence
>> only one CPU core was used.
>>
>>
>> Quick question: does this mean these requests are "Seaside session-less”?
>
>
> Uhhhh ... great question. I think they are indeed "quite session-less" ...
> but.. is there an easy way to check this? Of course, something easier than
> following code to see if I access session. Maybe simply putting some logging
> in session accessing code ?
>
>
>>
>> Because processing seaside requests in parallel is not possible (the
>> session object is locked).
>
>
>
> Yes, maybe this is the case for my scenario. But at least, the report can
> still be rendered ASAP rather than having to wait until fully done, even if
> all requests would not go parelel even if they are on multiple gems.
> But...honestly, I would love they go paralel....  do you have some reference
> or link explaining the locking? I remember some old Dale posts as well as
> some code but any hint would be appreciated.
> Have someone improved or make any progress on this locking / multi-gem
> issue?
>
> Thanks in advance!
>
>>
>>
>> Johan
>>
>> _______________________________________________
>> Glass mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/glass
>>
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
>
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass
>



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
In reply to this post by GLASS mailing list
Seems I answered to Mariano directly…
Sending this to list for reference..

-----

Mariano,

If your requests have a Seaside session key in it, and they are normal Seaside (ajax) callbacks, then I’m pretty sure that the Session lock will be applied in Gemstone.
The ‘session lock’ is not an actual lock: the request will abort and will be retried after some waiting time if another gem is processing a request for that same Seaside session.

However… that means your requests cannot be processed in parallel… even worse is that because of the waiting time, the net processing time will be longer that if they were queued to the same gem.
To avoid this, we use session affinity [1]

So… I’m wondering if the limit of parallel requests in the browser is really an issue if you cannot circumvent this session lock?

But you seem to describe you do so requests being processed in parallel (up to 6?)… so I’m still wondering what you are doing ;-)

Johan


On 21 Mar 2017, at 11:41, Mariano Martinez Peck <[hidden email]> wrote:



On Tue, Mar 21, 2017 at 3:20 AM, Johan Brichau via Glass <[hidden email]> wrote:
Hey Mariano,

On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass <[hidden email]> wrote:

The second thing is that making this on AJAX calls, that would end up on different Gems on my GemStone which was very performant. I have 10 Seaside gems on a 8 cores CPU so all those AJAX request were load balanced via nginx over the 10 seaside gems, which on the other hand were split across all cores. Previously, with a single request, only one Gem took care and hence only one CPU core was used. 

Quick question: does this mean these requests are "Seaside session-less”?

Uhhhh ... great question. I think they are indeed "quite session-less" ... but.. is there an easy way to check this? Of course, something easier than following code to see if I access session. Maybe simply putting some logging in session accessing code ? 

 
Because processing seaside requests in parallel is not possible (the session object is locked).


Yes, maybe this is the case for my scenario. But at least, the report can still be rendered ASAP rather than having to wait until fully done, even if all requests would not go parelel even if they are on multiple gems. 
But...honestly, I would love they go paralel....  do you have some reference or link explaining the locking? I remember some old Dale posts as well as some code but any hint would be appreciated. 
Have someone improved or make any progress on this locking / multi-gem issue?

Thanks in advance!
 

Johan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Workaround to browser maximum connection limit ?

GLASS mailing list
In reply to this post by GLASS mailing list
Also sent this one to Mariano directly instead of the list…

----

Hey Mariano,

The session-lock is only applicable if you call a service that is part of a Seaside session. Callbacks generated by Seaside are such an example.
If you would call a service that is not behind a Seaside session, it is not applicable.

It’s in WASession>>handle

So, Seaside-REST or Zinc-REST would not be affected.

So, this all depends if the data you need to retrieve is tied to a session or not...
If it is, you still might be able to register the data in the database and allow it to be retrieved session-less via the REST service. All you need is a way to uniquely and safely access it via the REST service.

This is, more-or-less what we do in a similar case where we pass generated reports to a Javascript app: a service vm produces the report data and it is being made available in a REST service where the Javascript app can pick it up.
All you need to do is some regular cleanup of the data, much like seaside session expiration.

…. that should allow you to burn that server’s cores :)

Hope this helps
Johan

On 21 Mar 2017, at 16:52, Mariano Martinez Peck <[hidden email]> wrote:



On Tue, Mar 21, 2017 at 11:04 AM, Johan Brichau <[hidden email]> wrote:
Mariano,

If your requests have a Seaside session key in it, and they are normal Seaside (ajax) callbacks, then I’m pretty sure that the Session lock will be applied in Gemstone.


Yeah, indeed, it's exactly as you describe :(
 
The ‘session lock’ is not an actual lock: the request will abort and will be retried after some waiting time if another gem is processing a request for that same Seaside session.

yes.. I found the method I was remembering (#seasideProcessRequestWithRetry: )
 

However… that means your requests cannot be processed in parallel… even worse is that because of the waiting time, the net processing time will be longer that if they were queued to the same gem.
To avoid this, we use session affinity [1]

Exactly. I understand that. 
 

So… I’m wondering if the limit of parallel requests in the browser is really an issue if you cannot circumvent this session lock?


Well, I was thinking that maybe the SSL shake, the POST, then nginx forwarding to FastCGi etc (I mean...all the work from one point up to the gem) may be solved if the ajax would happen immediatly even if stucked on the gem. But you are clearly right now this is not nearly as good as I thought it was :(
 
But you seem to describe you do so requests being processed in parallel (up to 6?)… so I’m still wondering what you are doing ;-)


Yeah, that SEEMED to while watching `htop` and see how cores were used. But I bet this was  UI thingy as probably the "usage" of each core was sequential in which each step was the gem that got the lock.


Honestly I am a bit sad that we cannot have multiple gems doing real parallel work. I guess my client is willing to pay for it. I understand that somehow the session must be synchronized across multiple possible gems, but can't we improve current implementation? I mean, a better approach that  lock + abort and retry....   Have you ever tried this? Did you have an idea even if you did not implement it? Does GemTalks ever provided a possible workaround? 


 
Johan


On 21 Mar 2017, at 11:41, Mariano Martinez Peck <[hidden email]> wrote:



On Tue, Mar 21, 2017 at 3:20 AM, Johan Brichau via Glass <[hidden email]> wrote:
Hey Mariano,

On 21 Mar 2017, at 01:01, Mariano Martinez Peck via Glass <[hidden email]> wrote:

The second thing is that making this on AJAX calls, that would end up on different Gems on my GemStone which was very performant. I have 10 Seaside gems on a 8 cores CPU so all those AJAX request were load balanced via nginx over the 10 seaside gems, which on the other hand were split across all cores. Previously, with a single request, only one Gem took care and hence only one CPU core was used. 

Quick question: does this mean these requests are "Seaside session-less”?

Uhhhh ... great question. I think they are indeed "quite session-less" ... but.. is there an easy way to check this? Of course, something easier than following code to see if I access session. Maybe simply putting some logging in session accessing code ? 

 
Because processing seaside requests in parallel is not possible (the session object is locked).


Yes, maybe this is the case for my scenario. But at least, the report can still be rendered ASAP rather than having to wait until fully done, even if all requests would not go parelel even if they are on multiple gems. 
But...honestly, I would love they go paralel....  do you have some reference or link explaining the locking? I remember some old Dale posts as well as some code but any hint would be appreciated. 
Have someone improved or make any progress on this locking / multi-gem issue?

Thanks in advance!
 

Johan

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




-- 




-- 


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Loading...