Increasing the performances of a Seaside application
Locked
Increasing the performances of a Seaside application
 
|
Hello,
In the next weeks, I hope that I will start my first application with Seaside. The first version is ready. It's a set of REST web services. Now, I'm doing differents stress tests for evaluating the charge that my product can support. I'm using Pharo 1.2.1, Seaside, Seaside-REST, SqueakDBX, LDAPlayer and XML-Parser. For the connector, I use AJPPharoAdaptator for the link between my program and Apache. Have you ideas or good practices for optimising the performances of a Seaside application ? Of course, I can installing my application on differents servers and made load balancing but I think that the first thing that I must improve is the code himself. Working faster with a database and a LDAP are the first problems that I want resolve. Someone worked on the pools of connections ? What are the errors that I must avoid making ? It's a large subject but it could be benefit to share experience in the search of performance. Best regards Olivier ;-) www.auverlot.fr _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
2011/5/26 Olivier Auverlot <[hidden email]>:
> Hello, > > In the next weeks, I hope that I will start my first application with > Seaside. The first version is ready. It's a set of REST web services. Now, > I'm doing differents stress tests for evaluating the charge that my product > can support. > > I'm using Pharo 1.2.1, Seaside, Seaside-REST, SqueakDBX, LDAPlayer and > XML-Parser. For the connector, I use AJPPharoAdaptator for the link between > my program and Apache. > > Have you ideas or good practices for optimising the performances of a > Seaside application ? > > Of course, I can installing my application on differents servers and made > load balancing but I think that the first thing that I must improve is the > code himself. > > Working faster with a database and a LDAP are the first problems that I want > resolve. Someone worked on the pools of connections ? We once used a proprietary connection pool for PostgreS. > What are the errors that I must avoid making ? AFAIK SqueakDBX uses FFI which to my understanding means that it calls block the entire VM. That reduces the amount of concurrency a single image can handle. It also makes a connection pool more important. > It's a large subject but it could be benefit to share experience in the > search of performance. Make sure you're measuring the right thing ;-) Cheers Philippe _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
Yes, FFI blocks the whole VM while a functions is being called. However, if and only if the database client library supports asynchronous queries, SqueakDBX will work asynchronously. Basically, we do a loop where we ask the backend if the query was ready. If it was not, we do a yield (we let other process of the image to run) and then continue the loop. http://www.squeakdbx.org/documentation/Asynchronous%20queries Eliot implemented a multi-threaded FFI for Cog but so far I am not sure if it is already working. Cheers Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
On Sat, 28 May 2011, Mariano Martinez Peck wrote:
>> >>> What are the errors that I must avoid making ? >> >> AFAIK SqueakDBX uses FFI which to my understanding means that it calls >> block the entire VM. That reduces the amount of concurrency a single >> image can handle. It also makes a connection pool more important. >> >> > Yes, FFI blocks the whole VM while a functions is being called. However, if > and only if the database client library supports asynchronous queries, > SqueakDBX will work asynchronously. Basically, we do a loop where we ask the > backend if the query was ready. If it was not, we do a yield (we let other > process of the image to run) and then continue the loop. FYI: The expression "Processor yield" doesn't let all processes run, just those which have the same priority. So using it in a loop starves all lower priority processes and results in busy waiting. Levente > > http://www.squeakdbx.org/documentation/Asynchronous%20queries > > Eliot implemented a multi-threaded FFI for Cog but so far I am not sure if > it is already working. > > Cheers > > -- > Mariano > http://marianopeck.wordpress.com > seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
In reply to this post by Mariano Martinez Peck
2011/5/28 Mariano Martinez Peck <[hidden email]>:
> >> >> > What are the errors that I must avoid making ? >> >> AFAIK SqueakDBX uses FFI which to my understanding means that it calls >> block the entire VM. That reduces the amount of concurrency a single >> image can handle. It also makes a connection pool more important. >> > > Yes, FFI blocks the whole VM while a functions is being called. However, if > and only if the database client library supports asynchronous queries, > SqueakDBX will work asynchronously. Basically, we do a loop where we ask the > backend if the query was ready. If it was not, we do a yield (we let other > process of the image to run) and then continue the loop. How do you avoid turning this into busy awaiting and still manage to get woken up when data is available? Cheers Philippe _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
On Sun, May 29, 2011 at 10:55 AM, Philippe Marschall <[hidden email]> wrote: 2011/5/28 Mariano Martinez Peck <[hidden email]>: What we do is
-- Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
On Sun, May 29, 2011 at 1:54 PM, Mariano Martinez Peck <[hidden email]> wrote:
gmail shortcuts... What we do is: DBXBackend >> processNextResultSet: aConnection querySettings: aQuerySettings "Gets the next resultSet of the query. Depending on the type of query, it will return a DBXResult or DBXResultSet. If there is a timeout, it will cicle till this is finished." | returnCode | [self nextResultSet: aConnection querySettings: aQuerySettings onReturn: [:handle :code | returnCode := code. code = OpenDBX resultWithRows ifTrue: [ ^ self processResultWithRows: aConnection resultHandle: handle querySettings: aQuerySettings]. code = OpenDBX resultWithoutRows ifTrue: [ ^ self processResultWithoutRows: aConnection resultHandle: handle querySettings: aQuerySettings]. code = OpenDBX resultDone ifTrue: [^ nil]. (code = OpenDBX resultTimeout) ifTrue: [ (Delay forMilliseconds: (aQuerySettings timeout asMiliseconds)) wait ]. ]] doWhileTrue: [returnCode = OpenDBX resultTimeout]. OpenDBXDriverError signal: 'Uknown problem with executeStatement'. so....that is the ugly part of dbx... the method #nextResultSet:querySettings:onReturn: basically answers what the backend answers to: OpenDBX current apiQueryResult: aConnection handle handle: handleArray timeout: aQuerySettings timeoutAsDBXTimeSpec chunk: aQuerySettings pageSize. that answers what we call in the previous method "code". That "code" can mean "results wtih no row" for example when it was a DDL (a create table for example), a "result done" when we have finishes iterating all results , and "result timeout" that means that the query is not yet finished. In this case we have to always loop until we have one the two previous possibilities. Non of us is expert with processes and friends so we may be doing something wrong. If true, please let us know. Mariano
-- Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
In reply to this post by Levente Uzonyi-2
On Sun, May 29, 2011 at 4:06 AM, Levente Uzonyi <[hidden email]> wrote:
Thanks. I was not clear. What we actually do is: (code = OpenDBX resultTimeout) ifTrue: [ (Delay forMilliseconds: (aQuerySettings timeout asMiliseconds)) wait ]. Is that better? Even if it lets just run processes of the same priority, this is good anyway because what we want is at least be able to process other queries. Probably, those other processes are being done from other Process. Thanks Mariano
-- Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
On Sun, 29 May 2011, Mariano Martinez Peck wrote:
> On Sun, May 29, 2011 at 4:06 AM, Levente Uzonyi <[hidden email]> wrote: > >> On Sat, 28 May 2011, Mariano Martinez Peck wrote: >> >> >>>> What are the errors that I must avoid making ? >>>>> >>>> >>>> AFAIK SqueakDBX uses FFI which to my understanding means that it calls >>>> block the entire VM. That reduces the amount of concurrency a single >>>> image can handle. It also makes a connection pool more important. >>>> >>>> >>>> Yes, FFI blocks the whole VM while a functions is being called. However, >>> if >>> and only if the database client library supports asynchronous queries, >>> SqueakDBX will work asynchronously. Basically, we do a loop where we ask >>> the >>> backend if the query was ready. If it was not, we do a yield (we let other >>> process of the image to run) and then continue the loop. >>> >> >> FYI: The expression "Processor yield" doesn't let all processes run, just >> those which have the same priority. So using it in a loop starves all lower >> priority processes and results in busy waiting. >> >> > Thanks. I was not clear. What we actually do is: > > (code = OpenDBX resultTimeout) ifTrue: [ (Delay forMilliseconds: > (aQuerySettings timeout asMiliseconds)) wait ]. > > Is that better? Even if it lets just run processes of the same priority, > this is good anyway because what we want is at least be able to process > other queries. Probably, those other processes are being done from other > Process. It's a bit better. There's no starvation if the timeout is greater than zero, but it's still a form of busy waiting, and it limits the number of queries per second per connection to at most 1000 (actually 1000 / timeout). To compare this with our native implementation - PostgresV3 - I measured 6k+ queries per second per connection and it's still not optimized for Cog (#perform: is slow on Cog). Levente > > Thanks > > Mariano > > > >> >> Levente >> >> >> >>> http://www.squeakdbx.org/documentation/Asynchronous%20queries >>> >>> Eliot implemented a multi-threaded FFI for Cog but so far I am not sure if >>> it is already working. >>> >>> Cheers >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >>> _______________________________________________ >> seaside mailing list >> [hidden email] >> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside >> > > > > -- > Mariano > http://marianopeck.wordpress.com > seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
On Sun, May 29, 2011 at 2:19 PM, Levente Uzonyi <[hidden email]> wrote:
Thanks Levente. Unfortunatly I guess that's all we can do with a blocking FFI :(
-- Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
>
> > It's a bit better. There's no starvation if the timeout is greater than zero, but it's still a form of busy waiting, and it limits the number of queries per second per connection to at most 1000 (actually 1000 / timeout). To compare this with our native implementation - PostgresV3 - I measured 6k+ queries per second per connection and it's still not optimized for Cog (#perform: is slow on Cog). > > > Thanks Levente. Unfortunatly I guess that's all we can do with a blocking FFI :( Just that I understand why you do not have the same constraints that levente? What are they? because he does not have either a threaded ffi. Stef_______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
On Sun, May 29, 2011 at 4:18 PM, stephane ducasse <[hidden email]> wrote:
Because he uses a native client for PostgreSQL, that is, a driver implemented directly in Smalltalk: you open a socket and you implement the protocol the database specify. Easy. They use a Socket, and do not need FFI because the do not delegate to an external library. They implement the same as the external library but in Smalltalk. Why we don't do that? Because that ONLY and ONLY works for open-sources databases where they can make available the protocol specification so that you can build as many drivers as you want. However, non open-source databases (Oracle, SQLServer, Sybase, etc) DO NOT make the protocol available so you cannot just create your own driver. Instead, you have to use the driver THEY give you. Of course, for java they build drivers, so java guys do not depend on anybody, they just have they jdbc driver. SqueakDBX appeared to solve that problem. We, Smalltalk community, do not have that luck and we have to use one of the available drivers they do. All of the databases provide drivers written in C, and these are the easier to use from Smalltalk. Hence, we depend on an external library. The same happens with ODBC for example. Cheers
-- Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
thanks mariano
Stef On May 29, 2011, at 4:26 PM, Mariano Martinez Peck wrote: > > > On Sun, May 29, 2011 at 4:18 PM, stephane ducasse <[hidden email]> wrote: > > > > > > It's a bit better. There's no starvation if the timeout is greater than zero, but it's still a form of busy waiting, and it limits the number of queries per second per connection to at most 1000 (actually 1000 / timeout). To compare this with our native implementation - PostgresV3 - I measured 6k+ queries per second per connection and it's still not optimized for Cog (#perform: is slow on Cog). > > > > > > Thanks Levente. Unfortunatly I guess that's all we can do with a blocking FFI :( > > Just that I understand why you do not have the same constraints that levente? > What are they? because he does not have either a threaded ffi. > > Because he uses a native client for PostgreSQL, that is, a driver implemented directly in Smalltalk: you open a socket and you implement the protocol the database specify. Easy. They use a Socket, and do not need FFI because the do not delegate to an external library. They implement the same as the external library but in Smalltalk. > Why we don't do that? > Because that ONLY and ONLY works for open-sources databases where they can make available the protocol specification so that you can build as many drivers as you want. However, non open-source databases (Oracle, SQLServer, Sybase, etc) DO NOT make the protocol available so you cannot just create your own driver. Instead, you have to use the driver THEY give you. Of course, for java they build drivers, so java guys do not depend on anybody, they just have they jdbc driver. SqueakDBX appeared to solve that problem. > > We, Smalltalk community, do not have that luck and we have to use one of the available drivers they do. All of the databases provide drivers written in C, and these are the easier to use from Smalltalk. Hence, we depend on an external library. The same happens with ODBC for example. > > Cheers > > > > > Stef_______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > > > > -- > Mariano > http://marianopeck.wordpress.com > > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
In reply to this post by Mariano Martinez Peck
2011/5/29 Mariano Martinez Peck <[hidden email]>:
> > > On Sun, May 29, 2011 at 4:18 PM, stephane ducasse <[hidden email]> > wrote: >> >> > >> > >> > It's a bit better. There's no starvation if the timeout is greater than >> > zero, but it's still a form of busy waiting, and it limits the number of >> > queries per second per connection to at most 1000 (actually 1000 / timeout). >> > To compare this with our native implementation - PostgresV3 - I measured 6k+ >> > queries per second per connection and it's still not optimized for Cog >> > (#perform: is slow on Cog). >> > >> > >> > Thanks Levente. Unfortunatly I guess that's all we can do with a >> > blocking FFI :( >> >> Just that I understand why you do not have the same constraints that >> levente? >> What are they? because he does not have either a threaded ffi. > > Because he uses a native client for PostgreSQL, that is, a driver > implemented directly in Smalltalk: you open a socket and you implement the > protocol the database specify. Easy. They use a Socket, and do not need FFI > because the do not delegate to an external library. They implement the same > as the external library but in Smalltalk. > Why we don't do that? > Because that ONLY and ONLY works for open-sources databases where they can > make available the protocol specification so that you can build as many > drivers as you want. However, non open-source databases (Oracle, SQLServer, > Sybase, etc) DO NOT make the protocol available so you cannot just create > your own driver. At least for SQLServer/Sybase it seems to be available [1]. You can get it for Oracle, just not for free and not in an OSS compatible way. [1] http://en.wikipedia.org/wiki/Tabular_Data_Stream Cheers Philippe _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
In reply to this post by Mariano Martinez Peck
On Sun, 29 May 2011, Mariano Martinez Peck wrote:
> On Sun, May 29, 2011 at 2:19 PM, Levente Uzonyi <[hidden email]> wrote: > >> On Sun, 29 May 2011, Mariano Martinez Peck wrote: >> >> On Sun, May 29, 2011 at 4:06 AM, Levente Uzonyi <[hidden email]> wrote: >>> >>> On Sat, 28 May 2011, Mariano Martinez Peck wrote: >>>> >>>> >>>> What are the errors that I must avoid making ? >>>>>> >>>>>>> >>>>>>> >>>>>> AFAIK SqueakDBX uses FFI which to my understanding means that it calls >>>>>> block the entire VM. That reduces the amount of concurrency a single >>>>>> image can handle. It also makes a connection pool more important. >>>>>> >>>>>> >>>>>> Yes, FFI blocks the whole VM while a functions is being called. >>>>>> However, >>>>>> >>>>> if >>>>> and only if the database client library supports asynchronous queries, >>>>> SqueakDBX will work asynchronously. Basically, we do a loop where we ask >>>>> the >>>>> backend if the query was ready. If it was not, we do a yield (we let >>>>> other >>>>> process of the image to run) and then continue the loop. >>>>> >>>>> >>>> FYI: The expression "Processor yield" doesn't let all processes run, just >>>> those which have the same priority. So using it in a loop starves all >>>> lower >>>> priority processes and results in busy waiting. >>>> >>>> >>>> Thanks. I was not clear. What we actually do is: >>> >>> (code = OpenDBX resultTimeout) ifTrue: [ (Delay forMilliseconds: >>> (aQuerySettings timeout asMiliseconds)) wait ]. >>> >>> Is that better? Even if it lets just run processes of the same priority, >>> this is good anyway because what we want is at least be able to process >>> other queries. Probably, those other processes are being done from other >>> Process. >>> >> >> It's a bit better. There's no starvation if the timeout is greater than >> zero, but it's still a form of busy waiting, and it limits the number of >> queries per second per connection to at most 1000 (actually 1000 / timeout). >> To compare this with our native implementation - PostgresV3 - I measured 6k+ >> queries per second per connection and it's still not optimized for Cog >> (#perform: is slow on Cog). >> >> > Thanks Levente. Unfortunatly I guess that's all we can do with a blocking > FFI :( That's right. You can write a custom plugin to avoid blocking if you really want to, but I guess the FFI version's performance is enough for most applications. Levente > > > >> >> Levente >> >> >> >>> Thanks >>> >>> Mariano >>> >>> >>> >>> >>>> Levente >>>> >>>> >>>> >>>> http://www.squeakdbx.org/documentation/Asynchronous%20queries >>>>> >>>>> Eliot implemented a multi-threaded FFI for Cog but so far I am not sure >>>>> if >>>>> it is already working. >>>>> >>>>> Cheers >>>>> >>>>> -- >>>>> Mariano >>>>> http://marianopeck.wordpress.com >>>>> >>>>> _______________________________________________ >>>>> >>>> seaside mailing list >>>> [hidden email] >>>> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside >>>> >>>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >>> _______________________________________________ >> seaside mailing list >> [hidden email] >> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside >> > > > > -- > Mariano > http://marianopeck.wordpress.com > seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
We tried to write our own plugin. Check package "OpenDBX-Plugin" in http://www.squeaksource.com/SqueakDBX. I am not sure why we didn't succeeded. Maybe because our knowledge was no enough or maybe because its performance was not as good as expected. Maybe Esteban remembers why... Cheers Levente -- Mariano http://marianopeck.wordpress.com _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Increasing the performances of a Seaside application
|
|
Maybe one can use the HydraVM to launch the data accessing code on a
different OS thread. http://squeakvm.org/~sig/hydravm/devnotes.html - Darius _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
|
|
In reply to this post by Mariano Martinez Peck
El 29/05/2011, a las 2:59p.m., Mariano Martinez Peck escribió:
we didn't succeed because at the time we hadn't time to do it (there was a lot of "sync" issues to handle, not an easy task), and when finally we talk about this, Eliot was talking about the threaded ffi, so we decided to wait :)
_______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
«
Return to Seaside General
|
1 view|%1 views
| Free forum by Nabble | Edit this page |