Fwd: [Seaside] Progress bar for database query

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: [Seaside] Progress bar for database query

GLASS mailing list
Hi Dario,

It looks strange to me that a query to GemStone could take that much. Did you profile it?  how many rows has the collection? which type of collection is it?   the complexity / time goes into the "condition" rather in the select?
I ask because many times I was in the same situation, after a quick profiling, I noticed that adding an index would solve it.

Cheers,
 

---------- Forwarded message ----------
From: [hidden email] <[hidden email]>
Date: Thu, Oct 15, 2015 at 9:23 AM
Subject: [Seaside] Progress bar for database query
To: seaside-wiki <[hidden email]>


Ciao,

        i have a seaside application with some user interface.

        Now i have one specific user request for query the database.

        This request required some long time on the server before answer.

        My goal is to :

                a) display a progress bar when the user submit this query

                b)  block the web user request until the server answer the data query result


        How i can implemented this solution?

        Thank for any considerations.


                Dario

        _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside



--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Seaside] Progress bar for database query

GLASS mailing list
Ciao Mariano,

thanks for your considerations.

Hi Dario,

It looks strange to me that a query to GemStone could take that much. Did you profile it?  how many rows has the collection? which type of collection is it?   the complexity / time goes into the "condition" rather in the select?
I ask because many times I was in the same situation, after a quick profiling,
I don't have experience into index.

I noticed that adding an index would solve it.

I defined my request as 'query'   but, i think, it is not a query on possible indexed data.

My query is relative to the some dictionaries ( one dictionary for any day ) everyone with a dictionary about the documents ( 200 - 300 )  managed on the day.

Every document is based on a bag of items ecc.  ecc.. ecc...

With my surprise when i have 2 - 3 moths on-line all works fine.  ( the SPC is set to 2097152 )

But now when i have 6 months on-line the system answer with some delay.

My goal for now is to have online 12 months.

I need to change  my internal structure ?

Cheers,
Dario



Cheers,
 

---------- Forwarded message ----------
From: [hidden email] <[hidden email]>
Date: Thu, Oct 15, 2015 at 9:23 AM
Subject: [Seaside] Progress bar for database query
To: seaside-wiki <[hidden email]>


Ciao,

        i have a seaside application with some user interface.

        Now i have one specific user request for query the database.

        This request required some long time on the server before answer.

        My goal is to :

                a) display a progress bar when the user submit this query

                b)  block the web user request until the server answer the data query result


        How i can implemented this solution?

        Thank for any considerations.


                Dario

        _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside



--
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Seaside] Progress bar for database query

GLASS mailing list
Dario,

With the years I have learned that I will  not spend a single second of my life anymore trying to optimize something without having previously profile it. Most of the times, the bottleneck is elsewhere where you imagined. So if I were you, before doing anything, I would profile the report and understand where is the bottleneck. And then analyze possible solutions: algorithmic, database index, kind of collection used, collection message used, gemstone tunning (do you have native code enabled? ), etc.

Let me know if you need help with the profiling.

Cheers, 


On Thu, Oct 15, 2015 at 11:28 AM, Trussardi Dario Romano via Glass <[hidden email]> wrote:
Ciao Mariano,

thanks for your considerations.

Hi Dario,

It looks strange to me that a query to GemStone could take that much. Did you profile it?  how many rows has the collection? which type of collection is it?   the complexity / time goes into the "condition" rather in the select?
I ask because many times I was in the same situation, after a quick profiling,
I don't have experience into index.

I noticed that adding an index would solve it.

I defined my request as 'query'   but, i think, it is not a query on possible indexed data.

My query is relative to the some dictionaries ( one dictionary for any day ) everyone with a dictionary about the documents ( 200 - 300 )  managed on the day.

Every document is based on a bag of items ecc.  ecc.. ecc...

With my surprise when i have 2 - 3 moths on-line all works fine.  ( the SPC is set to 2097152 )

But now when i have 6 months on-line the system answer with some delay.

My goal for now is to have online 12 months.

I need to change  my internal structure ?

Cheers,
Dario



Cheers,
 

---------- Forwarded message ----------
From: [hidden email] <[hidden email]>
Date: Thu, Oct 15, 2015 at 9:23 AM
Subject: [Seaside] Progress bar for database query
To: seaside-wiki <[hidden email]>


Ciao,

        i have a seaside application with some user interface.

        Now i have one specific user request for query the database.

        This request required some long time on the server before answer.

        My goal is to :

                a) display a progress bar when the user submit this query

                b)  block the web user request until the server answer the data query result


        How i can implemented this solution?

        Thank for any considerations.


                Dario

        _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside



--
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Seaside] Progress bar for database query

GLASS mailing list
| profMon |
profMon := ProfMonitorTree new.
profMon interval: 10.
 profMon startMonitoring.

self HERE_GOES_YOUR_CODE_EXECUTION.

 profMon
        stopMonitoring;
        gatherResults.

(FileStream forceNewFileNamed: '/Users/mariano/profile.txt') nextPutAll: (profMon reportDownTo: 1); cr.


Ok, replace "self HERE_GOES_YOUR_CODE_EXECUTION." with your real code you want to profile.
Change internal if you want samples to be more or less frequent (I would let 10).
Change reportDownTo: to see until how much you want to print in the file. If 1 is too much large file, you may want to try 10 or 100.
Change profile.txt location

Then simply inspect profile.txt, go to the very end and analyze where the time is spent.

Cheers,




On Thu, Oct 15, 2015 at 11:50 AM, Trussardi Dario Romano <[hidden email]> wrote:
Ciao Mariano,

Dario,

With the years I have learned that I will  not spend a single second of my life anymore trying to optimize something without having previously profile it. Most of the times, the bottleneck is elsewhere where you imagined. So if I were you, before doing anything, I would profile the report and understand where is the bottleneck. And then analyze possible solutions: algorithmic, database index, kind of collection used, collection message used, gemstone tunning (do you have native code enabled? ), etc.

Let me know if you need help with the profiling.

I don't have any knowledge about profiling.

Have you some considerations, reference about it ?

Thanks for any suggestion,

Dario


Cheers, 


On Thu, Oct 15, 2015 at 11:28 AM, Trussardi Dario Romano via Glass <[hidden email]> wrote:
Ciao Mariano,

thanks for your considerations.

Hi Dario,

It looks strange to me that a query to GemStone could take that much. Did you profile it?  how many rows has the collection? which type of collection is it?   the complexity / time goes into the "condition" rather in the select?
I ask because many times I was in the same situation, after a quick profiling,
I don't have experience into index.

I noticed that adding an index would solve it.

I defined my request as 'query'   but, i think, it is not a query on possible indexed data.

My query is relative to the some dictionaries ( one dictionary for any day ) everyone with a dictionary about the documents ( 200 - 300 )  managed on the day.

Every document is based on a bag of items ecc.  ecc.. ecc...

With my surprise when i have 2 - 3 moths on-line all works fine.  ( the SPC is set to 2097152 )

But now when i have 6 months on-line the system answer with some delay.

My goal for now is to have online 12 months.

I need to change  my internal structure ?

Cheers,
Dario



Cheers,
 

---------- Forwarded message ----------
From: [hidden email] <[hidden email]>
Date: Thu, Oct 15, 2015 at 9:23 AM
Subject: [Seaside] Progress bar for database query
To: seaside-wiki <[hidden email]>


Ciao,

        i have a seaside application with some user interface.

        Now i have one specific user request for query the database.

        This request required some long time on the server before answer.

        My goal is to :

                a) display a progress bar when the user submit this query

                b)  block the web user request until the server answer the data query result


        How i can implemented this solution?

        Thank for any considerations.


                Dario

        _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside



--
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--




--

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Seaside] Progress bar for database query

GLASS mailing list
Dario,

If you have tODE, you can use the `pf` command:

  pf start                                         #start profiling
  eval`Here goes your execution`
  pf view                                          #stop profiling and bring up profile viewer

Does pretty much what Mariano described, the profile viewer allows you interactively poke around in the profiling results while viewing the method source ...

Dale

On 10/15/2015 07:56 AM, Mariano Martinez Peck via Glass wrote:
| profMon |
profMon := ProfMonitorTree new.
profMon interval: 10.
 profMon startMonitoring.

self HERE_GOES_YOUR_CODE_EXECUTION.

 profMon
        stopMonitoring;
        gatherResults.

(FileStream forceNewFileNamed: '/Users/mariano/profile.txt') nextPutAll: (profMon reportDownTo: 1); cr.


Ok, replace "self HERE_GOES_YOUR_CODE_EXECUTION." with your real code you want to profile.
Change internal if you want samples to be more or less frequent (I would let 10).
Change reportDownTo: to see until how much you want to print in the file. If 1 is too much large file, you may want to try 10 or 100.
Change profile.txt location

Then simply inspect profile.txt, go to the very end and analyze where the time is spent.

Cheers,




On Thu, Oct 15, 2015 at 11:50 AM, Trussardi Dario Romano <[hidden email]> wrote:
Ciao Mariano,

Dario,

With the years I have learned that I will  not spend a single second of my life anymore trying to optimize something without having previously profile it. Most of the times, the bottleneck is elsewhere where you imagined. So if I were you, before doing anything, I would profile the report and understand where is the bottleneck. And then analyze possible solutions: algorithmic, database index, kind of collection used, collection message used, gemstone tunning (do you have native code enabled? ), etc.

Let me know if you need help with the profiling.

I don't have any knowledge about profiling.

Have you some considerations, reference about it ?

Thanks for any suggestion,

Dario


Cheers, 


On Thu, Oct 15, 2015 at 11:28 AM, Trussardi Dario Romano via Glass <[hidden email]> wrote:
Ciao Mariano,

thanks for your considerations.

Hi Dario,

It looks strange to me that a query to GemStone could take that much. Did you profile it?  how many rows has the collection? which type of collection is it?   the complexity / time goes into the "condition" rather in the select?
I ask because many times I was in the same situation, after a quick profiling,
I don't have experience into index.

I noticed that adding an index would solve it.

I defined my request as 'query'   but, i think, it is not a query on possible indexed data.

My query is relative to the some dictionaries ( one dictionary for any day ) everyone with a dictionary about the documents ( 200 - 300 )  managed on the day.

Every document is based on a bag of items ecc.  ecc.. ecc...

With my surprise when i have 2 - 3 moths on-line all works fine.  ( the SPC is set to 2097152 )

But now when i have 6 months on-line the system answer with some delay.

My goal for now is to have online 12 months.

I need to change  my internal structure ?

Cheers,
Dario



Cheers,
 

---------- Forwarded message ----------
From: [hidden email] <[hidden email]>
Date: Thu, Oct 15, 2015 at 9:23 AM
Subject: [Seaside] Progress bar for database query
To: seaside-wiki <[hidden email]>


Ciao,

        i have a seaside application with some user interface.

        Now i have one specific user request for query the database.

        This request required some long time on the server before answer.

        My goal is to :

                a) display a progress bar when the user submit this query

                b)  block the web user request until the server answer the data query result


        How i can implemented this solution?

        Thank for any considerations.


                Dario

        _______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside



--
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass




--




--


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass
Reply | Threaded
Open this post in threaded view
|

Re: [Seaside] Progress bar for database query

Richard Sargent
Administrator
In reply to this post by GLASS mailing list
GLASS mailing list wrote
Ciao Mariano,

        thanks for your considerations.

> Hi Dario,
>
> It looks strange to me that a query to GemStone could take that much. Did you profile it?  how many rows has the collection? which type of collection is it?   the complexity / time goes into the "condition" rather in the select?
> I ask because many times I was in the same situation, after a quick profiling,
       
        I don't have experience into index.

> I noticed that adding an index would solve it.

        I defined my request as 'query'   but, i think, it is not a query on possible indexed data.

        My query is relative to the some dictionaries ( one dictionary for any day ) everyone with a dictionary about the documents ( 200 - 300 )  managed on the day.

        Every document is based on a bag of items ecc.  ecc.. ecc...
Dario, it sounds like you have been using dictionaries to provide "access paths" to your objects. Indexes are the high-performance equivalent to this. In GemStone, you would want to have a single collection holding "all" instances of the same class, and possibly of related classes, too. (I say "all" in quotes, because it isn't an absolute.)

The indexes then allow you to readily locate those objects which have the necessary characteristics. e.g. created on a specific date, created on any date in a date range, etc. Authored by a specific user, about a specific customer, and so on. Indexes allow O(1) access to subsets, rather than O(n).


I am guessing you have used the dictionaries for a Pharo implementation, since it doesn't have indexes, and have strictly ported your application to GemStone/S. If you need to run in both environments, you have limited choices. Still, you could also add the documents to an "everything" collection which has indexes in GemStone and so use indexed queries in GemStone and use regular dictionary traversals in Pharo.



        With my surprise when i have 2 - 3 moths on-line all works fine.  ( the SPC is set to 2097152 )

        But now when i have 6 months on-line the system answer with some delay.

        My goal for now is to have online 12 months.

        I need to change  my internal structure ?

        Cheers,
                       
                Dario

       

>
> Cheers,
>  
>
> ---------- Forwarded message ----------
> From: [hidden email] <[hidden email]>
> Date: Thu, Oct 15, 2015 at 9:23 AM
> Subject: [Seaside] Progress bar for database query
> To: seaside-wiki <[hidden email]>
>
>
> Ciao,
>
>         i have a seaside application with some user interface.
>
>         Now i have one specific user request for query the database.
>
>         This request required some long time on the server before answer.
>
>         My goal is to :
>
>                 a) display a progress bar when the user submit this query
>
>                 b)  block the web user request until the server answer the data query result
>
>
>         How i can implemented this solution?
>
>         Thank for any considerations.
>
>
>                 Dario
>
>         _______________________________________________
> seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
>
> --
> Mariano
> http://marianopeck.wordpress.com
> _______________________________________________
> Glass mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/glass


_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass