code loading performance

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

code loading performance

George Ganea
Hi all,

Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?

Cheers,
George
Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

Pharo Smalltalk Developers mailing list
Hi Georges,

There have been quite some improvements those last weeks on the performance of loading classes and methods. But we are still waiting for the https://github.com/pharo-project/pharo/pull/5292 to be integrated.

And you have to encapsulate the loading code into: SourceFiles deferFlushDuring: [...] and use the latest pharo 8.0 image.

You can give a try with this and tell us how it goes!

Cheers,
Vincent

-----Original Message-----
From: Pharo-dev On Behalf Of George Ganea
Sent: Friday, 6 December 2019 17:08
To: [hidden email]
Cc: Chis Vasile Andrei <[hidden email]>
Subject: [Pharo-dev] code loading performance

Hi all,

Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?

Cheers,
George


Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

Sven Van Caekenberghe-2
In reply to this post by George Ganea
Yes, some improvements are needed.

There are many steps:

- downloading the code (networking)
- loading the code (file io)
- compiling the code
- writing to the changes file
- writing to the epicea log
- system notifications
- reactions to those system notifications

BTW, doing a cleanup for production is also too slow, so I am guessing most time is lost in the last 4 items.

> On 6 Dec 2019, at 17:07, George Ganea <[hidden email]> wrote:
>
> Hi all,
>
> Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?
>
> Cheers,
> George


Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

George Ganea
In reply to this post by George Ganea
Hi Vincent,

I will definitely try it out and report back. 

Thank you!
George
Hi Georges,

There have been quite some improvements those last weeks on the performance of loading classes and methods. But we are still waiting for the https://github.com/pharo-project/pharo/pull/5292 to be integrated.

And you have to encapsulate the loading code into: SourceFiles deferFlushDuring: [...] and use the latest pharo 8.0 image.

You can give a try with this and tell us how it goes!

Cheers,
Vincent

-----Original Message-----
From: Pharo-dev On Behalf Of George Ganea
Sent: Friday, 6 December 2019 17:08
To: pharo-dev at lists.pharo.org
Cc: Chis Vasile Andrei <dedal18 at yahoo.com>
Subject: [Pharo-dev] code loading performance

Hi all,

Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?

Cheers,
George

Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

George Ganea
In reply to this post by Sven Van Caekenberghe-2
Thank you Sven,

we’re already skipping writing to the epicea log, I’m now wondering can we also skip the notifications or not really?

Cheers,
George

> On 6 Dec 2019, at 19:01, Sven Van Caekenberghe <[hidden email]> wrote:
>
> Yes, some improvements are needed.
>
> There are many steps:
>
> - downloading the code (networking)
> - loading the code (file io)
> - compiling the code
> - writing to the changes file
> - writing to the epicea log
> - system notifications
> - reactions to those system notifications
>
> BTW, doing a cleanup for production is also too slow, so I am guessing most time is lost in the last 4 items.
>
>> On 6 Dec 2019, at 17:07, George Ganea <[hidden email]> wrote:
>>
>> Hi all,
>>
>> Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?
>>
>> Cheers,
>> George
>
>


Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

SergeStinckwich
In reply to this post by George Ganea


On Fri, Dec 6, 2019 at 5:08 PM George Ganea <[hidden email]> wrote:
Hi all,

Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?


Could be even worse when you dl from low-bandwith countries like in Africa.
While waiting a better solution, it possible to have build images available from PharoLauncher ?
Regards,

--
Serge Stinckwic
h

Int. Research Unit
 on Modelling/Simulation of Complex Systems (UMMISCO)
Sorbonne University
 (SU)
French National Research Institute for Sustainable Development (IRD)
U
niversity of Yaoundé I, Cameroon
"Programs must be written for people to read, and only incidentally for machines to execute."
https://twitter.com/SergeStinckwich
Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

George Ganea
In reply to this post by George Ganea
Hi Vincent,

In the end we tried it after it was merged :) Our CI uses the latest Pharo 8 image. 
Unfortunately we did not see an improvement, and even worse, it seems to break one of our builds that is downstream from GToolkit. We got all sorts of errors that seemed to be related to incomplete method code.
Two examples below.

Syntax Error on line 2: ''']'' expected'
========================================
1: privateState
2: ^ privateState ifNil: [ p

groupedByIntervalType: aTemporalIntervalType fromTimestamp: startTimestamp toTimestamp: endTimestamp
^ self
groupedByIntervalType: aTemporalIntervalType
inGroupOfType: self temporalGroupType
from: (aTemporalIntervalT')' expected ->

We’ve now switched our CI to a fixed version image that is prior to this merge.
Even though the code we’re loading is in a private repo, please let me know if there is anything I can do to help debug this.

Thank you,
George

On 6 Dec 2019, at 19:05, George Ganea <[hidden email]> wrote:

Hi Vincent,

I will definitely try it out and report back. 

Thank you!
George
Hi Georges,

There have been quite some improvements those last weeks on the performance of loading classes and methods. But we are still waiting for the https://github.com/pharo-project/pharo/pull/5292 to be integrated.

And you have to encapsulate the loading code into: SourceFiles deferFlushDuring: [...] and use the latest pharo 8.0 image.

You can give a try with this and tell us how it goes!

Cheers,
Vincent

-----Original Message-----
From: Pharo-dev On Behalf Of George Ganea
Sent: Friday, 6 December 2019 17:08
To: pharo-dev at lists.pharo.org
Cc: Chis Vasile Andrei <dedal18 at yahoo.com>
Subject: [Pharo-dev] code loading performance

Hi all,

Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?

Cheers,
George


Reply | Threaded
Open this post in threaded view
|

Re: code loading performance

Marcus Denker-4
Maybe the best is to revert for now 


On 11 Dec 2019, at 14:39, George Ganea <[hidden email]> wrote:

Hi Vincent,

In the end we tried it after it was merged :) Our CI uses the latest Pharo 8 image. 
Unfortunately we did not see an improvement, and even worse, it seems to break one of our builds that is downstream from GToolkit. We got all sorts of errors that seemed to be related to incomplete method code.
Two examples below.

Syntax Error on line 2: ''']'' expected'
========================================
1: privateState
2: ^ privateState ifNil: [ p

groupedByIntervalType: aTemporalIntervalType fromTimestamp: startTimestamp toTimestamp: endTimestamp
^ self
groupedByIntervalType: aTemporalIntervalType
inGroupOfType: self temporalGroupType
from: (aTemporalIntervalT')' expected ->

We’ve now switched our CI to a fixed version image that is prior to this merge.
Even though the code we’re loading is in a private repo, please let me know if there is anything I can do to help debug this.

Thank you,
George

On 6 Dec 2019, at 19:05, George Ganea <[hidden email]> wrote:

Hi Vincent,

I will definitely try it out and report back. 

Thank you!
George
Hi Georges,

There have been quite some improvements those last weeks on the performance of loading classes and methods. But we are still waiting for the https://github.com/pharo-project/pharo/pull/5292 to be integrated.

And you have to encapsulate the loading code into: SourceFiles deferFlushDuring: [...] and use the latest pharo 8.0 image.

You can give a try with this and tell us how it goes!

Cheers,
Vincent

-----Original Message-----
From: Pharo-dev On Behalf Of George Ganea
Sent: Friday, 6 December 2019 17:08
To: pharo-dev at lists.pharo.org
Cc: Chis Vasile Andrei <dedal18 at yahoo.com>
Subject: [Pharo-dev] code loading performance

Hi all,

Currently loading GToolkit takes quite some time (arount 18 minutes at the best of times) we were wondering if there’s been any attempts at improving code loading times in Metacello/Monticello. Or mabye there are some ideas on how one might start doing something like this?

Cheers,
George