create snapshots out of github based packages

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

create snapshots out of github based packages

Nicolai Hess-3-2
Is there any way to create a "snapshot" of a project from a git based repository.
Something that creates a "condensed" old style mcz package ?
Maybe there is already something and I missed it ?

Because now I am just a little shocked how much files are created during loading
a project.

I did some tests with gtoolkit bloc and calypso.
Sure, bloc or gtoollkit may be huge projects, but they produce thousands of files.
And I don't mean 1 or 2 thousand files.
Loading bloc for example creates over 20.000 files.

This is damn slow on windows.
Reply | Threaded
Open this post in threaded view
|

Re: create snapshots out of github based packages

CyrilFerlicot

On dim. 10 déc. 2017 at 15:07, Nicolai Hess <[hidden email]> wrote:
Is there any way to create a "snapshot" of a project from a git based repository.
Something that creates a "condensed" old style mcz package ?
Maybe there is already something and I missed it ?

Because now I am just a little shocked how much files are created during loading
a project.

I did some tests with gtoolkit bloc and calypso.
Sure, bloc or gtoollkit may be huge projects, but they produce thousands of files.
And I don't mean 1 or 2 thousand files.
Loading bloc for example creates over 20.000 files.

This is damn slow on windows.
Hi,

This is one of the reasons Esteban created Tonel I think. With Tonel there is 1 file par class instead of 1 file per method. Once it will be really stable, moving projects to Tonel should speed up things on Windows. 
--
Cyril Ferlicot
https://ferlicot.fr

http://www.synectique.eu
2 rue Jacques Prévert 01,
59650 Villeneuve d'ascq France
Reply | Threaded
Open this post in threaded view
|

Re: create snapshots out of github based packages

Stephan Eggermont-3
Op 10-12-2017 om 15:44 schreef Cyril Ferlicot:
> On dim. 10 déc. 2017 at 15:07, Nicolai Hess <[hidden email] wrote:
>>     And I don't mean 1 or 2 thousand files.
>>     Loading bloc for example creates over 20.000 files.
>>
>>     This is damn slow on windows.
>
> This is one of the reasons Esteban created Tonel I think. With Tonel
> there is 1 file par class instead of 1 file per method. Once it will be
> really stable, moving projects to Tonel should speed up things on Windows.

Indeed. An even larger problem is the write amplification: Minimium disk
file size of 4KB means a slow down of a factor 20. 1 (primary) file per
class and sorting the methods helps a lot. Doing that in a
cross-platform compatible way so we can easier share source code needs
some further refinement

Stephan


Reply | Threaded
Open this post in threaded view
|

Re: create snapshots out of github based packages

EstebanLM


> On 10 Dec 2017, at 21:37, Stephan Eggermont <[hidden email]> wrote:
>
> Op 10-12-2017 om 15:44 schreef Cyril Ferlicot:
>> On dim. 10 déc. 2017 at 15:07, Nicolai Hess <[hidden email] wrote:
>>>    And I don't mean 1 or 2 thousand files.
>>>    Loading bloc for example creates over 20.000 files.
>>>    This is damn slow on windows.
>> This is one of the reasons Esteban created Tonel I think. With Tonel there is 1 file par class instead of 1 file per method. Once it will be really stable, moving projects to Tonel should speed up things on Windows.
>
> Indeed. An even larger problem is the write amplification: Minimium disk file size of 4KB means a slow down of a factor 20. 1 (primary) file per class and sorting the methods helps a lot. Doing that in a cross-platform compatible way so we can easier share source code needs some further refinement

and btw since we moved pharo to tonel windows is a viable platform: commits became seconds instead minutes (lots of minutes).
still work to do there, but we are getting closer… ;)

Esteban

>
> Stephan
>
>


Reply | Threaded
Open this post in threaded view
|

Re: create snapshots out of github based packages

kilon.alios
nice to hear that my very old complaint about filetrees habit of breaking down everything to small files has caught and there is even a project I am not aware of that deal with this. Though my motivation was mostly readability and not speed of loading. Great work! :)

On Mon, Dec 11, 2017 at 11:35 AM Esteban Lorenzano <[hidden email]> wrote:


> On 10 Dec 2017, at 21:37, Stephan Eggermont <[hidden email]> wrote:
>
> Op 10-12-2017 om 15:44 schreef Cyril Ferlicot:
>> On dim. 10 déc. 2017 at 15:07, Nicolai Hess <[hidden email] wrote:
>>>    And I don't mean 1 or 2 thousand files.
>>>    Loading bloc for example creates over 20.000 files.
>>>    This is damn slow on windows.
>> This is one of the reasons Esteban created Tonel I think. With Tonel there is 1 file par class instead of 1 file per method. Once it will be really stable, moving projects to Tonel should speed up things on Windows.
>
> Indeed. An even larger problem is the write amplification: Minimium disk file size of 4KB means a slow down of a factor 20. 1 (primary) file per class and sorting the methods helps a lot. Doing that in a cross-platform compatible way so we can easier share source code needs some further refinement

and btw since we moved pharo to tonel windows is a viable platform: commits became seconds instead minutes (lots of minutes).
still work to do there, but we are getting closer… ;)

Esteban

>
> Stephan
>
>