.changes are now 37MB….

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

.changes are now 37MB….

Marcus Denker-4
... what do we do?

http://code.google.com/p/pharo/issues/detail?id=4843
(but that will not help that much. And someone would need to look at the tracker entry and make
it ready for integration).

or just do a condense? The history is in Monticello.

http://code.google.com/p/pharo/issues/detail?id=5379


        Marcus


--
Marcus Denker -- http://marcusdenker.de


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Stéphane Ducasse
May be it would be good to have a look at eliot changes but we need time for that.


> ... what do we do?
>
> http://code.google.com/p/pharo/issues/detail?id=4843
> (but that will not help that much. And someone would need to look at the tracker entry and make
> it ready for integration).
>
> or just do a condense? The history is in Monticello.
>
> http://code.google.com/p/pharo/issues/detail?id=5379
>
>
> Marcus
>
>
> --
> Marcus Denker -- http://marcusdenker.de
>
>


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Philippe Marschall-2-3
In reply to this post by Marcus Denker-4
On 06.03.2012 09:44, Marcus Denker wrote:
> ... what do we do?
>
> http://code.google.com/p/pharo/issues/detail?id=4843
> (but that will not help that much. And someone would need to look at the tracker entry and make
> it ready for integration).
>
> or just do a condense? The history is in Monticello.

Do people actually really rely on the .changes or just say they do? I'd
be fine with just a condense.

Cheers
Philippe


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Max Leske
This may be a stupid question… (As far as I understand, condensing the changes is not the same)
Why don't you increment the version number of the sources file to 1.1 and use a fresh changes file?

Max


On 06.03.2012, at 18:42, Philippe Marschall wrote:

> On 06.03.2012 09:44, Marcus Denker wrote:
>> ... what do we do?
>>
>> http://code.google.com/p/pharo/issues/detail?id=4843
>> (but that will not help that much. And someone would need to look at the tracker entry and make
>> it ready for integration).
>>
>> or just do a condense? The history is in Monticello.
>
> Do people actually really rely on the .changes or just say they do? I'd be fine with just a condense.
>
> Cheers
> Philippe
>
>


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Marcus Denker-4
In reply to this post by Philippe Marschall-2-3

On Mar 6, 2012, at 7:30 PM, Max Leske wrote:

> This may be a stupid question… (As far as I understand, condensing the changes is not the same)
> Why don't you increment the version number of the sources file to 1.1 and use a fresh changes file?
>
The sources file does not contain the sources and the changes just the changes. (even though one could think so taking
the names into account.)

The sources file containes the sources of ages ago (1.0), the .changes is "on top of that". This means that the .sources
file contains a lot of code from classes and methods that have been removed (like the changes),
as well as old versions.  

The .sources/.changes mechanism merges three responsabilties:

1) be a log of all edited code in case of a crash
2) store the current source of the methods (when you ask a method for code, it reads either from the .sources
(method not changed since 1.0) or from the .changes (method was changed).
3) provide a history for all code

And all that in 2 big files that one can only append to because? No idea, actually...

I personally think that it is a mistake today to merge all these into one such basic mechanism. Especialy if
you look at how complex the code is in the image... it's actually amazing.

When you do a #condenseSources (which is sadly broken right now), then it generates a .sources
file with just the code in the current image. It would be interesting to see how large that is.

        Marcus

--
Marcus Denker -- http://marcusdenker.de


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Camillo Bruni-3
max,

maybe you want to create a test git repos with all the sources back to 1.0 :D
and then we should record all changes into a local git-repos :)
if you want to got back some versions you simply start fetching the missing objects from a central repos somewhere...

so far dreaming a bit

best
cami

On 2012-03-06, at 20:01, Marcus Denker wrote:

>
> On Mar 6, 2012, at 7:30 PM, Max Leske wrote:
>
>> This may be a stupid question… (As far as I understand, condensing the changes is not the same)
>> Why don't you increment the version number of the sources file to 1.1 and use a fresh changes file?
>>
> The sources file does not contain the sources and the changes just the changes. (even though one could think so taking
> the names into account.)
>
> The sources file containes the sources of ages ago (1.0), the .changes is "on top of that". This means that the .sources
> file contains a lot of code from classes and methods that have been removed (like the changes),
> as well as old versions.  
>
> The .sources/.changes mechanism merges three responsabilties:
>
> 1) be a log of all edited code in case of a crash
> 2) store the current source of the methods (when you ask a method for code, it reads either from the .sources
> (method not changed since 1.0) or from the .changes (method was changed).
> 3) provide a history for all code
>
> And all that in 2 big files that one can only append to because? No idea, actually...
>
> I personally think that it is a mistake today to merge all these into one such basic mechanism. Especialy if
> you look at how complex the code is in the image... it's actually amazing.
>
> When you do a #condenseSources (which is sadly broken right now), then it generates a .sources
> file with just the code in the current image. It would be interesting to see how large that is.
>
> Marcus
>
> --
> Marcus Denker -- http://marcusdenker.de
>
>


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Eliot Miranda-2
In reply to this post by Marcus Denker-4


On Tue, Mar 6, 2012 at 11:01 AM, Marcus Denker <[hidden email]> wrote:

On Mar 6, 2012, at 7:30 PM, Max Leske wrote:

> This may be a stupid question… (As far as I understand, condensing the changes is not the same)
> Why don't you increment the version number of the sources file to 1.1 and use a fresh changes file?
>
The sources file does not contain the sources and the changes just the changes. (even though one could think so taking
the names into account.)

The sources file containes the sources of ages ago (1.0), the .changes is "on top of that". This means that the .sources
file contains a lot of code from classes and methods that have been removed (like the changes),
as well as old versions.

The .sources/.changes mechanism merges three responsabilties:

1) be a log of all edited code in case of a crash
2) store the current source of the methods (when you ask a method for code, it reads either from the .sources
(method not changed since 1.0) or from the .changes (method was changed).
3) provide a history for all code

And all that in 2 big files that one can only append to because? No idea, actually...

I personally think that it is a mistake today to merge all these into one such basic mechanism. Especialy if
you look at how complex the code is in the image... it's actually amazing.

When you do a #condenseSources (which is sadly broken right now), then it generates a .sources
file with just the code in the current image. It would be interesting to see how large that is.

IMO it's also interesting to see how big a condense of only the valid history is, i.e. removing history for missing classes and methods, and only retaining history that is referenced from current methods (which eliminates branches that don't lead to the current definition).  After all this is the history one wants, and I already provided code to compute the per-method history that leads to the current definition.  Why not have your cake and compress it too?
 

       Marcus

--
Marcus Denker -- http://marcusdenker.de





--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Max Leske
In reply to this post by Marcus Denker-4
That's what I meant… So that won't work…

Thanks for the explanation Marcus.


On 06.03.2012, at 20:01, Marcus Denker wrote:

When you do a #condenseSources (which is sadly broken right now), then it generates a .sources
file with just the code in the current image. It would be interesting to see how large that is.

Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Max Leske
In reply to this post by Camillo Bruni-3
If it wouldn't take so bloody long to run (approx. 9 hours for all Squeak version up to and including Pharo 1.0) and didn't require me to port back all the changes we made to FSGit to Pharogenesis I'd happily  do that. I just don't think the effort is worth it (I'll change my mind if people on the list start begging for it :) )

Ah yes, and the fetching… That would of course be missing too… :p

Cheers,
Max

On 06.03.2012, at 20:05, Camillo Bruni wrote:

> max,
>
> maybe you want to create a test git repos with all the sources back to 1.0 :D
> and then we should record all changes into a local git-repos :)
> if you want to got back some versions you simply start fetching the missing objects from a central repos somewhere...
>
> so far dreaming a bit
>
> best
> cami
>
> On 2012-03-06, at 20:01, Marcus Denker wrote:
>
>>
>> On Mar 6, 2012, at 7:30 PM, Max Leske wrote:
>>
>>> This may be a stupid question… (As far as I understand, condensing the changes is not the same)
>>> Why don't you increment the version number of the sources file to 1.1 and use a fresh changes file?
>>>
>> The sources file does not contain the sources and the changes just the changes. (even though one could think so taking
>> the names into account.)
>>
>> The sources file containes the sources of ages ago (1.0), the .changes is "on top of that". This means that the .sources
>> file contains a lot of code from classes and methods that have been removed (like the changes),
>> as well as old versions.  
>>
>> The .sources/.changes mechanism merges three responsabilties:
>>
>> 1) be a log of all edited code in case of a crash
>> 2) store the current source of the methods (when you ask a method for code, it reads either from the .sources
>> (method not changed since 1.0) or from the .changes (method was changed).
>> 3) provide a history for all code
>>
>> And all that in 2 big files that one can only append to because? No idea, actually...
>>
>> I personally think that it is a mistake today to merge all these into one such basic mechanism. Especialy if
>> you look at how complex the code is in the image... it's actually amazing.
>>
>> When you do a #condenseSources (which is sadly broken right now), then it generates a .sources
>> file with just the code in the current image. It would be interesting to see how large that is.
>>
>> Marcus
>>
>> --
>> Marcus Denker -- http://marcusdenker.de
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

ccrraaiigg
In reply to this post by Marcus Denker-4

> And all that in 2 big files that one can only append to because? No
> idea, actually...
>
> I personally think that it is a mistake today to merge all these into
> one such basic mechanism. Especialy if
> you look at how complex the code is in the image... it's actually
> amazing.

     Hear hear. I think it's insane to do it (only) this way at this
point. Objects everywhere...


-C

--
Craig Latta
www.netjam.org/resume
+31   6 2757 7177
+ 1 415  287 3547



Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Igor Stasenko
On 6 March 2012 23:30, Craig Latta <[hidden email]> wrote:

>
>> And all that in 2 big files that one can only append to because? No
>> idea, actually...
>>
>> I personally think that it is a mistake today to merge all these into
>> one such basic mechanism. Especialy if
>> you look at how complex the code is in the image... it's actually
>> amazing.
>
>     Hear hear. I think it's insane to do it (only) this way at this
> point. Objects everywhere...
>
can you elaborate? do what at which point?

>
> -C
>
> --
> Craig Latta
> www.netjam.org/resume
> +31 6 2757 7177
> + 1 415 287 3547
>
>
>



--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

ccrraaiigg

> > > And all that in 2 big files that one can only append to because?
> > > No idea, actually...
> > >
> > > I personally think that it is a mistake today to merge all these
> > > into one such basic mechanism. Especialy if
> > > you look at how complex the code is in the image... it's actually
> > > amazing.
> >
> >     Hear hear. I think it's insane to do it (only) this way at this
> > point. Objects everywhere...
> >
> can you elaborate? do what at which point?

     To use source code written on files to record history information,
at this point in history where we have more than enough processor,
network, and disk to just use live distributed object memories. I can
see keeping the old stuff around for a while as a redundancy mechanism,
but we needn't rely on it alone.


-C

--
Craig Latta
www.netjam.org/resume
+31   6 2757 7177
+ 1 415  287 3547



Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Eliot Miranda-2


On Wed, Mar 7, 2012 at 1:56 AM, Craig Latta <[hidden email]> wrote:

> > > And all that in 2 big files that one can only append to because?
> > > No idea, actually...
> > >
> > > I personally think that it is a mistake today to merge all these
> > > into one such basic mechanism. Especialy if
> > > you look at how complex the code is in the image... it's actually
> > > amazing.
> >
> >     Hear hear. I think it's insane to do it (only) this way at this
> > point. Objects everywhere...
> >
> can you elaborate? do what at which point?

    To use source code written on files to record history information,
at this point in history where we have more than enough processor,
network, and disk to just use live distributed object memories. I can
see keeping the old stuff around for a while as a redundancy mechanism,
but we needn't rely on it alone.

But both Occam's razor and IIABDFI (if-it-aint-broke-dont-fix-it) say we should keep with it.  It has worked extremely well with Squeak/Pharo, is robust, persistent, and simple.  Realistically we're not going to move to Spoon over night, so what to do about a 37Mb changes file is a pressing issue.  Being able to compress with history and/or write a new sources file with compressed history makes sense.  It's not being unimaginitive or regressive, or anti-innovation; merely pragmatic.



-C

--
Craig Latta
www.netjam.org/resume
<a href="tel:%2B31%20%20%206%202757%207177" value="+31627577177">+31 6 2757 7177
<a href="tel:%2B%201%20415%20%20287%203547" value="+14152873547">+ 1 415 287 3547






--
best,
Eliot

Reply | Threaded
Open this post in threaded view
|

Re: .changes are now 37MB….

Igor Stasenko
On 7 March 2012 19:31, Eliot Miranda <[hidden email]> wrote:

>
>
> On Wed, Mar 7, 2012 at 1:56 AM, Craig Latta <[hidden email]> wrote:
>>
>>
>> > > > And all that in 2 big files that one can only append to because?
>> > > > No idea, actually...
>> > > >
>> > > > I personally think that it is a mistake today to merge all these
>> > > > into one such basic mechanism. Especialy if
>> > > > you look at how complex the code is in the image... it's actually
>> > > > amazing.
>> > >
>> > >     Hear hear. I think it's insane to do it (only) this way at this
>> > > point. Objects everywhere...
>> > >
>> > can you elaborate? do what at which point?
>>
>>     To use source code written on files to record history information,
>> at this point in history where we have more than enough processor,
>> network, and disk to just use live distributed object memories. I can
>> see keeping the old stuff around for a while as a redundancy mechanism,
>> but we needn't rely on it alone.
>
>
> But both Occam's razor and IIABDFI (if-it-aint-broke-dont-fix-it) say we
> should keep with it.  It has worked extremely well with Squeak/Pharo, is
> robust, persistent, and simple.  Realistically we're not going to move to
> Spoon over night, so what to do about a 37Mb changes file is a pressing
> issue.  Being able to compress with history and/or write a new sources file
> with compressed history makes sense.  It's not being unimaginitive or
> regressive, or anti-innovation; merely pragmatic.
>

Well, it IS broken. I cannot say something else about direct uses of
'SourceFiles at: 1'/'SourceFiles at: 2' at random places in image.
But we didn't yet found time to fix that, not saying about binding to
something else like spoon,
so yes, a pragmatic way would be to just do the same thing as we did
before: condense/compres changes and source.



--
Best regards,
Igor Stasenko.