On Tue, Jan 17, 2012 at 6:31 PM, Mariano Martinez Peck <[hidden email]> wrote:
Well, with the following script it seems I am able to embed all source code in trailers: | trailer source key dict classes | dict := IdentityDictionary new. CompiledMethod allInstances do: [:aMethod | trailer := aMethod trailer. source := aMethod getSource. trailer sourceCode: source. dict at: aMethod put: (aMethod copyWithTrailerBytes: trailer). ]. dict keys asArray elementsExchangeIdentityWith: dict values asArray. Problem is that after executing the script, when a method is recompiled from the system browser, it puts back a trailer with a .changes pointer :( I will see if I find where I can change that. Cheers -- Mariano http://marianopeck.wordpress.com |
In another thread, Stefan Marr asked "Is there any general use for such an embedding, beside making the finder usable?" I would just give my answer: 1) From a design point of view, I think that CompiledMethodTrailer is the correct place where source code should be stored. 2) I can understand to put source code in files would make sense 20 years ago. But I don't think so anymore... a normal notebook has 8gb of RAM these days. 3) I think (I should measure it) that having source code in the image is faster, even thought that's the least I care about. 4) Ease the logic of pointers in the trailer... 5) Easy of versioning for a binary serializer ;) so for example in Fuel we can now be able to serialize compiled methods with trailer that have the source code. Hence, versioning it (Monticello) is easier. So...what I would do (that's just my opinion) is: 1) always store source code in trailer 2) do not use .sources anymore 3) Use .changes ONLY for recovery but not for getting source code (#getSource) 4) Provide a way to shrink an image for production cases where people is interested in image size. Can can very easily replace compiled methods trailers with empty trailers (zap) and then if it just happens that you need the sources..you decompile it. Maybe there are obvious drawbacks I am not seeing... Cheers Mariano http://marianopeck.wordpress.com |
On 17 Jan 2012, at 22:40, Mariano Martinez Peck wrote: > 1) From a design point of view, I think that CompiledMethodTrailer is the correct place where source code should be stored. Why? From my point of view, the source fits into the image, but does not belong into the CompiledMethod. CompiledMethod is the representation for execution purpose. Encoding it into the bytearray along the the bytecode just asks for trouble. CompiledMethod should have a counter part, which can also carry all kind of other meta data. I bet there is something in the ring framework to represent methods that is not based on CompiledMethod. That is where I would put the code. > 3) I think (I should measure it) that having source code in the image is faster, even thought that's the least I care about. The finder is obviously faster... Other tools wont benefit as much, because the cost of reading a single method is insignificant. Refactorings which might need to read code, or recompile everything might benefit. > 4) Ease the logic of pointers in the trailer... > 5) Easy of versioning for a binary serializer ;) so for example in Fuel we can now be able to serialize compiled methods with trailer that have the source code. Hence, versioning it (Monticello) is easier. I still do not see why you would want the extra complexity (with encoding and all) to put it into the ByteArray of compiled method. That makes everything harder, and one argument you might want to use, no, it does not guarantee consistency between the two parts. Putting the code into the image, yes, but not into the compiled method. At least I do not see any clear benefits. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 |
In reply to this post by Mariano Martinez Peck
On Tue, Jan 17, 2012 at 1:40 PM, Mariano Martinez Peck <[hidden email]> wrote:
I disagree. The changes file is synergistic: - provides crash recovery - removes significant space overhead from the image - changes file plus sources file provide method source history
it is not just a space trade-off. see above.
certainly. but doing a better job with readOnly copies of the changes file speeds up things a lot. there is no need to keep more than one read-only copy of the changes file. I have old Newspeak code for 3.9 if you're interested.
that's a fair reason. but with parcels having two files, objects (code) and source, worked well, e.g. deploying without source means not including the sources file.
best, Eliot |
In reply to this post by Mariano Martinez Peck
On Tue, Jan 17, 2012 at 1:40 PM, Mariano Martinez Peck
<[hidden email]> wrote: > > 2) I can understand to put source code in files would make sense 20 years > ago. But I don't think so anymore... a normal notebook has 8gb of RAM these > days. I believe the Windows VM still limits us to around 1/2 gig of image size, at least when starting up the image. If so, it would be nice to not increase the space too much by default (that is, windows users don't really have 8 gig...) |
In reply to this post by Mariano Martinez Peck
On 17 January 2012 19:22, Mariano Martinez Peck <[hidden email]> wrote:
> > > On Tue, Jan 17, 2012 at 6:31 PM, Mariano Martinez Peck > <[hidden email]> wrote: >> >> >>> If you don't bother about being slow, you can also make it much easier: >>> >>> >>> |trailer source m | >>> m := (Gofer>>#load). >>> trailer := m trailer. >>> source := m getSource. >>> trailer sourceCode: source. >>> m becomeForward: (m copyWithTrailerBytes: trailer) >>> >>> and if you want to do it for the whole image you can use one bulk become >>> and it will be fast. >>> >>> I will experiment and see if it is working. >>> >> > > > Well, with the following script it seems I am able to embed all source code > in trailers: > > | trailer source key dict classes | > dict := IdentityDictionary new. > CompiledMethod allInstances do: [:aMethod | > trailer := aMethod trailer. > source := aMethod getSource. > trailer sourceCode: source. > dict at: aMethod put: (aMethod copyWithTrailerBytes: trailer). > ]. > dict keys asArray elementsExchangeIdentityWith: dict values asArray. > > CompiledMethod allInstances do: [:aMethod | source := aMethod getSource. trailer := CompiledMethodTrailer new sourceCode: source. dict at: aMethod put: (aMethod copyWithTrailerBytes: trailer). ]. As for what we should encode in method trailer and what we shouldn't, i think we should put an id there, and then have a source code manager which will work as key-value storage to retrieve stuff at given key. And stuff can be not just a source code, but more compound object which will also carry things like, author, timestamp, link to previous version etc etc etc. Now, if we could have such abstraction at first place, then it is no longer relevant where the actual data is stored, in files, in image, or in networked database, because you can have any of them. I would really like to invest some time to that, or find someone who could do that, so i can help with it :) -- Best regards, Igor Stasenko. |
On 18 January 2012 10:22, Igor Stasenko <[hidden email]> wrote:
> On 17 January 2012 19:22, Mariano Martinez Peck <[hidden email]> wrote: >> >> >> On Tue, Jan 17, 2012 at 6:31 PM, Mariano Martinez Peck >> <[hidden email]> wrote: >>> >>> >>>> If you don't bother about being slow, you can also make it much easier: >>>> >>>> >>>> |trailer source m | >>>> m := (Gofer>>#load). >>>> trailer := m trailer. >>>> source := m getSource. >>>> trailer sourceCode: source. >>>> m becomeForward: (m copyWithTrailerBytes: trailer) >>>> >>>> and if you want to do it for the whole image you can use one bulk become >>>> and it will be fast. >>>> >>>> I will experiment and see if it is working. >>>> >>> >> >> >> Well, with the following script it seems I am able to embed all source code >> in trailers: >> >> | trailer source key dict classes | >> dict := IdentityDictionary new. >> CompiledMethod allInstances do: [:aMethod | >> trailer := aMethod trailer. >> source := aMethod getSource. >> trailer sourceCode: source. >> dict at: aMethod put: (aMethod copyWithTrailerBytes: trailer). >> ]. >> dict keys asArray elementsExchangeIdentityWith: dict values asArray. >> >> > you doing it a little bit wrong > > CompiledMethod allInstances do: [:aMethod | > source := aMethod getSource. > trailer := CompiledMethodTrailer new sourceCode: source. > dict at: aMethod put: (aMethod copyWithTrailerBytes: trailer). > ]. > > > As for what we should encode in method trailer and what we shouldn't, i think > we should put an id there, and then have a source code manager > which will work as key-value storage to retrieve stuff at given key. > And stuff can be not just a source code, but more compound object > which will also carry things like, author, timestamp, link to previous > version etc etc etc. Just thought that you can use Fuel to serialize such object and materialize it by demand by taking an id. :) > Now, if we could have such abstraction at first place, then it is no > longer relevant where the actual data is stored, in files, in image, > or in networked database, > because you can have any of them. > > I would really like to invest some time to that, or find someone who > could do that, so i can help with it :) > > -- > Best regards, > Igor Stasenko. -- Best regards, Igor Stasenko. |
In reply to this post by Stefan Marr-3
Hi-- If you have some way of uniquely identifying a method, then you can put the source wherever you like (e.g., in some other live object memory), and look it up when you need it. For an example, see the MethodID class in Spoon[1]. -C [1] http://dl.dropbox.com/u/15188004/spoon/Spoon%203%20alpha%203.zip -- Craig Latta www.netjam.org/resume +31 6 2757 7177 + 1 415 287 3547 |
On 19 February 2012 00:54, Craig Latta <[hidden email]> wrote:
> > Hi-- > > If you have some way of uniquely identifying a method, then you can > put the source wherever you like (e.g., in some other live object > memory), and look it up when you need it. For an example, see the > MethodID class in Spoon[1]. > indeed. it is really unimportant where to store the method(s). They can be stored in file, remote DB or remote image.. But first we need to change the system to work in that way, by removing hardcoded SourceFiles everywhere and replacing with nice polymorphic source manager(s). > > -C > > [1] http://dl.dropbox.com/u/15188004/spoon/Spoon%203%20alpha%203.zip > > -- > Craig Latta > www.netjam.org/resume > +31 6 2757 7177 > + 1 415 287 3547 > > > -- Best regards, Igor Stasenko. |
On Sun, Feb 19, 2012 at 2:37 PM, Igor Stasenko <[hidden email]> wrote:
+1 to both. having an abstact SourceManager with a LocalFileBaseSourceManager and Pharo defualt using that would be awesome. Imagine we can then plug different ones. This is a really interesting idea/project in case someone wants to investigate....
-- Mariano http://marianopeck.wordpress.com |
In reply to this post by Igor Stasenko
On 19 February 2012 13:37, Igor Stasenko <[hidden email]> wrote:
> On 19 February 2012 00:54, Craig Latta <[hidden email]> wrote: >> >> Hi-- >> >> If you have some way of uniquely identifying a method, then you can >> put the source wherever you like (e.g., in some other live object >> memory), and look it up when you need it. For an example, see the >> MethodID class in Spoon[1]. >> > indeed. > it is really unimportant where to store the method(s). > They can be stored in file, remote DB or remote image.. > But first we need to change the system to work in that way, by > removing hardcoded > SourceFiles everywhere and replacing with nice polymorphic source manager(s). It's unimportant right up until you're not online. Please don't write systems that assume a reliable network! (Reply: "You can always cache source, and possibly supply a prepopulated cache, called a .sources file.") frank >> -C >> >> [1] http://dl.dropbox.com/u/15188004/spoon/Spoon%203%20alpha%203.zip >> >> -- >> Craig Latta >> www.netjam.org/resume >> +31 6 2757 7177 >> + 1 415 287 3547 >> >> >> > > > > -- > Best regards, > Igor Stasenko. > |
> > > If you have some way of uniquely identifying a method, then you > > > can put the source wherever you like (e.g., in some other live > > > object memory), and look it up when you need it. For an example, > > > see the MethodID class in Spoon[1]. > > > > indeed. > > it is really unimportant where to store the method(s). > > They can be stored in file, remote DB or remote image.. > > But first we need to change the system to work in that way, by > > removing hardcoded SourceFiles everywhere and replacing with nice > > polymorphic source manager(s). > > It's unimportant right up until you're not online. Please don't write > systems that assume a reliable network! (Reply: "You can always cache > source, and possibly supply a prepopulated cache, called a .sources > file.") Just for the record, the typical Spoon history memory resides locally, remote messages are just going to and from localhost. It snapshots itself after every change. -C -- Craig Latta www.netjam.org/resume +31 6 2757 7177 + 1 415 287 3547 |
Free forum by Nabble | Edit this page |