Re: getting rid of Symbol >> new: ?
Locked
12
12
Re: getting rid of Symbol >> new: ?
 
|
On Tue, Dec 8, 2009 at 10:54 PM, Mariano Martinez Peck <[hidden email]> wrote:
Sorry, I forgot to said I found this in ObjectMemory >> markAndTrace: (self isWeakNonInt: oop) ifTrue: [ "Set lastFieldOffset before the weak fields in the receiver" lastFieldOffset := (self nonWeakFieldsOf: oop) << ShiftForWord. "And remember as weak root" weakRootCount := weakRootCount + 1. weakRoots at: weakRootCount put: oop. ] ifFalse: [ "Do it the usual way" lastFieldOffset := self lastPointerOf: oop. ]. But I don't know if this make sense or not. Thanks!! Mariano
_______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
|
|
In reply to this post by Henrik Sperre Johansen
The chore of loading all 500 fonts on a macintosh, rendering them all to show what they look like, was *quite* expensive and added many 10's of seconds to Sophie startup time.
At first we dumped the entire font menu oops tree into a image segment. The did a merge process by checking the current machine's view of the fonts, versus our historical view. usually fonts don't change much so the cost was only reading the directory. This worked oh, 99.99% of the time. The 0.01% was the hassle. Our fallback was then to store the rendered form for each font in an image segment. That seemed to work 100% As pointed out there is just a mess of eToys/Projects/whatever special casing going on in the default storage/reading of the segments, and when it goes bad, it's *hard* to figure out why. I did explore this area a bit more with WikiServer. In fact it's still on my to-do list, you can customize/override things to get a more simplistic set of steps, but I wasn't quite sure that sending a Pier Wiki to an image segment would be readable in the future, so I decided not to offer that as a storage option. On 2009-12-08, at 3:04 AM, Henrik Johansen wrote: > > On Dec 8, 2009, at 11:33 20AM, Mariano Martinez Peck wrote: > >> >> >> Thanks John. I am interested in your experience. However, I didn't understand this last paragraph where you said to finally make it work. >> I don't understand what did you change. What is the difference between "font menu" and " stored forms of >> each font face for the menu" ? > The way I read it, instead of storing the fonts themselves in an image segment, they stored a form with a string rendered by the font, then used that to display in list instead of actually loading the font, and rendering text with it. > That way the list could be shown fast, without having to load the fonts themselves until actually used/fonts on the machine changes. >> >> In addition, do you know why this has solved the problems ? > > If you look at the relative complexity of TTCFont and Form instances, it'd hardly come as a surprise that one may be loaded consistently, while the other is harder to get loaded correctly in all cases... > In other words, to me it seems like a workaround for non-repeatable errors when loading complex objects within ImageSegments (by storing simpler objects instead). > > Cheers, > Henry > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- =========================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com =========================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
In reply to this post by Mariano Martinez Peck
I don't have the VMMaker code at hand right now (or rather, don't have
time to look at it ;)), but I guess that after that piece of code you pasted below there is a loop from the first field (BaseHeader+4) to the last non-weak field (lastFieldOffset) in which the non-weak fields are traced. Note, that if an object both fixed and indexable weak fields the latter are always located after the former. You can see the memory layout and object formats on the pages 9 and 10 of the slides [1] (from a lecture about VMs I gave recently). Cheers, Adrian [1] http://www.adrian-lienhard.ch/presentations On Dec 8, 2009, at 22:57 , Mariano Martinez Peck wrote: > On Tue, Dec 8, 2009 at 10:54 PM, Mariano Martinez Peck < > [hidden email]> wrote: > >> >> >> On Tue, Dec 8, 2009 at 5:07 PM, Adrian Lienhard <[hidden email]> >> wrote: >> >>> >>> On Dec 8, 2009, at 12:08 , Mariano Martinez Peck wrote: >>> >>>> On Fri, Dec 4, 2009 at 12:05 PM, Adrian Lienhard <[hidden email]> >>>> wrote: >>> >>>> When you are creating your root of objects and you put symbols >>>> inside, they >>>> are not put in ourPointers but in ByteArray. >>>> This is due to the fact that the only object who is pointing to >>>> that >>>> symbol >>>> is inside the segment ? >>> >>> To be precise, the symbols are also pointed to by the symbol table, >>> but only by weak references. Since image segments use the GC mark >>> logic, these pointers are not considered. >>> >> >> Ahhh ok. Now I see Symbol class >> rehash where it sets to >> SymbolTable >> := WeakSet >> >> Now...my finally question is, where in the code you can see the GC >> only >> mark "normal" objects and that week objects are not being taken into >> account. Do you know ? I tried to search it but I didn't find it. >> >> > Sorry, I forgot to said I found this in ObjectMemory >> markAndTrace: > > (self isWeakNonInt: oop) ifTrue: [ > "Set lastFieldOffset before the weak fields in the receiver" > lastFieldOffset := (self nonWeakFieldsOf: oop) << ShiftForWord. > "And remember as weak root" > weakRootCount := weakRootCount + 1. > weakRoots at: weakRootCount put: oop. > ] ifFalse: [ > "Do it the usual way" > lastFieldOffset := self lastPointerOf: oop. > ]. > > > But I don't know if this make sense or not. > > Thanks!! > > Mariano > > >>>> What you do with this piece of code: >>>> >>>> symbolHolder := Symbol allSymbols. >>>> >>>> is to hold those symbols there. So, when ImageSegment uses the GC >>>> techniques >>>> to detect which objects are ONLY pointed from inside of the >>>> segment, >>>> the >>>> symbols is not found (because it is accessible trough that test) >>>> and >>>> thus, >>>> it goes to outPointers instead of ByteArray. >>>> >>>> And of course, if it is in outPointers instead of ByteArray when >>>> the >>>> segment >>>> is loaded again, yo don't create a symbol again but use the same >>>> object (the >>>> one of the oop). >>>> >>>> I am correct? or I understood anything ? >>> >>> yes. >>> >>> Cheers, >>> Adrian >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> [hidden email] >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
On Wed, Dec 9, 2009 at 9:40 AM, Adrian Lienhard <[hidden email]> wrote: I don't have the VMMaker code at hand right now (or rather, don't have Thanks for the explanation Adrian and the link Adrian...now, being a bit off-topic, I have the following situation about ImageSegment. I tried to reproduce it in a unit test, but I couldn't yet :( Suppose I create a segment with a tree of objects. Suppose inside of that tree I have an object that refers to an external object called XXX, thus it is put in the outPointers. Then I extract the segment, I write it to disk, and I put a nil to the segment variable. After that, for some reason, my object XXX is garbage collected from my image. Then I want to load again my segment (reading the file and installling it). What happens ? it fails? If true, suppose I have already "installed" some other objects, what happen with those objects? is this atomic ? Thanks a lot for anyone that can give me a hint. Mariano Cheers, _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
In reply to this post by Adrian Lienhard
Martin and I had been working towards the idea that ImageSegments should never create non-Canonical symbols, and the user should not have to use manual techniques to preserve symbols they are interested in keeping alive. I ended up getting stuck however, in trying to discover what process ImageSegment goes through while it is installing objects - ImageSegment #install has a call to an unimplemented method #readFromFileWithSymbols which it seems it should be using, but isn't....(wha' the?) install "This operation retrieves the segment if necessary from file storage, installs it in memory, and replaces (using become:) all the root stubs with the reconstructed roots of the segment." | newRoots | state = #onFile ifTrue: [self readFromFile]. state = #onFileWithSymbols ifTrue: [self readFromFileWithSymbols. endMarker := segment nextObject. "for enumeration of objects" endMarker == 0 ifTrue: [endMarker := 'End' clone]]. (state = #active) | (state = #imported) ifFalse: [self errorWrongState]. newRoots := self loadSegmentFrom: segment outPointers: outPointers. state = #imported ifTrue: ["just came in from exported file" arrayOfRoots := newRoots] ifFalse: [ arrayOfRoots elementsForwardIdentityTo: newRoots]. state := #inactive. Beeper beepPrimitive I think the next step in this trajectory is to trace is how the state is getting set - right now it is kind of a mystery... Does this seem reasonable? Thanks, Sheri |
Re: getting rid of Symbol >> new: ?
|
|
Sheridan Mahoney wrote:
> Martin and I had been working towards the idea that ImageSegments should > never create non-Canonical symbols, and the user should not have to use > manual techniques to preserve symbols they are interested in keeping alive. > > I ended up getting stuck however, in trying to discover what process > ImageSegment > goes through while it is installing objects - ImageSegment #install has a > call to an > unimplemented method #readFromFileWithSymbols which it seems it should be > using, but isn't....(wha' the?) > > > install > "This operation retrieves the segment if necessary from file storage, > installs it in memory, and replaces (using become:) all the root stubs with > the reconstructed roots of the segment." > > | newRoots | > state = #onFile ifTrue: [self readFromFile]. > state = #onFileWithSymbols ifTrue: [self readFromFileWithSymbols. My guess (and only a guess) is that #onFileWithSymbols never becomes a state in the test case that's failing. And my opinion is that it's suspicious to require something different for a image segment that contains symbols as opposed to one that doesn't; the ideal would be to have image segments just transparently handle Symbols correctly. Users just shouldn't have to deal with details at that level; the correct semantics are extremely clear. Regards, -Martin _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
On Thu, Dec 10, 2009 at 3:32 AM, Martin McClure <[hidden email]> wrote:
Yes! And my opinion is that it's suspicious to require something different 142% agree. So...there are a couple of things I think ImageSegment should do instead of the user: 1) If you put in the segment a class, ImageSegment automatically puts also the metcaclass. 2) Deal transparently if you put symbols in the segment. 3) Evaluate "Smalltalk garbageCollect" after the message "extract". This is most of the cases something you want to do. I addition, look the senders of extract and you will see that in most of the cases, after doing the extract, it evaluates a garbageCollect. Cheers, Mariano
_______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
In reply to this post by Mariano Martinez Peck
Hi Mariano,
I don't understand your question(s)... If you have some code, that would probably help. Cheers, Adrian On Dec 9, 2009, at 21:06 , Mariano Martinez Peck wrote: > On Wed, Dec 9, 2009 at 9:40 AM, Adrian Lienhard <[hidden email]> > wrote: > >> I don't have the VMMaker code at hand right now (or rather, don't >> have >> time to look at it ;)), but I guess that after that piece of code you >> pasted below there is a loop from the first field (BaseHeader+4) to >> the last non-weak field (lastFieldOffset) in which the non-weak >> fields >> are traced. Note, that if an object both fixed and indexable weak >> fields the latter are always located after the former. You can see >> the >> memory layout and object formats on the pages 9 and 10 of the slides >> [1] (from a lecture about VMs I gave recently). >> >> > Thanks for the explanation Adrian and the link Adrian...now, being a > bit > off-topic, I have the following situation about ImageSegment. I > tried to > reproduce it in a unit test, but I couldn't yet :( > > Suppose I create a segment with a tree of objects. Suppose inside of > that > tree I have an object that refers to an external object called XXX, > thus it > is put in the outPointers. > Then I extract the segment, I write it to disk, and I put a nil to the > segment variable. > After that, for some reason, my object XXX is garbage collected from > my > image. > Then I want to load again my segment (reading the file and > installling it). > What happens ? it fails? If true, suppose I have already "installed" > some > other objects, what happen with those objects? is this atomic ? > > Thanks a lot for anyone that can give me a hint. > > Mariano > > > >> Cheers, >> Adrian >> >> [1] http://www.adrian-lienhard.ch/presentations >> >> On Dec 8, 2009, at 22:57 , Mariano Martinez Peck wrote: >> >>> On Tue, Dec 8, 2009 at 10:54 PM, Mariano Martinez Peck < >>> [hidden email]> wrote: >>> >>>> >>>> >>>> On Tue, Dec 8, 2009 at 5:07 PM, Adrian Lienhard <[hidden email]> >>>> wrote: >>>> >>>>> >>>>> On Dec 8, 2009, at 12:08 , Mariano Martinez Peck wrote: >>>>> >>>>>> On Fri, Dec 4, 2009 at 12:05 PM, Adrian Lienhard >>>>>> <[hidden email]> >>>>>> wrote: >>>>> >>>>>> When you are creating your root of objects and you put symbols >>>>>> inside, they >>>>>> are not put in ourPointers but in ByteArray. >>>>>> This is due to the fact that the only object who is pointing to >>>>>> that >>>>>> symbol >>>>>> is inside the segment ? >>>>> >>>>> To be precise, the symbols are also pointed to by the symbol >>>>> table, >>>>> but only by weak references. Since image segments use the GC mark >>>>> logic, these pointers are not considered. >>>>> >>>> >>>> Ahhh ok. Now I see Symbol class >> rehash where it sets to >>>> SymbolTable >>>> := WeakSet >>>> >>>> Now...my finally question is, where in the code you can see the GC >>>> only >>>> mark "normal" objects and that week objects are not being taken >>>> into >>>> account. Do you know ? I tried to search it but I didn't find it. >>>> >>>> >>> Sorry, I forgot to said I found this in ObjectMemory >> >>> markAndTrace: >>> >>> (self isWeakNonInt: oop) ifTrue: [ >>> "Set lastFieldOffset before the weak fields in the receiver" >>> lastFieldOffset := (self nonWeakFieldsOf: oop) << >>> ShiftForWord. >>> "And remember as weak root" >>> weakRootCount := weakRootCount + 1. >>> weakRoots at: weakRootCount put: oop. >>> ] ifFalse: [ >>> "Do it the usual way" >>> lastFieldOffset := self lastPointerOf: oop. >>> ]. >>> >>> >>> But I don't know if this make sense or not. >>> >>> Thanks!! >>> >>> Mariano >>> >>> >>>>>> What you do with this piece of code: >>>>>> >>>>>> symbolHolder := Symbol allSymbols. >>>>>> >>>>>> is to hold those symbols there. So, when ImageSegment uses the GC >>>>>> techniques >>>>>> to detect which objects are ONLY pointed from inside of the >>>>>> segment, >>>>>> the >>>>>> symbols is not found (because it is accessible trough that test) >>>>>> and >>>>>> thus, >>>>>> it goes to outPointers instead of ByteArray. >>>>>> >>>>>> And of course, if it is in outPointers instead of ByteArray when >>>>>> the >>>>>> segment >>>>>> is loaded again, yo don't create a symbol again but use the same >>>>>> object (the >>>>>> one of the oop). >>>>>> >>>>>> I am correct? or I understood anything ? >>>>> >>>>> yes. >>>>> >>>>> Cheers, >>>>> Adrian >>>>> >>>>> _______________________________________________ >>>>> Pharo-project mailing list >>>>> [hidden email] >>>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo- >>>>> project >>>>> >>>> >>>> >>> _______________________________________________ >>> Pharo-project mailing list >>> [hidden email] >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
In reply to this post by Sheridan Mahoney
The objects in the segment's byte array are "instantiated" by the
primitive #loadSegmentFrom:outPointers:. In fact they are not instantiated, but the byte array is just twisted a bit to represent real objects on the heap (internal references and indexes into outPointers are remapped). Adrian On Dec 10, 2009, at 01:59 , Sheridan Mahoney wrote: > > > > Adrian Lienhard wrote: >> >> >> On Dec 8, 2009, at 12:08 , Mariano Martinez Peck wrote: >> >>> On Fri, Dec 4, 2009 at 12:05 PM, Adrian Lienhard <[hidden email]> >>> wrote: >> >>> When you are creating your root of objects and you put symbols >>> inside, they >>> are not put in ourPointers but in ByteArray. >>> This is due to the fact that the only object who is pointing to that >>> symbol >>> is inside the segment ? >> >> To be precise, the symbols are also pointed to by the symbol table, >> but only by weak references. Since image segments use the GC mark >> logic, these pointers are not considered. >> >> > > Martin and I had been working towards the idea that ImageSegments > should > never create non-Canonical symbols, and the user should not have to > use > manual techniques to preserve symbols they are interested in keeping > alive. > > I ended up getting stuck however, in trying to discover what process > ImageSegment > goes through while it is installing objects - ImageSegment #install > has a > call to an > unimplemented method #readFromFileWithSymbols which it seems it > should be > using, but isn't....(wha' the?) > > > install > "This operation retrieves the segment if necessary from file storage, > installs it in memory, and replaces (using become:) all the root > stubs with > the reconstructed roots of the segment." > > | newRoots | > state = #onFile ifTrue: [self readFromFile]. > state = #onFileWithSymbols ifTrue: [self readFromFileWithSymbols. > endMarker := segment nextObject. "for enumeration of objects" > endMarker == 0 ifTrue: [endMarker := 'End' clone]]. > (state = #active) | (state = #imported) ifFalse: [self > errorWrongState]. > newRoots := self loadSegmentFrom: segment outPointers: outPointers. > state = #imported > ifTrue: ["just came in from exported file" > arrayOfRoots := newRoots] > ifFalse: [ > arrayOfRoots elementsForwardIdentityTo: newRoots]. > state := #inactive. > Beeper beepPrimitive > > > I think the next step in this trajectory is to trace is how the > state is > getting > set - right now it is kind of a mystery... > > Does this seem reasonable? > > Thanks, > > Sheri > > > -- > View this message in context: http://n2.nabble.com/Re-getting-rid-of-Symbol-new-tp4109230p4143090.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
In reply to this post by Adrian Lienhard
On Thu, Dec 10, 2009 at 9:21 AM, Adrian Lienhard <[hidden email]> wrote: Hi Mariano, Ok. I commit a test case called testInstallSegmenetWithObjectRemoved which is failing in the version Tests-Mariano.40 of the repository http://www.squeaksource.com/MarianoPhD Maybe what is happening is that "anExternalObject" is not being garbageCollected because there are references from the outPointers array. That sounds logic. What do you think? Kind regards, Mariano Cheers, _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: getting rid of Symbol >> new: ?
|
|
In reply to this post by Martin McClure-2
On Thu, Dec 10, 2009 at 3:32 AM, Martin McClure <[hidden email]> wrote:
Even more...#onFileWithSymbols becomes as state in the method writeToFileWithSymbols But, look the first line of this method: writeToFileWithSymbols | symbols nonSymbols pound | state = #extracted ifFalse: [self error: 'wrong state']. The state #extracted DOES NOT EXISTS!!! Look the senders and you will find only that method hahahahahah I guess it should be #active, because, as you can see in the method extract, the final states it sets is #active. So....you always get "wrong state" ;) Cheers, Mariano And my opinion is that it's suspicious to require something different _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Re: [Pharo-project] getting rid of Symbol >> new: ?
|
|
In reply to this post by Sheridan Mahoney
Martin and I had been working towards the idea that ImageSegments should Do you believe me if I told you that I checked in squeak 2.6 and the method (readFromFileWithSymbols) is still not implemented ? I think it never was hahahah _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
«
Return to Pharo Smalltalk Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |