isInMemory :)

> Mariano
>
> I was wondering (since we already removed a lot of the behavior related to inMemory) if we could not remove is in
>
> traitNames
> "Answer a SortedCollection of all traits (not including class-traits) names."
> | names |
> names := OrderedCollection new.
> self do:
> [:cl | (cl isInMemory
> and: [(cl isKindOf: Trait)
> and: [(cl name beginsWith: 'AnObsolete') not]])
> ifTrue: [names add: cl name]].
> ^ names
>
> and friend
>

>
> On Mar 20, 2010, at 3:46 PM, stephane ducasse wrote:
>
>> Mariano
>>
>> I was wondering (since we already removed a lot of the behavior related to inMemory) if we could not remove is in
>>
>> traitNames
>> "Answer a SortedCollection of all traits (not including class-traits) names."
>> | names |
>> names := OrderedCollection new.
>> self do:
>> [:cl | (cl isInMemory
>> and: [(cl isKindOf: Trait)
>> and: [(cl name beginsWith: 'AnObsolete') not]])
>> ifTrue: [names add: cl name]].
>> ^ names
>>
>> and friend
>>
>
> Yes... #isInMemory is clearly not needed anymore.

>
> I have a changeset that I will upload to the bug tracker.
>
> Marcus
>
>
> --
> Marcus Denker  -- http://www.marcusdenker.de
> INRIA Lille -- Nord Europe. Team RMoD.
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

>
> On Mar 21, 2010, at 8:40 PM, Adrian Lienhard wrote:
>
>> On Mar 21, 2010, at 19:50 , Marcus Denker wrote:
>>
>>>
>>> Yes... #isInMemory is clearly not needed anymore.
>>
>> Well... it is needed for the swapping out of classes. If we remove #isInMemory, we should also remove the whole image segment stub mechanism because it becomes useless without the isInMemory protection.
>>
>
>
> But was this ever working? Was this ever *used* the last 10 years? Are actually all cases covered? So are there enough #isInmemory checks to not end up with loading
> everything anyways?
>
> We already removed the *gently variants for iterating over subclasses, for example. or the "ifPresentAndInMemory" things....

>
> --
> Marcus Denker  -- http://www.marcusdenker.de
> INRIA Lille -- Nord Europe. Team RMoD.
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

>>>
>>
>>
>> But was this ever working? Was this ever *used* the last 10 years? Are actually all cases covered? So are there enough #isInmemory checks to not end up with loading
>> everything anyways?
>>
>> We already removed the *gently variants for iterating over subclasses, for example. or the "ifPresentAndInMemory" things....
>
> Yes, I did some experiments with it lately and managed to create 4MB images. Cases which are not protected are indeed a problem.
>
>>> I would like to keep that working and extract it with the image segments code into an external package. How to do this cleanly is an interesting question, though...
>>>
>>
>> I think we should ask the question: Do we intent to *use* this mechanism *at all* the next 5 years?
>
> Probably not. Nevertheless, I think its a nice idea but if it is not properly working the chances are small anybody will use.
>
> In any case, the question of how this *could* be factored out in a clean way is interesting if we want to further strip and modularize the image. It seems we are missing appropriate abstractions to modify from external packages behavior like the #isInMemory sends. Overrides are an unsatisfactory solution. So there is a limitation to the kind of code can be made external packages.
>
>
>
>>
>> --
>> Marcus Denker  -- http://www.marcusdenker.de
>> INRIA Lille -- Nord Europe. Team RMoD.
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

>
> On Mar 21, 2010, at 10:51 PM, Stéphane Ducasse wrote:
>
>>>>> Well... it is needed for the swapping out of classes. If we remove #isInMemory, we should also remove the whole image segment stub mechanism because it becomes useless without the isInMemory protection.
>>
>> what I was thinking to remove is the use of isInMemory from SystemDictionary>>allClasses and friends.
>> Now I do not get 100% what you are saying.
>> Do you say that all the queries should be protected against bringing back classes on disc?
>
> The reason why the #isInmemory checks are there at all is the following:
>
> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
> -> Now if you access the class, it is loaded again.
> -> doing anything that looks at "all classes" would load the classes. (all of them)
> -> this includes things like "Object subclasses". or "Smalltalk classNames".
> -> e.g. Openign a browser would load all classes.
> -> so we put #isInMemory everywhere

>
> Marcus
>
> --
> Marcus Denker  -- http://www.marcusdenker.de
> INRIA Lille -- Nord Europe. Team RMoD.
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

>
> On Mar 21, 2010, at 10:59 PM, Marcus Denker wrote:
>>
>> -> this of course means that #subclasses would just return those that by chance are
>>            in memory... I don't understand how that can work. Honestly :-)
>>
>
> of course, they realized that, too. Thus: #subclasses never checked for #isInMemory.
>
> But there where strangely names methods to iterate over all subclasses in memory (called #allSubclassesDoGently:).
>
> But Smalltalk>>#classNames foes check for #isInmemory. Which means you could have a case where a subclass would not be
> in Smalltalk classNames...
>
> This is all fundamentally broken, sorry.
>
> Marcus
>
> --
> Marcus Denker  -- http://www.marcusdenker.de
> INRIA Lille -- Nord Europe. Team RMoD.
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

>>
>> The reason why the #isInmemory checks are there at all is the following:
>>
>> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
>> -> Now if you access the class, it is loaded again.
>> -> doing anything that looks at "all classes" would load the classes. (all of them)
>> -> this includes things like "Object subclasses". or "Smalltalk classNames".
>> -> e.g. Openign a browser would load all classes.
>> -> so we put #isInMemory everywhere
>
> Yes I know that :)
> Now I was wondering what adrian meant.
>
>> -> this of course means that #subclasses would just return those that by chance are
>>            in memory... I don't understand how that can work. Honestly :-)
>>
>> I think we shoud remove all that stuff and do it "for real". c.f. Loom.
>
> probably.

>
> On Mar 21, 2010, at 10:59 PM, Marcus Denker wrote:
>
>>
>> On Mar 21, 2010, at 10:51 PM, Stéphane Ducasse wrote:
>>
>>>>>> Well... it is needed for the swapping out of classes. If we remove #isInMemory, we should also remove the whole image segment stub mechanism because it becomes useless without the isInMemory protection.
>>>
>>> what I was thinking to remove is the use of isInMemory from SystemDictionary>>allClasses and friends.
>>> Now I do not get 100% what you are saying.
>>> Do you say that all the queries should be protected against bringing back classes on disc?
>>
>> The reason why the #isInmemory checks are there at all is the following:
>>
>> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
>> -> Now if you access the class, it is loaded again.
>> -> doing anything that looks at "all classes" would load the classes. (all of them)
>> -> this includes things like "Object subclasses". or "Smalltalk classNames".
>> -> e.g. Openign a browser would load all classes.
>> -> so we put #isInMemory everywhere
>
> Yes I know that :)
> Now I was wondering what adrian meant.

>
> On Mar 21, 2010, at 11:18 PM, Stéphane Ducasse wrote:
>>>
>>> The reason why the #isInmemory checks are there at all is the following:
>>>
>>> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
>>> -> Now if you access the class, it is loaded again.
>>> -> doing anything that looks at "all classes" would load the classes. (all of them)
>>> -> this includes things like "Object subclasses". or "Smalltalk classNames".
>>> -> e.g. Openign a browser would load all classes.
>>> -> so we put #isInMemory everywhere
>>
>> Yes I know that :)
>> Now I was wondering what adrian meant.
>>
>>> -> this of course means that #subclasses would just return those that by chance are
>>>           in memory... I don't understand how that can work. Honestly :-)
>>>
>>> I think we shoud remove all that stuff and do it "for real". c.f. Loom.
>>
>> probably.
>
>
> so I was wondering: what exactly is bad in swapping in all classes when opening a browser?
>
> 1) no need to swap in *everything*. Classe are large because they reference *lots* of methods (which in turn reference
>             lots of literals).
>            But most cases where we iterate over all classes we will not touch the methods (other than when we *want*
>            the methods).
>
> => be fine grained. Loading all classes does not imply loading all methodDictcs.
>
> 2) If I am interested in all classes, I am interested in all classes. Give them to me!
>              And get rid of them as soon as I am not interested anymore
>
> => on-demand loading is half of the story. On-No-Demand-*Unloading* is needed, too.
>
> This means, when developing, we will have all classes (or at least parts of them) in memory.
>                (*because we look at them*). But in Deployment, the working set of objects will be different, and
>               only those classes that are needed for *execution* will stay in memory.
>
> 3) Intelligent caching. Of course we don't load objects one-by-one, we will have an intelligent cache.
>             We will do intelligent pre-fetching, too, to not have to go to disk for each object when it's clear that
>             probably we will load more that just that one object (or it's cheap to load more).
>
>     But that is orthogonal and invisible.

>>>> The reason why the #isInmemory checks are there at all is the following:
>>>>
>>>> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
>>>> -> Now if you access the class, it is loaded again.
>>>> -> doing anything that looks at "all classes" would load the classes. (all of them)
>>>> -> this includes things like "Object subclasses". or "Smalltalk classNames".
>>>> -> e.g. Openign a browser would load all classes.
>>>> -> so we put #isInMemory everywhere
>>>
>>> Yes I know that :)
>>> Now I was wondering what adrian meant.
>>>
>>>> -> this of course means that #subclasses would just return those that by chance are
>>>>          in memory... I don't understand how that can work. Honestly :-)
>>>>
>>>> I think we shoud remove all that stuff and do it "for real". c.f. Loom.
>>>
>>> probably.
>>
>>
>> so I was wondering: what exactly is bad in swapping in all classes when opening a browser?
>>
>> 1) no need to swap in *everything*. Classe are large because they reference *lots* of methods (which in turn reference
>>            lots of literals).
>>           But most cases where we iterate over all classes we will not touch the methods (other than when we *want*
>>           the methods).
>>
>> => be fine grained. Loading all classes does not imply loading all methodDictcs.
>>
>> 2) If I am interested in all classes, I am interested in all classes. Give them to me!
>>             And get rid of them as soon as I am not interested anymore
>>
>> => on-demand loading is half of the story. On-No-Demand-*Unloading* is needed, too.
>>
>> This means, when developing, we will have all classes (or at least parts of them) in memory.
>>               (*because we look at them*). But in Deployment, the working set of objects will be different, and
>>              only those classes that are needed for *execution* will stay in memory.
>>
>> 3) Intelligent caching. Of course we don't load objects one-by-one, we will have an intelligent cache.
>>            We will do intelligent pre-fetching, too, to not have to go to disk for each object when it's clear that
>>            probably we will load more that just that one object (or it's cheap to load more).
>>
>>     But that is orthogonal and invisible.
>
> Exactly! If granularity even was at the method level (executing 100 methods of Morph would not bring in all 1000 methods), we could get a system that is *really* small (and also faster because GC has to do less work).
>
> Adrian
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> so do we remove imageSegments support?
>
> Stef
>
>>>>> The reason why the #isInmemory checks are there at all is the following:
>>>>>
>>>>> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
>>>>> -> Now if you access the class, it is loaded again.
>>>>> -> doing anything that looks at "all classes" would load the classes. (all of them)
>>>>> -> this includes things like "Object subclasses". or "Smalltalk classNames".
>>>>> -> e.g. Openign a browser would load all classes.
>>>>> -> so we put #isInMemory everywhere
>>>>
>>>> Yes I know that :)
>>>> Now I was wondering what adrian meant.
>>>>
>>>>> -> this of course means that #subclasses would just return those that by chance are
>>>>>         in memory... I don't understand how that can work. Honestly :-)
>>>>>
>>>>> I think we shoud remove all that stuff and do it "for real". c.f. Loom.
>>>>
>>>> probably.
>>>
>>>
>>> so I was wondering: what exactly is bad in swapping in all classes when opening a browser?
>>>
>>> 1) no need to swap in *everything*. Classe are large because they reference *lots* of methods (which in turn reference
>>>           lots of literals).
>>>          But most cases where we iterate over all classes we will not touch the methods (other than when we *want*
>>>          the methods).
>>>
>>> => be fine grained. Loading all classes does not imply loading all methodDictcs.
>>>
>>> 2) If I am interested in all classes, I am interested in all classes. Give them to me!
>>>            And get rid of them as soon as I am not interested anymore
>>>
>>> => on-demand loading is half of the story. On-No-Demand-*Unloading* is needed, too.
>>>
>>> This means, when developing, we will have all classes (or at least parts of them) in memory.
>>>              (*because we look at them*). But in Deployment, the working set of objects will be different, and
>>>             only those classes that are needed for *execution* will stay in memory.
>>>
>>> 3) Intelligent caching. Of course we don't load objects one-by-one, we will have an intelligent cache.
>>>           We will do intelligent pre-fetching, too, to not have to go to disk for each object when it's clear that
>>>           probably we will load more that just that one object (or it's cheap to load more).
>>>
>>>     But that is orthogonal and invisible.
>>
>> Exactly! If granularity even was at the method level (executing 100 methods of Morph would not bring in all 1000 methods), we could get a system that is *really* small (and also faster because GC has to do less work).
>>
>> Adrian
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

> My plan is to extract ImageSegments into an external package and maintain it there. Since I will probably not be able to make the class swapping work, this part of image segments will be dropped.
>
> If you start with removing #isInMemory, thats fine with me. I can do the rest later.
>
> Cheers,
> Adrian
>
>
> On Mar 22, 2010, at 14:42 , Stéphane Ducasse wrote:
>
>> so do we remove imageSegments support?
>>
>> Stef
>>
>>>>>> The reason why the #isInmemory checks are there at all is the following:
>>>>>>
>>>>>> -> they used imageSegements (and before, the ObjectOut stuff) to swap out objecs to disk
>>>>>> -> Now if you access the class, it is loaded again.
>>>>>> -> doing anything that looks at "all classes" would load the classes. (all of them)
>>>>>> -> this includes things like "Object subclasses". or "Smalltalk classNames".
>>>>>> -> e.g. Openign a browser would load all classes.
>>>>>> -> so we put #isInMemory everywhere
>>>>>
>>>>> Yes I know that :)
>>>>> Now I was wondering what adrian meant.
>>>>>
>>>>>> -> this of course means that #subclasses would just return those that by chance are
>>>>>>        in memory... I don't understand how that can work. Honestly :-)
>>>>>>
>>>>>> I think we shoud remove all that stuff and do it "for real". c.f. Loom.
>>>>>
>>>>> probably.
>>>>
>>>>
>>>> so I was wondering: what exactly is bad in swapping in all classes when opening a browser?
>>>>
>>>> 1) no need to swap in *everything*. Classe are large because they reference *lots* of methods (which in turn reference
>>>>          lots of literals).
>>>>         But most cases where we iterate over all classes we will not touch the methods (other than when we *want*
>>>>         the methods).
>>>>
>>>> => be fine grained. Loading all classes does not imply loading all methodDictcs.
>>>>
>>>> 2) If I am interested in all classes, I am interested in all classes. Give them to me!
>>>>           And get rid of them as soon as I am not interested anymore
>>>>
>>>> => on-demand loading is half of the story. On-No-Demand-*Unloading* is needed, too.
>>>>
>>>> This means, when developing, we will have all classes (or at least parts of them) in memory.
>>>>             (*because we look at them*). But in Deployment, the working set of objects will be different, and
>>>>            only those classes that are needed for *execution* will stay in memory.
>>>>
>>>> 3) Intelligent caching. Of course we don't load objects one-by-one, we will have an intelligent cache.
>>>>          We will do intelligent pre-fetching, too, to not have to go to disk for each object when it's clear that
>>>>          probably we will load more that just that one object (or it's cheap to load more).
>>>>
>>>>     But that is orthogonal and invisible.
>>>
>>> Exactly! If granularity even was at the method level (executing 100 methods of Morph would not bring in all 1000 methods), we could get a system that is *really* small (and also faster because GC has to do less work).
>>>
>>> Adrian
>>> _______________________________________________
>>> Pharo-project mailing list
>>> [hidden email]
>>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>>
>>
>> _______________________________________________
>> Pharo-project mailing list
>> [hidden email]
>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
>
>
> _______________________________________________
> Pharo-project mailing list
> [hidden email]
> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project