About CompactClasses and their future [WAS] how many free bits we have in Object Header?

Hi Mariano,

I see two solutions
- make the identityHash (even) smaller
- remove compact classes

The former would probably not be hard to implement (just change some bit masks and then rehash all sets). I've brought up the idea about the latter some time ago on this mailing list. The idea is to remove compact classes to get a larger identityHash (trading memory against speed). After the removal, only two header types are left and hence you gain a spare bit. I think this shouldn't be too much work either, but I haven't come around to implementing it (yet).

Good luck ;)
Adrian

On May 4, 2010, at 13:13 , Mariano Martinez Peck wrote:

>>>
>>> Now...the question is, is there another free bit in the object header ? I
>>> need another one :(
>>
>> Based on the comment at the bottom of Interpreter>>internalIsImmutable:
>> I suspect that somebody else has their eye on that last available bit
>> also ;-)
>>
>>
> Yes, I know :(   But for the moment I don't care to "integrate" my stuff.  I
> just want to play and experiment. And even for that, I need 2 bits :(
> Of course, if I once make something good to integrate, I will have double
> problem :(
>
> So...no solution ?

For measuring, I wouldn't look at the current process' memory consumption because the VM has some headroom to not have to allocate new memory too often. Depending on when memory is grown or shrunken, this may yield very different results. What I would do is to GC and save the image to disk and then compare the size of the .image files. I wouldn't run the tests either.

Making more classes compact, may make sense. But I doubt that you get 3 or even 5 MB. That sounds way too good ;)

Cheers,
Adrian

On May 6, 2010, at 11:24 , Mariano Martinez Peck wrote:

> Hi folks. I was just trying to understand this optimization of the Compact
> Classes. If I understood correctly, most objects needs a second word there
> they have 30 bits of OOP of their class and 2 for the header type.
>
> There are classes that have a LOT of instances. For those cases we have
> compact classes where their instances have a number using those 5 bits in
> the first word of the header, and thus, they don't need a second header.
> Then, less memory is used.
>
> So...I understood ok ?
>
> Now, if that is the situation. I have evaluated Smalltalk
> compactClassesArray  and I see that in the array there are a lot of nils. 16
> classes in Squeak and 15 in Pharo. The rest of the 31, are just nil.
>
> Now I wonder if it is worth to do anything with that :
>
> a) Use 4 bits instead of 5 and free one more bit in the OH
> b) remove completely CompactClasses as Adrian said
> c) Let 5 bits but then add more classes (up to 31) so that we can get
> benefits of that. Do you know if there are side effects of becomingCompact a
> class ?
>
> If I evaluate SpaceTally new printSpaceAnalysis  I get something like this:
>
> Class                           code space # instances  inst space percent
> ByteString                            2217       71317       3621735    19.3
> CompiledMethod                       21064       56057       3517772    18.7
> Bitmap                                3893        2256       3424604    18.2
> Array                                 2478       71108       2198420    11.7
> ByteSymbol                             959       37896        962465     5.1
> ByteArray                             2778         317        778297     4.1
> MethodDictionary                      1819        5964        449252     2.4
> MCVersionInfo                         1313       10369        414760     2.2
> DateAndTime                           6974       13793        331032     1.8
> WeakArray                              827         286        326900     1.7
> Association                            749       26784        321408     1.7
> UUID                                  1528       10370        248880     1.3
> Float                                11442       16450        197400     1.1
> ClassOrganizer                        1398        5964        190848     1.0
> ODatedEntry                            263        7049        169176     0.9
> Time                                  5530       10369        165904     0.9
> Duration                              3025       10290        164640     0.9
> Date                                  3167       10289        164624     0.9
> LargePositiveInteger                  2515       14340        118169     0.6
> Metaclass                             4250        2876        115040     0.6
> Point                                 6404        8476        101712     0.5
> GlyphForm                              307        2030         73080     0.4
> FreeTypeCacheEntry                     536        2238         71616     0.4
> SpaceTallyItem                         651        2509         60216     0.3
> OrderedCollection                     5351        2831         56620     0.3
> FreeTypeFileInfo                       797         321         30816     0.2
> FreeTypeFontFamilyMember              1274         537         25776     0.1
> Color                                21202        1255         25100     0.1
> TraitComposition                      2651        2013         24156     0.1
> Set                                   3510        1358         21728     0.1
> Dictionary                            5742        1344         21504     0.1
> RemoteString                          1533        1236         19776     0.1
> SparseLargeTable                      2376           2         18708     0.1
> WordArray                             1821          30         18608     0.1
> SoundBuffer                           3299           3         18508     0.1
> Pragma                                1558         862         17240     0.1
> Preference                            1534         379         16676     0.1
> EventHandler                          3216         125         14500     0.1
> MorphExtension                        3249         277         14404     0.1
> Character                             6428         939         11268     0.1
> SortedCollection                      1496         462         11088     0.1
> AdditionalMethodState                 2576         859         10304     0.1
> ISOLanguageDefinition                18114         402          9648     0.1
> TimeStamp                              861         392          9408     0.1
>
>
> So...for this quick evaluation, maybe we can become compact also the
> following classes for example:
>
> Smalltalk compactClassesArray
>
> #( ByteSymbol ByteArray MethodDictionary "Association" "WeakArray"
> "Metaclass" DateAndTime UUID Time Duration Date "MethodDictionary"
> ClassOrganizer Metaclass Pragma OrderedCollection RemoteString WordArray
> Color Character )
> do: [:each | (Smalltalk at: each) becomeCompact]
>
> Do this make sense ? How could I measure this?  I took a Pharo image and
> compare the memory used by the VM running all test before and after those
> new compact classes. From the "Activity monitor" I noticed between 3 and 5
> MB less used with these new compact classes.
>
> Thanks in advance
>
> Mariano
>
>
> On Tue, May 4, 2010 at 2:12 PM, Adrian Lienhard <[hidden email]> wrote:
>
>>
>> Hi Mariano,
>>
>> I see two solutions
>> - make the identityHash (even) smaller
>> - remove compact classes
>>
>> The former would probably not be hard to implement (just change some bit
>> masks and then rehash all sets). I've brought up the idea about the latter
>> some time ago on this mailing list. The idea is to remove compact classes to
>> get a larger identityHash (trading memory against speed). After the removal,
>> only two header types are left and hence you gain a spare bit. I think this
>> shouldn't be too much work either, but I haven't come around to implementing
>> it (yet).
>>
>> Good luck ;)
>> Adrian
>>
>> On May 4, 2010, at 13:13 , Mariano Martinez Peck wrote:
>>
>>>>>
>>>>> Now...the question is, is there another free bit in the object header ?
>> I
>>>>> need another one :(
>>>>
>>>> Based on the comment at the bottom of Interpreter>>internalIsImmutable:
>>>> I suspect that somebody else has their eye on that last available bit
>>>> also ;-)
>>>>
>>>>
>>> Yes, I know :(   But for the moment I don't care to "integrate" my stuff.
>> I
>>> just want to play and experiment. And even for that, I need 2 bits :(
>>> Of course, if I once make something good to integrate, I will have double
>>> problem :(
>>>
>>> So...no solution ?
>>
>>