Big objects and GC

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Big objects and GC

MrGwen
Hi,

   Currently in GST big objects are allocated in the old space
and won't move during the GC. The non-moving property is nice
but putting  big objects in the old space is not good since we
need to do a full GC to free them.

   I've made a small patch not yet finished (there is one last bug when
I do a make check) which creates a new kind of object:

   F_LARGE that can be in young space and after tenured to the old space.
The difference is that the object won't move during the copy gc and the
tenure step.

   git://github.com/MrGwen/GNU-Smalltalk.git in the branch LargeOOP.

Cheers,
Gwen

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: Big objects and GC

MrGwen
On 05/10/2011 11:59 AM, Gwenael Casaccio wrote:

> Hi,
>
> Currently in GST big objects are allocated in the old space
> and won't move during the GC. The non-moving property is nice
> but putting big objects in the old space is not good since we
> need to do a full GC to free them.
>
> I've made a small patch not yet finished (there is one last bug when
> I do a make check) which creates a new kind of object:
>
> F_LARGE that can be in young space and after tenured to the old space.
> The difference is that the object won't move during the copy gc and the
> tenure step.
>
> git://github.com/MrGwen/GNU-Smalltalk.git in the branch LargeOOP.
>
> Cheers,
> Gwen
Hi,

Here is the patch, all the tests are green ;-)
Basically I've added a new flag F_LARGE, and a new method for allocating
large object: gst_alloc_large_obj. The object is fixed and won't move
during the GC, so that implies that when tenuring an object I check if
the object is F_LARGE. Also a special care is taken when the object is
sweep, I use the correct heap. The image loading is updated to take care
of the F_LARGE field.

Cheers,
Gwen

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

large-oop.patch (9K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Big objects and GC

Paolo Bonzini-2

> Here is the patch, all the tests are green ;-)

Nice patch!

I have a few comments.

> diff --git a/libgst/oop.c b/libgst/oop.c
> index ad264d9..018ec95 100644
> --- a/libgst/oop.c
> +++ b/libgst/oop.c
> @@ -66,6 +66,7 @@
>
>  /* Define this flag to turn on debugging code for OOP table management */
>  /* #define GC_DEBUGGING */
> +#define GC_DEBUGGING

Spurious change.

> @@ -701,7 +702,7 @@ _gst_make_oop_fixed (OOP oop)
>  {
>    gst_object newObj;
>    int size;
> -  if (oop->flags & F_FIXED)
> +  if ((oop->flags & F_FIXED) || (oop->flags & F_LARGE))
>      return;

Can you try representing large objects with F_FIXED but not F_OLD?  At
the same time, makeFixed would not cause objects to become old, which is
useful for objects passed to C.

Apparently, I even had some code to handle this in _gst_tenure_oop:

   if (oop->flags & F_OLD)
     return;

   if (!(oop->flags & F_FIXED))
     {
       int size = SIZE_TO_BYTES (TO_INT(oop->object->objSize));
       newObj = (gst_object) _gst_mem_alloc (_gst_mem.old, size);
       if (!newObj)
         abort ();

       memcpy (newObj, oop->object, size);
       _gst_mem.numOldOOPs++;

       oop->object = newObj;
     }

where the increment of numOldOOPs should be moved outside the if.

> @@ -780,6 +781,38 @@ _gst_alloc_obj (size_t size,
>  }
>
>  gst_object
> +_gst_alloc_large_obj (size_t size,
> +                      OOP *p_oop)
> +{
> +  gst_object p_instance;
> +
> +  size = ROUNDED_BYTES (size);
> +
> +  /* If the object is big enough, we put it directly in fixedspace.  */
> +  p_instance = (gst_object) _gst_mem_alloc (_gst_mem.fixed, size);
> +  if COMMON (p_instance)
> +    goto ok;
> +
> +  _gst_global_gc (size);
> +  p_instance = (gst_object) _gst_mem_alloc (_gst_mem.fixed, size);
> +  if COMMON (p_instance)
> +    goto ok;
> +
> +  _gst_compact (0);

Compacting is not going to be helpful, for fixed objects.

> @@ -1452,7 +1485,7 @@ _gst_sweep_oop (OOP oop)
>      _gst_make_oop_non_weak (oop);
>
>    /* Free unreachable oldspace objects.  */
> -  if UNCOMMON (oop->flags & F_FIXED)
> +  if UNCOMMON ((oop->flags & F_FIXED) || (oop->flags & F_LARGE))
>      {
>        _gst_mem.numOldOOPs--;

This makes stats wrong.  With the idea above, the code would be like:

   if ((oop->flags & F_LOADED) == 0)
     {
       if UNCOMMON (oop->flags & F_FIXED)
         _gst_mem_free (_gst_mem.fixed, oop->object);
       else if UNCOMMON (oop->flags & F_OLD)
         _gst_mem_free (_gst_mem.old, oop->object);
     }

   if UNCOMMON (oop->flags & F_OLD)
     {
       _gst_mem.numOldOOPs--;
       stats.reclaimedOldSpaceBytesSinceLastGlobalGC +=
         SIZE_TO_BYTES (TO_INT (OOP_TO_OBJ (oop)->objSize));
     }

It is a very good start though, thanks!

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: Big objects and GC

MrGwen
On 05/11/2011 11:45 AM, Paolo Bonzini wrote:

>
>> Here is the patch, all the tests are green ;-)
>
> Nice patch!
>
> I have a few comments.
>
>> diff --git a/libgst/oop.c b/libgst/oop.c
>> index ad264d9..018ec95 100644
>> --- a/libgst/oop.c
>> +++ b/libgst/oop.c
>> @@ -66,6 +66,7 @@
>>
>> /* Define this flag to turn on debugging code for OOP table management */
>> /* #define GC_DEBUGGING */
>> +#define GC_DEBUGGING
>
> Spurious change.
>
>> @@ -701,7 +702,7 @@ _gst_make_oop_fixed (OOP oop)
>> {
>> gst_object newObj;
>> int size;
>> - if (oop->flags & F_FIXED)
>> + if ((oop->flags & F_FIXED) || (oop->flags & F_LARGE))
>> return;
>
> Can you try representing large objects with F_FIXED but not F_OLD? At
> the same time, makeFixed would not cause objects to become old, which is
> useful for objects passed to C.
>
> Apparently, I even had some code to handle this in _gst_tenure_oop:
>
> if (oop->flags & F_OLD)
> return;
>
> if (!(oop->flags & F_FIXED))
> {
> int size = SIZE_TO_BYTES (TO_INT(oop->object->objSize));
> newObj = (gst_object) _gst_mem_alloc (_gst_mem.old, size);
> if (!newObj)
> abort ();
>
> memcpy (newObj, oop->object, size);
> _gst_mem.numOldOOPs++;
>
> oop->object = newObj;
> }
>
> where the increment of numOldOOPs should be moved outside the if.
>
>> @@ -780,6 +781,38 @@ _gst_alloc_obj (size_t size,
>> }
>>
>> gst_object
>> +_gst_alloc_large_obj (size_t size,
>> + OOP *p_oop)
>> +{
>> + gst_object p_instance;
>> +
>> + size = ROUNDED_BYTES (size);
>> +
>> + /* If the object is big enough, we put it directly in fixedspace. */
>> + p_instance = (gst_object) _gst_mem_alloc (_gst_mem.fixed, size);
>> + if COMMON (p_instance)
>> + goto ok;
>> +
>> + _gst_global_gc (size);
>> + p_instance = (gst_object) _gst_mem_alloc (_gst_mem.fixed, size);
>> + if COMMON (p_instance)
>> + goto ok;
>> +
>> + _gst_compact (0);
>
> Compacting is not going to be helpful, for fixed objects.
>
>> @@ -1452,7 +1485,7 @@ _gst_sweep_oop (OOP oop)
>> _gst_make_oop_non_weak (oop);
>>
>> /* Free unreachable oldspace objects. */
>> - if UNCOMMON (oop->flags & F_FIXED)
>> + if UNCOMMON ((oop->flags & F_FIXED) || (oop->flags & F_LARGE))
>> {
>> _gst_mem.numOldOOPs--;
>
> This makes stats wrong. With the idea above, the code would be like:
>
> if ((oop->flags & F_LOADED) == 0)
> {
> if UNCOMMON (oop->flags & F_FIXED)
> _gst_mem_free (_gst_mem.fixed, oop->object);
> else if UNCOMMON (oop->flags & F_OLD)
> _gst_mem_free (_gst_mem.old, oop->object);
> }
>
> if UNCOMMON (oop->flags & F_OLD)
> {
> _gst_mem.numOldOOPs--;
> stats.reclaimedOldSpaceBytesSinceLastGlobalGC +=
> SIZE_TO_BYTES (TO_INT (OOP_TO_OBJ (oop)->objSize));
> }
>
> It is a very good start though, thanks!
>
> Paolo
Here is the new patch, I've removed like you've said the F_LARGE and
make the F_FIXED working with both new and old generation the spurious
change is removed ;-). I've included your idea just swap the two if
because the memory can be freed before the size is accessed.

Gwen


_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

large-oop.patch (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Big objects and GC

Paolo Bonzini-2
On 05/11/2011 01:47 PM, Gwenael Casaccio wrote:
>
> Here is the new patch, I've removed like you've said the F_LARGE and
> make the F_FIXED working with both new and old generation the spurious
> change is removed ;-). I've included your idea just swap the two if
> because the memory can be freed before the size is accessed.

Committed, thanks!

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk