Re: [Vm-dev] A trick to speedup #become: (but not becomeForward:)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] A trick to speedup #become: (but not becomeForward:)

Chris Muller-3
I'm not sure how easy it would be to get the physical size of the
objects which weren't retrieved.  As you said, encoding it in the
referencing oid might be the only way - which would require rewriting
of the OidMap and upgrade of legacy repositories.  :(

I understand why it won't work with becomeForward: - because that
would be creating two copies of the object.  But Magma needs to
becomeForward:, rather than become:, its proxies... (pause to remember
for sure why...).  I think because of proxies to Symbol selectors -
you can't become: any object to a Symbol selector or else their
CompiledMethod literals would refer to the Proxy...  So that would be
another hurdle to overcome to succeed with your idea.

Just so you know, I _did_ implement your other workaround idea - where
reified proxies are "saved up" into a OrderedCollection which is then
bulk-becomed only once every 30 seconds.  I didn't know if you saw it
in the last release (Magma 1.2) - it was a great performance
improvement!  Your creative ideas are really helping Magma.

Thanks,
  Chris


On Sat, Jul 30, 2011 at 8:12 PM, Igor Stasenko <[hidden email]> wrote:

>
> An interesting observation:
>
> if two oops which we need to swap: oop1 and oop2 taking exactly same
> space in memory,
> then all we need to do is to swap their memory contents, instead of
> scanning heap and updating pointers!
>
> This makes a become operation extremely cheap for such pairs!
> As for objects of different sizes, we could still use slow algorithm.
>
> But knowing that if two objects with same size will be swapped much
> faster, a developers could adapt their algorithms to exploit this
> feature
> (use fixed-size objects instead of variable-sized ones etc).
>
> Magma using proxies, which then #becomeForward: to real objects when reified.
> This is very costly operation and main reason why it so slow on
> loading objects from server.
> To speed thing up, a special trick can be used:
>  - each time server sends an object ID (instead of real object) it
> could also send its size in bytes (or even more clever - you can
> encode object size in its ID ;) ).
> Then a client will create a proxy for given object and will try to
> match the size, if its possible. (nobody said that proxy can't be
> variable-sized, right?)
>
> Then once client requesting to reify given proxy with real object from
> database, VM will just overwrite a proxy's memory contents with real
> object.
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] A trick to speedup #become: (but not becomeForward:)

Igor Stasenko
On 1 August 2011 00:02, Chris Muller <[hidden email]> wrote:
> I'm not sure how easy it would be to get the physical size of the
> objects which weren't retrieved.  As you said, encoding it in the
> referencing oid might be the only way - which would require rewriting
> of the OidMap and upgrade of legacy repositories.  :(
>

Yes. And if you remember i gave you the code with another oid mapping
implementation,
which avoids using big-integers and a bit more simpler (i hope).
So maybe it is time to check what could be done there?

I don't having time for it right now, hoping that you have it.

> I understand why it won't work with becomeForward: - because that
> would be creating two copies of the object.  But Magma needs to
> becomeForward:, rather than become:, its proxies... (pause to remember
> for sure why...).  I think because of proxies to Symbol selectors -
> you can't become: any object to a Symbol selector or else their
> CompiledMethod literals would refer to the Proxy...  So that would be
> another hurdle to overcome to succeed with your idea.
>

Symbols require special handling anyways: you must check/intern them
once reified.
But for majority of other objects, using these prims could mean a vast
difference.

> Just so you know, I _did_ implement your other workaround idea - where
> reified proxies are "saved up" into a OrderedCollection which is then
> bulk-becomed only once every 30 seconds.  I didn't know if you saw it
> in the last release (Magma 1.2) - it was a great performance
> improvement!  Your creative ideas are really helping Magma.
>

:)

> Thanks,
>  Chris
>
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] A trick to speedup #become: (but not becomeForward:)

Chris Muller-3
> Yes. And if you remember i gave you the code with another oid mapping
> implementation,
> which avoids using big-integers and a bit more simpler (i hope).

The current Magma relies on the ".idx" file to lookup locations of
objects in the objects.dat file.  Since oids are allocated
consecutively, the file only grows as much as the db allocates new
oids.

Do you happen to remember; whether you had a new scheme for looking up
location of objects in the objects.dat file, since I assume you can no
longer allocate oids consecutively?
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma
Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] A trick to speedup #become: (but not becomeForward:)

Igor Stasenko
On 1 August 2011 21:30, Chris Muller <[hidden email]> wrote:

>> Yes. And if you remember i gave you the code with another oid mapping
>> implementation,
>> which avoids using big-integers and a bit more simpler (i hope).
>
> The current Magma relies on the ".idx" file to lookup locations of
> objects in the objects.dat file.  Since oids are allocated
> consecutively, the file only grows as much as the db allocates new
> oids.
>
> Do you happen to remember; whether you had a new scheme for looking up
> location of objects in the objects.dat file, since I assume you can no
> longer allocate oids consecutively?
>

no. i didn't changed anything like that. you can allocate oid consecutively.
In my changes i just rearranged a id space between special object ,
like ints, big ints , floats and chars,
to try to avoid jumping in large integer range for object id if its possible.



--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma