Blocks that don't leak memory

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Blocks that don't leak memory

webwarrior
This post was updated on .
BlockClosure objects hold reference to object that created them via outerContext receiver. This can cause memory leaks when moving blocks around/deepcopying/serializing.

Is there a way to get "lightweight" block that has empty outerContext (like when executing code in Playground) or is there another class for such "lightweight" blocks somewhere?

Obviously I'm talking about blocks that don't reference variables in enclosing scope and don't do nonlocal return.
Reply | Threaded
Open this post in threaded view
|

Re: Block that doesn't leak memory

Clément Béra
There is no way of creating such blocks currently.

In fact only non local returns and debugging are problems as remote variables are accessed through an indirection. A lightweight block (to reuse your terms) with no outer context but accesses to remote temporary variables would work fine. 

The common terminology is as follow:
- Blocks are called clean blocks if there are no remote temporary variable access and no outer context references
- They're called copying block if there are remote temporary access but still no outer context
- They're called full blocks if they have an outer context (One can sometimes distinguish also full and fullcopying blocks, but it does not matter)

Pharo supports only full blocks, on the contrary to other smalltalk such as VW. I did a working implementation of clean blocks, as you describe, but I didn't integrate it because I didn't like loosing part of the debugging features. That was years ago I am not sure I can find the code anymore. I can explain the design though as it's pretty straightforward.

To implement such blocks, you need to share a fake outerContext between all the clean blocks created in the same method. You can't have nil as an outer context else the VM behaves badly. Instead of the block creation bytecode, you can use the pushLiteral instruction to push the BlockClosure created at compilation time and stored in the literal frame of the method. You can put the bytecode of clean blocks at the end of the bytecode zone of the method, followed by a returnTop instruction at the end (without the return instruction the JIT is confused on where the end of the bytecoded method is). Let me know if you want to implement that I can help. I still think it should be an external library and not in the base Pharo though for debugging purpose.

To solve your problem of moving blocks around, it may also be possible to use ephemerons to remove the dependency. Maybe some ephemeron experts can help you there.

2015-12-26 16:47 GMT+01:00 webwarrior <[hidden email]>:
BlockClosure objects hold reference to object that created them via
outerContext receiver. This can cause memory leaks when moving blocks
around/deepcopying/serializing.

Is there a way to get "lightweight" block that has empty outerContext (like
when executing code in Playground) or is there another class for such
"lightweight" blocks somewhere?

Obviously I'm talking about blocks that don't reference variables in
enclosing scope and don't do nonlocal return.



--
View this message in context: http://forum.world.st/Block-that-doesn-t-leak-memory-tp4868529.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.


Reply | Threaded
Open this post in threaded view
|

Re: Block that doesn't leak memory

Mariano Martinez Peck
This is halt related, but you can see how Fuel checks wether the closure is "clean" and if true, it serializes a pruned version of it so that to avoid a whole stack. See BlockClosure >> fuelAccept:

fuelAccept: aGeneralMapper

^ self shouldBeSubstitutedByCleanCopy
"The 'onRecursionDo:' is just to avoid an infinitive loop for the substitution. The cleanCopy MUST be a clean copy so it can be serialized normally"
ifTrue: [ aGeneralMapper visitSubstitution: self by: self cleanCopy onRecursionDo: [ aGeneralMapper visitVariableObject: self ]  ]
ifFalse: [ aGeneralMapper visitVariableObject: self ]


Cheers,

On Sun, Dec 27, 2015 at 1:31 PM, Clément Bera <[hidden email]> wrote:
There is no way of creating such blocks currently.

In fact only non local returns and debugging are problems as remote variables are accessed through an indirection. A lightweight block (to reuse your terms) with no outer context but accesses to remote temporary variables would work fine. 

The common terminology is as follow:
- Blocks are called clean blocks if there are no remote temporary variable access and no outer context references
- They're called copying block if there are remote temporary access but still no outer context
- They're called full blocks if they have an outer context (One can sometimes distinguish also full and fullcopying blocks, but it does not matter)

Pharo supports only full blocks, on the contrary to other smalltalk such as VW. I did a working implementation of clean blocks, as you describe, but I didn't integrate it because I didn't like loosing part of the debugging features. That was years ago I am not sure I can find the code anymore. I can explain the design though as it's pretty straightforward.

To implement such blocks, you need to share a fake outerContext between all the clean blocks created in the same method. You can't have nil as an outer context else the VM behaves badly. Instead of the block creation bytecode, you can use the pushLiteral instruction to push the BlockClosure created at compilation time and stored in the literal frame of the method. You can put the bytecode of clean blocks at the end of the bytecode zone of the method, followed by a returnTop instruction at the end (without the return instruction the JIT is confused on where the end of the bytecoded method is). Let me know if you want to implement that I can help. I still think it should be an external library and not in the base Pharo though for debugging purpose.

To solve your problem of moving blocks around, it may also be possible to use ephemerons to remove the dependency. Maybe some ephemeron experts can help you there.

2015-12-26 16:47 GMT+01:00 webwarrior <[hidden email]>:
BlockClosure objects hold reference to object that created them via
outerContext receiver. This can cause memory leaks when moving blocks
around/deepcopying/serializing.

Is there a way to get "lightweight" block that has empty outerContext (like
when executing code in Playground) or is there another class for such
"lightweight" blocks somewhere?

Obviously I'm talking about blocks that don't reference variables in
enclosing scope and don't do nonlocal return.



--
View this message in context: http://forum.world.st/Block-that-doesn-t-leak-memory-tp4868529.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.





--
Reply | Threaded
Open this post in threaded view
|

Re: Block that doesn't leak memory

webwarrior
I've looked at implementation of cleanCopy. It removes sender, but my
problem is with receiver.

However I wrote similar set of methods that also set receiver to nil.
Seems to work fine, in debugger such blocks are seen as defined in DoIt.


On 29.12.2015 3:42, Mariano Martinez Peck [via Smalltalk] wrote:

> This is halt related, but you can see how Fuel checks wether the closure
> is "clean" and if true, it serializes a pruned version of it so that to
> avoid a whole stack. See BlockClosure >> fuelAccept:
>
> fuelAccept: aGeneralMapper
>
> ^ self *shouldBeSubstitutedByCleanCopy*
> "The 'onRecursionDo:' is just to avoid an infinitive loop for the
> substitution. The cleanCopy MUST be a clean copy so it can be serialized
> normally"
> ifTrue: [ aGeneralMapper visitSubstitution: self by: *self cleanCopy
> *onRecursionDo: [ aGeneralMapper visitVariableObject: self ]  ]
> ifFalse: [ aGeneralMapper visitVariableObject: self ]