Request for feedback: mirrors

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Request for feedback: mirrors

Colin Putney-3
Hi all,

I've just uploaded a package to the Inbox for review by the community:


It's a simple first step at creating a mirror API for Squeak. I've only implemented ObjectMirror and ObjectVmMirror, which provide high- and low-level reflection on an object. This implementation has a twist, however: it sends no messages to the object it's reflecting. That allows us to reflect on network proxies, ORM stubs, mock objects and the like without triggering any state changes. 

As a proof of the concept, I've implemented a rudimentary non-invasive MirrorInspector. The main thing that's missing is non-invasive printing, which would take a bit of work to implement, but isn't necessary for a demo. To see it in action do try the following in a workspace.

WARNING: Read the comments carefully, and don't do this in an image you care about!

"This creates a very dangerous object"
Object subclass: #Nuke
instanceVariableNames: 'one two three'
classVariableNames: ''
poolDictionaries: ''
category: 'Test'.
class := Smalltalk at: #Nuke.
class superclass: nil.
inst := class basicNew.

"This is harmless"
MirrorInspector inspect: inst.

"This will crash your image"
inst yourself.

The implementation uses primitive 188 to directly execute CompiledMethods with the reflected object as the receiver, thus giving us access to the object's state without sending it a message. 

So, what do you think? 

Colin


Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Frank Shearar-3
On 22 May 2012 08:07, Colin Putney <[hidden email]> wrote:

> Hi all,
>
> I've just uploaded a package to the Inbox for review by the community:
>
> http://source.squeak.org/inbox/Mirrors-cwp.2.mcz
>
> It's a simple first step at creating a mirror API for Squeak. I've only
> implemented ObjectMirror and ObjectVmMirror, which provide high- and
> low-level reflection on an object. This implementation has a twist, however:
> it sends no messages to the object it's reflecting. That allows us to
> reflect on network proxies, ORM stubs, mock objects and the like without
> triggering any state changes.
>
> As a proof of the concept, I've implemented a rudimentary non-invasive
> MirrorInspector. The main thing that's missing is non-invasive printing,
> which would take a bit of work to implement, but isn't necessary for a demo.
> To see it in action do try the following in a workspace.
>
> WARNING: Read the comments carefully, and don't do this in an image you care
> about!
>
> "This creates a very dangerous object"
> Object subclass: #Nuke
> instanceVariableNames: 'one two three'
> classVariableNames: ''
> poolDictionaries: ''
> category: 'Test'.
> class := Smalltalk at: #Nuke.
> class superclass: nil.
> inst := class basicNew.
>
> "This is harmless"
> MirrorInspector inspect: inst.
>
> "This will crash your image"
> inst yourself.
>
> The implementation uses primitive 188 to directly execute CompiledMethods
> with the reflected object as the receiver, thus giving us access to the
> object's state without sending it a message.
>
> So, what do you think?

MessageShunt can't load because it has a nil superclass, causing an
MNU for #allowsSubInstVars in ClassBuilder >>
#validateInstVars:from:forSuper: because the last arg - newSuper - is
nil.

frank

> Colin

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Colin Putney-3

On 2012-05-22, at 3:09 AM, Frank Shearar wrote:

MessageShunt can't load because it has a nil superclass, causing an
MNU for #allowsSubInstVars in ClassBuilder >>
#validateInstVars:from:forSuper: because the last arg - newSuper - is
nil.

Drat. That turns out to more of an issue than I thought. I've uploaded a new version that has ProtoObject as a superclass. 


Colin


Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Frank Shearar-3
On 22 May 2012 16:06, Colin Putney <[hidden email]> wrote:

>
> On 2012-05-22, at 3:09 AM, Frank Shearar wrote:
>
> MessageShunt can't load because it has a nil superclass, causing an
> MNU for #allowsSubInstVars in ClassBuilder >>
> #validateInstVars:from:forSuper: because the last arg - newSuper - is
> nil.
>
>
> Drat. That turns out to more of an issue than I thought. I've uploaded a new
> version that has ProtoObject as a superclass.
>
> http://source.squeak.org/inbox/Mirrors-cwp.3.mcz

Ah yes, that works, and the tests pass. Now to actually read the code :) Thanks!

frank

> Colin
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Eliot Miranda-2
In reply to this post by Colin Putney-3


On Tue, May 22, 2012 at 12:07 AM, Colin Putney <[hidden email]> wrote:
Hi all,

I've just uploaded a package to the Inbox for review by the community:


It's a simple first step at creating a mirror API for Squeak. I've only implemented ObjectMirror and ObjectVmMirror, which provide high- and low-level reflection on an object. This implementation has a twist, however: it sends no messages to the object it's reflecting. That allows us to reflect on network proxies, ORM stubs, mock objects and the like without triggering any state changes. 

As a proof of the concept, I've implemented a rudimentary non-invasive MirrorInspector. The main thing that's missing is non-invasive printing, which would take a bit of work to implement, but isn't necessary for a demo. To see it in action do try the following in a workspace.

WARNING: Read the comments carefully, and don't do this in an image you care about!

"This creates a very dangerous object"
Object subclass: #Nuke
instanceVariableNames: 'one two three'
classVariableNames: ''
poolDictionaries: ''
category: 'Test'.
class := Smalltalk at: #Nuke.
class superclass: nil.
inst := class basicNew.

"This is harmless"
MirrorInspector inspect: inst.

"This will crash your image"
inst yourself.

The implementation uses primitive 188 to directly execute CompiledMethods with the reflected object as the receiver, thus giving us access to the object's state without sending it a message. 

In part you can use the light-weigth mirror methods in ContextPart, these take the object operated on as an argument.  Non-invasive printing could be done via execution simulation.  I've already modified the Squeak debugger in Qwaq/Teleplace images to use these light-weight mirror primitives so that messages are not sent to receivers when simulating execution in the debugger (i.e. when doing send instead of step).  If the mirror primitives now work on the interpreter I can fold this into trunk.  David, do the mirror primitive tests pass on the interpreter?

Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.
 

So, what do you think? 

Colin






--
best,
Eliot



Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Colin Putney-3
On Tue, May 22, 2012 at 3:24 PM, Eliot Miranda <[hidden email]> wrote:

> In part you can use the light-weigth mirror methods in ContextPart, these
> take the object operated on as an argument.  Non-invasive printing could be
> done via execution simulation.  I've already modified the Squeak debugger in
> Qwaq/Teleplace images to use these light-weight mirror primitives so that
> messages are not sent to receivers when simulating execution in the debugger
> (i.e. when doing send instead of step).  If the mirror primitives now work
> on the interpreter I can fold this into trunk.  David, do the mirror
> primitive tests pass on the interpreter?

Yeah, I looked at those, but they seem unnecessary given the existence
of primitive 188, (i.e., CompiledMethod
class>>receiver:withArguments:executeMethod:). With that one primitive
we can do anything we need, with all the logic in the image, instead
of the VM.

Is there some reason you prefer separate primitives for all these
operations? You called them light-weight—what do you mean by that?

> Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.

Yes!

I guess we have two separate issues here, which I conflated in my
first post. On the one hand, there's non-invasive tool support. That
would be good, and it can be implemented in various ways. Your
debugger changes sound like a good first step in that direction. We
could also implement an inspector based on the light-weight mirror
primitives.

On the other hand, there's the question of a mirror API. That's one of
the things I really like about the Self-Strongtalk-Newspeak-Dart
thread of language design, and hey, it would be great to have that in
Squeak as well. (We stole Morphic from Self, why not mirrors?)

If we did decided to move to a mirror-based API, then non-invasive
mirrors seems like the way to go, and this approach based on prim 188
seems like a good way to implement that. This experiment makes me
think it's feasible. Laying aside the (very real) issues of how to get
from here to there, is this a direction worth exploring, or is
Smalltalk-80-style reflection good enough?

Colin

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Igor Stasenko
On 23 May 2012 03:59, Colin Putney <[hidden email]> wrote:

> On Tue, May 22, 2012 at 3:24 PM, Eliot Miranda <[hidden email]> wrote:
>
>> In part you can use the light-weigth mirror methods in ContextPart, these
>> take the object operated on as an argument.  Non-invasive printing could be
>> done via execution simulation.  I've already modified the Squeak debugger in
>> Qwaq/Teleplace images to use these light-weight mirror primitives so that
>> messages are not sent to receivers when simulating execution in the debugger
>> (i.e. when doing send instead of step).  If the mirror primitives now work
>> on the interpreter I can fold this into trunk.  David, do the mirror
>> primitive tests pass on the interpreter?
>
> Yeah, I looked at those, but they seem unnecessary given the existence
> of primitive 188, (i.e., CompiledMethod
> class>>receiver:withArguments:executeMethod:). With that one primitive
> we can do anything we need, with all the logic in the image, instead
> of the VM.
>
> Is there some reason you prefer separate primitives for all these
> operations? You called them light-weight—what do you mean by that?
>
>> Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.
>
> Yes!
>
> I guess we have two separate issues here, which I conflated in my
> first post. On the one hand, there's non-invasive tool support. That
> would be good, and it can be implemented in various ways. Your
> debugger changes sound like a good first step in that direction. We
> could also implement an inspector based on the light-weight mirror
> primitives.
>
> On the other hand, there's the question of a mirror API. That's one of
> the things I really like about the Self-Strongtalk-Newspeak-Dart
> thread of language design, and hey, it would be great to have that in
> Squeak as well. (We stole Morphic from Self, why not mirrors?)
>
> If we did decided to move to a mirror-based API, then non-invasive
> mirrors seems like the way to go, and this approach based on prim 188
> seems like a good way to implement that. This experiment makes me
> think it's feasible. Laying aside the (very real) issues of how to get
> from here to there, is this a direction worth exploring, or is
> Smalltalk-80-style reflection good enough?
>
> Colin
>

Nice trick with 188 prim..
yes, actually using this prim you can attach any behavior to any
object, without it's permission..
so why not mirrors ;)

I sure this looks not very good in the eyes of pro-security guys, but
as to me it is fine.
Maybe mirror primitives, mentioned by Eliot is kind of providing the
same but in more conservative/secure way and probably a bit faster.
But unless we ban primitive 188, i don't see why we need something
more/else for mirrors.

And yes, i vote for mirrors.

--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

David T. Lewis
In reply to this post by Eliot Miranda-2
On Tue, May 22, 2012 at 03:24:34PM -0700, Eliot Miranda wrote:

> On Tue, May 22, 2012 at 12:07 AM, Colin Putney <[hidden email]> wrote:
>
> > Hi all,
> >
> > I've just uploaded a package to the Inbox for review by the community:
> >
> > http://source.squeak.org/inbox/Mirrors-cwp.2.mcz
> >
> > It's a simple first step at creating a mirror API for Squeak. I've only
> > implemented ObjectMirror and ObjectVmMirror, which provide high- and
> > low-level reflection on an object. This implementation has a twist,
> > however: it sends no messages to the object it's reflecting. That allows us
> > to reflect on network proxies, ORM stubs, mock objects and the like without
> > triggering any state changes.
> >
> > As a proof of the concept, I've implemented a rudimentary non-invasive
> > MirrorInspector. The main thing that's missing is non-invasive printing,
> > which would take a bit of work to implement, but isn't necessary for a
> > demo. To see it in action do try the following in a workspace.
> >
> > WARNING: Read the comments carefully, and don't do this in an image you
> > care about!
> >
> > "This creates a very dangerous object"
> > Object subclass: #Nuke
> > instanceVariableNames: 'one two three'
> > classVariableNames: ''
> > poolDictionaries: ''
> > category: 'Test'.
> > class := Smalltalk at: #Nuke.
> > class superclass: nil.
> > inst := class basicNew.
> >
> > "This is harmless"
> > MirrorInspector inspect: inst.
> >
> > "This will crash your image"
> > inst yourself.
> >
> > The implementation uses primitive 188 to directly execute CompiledMethods
> > with the reflected object as the receiver, thus giving us access to the
> > object's state without sending it a message.
> >
>
> In part you can use the light-weigth mirror methods in ContextPart, these
> take the object operated on as an argument.  Non-invasive printing could be
> done via execution simulation.  I've already modified the Squeak debugger
> in Qwaq/Teleplace images to use these light-weight mirror primitives so
> that messages are not sent to receivers when simulating execution in the
> debugger (i.e. when doing send instead of step).  If the mirror primitives
> now work on the interpreter I can fold this into trunk.  David, do the
> mirror primitive tests pass on the interpreter?

All mirror primitive tests pass on the interpreter VM if built with the
latest VMMaker and SVN sources. However, the mirror primitive support is
not yet present in any officially released standard VM.

The updates for mirror primitive support were added in VMMaker-dtl.262 and
VMMaker-dtl.261. Note the update comment in VMMaker-dtl.262, as the
implementation differs from Cog and a code review would be welcome:

    Name: VMMaker-dtl.262
    Author: dtl
    Time: 5 January 2012, 12:41:29.745 am
   
    VMMaker 4.7.19
   
    Reference Mantis 7429: Add Mirror Primitives to the VM
   
    Update primitivePerformInSuperclass to support ContextPart>>object:perform:withArguments:inClass:
   
    Implementation differs from that of oscog in that the original primitivePerformAt:
    is retained unmodified, and the necessary stack adjustments are done in
    primitivePerformInSuperclass for the special case of argumentCount 4
    (mirror primitive call) rather than 3. The oscog approach may be prefered
    (not least for its clearer method naming), but making the change in
    primitivePerformInSuperclass is low risk and more easily implemented by
    a Sunday Squeaker.
   
    All MirrorPrimitiveTests pass.

Dave


Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Eliot Miranda-2
In reply to this post by Colin Putney-3


On Tue, May 22, 2012 at 6:59 PM, Colin Putney <[hidden email]> wrote:
On Tue, May 22, 2012 at 3:24 PM, Eliot Miranda <[hidden email]> wrote:

> In part you can use the light-weigth mirror methods in ContextPart, these
> take the object operated on as an argument.  Non-invasive printing could be
> done via execution simulation.  I've already modified the Squeak debugger in
> Qwaq/Teleplace images to use these light-weight mirror primitives so that
> messages are not sent to receivers when simulating execution in the debugger
> (i.e. when doing send instead of step).  If the mirror primitives now work
> on the interpreter I can fold this into trunk.  David, do the mirror
> primitive tests pass on the interpreter?

Yeah, I looked at those, but they seem unnecessary given the existence
of primitive 188, (i.e., CompiledMethod
class>>receiver:withArguments:executeMethod:). With that one primitive
we can do anything we need, with all the logic in the image, instead
of the VM.

Is there some reason you prefer separate primitives for all these
operations? You called them light-weight—what do you mean by that?

> Doing this is also a good idea in general since it allows the debugger to correctly debug proxies also.

Yes!

I guess we have two separate issues here, which I conflated in my
first post. On the one hand, there's non-invasive tool support. That
would be good, and it can be implemented in various ways. Your
debugger changes sound like a good first step in that direction. We
could also implement an inspector based on the light-weight mirror
primitives.

On the other hand, there's the question of a mirror API. That's one of
the things I really like about the Self-Strongtalk-Newspeak-Dart
thread of language design, and hey, it would be great to have that in
Squeak as well. (We stole Morphic from Self, why not mirrors?)

If we did decided to move to a mirror-based API, then non-invasive
mirrors seems like the way to go, and this approach based on prim 188
seems like a good way to implement that. This experiment makes me
think it's feasible. Laying aside the (very real) issues of how to get
from here to there, is this a direction worth exploring, or is
Smalltalk-80-style reflection good enough?

To not address your core questions, use of 188 is I think clunky.  Instead, implementing the mirror primitives in the mirrors seems simple and efficient.  Putting the mirror primitives in ContextPart is insecure, since anyone can to thisContext object: foo instVarAt: n.  But fixing this means a major design overhaul of the system re debugging and mirrors, so it can stand for now.  (After all anyone can do instVarAt: now, so we're a ways away from a secure Newspeak-style sandbox).


Colin




--
best,
Eliot



Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Colin Putney-3
On Wed, May 23, 2012 at 11:54 AM, Eliot Miranda <[hidden email]> wrote:

> To not address your core questions, use of 188 is I think clunky.  Instead,
> implementing the mirror primitives in the mirrors seems simple and
> efficient.

You find it clunky because you can't just call the primitive, you have
to do a manual lookup to find the appropriate CompiledMethod first?
Fair enough. I don't think it's an issue with mirrors, because the
mirror encapsulates that clunky lookup behaviour, and lets us make us
of it through a clean and simple protocol. I prefer a smaller, more
general VM/image interface, with as much of the logic as possible in
the image.

But 188 does have the following practical advantages:

- it's present in older VMs that are already out there
- in the future we'll be able to make changes at the image level,
rather than having to ship new VMs

The other thread on your debugger improvements would be moot if they
had been based on 188 instead of custom primitives.

> Putting the mirror primitives in ContextPart is insecure, since
> anyone can to thisContext object: foo instVarAt: n.  But fixing this means a
> major design overhaul of the system re debugging and mirrors, so it can
> stand for now.  (After all anyone can do instVarAt: now, so we're a ways
> away from a secure Newspeak-style sandbox).

Agreed.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Igor Stasenko
On 24 May 2012 00:29, Colin Putney <[hidden email]> wrote:
> On Wed, May 23, 2012 at 11:54 AM, Eliot Miranda <[hidden email]> wrote:
>
>> To not address your core questions, use of 188 is I think clunky.  Instead,
>> implementing the mirror primitives in the mirrors seems simple and
>> efficient.
>
> You find it clunky because you can't just call the primitive, you have
> to do a manual lookup to find the appropriate CompiledMethod first?

Sorry, but why you need a lookup? Do i miss something? I didn't
checked your implementation.

For instance, Pharo no longer uses shortcut for #class message,
sending a class to an object is a normal message send with usual
lookup procedure.
Therefore , sending #class to an object does not guarantees that it
will answer its class (especially,
if it can be simply not implemented).

Now i don't see where you need to do a manual lookup in following:

(Mirror on: arbitraryObject) subjectClass

Mirror>>subjectClass
   ^ CompiledMethod receiver: subject withArguments: argArray
executeMethod: answerSubjectClass

(here the subject is an object which mirror reflects).


> Fair enough. I don't think it's an issue with mirrors, because the
> mirror encapsulates that clunky lookup behaviour, and lets us make us
> of it through a clean and simple protocol. I prefer a smaller, more
> general VM/image interface, with as much of the logic as possible in
> the image.
>
> But 188 does have the following practical advantages:
>
> - it's present in older VMs that are already out there
> - in the future we'll be able to make changes at the image level,
> rather than having to ship new VMs
>
> The other thread on your debugger improvements would be moot if they
> had been based on 188 instead of custom primitives.
>

Indeed, i think 188 having a great potential if used by good hands.


>> Putting the mirror primitives in ContextPart is insecure, since
>> anyone can to thisContext object: foo instVarAt: n.  But fixing this means a
>> major design overhaul of the system re debugging and mirrors, so it can
>> stand for now.  (After all anyone can do instVarAt: now, so we're a ways
>> away from a secure Newspeak-style sandbox).
>
> Agreed.
>
> Colin
>

--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Igor Stasenko
ok, i checked your implementation and now i see where you doing lookup

receiver: anObject perform: aSelector arguments: anArray
        | cm |
        cm := self compiledMethodAt: aSelector ifAbsent:
                [^ anObject doesNotUnderstand: (Message selector: aSelector
arguments: anArray)].
        ^ self receiver: anObject withArguments: anArray executeMethod: cm

But you can avoid it, consider a following:

ObjectVmMirror>>fixedAt: anInteger
        ^ Reflection
                receiver: object
                perform: #fixedAt:
                arguments: {anInteger}

you can rewrite it as:

ObjectVmMirror>>fixedAt: anInteger
        ^ self execute: GetFixedSlotMethod
                arguments: {anInteger}

where GetFixedSlotMethod is a class variable holding an appropriate
CompiledMethod instance,
which you can initialize just once at class initialization (and
reinitialize if it changes).

and #execute:arguments: is method which using prim 188 for executing
given method
with object as receiver.

--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Colin Putney-3
On Wed, May 23, 2012 at 5:13 PM, Igor Stasenko <[hidden email]> wrote:
> ok, i checked your implementation and now i see where you doing lookup

[snip]

> ObjectVmMirror>>fixedAt: anInteger
>        ^ self execute: GetFixedSlotMethod
>                arguments: {anInteger}
>
> where GetFixedSlotMethod is a class variable holding an appropriate
> CompiledMethod instance,
> which you can initialize just once at class initialization (and
> reinitialize if it changes).
>
> and #execute:arguments: is method which using prim 188 for executing
> given method
> with object as receiver.

Sure, that would be cleaner and a bit faster. I'll make that change, thanks.

I don't think it'll please Eliot though - we still have to get the
CompiledMethod from *somewhere* even if it's a class variable.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: Request for feedback: mirrors

Igor Stasenko
On 24 May 2012 02:34, Colin Putney <[hidden email]> wrote:

> On Wed, May 23, 2012 at 5:13 PM, Igor Stasenko <[hidden email]> wrote:
>> ok, i checked your implementation and now i see where you doing lookup
>
> [snip]
>
>> ObjectVmMirror>>fixedAt: anInteger
>>        ^ self execute: GetFixedSlotMethod
>>                arguments: {anInteger}
>>
>> where GetFixedSlotMethod is a class variable holding an appropriate
>> CompiledMethod instance,
>> which you can initialize just once at class initialization (and
>> reinitialize if it changes).
>>
>> and #execute:arguments: is method which using prim 188 for executing
>> given method
>> with object as receiver.
>
> Sure, that would be cleaner and a bit faster. I'll make that change, thanks.
>
> I don't think it'll please Eliot though - we still have to get the
> CompiledMethod from *somewhere* even if it's a class variable.
>

i like the concept. it is quite elegant, to my look:
 - things like GetFixedSlotMethod & others is actually a VM contracts
which language side must know about
to properly reflect the objects without sending any message to them
(same as VM does).

So you end up with things like

GetObjectBehaviorMethod
GetObjectVarSizeMethod

GetBehaviorSuperclassMethod
GetBehaviorMDMethod
GetBehaviorFormatMethod
etc


i would even create a separate class for that.. which will hold all
those contracts.
So it can be useful for educational purposes too.

> Colin


--
Best regards,
Igor Stasenko.