test crashing the cog vm

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
50 messages Options
123
Reply | Threaded
Open this post in threaded view
|

test crashing the cog vm

Tudor Girba
Hi,

Alex recently wrote a test in Moose that seems to crash the Cog VM at least on Mac.

How to reproduce:

- download the following image
http://dl.dropbox.com/u/18323746/Tmp/moose-crashing-cog-jit.zip

- execute the following code in the workspace (already provided in the image)
        cls := Class new superclass: MooseElement; yourself.
        cls compileSilently: 'mooseName   ^ 1/0'.
        element := cls new.

- I used all of the followings and they all crashed:
https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/Cog%20Mac%20Cocoa/4/artifact/cog/build/Cog.zip
https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/StackVM%20Mac%20Cocoa/2/artifact/cog/build/StackVM.zip
http://www.mirandabanda.org/files/Cog/VM/VM.r2370/Cog.app.tgz
http://www.mirandabanda.org/files/Cog/VM/VM.r2361/Cog.app.tgz


The strange thing is that the crash only happens when we subclass MooseElement, but not another class.

Could someone take a look?

Cheers,
Doru


--
www.tudorgirba.com

"Every thing should have the right to be different."




Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Toon Verwaest-2
This does crash whenever you subclass a class which has instance
variables and you try to access those instance variables. The problem is
that you don't properly initialize your class, leaving you with a Class
that has a wrong format. For example:

cls := Class new superclass: Class; yourself.
cls format

returns 2. 2 basically means it's an object with pointers but with 0
instance variables. If you instantiate the 'cls' I just made it also
crashes. Why? Well, class has an initialize method that is compiled to
write to the fields of the new instance. It puts an empty method
dictionary into the class you create as an instance of my cls. This
segfaults because you are writing outside of memory.

So just make sure you properly create classes, with a proper format!

This test should crash all VMs btw... at least at some point. Since you
are writing in random memory it might take longer to notice it in some
cases; especially when padded memory is owned by the garbage collector :)

cheers,
Toon

On 03/21/2011 10:24 AM, Tudor Girba wrote:

> Hi,
>
> Alex recently wrote a test in Moose that seems to crash the Cog VM at least on Mac.
>
> How to reproduce:
>
> - download the following image
> http://dl.dropbox.com/u/18323746/Tmp/moose-crashing-cog-jit.zip
>
> - execute the following code in the workspace (already provided in the image)
> cls := Class new superclass: MooseElement; yourself.
> cls compileSilently: 'mooseName   ^ 1/0'.
> element := cls new.
>
> - I used all of the followings and they all crashed:
> https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/Cog%20Mac%20Cocoa/4/artifact/cog/build/Cog.zip
> https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/StackVM%20Mac%20Cocoa/2/artifact/cog/build/StackVM.zip
> http://www.mirandabanda.org/files/Cog/VM/VM.r2370/Cog.app.tgz
> http://www.mirandabanda.org/files/Cog/VM/VM.r2361/Cog.app.tgz
>
>
> The strange thing is that the crash only happens when we subclass MooseElement, but not another class.
>
> Could someone take a look?
>
> Cheers,
> Doru
>
>
> --
> www.tudorgirba.com
>
> "Every thing should have the right to be different."
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Tudor Girba
Thanks, Toon!

I changed the code to explicitly set the format, and it seems to fix the problem:

        cls := Class new superclass: MooseElement;
                setFormat: MooseElement format;
                yourself.
        cls compileSilently: 'mooseName   ^ 1/0'.
        element := cls new.

Is this correct?

Cheers,
Doru


On 21 Mar 2011, at 10:57, Toon Verwaest wrote:

> This does crash whenever you subclass a class which has instance variables and you try to access those instance variables. The problem is that you don't properly initialize your class, leaving you with a Class that has a wrong format. For example:
>
> cls := Class new superclass: Class; yourself.
> cls format
>
> returns 2. 2 basically means it's an object with pointers but with 0 instance variables. If you instantiate the 'cls' I just made it also crashes. Why? Well, class has an initialize method that is compiled to write to the fields of the new instance. It puts an empty method dictionary into the class you create as an instance of my cls. This segfaults because you are writing outside of memory.
>
> So just make sure you properly create classes, with a proper format!
>
> This test should crash all VMs btw... at least at some point. Since you are writing in random memory it might take longer to notice it in some cases; especially when padded memory is owned by the garbage collector :)
>
> cheers,
> Toon
>
> On 03/21/2011 10:24 AM, Tudor Girba wrote:
>> Hi,
>>
>> Alex recently wrote a test in Moose that seems to crash the Cog VM at least on Mac.
>>
>> How to reproduce:
>>
>> - download the following image
>> http://dl.dropbox.com/u/18323746/Tmp/moose-crashing-cog-jit.zip
>>
>> - execute the following code in the workspace (already provided in the image)
>> cls := Class new superclass: MooseElement; yourself.
>> cls compileSilently: 'mooseName   ^ 1/0'.
>> element := cls new.
>>
>> - I used all of the followings and they all crashed:
>> https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/Cog%20Mac%20Cocoa/4/artifact/cog/build/Cog.zip
>> https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/StackVM%20Mac%20Cocoa/2/artifact/cog/build/StackVM.zip
>> http://www.mirandabanda.org/files/Cog/VM/VM.r2370/Cog.app.tgz
>> http://www.mirandabanda.org/files/Cog/VM/VM.r2361/Cog.app.tgz
>>
>>
>> The strange thing is that the crash only happens when we subclass MooseElement, but not another class.
>>
>> Could someone take a look?
>>
>> Cheers,
>> Doru
>>
>>
>> --
>> www.tudorgirba.com
>>
>> "Every thing should have the right to be different."
>>
>>
>>
>>
>
>

--
www.tudorgirba.com

"Every thing has its own flow."





Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Toon Verwaest-2
If this is all you are doing then yes, that is correct. If you expect
the format to be different in the new subclass you will have to
recalculate it, but since you aren't doing that in this example, this is
fine.

cheers,
Toon

On 03/21/2011 11:29 AM, Tudor Girba wrote:

> Thanks, Toon!
>
> I changed the code to explicitly set the format, and it seems to fix the problem:
>
> cls := Class new superclass: MooseElement;
> setFormat: MooseElement format;
> yourself.
> cls compileSilently: 'mooseName   ^ 1/0'.
> element := cls new.
>
> Is this correct?
>
> Cheers,
> Doru
>
>
> On 21 Mar 2011, at 10:57, Toon Verwaest wrote:
>
>> This does crash whenever you subclass a class which has instance variables and you try to access those instance variables. The problem is that you don't properly initialize your class, leaving you with a Class that has a wrong format. For example:
>>
>> cls := Class new superclass: Class; yourself.
>> cls format
>>
>> returns 2. 2 basically means it's an object with pointers but with 0 instance variables. If you instantiate the 'cls' I just made it also crashes. Why? Well, class has an initialize method that is compiled to write to the fields of the new instance. It puts an empty method dictionary into the class you create as an instance of my cls. This segfaults because you are writing outside of memory.
>>
>> So just make sure you properly create classes, with a proper format!
>>
>> This test should crash all VMs btw... at least at some point. Since you are writing in random memory it might take longer to notice it in some cases; especially when padded memory is owned by the garbage collector :)
>>
>> cheers,
>> Toon
>>
>> On 03/21/2011 10:24 AM, Tudor Girba wrote:
>>> Hi,
>>>
>>> Alex recently wrote a test in Moose that seems to crash the Cog VM at least on Mac.
>>>
>>> How to reproduce:
>>>
>>> - download the following image
>>> http://dl.dropbox.com/u/18323746/Tmp/moose-crashing-cog-jit.zip
>>>
>>> - execute the following code in the workspace (already provided in the image)
>>> cls := Class new superclass: MooseElement; yourself.
>>> cls compileSilently: 'mooseName   ^ 1/0'.
>>> element := cls new.
>>>
>>> - I used all of the followings and they all crashed:
>>> https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/Cog%20Mac%20Cocoa/4/artifact/cog/build/Cog.zip
>>> https://pharo-ic.lille.inria.fr/hudson/view/Cog/job/StackVM%20Mac%20Cocoa/2/artifact/cog/build/StackVM.zip
>>> http://www.mirandabanda.org/files/Cog/VM/VM.r2370/Cog.app.tgz
>>> http://www.mirandabanda.org/files/Cog/VM/VM.r2361/Cog.app.tgz
>>>
>>>
>>> The strange thing is that the crash only happens when we subclass MooseElement, but not another class.
>>>
>>> Could someone take a look?
>>>
>>> Cheers,
>>> Doru
>>>
>>>
>>> --
>>> www.tudorgirba.com
>>>
>>> "Every thing should have the right to be different."
>>>
>>>
>>>
>>>
>>
> --
> www.tudorgirba.com
>
> "Every thing has its own flow."
>
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Igor Stasenko
In reply to this post by Tudor Girba
On 21 March 2011 11:29, Tudor Girba <[hidden email]> wrote:

> Thanks, Toon!
>
> I changed the code to explicitly set the format, and it seems to fix the problem:
>
>        cls := Class new superclass: MooseElement;
>                setFormat: MooseElement format;
>                yourself.
>        cls compileSilently: 'mooseName   ^ 1/0'.
>        element := cls new.
>
> Is this correct?
>

Yes, you should pay attention to have correct format.
But of course VM could also be more cautios  and verify if your class
are well formed before creating an instance of it..



> Cheers,
> Doru
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Toon Verwaest-2
I couldn't agree more!

However, here the problem is not that the format isn't a correct format.
The problem is that a bytecode is activated that is incompatible with
the instance format. So basically on every bytecode accessing an
instance you would have to check if this is a valid access. And as we
all know, Smalltalk avoids this; it makes things faster :)

Toon

On 03/21/2011 12:49 PM, Igor Stasenko wrote:

> On 21 March 2011 11:29, Tudor Girba<[hidden email]>  wrote:
>> Thanks, Toon!
>>
>> I changed the code to explicitly set the format, and it seems to fix the problem:
>>
>>         cls := Class new superclass: MooseElement;
>>                 setFormat: MooseElement format;
>>                 yourself.
>>         cls compileSilently: 'mooseName   ^ 1/0'.
>>         element := cls new.
>>
>> Is this correct?
>>
> Yes, you should pay attention to have correct format.
> But of course VM could also be more cautios  and verify if your class
> are well formed before creating an instance of it..
>
>
>
>> Cheers,
>> Doru
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Igor Stasenko
On 21 March 2011 13:14, Toon Verwaest <[hidden email]> wrote:
> I couldn't agree more!
>
> However, here the problem is not that the format isn't a correct format. The
> problem is that a bytecode is activated that is incompatible with the
> instance format. So basically on every bytecode accessing an instance you
> would have to check if this is a valid access. And as we all know, Smalltalk
> avoids this; it makes things faster :)
>

In other words, a responsibility of having correct bytecode lies on
image side, not on VM side.

Adding a bytecode verifier to VM will be a lot of work.. And of course
it could verify it only when you installing a method,
not when you activating it.

> Toon
>
> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>
>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>  wrote:
>>>
>>> Thanks, Toon!
>>>
>>> I changed the code to explicitly set the format, and it seems to fix the
>>> problem:
>>>
>>>        cls := Class new superclass: MooseElement;
>>>                setFormat: MooseElement format;
>>>                yourself.
>>>        cls compileSilently: 'mooseName   ^ 1/0'.
>>>        element := cls new.
>>>
>>> Is this correct?
>>>
>> Yes, you should pay attention to have correct format.
>> But of course VM could also be more cautios  and verify if your class
>> are well formed before creating an instance of it..
>>
>>
>>
>>> Cheers,
>>> Doru
>>>
>>
>>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Toon Verwaest-2
Yes you are right! Bytecode validation would be very important here. If
you ask me, the VM should only allow verified bytecode to be executed.
Here it could have a dual model, where bytecodes are "internalized"
whenever you give them to the VM first. At that point they are
validated. Once it's internalized, it can't be modified anymore from the
outside, since the Smalltalk world doesn't have direct access to the
internalized version. All changes to the bytecode "source", needs to be
propagated to the VM by re-internalizing it.

This is exactly the model I'm using in Pinocchio... and that allows us
to not use bytecodes in the first place :) You just need a way to map
the internal version back onto the highlevel version for debugging. (We
don't verify the bytecode yet though).

Bytecode verification btw shouldn't be too hard. You just need to make
sure that all the codes are safe for the instance at hand, and for the
method contexts you create. Anything else is just a runtime semantical
error which you can't guarantee, since you don't know what the
programmer is trying to do. We don't verify it yet since I'm still
designing my register-based bytecodes; and since I atm have too much
work at hand :)

cheers,
Toon

On 03/21/2011 02:22 PM, Igor Stasenko wrote:

> On 21 March 2011 13:14, Toon Verwaest<[hidden email]>  wrote:
>> I couldn't agree more!
>>
>> However, here the problem is not that the format isn't a correct format. The
>> problem is that a bytecode is activated that is incompatible with the
>> instance format. So basically on every bytecode accessing an instance you
>> would have to check if this is a valid access. And as we all know, Smalltalk
>> avoids this; it makes things faster :)
>>
> In other words, a responsibility of having correct bytecode lies on
> image side, not on VM side.
>
> Adding a bytecode verifier to VM will be a lot of work.. And of course
> it could verify it only when you installing a method,
> not when you activating it.
>
>> Toon
>>
>> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>    wrote:
>>>> Thanks, Toon!
>>>>
>>>> I changed the code to explicitly set the format, and it seems to fix the
>>>> problem:
>>>>
>>>>         cls := Class new superclass: MooseElement;
>>>>                 setFormat: MooseElement format;
>>>>                 yourself.
>>>>         cls compileSilently: 'mooseName   ^ 1/0'.
>>>>         element := cls new.
>>>>
>>>> Is this correct?
>>>>
>>> Yes, you should pay attention to have correct format.
>>> But of course VM could also be more cautios  and verify if your class
>>> are well formed before creating an instance of it..
>>>
>>>
>>>
>>>> Cheers,
>>>> Doru
>>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

abergel
Thanks for this fruitful discussion. Thanks Doru for fixing my test. It was hard to predict this problem, since it went green on my machine.

Good job Guys!

Alexandre



Le 21 mars 2011 à 09:57, Toon Verwaest <[hidden email]> a écrit :

> Yes you are right! Bytecode validation would be very important here. If you ask me, the VM should only allow verified bytecode to be executed. Here it could have a dual model, where bytecodes are "internalized" whenever you give them to the VM first. At that point they are validated. Once it's internalized, it can't be modified anymore from the outside, since the Smalltalk world doesn't have direct access to the internalized version. All changes to the bytecode "source", needs to be propagated to the VM by re-internalizing it.
>
> This is exactly the model I'm using in Pinocchio... and that allows us to not use bytecodes in the first place :) You just need a way to map the internal version back onto the highlevel version for debugging. (We don't verify the bytecode yet though).
>
> Bytecode verification btw shouldn't be too hard. You just need to make sure that all the codes are safe for the instance at hand, and for the method contexts you create. Anything else is just a runtime semantical error which you can't guarantee, since you don't know what the programmer is trying to do. We don't verify it yet since I'm still designing my register-based bytecodes; and since I atm have too much work at hand :)
>
> cheers,
> Toon
>
> On 03/21/2011 02:22 PM, Igor Stasenko wrote:
>> On 21 March 2011 13:14, Toon Verwaest<[hidden email]>  wrote:
>>> I couldn't agree more!
>>>
>>> However, here the problem is not that the format isn't a correct format. The
>>> problem is that a bytecode is activated that is incompatible with the
>>> instance format. So basically on every bytecode accessing an instance you
>>> would have to check if this is a valid access. And as we all know, Smalltalk
>>> avoids this; it makes things faster :)
>>>
>> In other words, a responsibility of having correct bytecode lies on
>> image side, not on VM side.
>>
>> Adding a bytecode verifier to VM will be a lot of work.. And of course
>> it could verify it only when you installing a method,
>> not when you activating it.
>>
>>> Toon
>>>
>>> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>    wrote:
>>>>> Thanks, Toon!
>>>>>
>>>>> I changed the code to explicitly set the format, and it seems to fix the
>>>>> problem:
>>>>>
>>>>>        cls := Class new superclass: MooseElement;
>>>>>                setFormat: MooseElement format;
>>>>>                yourself.
>>>>>        cls compileSilently: 'mooseName   ^ 1/0'.
>>>>>        element := cls new.
>>>>>
>>>>> Is this correct?
>>>>>
>>>> Yes, you should pay attention to have correct format.
>>>> But of course VM could also be more cautios  and verify if your class
>>>> are well formed before creating an instance of it..
>>>>
>>>>
>>>>
>>>>> Cheers,
>>>>> Doru
>>>>>
>>>>
>>>
>>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Stéphane Ducasse
In reply to this post by Toon Verwaest-2
This would be a nice little topic... this bytecode verifier.

Stef

On Mar 21, 2011, at 2:57 PM, Toon Verwaest wrote:

> Yes you are right! Bytecode validation would be very important here. If you ask me, the VM should only allow verified bytecode to be executed. Here it could have a dual model, where bytecodes are "internalized" whenever you give them to the VM first. At that point they are validated. Once it's internalized, it can't be modified anymore from the outside, since the Smalltalk world doesn't have direct access to the internalized version. All changes to the bytecode "source", needs to be propagated to the VM by re-internalizing it.
>
> This is exactly the model I'm using in Pinocchio... and that allows us to not use bytecodes in the first place :) You just need a way to map the internal version back onto the highlevel version for debugging. (We don't verify the bytecode yet though).
>
> Bytecode verification btw shouldn't be too hard. You just need to make sure that all the codes are safe for the instance at hand, and for the method contexts you create. Anything else is just a runtime semantical error which you can't guarantee, since you don't know what the programmer is trying to do. We don't verify it yet since I'm still designing my register-based bytecodes; and since I atm have too much work at hand :)
>
> cheers,
> Toon
>
> On 03/21/2011 02:22 PM, Igor Stasenko wrote:
>> On 21 March 2011 13:14, Toon Verwaest<[hidden email]>  wrote:
>>> I couldn't agree more!
>>>
>>> However, here the problem is not that the format isn't a correct format. The
>>> problem is that a bytecode is activated that is incompatible with the
>>> instance format. So basically on every bytecode accessing an instance you
>>> would have to check if this is a valid access. And as we all know, Smalltalk
>>> avoids this; it makes things faster :)
>>>
>> In other words, a responsibility of having correct bytecode lies on
>> image side, not on VM side.
>>
>> Adding a bytecode verifier to VM will be a lot of work.. And of course
>> it could verify it only when you installing a method,
>> not when you activating it.
>>
>>> Toon
>>>
>>> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>    wrote:
>>>>> Thanks, Toon!
>>>>>
>>>>> I changed the code to explicitly set the format, and it seems to fix the
>>>>> problem:
>>>>>
>>>>>        cls := Class new superclass: MooseElement;
>>>>>                setFormat: MooseElement format;
>>>>>                yourself.
>>>>>        cls compileSilently: 'mooseName   ^ 1/0'.
>>>>>        element := cls new.
>>>>>
>>>>> Is this correct?
>>>>>
>>>> Yes, you should pay attention to have correct format.
>>>> But of course VM could also be more cautios  and verify if your class
>>>> are well formed before creating an instance of it..
>>>>
>>>>
>>>>
>>>>> Cheers,
>>>>> Doru
>>>>>
>>>>
>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Mariano Martinez Peck


On Mon, Mar 21, 2011 at 7:23 PM, Stéphane Ducasse <[hidden email]> wrote:
This would be a nice little topic... this bytecode verifier.

Probably on top of Opal

 

Stef

On Mar 21, 2011, at 2:57 PM, Toon Verwaest wrote:

> Yes you are right! Bytecode validation would be very important here. If you ask me, the VM should only allow verified bytecode to be executed. Here it could have a dual model, where bytecodes are "internalized" whenever you give them to the VM first. At that point they are validated. Once it's internalized, it can't be modified anymore from the outside, since the Smalltalk world doesn't have direct access to the internalized version. All changes to the bytecode "source", needs to be propagated to the VM by re-internalizing it.
>
> This is exactly the model I'm using in Pinocchio... and that allows us to not use bytecodes in the first place :) You just need a way to map the internal version back onto the highlevel version for debugging. (We don't verify the bytecode yet though).
>
> Bytecode verification btw shouldn't be too hard. You just need to make sure that all the codes are safe for the instance at hand, and for the method contexts you create. Anything else is just a runtime semantical error which you can't guarantee, since you don't know what the programmer is trying to do. We don't verify it yet since I'm still designing my register-based bytecodes; and since I atm have too much work at hand :)
>
> cheers,
> Toon
>
> On 03/21/2011 02:22 PM, Igor Stasenko wrote:
>> On 21 March 2011 13:14, Toon Verwaest<[hidden email]>  wrote:
>>> I couldn't agree more!
>>>
>>> However, here the problem is not that the format isn't a correct format. The
>>> problem is that a bytecode is activated that is incompatible with the
>>> instance format. So basically on every bytecode accessing an instance you
>>> would have to check if this is a valid access. And as we all know, Smalltalk
>>> avoids this; it makes things faster :)
>>>
>> In other words, a responsibility of having correct bytecode lies on
>> image side, not on VM side.
>>
>> Adding a bytecode verifier to VM will be a lot of work.. And of course
>> it could verify it only when you installing a method,
>> not when you activating it.
>>
>>> Toon
>>>
>>> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>    wrote:
>>>>> Thanks, Toon!
>>>>>
>>>>> I changed the code to explicitly set the format, and it seems to fix the
>>>>> problem:
>>>>>
>>>>>        cls := Class new superclass: MooseElement;
>>>>>                setFormat: MooseElement format;
>>>>>                yourself.
>>>>>        cls compileSilently: 'mooseName   ^ 1/0'.
>>>>>        element := cls new.
>>>>>
>>>>> Is this correct?
>>>>>
>>>> Yes, you should pay attention to have correct format.
>>>> But of course VM could also be more cautios  and verify if your class
>>>> are well formed before creating an instance of it..
>>>>
>>>>
>>>>
>>>>> Cheers,
>>>>> Doru
>>>>>
>>>>
>>>
>>>
>>
>>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Eliot Miranda-2


On Mon, Mar 21, 2011 at 12:13 PM, Mariano Martinez Peck <[hidden email]> wrote:


On Mon, Mar 21, 2011 at 7:23 PM, Stéphane Ducasse <[hidden email]> wrote:
This would be a nice little topic... this bytecode verifier.

Probably on top of Opal

The bytecode verifier must be in the VM.  If it is up in the image it can be side-stepped.  The VM is the ultimate executor of code and so it must apply verification.  It could be written in Smalltalk and verified in Smalltalk and then translated.  But it must be part of the VM and used by the VM before running any previously unverified method.

best,
Eliot


 

Stef

On Mar 21, 2011, at 2:57 PM, Toon Verwaest wrote:

> Yes you are right! Bytecode validation would be very important here. If you ask me, the VM should only allow verified bytecode to be executed. Here it could have a dual model, where bytecodes are "internalized" whenever you give them to the VM first. At that point they are validated. Once it's internalized, it can't be modified anymore from the outside, since the Smalltalk world doesn't have direct access to the internalized version. All changes to the bytecode "source", needs to be propagated to the VM by re-internalizing it.
>
> This is exactly the model I'm using in Pinocchio... and that allows us to not use bytecodes in the first place :) You just need a way to map the internal version back onto the highlevel version for debugging. (We don't verify the bytecode yet though).
>
> Bytecode verification btw shouldn't be too hard. You just need to make sure that all the codes are safe for the instance at hand, and for the method contexts you create. Anything else is just a runtime semantical error which you can't guarantee, since you don't know what the programmer is trying to do. We don't verify it yet since I'm still designing my register-based bytecodes; and since I atm have too much work at hand :)
>
> cheers,
> Toon
>
> On 03/21/2011 02:22 PM, Igor Stasenko wrote:
>> On 21 March 2011 13:14, Toon Verwaest<[hidden email]>  wrote:
>>> I couldn't agree more!
>>>
>>> However, here the problem is not that the format isn't a correct format. The
>>> problem is that a bytecode is activated that is incompatible with the
>>> instance format. So basically on every bytecode accessing an instance you
>>> would have to check if this is a valid access. And as we all know, Smalltalk
>>> avoids this; it makes things faster :)
>>>
>> In other words, a responsibility of having correct bytecode lies on
>> image side, not on VM side.
>>
>> Adding a bytecode verifier to VM will be a lot of work.. And of course
>> it could verify it only when you installing a method,
>> not when you activating it.
>>
>>> Toon
>>>
>>> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>    wrote:
>>>>> Thanks, Toon!
>>>>>
>>>>> I changed the code to explicitly set the format, and it seems to fix the
>>>>> problem:
>>>>>
>>>>>        cls := Class new superclass: MooseElement;
>>>>>                setFormat: MooseElement format;
>>>>>                yourself.
>>>>>        cls compileSilently: 'mooseName   ^ 1/0'.
>>>>>        element := cls new.
>>>>>
>>>>> Is this correct?
>>>>>
>>>> Yes, you should pay attention to have correct format.
>>>> But of course VM could also be more cautios  and verify if your class
>>>> are well formed before creating an instance of it..
>>>>
>>>>
>>>>
>>>>> Cheers,
>>>>> Doru
>>>>>
>>>>
>>>
>>>
>>
>>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Igor Stasenko
On 21 March 2011 21:14, Eliot Miranda <[hidden email]> wrote:

>
>
> On Mon, Mar 21, 2011 at 12:13 PM, Mariano Martinez Peck
> <[hidden email]> wrote:
>>
>>
>> On Mon, Mar 21, 2011 at 7:23 PM, Stéphane Ducasse
>> <[hidden email]> wrote:
>>>
>>> This would be a nice little topic... this bytecode verifier.
>>
>> Probably on top of Opal
>
> The bytecode verifier must be in the VM.  If it is up in the image it can be
> side-stepped.  The VM is the ultimate executor of code and so it must apply
> verification.  It could be written in Smalltalk and verified in Smalltalk
> and then translated.  But it must be part of the VM and used by the VM
> before running any previously unverified method.

indeed

> best,
> Eliot
>>
>>
>>>
>>> Stef
>>>
>>> On Mar 21, 2011, at 2:57 PM, Toon Verwaest wrote:
>>>
>>> > Yes you are right! Bytecode validation would be very important here. If
>>> > you ask me, the VM should only allow verified bytecode to be executed. Here
>>> > it could have a dual model, where bytecodes are "internalized" whenever you
>>> > give them to the VM first. At that point they are validated. Once it's
>>> > internalized, it can't be modified anymore from the outside, since the
>>> > Smalltalk world doesn't have direct access to the internalized version. All
>>> > changes to the bytecode "source", needs to be propagated to the VM by
>>> > re-internalizing it.
>>> >
>>> > This is exactly the model I'm using in Pinocchio... and that allows us
>>> > to not use bytecodes in the first place :) You just need a way to map the
>>> > internal version back onto the highlevel version for debugging. (We don't
>>> > verify the bytecode yet though).
>>> >
>>> > Bytecode verification btw shouldn't be too hard. You just need to make
>>> > sure that all the codes are safe for the instance at hand, and for the
>>> > method contexts you create. Anything else is just a runtime semantical error
>>> > which you can't guarantee, since you don't know what the programmer is
>>> > trying to do. We don't verify it yet since I'm still designing my
>>> > register-based bytecodes; and since I atm have too much work at hand :)
>>> >
>>> > cheers,
>>> > Toon
>>> >
>>> > On 03/21/2011 02:22 PM, Igor Stasenko wrote:
>>> >> On 21 March 2011 13:14, Toon Verwaest<[hidden email]>  wrote:
>>> >>> I couldn't agree more!
>>> >>>
>>> >>> However, here the problem is not that the format isn't a correct
>>> >>> format. The
>>> >>> problem is that a bytecode is activated that is incompatible with the
>>> >>> instance format. So basically on every bytecode accessing an instance
>>> >>> you
>>> >>> would have to check if this is a valid access. And as we all know,
>>> >>> Smalltalk
>>> >>> avoids this; it makes things faster :)
>>> >>>
>>> >> In other words, a responsibility of having correct bytecode lies on
>>> >> image side, not on VM side.
>>> >>
>>> >> Adding a bytecode verifier to VM will be a lot of work.. And of course
>>> >> it could verify it only when you installing a method,
>>> >> not when you activating it.
>>> >>
>>> >>> Toon
>>> >>>
>>> >>> On 03/21/2011 12:49 PM, Igor Stasenko wrote:
>>> >>>> On 21 March 2011 11:29, Tudor Girba<[hidden email]>    wrote:
>>> >>>>> Thanks, Toon!
>>> >>>>>
>>> >>>>> I changed the code to explicitly set the format, and it seems to
>>> >>>>> fix the
>>> >>>>> problem:
>>> >>>>>
>>> >>>>>        cls := Class new superclass: MooseElement;
>>> >>>>>                setFormat: MooseElement format;
>>> >>>>>                yourself.
>>> >>>>>        cls compileSilently: 'mooseName   ^ 1/0'.
>>> >>>>>        element := cls new.
>>> >>>>>
>>> >>>>> Is this correct?
>>> >>>>>
>>> >>>> Yes, you should pay attention to have correct format.
>>> >>>> But of course VM could also be more cautios  and verify if your
>>> >>>> class
>>> >>>> are well formed before creating an instance of it..
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>>> Cheers,
>>> >>>>> Doru
>>> >>>>>
>>> >>>>
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >
>>> >
>>>
>>>
>>
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Toon Verwaest-2
In reply to this post by Eliot Miranda-2

On Mon, Mar 21, 2011 at 7:23 PM, Stéphane Ducasse <[hidden email]> wrote:
This would be a nice little topic... this bytecode verifier.

Probably on top of Opal

The bytecode verifier must be in the VM.  If it is up in the image it can be side-stepped.  The VM is the ultimate executor of code and so it must apply verification.  It could be written in Smalltalk and verified in Smalltalk and then translated.  But it must be part of the VM and used by the VM before running any previously unverified method.

best,
Eliot
Indeed, Mariano's response made me want to scream: NOOOOO! :)

HOWEVER!, Thinking about it now, it's actually a very interesting proposal! What if we write this piece of software in Smalltak and give it to the VM as part of the VM definition. The VM decides what the application can see, so it might as well not give access to this smalltalk-written bytecode validator that it knows to be safe. If it's built by a separate classbuilder in a separate environment, it can be clearly separated, invulnerable to become: and other evil-doing. So, in short. It's written in Smalltalk (YAY!), but becomes part of the VM runtime, not of the image running on the VM. If we just make sure that we can write out image segments that the VM can use as part of its core (exactly what I do in Pinocchio incidentally), we can just develop and test it nicely in Pharo. Ain't that super-duper cool?

Ok, I think I drank too much Yerba Mate. Bad bad caffeine :)

cheers,
Toon
Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Marcus Denker-4
In reply to this post by Eliot Miranda-2

On Mar 22, 2011, at 12:17 AM, Toon Verwaest wrote:


On Mon, Mar 21, 2011 at 7:23 PM, Stéphane Ducasse <[hidden email]> wrote:
This would be a nice little topic... this bytecode verifier.

Probably on top of Opal

The bytecode verifier must be in the VM.  If it is up in the image it can be side-stepped.  The VM is the ultimate executor of code and so it must apply verification.  It could be written in Smalltalk and verified in Smalltalk and then translated.  But it must be part of the VM and used by the VM before running any previously unverified method.

best,
Eliot
Indeed, Mariano's response made me want to scream: NOOOOO! :)

HOWEVER!, Thinking about it now, it's actually a very interesting proposal! What if we write this piece of software in Smalltak and give it to the VM as part of the VM definition. The VM decides what the application can see, so it might as well not give access to this smalltalk-written bytecode validator that it knows to be safe. If it's built by a separate classbuilder in a separate environment, it can be clearly separated, invulnerable to become: and other evil-doing. So, in short. It's written in Smalltalk (YAY!), but becomes part of the VM runtime, not of the image running on the VM. If we just make sure that we can write out image segments that the VM can use as part of its core (exactly what I do in Pinocchio incidentally), we can just develop and test it nicely in Pharo. Ain't that super-duper cool?


If we would have solved the security problem already, then we could use it for the bytecode verifyer, too... (guarantee that nobody tampers with the verifyer).

We need to reify that was makes "the VM" special and provide it for all smalltalk code. But that is of course research and completely  not clear
yet what it means.

Marcus


--
Marcus Denker  -- http://www.marcusdenker.de
INRIA Lille -- Nord Europe. Team RMoD.

Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Igor Stasenko
On 22 March 2011 08:55, Marcus Denker <[hidden email]> wrote:

>
> On Mar 22, 2011, at 12:17 AM, Toon Verwaest wrote:
>
>> On Mon, Mar 21, 2011 at 7:23 PM, Stéphane Ducasse
>> <[hidden email]> wrote:
>>>
>>> This would be a nice little topic... this bytecode verifier.
>>
>> Probably on top of Opal
>
> The bytecode verifier must be in the VM.  If it is up in the image it can be
> side-stepped.  The VM is the ultimate executor of code and so it must apply
> verification.  It could be written in Smalltalk and verified in Smalltalk
> and then translated.  But it must be part of the VM and used by the VM
> before running any previously unverified method.
> best,
> Eliot
>
> Indeed, Mariano's response made me want to scream: NOOOOO! :)
>
> HOWEVER!, Thinking about it now, it's actually a very interesting proposal!
> What if we write this piece of software in Smalltak and give it to the VM as
> part of the VM definition. The VM decides what the application can see, so
> it might as well not give access to this smalltalk-written bytecode
> validator that it knows to be safe. If it's built by a separate classbuilder
> in a separate environment, it can be clearly separated, invulnerable to
> become: and other evil-doing. So, in short. It's written in Smalltalk
> (YAY!), but becomes part of the VM runtime, not of the image running on the
> VM. If we just make sure that we can write out image segments that the VM
> can use as part of its core (exactly what I do in Pinocchio incidentally),
> we can just develop and test it nicely in Pharo. Ain't that super-duper
> cool?
>
>
> If we would have solved the security problem already, then we could use it
> for the bytecode verifyer, too... (guarantee that nobody tampers with the
> verifyer).
> We need to reify that was makes "the VM" special and provide it for all
> smalltalk code. But that is of course research and completely  not clear
> yet what it means.

By stretching, you can treat VM as a hardware which you can't change
and must follow its rules.
But as someone said, hardware is an early crystallized software, which
means that we can change it,
but at much higher cost(s) in terms of engineering effort and time.
As we hopefully approach closer to self-sustaining systems (like
SqueakNOS shows) the need in VM, as
a separate software layer between hardware and a language is smearing.
And i would really like to see it completely disappeared at some day.

There are a family of languages which is already very close to
self-sustaining form - a concatenative languages, like Forth or
Factor.
Most of them are implemented in themselves, and have very thin layer
between language and hadrware implemented in C,
the rest is already in language.

See http://concatenative.org/wiki/view/Factor/Optimizing%20compiler


> Marcus
>
> --
> Marcus Denker  -- http://www.marcusdenker.de
> INRIA Lille -- Nord Europe. Team RMoD.
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Stéphane Ducasse
In reply to this post by Eliot Miranda-2
But why we could not have a byecode validator at the image level that first make sure that byte code are in sync with the format of the objects.
Why this has to be done in the vm.

Stef
Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Marcus Denker-4
In reply to this post by Eliot Miranda-2

On Mar 22, 2011, at 1:35 PM, Stéphane Ducasse wrote:

> But why we could not have a byecode validator at the image level that first make sure that byte code are in sync with the format of the objects.
> Why this has to be done in the vm.
>
Because you can not gurantee that they are not changed after. Or that the code is changed that does the checking. or...

        Marcus


--
Marcus Denker  -- http://www.marcusdenker.de
INRIA Lille -- Nord Europe. Team RMoD.


Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

abergel
In reply to this post by Stéphane Ducasse
> But why we could not have a byecode validator at the image level that first make sure that byte code are in sync with the format of the objects.
> Why this has to be done in the vm.


I agree with Stef. It is not obvious for me why it has to be done at the VM.

Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.






Reply | Threaded
Open this post in threaded view
|

Re: test crashing the cog vm

Toon Verwaest-2
The problem is exactly what you had at hand. Your bytecode WAS valid,
but it was used in combination with an incompatible class layout. So
validation here wouldn't solve anything. You always need to validate in
a closed world to ensure you don't accidentally break everything.
Whenever you change something, you need to ensure that you revalidate
the relevant parts. In this case you could for example just have
validated that your new class is a valid subclass of its superclass;
which it was not.

To make the system more secure obviously you would have to check those
things, but if you add part of the API that circumvents these checks,
like you were doing by calling "Class new" rather than using the
ClassBuilder which does do the checks, everything breaks. The only
authority that can actually ensure that you don't circumvent these
checks is the VM.

Obviously you can make sure already in your image that you have enough
checks everywhere. You just don't have a crashproof mechanism that will
chain your users down avoiding that they shoot themselves in the foot
with segfaults. If you put it inside of the VM you -can- provide such a
mechanism, because you don't execute anything unless you know it's safe.
And as I said, this piece of code could be a piece of prevalidated
Smalltalk code that's immutable from the rest of the image.

cheers,
Toon

On 03/22/2011 02:37 PM, Alexandre Bergel wrote:
>> But why we could not have a byecode validator at the image level that first make sure that byte code are in sync with the format of the objects.
>> Why this has to be done in the vm.
>
> I agree with Stef. It is not obvious for me why it has to be done at the VM.
>
> Alexandre


123