Quantcast

script to add internal plugins ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

script to add internal plugins ?

SophieKaleba
 
Hi again

I want to modify the plugins.int files to add some internal plugins to the different pharo distributions (http://forum.world.st/Missing-plugins-to-make-the-vmProfiler-work-on-Pharo-td4946444.html).

Is there an already existing script that I could use for this matter ?

Thanks
Sophie
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

timrowledge
 

> On 10-05-2017, at 10:47 AM, Sophie Kaleba <[hidden email]> wrote:
>
> Hi again
>
> I want to modify the plugins.int files to add some internal plugins to the different pharo distributions (http://forum.world.st/Missing-plugins-to-make-the-vmProfiler-work-on-Pharo-td4946444.html).

It’s just a text file so you edit with your favourite text editing tools (smalltalk, obviously!) and add another line with the name of your plugin.

If the plugin is already listed in the *other* plugins.* file then you’d want to remove it as well - ie to change FooPlugin from being built internal to external, delete it from plugins.int and add to plugins.ext

You could very simply try this out by moving pretty much any current plugin from one list to another and running a build, just to convince yourself it works ok. You might also take a look though the assorted plugins.* files in the many `build.*/build*` directories to see a lot of examples. And indeed a lot of cases where a plugin appears to be near-randlomly included or not, something that could do with tidying up. And if you were to conclude that making things tidier and doing test builds was a project worth spending a little time on then you would certainly earn my thanks...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- LE ROI EST MORT. JIVE LE ROI - The King is dead.  No kidding.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

SophieKaleba
 
Hi Tim,

I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins

I knew indeed that I could edit all the plugins.*files by hand, that was the reason why I was looking for a script. Please forgive my laziness :-)

Sophie


2017-05-10 20:02 GMT+02:00 tim Rowledge <[hidden email]>:


> On 10-05-2017, at 10:47 AM, Sophie Kaleba <[hidden email]> wrote:
>
> Hi again
>
> I want to modify the plugins.int files to add some internal plugins to the different pharo distributions (http://forum.world.st/Missing-plugins-to-make-the-vmProfiler-work-on-Pharo-td4946444.html).

It’s just a text file so you edit with your favourite text editing tools (smalltalk, obviously!) and add another line with the name of your plugin.

If the plugin is already listed in the *other* plugins.* file then you’d want to remove it as well - ie to change FooPlugin from being built internal to external, delete it from plugins.int and add to plugins.ext

You could very simply try this out by moving pretty much any current plugin from one list to another and running a build, just to convince yourself it works ok. You might also take a look though the assorted plugins.* files in the many `build.*/build*` directories to see a lot of examples. And indeed a lot of cases where a plugin appears to be near-randlomly included or not, something that could do with tidying up. And if you were to conclude that making things tidier and doing test builds was a project worth spending a little time on then you would certainly earn my thanks...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Fractured Idiom:- LE ROI EST MORT. JIVE LE ROI - The King is dead.  No kidding.



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

timrowledge
 

> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>
> Hi Tim,
>
> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins

I’ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I’d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there’d be problems if the internal version were compiled with flag A, the ‘.o’ file were left around, the external version compile process found it, therefore didn’t compile with flag B, thus making something fail?

>
> I knew indeed that I could edit all the plugins.*files by hand, that was the reason why I was looking for a script. Please forgive my laziness :-)

Creatively applied laziness is an admirable thing. It’s what makes us write software and make tools so we can do more with less effort.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Always try to be modest. And be damn proud of it!


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

David T. Lewis
 
On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:

>  
>
> > On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
> >
> > Hi Tim,
> >
> > I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>
> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>

I have not checked in a while, but I am pretty sure that it is perfectly
ok to have both an internally compiled plugin as part of the main VM
executable, and also an externally compiled version of the same plugin
(a loadable module in a file separate from the main VM executable). In
that case, the external plugin should override the internal one.

The idea is that it should be easy to make an external plugin that might
add some new behaviour, and the plugin developer should be able to do
this without waiting for somebody else to distribute a new VM.

If this is not the case, then it certainly /should/ be. Tim, weren't
you one of the people who invented this loadable module stuff in the
first place?

Dave
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

timrowledge
 

> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>
>
> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>
>>
>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>>
>>> Hi Tim,
>>>
>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>
>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>
>
> I have not checked in a while, but I am pretty sure that it is perfectly
> ok to have both an internally compiled plugin as part of the main VM
> executable, and also an externally compiled version of the same plugin
> (a loadable module in a file separate from the main VM executable). In
> that case, the external plugin should override the internal one.
>
> The idea is that it should be easy to make an external plugin that might
> add some new behaviour, and the plugin developer should be able to do
> this without waiting for somebody else to distribute a new VM.
>
> If this is not the case, then it certainly /should/ be. Tim, weren't
> you one of the people who invented this loadable module stuff in the
> first place?

D’oh, of course you’re right. Oh the joys of aging…

And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- So dumb, he faxes face up.


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

Ben Coman
 
On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:

>
>
>> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>
>>
>> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>>
>>>
>>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>>>
>>>> Hi Tim,
>>>>
>>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>>
>>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>>
>>
>> I have not checked in a while, but I am pretty sure that it is perfectly
>> ok to have both an internally compiled plugin as part of the main VM
>> executable, and also an externally compiled version of the same plugin
>> (a loadable module in a file separate from the main VM executable). In
>> that case, the external plugin should override the internal one.
>>
>> The idea is that it should be easy to make an external plugin that might
>> add some new behaviour, and the plugin developer should be able to do
>> this without waiting for somebody else to distribute a new VM.
>>
>> If this is not the case, then it certainly /should/ be. Tim, weren't
>> you one of the people who invented this loadable module stuff in the
>> first place?
>
> D’oh, of course you’re right. Oh the joys of aging…
>
> And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
> So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
> I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...

Remember, just because you think you're paranoid doesn't mean they're
not watching you.

HTH  ;)
cheers -ben
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

SophieKaleba
 
Update :

I did add the plugins to the file. That was the easy part, it went well.

Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :

- at first, I had an error : "can't find PharoV50.sources".

(I cloned the opensmalltalk-vm directory,
 ran updateSCCSVersions
and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )

Manually adding a "sources" directory containing the missing file solved the problem
Did I miss something on that part?


-OK, now, the VM is built with this small hack.

When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :

*** stack smashing detected ***: ./squeak terminated
output file stack is full.

I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.

Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
I will also try to update some libs, just in case.

In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.

Sophie



2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:

On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
>
>
>> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>
>>
>> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>>
>>>
>>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>>>
>>>> Hi Tim,
>>>>
>>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>>
>>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>>
>>
>> I have not checked in a while, but I am pretty sure that it is perfectly
>> ok to have both an internally compiled plugin as part of the main VM
>> executable, and also an externally compiled version of the same plugin
>> (a loadable module in a file separate from the main VM executable). In
>> that case, the external plugin should override the internal one.
>>
>> The idea is that it should be easy to make an external plugin that might
>> add some new behaviour, and the plugin developer should be able to do
>> this without waiting for somebody else to distribute a new VM.
>>
>> If this is not the case, then it certainly /should/ be. Tim, weren't
>> you one of the people who invented this loadable module stuff in the
>> first place?
>
> D’oh, of course you’re right. Oh the joys of aging…
>
> And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
> So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
> I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...

Remember, just because you think you're paranoid doesn't mean they're
not watching you.

HTH  ;)
cheers -ben

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

Eliot Miranda-2
 
Hi Sophie,

On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]> wrote:
 
Update :

I did add the plugins to the file. That was the easy part, it went well.

Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :

- at first, I had an error : "can't find PharoV50.sources".

(I cloned the opensmalltalk-vm directory,
 ran updateSCCSVersions
and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )

Manually adding a "sources" directory containing the missing file solved the problem
Did I miss something on that part?


-OK, now, the VM is built with this small hack.

When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :

*** stack smashing detected ***: ./squeak terminated
output file stack is full.

I wonder if this is some checking feature supported by the C compiler you're using.  I expect that it is either disabled or not available in the VMs built on binary.  I say this because there is no occurrence of the string 'stack smashing detected' in our sources and so expect it is being produced by the C runtime on your version of linux.

The stack organization in the VM is a little unusual.  The VM reserves space for a set of "stack pages" upon which Smalltalk activations exist.  The space for the stack pages is allocated via alloca on the C stack at startup.  It could be that this strange organization, which means that at some times, when the VM is executing C, the stack pointer is at the bottom end of the C stack, but when it is executing JITted Smalltalk code the stack pointer is somewhere in the middle of the C stack.  I expect that the C runtime sees this happening and misinterprets it as an error.

I expect further that you can, and should, disable such checking.

HTH

 

I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.

Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
I will also try to update some libs, just in case.

In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.

Sophie



2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:

On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
>
>
>> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>
>>
>> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>>
>>>
>>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>>>
>>>> Hi Tim,
>>>>
>>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>>
>>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>>
>>
>> I have not checked in a while, but I am pretty sure that it is perfectly
>> ok to have both an internally compiled plugin as part of the main VM
>> executable, and also an externally compiled version of the same plugin
>> (a loadable module in a file separate from the main VM executable). In
>> that case, the external plugin should override the internal one.
>>
>> The idea is that it should be easy to make an external plugin that might
>> add some new behaviour, and the plugin developer should be able to do
>> this without waiting for somebody else to distribute a new VM.
>>
>> If this is not the case, then it certainly /should/ be. Tim, weren't
>> you one of the people who invented this loadable module stuff in the
>> first place?
>
> D’oh, of course you’re right. Oh the joys of aging…
>
> And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
> So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
> I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...

Remember, just because you think you're paranoid doesn't mean they're
not watching you.

HTH  ;)
cheers -ben





--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

David T. Lewis
 
On Fri, May 12, 2017 at 09:00:08PM -0700, Eliot Miranda wrote:

>  
> Hi Sophie,
>
> On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]>
> wrote:
>
> >
> > Update :
> >
> > I did add the plugins to the file. That was the easy part, it went well.
> >
> > Then it got trickier when I tried to actually build the (Pharo) VM
> > (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new
> > plugins :
> >
> > - at first, I had an error : "can't find PharoV50.sources".
> >
> > (I cloned the opensmalltalk-vm directory,
> >  ran updateSCCSVersions
> > and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat"
> > and ran ./mvm -f )
> >
> > Manually adding a "sources" directory containing the missing file solved
> > the problem
> > Did I miss something on that part?
> >
> >
> > -OK, now, the VM is built with this small hack.
> >
> > When I try to use the VMProfiler (which works fine using the latest squeak
> > VM from bintray), I get a segmentation fault, with this message :
> >
> > *** stack smashing detected ***: ./squeak terminated
> > output file stack is full.
> >
>
> I wonder if this is some checking feature supported by the C compiler
> you're using.  I expect that it is either disabled or not available in the
> VMs built on binary.  I say this because there is no occurrence of the
> string 'stack smashing detected' in our sources and so expect it is being
> produced by the C runtime on your version of linux.
>
> The stack organization in the VM is a little unusual.  The VM reserves
> space for a set of "stack pages" upon which Smalltalk activations exist.
> The space for the stack pages is allocated via alloca on the C stack at
> startup.  It could be that this strange organization, which means that at
> some times, when the VM is executing C, the stack pointer is at the bottom
> end of the C stack, but when it is executing JITted Smalltalk code the
> stack pointer is somewhere in the middle of the C stack.  I expect that the
> C runtime sees this happening and misinterprets it as an error.
>
> I expect further that you can, and should, disable such checking.
>


It seems likely that this is a real stack overflow error.

The "stack smashing detected" message comes from the C runtime. I googled it
and found several good explanations.

Here is a tip on how to partially disable the checking (although I do not
think it will help much in this case);

  https://reverseengineering.stackexchange.com/questions/11777/how-to-effectively-bypass-gcc-stack-smashing-detection

The other error message "output file stack is full" comes from the VM itself,
in the pushOutputFile() function in platforms/Cross/vm/sqVirtualMachine.c.

I cannot guess what might be the cause of this, but for purposes of debugging
it might help to try building the VM without the new plugins. If the problem
goes away, then it may indicate a problem in the plugins that are being added.

Dave

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

Ben Coman
In reply to this post by Eliot Miranda-2
 
On Sat, May 13, 2017 at 12:00 PM, Eliot Miranda <[hidden email]> wrote:

>
> Hi Sophie,
>
> On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]> wrote:
>>
>>
>> Update :
>>
>> I did add the plugins to the file. That was the easy part, it went well.
>>
>> Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :
>>
>> - at first, I had an error : "can't find PharoV50.sources".

You're the fifth newcomer compiling the VM that I've seen bump into
this problem.
The fix PR has been available since last October...
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/79


>>
>> (I cloned the opensmalltalk-vm directory,
>>  ran updateSCCSVersions
>> and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )
>>
>> Manually adding a "sources" directory containing the missing file solved the problem
>> Did I miss something on that part?
>>
>>
>> -OK, now, the VM is built with this small hack.
>>
>> When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :
>>
>> *** stack smashing detected ***: ./squeak terminated
>> output file stack is full.
>
>
> I wonder if this is some checking feature supported by the C compiler you're using.  I expect that it is either disabled or not available in the VMs built on binary.  I say this because there is no occurrence of the string 'stack smashing detected' in our sources and so expect it is being produced by the C runtime on your version of linux.

The best description I found in a quick search is...
http://wiki.osdev.org/Stack_Smashing_Protector

This implies a canary variable is being overwritten, which implies(??)
a legitimate smashed-stack, with the culprit perhaps being some other
protection mechanism like address space layout randomisation.

Here is a detailed explanation including sample assembly code on the
second page.
http://www.drdobbs.com/security/anatomy-of-a-stack-smashing-attack-and-h/240001832

>
> The stack organization in the VM is a little unusual.  The VM reserves space for a set of "stack pages" upon which Smalltalk activations exist.  The space for the stack pages is allocated via alloca on the C stack at startup.  It could be that this strange organization, which means that at some times, when the VM is executing C, the stack pointer is at the bottom end of the C stack, but when it is executing JITted Smalltalk code the stack pointer is somewhere in the middle of the C stack.  I expect that the C runtime sees this happening and misinterprets it as an error.

@Eliot, could ASLR cause assumptions about this memory layout to be violated?



It could be related to EBP register "position independent executables"
http://forum.world.st/IMPORTANT-GCC-6-generates-position-independent-executables-by-default-on-Linux-td4935173.html

This article mentions both Stack Smashing and EBP register but I can't
exactly tie them together.
https://users.ece.cmu.edu/~vsekar/Teaching/Fall16/18487-f16/reading/Makowski_2011_Smashing%20the%20Stack%20in%202011.pdf

@Sophie,  Could you report your OS and compiler version here? (sorry
if you've already done this elsewhere)

cheers -ben

>
> I expect further that you can, and should, disable such checking.
>
> HTH
>
>
>>
>>
>> I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
>> Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
>> I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.
>>
>> Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
>> I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
>> I will also try to update some libs, just in case.
>>
>> In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.
>>
>> Sophie
>>
>>
>>
>> 2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:
>>>
>>>
>>> On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
>>> >
>>> >
>>> >> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>> >>
>>> >>
>>> >> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>> >>>
>>> >>>
>>> >>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>> >>>>
>>> >>>> Hi Tim,
>>> >>>>
>>> >>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>> >>>
>>> >>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>> >>>
>>> >>
>>> >> I have not checked in a while, but I am pretty sure that it is perfectly
>>> >> ok to have both an internally compiled plugin as part of the main VM
>>> >> executable, and also an externally compiled version of the same plugin
>>> >> (a loadable module in a file separate from the main VM executable). In
>>> >> that case, the external plugin should override the internal one.
>>> >>
>>> >> The idea is that it should be easy to make an external plugin that might
>>> >> add some new behaviour, and the plugin developer should be able to do
>>> >> this without waiting for somebody else to distribute a new VM.
>>> >>
>>> >> If this is not the case, then it certainly /should/ be. Tim, weren't
>>> >> you one of the people who invented this loadable module stuff in the
>>> >> first place?
>>> >
>>> > D’oh, of course you’re right. Oh the joys of aging…
>>> >
>>> > And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
>>> > So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
>>> > I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...
>>>
>>> Remember, just because you think you're paranoid doesn't mean they're
>>> not watching you.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

Ben Coman
 
On Sun, May 14, 2017 at 12:22 AM, Ben Coman <[hidden email]> wrote:

> On Sat, May 13, 2017 at 12:00 PM, Eliot Miranda <[hidden email]> wrote:
>>
>> Hi Sophie,
>>
>> On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]> wrote:
>>>
>>>
>>> Update :
>>>
>>> I did add the plugins to the file. That was the easy part, it went well.
>>>
>>> Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :
>>>
>>> - at first, I had an error : "can't find PharoV50.sources".
>
> You're the fifth newcomer compiling the VM that I've seen bump into
> this problem.
> The fix PR has been available since last October...
> https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/79
>
>
>>>
>>> (I cloned the opensmalltalk-vm directory,
>>>  ran updateSCCSVersions
>>> and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )
>>>
>>> Manually adding a "sources" directory containing the missing file solved the problem
>>> Did I miss something on that part?
>>>
>>>
>>> -OK, now, the VM is built with this small hack.
>>>
>>> When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :
>>>
>>> *** stack smashing detected ***: ./squeak terminated
>>> output file stack is full.
>>
>>
>> I wonder if this is some checking feature supported by the C compiler you're using.  I expect that it is either disabled or not available in the VMs built on binary.  I say this because there is no occurrence of the string 'stack smashing detected' in our sources and so expect it is being produced by the C runtime on your version of linux.
>
> The best description I found in a quick search is...
> http://wiki.osdev.org/Stack_Smashing_Protector
>
> This implies a canary variable is being overwritten, which implies(??)
> a legitimate smashed-stack, with the culprit perhaps being some other
> protection mechanism like address space layout randomisation.
>
> Here is a detailed explanation including sample assembly code on the
> second page.
> http://www.drdobbs.com/security/anatomy-of-a-stack-smashing-attack-and-h/240001832
>
>>
>> The stack organization in the VM is a little unusual.  The VM reserves space for a set of "stack pages" upon which Smalltalk activations exist.  The space for the stack pages is allocated via alloca on the C stack at startup.  It could be that this strange organization, which means that at some times, when the VM is executing C, the stack pointer is at the bottom end of the C stack, but when it is executing JITted Smalltalk code the stack pointer is somewhere in the middle of the C stack.  I expect that the C runtime sees this happening and misinterprets it as an error.
>
> @Eliot, could ASLR cause assumptions about this memory layout to be violated?
>
>
>
> It could be related to EBP register "position independent executables"
> http://forum.world.st/IMPORTANT-GCC-6-generates-position-independent-executables-by-default-on-Linux-td4935173.html
>
> This article mentions both Stack Smashing and EBP register but I can't
> exactly tie them together.
> https://users.ece.cmu.edu/~vsekar/Teaching/Fall16/18487-f16/reading/Makowski_2011_Smashing%20the%20Stack%20in%202011.pdf
>
> @Sophie,  Could you report your OS and compiler version here? (sorry
> if you've already done this elsewhere)

Whoops, I meant add a few other links for background rading...

* http://stackoverflow.com/questions/5863252/disabling-stack-smashing-protection-in-ubuntu-11-04
* https://unix.stackexchange.com/questions/46716/is-there-a-way-to-deactivate-buffer-overflow-protection-on-my-machine
* https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization
* https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization


>
> cheers -ben
>
>>
>> I expect further that you can, and should, disable such checking.
>>
>> HTH
>>
>>
>>>
>>>
>>> I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
>>> Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
>>> I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.
>>>
>>> Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
>>> I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
>>> I will also try to update some libs, just in case.
>>>
>>> In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.
>>>
>>> Sophie
>>>
>>>
>>>
>>> 2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:
>>>>
>>>>
>>>> On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
>>>> >
>>>> >
>>>> >> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>>> >>
>>>> >>
>>>> >> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>>> >>>
>>>> >>>
>>>> >>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>>> >>>>
>>>> >>>> Hi Tim,
>>>> >>>>
>>>> >>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>>> >>>
>>>> >>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>>> >>>
>>>> >>
>>>> >> I have not checked in a while, but I am pretty sure that it is perfectly
>>>> >> ok to have both an internally compiled plugin as part of the main VM
>>>> >> executable, and also an externally compiled version of the same plugin
>>>> >> (a loadable module in a file separate from the main VM executable). In
>>>> >> that case, the external plugin should override the internal one.
>>>> >>
>>>> >> The idea is that it should be easy to make an external plugin that might
>>>> >> add some new behaviour, and the plugin developer should be able to do
>>>> >> this without waiting for somebody else to distribute a new VM.
>>>> >>
>>>> >> If this is not the case, then it certainly /should/ be. Tim, weren't
>>>> >> you one of the people who invented this loadable module stuff in the
>>>> >> first place?
>>>> >
>>>> > D’oh, of course you’re right. Oh the joys of aging…
>>>> >
>>>> > And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
>>>> > So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
>>>> > I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...
>>>>
>>>> Remember, just because you think you're paranoid doesn't mean they're
>>>> not watching you.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

aa72
 
Hi, this info might be helpful.
From my OpenBSD porting efforts, currently:

.if ! ${PROPERTIES:Mlp64}
# Any combination of one or more will build a non-crashing vm.
#CC =                   clang
#CFLAGS +=              -fno-pie
MAKE_FLAGS +=           INTERP=cointerp
.endif


Stu

> ---------- Original Message ----------
> From: Ben Coman <[hidden email]>
> Date: May 13, 2017 at 12:26 PM
>
>
>  
> On Sun, May 14, 2017 at 12:22 AM, Ben Coman <[hidden email]> wrote:
> > On Sat, May 13, 2017 at 12:00 PM, Eliot Miranda <[hidden email]> wrote:
> >>
> >> Hi Sophie,
> >>
> >> On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]> wrote:
> >>>
> >>>
> >>> Update :
> >>>
> >>> I did add the plugins to the file. That was the easy part, it went well.
> >>>
> >>> Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :
> >>>
> >>> - at first, I had an error : "can't find PharoV50.sources".
> >
> > You're the fifth newcomer compiling the VM that I've seen bump into
> > this problem.
> > The fix PR has been available since last October...
> > https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/79
> >
> >
> >>>
> >>> (I cloned the opensmalltalk-vm directory,
> >>>  ran updateSCCSVersions
> >>> and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )
> >>>
> >>> Manually adding a "sources" directory containing the missing file solved the problem
> >>> Did I miss something on that part?
> >>>
> >>>
> >>> -OK, now, the VM is built with this small hack.
> >>>
> >>> When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :
> >>>
> >>> *** stack smashing detected ***: ./squeak terminated
> >>> output file stack is full.
> >>
> >>
> >> I wonder if this is some checking feature supported by the C compiler you're using.  I expect that it is either disabled or not available in the VMs built on binary.  I say this because there is no occurrence of the string 'stack smashing detected' in our sources and so expect it is being produced by the C runtime on your version of linux.
> >
> > The best description I found in a quick search is...
> > http://wiki.osdev.org/Stack_Smashing_Protector
> >
> > This implies a canary variable is being overwritten, which implies(??)
> > a legitimate smashed-stack, with the culprit perhaps being some other
> > protection mechanism like address space layout randomisation.
> >
> > Here is a detailed explanation including sample assembly code on the
> > second page.
> > http://www.drdobbs.com/security/anatomy-of-a-stack-smashing-attack-and-h/240001832
> >
> >>
> >> The stack organization in the VM is a little unusual.  The VM reserves space for a set of "stack pages" upon which Smalltalk activations exist.  The space for the stack pages is allocated via alloca on the C stack at startup.  It could be that this strange organization, which means that at some times, when the VM is executing C, the stack pointer is at the bottom end of the C stack, but when it is executing JITted Smalltalk code the stack pointer is somewhere in the middle of the C stack.  I expect that the C runtime sees this happening and misinterprets it as an error.
> >
> > @Eliot, could ASLR cause assumptions about this memory layout to be violated?
> >
> >
> >
> > It could be related to EBP register "position independent executables"
> > http://forum.world.st/IMPORTANT-GCC-6-generates-position-independent-executables-by-default-on-Linux-td4935173.html
> >
> > This article mentions both Stack Smashing and EBP register but I can't
> > exactly tie them together.
> > https://users.ece.cmu.edu/~vsekar/Teaching/Fall16/18487-f16/reading/Makowski_2011_Smashing%20the%20Stack%20in%202011.pdf
> >
> > @Sophie,  Could you report your OS and compiler version here? (sorry
> > if you've already done this elsewhere)
>
> Whoops, I meant add a few other links for background rading...
>
> * http://stackoverflow.com/questions/5863252/disabling-stack-smashing-protection-in-ubuntu-11-04
> * https://unix.stackexchange.com/questions/46716/is-there-a-way-to-deactivate-buffer-overflow-protection-on-my-machine
> * https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization
> * https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization
>
>
> >
> > cheers -ben
> >
> >>
> >> I expect further that you can, and should, disable such checking.
> >>
> >> HTH
> >>
> >>
> >>>
> >>>
> >>> I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
> >>> Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
> >>> I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.
> >>>
> >>> Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
> >>> I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
> >>> I will also try to update some libs, just in case.
> >>>
> >>> In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.
> >>>
> >>> Sophie
> >>>
> >>>
> >>>
> >>> 2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:
> >>>>
> >>>>
> >>>> On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
> >>>> >
> >>>> >
> >>>> >> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
> >>>> >>
> >>>> >>
> >>>> >> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
> >>>> >>>
> >>>> >>>
> >>>> >>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
> >>>> >>>>
> >>>> >>>> Hi Tim,
> >>>> >>>>
> >>>> >>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
> >>>> >>>
> >>>> >>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
> >>>> >>>
> >>>> >>
> >>>> >> I have not checked in a while, but I am pretty sure that it is perfectly
> >>>> >> ok to have both an internally compiled plugin as part of the main VM
> >>>> >> executable, and also an externally compiled version of the same plugin
> >>>> >> (a loadable module in a file separate from the main VM executable). In
> >>>> >> that case, the external plugin should override the internal one.
> >>>> >>
> >>>> >> The idea is that it should be easy to make an external plugin that might
> >>>> >> add some new behaviour, and the plugin developer should be able to do
> >>>> >> this without waiting for somebody else to distribute a new VM.
> >>>> >>
> >>>> >> If this is not the case, then it certainly /should/ be. Tim, weren't
> >>>> >> you one of the people who invented this loadable module stuff in the
> >>>> >> first place?
> >>>> >
> >>>> > D’oh, of course you’re right. Oh the joys of aging…
> >>>> >
> >>>> > And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
> >>>> > So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
> >>>> > I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...
> >>>>
> >>>> Remember, just because you think you're paranoid doesn't mean they're
> >>>> not watching you.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

SophieKaleba
In reply to this post by Ben Coman
 
Thanks all, I will have a look and have a look this evening.

@Ben : I was using ubuntu 15.10 (64bits) and gcc 5.1 when I got the errors.
@David : it could be caused by the plugin indeed... But these are default plugins of the squeak vm, and I guess I should have had the problem while using the profiler on the latest squeak vm but I didn't, hm... I will try anyway

I upgraded yesterday to ubuntu 16.04 and gcc 5.4. I just tried to build a pharo image but it failed due to some missing libs (due to the upgrade I guess).
So I will fix that first and then try out the suggested solutions.

 Sophie

2017-05-13 18:26 GMT+02:00 Ben Coman <[hidden email]>:

On Sun, May 14, 2017 at 12:22 AM, Ben Coman <[hidden email]> wrote:
> On Sat, May 13, 2017 at 12:00 PM, Eliot Miranda <[hidden email]> wrote:
>>
>> Hi Sophie,
>>
>> On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]> wrote:
>>>
>>>
>>> Update :
>>>
>>> I did add the plugins to the file. That was the easy part, it went well.
>>>
>>> Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :
>>>
>>> - at first, I had an error : "can't find PharoV50.sources".
>
> You're the fifth newcomer compiling the VM that I've seen bump into
> this problem.
> The fix PR has been available since last October...
> https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/79
>
>
>>>
>>> (I cloned the opensmalltalk-vm directory,
>>>  ran updateSCCSVersions
>>> and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )
>>>
>>> Manually adding a "sources" directory containing the missing file solved the problem
>>> Did I miss something on that part?
>>>
>>>
>>> -OK, now, the VM is built with this small hack.
>>>
>>> When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :
>>>
>>> *** stack smashing detected ***: ./squeak terminated
>>> output file stack is full.
>>
>>
>> I wonder if this is some checking feature supported by the C compiler you're using.  I expect that it is either disabled or not available in the VMs built on binary.  I say this because there is no occurrence of the string 'stack smashing detected' in our sources and so expect it is being produced by the C runtime on your version of linux.
>
> The best description I found in a quick search is...
> http://wiki.osdev.org/Stack_Smashing_Protector
>
> This implies a canary variable is being overwritten, which implies(??)
> a legitimate smashed-stack, with the culprit perhaps being some other
> protection mechanism like address space layout randomisation.
>
> Here is a detailed explanation including sample assembly code on the
> second page.
> http://www.drdobbs.com/security/anatomy-of-a-stack-smashing-attack-and-h/240001832
>
>>
>> The stack organization in the VM is a little unusual.  The VM reserves space for a set of "stack pages" upon which Smalltalk activations exist.  The space for the stack pages is allocated via alloca on the C stack at startup.  It could be that this strange organization, which means that at some times, when the VM is executing C, the stack pointer is at the bottom end of the C stack, but when it is executing JITted Smalltalk code the stack pointer is somewhere in the middle of the C stack.  I expect that the C runtime sees this happening and misinterprets it as an error.
>
> @Eliot, could ASLR cause assumptions about this memory layout to be violated?
>
>
>
> It could be related to EBP register "position independent executables"
> http://forum.world.st/IMPORTANT-GCC-6-generates-position-independent-executables-by-default-on-Linux-td4935173.html
>
> This article mentions both Stack Smashing and EBP register but I can't
> exactly tie them together.
> https://users.ece.cmu.edu/~vsekar/Teaching/Fall16/18487-f16/reading/Makowski_2011_Smashing%20the%20Stack%20in%202011.pdf
>
> @Sophie,  Could you report your OS and compiler version here? (sorry
> if you've already done this elsewhere)

Whoops, I meant add a few other links for background rading...

* http://stackoverflow.com/questions/5863252/disabling-stack-smashing-protection-in-ubuntu-11-04
* https://unix.stackexchange.com/questions/46716/is-there-a-way-to-deactivate-buffer-overflow-protection-on-my-machine
* https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization
* https://askubuntu.com/questions/318315/how-can-i-temporarily-disable-aslr-address-space-layout-randomization


>
> cheers -ben
>
>>
>> I expect further that you can, and should, disable such checking.
>>
>> HTH
>>
>>
>>>
>>>
>>> I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
>>> Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
>>> I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.
>>>
>>> Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
>>> I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
>>> I will also try to update some libs, just in case.
>>>
>>> In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.
>>>
>>> Sophie
>>>
>>>
>>>
>>> 2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:
>>>>
>>>>
>>>> On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
>>>> >
>>>> >
>>>> >> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>>> >>
>>>> >>
>>>> >> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>>> >>>
>>>> >>>
>>>> >>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>>> >>>>
>>>> >>>> Hi Tim,
>>>> >>>>
>>>> >>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>>> >>>
>>>> >>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>>> >>>
>>>> >>
>>>> >> I have not checked in a while, but I am pretty sure that it is perfectly
>>>> >> ok to have both an internally compiled plugin as part of the main VM
>>>> >> executable, and also an externally compiled version of the same plugin
>>>> >> (a loadable module in a file separate from the main VM executable). In
>>>> >> that case, the external plugin should override the internal one.
>>>> >>
>>>> >> The idea is that it should be easy to make an external plugin that might
>>>> >> add some new behaviour, and the plugin developer should be able to do
>>>> >> this without waiting for somebody else to distribute a new VM.
>>>> >>
>>>> >> If this is not the case, then it certainly /should/ be. Tim, weren't
>>>> >> you one of the people who invented this loadable module stuff in the
>>>> >> first place?
>>>> >
>>>> > D’oh, of course you’re right. Oh the joys of aging…
>>>> >
>>>> > And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
>>>> > So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
>>>> > I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...
>>>>
>>>> Remember, just because you think you're paranoid doesn't mean they're
>>>> not watching you.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: script to add internal plugins ?

Eliot Miranda-2
In reply to this post by Ben Coman
 
Hi Ben,

On Sat, May 13, 2017 at 9:22 AM, Ben Coman <[hidden email]> wrote:

On Sat, May 13, 2017 at 12:00 PM, Eliot Miranda <[hidden email]> wrote:
>
> Hi Sophie,
>
> On Thu, May 11, 2017 at 8:04 AM, Sophie Kaleba <[hidden email]> wrote:
>>
>>
>> Update :
>>
>> I did add the plugins to the file. That was the easy part, it went well.
>>
>> Then it got trickier when I tried to actually build the (Pharo) VM (build.linux32x86/pharo.cog.spur/build.itimerheatbeat) with my brand new plugins :
>>
>> - at first, I had an error : "can't find PharoV50.sources".

You're the fifth newcomer compiling the VM that I've seen bump into
this problem.
The fix PR has been available since last October...
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/79


>>
>> (I cloned the opensmalltalk-vm directory,
>>  ran updateSCCSVersions
>> and then went in the "build.linux32x86/pharo.cog.spur/build.itimerheatbeat" and ran ./mvm -f )
>>
>> Manually adding a "sources" directory containing the missing file solved the problem
>> Did I miss something on that part?
>>
>>
>> -OK, now, the VM is built with this small hack.
>>
>> When I try to use the VMProfiler (which works fine using the latest squeak VM from bintray), I get a segmentation fault, with this message :
>>
>> *** stack smashing detected ***: ./squeak terminated
>> output file stack is full.
>
>
> I wonder if this is some checking feature supported by the C compiler you're using.  I expect that it is either disabled or not available in the VMs built on binary.  I say this because there is no occurrence of the string 'stack smashing detected' in our sources and so expect it is being produced by the C runtime on your version of linux.

The best description I found in a quick search is...
http://wiki.osdev.org/Stack_Smashing_Protector

This implies a canary variable is being overwritten, which implies(??)
a legitimate smashed-stack, with the culprit perhaps being some other
protection mechanism like address space layout randomisation.

Here is a detailed explanation including sample assembly code on the
second page.
http://www.drdobbs.com/security/anatomy-of-a-stack-smashing-attack-and-h/240001832

>
> The stack organization in the VM is a little unusual.  The VM reserves space for a set of "stack pages" upon which Smalltalk activations exist.  The space for the stack pages is allocated via alloca on the C stack at startup.  It could be that this strange organization, which means that at some times, when the VM is executing C, the stack pointer is at the bottom end of the C stack, but when it is executing JITted Smalltalk code the stack pointer is somewhere in the middle of the C stack.  I expect that the C runtime sees this happening and misinterprets it as an error.

@Eliot, could ASLR cause assumptions about this memory layout to be violated?

I don't think so.  Given that the Stack Zone is alloca'ed on the C stack it effectively stays in the same relative position to the C stack.
 
It could be related to EBP register "position independent executables"
http://forum.world.st/IMPORTANT-GCC-6-generates-position-independent-executables-by-default-on-Linux-td4935173.html

That's an interesting thought.  There is code executed at start-up (see platforms/Cross/vm/sqCogStackAlignment.h: extern sqInt cFramePointerInUse; platforms//unix/vm/sqUnixMain.c:isCFramePointerInUse()) t6hat attempts to find out if the frame pointer is in use and there-fore whether the glue code between the JIT and the C run-time should save and restore the frame pointer or not.

This article mentions both Stack Smashing and EBP register but I can't
exactly tie them together.
https://users.ece.cmu.edu/~vsekar/Teaching/Fall16/18487-f16/reading/Makowski_2011_Smashing%20the%20Stack%20in%202011.pdf

@Sophie,  Could you report your OS and compiler version here? (sorry
if you've already done this elsewhere)

cheers -ben

Sophie, Ben, David, regarding the stack smashing, I did notice that there is a variable intended to stop recursive error reporting within the sigsegv handler (which also handles SIGBUS and SIGILL) that was not being set when an error occurred.  This would have indeed causes an infinite recursion if the error reporting was, for example, fooled by the use of EBP as a general purpose register into crashing while trying to report an error.  I committed the fix, and fixed the Mac VM.  But we should still investigate exactly what the recursive crash is.  BTW, the variable is 

oscogvm$ grep inFault platforms/unix/vm/* platforms/iOS/vm/Common/Classes/*
platforms/unix/vm/sqUnixMain.c:static int inFault = 0;
platforms/unix/vm/sqUnixMain.c: if (!inFault) {
platforms/unix/vm/sqUnixMain.c: inFault = 1;
platforms/iOS/vm/Common/Classes/sqSqueakMainApp.m:static int inFault = 0;
platforms/iOS/vm/Common/Classes/sqSqueakMainApp.m: if (!inFault) {
platforms/iOS/vm/Common/Classes/sqSqueakMainApp.m: inFault = 1;


>
> I expect further that you can, and should, disable such checking.
>
> HTH
>
>
>>
>>
>> I already ran into this error when I first tried to run the VMProfiler a few weeks ago. At that time, I was using a built Squeak VM (still linux32x86 , cog.spur, itimer).
>> Seeing that I got the same problem building both pharo and squeak vm (and not having it with vm I haven't actually built), I guess it has something to do with my environment.
>> I was told it could be caused by a near-full disk, but I still have 8Gb+ of free space so, well.
>>
>> Does somebody have any idea about what could cause this error ? It would actually help me narrowing down the possibilities...
>> I am using ubuntu 15.10 (old) so I will update it later today and check if it solves the problem.
>> I will also try to update some libs, just in case.
>>
>> In the meantime, i will make a PR with the modified plugins files anyway, and see if it gets rejected or not.
>>
>> Sophie
>>
>>
>>
>> 2017-05-11 3:18 GMT+02:00 Ben Coman <[hidden email]>:
>>>
>>>
>>> On Thu, May 11, 2017 at 8:08 AM, tim Rowledge <[hidden email]> wrote:
>>> >
>>> >
>>> >> On 10-05-2017, at 4:57 PM, David T. Lewis <[hidden email]> wrote:
>>> >>
>>> >>
>>> >> On Wed, May 10, 2017 at 02:36:20PM -0700, tim Rowledge wrote:
>>> >>>
>>> >>>
>>> >>>> On 10-05-2017, at 1:47 PM, Sophie Kaleba <[hidden email]> wrote:
>>> >>>>
>>> >>>> Hi Tim,
>>> >>>>
>>> >>>> I hadn't thought about that, thanks ! I will make sure not to create any duplicate of these plugins
>>> >>>
>>> >>> I???ve never thought to experiment and see what happens if one tried to make both internal and external versions of the same plugin. I???d guess that it might not cause any problems in most cases.. hmm, unless on some platform you actually have to compile with different flags, and then maybe there???d be problems if the internal version were compiled with flag A, the ???.o??? file were left around, the external version compile process found it, therefore didn???t compile with flag B, thus making something fail?
>>> >>>
>>> >>
>>> >> I have not checked in a while, but I am pretty sure that it is perfectly
>>> >> ok to have both an internally compiled plugin as part of the main VM
>>> >> executable, and also an externally compiled version of the same plugin
>>> >> (a loadable module in a file separate from the main VM executable). In
>>> >> that case, the external plugin should override the internal one.
>>> >>
>>> >> The idea is that it should be easy to make an external plugin that might
>>> >> add some new behaviour, and the plugin developer should be able to do
>>> >> this without waiting for somebody else to distribute a new VM.
>>> >>
>>> >> If this is not the case, then it certainly /should/ be. Tim, weren't
>>> >> you one of the people who invented this loadable module stuff in the
>>> >> first place?
>>> >
>>> > D’oh, of course you’re right. Oh the joys of aging…
>>> >
>>> > And what I *should* have said was that I haven’t experimented with making internal and external versions within the same build process, given faint concerns about the object files maybe confusing the matter if one version is built and (incorrect) object files are left for the next version make-rule to find.
>>> > So er, something like build BlobbyPlugin.c -> BlobbyPlugin.o -> link to make vm with internal BlobbyPlugin followed almost immediately by build BlobbyPlugin.c -> ooh, already got BlobbyPlugin.o -> make library BlobbyPlugin.la
>>> > I’m imagining a situation where BlobbyPlugin.o actually ought to have been (re)compiled with subtly different flags in order to make a correct library. I may be being paranoid. Or maybe the OS is actually out to get me...
>>>
>>> Remember, just because you think you're paranoid doesn't mean they're
>>> not watching you.


_,,,^..^,,,_
best, Eliot
Loading...