[UPD] NativeBoost runs on new Cog VMs

Hello,
i just wanna to inform you that i adopted NativeBoost plugin for
building with latest Cog vms.

To build VM, you should have an image with VMMaker + CMakeVMMaker
packages installed.

Then load NBIntaller package and do:
NBInstaller installCogPlugin

There are new cmake configurations i added to build VMs with NB plugin support.

So, simply do as usual:
<NB config name> generateWithSources

and then  cmake. / make

P.S. Im happy knowing that the OpenGL fonts rendering demo, which i
made almost a year ago works out of the box on Windoze :)

--
Best regards,
Igor Stasenko AKA sig.

Hi Igor,

Exciting :)

Cheers,
Doru


On 20 Mar 2011, at 12:51, Igor Stasenko wrote:

--
www.tudorgirba.com

"There are no old things, there are only old ways of looking at them."

On 20 March 2011 17:18, Eliot Miranda <[hidden email]> wrote:
erm.. in terms of time, like 10 minutes to check everything and add
new configurations for building VM with it.
It took more time to build VM and verify it works :)

So its not much.. All code which i did before works out of the box.
The code generator using InterpreterProxy API, and since it hard to
believe it could change from one VM to another,
it is quite stable.

Of course language callbacks are not functioning. But that was expected.



--
Best regards,
Igor Stasenko AKA sig.

On 21 March 2011 02:43, Levente Uzonyi <[hidden email]> wrote:
It is disabled by default, when you starting a new image.
This allows me to not care if some of the methods were carrying native
code saved into an image.

For enabling a native code, one should invoke #primitiveEnable first.
You can customize a mechanism of enabling it in
NativeBoost class>>enableNativeCode
and put checking of command-line argument(s) there.

And yes, i am against putting extra logic at plugin side :)
The reason is simple: if you using it, then you know what you doing,
and if you don't - then you safe anyways.

Also, plugin can be made external, if you want.
But there is certainly some intimate connection with VM, so i wouldn't
recommend making it external, because it can easy break things,
if you mismatch VM/plugin pairs.

I am also looking forward for making JITted code to make direct calls
to NB native code.
In a way, that  JIT code would not call #primitiveNativeCall (and only
then enter native code) but instead directly call the native code
which method contains.

It will , of course,  bypass all logic placed in #primitiveNativeCall,
like checking if NB is enabled etc.. and could be dangerous..
but the speed will be... :)

In a longer perspective this means that eventually JIT compiler could
be moved from VM to language side (like Exupery does).


--
Best regards,
Igor Stasenko AKA sig.

On 21.03.2011, at 03:12, Igor Stasenko wrote:

IMHO it's a good idea to not allow native code execution if the VM sandbox is enabled. Just like e.g. OSProcess checks the SecurityPlugin for this.

- Bert -

On 21 March 2011 11:54, Bert Freudenberg <[hidden email]> wrote:
Bert, but you cannot state that your deployment are more secure just
because you are passing "right" command-line arguments to VM.
As i said before , if you want security for real, then you should
build own "sandboxed" version of VM, which simply having no way to use
such features.
Otherwise that's just a painting on the water pool.

> - Bert -
>

--
Best regards,
Igor Stasenko AKA sig.

On 21.03.2011, at 12:42, Igor Stasenko wrote:

I didn't say anything about command line arguments.

> As i said before , if you want security for real, then you should
> build own "sandboxed" version of VM, which simply having no way to use
> such features.

I disagree. There should be a single version of the VM that can be used in almost all circumstances. E.g. there should be a single Linux package for the VM. Possibly three for interpreter/stack/cog, but it makes no sense at all to have more than one interpreter, IMHO.

It would be a shame if e.g. Etoys would have to ship its own VM because you made it less secure in newer versions. After all, it was Etoys that convinced all the major Linux distros to even include a VM.

> Otherwise that's just a painting on the water pool.

I don't quite get that analogy ;)

And we're not after absolute security, which can't be achieved anyway. But why taking a step back from where we are already?

- Bert -

On 21 March 2011 13:03, Bert Freudenberg <[hidden email]> wrote:
It sounds as contradiction to me. From one side you want to make VM
being responsible for certain level of security, but from another
side, you want it to be open for "insecure" stuff like NativeBoost.
Apparently you can't have both at once. You should choose one.
Unless you change VM to disable external module loading mechanism,
there is no any security guarantees, only imaginable ones.
I could download plugin from web, place it into 'secure' directory and
then tell VM to load it as external plugin. And where is your
command-line option(s) now?

For things like NativeBoost, it is responsibility of developer(s) ,
what level of security is desirable/enough for them.
In this regard, VM is just a tool which developers using.

And may i ask, why you don't want to put a security measures at
language side? You can disable or completely remove compiler. You can
disable a primitives which allow doing "nasty" stuff,
and of course you can make it controllable from command line. And
because it is at language side, it is much easier to maintain and fix
and develop in general. So, why not?

I am really enjoying  that smalltalk has a wide open architecture, but
at the same time it missing one simple thing: a 'deployment' mode,
where you have certain security guarantees out of the box.
So that's an open question, why we never had that in Squeak.. VM
cannot do such magic, it is too stupid and cannot really tell if given
sequence of bytecodes/class format/external plugin are secure enough
or not.
This logic is best to be put at image side where you can reason about
it much easier because you have reflection ( and smalltalk instead of
C :)

> It would be a shame if e.g. Etoys would have to ship its own VM because you made it less secure in newer versions. After all, it was Etoys that convinced all the major Linux distros to even include a VM.
>

No no no.. Please stop looking at VM as a black-box with two inputs
and one TV output. This approach is too naive and will never pass any
real-world testing, moreover it hinders any progress towards improving
state of art.
If you approach with our "standard" VM to some serious guys, first
thing they will do is to ask a security expert to evaluate it..
And then any your proposal which is based on such technology will be
rejected instantly, and i fear that project budget will be spent on
making same thing but in java.

So, it is a shame that we don't have a good answer to tough guys, and
instead of really looking how to make system sound and secure, we're
like sitting ducks with all these command-line options, which can be
cracked by
some clever guy in few minutes.

In my vision, VM should be made a dynamically loadable library with
nice API to interact with host application. Currently we have a
monolithic VM, which is host and library in one. But in future i'd
like (and not just me btw) to have
VM which can be easily embedded into a host application, and then host
application can control security as well as many other aspects.

For things like browser plugin, you simply can't use same things for
everything. And so, we should provide a way for making it easy to
customize VM for own purpose(s),
but at same time make it flexible enough to make forking pointless.

And that's exactly why we building an infrastructure for VMs: it will
enable us to build own custom VMs in much easier, much less painful
and much more controllable way.

And for Etoys this is a way to go: you can share the same code base,
but in own corner you could have mods, which will disable or stub-out
unwanted/insecure functionality.
In that way you can be confident that application you distributing is secure,
and instead of arguing with me (or some other VM developer guy) what
is best for you, you can do it yourself without much stress. :)

Just don't tell me that you don't want this, or you never considered
it as a good perspective. ( I won't believe you anyways ;)


>> Otherwise that's just a painting on the water pool.
>
> I don't quite get that analogy ;)
>

Standing on the bank of river and trying to paint on water stream.

> And we're not after absolute security, which can't be achieved anyway. But why taking a step back from where we are already?
>
> - Bert -
>



--
Best regards,
Igor Stasenko AKA sig.

On 21.03.2011, at 14:09, Igor Stasenko wrote:

> It sounds as contradiction to me. From one side you want to make VM
> being responsible for certain level of security, but from another
> side, you want it to be open for "insecure" stuff like NativeBoost.
> Apparently you can't have both at once. You should choose one.
> Unless you change VM to disable external module loading mechanism,
> there is no any security guarantees, only imaginable ones.
> I could download plugin from web, place it into 'secure' directory and
> then tell VM to load it as external plugin.

If the VM sandbox is enabled, file access is restricted to a single directory. So you cannot write the plugin to the plugin directory.

(if there are problems with the current implementation, we should fix those, obviously)

>  And where is your command-line option(s) now?

This has nothing to do with command line options. The sandbox is enabled by a primitive. Etoys enables it before downloading code. There is no way to disable it once engaged.

> For things like NativeBoost, it is responsibility of developer(s) ,
> what level of security is desirable/enough for them.
> In this regard, VM is just a tool which developers using.

Yes, but the virtual machine defines the boundary

> And may i ask, why you don't want to put a security measures at
> language side? You can disable or completely remove compiler.

Not if you want to allow the user to write code.

> You can
> disable a primitives which allow doing "nasty" stuff,
> and of course you can make it controllable from command line. And
> because it is at language side, it is much easier to maintain and fix
> and develop in general. So, why not?

That is precisely what the SecurityPlugin is designed to do. It is very simple, but all VM extensions (like NativeBoost) would have to use it.

The VM is the interface between Smalltalk code and the outside world. Traditionally we have been keeping this interface as small as possible. It's an ideal place to ensure that bad things won't happen. VM extensions like FFI, OSProcess, NativeBoost allows code to circumvent the safe environment of the VM.

By widening the VM interface you make security analysis even harder. Or even unnecessary, by allowing FFI or arbitrary native code there is no security anyway.

> So, it is a shame that we don't have a good answer to tough guys, and
> instead of really looking how to make system sound and secure, we're
> like sitting ducks with all these command-line options, which can be
> cracked by
> some clever guy in few minutes.

I have no idea what your beef with command-line options is. It's a red herring anyway. Etoys does not use security-related command-line options.

> In my vision, VM should be made a dynamically loadable library with
> nice API to interact with host application. Currently we have a
> monolithic VM, which is host and library in one. But in future i'd
> like (and not just me btw) to have
> VM which can be easily embedded into a host application, and then host
> application can control security as well as many other aspects.

So I need a C compiler to build my apps instead of doing it in Smalltalk? No thank you. I do not share that vision.

> For things like browser plugin, you simply can't use same things for
> everything.

I'm sorry, but that's bullshit. The browser plugin executes the same VM binary as a standalone app.

> And so, we should provide a way for making it easy to
> customize VM for own purpose(s),
> but at same time make it flexible enough to make forking pointless.

Precisely. And what I want is run-time customization, not compile-time.

> And that's exactly why we building an infrastructure for VMs: it will
> enable us to build own custom VMs in much easier, much less painful
> and much more controllable way.

I very much like the build infrastructure you are setting up. It's very useful for development, for working together, for shorter turn-around times etc.

But that is totally unrelated to the question how many different Linux packages there should be for the VM. Because those packages are not built by your Hudson installation. They are built by the Fedora maintainer, the Debian packager, etc.

Do you know how many different Python interpreters there are in a typical Linux distro?

> And for Etoys this is a way to go: you can share the same code base,
> but in own corner you could have mods, which will disable or stub-out
> unwanted/insecure functionality.

It is reasonably simple to just have a flag in the VM to disable the insecure parts. Why would I need a C compiler to do that? Why can't we share even more than just the code base?

> In that way you can be confident that application you distributing is secure,
> and instead of arguing with me (or some other VM developer guy) what
> is best for you, you can do it yourself without much stress. :)

So this is you telling me to just shut up? When all I am asking for is to make things configurable at runtime (in true Smalltalk spirit) rather than having to compile my own VM? I'm not arguing against native boost or FFI or the like, but simply for an option to disable it at runtime.

> Just don't tell me that you don't want this, or you never considered
> it as a good perspective. ( I won't believe you anyways ;)

Why wouldn't you believe it? I went to *considerable* effort to *not* bundle a VM with Etoys. That would have been much easier for me indeed, and in fact we used to bundle a Linux VM until about 3 years ago. I then built the original Etoys RPM so that it depends on a VM RPM. And thanks to that effort (and people doing the packaging work) today we have Etoys and VM packages in many Linux distros:

        http://pkgs.org/package/squeak-vm

Call me a dreamer, but I find it important to being a good player in the wider open-source community. And that community has evolved considerably from "download the sources and configure it to taste and compile your own". A single binary package is standard nowadays. You don't really recompile Firefox to change a preferences, do you?

- Bert -

On 21 March 2011 15:34, Bert Freudenberg <[hidden email]> wrote:
i can hardly imagine a sandbox which allows to write files on file system.
How much you allowed to write? What if it will just consume all disk space?

But ok. I understand that there are certain security model exists in VM.
One problem with SecurityPlugin that its not really a plugin but
instead an integral part of VM, because it is hardwired with many
other plugins who using it.

I wonder why each of the involved plugins don't implement own,
in-place security mechanisms instead.

> (if there are problems with the current implementation, we should fix those, obviously)
>

Well, if you can crash VM using simple #become, or infinite
recursion.. who cares about the rest? :)

>>  And where is your command-line option(s) now?
>
> This has nothing to do with command line options. The sandbox is enabled by a primitive. Etoys enables it before downloading code. There is no way to disable it once engaged.
>

How about this:
 - a single primitive which once activated, freezes the VM's plugin
set. No new plugins can be loaded/initialized (even internal ones).

Then at startup time, first thing you do is checking if you have all
plugins you need for your application to work, and for the rest you
don't care and simply disabling plugin loading mechanism.
In this way, VM could carry anything, but you have a precise control
what you want to use.


>> For things like NativeBoost, it is responsibility of developer(s) ,
>> what level of security is desirable/enough for them.
>> In this regard, VM is just a tool which developers using.
>
> Yes, but the virtual machine defines the boundary
>
For me, as a developer i prefer an OS to be the boundary, not VM.

>> And may i ask, why you don't want to put a security measures at
>> language side? You can disable or completely remove compiler.
>
> Not if you want to allow the user to write code.
>

You can control what you compiling. For instance in sandbox mode you
can simply reject compiling code which having primitive invocations.

>> You can
>> disable a primitives which allow doing "nasty" stuff,
>> and of course you can make it controllable from command line. And
>> because it is at language side, it is much easier to maintain and fix
>> and develop in general. So, why not?
>
> That is precisely what the SecurityPlugin is designed to do. It is very simple, but all VM extensions (like NativeBoost) would have to use it.
>
That approach won't scale.
Patching SecurityPlugin every time someone implements new plugin... it
won't work.
IMO individual plugins should provide their own security control
mechanisms, if necessary.
And "keeping as small as possible" to me sounds "do not put extra
complexity to VM, if you can do it at language side". That what i was
trying to say.

Hmm.. where you draw conclusion that i am for widening VM interface?
In contrary i am against it, i am for putting things at language side,
and leave only essential ones at VM.

>> So, it is a shame that we don't have a good answer to tough guys, and
>> instead of really looking how to make system sound and secure, we're
>> like sitting ducks with all these command-line options, which can be
>> cracked by
>> some clever guy in few minutes.
>
> I have no idea what your beef with command-line options is. It's a red herring anyway.

It was proposed by Levente up in this thread to "control" security by
command-line argument(s).

> Etoys does not use security-related command-line options.
>

Yeah. Good. :)

>> In my vision, VM should be made a dynamically loadable library with
>> nice API to interact with host application. Currently we have a
>> monolithic VM, which is host and library in one. But in future i'd
>> like (and not just me btw) to have
>> VM which can be easily embedded into a host application, and then host
>> application can control security as well as many other aspects.
>
> So I need a C compiler to build my apps instead of doing it in Smalltalk? No thank you. I do not share that vision.
>

Heh.. neither do i. But world is imperfect and we have to make some compromises.

>> For things like browser plugin, you simply can't use same things for
>> everything.
>
> I'm sorry, but that's bullshit. The browser plugin executes the same VM binary as a standalone app.
>

You cannot use same thing in every possible scenario. Browser was just
a first example came to my mind.


>> And so, we should provide a way for making it easy to
>> customize VM for own purpose(s),
>> but at same time make it flexible enough to make forking pointless.
>
> Precisely. And what I want is run-time customization, not compile-time.
>

So, again, why a language-side runtime customization is not good enough for you?


Never interested.. Just asked around , people counting 4 or 5
different interpreters.
But i don't care about standalone interpreter/VM.
Now take a look at Lua.

>> And for Etoys this is a way to go: you can share the same code base,
>> but in own corner you could have mods, which will disable or stub-out
>> unwanted/insecure functionality.
>
> It is reasonably simple to just have a flag in the VM to disable the insecure parts. Why would I need a C compiler to do that? Why can't we share even more than just the code base?
>

No. I'm not taking about disabling it. I am talking about _not_
including it. You cannot use something which is not there, and
therefore you don't have to put extra security measures around it.
Just applying the Principle of least privilege.
(http://en.wikipedia.org/wiki/Principle_of_least_privilege)


>> In that way you can be confident that application you distributing is secure,
>> and instead of arguing with me (or some other VM developer guy) what
>> is best for you, you can do it yourself without much stress. :)
>
> So this is you telling me to just shut up?

No. I just telling what i thinking would be best in terms of security.
I meant that if you can control it by yourself without asking people
to do(or not do) something, then you are in much better position.
And of course i didn't meant to insult you.


>When all I am asking for is to make things configurable at runtime (in true Smalltalk spirit) rather than having to compile my own VM? I'm not arguing against native boost or FFI or the like, but simply for an option to disable it at runtime.
>

Point taken.

>> Just don't tell me that you don't want this, or you never considered
>> it as a good perspective. ( I won't believe you anyways ;)
>
> Why wouldn't you believe it? I went to *considerable* effort to *not* bundle a VM with Etoys. That would have been much easier for me indeed, and in fact we used to bundle a Linux VM until about 3 years ago. I then built the original Etoys RPM so that it depends on a VM RPM. And thanks to that effort (and people doing the packaging work) today we have Etoys and VM packages in many Linux distros:
>
>        http://pkgs.org/package/squeak-vm
>
> Call me a dreamer, but I find it important to being a good player in the wider open-source community. And that community has evolved considerably from "download the sources and configure it to taste and compile your own". A single binary package is standard nowadays. You don't really recompile Firefox to change a preferences, do you?
>

So, you are at another side of extreme. You want a universal binary
which solves all of your problems. But it is illusion: it certainly
can't solve all problems of all people. What works for you may not
work for others and vise verse.
I see nothing bad in allowing people to deviate from "standard" VM, as
long as they stay in community and contribute userful pieces back.
So, i want both ways to be possible: we could have universal (kind of)
VM, but should also make sure its easy to customize VM in case of
need.


Thanks for discussion.
(I am really interested in what you think about providing a feature to
"freeze" plugin set, which i described above).

--
Best regards,
Igor Stasenko AKA sig.