Suggestions for C++ or DLL code

Hi--

        John writes:

> Yes yes, someday I promised making a list of the entry points, however
> workload seems to be stalling that event. Perhaps someone would be
> interested?

        I also intend to do this eventually. I would do it now if it were paid.


-C

--
Craig Latta
http://netjam.org/resume

nicolas cellier wrote:
> FFI is unsafe of course as C code is unsafe.

No, that's only true for plugins. For the FFI it's a lot worse. Even if
you have a perfectly safe function foo in bar.dll how do you prevent a
malicious attacker instead to, e.g., execute "format c:" from
"system.dll"? The FFI won't care.

> But you have to explain why a plugin would be safer.

A plugin provides access that has been explicitly granted. You write a
primitive in a plugin to provide access to that function and NO MORE.
Without the FFI for example, it is not possible to divert any of the
plugins to do the moral equivalent of "format c:". For plugins your
statement is correct - the plugin is as safe as the code it uses and if
there is a bug in the underlying C code that bug might be exploited. But
beyond that, there is no authority that a plugin grants.

> In the plugin case, user has to deal with pointers on data too, he has to do
> conversions by hands where FFI would automate part of this process and he has
> to be extremely carefull with object creation that might break previously
> collected pointers in Smalltalk space.

This is true in both situations. Unless you consider utterly trivial
examples, managing complex data structures using the FFI is a nightmare
- one of the reasons why I prefer to deal with them plugin-style.

> So, the plugin are maybe safer because only a few well experienced and well
> behaved programmers dare jump the hurd, in a word because it is rarely
> used...

Given the amount of plugins and the amount of FFI usage I think plugin
usage outnumbers FFI usage by an order of magnitude at least.

Cheers,
   - Andreas

John M McIntosh wrote:
> Ah, I'll note that the squeak VM really hasn't been hardened against
> attack, it's much less paranoid than the VW VM.
> In many places we might pass a ByteArray and a length, where the length
> is calculated from the ByteArray in Smalltalk however
> nothing prevents someone from making that VM call with a bogus ByteArray
> and length and see if something interesting will happen.

Which places are that?

Cheers,
   - Andreas

mmm, in looking it seems we check length versus data in all the  
places a cursory scan show, but.

What if I pass -1 as width to
primitiveWrite24BmpLine

or
perhaps pass something odd to
primAECoerceDesc: typeCode to: result.



On 31-Jul-06, at 10:49 PM, Andreas Raab wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

John M McIntosh wrote:
> mmm, in looking it seems we check length versus data in all the places a
> cursory scan show, but.
>
> What if I pass -1 as width to primitiveWrite24BmpLine

Yes, this would cause problems. Report and fix available at:

     http://bugs.impara.de/view.php?id=4360

> or perhaps pass something odd to
> primAECoerceDesc: typeCode to: result.

Don't know. Given that this is Applescript it should probably be treated
like the FFI, e.g., not be used in any environment where we care about
random access to the OS by untrusted code.

Cheers,
   - Andreas

On Tue, 01 Aug 2006 00:19:00 +0200, John M McIntosh wrote:
...
> Really *all* VM entry point would need to be looked at in a proper audit  
> to avoid buffer overflow issues, even perhaps accidents which generally  
> are fatal.

If InterpreterSimulatorXyz where up-to-date and usable, whatever the VM  
uses from the host OS can easily be looked up. The list below is from a  
3.7-5989 simulation run by a 3.7-1 VM (itself ;-) which was compiled with  
the Goya extension. And if it had TestCases for the full VM->OS interface,  
the list would've been complete.

/Klaus

---------------------

InterpreterSimulator 30 ioGetNextEvent: 1 1
        by Interpreter>>#primitiveGetNextEvent
InterpreterSimulator 30 ioSeconds 4 1
        by Interpreter>>#primitiveSecondsClock
InterpreterSimulator 30 ioMSecs 4 1
        by Interpreter>>#checkForInterrupts
        by Interpreter>>#primitiveExternalCall
        by Interpreter>>#primitiveMillisecondClock
InterpreterSimulator 30 ioSetInputSemaphore: 1 1
        by Interpreter>>#primitiveInputSemaphore
InterpreterSimulator 30 vmPathSize 4 1
        by Interpreter>>#primitiveVMPath
InterpreterSimulator 30 showDisplayBits:w:h:d:left:right:top:bottom: 6 1
        by Interpreter>>#displayBitsOf:Left:Top:Right:Bottom:
                actually ioShowDisplay(...)
InterpreterSimulator 30 ioMicroMSecs 4 1
        by ObjectMemory>>#fullGC
        by ObjectMemory>>#incrementalGC
InterpreterSimulator 30 ioHasDisplayDepth: 4 1
        by Interpreter>>#primitiveTestDisplayDepth
InterpreterSimulator 30 sqMemoryExtraBytesLeft: 0 1
        by ObjectMemory>>#bytesLeft:
InterpreterSimulator 30 vmPathGet:Length: 5 1
        by Interpreter>>#primitiveVMPath
InterpreterSimulator 30 ioLowResMSecs 4 1
        by Interpreter>>#primitiveExternalCall
BitBltSimulation 31 lockSurfaces 12 1
        by self>>#copyBits
        by self>>#warpBits
InterpreterSimulator 30 sqFileAtEnd: 4 2
        by FilePlugin>>#primitiveFileAtEnd
                actually sqFileAtEnd()
InterpreterSimulator 30 primitiveFileOpen 15 2
        by StandardFileStream>>#primOpen:writable:
InterpreterSimulator 30 sqFileClose: 4 2
        by FilePlugin>>#primitiveFileClose
InterpreterSimulatorLSB 30 nextLongFrom: 12 2
        via Interpreter>>#readImageFromFile:HeapSize:StartingAt:

Klaus,

Thank you for the starting point.

Ron Teitelbaum

> From: Klaus D. Witzel
> Sent: Tuesday, August 01, 2006 8:35 AM

Am 01.08.2006 um 07:43 schrieb Andreas Raab:

> nicolas cellier wrote:
>
>> So, the plugin are maybe safer because only a few well experienced  
>> and well behaved programmers dare jump the hurd, in a word because  
>> it is rarely used...
>
> Given the amount of plugins and the amount of FFI usage I think  
> plugin usage outnumbers FFI usage by an order of magnitude at least.

Indeed. Just do a "Smalltalk listLoadedModules" to see which plugins  
you are using *right now*.

You can think of plugins as an extension of the VM, so in principle  
they can be made as safe as the VM itself is - as long as each  
primitive they provide has one specific function that can be checked  
for inputs and side-effects. This is not the case for the FFI  
primitives.

- Bert -

On 1-Aug-06, at 5:34 AM, Klaus D. Witzel wrote:

> On Tue, 01 Aug 2006 00:19:00 +0200, John M McIntosh wrote:
> ...
>> Really *all* VM entry point would need to be looked at in a proper  
>> audit to avoid buffer overflow issues, even perhaps accidents  
>> which generally are fatal.
>
> If InterpreterSimulatorXyz where up-to-date and usable,

If *anybody* has time and interest available to improve the  
simulator, please feel free to work on it. My time for that  
particular corner is currently zero.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: QVC: Question Valid Command

All,

I'll have to preface this comment with "I JUST GOT FFI WORKING" but consider
the potential risk of FFI compared to any other external communication
mechanisms.  

"IF" the image is secured so that no changes can be made to a running image
which I suppose would include calls to FFI <...> (he says not really knowing
how the calls are made or interpreted) then there are benefits to direct
calls.  

First the implementation of the dll wrapper would not be in question only
the direct calls to the OS provided calls would need to be reviewed.  

Second securing the image by removing FFI is too drastic in my opinion.  How
will we get Crystal Reports or any other direct API to work without it?

Third support for external API's are simplified for developers by having the
calls in the image not wrapped in a DLL.  Everything is much more
transparent.

Forth there is not much difference in terms of risk to the system between
the execution of the DLL function and the FFI call if parameters are passed
in.  Both methods need to have proper handling for error conditions before
the primitive call.  I'm not convinced that error checking is better in a
DLL then in Smalltalk and on the contrary if we make the error checking
harder to apply (by pushing it into the DLL) it is less likely that changes
will be made to prevent the error condition.

Ron Teitelbaum

Totally agree

The same level of carefull precondition testing can be achieved in Smalltalk
and FFI than through plugin.

The major difference is that with FFI, we cannot prevent user to bypass those
protections and precondition tests.

But if the user also have access to plugin source code, you cannot prevent
either... The only difference is that it is more difficult to program the
plugin than FFI, and that is why it is safer in a way... Also, i am not sure
if we can modify the plugin without restarting the image... Can we?
If not, that's a good point for plugins and security.

If we are lazy, both FFI and plugin implementations will have security holes.
But maybe, code will be corrected faster in a FFI framework.

I have no experience with Squeak plugin, but practiced both user primitives
and DLLCC in VW. And the latter gave me more flexibility and rapid
development than the former (except for cross machine portability for which
good old dirty C preprocessor directives are somehow more efficient).

I would encourage FFI because it does open Squeak on the external world. Of
course it does scratch the "do it all in Smalltalk" paradigm, but from a
strategic point of view, opening Squeak is really something needed if you
want to convince more people to join (think of how easy it is in python to
call external code).

And sorry, the plugin framework is not simple enough. You have to learn a new
language that is neither Smalltalk, nor C, you have to go through code
generation for generating the wrapper function to call your external one,
then you are on your own to compile the wrapper code and integrate it in the
image... Rather a long development cycle for Smalltalkers. That is only my
opinion, and of course, good arguments can make it change.

Nicolas


Le Mardi 01 Août 2006 18:19, Ron Teitelbaum a écrit :

Another point to consider is that a plugin run at the same
 speed as C code and can't be optimized dynamically
 by a Jitter implementation.
imho enhancements on the VM will apply to clients
 of FFI, but will not be applicable to plugins.
Ale.



----- Original Message -----
From: "nicolas cellier" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Tuesday, August 01, 2006 3:14 PM
Subject: Re: FFI vs DLL (was Suggestions for C++ or DLL code)


Totally agree

The same level of carefull precondition testing can be achieved in Smalltalk
and FFI than through plugin.

The major difference is that with FFI, we cannot prevent user to bypass
those
protections and precondition tests.

But if the user also have access to plugin source code, you cannot prevent
either... The only difference is that it is more difficult to program the
plugin than FFI, and that is why it is safer in a way... Also, i am not sure
if we can modify the plugin without restarting the image... Can we?
If not, that's a good point for plugins and security.

If we are lazy, both FFI and plugin implementations will have security
holes.
But maybe, code will be corrected faster in a FFI framework.

I have no experience with Squeak plugin, but practiced both user primitives
and DLLCC in VW. And the latter gave me more flexibility and rapid
development than the former (except for cross machine portability for which
good old dirty C preprocessor directives are somehow more efficient).

I would encourage FFI because it does open Squeak on the external world. Of
course it does scratch the "do it all in Smalltalk" paradigm, but from a
strategic point of view, opening Squeak is really something needed if you
want to convince more people to join (think of how easy it is in python to
call external code).

And sorry, the plugin framework is not simple enough. You have to learn a
new
language that is neither Smalltalk, nor C, you have to go through code
generation for generating the wrapper function to call your external one,
then you are on your own to compile the wrapper code and integrate it in the
image... Rather a long development cycle for Smalltalkers. That is only my
opinion, and of course, good arguments can make it change.

Nicolas


Le Mardi 01 Août 2006 18:19, Ron Teitelbaum a écrit :
> All,
>
> I'll have to preface this comment with "I JUST GOT FFI WORKING" but
> consider the potential risk of FFI compared to any other external
> communication mechanisms.
>
> "IF" the image is secured so that no changes can be made to a running
image
> which I suppose would include calls to FFI <...> (he says not really
> knowing how the calls are made or interpreted) then there are benefits to
> direct calls.
>
> First the implementation of the dll wrapper would not be in question only
> the direct calls to the OS provided calls would need to be reviewed.
>
> Second securing the image by removing FFI is too drastic in my opinion.
> How will we get Crystal Reports or any other direct API to work without
it?
>
> Third support for external API's are simplified for developers by having
> the calls in the image not wrapped in a DLL.  Everything is much more
> transparent.
>
> Forth there is not much difference in terms of risk to the system between
> the execution of the DLL function and the FFI call if parameters are
passed
> in.  Both methods need to have proper handling for error conditions before
> the primitive call.  I'm not convinced that error checking is better in a
> DLL then in Smalltalk and on the contrary if we make the error checking
> harder to apply (by pushing it into the DLL) it is less likely that
changes