Cog: A question about: setInterruptCheckChain()

On 9/29/2010 1:03 PM, Eliot Miranda wrote:

I think it /is/, and you shouldn't allow your disinclination for the FFI
approach blind you to the fact that one way of keeping the VM small is
to produce a really strong flexible FFI that supports threading and
callbacks and to implement interfaces above the line.  Do you seriously
believe things like the ODBC connect should be implemented in plugins?

Absolutely. Memory safety being the prime reason.

In the case of event callbacks its a different issue.  The current event
queue abstraction is broken in that it prevents Smalltalk code from
answering questions asked through events such as WM_QUERYENDSESSION
because the VM is forced to queue events, and hence can only provide a
default answer in the window proc.  So there are areas where having
proper callbacks is required to be able to integrate properly with the
host operating system.

Please. The issue here is that our event mechanism is a bit simplistic but the Android VM is a great example of how to deal with these issues properly. Sure, callbacks could address this too, but claiming *only* callbacks can address this is simply wrong.

I don't disagree with what you say about plugins, or about maintaining
cross-platform abstractions in the VM for core functionality that is
common across platforms, but I think its clear one needs true callbacks,
and that at least for integrating with the GUI this is the best way to
implement it.

A solution that inherently implies exposure of raw pointers to users isn't what I would call the "best" solution. That said, I'm not arguing against callbacks - when you need them you need them. And there are practical situations where it's incredibly handy to have them. And yet, would I design the system so that it depends at its core on such facilities? Hell, no, because in addition to being memory-unsafe, porting an FFI is *much* harder than porting a bit of C support code.

   It really is a matter of where you want to deal with certain aspects
   of the system. I prefer to deal with that level of stuff in a plugin
   via C because:
   + it provides a way to do ensure memory safety

there's nothing in plugins per-se to maintain memory safety.  A mistake
in a plugin is as destructive as a mistake in an FFI call.  In either
case a wrapper around a particular call can validate and provide safety.

In theory, yes. In practice such wrappers *do* exist in plugins and they do *not* exist in FFI calls. Why? Good question. Probably because people understand that in C land there are more constraints and they are mentally "set up" to keep an eye on the constraints and to make sure the incoming stuff is okay.

   + it provides a way to provide abstractions

- so does Smalltalk, and t least in SMalltalk we have inheritance to
enable us to express cross-platform APIs in abstract classes and map
these down onto particular platforms in concrete subclasses.  This is
/far better/ than the sprawling mess that is the platforms tree in the VM.

In theory yes. In practice, who knows. Having looked at some Smalltalks that expose all the platform details (like Dolphin) it seems to me that the equation of "platform FFI code * number of platforms" will be /far more/ of a sprawling mess. Which is probably why (unless I'm mistaken in which case I'll appreciate a correction) precisely none of the cross-platform Smalltalks do that.

   + the code is faster to write and debug in C

this is debatable.

Hardly. Let's start with "#include <windows.h>" (or whatever platform stuff you happen to need). You're losing right there. The problem is that the system isn't set up to seamlessly interact with C. Even if you've got some auto-magic header file importer that creates structs and defines and stuff, as long as it takes more than, i.e.,

winMessageBox: contents title: title
       <include: 'windows.h'>
       MessageBox(NULL, contents, title, MB_OK + MB_ICONWARNING).

you're going to lose because it takes so much longer to get to the point of actually writing the code you're trying to write.

Secondly, you *really* need to try the latest Visual Studio and play with its debugger - I was blown away while debugging SqueakSSL when I hit a typical NULL pointer, fixed the code in my plugin DLL, continued in VS and the underlying Squeak KEPT RUNNING! Holy crap. This stuff has come a looooong way.

In the VW FFI with the ability to catch exceptions
and pass them back up (doesn't always work, a bad emory corruption may
crash the entire system) makes thing easier.  But low-level debugging is
painful in general.

   + access to facilities (threads, interrupts, atomicity) not
   available otherwise

Again, a strong FFI provides at least some of these facilities.  Not
everything can be done through the FFI, but a lot can be done, and
elegantly and extensibly, by Smalltalk programmers, not a few VM
specialists.

I call BS. The reality is that one needs to know three kinds of things, only one of which would be helped by using the FFI:

1) The "mechanics" of writing a plugin, i.e., slang and compilation. This is the part where the FFI would help, but so would better APIs for marshaling. While it's "cool" that one can run slang plugins in Squeak it is also completely and utterly *useless* when it comes to integrating platform specific stuff and having to deal with Slang, VMMaker and the awful build setups, and then -on top of that- the actual work you're trying to do is just a bit too much.

2) Understanding the rules of interfacing the external world, i.e., when can pointers be kept, what does GC do to your objects etc. None of this is helped by using the FFI.

3) Understanding the actual domain code. Again, none of this is helped by the FFI (to the contrary because nobody will find examples for the weird Smalltalk selectors we use on the web and people will in turn not know where to find the actual value for MB_OK or somesuch).

Basically, the assumption that *all* the difficulty is in writing the marshaling code is nonsense. There is some of it, true, but that is entirely our fault for providing poor integration APIs.

   On the downside, there is:
   - it's harder to modify for people who don't know C and are not set
   up to build a plugin
   - it's often aimed at the smallest common denominator (though not
   necessarily)

- it is a far more fixed interface that is far harder to evolve

Yes, but when you call it "stable" instead of "fixed" and "robust" instead of "harder to evolve" it becomes a plus :-)

Cheers,
 - Andreas

   Cheers,
     - Andreas


       On the other hand, a VM providing a larger de facto immutable API:
       - somehow is more secure (a guaranty we can run very old images in
       very new OSes).
       - can maintain the illusion that despite the efforts of OS and
       hardware designers to make it ever more complex, a single person can
       still understand (almost) the whole system.
       - avoid the necessity to embed knowledge of dozens of different
       variants of different OSes in your image.

       Of course, the complexity still exists under the carpet... When I
       contemplate all the unecessary complex knowledge from those
       beautiful
       configure/cmake scripts and macros, I'm not convinced all this cruft
       will be easier to modify in Smalltalk than it is in C world.

       I perfectly understand too the desire of a VM maintainer to lighten
       the burden ;) though.

       Nicolas

       2010/9/29 Eliot Miranda<[hidden email]

       <mailto:[hidden email]>>:

           On Wed, Sep 29, 2010 at 10:22 AM, Andreas

           Raab<[hidden email] <mailto:[hidden email]>>  wrote:

               On 9/29/2010 9:36 AM, Eliot Miranda wrote:


                       See the recordMouseEvent() and its friends
                   (sqNextEventPut,
                       eventBuffer[1024] ...) in sqWin32Window, for
                   better understanding my
                       point :)


                   Which makes my point well.  These are simply
                   maintaining a queue.  If
                   one has callbacks then the bulk of the Windows event
                   handling system can
                   be lifted up into the image.  This is what Vassili
                   Bykov did in Newspeak
                   above my Alien callbacks.  In Newspeak the Windows
                   MainWndProc is a
                   callback:


               Thus losing any level of platform abstraction. A bad
               trade-off, IMO.


           The platform abstraction merely moves from the VM to classes
           in the image.  But it does mean that one can properly
           implement Windows event semantics (things like query quit
           events) that the queue prevents.  And it means that the VM
           gets simpler and complexity is handled by better facilities
           (in Smalltalk with its many advantages for creating and
           maintaining abstractions with concrete variant implementations).
           But you and I have had this discussion a number of times and
           I take your point about the VM maintaining a black-box
           abstraction boundary whereas the in-image implementation is
           in theory porous.  As I've said before this doesn't strike
           me as that important when Smalltalk has had boundary
           violators such as instVarAt:[put:] for ever and they are
           rarely abused.  If one were to implement a native GUI event
           interface in Smalltalk using callbacks one would have to
           police the abstraction boundary.  But that's easily done,
           and there are other benefits
           - the VM, being simpler, gets more longevity since changing
           the event interface doesn't imply a VM change
           - the system, having native interfaces, can support each
           platform's facilities in full instead of providing the
           lowest common denominator
           So IMO it is a good trade-off.
           best
           Eliot


               Cheers,
                 - Andreas