JNIPort for VisualWorks

Hi all,

I think it's about time to get this off my chest:

I've been working on porting Chris Uppal's JNIPort to VisualWorks. JNIPort is a class library which allows Java code to be invoked from Smalltalk, using a regular Java VM to execute the Java code. If you are interested in details, have a look at
http://www.metagnostic.org/DolphinSmalltalk/JNIPort.html

JNIPort would allow instant access to any Java class library, and even calling back into VisualWorks (although with some limitations when multithreading is involved). This would open up a whole new world - e.g. monitoring a VisualWorks application server using JMX, integration of Java messaging using JMS, using all of those free and commercial Java libraries available on the market. Chris even has an example for opening a Swing GUI showing a hierarchy of Smalltalk classes - certainly not something you would want to do in a real application, but it shows what's possible.

The current state of the VisualWorks port is "can calculate the abs of a float, but crashes with access violations when trying to do something useful (after successfully generating wrappers for about 100 real Java classes)". Currently, it's Windows only, but it should be easy to include other platforms. The original Dolphin Smalltalk version works perfectly, of course - the access violations are a feature which I introduced. Once this bug is corrected, the rest should be easy.

For various reasons, I haven't had enough time to finish the port during the last few months, and I hope that someone from the community might help with debugging the access violations. This does not mean that I'm abandoning the project, leaving the mess to someone else. I still want to do it, but given the circumstances, I think that it might make more sense if I give others a chance to participate. ;-) I'll post the current state of the code to the public repository as soon as possible (probably next weekend) together with instructions about other files from metagnostic.org which you'll need.

Just a quick check to see if it's worth the effort: Anybody interested, either in using JNIPort, or in participating?

Joachim Geidel

Hi Joachim,

I'm definitively interested on that, because I think it is strategically
important for Smalltalkers to have a "bridge" to Java world. I think a
Smalltalk community knowledgeable on Java should help you to keep going
on. I can help as a user, because I don't have any experience with Java
development.

Thanks for you effort and best regards
Janko

[hidden email] wrote:
--
Janko Mivšek
Svetovalec za informatiko
EraNova d.o.o.
Ljubljana, Slovenija
www.eranova.si
tel:  01 514 22 55
faks: 01 514 22 56
gsm: 031 674 565

But according to Steve Jobs, nobody uses Java anymore.

> Hi Joachim,

> I'm definitively interested on that, because I think it is strategically
> important for Smalltalkers to have a "bridge" to Java world. I think a
> Smalltalk community knowledgeable on Java should help you to keep going
> on. I can help as a user, because I don't have any experience with Java
> development.

> Thanks for you effort and best regards
> Janko

Can you post what you have done into the Public Store Repository, so
that others can get at it?

At 03:58 AM 1/16/2007, you wrote:
<Talk Small and Carry a Big Class Library>
James Robertson, Product Manager, Cincom Smalltalk
http://www.cincomsmalltalk.com/blog/blogView

well, if Steve says so :)

and apparently Java people don't use EJBs anymore i.e. according to my  
Java friends, but I do know for one a colleague of mine that decided to do  
a small project in Java just because they had a PDF wrapper library, it  
did something with fonts that's Bruce's wrapper does not.

The glue would be good

-Charles
------------------------------------
http://www.monteirosfusion.com/forum
http://monteirofusion.blogspot.com

On Tue, 16 Jan 2007 05:55:58 -0500, Michael Lucas-Smith  
<[hidden email]> wrote:



--
Charles A. Monteiro
http://wiki.nycsmalltalk.org
http://www.monteirosfusion.com
http://monteirofusion.blogspot.com

Hi James,

>Can you post what you have done into the Public Store Repository, so
>that others can get at it?

yes, of course. But you'll have to wait until next weekend, because I don't have easy access to the repository from the place where I am now. I also have to write some documentation about which additional files are needed and where to get them.

I'll post an announcement to the list as soon as the code is available.

Joachim

Hi all,

I just published a first version of a VisualWorks port of Chris Uppal's
JNIPort to the Cincom Public Repository.

JNIPort is an interface which makes it possible to use Java libraries
from Smalltalk. It invokes a Java VM using the Invocation Interface
functions of the Java Native Interface (JNI).

JNIPort consists of four bundles:
        JNIPort Prerequisites
        JNIPort
        JNIPort Tests
        JNIPort Tools
You will also need recent versions of the following packages:
        Registry
        Weaklings

For detailed documentation of JNIPort including examples see
        http://www.metagnostic.org/DolphinSmalltalk/JNIPort.html

To use callbacks from Java to Smalltalk and to run the unit tests, you
will need the files contained in the 'Extras' directory contained in the
archive which can be downloaded from Chris Uppal's web site:
        http://www.metagnostic.org/DolphinSmalltalk/JNIPort-Complete.zip
The archive also contains a copy of the complete documentation.

The bundle comment of JNIPort contains a few additional code samples:
computing "-3 abs" the hard way, and querying the JVM for the JNI
version number.

The current state of the port is clearly "work in progress". It does not
yet work, and the unit tests are failing. I have done the port on
Windows XP with Java 1.5.0 and have not tried any other configuration.

So far, it is possible to create a JVM instance connected to a real Java
VM ("JVM newWithDefaultSettings") and to execute low level JNI function
calls as shown in the code samples in the bundle comment. To run the
tests, you have to execute "JVM newWithDefaultSettings" first.

When a call of a Java method raises a Java exception (a Throwable),
JNIPort can not yet cope with the situation. It tries to find the
superclass of the Throwable, which fails with an "External access
failed". You'll run into this in the unit tests when you have the
complete code of JNIPort loaded and do not have the jar-files from the
Extras directory in the classpath.

I have tried to preserve the structure of the original code as much as
possible, but as the low level C APIs of VisualWorks and Dolphin
Smalltalk are quite different, there were lots of changes of low level
code. I also removed many of the methods which are specific to Dolphin,
and stripped down some supporting class libraries (in JNIPort
Prerequisites) to the bare minimum.

The VisualWorks port does not yet contain any GUI. Chris has implemented
a status monitoring tool with a configuration wizard, a tree view of the
(Java) class registry. When the mechanics are working, I would like to
add something similar to the VisualWorks version. Have a look at
http://www.metagnostic.org/DolphinSmalltalk/JNIPort/status-monitor.html
to see what the tools look like.

What's next:
- The tests must not fail.
- Creating a JVM for the tests might be implemented using a TestResource.
- The hard coded library path and library file names in JNIInterface
should be avoided, e.g. using the package External-Interface-Pragmas.
- Check the package FastCMethodPointers from the public repository; it
might enhance the performance of JNI function calls.
- General code cleaning. Remove unused and Dolphin specific methods.
- Make JVM creation more robust when Java classes corresponding to
wrapper classes in the image cannot be found in the classpath.
- Add a GUI.

If anybody would like to help finishing the port: load the code, and go
ahead. However, it might be a good idea to contact me, such that we can
avoid unnecessary parallel work. Please set the blessing level to "Work
in Progress" as long as the unit tests are failing.

Best regards,
Joachim Geidel

Hi all,

I have posted an updated version of JNIPort for VisualWorks to the
Cincom Public Repository.

JNIPort is an interface which makes it possible to transparently use
Java libraries from Smalltalk. It invokes a Java VM using the Invocation
Interface functions of the Java Native Interface (JNI). Once the full
functionality works, it will automatically generate wrapper classes for
Java classes, enabling to write code like this:

        jvm := JVM current.
        zfClass := jvm findClass: #'java.util.zip.ZipFile'.
        zipfile := zfClass new_String: aFilenameAsString.
        zipfile size_null. "--> answers an Integer"
        entries := zipfile entries_null.
        entries asAnEnumeration do:
                [:each | Transcript print: each; cr].

The methods new_String:, size_null, entries_null will be generated in a
"ghost class" when the JVM accesses the Java class - no need to write code!

Chris Uppal has sent me several bug fixes and changes to bring JNIPort
up to date. I have changed the unit tests to use SUnitToo instead of
SUnit and added some more bug fixes.

It is already possible to execute Java methods and access Java objects.
Chris' example for accessing ZIP files using java.util.zip.ZipFile works:
http://www.metagnostic.org/DolphinSmalltalk/JNIPort/java-base-example-1.html

The unit tests for the base layer are almost all green. There are 6
tests which fail because out-of-range values for short and byte values
do not raise exceptions as expected, and one test fails because of a bug
somewhere in the conversion between Java and Smalltalk strings (only for
two-byte characters).

If you want to run the tests, the corresponding Java code (files
"Extras\JNIPort.jar" and "Extras\JNIPort-Tests.jar" in the JNIPort
download) must be on the Java classpath. One way to ensure this is to
set the CLASSPATH environment variable like this if you have copied the
two jar files to the working directory:
        CLASSPATH=.;JNIPort.jar;JNIPort-Tests.jar
It is better to use absolute paths, as VisualWorks 7.4.1 changes its
current directory when using the Windows file dialog - this can have the
consequence that the Java VM looks in the wrong directory.

To run the tests in the package "CU Java Base Tests", first execute
        JVM default
in a workspace, which initializes a running Java VM. The tests for
wrapper generation and for callbacks from Java will break when the
default JVM is used. You have to use a JVM configured for wrapper
generation and callbacks to run them.

What's next (feel free to help me):
- Fix the bug in String conversion.
- Add checks for out-of-bounds values for short and byte values.
- Make finalization of JavaClassInstances work.
- Make wrapper class generation work (if it doesn't already by
  accident).
- Make callbacks work.
- Add a GUI for configuration, monitoring, and inspecting ghost classes.
- See what happens when using Java 1.6.

Give it a try!

Joachim Geidel

Joachim Geidel schrieb am 20.01.2007 17:54: