[ANN] Squeak on Android

Folks -

Sorry for being quiet over the weekend but I was busy with my new toy -
a shiny new unlocked Nexus One which arrived last week fresh from the
Google store.

As a result of my weekend distraction you can now download and run
Squeak on any Android based phone. If you have an Android phone, simply
search for "squeak" in the Android market place and it should find it
right there (sorry, Google doesn't seem to give web access to the apps
in the store so I can't send a link).

Please be aware that this is not a full port yet. It's a weekend effort
to show the basic feasibility. Lots (and I mean *lots*) of things are
still missing from a full port (among those is text input and network
support to name just two of the more glaring ones). However, I would be
*very* interested to hear if (and how well) it works for other
Android-based cell phones. So if you have a Motorola Droid or or a
T-Mobile G1 give it a shot and post some benchmark results.

Performance on the Nexus One is about what one would expect: With
roughly 1M sends/sec and 30M bytecodes/sec it's not exactly rocking but
it's quite usable for most tasks on a mobile device. (Input is
*terrible* though; Squeak's UI is not made for fat-fingered clicks like
mine :-)

Since the code itself is quite messy at this point, it isn't published
anywhere yet. If people are interested in contributing, please let me
know and I'll share what I have even before it's cleaned up. Be warned
though, unless you know how to deal with both the Android SDK and NDK,
Java, JNI, and the Squeak VM it will be a very steep learning curve.

And of course, attached a little screenshot (taken on the emulator; I
haven't found out how to use my phone to make pictures of my phone :)

Cheers,
   - Andreas

PS. I don't recall micro benchmarks on the iPhone - John can you repost
those to see how well the hardware stacks up?

On 2010-01-17, at 10:18 PM, Andreas Raab wrote:

> Performance on the Nexus One is about what one would expect: With roughly 1M sends/sec and 30M bytecodes/sec it's not exactly rocking but it's quite usable for most tasks on a mobile device. (Input is *terrible* though; Squeak's UI is not made for fat-fingered clicks like mine :-)

> PS. I don't recall micro benchmarks on the iPhone - John can you repost those to see how well the hardware stacks up?

A dull iPhone 2nd generation gives
21,548,821 bytecode/sec; and 564,465 sends/sec

a shinny newish iPhone 3G S gives:
42,xxxx,xxx bytescodes/sec; and 1,1xx,xxx sends/sec.

Also how many frames per second can it do?  higher performance turned on, oh say bouncing atoms up...


--
===========================================================================
John M. McIntosh <[hidden email]>   Twitter:  squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
===========================================================================

Great news!
I know at least one person who is interested in that - Goran :)

2010/1/18 Andreas Raab <[hidden email]>:


--
Best regards,
Igor Stasenko AKA sig.

Hi folks!

Igor Stasenko wrote:
> Great news!
> I know at least one person who is interested in that - Goran :)
>
> 2010/1/18 Andreas Raab <[hidden email]>:
>> Folks -
>>
>> Sorry for being quiet over the weekend but I was busy with my new toy - a
>> shiny new unlocked Nexus One which arrived last week fresh from the Google
>> store.

Andreas, I *love* you! :) This made my day.

I was myself toying with different approaches and will definitely take a
look if you can share the code - github is nice... :)

Mikael Kindborg has been doing lots of cool fun stuff on Android by
remote controlling a Rhinoscript engine using HTTP from an extended
Workspace within Squeak with stuff like "do it in Android" etc.

We are focusing hard on Android here at MSC and having Squeak on it is
extremely interesting.

Now, the absolutely coolest thing would be if we somehow can "tap into"
all that java code. Have you given that any though? I even looked at the
Dalvik source code to play with the thought of merging a Squeak VM with
a Dalvik VM...

regards, Göran

John M McIntosh wrote:
> A dull iPhone 2nd generation gives
> 21,548,821 bytecode/sec; and 564,465 sends/sec
>
> a shinny newish iPhone 3G S gives:
> 42,xxxx,xxx bytescodes/sec; and 1,1xx,xxx sends/sec.

Interesting! I think that means I need to play with optimizations a bit
more - from what I see it looks like the 3GS runs a 600MHz ARM where the
Nexus runs a 1GHz. Did you apply any optimizations i.e., gnuify, or any
ARM specific stuff?

> Also how many frames per second can it do?  higher performance turned on, oh say bouncing atoms up...

Can't do those yet. I'm using an event VM for the Android and haven't
gotten around to integrating timer events. At this point the VM requires
touch input for *any* operation :-) Plus, every cycle is a fullscreen
update both from Squeak to Java and from Java to the display. A bit more
optimization work to be left done here.

Cheers,
   - Andreas

Göran Krampe wrote:
> Andreas, I *love* you! :) This made my day.

Heh, heh, you're welcome. It's still a long way to go though.

> We are focusing hard on Android here at MSC and having Squeak on it is
> extremely interesting.

Great! Think you can spare some resources to move things forward? :)

> Now, the absolutely coolest thing would be if we somehow can "tap into"
> all that java code. Have you given that any though? I even looked at the
> Dalvik source code to play with the thought of merging a Squeak VM with
> a Dalvik VM...

There are two approaches here, with different tradeoffs:

1) Write generic bridge to Java (using JNI). That's a sizable chunk of
initial work, but once it's done you will be able to interact seamlessly
between Squeak and Java. I've built a similar bridge (to Python) and it
works extremely well.

2) Build an AndroidPlugin that abstracts the relevant functionality and
implement whatever needs to be done from the plugin via JNI. This
doesn't require you to implement a generic bridge but rather only the
pieces that you really need in your app. Plus, you can have proper
fallback code if the plugin is absent.

Personally, I'll follow the approach #2 for several reasons. Most of
all, it's easier to do in the hobbyist mode that I'm working on this
project. I can add one function at a time, test it, commit it, get on to
the next. The Java bridge would be intellectually challenging but is
just not doable with the amount of time it would require (about a month
of full time work for me - translated into weekends and evenings that's
roughly 6 months or so). Plus, it's not all that much fun compared with
the alternatives that I could spend time on. I *could*, however, give a
good bit of advice on how such a bridge would need to be designed if
someone were to take this on (it's actually rather simple - the trick is
in transparently mapping oops so that you can pass objects back and
forth without them being "accidentally" collected).

Cheers,
   - Andreas

On 2010-01-18, at 7:36 PM, Andreas Raab wrote:

> John M McIntosh wrote:
>> A dull iPhone 2nd generation gives 21,548,821 bytecode/sec; and 564,465 sends/sec
>> a shinny newish iPhone 3G S gives: 42,xxxx,xxx bytescodes/sec; and 1,1xx,xxx sends/sec.
>
> Interesting! I think that means I need to play with optimizations a bit more - from what I see it looks like the 3GS runs a 600MHz ARM where the Nexus runs a 1GHz. Did you apply any optimizations i.e., gnuify, or any ARM specific stuff?

Well yes, I'm sure there has to be a beer in here somewhere, don't see it tho...

No ARM specific stuff unless you consider it might have more registers than an i386...

--
===========================================================================
John M. McIntosh <[hidden email]>   Twitter:  squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
===========================================================================

John M McIntosh wrote:
> On 2010-01-18, at 7:36 PM, Andreas Raab wrote:
>
>> John M McIntosh wrote:
>>> A dull iPhone 2nd generation gives 21,548,821 bytecode/sec; and 564,465 sends/sec
>>> a shinny newish iPhone 3G S gives: 42,xxxx,xxx bytescodes/sec; and 1,1xx,xxx sends/sec.
>> Interesting! I think that means I need to play with optimizations a bit more - from what I see it looks like the 3GS runs a 600MHz ARM where the Nexus runs a 1GHz. Did you apply any optimizations i.e., gnuify, or any ARM specific stuff?
>
> Well yes, I'm sure there has to be a beer in here somewhere, don't see it tho...

If you're going to C5 we can take care of it there :-)

> No ARM specific stuff unless you consider it might have more registers than an i386...

Nah, I was just wondering if there was something specific to learn about
compiler switches on ARM in terms of optimizations, say
-fwoohoo-enable-super-fast-arm-dispatch or something :-)

BTW, one of the things I'm considering is to change the event type for
touch events. That's mostly because it seems like on Android you
actually get a coverage for a touch event, i.e., how many pixels were
covered by your fatty fingers. And what I'm thinking here is that
interaction could be *greatly* improved if the dispatch of such events
would take coverage of the target area into account. That means that
even if a click doesn't land "on" a relatively small button that the
button will get the click if it's the ui element with the largest
coverage in the region. I could see this working very well for
relatively small UI elements. However, that does require extra
information (pressure, coverage) and I'm wondering if it's worthwhile to
package and dispatch those events separately. How does this level of
stuff work on the iPhone? Are you dispatching touch events ore are you
dispatching mouse events? Any additional parameters you're passing along?

Cheers,
   - Andreas

Andreas Raab wrote:
> 1) Write generic bridge to Java (using JNI). That's a sizable chunk of
> initial work, but once it's done you will be able to interact seamlessly
> between Squeak and Java.

And now that I've been thinking about it, it actually even seems more
reasonable because you can leverage the FFI to do it. What I'd do is to
add a new calling convention called 'java' and then use the FFI to spec
it along the lines of:

JAssetManager>>open: fileName
   "public final InputStream open (String fileName)"
   <java: JInputStream 'android.content.res.AssetManager.open' (JString)>

(see
http://developer.android.com/reference/android/content/res/AssetManager.html#open%28java.lang.String%29)

and then provide the necessary level of primitive coercion via having
the FFI recognize the <java:> calling convention and dispatch to a
JNIPlugin which implements the necessary glue code. Being able to ride
on top of the FFI really simplifies implementing all of the type
manipulation so you can concentrate on the glue code itself. I'd say
this cuts down on the required time significantly (to the point where I
might even spend another weekend looking at it).

Cheers,
   - andreas

> There are two approaches here, with different tradeoffs:
>
> 1) Write generic bridge to Java (using JNI). That's a sizable chunk of
> initial work, but once it's done you will be able to interact seamlessly
> between Squeak and Java. I've built a similar bridge (to Python) and it
> works extremely well.

You are aware of
http://jniport.wikispaces.com/
aren't you?

Cheers

Markus

On Tue, Jan 19, 2010 at 11:14:19AM +0100, [hidden email] wrote:
There is also JavaConnect:

  http://news.squeak.org/2008/04/30/using-smalltalk-with-java-javaconnect-and-jniport/
  http://www.info.ucl.ac.be/~jbrichau/javaconnect.html

Dave
 

Hi!

Andreas Raab wrote:
> Göran Krampe wrote:
>> Andreas, I *love* you! :) This made my day.
>
> Heh, heh, you're welcome. It's still a long way to go though.

Of course, but it's a start!

>> We are focusing hard on Android here at MSC and having Squeak on it is
>> extremely interesting.
>
> Great! Think you can spare some resources to move things forward? :)

Might be able to.

Yeah, and yes, I read your other ride-on-FFI-post. Also, the JNIPort
someone mentioned looked quite interesting too, and with a reasonable
license it seems.

But of course, a plugin ...not sure I get what that *is* in this lineup
- a C plugin? Or is it a piece of java code in the java "harness" you
have that we call using.... some kind of protocol?

Anyway, we would love to take a look at your "messy code" and promise
that we will not harass you with questions about it :)

regards, Göran

Göran Krampe wrote:
> But of course, a plugin ...not sure I get what that *is* in this lineup
> - a C plugin? Or is it a piece of java code in the java "harness" you
> have that we call using.... some kind of protocol?

I was thinking about a regular plain old plugin written in C that uses
specific JNI calls to do its work. Basically Squeak -> plugin (C) -> JNI
-> Java. It's more work but it works incrementally which is a huge
advantage from my position.

> Anyway, we would love to take a look at your "messy code" and promise
> that we will not harass you with questions about it :)

I've uploaded the current state to

   http://squeakvm.org/win32/experimental/androidvm.zip

Some comments on the setup:
- The VM itself is a shared library (libsqueakvm.so) the "app" is a
wrapper written in Java.
- You'll need the Android NDK in order to compile the shared lib. Make
sure you can compile hellojni or some of the others to ensure your tools
are in place.
- Place the entire squeakvm directory in the NDK's apps/ directory (next
to hellojni and friends)
- To compile the app you need Eclipse and the various SDK bits (see
Android docs for that; it works pretty much as advertised)
- Apparently, Eclipse does not "see" the dependencies between the shared
library and the app - you need to manually remake the shared lib if
you've changed a .C file
- Make absolutely sure you how to work the debugger (ddms). It's pretty
much your only source of debug info since it shows the android logs that
you can write to by using either System.out or dprintf().

BTW, I'll say that more than half of my time on the weekend was spent
with the issues like the above. The setup is lengthy and just drags on
and on and on, no reason to get discouraged by that.

Some comments on the implementation:
- The image is broken into pieces since Android's AssetManager doesn't
like assets larger than 1MB. There's probably a better way of doing that
but here's the code I used to break up an image:

        infile := FileStream readOnlyFileNamed: 'android.image'.
        infile binary.
        count := 0.
        [infile atEnd] whileFalse:[
                outfile := FileStream newFileNamed: 'android.image.', (count := count+1).
                outfile binary.
                outfile nextPutAll: (infile next: 1024*1024).
                outfile close.
        ].
- All the Android specific code is currently in one file
project/jni/squeakvm.c for simplicity (this should be broken out to
separate files as the project progresses).
- Since the image file is compressed, it's preloaded into the object
memory, and then the sqImageFile* and sqAlloc* functions are stubbed out
"just so" that they operate on the preloaded image in-memory. It's a
really neat hack but you probably wouldn't get that from reading the
code :-)
- I haven't been using "proper" JNI callbacks for implementing display
updates; but then I don't *really* think that should be necessary (we
should rather fix the Morphic update cycle not to leave bits behind).
But if you want to start with something I'd say a good place is to
implement the callbacks from Squeak to Java in ioForceToScreen and
ioShowDisplay.

Good luck!

Cheers,
   - Andreas

On Tuesday 19 January 2010 11:12:09 pm Andreas Raab wrote:
> Some comments on the setup:
> - The VM itself is a shared library (libsqueakvm.so) the "app" is a
> wrapper written in Java.
Way to go! I feel as a library squeakvm is more portable to emerging platforms
than as a standalone command line or X11 app. As a library, squeakvm can be
easily integrated into desktop systems based on GTK+, Qt etc.
> Some comments on the implementation:
> - The image is broken into pieces since Android's AssetManager doesn't
> like assets larger than 1MB.
Another excellent idea that I would like to see merged into squeakvm trunk.

Perhaps a slight modification - instead of squeak.image.<n>, pick segments from
a directory say, squeak.imd/<n>. This would allow large images to be carried
around on (or transferred through) a traditional filesystems with 8.3 or per-
file size limits.

Subbu

In

MCHttpRepository
    location: 'http://www.squeaksource.com/iPhone'
    user: ''
    password: ''

Name: iPhone-VMMaker-Only-JMM.1


I had some changes to handle:

(a) readImageFromFile: f HeapSize: desiredHeapSize StartingAt: imageOffset
This was to handle mmap allocation, but it's obsoleted and replaced a year back with the macros we talked about earlier on image reading

(b) primitiveGetNextEvent

I have a change here where eventTypeIs == 6  if so that's a complex event and you'll find a description of that in the sq.h that I submitted in my microsecond clock changes last week or so.
One of the issues is that the event type coming up is assumed to be an integers.  However I change slot 4 to point to an oops.   What is in the oops is platform dependent.  I did have some
constants I added for iphone touchs, and the generic location, acceleration data.

The ioGetNextEvent then is responsible for population slot 4 with smalltalk object.   Althought the engineers at Apple tell me there *might* be touch area & pressure data, it's not accessible
so you get the four types of events, the object is a set containing one or more items to enable the handling of 4 fingers down... 2 fingers up... with twist rotation on the other two fingers.
Then in EventSensor(whatever) I deal with the interpretation of the complex object type.  So no decision is done in the VM.

 
typedef struct sqComplexEvent
        {
                int type; /* type of event;  EventTypeComplex */
                unsigned int timeStamp; /* time stamp */
                /* the interpretation of the following fields depend on the type  of the event */
                int action;        /* one of ComplexEventXXX (see below) */
                usqInt objectPointer; /* used to point to object */
                int unused1; /*  */
                int unused2; /*  */
                int unused3; /*  */
                int windowIndex; /* host window structure */
        } sqComplexEvent;

#define ComplexEventTypeTouchsDown 1 /*  */
#define ComplexEventTypeTouchsUp 2 /*  */
#define ComplexEventTypeTouchsMoved 3 /*  */
#define ComplexEventTypeTouchsStationary 4 /*  */
#define ComplexEventTypeTouchsCancelled 5 /*  */
#define ComplexEventTypeAccelerationData 6 /*  */
#define ComplexEventTypeLocationData 7 /*  */
#define ComplexEventTypeApplicationData 8 /*  */



On 2010-01-18, at 11:30 PM, Andreas Raab wrote:

> BTW, one of the things I'm considering is to change the event type for touch events. That's mostly because it seems like on Android you actually get a coverage for a touch event, i.e., how many pixels were covered by your fatty fingers. And what I'm thinking here is that interaction could be *greatly* improved if the dispatch of such events would take coverage of the target area into account. That means that even if a click doesn't land "on" a relatively small button that the button will get the click if it's the ui element with the largest coverage in the region. I could see this working very well for relatively small UI elements. However, that does require extra information (pressure, coverage) and I'm wondering if it's worthwhile to package and dispatch those events separately. How does this level of stuff work on the iPhone? Are you dispatching touch events ore are you dispatching mouse events? Any additional parameters you're passing along?
>
> Cheers,
>  - Andreas

--
===========================================================================
John M. McIntosh <[hidden email]>   Twitter:  squeaker68882
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
===========================================================================

On 19.01.2010, at 04:57, Göran Krampe wrote:
Just came across this article via my linux-squeak trigger:

http://www.infoq.com/news/2010/01/new-python-gil-squeak-on-android

It suggests to use Android Scripting Environment (ASE) for calling into Android:

http://code.google.com/p/android-scripting/

- Bert -

Bert Freudenberg wrote:
> Just came across this article via my linux-squeak trigger:
>
> http://www.infoq.com/news/2010/01/new-python-gil-squeak-on-android
>
> It suggests to use Android Scripting Environment (ASE) for calling into Android:
>
> http://code.google.com/p/android-scripting/

Interesting, thanks for the pointer!

Cheers,
   - Andreas

Folks -

I finally set up a project at Google to have a place people can more
easily contribute to:

   http://code.google.com/p/squeak-android-vm/

The code that's checked in is still the same that I put up earlier
(didn't have much time to play with things) but hopefully I'll find a
few hours soon to improve things.

If you'd like to be added as developer, please send me your Google
Account email address so that I can add you to the project.

Cheers,
   - Andreas

Truly great! Thanks!
Micke :-)

On 9 February 2010 03:31, Andreas Raab <[hidden email]> wrote: