Smalltalk › Squeak › Croquet › Croquet - User

Controlling Avatars with a Java Program

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

3 messages Options

jyoshimi

Controlling Avatars with a Java Program

Hello and kudos to those who made croquet. It's quite nice. I'm writing to ask how hard it would be to control avatars in Croquet with a separately running java program (in particular, this one: http://www.simbrain.net/). The idea would be a user could run Simbrain, which is used to build neural networks (simulated brain circuits) and have a client window of croquet running with an avatar in it, and link the neural network to the avatar to control it. As the avatar walked around the user could observe and manipulate its simulated brain. If this wouldn't be too hard any pointers would be welcome.

I did a few quick searches on the topic of java and croquet and did not find the answer, but apologies if I overlooked something obvious in the docs or forums.

With best regards,

- Jeff

Les Howell

Re: Controlling Avatars with a Java Program

On Sun, 2007-09-30 at 20:47 -0700, jyoshimi wrote:

> Hello and kudos to those who made croquet. It's quite nice. I'm writing to
> ask how hard it would be to control avatars in Croquet with a separately
> running java program (in particular, this one: http://www.simbrain.net/).
> The idea would be a user could run Simbrain, which is used to build neural
> networks (simulated brain circuits) and have a client window of croquet
> running with an avatar in it, and link the neural network to the avatar to
> control it. As the avatar walked around the user could observe and
> manipulate its simulated brain. If this wouldn't be too hard any pointers
> would be welcome.
>
> I did a few quick searches on the topic of java and croquet and did not find
> the answer, but apologies if I overlooked something obvious in the docs or
> forums.
>

Well, Lets think about this a bit. First of all have you already worked
with simbrain? It apparently can generate a creature of some sort in a
world you can manipulate. That would be the first thing to do. Next,
what are you wanting to get the avatar to do? The avatar interface is
made up of a viewscreen which is a two dimensional projection of a 3
dimensional world (4 if you count time). It has ordered things about it
which are the objects in the island you are currently visiting and its
entire environmnet, but your interaction with that environment is via
the view screen, mouse and keyboard. So on one level, the immediate
thought would be to use the screen for input (one very large neural net
to process what is being seen, maybe), And to have the IM bar scanned by
a second network for interaction with other denizens. Finally your
neural net would have to generate mouse and or keyboard input to get the
avatar to interact in Second Life. But to achieve anything approaching
real autonomous effects, the full neuronet would have to run inside
croquet. I wouldn't want to tackle that by myself, but maybe you are up
to the challenge. However my experience (which is quite limited)
suggests that with the number of inputs, the random interaction of
others, and the inability to deal directly with the environment would
make this a real test of your patience and programming skills. Anything
in software can be done, if you can envision the right approach, can do
the coding, and can establish the working boundaries within which you
wish to work. One problem is that in Croquet, you will not easily have
control over the environment if your project begins to "crawl the space"
of all the croquet worlds.

So first learn the simbrain system, then with some experience behind
you, design a creature in the world of simbrain, and see how complex it
can become. If you succeed in reasonable fashion, then by all means go
for it. I don't know what to tell you about the incompatibilities of
Java/C++ relative to Squeak or smalltalk, but lets just say they are
different languages used to achieve nearly the same thing. But the devil
as always is in the details. Someone may give you more or better
information than this.

Regards,
Les H

jyoshimi

Re: Controlling Avatars with a Java Program

Hi Les, thanks for your prompt response. Let me give you more
background.

I created Simbrain so I know it pretty well. We are working towards
a next release and are adding features, and one feature we'd like to
add is an ability to control agents outside of Simbrain, e.g. an
avatar in Croquet. I don't anticipate being able to do much
interesting initially in terms of intelligently controlling an
avatar; as a first step I'd like to be able to set up the linkage (I
do have a pretty good sense of what the basic network / agent
interface would look like). I also wouldn't mind setting up a really
simple toy metaverse and controlling agents in that but that's a
separate issue...

So, really, my question is probably pretty naive (after all these
years coding on the side I'm still a neophyte). All I'm wondering
is how hard it would be to have a instance of a java program talk to
an agent in a croquet world. I figured using RMI or just sending
requests over internet or something could do this, but I'm not sure.

- Jeff

On Sep 30, 2007, at 10:37 PM, Les wrote:

> On Sun, 2007-09-30 at 20:47 -0700, jyoshimi wrote:
>> Hello and kudos to those who made croquet. It's quite nice. I'm
>> writing to
>> ask how hard it would be to control avatars in Croquet with a
>> separately
>> running java program (in particular, this one: http://
>> www.simbrain.net/).
>> The idea would be a user could run Simbrain, which is used to
>> build neural
>> networks (simulated brain circuits) and have a client window of
>> croquet
>> running with an avatar in it, and link the neural network to the
>> avatar to
>> control it. As the avatar walked around the user could observe and
>> manipulate its simulated brain. If this wouldn't be too hard any
>> pointers
>> would be welcome.
>>
>> I did a few quick searches on the topic of java and croquet and
>> did not find
>> the answer, but apologies if I overlooked something obvious in the
>> docs or
>> forums.
>>
> Well, Lets think about this a bit. First of all have you already
> worked
> with simbrain? It apparently can generate a creature of some sort in a
> world you can manipulate. That would be the first thing to do.
> Next,
> what are you wanting to get the avatar to do? The avatar interface is
> made up of a viewscreen which is a two dimensional projection of a 3
> dimensional world (4 if you count time). It has ordered things
> about it
> which are the objects in the island you are currently visiting and its
> entire environmnet, but your interaction with that environment is via
> the view screen, mouse and keyboard. So on one level, the immediate
> thought would be to use the screen for input (one very large neural
> net
> to process what is being seen, maybe), And to have the IM bar
> scanned by
> a second network for interaction with other denizens. Finally your
> neural net would have to generate mouse and or keyboard input to
> get the
> avatar to interact in Second Life. But to achieve anything
> approaching
> real autonomous effects, the full neuronet would have to run inside
> croquet. I wouldn't want to tackle that by myself, but maybe you
> are up
> to the challenge. However my experience (which is quite limited)
> suggests that with the number of inputs, the random interaction of
> others, and the inability to deal directly with the environment would
> make this a real test of your patience and programming skills.
> Anything
> in software can be done, if you can envision the right approach,
> can do
> the coding, and can establish the working boundaries within which you
> wish to work. One problem is that in Croquet, you will not easily
> have
> control over the environment if your project begins to "crawl the
> space"
> of all the croquet worlds.
>
> So first learn the simbrain system, then with some experience behind
> you, design a creature in the world of simbrain, and see how
> complex it
> can become. If you succeed in reasonable fashion, then by all
> means go
> for it. I don't know what to tell you about the incompatibilities of
> Java/C++ relative to Squeak or smalltalk, but lets just say they are
> different languages used to achieve nearly the same thing. But the
> devil
> as always is in the details. Someone may give you more or better
> information than this.
>
> Regards,
> Les H
>