Objects through sockets between dolphin applications

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Objects through sockets between dolphin applications

Smalltalkiano
Hi all,

    we need some orientation with the process of sending objects trhoug
sockets.

    We made our own little framework that mostly works, but it still need
adjustments.

    The millon dollar question is:

    How do you send objects between your dolphins applications? There is a
simple way to do that?

    At first we test Socket>>send: anObject but it doesn't seems to work
whit big and very composed objects.

    best regards..

Seb


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Bill Schwab-2
Seb,

>     How do you send objects between your dolphins applications? There is a
> simple way to do that?
>
>     At first we test Socket>>send: anObject but it doesn't seems to work
> whit big and very composed objects.

Can you tell us more?  #send: uses the binary filer, which seems to work
quite well; if you have counter examples I'm very interested in seeing them.
There was a bug in one of the stream classes that resulted in a single byte
being dropped in rare circumstances, but, that's been fixed.

Your problem is probably caused by multiple copies of some objects.  The
binary filer lives to make copies; without some means of establishing
proxies, you will find that one object will quickly turn into N copies of
itself.  Of course, you don't want to proxy everything, otherwise your
network traffic will explode.

One solution to all of this is to seek out an off the shelf ORB.  The
objects tend to stay in place, and parameters get marshaled according to
well defined rules.  I emphatically UNrecommend DCOM; I tried it.  A major
problem with DCOM is that it refuses to involve the programmer (and perhaps
indirectly the user) in decisions about when to give up on a connection.
CORBA ORBs tend to be far better about reporting errors in a timely fashion,
but, none of them are bound to Dolphin.  A fellow Dolphin user has convinced
me that the best way to add CORBA support to Dolphin is to write IIOP
support in Dolphin.  Then are stubbing issues; T-Gen might help with that.
And of course, you have to write IDL for the interfaces, which isn't fun in
itself.

There are some freeware CORBA ORBs; ILU is one example, and worked pretty
well.  For various reasons, we ended up using the C binding to write a small
app that talked CORBA to a Java/Unix box at another site, and used DCOM to
talk to a Dolphin box on our network.  We had less trouble with the 1000
mile cross-language/cross-platform/cross-ORB CORBA connections than with the
100 ft (no routers!!) DCOM link.

VW's Distributed Smalltalk is another possibility.  It's price put it out of
the running; that was in the ObjectShare days.

SOAP is a possibility.  I don't know much about it.

Not intending to discourage you, but, I'm not aware of a simple answer.
More general solutions will present interesting debugging challenges and
puzzling decisions re the correct behavior.  Purchased products put you at
the mercy of the vendor and license agreements.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Smalltalkiano
I do tell you more...

    Using Socket>>send: anObject (in the case of a listening socket), I've
found that >>onAsyncRead triggers many times when you receive one big
object. I tried a big String and pass ok but, I dont know why a
considerabily big(?) object (4KB serialized), composed with many others does
not.

    Off course the images had an identical package in common to success in
serializing and deserializing the voyager objects.

    In the case of an talker Socket, sometimes the application gets a lot of
processes trying to deploy a voyager but stays waiting indefinitely without
resolving their attempt. We saw that the socket stays on a SocketWouldBlock
state into a while (I don't like whiles too much).

we keep talking...

regards

Seb




"Bill Schwab" <[hidden email]> escribió en el mensaje
news:9sot5v$14urd1$[hidden email]...
> Seb,
>
> >     How do you send objects between your dolphins applications? There is
a
> > simple way to do that?
> >
> >     At first we test Socket>>send: anObject but it doesn't seems to work
> > whit big and very composed objects.
>
> Can you tell us more?  #send: uses the binary filer, which seems to work
> quite well; if you have counter examples I'm very interested in seeing
them.
> There was a bug in one of the stream classes that resulted in a single
byte

> being dropped in rare circumstances, but, that's been fixed.
>
> Your problem is probably caused by multiple copies of some objects.  The
> binary filer lives to make copies; without some means of establishing
> proxies, you will find that one object will quickly turn into N copies of
> itself.  Of course, you don't want to proxy everything, otherwise your
> network traffic will explode.
>
> One solution to all of this is to seek out an off the shelf ORB.  The
> objects tend to stay in place, and parameters get marshaled according to
> well defined rules.  I emphatically UNrecommend DCOM; I tried it.  A major
> problem with DCOM is that it refuses to involve the programmer (and
perhaps
> indirectly the user) in decisions about when to give up on a connection.
> CORBA ORBs tend to be far better about reporting errors in a timely
fashion,
> but, none of them are bound to Dolphin.  A fellow Dolphin user has
convinced
> me that the best way to add CORBA support to Dolphin is to write IIOP
> support in Dolphin.  Then are stubbing issues; T-Gen might help with that.
> And of course, you have to write IDL for the interfaces, which isn't fun
in
> itself.
>
> There are some freeware CORBA ORBs; ILU is one example, and worked pretty
> well.  For various reasons, we ended up using the C binding to write a
small
> app that talked CORBA to a Java/Unix box at another site, and used DCOM to
> talk to a Dolphin box on our network.  We had less trouble with the 1000
> mile cross-language/cross-platform/cross-ORB CORBA connections than with
the
> 100 ft (no routers!!) DCOM link.
>
> VW's Distributed Smalltalk is another possibility.  It's price put it out
of

> the running; that was in the ObjectShare days.
>
> SOAP is a possibility.  I don't know much about it.
>
> Not intending to discourage you, but, I'm not aware of a simple answer.
> More general solutions will present interesting debugging challenges and
> puzzling decisions re the correct behavior.  Purchased products put you at
> the mercy of the vendor and license agreements.
>
> Have a good one,
>
> Bill
>
> --
> Wilhelm K. Schwab, Ph.D.
> [hidden email]
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Smalltalkiano
Anybody knows why Socket>>send: anObject doesn't work?
and why it works for large Strings?

any ideas?



"Smalltalkiano" <[hidden email]> escribió en el mensaje
news:9sp2d8$14uljb$[hidden email]...
> I do tell you more...
>
>     Using Socket>>send: anObject (in the case of a listening socket), I've
> found that >>onAsyncRead triggers many times when you receive one big
> object. I tried a big String and pass ok but, I dont know why a
> considerabily big(?) object (4KB serialized), composed with many others
does
> not.
>
>     Off course the images had an identical package in common to success in
> serializing and deserializing the voyager objects.
>
>     In the case of an talker Socket, sometimes the application gets a lot
of
> processes trying to deploy a voyager but stays waiting indefinitely
without
> resolving their attempt. We saw that the socket stays on a
SocketWouldBlock

> state into a while (I don't like whiles too much).
>
> we keep talking...
>
> regards
>
> Seb
>
>
>
>
> "Bill Schwab" <[hidden email]> escribió en el mensaje
> news:9sot5v$14urd1$[hidden email]...
> > Seb,
> >
> > >     How do you send objects between your dolphins applications? There
is
> a
> > > simple way to do that?
> > >
> > >     At first we test Socket>>send: anObject but it doesn't seems to
work

> > > whit big and very composed objects.
> >
> > Can you tell us more?  #send: uses the binary filer, which seems to work
> > quite well; if you have counter examples I'm very interested in seeing
> them.
> > There was a bug in one of the stream classes that resulted in a single
> byte
> > being dropped in rare circumstances, but, that's been fixed.
> >
> > Your problem is probably caused by multiple copies of some objects.  The
> > binary filer lives to make copies; without some means of establishing
> > proxies, you will find that one object will quickly turn into N copies
of
> > itself.  Of course, you don't want to proxy everything, otherwise your
> > network traffic will explode.
> >
> > One solution to all of this is to seek out an off the shelf ORB.  The
> > objects tend to stay in place, and parameters get marshaled according to
> > well defined rules.  I emphatically UNrecommend DCOM; I tried it.  A
major
> > problem with DCOM is that it refuses to involve the programmer (and
> perhaps
> > indirectly the user) in decisions about when to give up on a connection.
> > CORBA ORBs tend to be far better about reporting errors in a timely
> fashion,
> > but, none of them are bound to Dolphin.  A fellow Dolphin user has
> convinced
> > me that the best way to add CORBA support to Dolphin is to write IIOP
> > support in Dolphin.  Then are stubbing issues; T-Gen might help with
that.
> > And of course, you have to write IDL for the interfaces, which isn't fun
> in
> > itself.
> >
> > There are some freeware CORBA ORBs; ILU is one example, and worked
pretty
> > well.  For various reasons, we ended up using the C binding to write a
> small
> > app that talked CORBA to a Java/Unix box at another site, and used DCOM
to
> > talk to a Dolphin box on our network.  We had less trouble with the 1000
> > mile cross-language/cross-platform/cross-ORB CORBA connections than with
> the
> > 100 ft (no routers!!) DCOM link.
> >
> > VW's Distributed Smalltalk is another possibility.  It's price put it
out
> of
> > the running; that was in the ObjectShare days.
> >
> > SOAP is a possibility.  I don't know much about it.
> >
> > Not intending to discourage you, but, I'm not aware of a simple answer.
> > More general solutions will present interesting debugging challenges and
> > puzzling decisions re the correct behavior.  Purchased products put you
at

> > the mercy of the vendor and license agreements.
> >
> > Have a good one,
> >
> > Bill
> >
> > --
> > Wilhelm K. Schwab, Ph.D.
> > [hidden email]
> >
> >
> >
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Steve Waring-2
Hi Seb,

Can you send me some code that shows what you are doing?

My email address is; [hidden email]

Thanks,
Steve
www.dolphinharbor.org


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Bill Schwab-2
In reply to this post by Smalltalkiano
Seb,

> I do tell you more...
>
>     Using Socket>>send: anObject (in the case of a listening socket), I've
> found that >>onAsyncRead triggers many times when you receive one big
> object.

It depends on what you mean by "many times", but, this could be as simple as
your network chopping the blob up into blocks.  IIRC, datagrams (UDP) always
travel as sent, but, TCP (aka "stream") sockets reserve the right to break
up large messages and even to combine small ones.  While that might sound
bad, it's really not, because TCP sockets guarantee to get the data through
the channel in the right order, or to report an error.  The caveat is that
neither protocol has a magic solution to knowing when the other side has
dropped out; sometimes you get an error message, and sometimes you don't -
that's why all the talk about pinging algoritms.

Something to avoid - keep alive packets.  MSDN still contains the following
article:

  http://support.microsoft.com/support/kb/articles/Q140/3/25.asp

Keep alive packets sound great, but, turn out not to be practical.  The idea
was basically hobbled from the beginning, with the defaults taking something
like 40 hours(???) to finally time out a connection.  If you want to ping
(which you might need to do eventually), make it part of your protocol that
runs over the socket.


> I tried a big String and pass ok but, I dont know why a
> considerabily big(?) object (4KB serialized), composed with many others
does
> not.

Can you post an example?


>     Off course the images had an identical package in common to success in
> serializing and deserializing the voyager objects.

That's fine, but, not strictly necessary if you have working STB conversions
and are sending from old image to new image.  Of course, a round trip will
need both images at the same version, otherwise one of them will get an STB
blob that it can't read.


>     In the case of an talker Socket, sometimes the application gets a lot
of
> processes trying to deploy a voyager but stays waiting indefinitely
without
> resolving their attempt. We saw that the socket stays on a
SocketWouldBlock
> state into a while (I don't like whiles too much).

You could simply be getting hung up on your message loop.  If you block the
main UI thread (Smalltalk Process), then your image can find itself unable
to "see" the arriving socket data, and can deadlock.

You said you sent a big string - how big?  Try something larger than the 4kB
serialized object that's giving you trouble.  If that works then I'd start
to suspect something about the in-filing of your object.  If your image
deadlocks on the string, then there might be need for another
#forkMainIfMain somewhere in the base system.  You could try forking the
read away from the main thread to see if it works.  However, you are then
venturing into multi-threaded programming, which (while ultimately worth the
effort) can be very challenging.

I use #send: and #receive:, but, only for small objects.  Most of my heavy
socket I/O runs on other threads and is (hopefully!!) synchronized after the
fact.  If there were some kind of deadlock potential in #send:/#receive:, I
wouldn't see it.  The good news if you are starting down this path now, is
that Dolphin (especially since a patch or two to 3.06 and continuing with
4.x) has become very robust with respect to shared (weak) collections,
ensured execution, and terminating threads.  I still have a few dangling
work arounds for old bugs, but, am gradually identifying and removing them.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

James Foster-3
In reply to this post by Smalltalkiano
Can you move objects between Dolphin applications using STBFiler? That is,
do you think the problem is the sockets (TCP/IP) or the STBFiler? Try to
reduce the number of variables (serialization vs. communication) and come up
with as simple a failure as possible. See my recent post for an
fix/enhancement to STBInFiler.

James Foster
Fargo, ND

"Smalltalkiano" <[hidden email]> wrote in message
news:9sp2d8$14uljb$[hidden email]...
> I do tell you more...
>
>     Using Socket>>send: anObject (in the case of a listening socket), I've
> found that >>onAsyncRead triggers many times when you receive one big
> object. I tried a big String and pass ok but, I dont know why a
> considerabily big(?) object (4KB serialized), composed with many others
does
> not.
>
>     Off course the images had an identical package in common to success in
> serializing and deserializing the voyager objects.
>
>     In the case of an talker Socket, sometimes the application gets a lot
of
> processes trying to deploy a voyager but stays waiting indefinitely
without
> resolving their attempt. We saw that the socket stays on a
SocketWouldBlock

> state into a while (I don't like whiles too much).
>
> we keep talking...
>
> regards
>
> Seb
>
>
>
>
> "Bill Schwab" <[hidden email]> escribió en el mensaje
> news:9sot5v$14urd1$[hidden email]...
> > Seb,
> >
> > >     How do you send objects between your dolphins applications? There
is
> a
> > > simple way to do that?
> > >
> > >     At first we test Socket>>send: anObject but it doesn't seems to
work

> > > whit big and very composed objects.
> >
> > Can you tell us more?  #send: uses the binary filer, which seems to work
> > quite well; if you have counter examples I'm very interested in seeing
> them.
> > There was a bug in one of the stream classes that resulted in a single
> byte
> > being dropped in rare circumstances, but, that's been fixed.
> >
> > Your problem is probably caused by multiple copies of some objects.  The
> > binary filer lives to make copies; without some means of establishing
> > proxies, you will find that one object will quickly turn into N copies
of
> > itself.  Of course, you don't want to proxy everything, otherwise your
> > network traffic will explode.
> >
> > One solution to all of this is to seek out an off the shelf ORB.  The
> > objects tend to stay in place, and parameters get marshaled according to
> > well defined rules.  I emphatically UNrecommend DCOM; I tried it.  A
major
> > problem with DCOM is that it refuses to involve the programmer (and
> perhaps
> > indirectly the user) in decisions about when to give up on a connection.
> > CORBA ORBs tend to be far better about reporting errors in a timely
> fashion,
> > but, none of them are bound to Dolphin.  A fellow Dolphin user has
> convinced
> > me that the best way to add CORBA support to Dolphin is to write IIOP
> > support in Dolphin.  Then are stubbing issues; T-Gen might help with
that.
> > And of course, you have to write IDL for the interfaces, which isn't fun
> in
> > itself.
> >
> > There are some freeware CORBA ORBs; ILU is one example, and worked
pretty
> > well.  For various reasons, we ended up using the C binding to write a
> small
> > app that talked CORBA to a Java/Unix box at another site, and used DCOM
to
> > talk to a Dolphin box on our network.  We had less trouble with the 1000
> > mile cross-language/cross-platform/cross-ORB CORBA connections than with
> the
> > 100 ft (no routers!!) DCOM link.
> >
> > VW's Distributed Smalltalk is another possibility.  It's price put it
out
> of
> > the running; that was in the ObjectShare days.
> >
> > SOAP is a possibility.  I don't know much about it.
> >
> > Not intending to discourage you, but, I'm not aware of a simple answer.
> > More general solutions will present interesting debugging challenges and
> > puzzling decisions re the correct behavior.  Purchased products put you
at

> > the mercy of the vendor and license agreements.
> >
> > Have a good one,
> >
> > Bill
> >
> > --
> > Wilhelm K. Schwab, Ph.D.
> > [hidden email]
> >
> >
> >
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Smalltalkiano
In reply to this post by Steve Waring-2
Open the Chat, connect two machines and send a String with 30000 characters
and see wath happens.


"Steve Waring" <[hidden email]> escribió en el mensaje
news:9srbkb$14bcc2$[hidden email]...

> Hi Seb,
>
> Can you send me some code that shows what you are doing?
>
> My email address is; [hidden email]
>
> Thanks,
> Steve
> www.dolphinharbor.org
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Smalltalkiano
In reply to this post by Bill Schwab-2
Dear Bill,

> > I do tell you more...
> >
> >     Using Socket>>send: anObject (in the case of a listening socket),
I've
> > found that >>onAsyncRead triggers many times when you receive one big
> > object.
>
> It depends on what you mean by "many times", but, this could be as simple
as
> your network chopping the blob up into blocks.  IIRC, datagrams (UDP)
always
> travel as sent, but, TCP (aka "stream") sockets reserve the right to break
> up large messages and even to combine small ones.  While that might sound
> bad, it's really not, because TCP sockets guarantee to get the data
through
> the channel in the right order, or to report an error.  The caveat is that
> neither protocol has a magic solution to knowing when the other side has
> dropped out; sometimes you get an error message, and sometimes you don't -
> that's why all the talk about pinging algoritms.

I mean, many times per object serialized. You pass an array (it probably be
chopped, we use TCP/IP alone) and get many onAsyncRead on the receptionist
socket. We use a little state machine to deaden them, we read a header
(fixed size array of ten bytes) wich tell us the size of the incomming
serialized object, then we proceed to read the "body" of this incoming
object. After reading that byteArray we use the STBInFiler.
If you don't deaden those redundant onAsyncRead, then you will get problems
to deserialize your incoming object. (Not STB format). Because it will make
you process an array that is incomplete (to build the incommingObject) by
that time.

> Something to avoid - keep alive packets.  MSDN still contains the
following
> article:

    OK, thank you for that

>   http://support.microsoft.com/support/kb/articles/Q140/3/25.asp
>
> Keep alive packets sound great, but, turn out not to be practical.  The
idea
> was basically hobbled from the beginning, with the defaults taking
something
> like 40 hours(???) to finally time out a connection.  If you want to ping
> (which you might need to do eventually), make it part of your protocol
that
> runs over the socket.
>
>
> > I tried a big String and pass ok but, I dont know why a
> > considerabily big(?) object (4KB serialized), composed with many others
> does
> > not.

I have tested strings sized in 700KB

> Can you post an example?
>
>
> >     Off course the images had an identical package in common to success
in
> > serializing and deserializing the voyager objects.
>
> That's fine, but, not strictly necessary if you have working STB
conversions
> and are sending from old image to new image.  Of course, a round trip will
> need both images at the same version, otherwise one of them will get an
STB
> blob that it can't read.
>
>
> >     In the case of an talker Socket, sometimes the application gets a
lot
> of
> > processes trying to deploy a voyager but stays waiting indefinitely
> without
> > resolving their attempt. We saw that the socket stays on a
> SocketWouldBlock
> > state into a while (I don't like whiles too much).
>
> You could simply be getting hung up on your message loop.  If you block
the
> main UI thread (Smalltalk Process), then your image can find itself unable
> to "see" the arriving socket data, and can deadlock.

    The UI work perfecly even when that happens. You can simulate this
unplugging the RJ45 of one of the machines when they are talking. The talker
don't get a nice close and the sender can't complete their sendings.
    Off course this isn't a normal operation but (I don't know why) it
happens with time.

> You said you sent a big string - how big?  Try something larger than the
4kB
> serialized object that's giving you trouble.  If that works then I'd start
> to suspect something about the in-filing of your object.  If your image
> deadlocks on the string, then there might be need for another
> #forkMainIfMain somewhere in the base system. You could try forking the
> read away from the main thread to see if it works. However, you are then
> venturing into multi-threaded programming, which (while ultimately worth
the
> effort) can be very challenging.

    I already fork the readings and the writings.

> I use #send: and #receive:, but, only for small objects.  Most of my heavy
> socket I/O runs on other threads and is (hopefully!!) synchronized after
the
> fact.

    Whay do you mean when you say "heavy socket I/O" ? fecuency? size?
complexity (high degree of composition)? all together? between many
applications?

 If there were some kind of deadlock potential in #send:/#receive:, I
> wouldn't see it.  The good news if you are starting down this path now, is
> that Dolphin (especially since a patch or two to 3.06 and continuing with
> 4.x) has become very robust with respect to shared (weak) collections,
> ensured execution, and terminating threads.  I still have a few dangling
> work arounds for old bugs, but, am gradually identifying and removing
them.

        Yes, we can see that, but I think that the source of this processes
problem is not in processes itself but in the socket solution that could be
improved.

> Have a good one,

Have a good you too,

Seb

>
> Bill
>
> --
> Wilhelm K. Schwab, Ph.D.
> [hidden email]
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Steve Waring-2
"Smalltalkiano" <[hidden email]> wrote in message
news:9srrre$1533tl$[hidden email]...
> Open the Chat, connect two machines and send a String with 30000
characters
> and see wath happens.

Hi Seb,

I just tried that over my local network and it worked as expected. What
happens for you?

>  Whay do you mean when you say "heavy socket I/O" ? fecuency? size?
> complexity (high degree of composition)? all together? between many
> applications?

I use the Dolphin Sockets package regularly, and I have had no problems. I
have stress tested up to 100 simultaneous connections to the limit of my
local network (10/100) bandwidth. Dolphin implements the asynchronous IO
model which can be a limiting factor, but Heurisitic 6 at [1] indicates that
this model is a good choice if you only need to support between 100 and 1000
simultaneous connections.

I suspect that you either have a problem with your network configuration or
your client/server model. The Winsock Programmers FAQ [2] contains good
information on effective client/server models.

> We use a little state machine to deaden them, we read a header
> (fixed size array of ten bytes) wich tell us the size of the incomming
> serialized object, then we proceed to read the "body" of this incoming
> object.

If you know the number of bytes you need to read, and you are reading the
socket in a background process, why dont you just ignore the onAsyncRead
event and block until you have read the required number of bytes?

You refer to the sockets as "talker" and "application". During a connection
it is handy to think in these terms (ie client/server?), with the server
being the image with the listening socket. Once the connection is
established, you simply have two streams, each of which can be closed
independantly of the other.

It is hard to provide more information because I am still not sure of what
you are doing. Specifically I do not understand what you mean by;

>In the case of an talker Socket, sometimes the application
>gets a lot of processes trying to deploy a voyager
>but stays waiting indefinitely without resolving their attempt.

I have found that a good understanding of how sockets close, and what the
socket error messages indicate, is important in implementing client/server
models, MSDN [3] and [2] have good information.

Steve
www.dolphinharbor.org


[1] http://tangentsoft.net/wskfaq/articles/io-strategies.html
[2] http://tangentsoft.net/wskfaq/
[3] http://msdn.microsoft.com/library/en-us/winsock/apistart_9g1e.asp


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Bill Schwab
Steve,

> I have found that a good understanding of how sockets close, and what the
> socket error messages indicate, is important in implementing client/server
> models, MSDN [3] and [2] have good information.
> www.dolphinharbor.org
> [1] http://tangentsoft.net/wskfaq/articles/io-strategies.html
> [2] http://tangentsoft.net/wskfaq/
> [3] http://msdn.microsoft.com/library/en-us/winsock/apistart_9g1e.asp

Thanks for the links.  If there is anything else you would care to write as
a summary of error messages or what can go wrong when sockets close, etc.,
please feel encouraged to post it either here or on the Wiki.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Smalltalkiano
In reply to this post by Steve Waring-2
Hi Steve,

"Steve Waring" <[hidden email]> escribió en el mensaje
news:9ssicb$iki5o$[hidden email]...

> "Smalltalkiano" <[hidden email]> wrote in message
> news:9srrre$1533tl$[hidden email]...
> > Open the Chat, connect two machines and send a String with 30000
> characters
> > and see wath happens.
>
> Hi Seb,
>
> I just tried that over my local network and it worked as expected. What
> happens for you?

    When I do that, I get an STB format error. We found that, when you send
an object, the first onAsyncRead come and the deserializing process begin,
but in the middle of this deserialization (big object) come another
onAsyncRead and the deserialization fails. Perhaps because this serialized
object is chopped.

> >  Whay do you mean when you say "heavy socket I/O" ? fecuency? size?
> > complexity (high degree of composition)? all together? between many
> > applications?
>
> I use the Dolphin Sockets package regularly, and I have had no problems. I
> have stress tested up to 100 simultaneous connections to the limit of my
> local network (10/100) bandwidth. Dolphin implements the asynchronous IO
> model which can be a limiting factor, but Heurisitic 6 at [1] indicates
that
> this model is a good choice if you only need to support between 100 and
1000
> simultaneous connections.

Wow, we need something like that! . We had 10/100 Ethernet adapters on every
machine and a 3Com HUB of 10Mb (3Com TP16C).
I'm talking rigth now with my partners and we though that could be that your
test goes ok because the generous band width of your HUB. Have do you tested
on a 10Mb HUB too? I'm interested to know your results.

>
> I suspect that you either have a problem with your network configuration
or
> your client/server model. The Winsock Programmers FAQ [2] contains good
> information on effective client/server models.

The client/server model that we use is the same as Dolphin, we also have a
state for the reading process.

> > We use a little state machine to deaden them, we read a header
> > (fixed size array of ten bytes) wich tell us the size of the incomming
> > serialized object, then we proceed to read the "body" of this incoming
> > object.
>
> If you know the number of bytes you need to read, and you are reading the
> socket in a background process, why dont you just ignore the onAsyncRead
> event and block until you have read the required number of bytes?

    We do ignore the redundant onAsyncRead. Thanks to the serverSocketState.

>
> You refer to the sockets as "talker" and "application". During a
connection
> it is handy to think in these terms (ie client/server?), with the server
> being the image with the listening socket. Once the connection is
> established, you simply have two streams, each of which can be closed
> independantly of the other.

    Yes it is. Thanks. I refer as tarker and lisener because we have
applications chatting so for each application we have a clientSocket (the
talker) and a serverSocker (the listener). The talker of application A is
linked with the listener of application B and viceversa.

>
> It is hard to provide more information because I am still not sure of what
> you are doing. Specifically I do not understand what you mean by;

    When I improve the prolixity of my package I´ll send it to you.
>
> >In the case of an talker Socket, sometimes the application
> >gets a lot of processes trying to deploy a voyager
> >but stays waiting indefinitely without resolving their attempt.
>
> I have found that a good understanding of how sockets close, and what the
> socket error messages indicate, is important in implementing client/server
> models, MSDN [3] and [2] have good information.

    I already hooked all the triggers for closing properly when things occur
to make an appropiate housekeeping.
>
> Steve
> www.dolphinharbor.org
>
>
> [1] http://tangentsoft.net/wskfaq/articles/io-strategies.html
> [2] http://tangentsoft.net/wskfaq/
> [3] http://msdn.microsoft.com/library/en-us/winsock/apistart_9g1e.asp
>

thank you very much for your information, we appreciate talking about our
challenges because it helps to think

best regards,

Seb


Reply | Threaded
Open this post in threaded view
|

Re: Objects through sockets between dolphin applications

Steve Waring-2
Hi Seb,

>     When I do that, I get an STB format error. We found that, when you
send
> an object, the first onAsyncRead come and the deserializing process begin,
> but in the middle of this deserialization (big object) come another
> onAsyncRead and the deserialization fails. Perhaps because this serialized
> object is chopped.

In the work that I have done, I have not made use of this event and do not
register a handler for it. I can see that OA do use it in the Chat example.
You mentioned in a previous message that you did not like the while loop,
however this loop will end up waiting on a semaphore so it is an efficient
way to block a process reading from a socket.

> test goes ok because the generous band width of your HUB. Have do you
tested
> on a 10Mb HUB too? I'm interested to know your results.

I did use it once over a 56K connection :) I have never seen an STB format
error. Have you tried this between two fresh images? Can you post the
walkback?

> We do ignore the redundant onAsyncRead. Thanks to the serverSocketState.

As a test you could try using #removeEventsTriggeredFor: Which could show
you if the triggering or handling of this event (even when you ignore it) is
causing the problem.

> applications chatting so for each application we have a clientSocket (the
> talker) and a serverSocker (the listener). The talker of application A is
> linked with the listener of application B and viceversa.

Is it always 1 taker to 1 listener?

>     When I improve the prolixity of my package I´ll send it to you.

Great, I do enjoy playing around with this stuff, and I would like to take a
look at what you are doing.

The other thing you may want to try is to remove the STBFiler from your code
and test with raw byteArrays. It can be hard to track down errors in the
STBFiler, but if you can send and receive the same byteArray then you can
further isolate the problem.

I dont have a permanent IP, but if you email me ([hidden email]) I
can send one through for a Chat test!

Thanks,
Steve
www.dolphinharbor.org