Threads, Events and Sockets

I am writing a program that will use threads, events and sockets.  I want to
make sure I understand the interactions of those three so that I can get it
right.

I have three objects - A, B and C -  each of them running in their own
thread (let's call them Ta, Tb, Tc), blocking on a socket read.  All of them
have a reference to the _same_ dictionary wrapped in a class to make it
thread-safe.  They all have registered interest in any changes made to the
dictionary. One of them - say A - gets a change request from it's TCP/IP
client.  It makes a change to the dictionary which in turn sends an event to
all registered objects - A, B and C.

The event will trigger a method in each object.  Now if I understand it
right,  the methods (in A, B and C) triggered by the event will all run in
Ta.  Is this correct?

I want to make sure I understand this part before I dive in to my sockets
questions.

TIA

Elliot

Elliot Finley wrote:

> The event will trigger a method in each object.  Now if I understand it
> right,  the methods (in A, B and C) triggered by the event will all run in
> Ta.  Is this correct?

Yes, that's right.

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:3ffa6a53$0$61070$[hidden email]...
> Elliot Finley wrote:
>
> > The event will trigger a method in each object.  Now if I understand it
> > right,  the methods (in A, B and C) triggered by the event will all run
in
> > Ta.  Is this correct?
>
> Yes, that's right.

Now for one of my socket questions:

Can I have a reader in thread X and a writer in thread Y reading and writing
on the same socket at the same time?

TIA

Elliot

Elliot,

Unless the VM prevents you, I think you should be good.  Windows doesn't
really care which threads read and write to/from sockets.  Reads and writes
on sockets are not interlinked on any OS I've ever programmed for.

The only issues you run into occur then two or more threads are reading from
or writing to the same socket.  But this is par for the course with
threads--and you sound like you've avoided it.

I've often done things like this in other languages, so anything preventing
it would have to be Dolphin specific (I highly doubt this to be the case).

Cheers,

--GK

Elliot Finley wrote:

> Now for one of my socket questions:
>
> Can I have a reader in thread X and a writer in thread Y reading and
> writing on the same socket at the same time?

I don't know if this will help, but Dolphin's "threads" are Processes not real
Window's threads.  That's to say that they are lightweight executions streams
implemented entirely in the Dolphin VM -- Windows itself sees all Smalltalk
code executing on one OS thread.

The current socket implementation (this is scheduled to change, I think) uses
asynchronous reads and writes at the OS/socket level, and arranges for Windows
to notify Dolphin whenever it's possible to read or write to the socket (i.e.
when there's data available to read, or enough free buffer space to allow a
write without blocking).  And Dolphin in turn blocks/enables the
reading/writing Process to emulate a "synchronous" API.  So I'd *guess* that
there's no reason in principle why you shouldn't be able to use multiple
Processes.

However, the Socket object does not have Mutex protection internally, so I'd
expect that you'd have take some care (using your own Mutex if necessary) to
ensure that you weren't breaking the Dolphin Socket implementation.  You'd have
to study the Socket code to see if its safe (unless Andy or Blair pops up to
say that it's OK by design).   From a *very quick* look at the Socket code, it
*appears* that it should Just Work for the case where there's one reader
Process and one writer Process, but...

FWIW, you might find it easier to use the asynchronous interface to Dolphin
sockets yourself, rather than using two threads.  But, again, I haven't tried
this myself, so I can't say how easy/difficult it would be.

    -- chris

Elliot, Chris,

> > Can I have a reader in thread X and a writer in thread Y reading and
> > writing on the same socket at the same time?

Yes, which is why you don't want to allow it to happen :)  As Chris points
out, Dolphin uses "green threads" which are somewhat limited in their
ability to cause trouble, but only slightly limited.  The socket streams
will be even more willing to allow threads to stomp on each other, so mutex
protection is in order.  When (note I didn't say "if"<g>) your code
deadlocks in the early stages, consider using my ProcessViewer goodie.  The
Dolphin ProcessBrowser is more powerful, but is sometimes a little harder to
use for tracking down simple deadlocks.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

"Bill Schwab" <[hidden email]> wrote in message
news:btkhs1$8avqo$[hidden email]...
> Elliot, Chris,
>
> > > Can I have a reader in thread X and a writer in thread Y reading and
> > > writing on the same socket at the same time?
>
> Yes, which is why you don't want to allow it to happen :)  As Chris points
> out, Dolphin uses "green threads" which are somewhat limited in their
> ability to cause trouble, but only slightly limited.  The socket streams
> will be even more willing to allow threads to stomp on each other, so
mutex
> protection is in order.  When (note I didn't say "if"<g>) your code
> deadlocks in the early stages, consider using my ProcessViewer goodie.
The
> Dolphin ProcessBrowser is more powerful, but is sometimes a little harder
to
> use for tracking down simple deadlocks.

Well, this puts a _huge_ damper on my project.

The reason I wanted a reader thread and a writer thread for every socket is
because:

For every client that the server serves, the reader thread blocks reading an
object from the socket.  The writer thread blocks reading an object from its
output queue, then when it gets its object off the queue, it writes it to
the socket.  But a reader on socketA can cause a write to happen on socketB
(but threadB is still blocking on a read from socketB).

How to accomplish this?

Elliot

P.S.  Dolphin Smalltalk is the first threaded language that I've used that I
haven't been able to have both a reader and a writer on the same socket at
the same time.

Elliot,

> Well, this puts a _huge_ damper on my project.

I think that Bill was probably being a bit too pessimistic.  As I said, it
*looks to me* as if the Dolphin implementation is OK for use with 1 reader and
1 writer -- but of course it's up to you to verify that.  And you'll have to be
careful about how you open and close the connection -- but that should be
simple enough.


> The reason I wanted a reader thread and a writer thread for every socket
> is because:
>
> For every client that the server serves, the reader thread blocks reading
> an object from the socket.  The writer thread blocks reading an object
> from its output queue, then when it gets its object off the queue, it
> writes it to the socket.  But a reader on socketA can cause a write to
> happen on socketB (but threadB is still blocking on a read from socketB).

Ah now, that sounds more complicated.  Maybe I'm misunderstanding you, but it
sounds as if you can have two Processes writing to the same socket at the same
time -- which would be a no-no in any language/implementation.

Perhaps you can clarify what you are trying to do a little more ?

    -- chris

"Chris Uppal" <[hidden email]> wrote in message
news:3ffffdb7$1$61063$[hidden email]...
> > The reason I wanted a reader thread and a writer thread for every socket
> > is because:
> >
> > For every client that the server serves, the reader thread blocks
reading
> > an object from the socket.  The writer thread blocks reading an object
> > from its output queue, then when it gets its object off the queue, it
> > writes it to the socket.  But a reader on socketA can cause a write to
> > happen on socketB (but threadB is still blocking on a read from
socketB).
>
> Ah now, that sounds more complicated.  Maybe I'm misunderstanding you, but
it
> sounds as if you can have two Processes writing to the same socket at the
same
> time -- which would be a no-no in any language/implementation.
>
> Perhaps you can clarify what you are trying to do a little more ?

Basically I have a server that multiple simultaneous IP clients can connect
to.  The server is just a central information repository.  A client can push
information to the server, update existing information on the server and it
can register for updates on particular pieces of information on the server.

That last part - registering for updates - is the reason that an update from
socketA can cause an object to be put on the outputQueue of socketWriterB.

There is only one reader and one writer per socket.

I also looked at the socket code, and it _appears_ that it would be okay to
have a reader and a writer on a socket at the same time.  But it also
_appears_ that Bill was speaking from experience.

Andy? Blair?  Now would be a really good time to chime in. :-)

Bill - What specific deadlock problems did you run into?  and what caused
them?

TIA

Elliot

Elliot,

> That last part - registering for updates - is the reason that an update
> from socketA can cause an object to be put on the outputQueue of
> socketWriterB.

Ah, I see -- the *queue* can have multiple writers, but it is serviced by
exactly one Process that takes data off and stuffs it down the socket.

I took a closer look at the socket code, and I still can't see any reason why 1
reader + 1 writer shouldn't work in theory (it certainly does work in
practice), provided that opening and closing is handled sensitively.

One slightly dodgy area is the handling of errors, though I think it'd still be
OK.  The "problem" is that an error reading (say) is notified to both
Processes, so if there's a blocked writer at the same time, it will be woken up
and receive an error too.  OTOH, if the writer happens not to be in the socket
code at that time, then it'll proceed unhampered.  I don't expect that to be a
problem, though it may be worth a little thought.  The special case when the
socket is partially shutdown at the far end is handed OK -- e.g. if the far end
closes just the send side of its socket (i.e. uses shutdown(SD_SEND)*), then
the reader Process gets socket closed notification, but the writer Process is
(correctly) unaffected, and is able to continue sending data.

So I guess you're OK.

    -- chris

[*] Dolphin doesn't expose an API for doing partial shutdowns, though it's
trivial to add.

Elliot,

> Well, this puts a _huge_ damper on my project.

You should not look at it that way.  However, you do need to protect the
various streams or things will get ugly.


> The reason I wanted a reader thread and a writer thread for every socket
is
> because:
>
> For every client that the server serves, the reader thread blocks reading
an
> object from the socket.  The writer thread blocks reading an object from
its
> output queue, then when it gets its object off the queue, it writes it to
> the socket.  But a reader on socketA can cause a write to happen on
socketB
> (but threadB is still blocking on a read from socketB).
>
> How to accomplish this?

The best thing to do is to set up fairly strict mutex protection (probably
one mutex per connection and one guarding the mapping of connections to
machines, unless you use a shared lookup table for the latter, in which case
it is already thread safe).  You will be much happier if you have to track
down deadlocks than you would be with dirty read problems, hence my advice
to generously protect your sockets.

Using a connection-level mutex to protect the socket write stream will avoid
corrupting the data sent to the other side, and might avoid the need for
your writer threads.  If you have only one thread that reads from any one
socket, then you will probably be ok allowing that thread to read without
enclosing the reads in critical sections, or you can add a separate mutex
for the read stream.

If you find a deadlock, then a good place to start is to look at waits
inside of critical sections.  You might be making the call using the wrong
mutex, though if you think of connection-level mutexes and one mutex to
guard connection mapping, you will probably not have that problem.  Perhaps
the critical section covers too much code (be careful before concluding that
however, because it could cause worse problems if you are wrong), or moving
the call to another thread might help, which your writer threads are likely
doing for you.  An alternative is to use short-lived threads to make
blocking calls that would otherwise cause trouble.


> P.S.  Dolphin Smalltalk is the first threaded language that I've used that
I
> haven't been able to have both a reader and a writer on the same socket at
> the same time.

I probably said that, but did not intend to say that.  My concern was based
on your emphasis on the event system (or am I confusing you with another
poster?), which is not thread safe, and might eventually introduce callers
from the UI thread.

The more you have started with sound protection, the easier it will be to
get it working.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]