STB Bug???

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

STB Bug???

Bernardo Clavijo
Hello,
I'm working in a project using Dolphin Smalltalk XP over W2K, and we're
using STB to send objects between a server and some clients.
We started in a single threaded aproach, and it worked fine. But then we
make it multi threaded, and when 2 or more thread use STBOut, the things
started to went... just bad.
In the receiver side we get an "Not in STB format" error.
So far... I've found some references to a similar problem in google
groups... so I've tried to reproduce the error using stb over a view as
they do, and the results are the same, fine in single-thread, bad in multi
thread.
So far, the proposed solution to the bug, that was uncomment a line in some
method of STB, didn't work in our case.
Any help or info would be greatly appreciated. I've read that some people
re-implement all STB functions in other class for their works, that would
be fine too :P

Sorry 'bout my bad english.

Saludos

    BJ
Bernardo Clavijo


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Christopher J. Demers
"Bernardo Clavijo" <[hidden email]> wrote in message
news:boj97h$1fbs9s$[hidden email]...
> Hello,
> I'm working in a project using Dolphin Smalltalk XP over W2K, and we're
> using STB to send objects between a server and some clients.
> We started in a single threaded aproach, and it worked fine. But then we
> make it multi threaded, and when 2 or more thread use STBOut, the things
> started to went... just bad.
> In the receiver side we get an "Not in STB format" error.
...

I think what you are experiencing may not be quite the same kind of STB
corruption Bill and I experienced (I am assume that is what you found via
Google).  For me at least the corruption was usually class names, and
returned a different error from what you are getting.  The error you are
experiencing is due to a incorrect STB prefix.  See STBFiler
class<<peekForSignatureIn: to see the code.  It expects a valid STB stream
to start with the prefix '!STB '.  Look at what bytes are actually being
sent.  Perhaps the threads are corrupting the stream in some way.  You might
make sure the two threads are not trying to send multiple objects at the
same time and mangling the data stream.  You may need to add a Mutex of
Semaphore.  I don't have a great deal of experience with threads so others
may be able to offer better advice.

Chris


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Schwab,Wilhelm K
Bernardo,

> > I'm working in a project using Dolphin Smalltalk XP over W2K, and we're
> > using STB to send objects between a server and some clients.
> > We started in a single threaded aproach, and it worked fine. But then we
> > make it multi threaded, and when 2 or more thread use STBOut, the things
> > started to went... just bad.
>
> I think what you are experiencing may not be quite the same kind of STB
> corruption Bill and I experienced (I am assume that is what you found via
> Google).  For me at least the corruption was usually class names, and
> returned a different error from what you are getting.  The error you are
> experiencing is due to a incorrect STB prefix.  See STBFiler
> class<<peekForSignatureIn: to see the code.  It expects a valid STB stream
> to start with the prefix '!STB '.  Look at what bytes are actually being
> sent.  Perhaps the threads are corrupting the stream in some way.  You
might
> make sure the two threads are not trying to send multiple objects at the
> same time and mangling the data stream.  You may need to add a Mutex of
> Semaphore.

I think Chris is correct in his analysis of your problem.  You will need to
mutex protect your (socket?) stream to prevent the multiple threads from
stomping on each other's output.  You will likely deadlock somewhere along
the way, but that is fairly easy to fix with the correct tools.  Dolphin's
process browser is great, but for diagnosing deadlocks, I still like my less
powerful process viewer goodie because provides more immediate access to the
call stacks.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Bernardo Clavijo
> I think Chris is correct in his analysis of your problem.  You will need
> to mutex protect your (socket?) stream to prevent the multiple threads
> from
> stomping on each other's output.  You will likely deadlock somewhere along
> the way, but that is fairly easy to fix with the correct tools.  Dolphin's
> process browser is great, but for diagnosing deadlocks, I still like my
> less powerful process viewer goodie because provides more immediate access
> to the call stacks.

Mmmm, my damn memory!!!!
I didn't mention, but we already try mutexes... with no luck :(

But.... well, maybe i'll give it a try again... don't know... i don't think
that will solve the problem :(

Saludos

    BJ
Bernardo CLavijo


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Chris Uppal-3
Bernardo Clavijo wrote:

> Mmmm, my damn memory!!!!
> I didn't mention, but we already try mutexes... with no luck :(

Can you give us more detail about how you are using the STB streams ?  It
sounds almost as if you are using the same STB stream from more than one
Smalltalk Process, but I find it hard to imagine why you'd do that.

Also, are the connections between your clients and server long lived, or do you
make a connection, send some data (and perhaps get a reply) and then close the
connection ?  If you do have long-lived connections, do you have an STB stream
permanently associated with each connection, or do you use a new STB in/out
stream each time you convert an object to binary before sending it ?

Just questions, I'm afraid, but I suspect that the your problem is something to
do with the way you are using STB and/or the sockets.

    -- chris


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Bernardo Clavijo
Chris Uppal wrote:

> Bernardo Clavijo wrote:
>
>> Mmmm, my damn memory!!!!
>> I didn't mention, but we already try mutexes... with no luck :(
>
> Can you give us more detail about how you are using the STB streams ?  It
> sounds almost as if you are using the same STB stream from more than one
> Smalltalk Process, but I find it hard to imagine why you'd do that.
>
> Also, are the connections between your clients and server long lived, or
> do you make a connection, send some data (and perhaps get a reply) and
> then close the
> connection ?  If you do have long-lived connections, do you have an STB
> stream permanently associated with each connection, or do you use a new
> STB in/out stream each time you convert an object to binary before sending
> it ?

I've managed to my program "strange feature" whitout sockets, so I think it
ain't a socket problem. I've got an script at work, so I'll send it as soon
as I can.
 
> Just questions, I'm afraid, but I suspect that the your problem is
> something to do with the way you are using STB and/or the sockets.

Well, I'll post the script that has my "easiest & shortest way (tm)" to
reproduce the situation, at least in my environment.

Thanks for your patience and great help.

Saludos

    BJ
Bernardo Clavijo


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Bill Schwab-2
In reply to this post by Chris Uppal-3
Chris,

> Can you give us more detail about how you are using the STB streams ?  It
> sounds almost as if you are using the same STB stream from more than one
> Smalltalk Process, but I find it hard to imagine why you'd do that.

It's easy to end up with such constructs, but of course, the access to the
filer must be properly synchronized or it will turn ugly.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Bernardo Clavijo
Bill Schwab wrote:

> Chris,
>
>> Can you give us more detail about how you are using the STB streams ?  It
>> sounds almost as if you are using the same STB stream from more than one
>> Smalltalk Process, but I find it hard to imagine why you'd do that.
>
> It's easy to end up with such constructs, but of course, the access to the
> filer must be properly synchronized or it will turn ugly.


Well, my script didn't give me any errors once I put a mutex in, but the
application still fails.
Can you explain what you call " properly synchronized? maybe it could light
me a dark point in my "working method".

Saludos

    BJ
Bernardo Clavijo


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Chris Uppal-3
In reply to this post by Bill Schwab-2
Bill,

> > It sounds almost as if you are using the same STB stream from more than
> > one Smalltalk Process, but I find it hard to imagine why you'd do that.
>
> It's easy to end up with such constructs,

You must have a better imagination than me ;-)  I've been trying to think of a
natural scenario that would have different threads writing to the same STB
stream, and have only found one so far (and it's pretty damned esoteric).

> but of course, the access to the
> filer must be properly synchronized or it will turn ugly.

And the objects being written, if they could potentially be mutated at the same
time.  Though I'm not sure that that would result in *structurally* invalid STB
data.

    -- chris


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Bill Schwab-2
Chris,

> > > It sounds almost as if you are using the same STB stream from more
than
> > > one Smalltalk Process, but I find it hard to imagine why you'd do
that.
> >
> > It's easy to end up with such constructs,
>
> You must have a better imagination than me ;-)  I've been trying to think
of a
> natural scenario that would have different threads writing to the same STB
> stream, and have only found one so far (and it's pretty damned esoteric).

Necessity is the mother of invention: read data from several serial ports
and a few sockets, and do a background STB save every five minutes.


> > but of course, the access to the
> > filer must be properly synchronized or it will turn ugly.
>
> And the objects being written, if they could potentially be mutated at the
same
> time.  Though I'm not sure that that would result in *structurally*
invalid STB
> data.

True, but that's probably better left to the objects being serialized.  I
tend to make copies under protection of one mutex and then pass the "frozen"
result to lower levels that synchronize to solve their own problems, but can
safely consider the objects passing by to immutable.  The copies are also
useful in printing, because there is no concern about interfering (or
deadlocking) with the live document.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Bill Schwab-2
In reply to this post by Bernardo Clavijo
BJ,

> Well, my script didn't give me any errors once I put a mutex in, but the
> application still fails.
> Can you explain what you call " properly synchronized? maybe it could
light
> me a dark point in my "working method".

Most problems that can be solved with threads can also be solved in other
ways.  Why use them?

Threads are perhaps their most crucial when calling something that might not
return.  Given the choice between hanging the entire program or just one
thread, take the latter option.

Beyond that, IMHO, threads are often easier to design, write and maintain.
That's especially true in Smalltalk, in large part thanks to blocks.  Each
thread has one job, and the code reads that way, with occaisional effort to
cooperate with other threads.

Take a look at

   http://www.object-arts.co.uk/wiki/html/Dolphin/MultipleThreads.htm

The Wiki appears not to be accepting updates, but it will at least give you
a starting point for questions.

Re "proper" synchronization, you are correct to find that vague.  The bad
news is that _you_ have to define "proper" for your design problem.
Clearly, you want it to run (not deadlock), and you want sufficient
synchronization to avoid corrupt data (which sounds like your problem).  A
good starting point would be to give each owner of an STB filer its own
mutex.  You then will need to create/destroy "connections", which might
require another mutex to protect the connections themselves.  You might find
that one of Dolphin's shared collections would suffice for the latter.

As Chris noted, you might require some form of thread safety for you domain
classes, but that is probably better addressed separately from the STB
streams.

While it will do no good (and probably quite a lot of harm) to wrap every
line of code in a critical section, you would probably be wise to SLIGHTLY
over-protect.  That way, your sign that something is wrong will be a
deadlock rather than nonsense data or a crash.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: STB Bug???

Chris Uppal-3
In reply to this post by Bernardo Clavijo
Bernardo Clavijo wrote:

> Can you explain what you call " properly synchronized? maybe it could
> light me a dark point in my "working method".

Nobody had answered this request directly.  I can't speak for anyone else, but
for me it's because I still don't know what you are trying to do.  Which makes
it difficult to offer a suggestion about how to do it correctly.

    -- chris