Are Collections threadsafe?

Hello list,

If I access global collections of objects within my Web Application, do I have to synchronize access by myself?

My concerns are whether Dictionaries and OrderedCollections are implemented threadsafe or not,
I found some documents that said they aren't but they date back a fair while, so I'd like to ask whether any of you has an answer for me..

Greetings,
Malte Zacharias
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Hi!

Malte Zacharias <[hidden email]> wrote:
> Hello list,
>
> If I access global collections of objects within my Web Application, do
> I have to synchronize access by myself?=20

Yes.
 
> My concerns are whether Dictionaries and OrderedCollections are
> implemented threadsafe or not,=20
> I found some documents that said they aren't but they date back a fair
> while, so I'd like to ask whether any of you has an answer for me..

They are not thread safe. An easy way to serialize accesses is using a
Monitor, see its class comment and references to the class for examples.

regards, Göran
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/25, Malte Zacharias <[hidden email]>:
>
>  Hello list,
>
>  If I access global collections of objects within my Web Application, do I
> have to synchronize access by myself?

Yes.

>  My concerns are whether Dictionaries and OrderedCollections are implemented
> threadsafe or not,

They're not thread safe .

Cheers
Philippe
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/25, [hidden email] <[hidden email]>: Monitor, Mutex or Semaphore. Which shall it be? And which works with
with patch set posted where in which Image with which VM? Because
stock Semaphores in stock Images with stock VMs are broken and Mutex
and Monitor build on Semaphore.

Cheers
Philippe

>  regards, Göran
>
> _______________________________________________
>  seaside mailing list
>  [hidden email]
>  http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Thanks for the prompt answer.

But in what way are Semaphores broken? Completely or are there certain cirumstances that need to be avoided?
I have to admit, I am quite surprised to hear that. Is the same true for seaside-mini.image and/or squeak web-dev image?

Should I have found that information online already without asking the list?

Again thanks and I'm curious to hear the answer(s)..


_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/25, Malte Zacharias <[hidden email]>:
>
>  Thanks for the prompt answer.
>
>  But in what way are Semaphores broken? Completely or are there certain
> cirumstances that need to be avoided?

Under some circumstances a critical section might not behave like a
critical section.

>  I have to admit, I am quite surprised to hear that. Is the same true for
> seaside-mini.image and/or squeak web-dev image?

Dunno. As I said, I don't know which patch set is supposed to work.

>  Should I have found that information online already without asking the
> list?

http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-July/118942.html
http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-July/119093.html

Read the whole threads including follow up posts. In most cases the
initial patch didn't really fix it. AFAIK there were also some other
follow up threads on Delays.

Cheers
Philippe
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

note that in the 3.9.1 imae some fixes of andreas on semaphores and  
delay + adrian fixes are harvested.

Stef

On Feb 25, 2008, at 8:31 PM, Philippe Marschall wrote:

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 25/02/2008, stephane ducasse <[hidden email]> wrote:
> note that in the 3.9.1 imae some fixes of andreas on semaphores and
>  delay + adrian fixes are harvested.
>
Including this one? http://bugs.squeak.org/view.php?id=6822


--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Hi again!

Malte Zacharias <[hidden email]> wrote:
> Thanks for the prompt answer.=20
>
> But in what way are Semaphores broken? Completely or are there certain
> cirumstances that need to be avoided?=20
> I have to admit, I am quite surprised to hear that. Is the same true for
> seaside-mini.image and/or squeak web-dev image?

Let me just say that Semaphore and Monitor have been around and used a
LOT - and I mean, LOT LOT LOT.

So saying that these things are "broken" is an overstatement. :) Sure,
we have discovered some issues - but we are talking things that have
gone undetected a loooong time.

IMHO - while you can learn something from digging into these discussions
and patches, you could also just move along and use a Monitor (or
Semaphore - but a Monitor is probably "smarter") and be done with it.

regards, Göran
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Thanks to all of you for the numerous replies, I have read a fair bit of
the referenced discussion and certainly hope it will be a non-issue for
me, I kind of expected the one major mechanism for process
synchronisation to "just work", just as the mayority of people I
suppose. Strange thing is, that immediately after starting to work with
Mutex and Delays my image got wrecked.. Wrecked as in UI unbearable
slow, mouse clicks only got noticed after lots of time, etc.. Mind you
the seaside application did not show those effects. and the Process
browser only showed that the UI process consumed an awful lot of time..
I hope not to experience the same with the actual image running the
server. (fortunately the Data is also stored externally now, thanks to
Ramon Leons description of a simple ReferenceStream based Persistence)
Sad that I can't be certain whether it was just a strange coincidence..

> So saying that these things are "broken" is an overstatement. :) Sure,
> we have discovered some issues - but we are talking things that have
> gone undetected a loooong time.
Well, concurrency tends to alway shoots you in the back when you expect
it the least.. And how many people would have done like I did; thinking
Image's broken, let's use a fresh one. Sad, loosing all the
customizations though..

> IMHO - while you can learn something from digging into these discussions
> and patches, you could also just move along and use a Monitor (or
> Semaphore - but a Monitor is probably "smarter") and be done with it.
Just FYI I am using Mutexes right now. I wasn't expecting lots of
problems anyway, we only have very few concurrent user (averaged to
below 1 I'd guess..) with the occasional spikes, so it's more a better
safe than sorry approach.

Big thanks,
Malte Zacharias
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/25, [hidden email] <[hidden email]>: Not undetected. They do show up in production. We had for example once
a problem with a db connection pool that was protected by a Semaphore.
Whether it worked or not depended on the priority of the thread
accessing it. But of course implying there is something wrong with the
VM or Image is considered heresy. If you have such issues, better just
fix them yourself. This is easy because "the source it there" (tm). It
was probably done intentionally so that you could expand your
Smalltalk knowledge.

Cheers
Philippe

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Hi!

Phillippe, could you possibly turn off base64 encoding or such?

"Philippe Marschall" <[hidden email]> wrote:
> MjAwOC8yLzI1LCBnb3JhbkBrcmFtcGUuc2UgPGdvcmFuQGtyYW1wZS5zZT46Cj4gSGkgYWdhaW4h

Eh... :) anyway...

I didn't imply the problems have been undetected - I just meant that
they have gone undetected for a looong time. And most users deploy stuff
with Squeak and do just fine. That is all I am saying.

Of course we should fix it. But we should also not scare people into
thinking that Semaphores are *totally* broken and that Squeak is total
crap when it comes to concurrency. :)

regards, Göran
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On Feb 25, 2008, at 9:46 PM, Philippe Marschall wrote:


Really the problem is actually capturing meaningful data and debugging  
semaphore and delay
issues is very difficult because the VM usually is catatonic at this  
point.  Or something is wrong
but you don't know what.    There are specialized tools to do VM  
message tracing or snapshots of
the squeak VM stacks outside of the image, using perhaps unfortunately  
special versions of the VM
which require folks to build their own.

This is not to say we ignore things.

I'll note in the late 90's for example Dan called me and said his  
squeak image hung after running for a
month. This was the first clue that macintosh operating systems had  
become stable enough for you to
actually run an entire month without rebooting. Issue was millisecond  
clock rollover cause event loop
processing to stop after 23+ days.

At camp smaltlalk 4 a bunch of us huddled about a table for an  
afternoon trying to figure out why
Delay occasional froze, although we had no data, in the code review we  
traced to an exposure between
a check for a condition and a message send that introduced an  
opportunity for a higher priority task to
attempt to setup a Delay when the Delay logic was performing a  
critical structure change.

Then someone say Oh they had really really long delays and that  
results in large integers going into
the delay logic, yet the delay logic is dependent on the millisecond  
clock which only thinks in a much smaller
number. I recall Tim spent days/weeks figuring out an optimal  
solution.  Noting of course tampering with
Delay on the fly, make a mistake and you get to restart your image, or  
toss it...

We had MC servers lock up and drive people crazy, even the Sophie team  
which I am on.  Someone else
finally figured out, no it wasn't a problem with unix sockets, it was  
a problem in the Morphic Event loop and how
it services morphs and how it wants to ensure a 60 fps rate by real  
time calculation  (which I added years ago).
That prevented the servicing of MC server work threads.

Lately Andreas because he is using Squeak for qwaq pushed out quite a  
number of fixed to Semaphores, are all
the issues fixed? Well you need to apply all the patches and see what  
happens.  So make a mantis report, we'd
like to see that thought: "Whether it worked or not depended on the  
priority of the thread
> accessing it."   I'm sure that would an Andreas think more about  
> what is going on.

What you see now is Squeak is being much more important to people and  
issues with stability cannot be
ignored or glossed over.

Shameless job plug:

        Tim and I of course for a fee are quite willing to assist people who  
think they have a VM issue, if your
        "downtime" is costing you $$$ or causing people to think "Who picked  
this crummy software?"
        Then by all means give us a call.



--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
=
=
=
========================================================================


_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

[hidden email] wrote:
> I didn't imply the problems have been undetected - I just meant that
> they have gone undetected for a looong time. And most users deploy stuff
> with Squeak and do just fine. That is all I am saying.

The real trouble is that some of the people (like Philippe) that are
closest to the point of the problem end up complaining in general
instead of gathering valuable data. Basically it's all flawed logic of
the form "clearly, our code can't possibly be wrong so the VM must be
broken and why don't you guys just get your act together and fix it".

What people *really* need to do in such a situation is to gather as much
data as possible. If you can still save the image, save it. If you can
still get a bunch of stack traces, get them. Attach gdb to the VM and do
a printAllStacks() - this is probably the most important information you
get in a situation like this (at Qwaq, we have hooked this up to a USR1
signal so that when we need to restart the servers we first get a full
stack trace and then restart the images just in case).

Once you have gathered all that information, post it to Squeak-dev.
There are actually people out there who care about it. They just don't
care very much about editorial comments of the form "TEH SQUEAK SUCKZ!".
If you want a solution, then provide the input that helps other people
resolving your problem. The short form of that equation is:
   complaints == no data
   no data == no solution
And if you keep this in mind (and your frustration to a minimum) you
will likely find that *with* data the probability of actually fixing
your problems goes up dramatically.

> Of course we should fix it. But we should also not scare people into
> thinking that Semaphores are *totally* broken and that Squeak is total
> crap when it comes to concurrency. :)

Absolutely. At Qwaq, we routinely run servers with hundreds of
concurrent connections piping through gigabytes of data per day.

Cheers,
   - Andreas
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Andreas is right.
When your project is based on some proprietary libraries with bugs,
which you can't fix because you don't have access to sources, all you
can do is to blame developers.
In contrast, Squeak is open-source.
With Squeak you have a chance to find bug, and fix it with or without
help of community.
So, when you found a bug, least you can do is report it and try to
fix, or provide enough information to others who can fix it.
And your actions will be considered as act of gratitude to people who
created Squeak and spent and spending own time to support it, without
being paid a penny by you.

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/27, Andreas Raab <[hidden email]>:
That's simple if your image blocks. If your image doesn't blocks but
the behavior you get looks as if your critical sections are not
critical what do you do then? Attach gdb at random points in time?
Would that really help?

Cheers
Philippe
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/27, Andreas Raab <[hidden email]>:
http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-December/123045.html
http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-December/123508.html
http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-December/123498.html

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Philippe Marschall wrote:
> That's simple if your image blocks. If your image doesn't blocks but
> the behavior you get looks as if your critical sections are not
> critical what do you do then? Attach gdb at random points in time?
> Would that really help?

Taking a couple of samples that you can analyze off-line is definitely
worthwhile (I've done it actually today to get a feel for a particular
slowdown in our app). The other thing you can do is to add sanity
checks. For example: It can be helpful to make your own kind of mutex so
that you can see whether the signal count is ever more than one (this is
one of the things that helped me tracking some of the Delay issues).

Generally speaking: Define the invariants that you believe to be true
(like: Semaphore must have either zero or one signals) and attribute
your code to verify these assumptions. If you see them violated try to
find the first place where things go wrong and then debug it from there.

Cheers,
   - Andreas
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Philippe Marschall wrote:
I'm not sure what point you're trying to make here but this is actually
a great example for what I mean. Lukas gave meaningful information in
this exchange and as a result we developed a couple of theories and
disproved them based on available evidence:

http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-December/123047.html
http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-December/123499.html
http://lists.squeakfoundation.org/pipermail/squeak-dev/2007-December/123511.html

I'm not claiming that you will *always* get to a result that satisfies
your needs but the above is a great example for the kind of discourse
that you can engage in when you provide some actual data. Also, I think
we both learned something in this exchange which is where the real value
lies - if I ever happen to run into a similar situation the data that
Lukas gave will definitely be helpful in tracking this down. And don't
worry, we'll get to it but these are hard problems and I think we simply
haven't got enough data to determine the root cause of the problem. So
if you have more evidence, please share it and please don't be shy to
post repeatedly because that's what is needed to fix it.

Cheers,
   - Andreas

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

2008/2/27, Andreas Raab <[hidden email]>:
That we are not uncooperative, don't give out any information and
simply post "SQUEAK IS TEH SUX"

Cheers
Philippe
_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside