Squeaksource (in)stability

> Hi -
>
> Our local Squeaksource installation has recently taken a turn to the
> worse, probably because we hit some magic number of packages or versions
> or so. While in the past we needed to restart SS about 2-3 times a week,
> it has recently gone up to needing a restart twice a day (we get emails
> whenever that happens so I'm not just imagining this).
>
> Help! This is completely unusable. If I hadn't added my little patch
> that recovers the versions involved in the lock up there would be open
> mutiny here by now. How can you run a Squeaksource installation with a
> *bit* of stability?

> Hi -
>
> Our local Squeaksource installation has recently taken a turn to the
> worse, probably because we hit some magic number of packages or versions
> or so. While in the past we needed to restart SS about 2-3 times a week,
> it has recently gone up to needing a restart twice a day (we get emails
> whenever that happens so I'm not just imagining this).
>
> Help! This is completely unusable. If I hadn't added my little patch
> that recovers the versions involved in the lock up there would be open
> mutiny here by now. How can you run a Squeaksource installation with a
> *bit* of stability?

>
> Ken
>
> On Tue, 2007-10-02 at 11:10 -0700, Andreas Raab wrote:
>> Hi -
>>
>> Our local Squeaksource installation has recently taken a turn to the
>> worse, probably because we hit some magic number of packages or versions
>> or so. While in the past we needed to restart SS about 2-3 times a week,
>> it has recently gone up to needing a restart twice a day (we get emails
>> whenever that happens so I'm not just imagining this).
>>
>> Help! This is completely unusable. If I hadn't added my little patch
>> that recovers the versions involved in the lock up there would be open
>> mutiny here by now. How can you run a Squeaksource installation with a
>> *bit* of stability?
>>
>> Thanks,
>>    - Andreas
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>>

> 2007/10/2, Andreas Raab <[hidden email]>:
>> Help! This is completely unusable. If I hadn't added my little patch
>> that recovers the versions involved in the lock up there would be open
>> mutiny here by now. How can you run a Squeaksource installation with a
>> *bit* of stability?
>
> Hey, it's open source man! Either pay me a million dollars of fix it yourself.
>
> If you want something reliable use professional software like Apache +
> WebDAV or Gemstone.
>
> *just kidding*

> Philippe Marschall wrote:
> > 2007/10/2, Andreas Raab <[hidden email]>:
> >> Help! This is completely unusable. If I hadn't added my little patch
> >> that recovers the versions involved in the lock up there would be open
> >> mutiny here by now. How can you run a Squeaksource installation with a
> >> *bit* of stability?
> >
> > Hey, it's open source man! Either pay me a million dollars of fix it yourself.
> >
> > If you want something reliable use professional software like Apache +
> > WebDAV or Gemstone.
> >
> > *just kidding*
>
> Well, I'm not. I'd be perfectly happy to pay my share if someone were to
> offer a solution that actually works. Not quite a million dollars but a
> couple of hundred for sure. After all it's our source repository we are
> talking about.

> > I don't want to point fingers but all the information we have so far
> > hints that the bug is not in SqueakSource but in Squeak itself or the
> > VM (Sockets, Delays, Semaphores, ....). If you have some evidence that
> > points to SqueakSource I'll be happy to look into it.
>
> Thanks I'll try this. I hadn't applied these changes to that particular
> image (mostly because I try to touch a working installation as little as
> possible) but to the best of my understanding of the situation the
> problem always occurs when it tries to save a snapshot of the data.
> Which is really not that surprising if you consider what I wrote a while
> back about saves being locked against each other but not serialized with
> incoming modification requests.

> > * me not understanding how Seaside / Squeaksource protect against
> > concurrent modifications internally
>
> Seaside serializes requests for a single session. It does not
> serialize requests among multiple sessions.
>
> > * the singular use of a single critical section for saves only (!)
>
> I don't remember exactly what SqueakSource is doing. I don't have an
> image accessible right here, but maybe Philippe can comment on that?

> Andreas Raab wrote:
>> Hi -
>> Our local Squeaksource installation has recently taken a turn to  
>> the worse, probably because we hit some magic number of packages  
>> or versions or so. While in the past we needed to restart SS about  
>> 2-3 times a week, it has recently gone up to needing a restart  
>> twice a day (we get emails whenever that happens so I'm not just  
>> imagining this).
>> Help! This is completely unusable. If I hadn't added my little  
>> patch that recovers the versions involved in the lock up there  
>> would be open mutiny here by now. How can you run a Squeaksource  
>> installation with a *bit* of stability?
>
> Welcome to the club.
> I used the latest squeaksource version for sophieproject (lots and  
> lots of packages and versions) that also runs on the esug server (I  
> think) and added all your patches.
> It seems to be really stable, so your patches seem to work their  
> magic :-)
>
> The only thing that makes problems is that it screws up the web  
> connection (see previous postings).
>
> And we use image storage, which sucks big time as we already ran  
> into the problem that on a rare bug the image got saved with the  
> debugger open and caused problems. We don't have the resources  
> right now to try out the Magma backend Bert did a while ago.
>
>
> Michael
>
>
>

> Philippe Marschall wrote:
>
>> I don't want to point fingers but all the information we have so far
>> hints that the bug is not in SqueakSource but in Squeak itself or the
>> VM (Sockets, Delays, Semaphores, ....). If you have some evidence  
>> that
>> points to SqueakSource I'll be happy to look into it.
>
> Well, part of the problem is that SqueakSource doesn't have a  
> reliable storage solution. The serialization doesn't scale and
> <flame bait>
> image storage is one of the dumbest things that I ever came across.
> </flame bait>
>
> Michael
>
>

> 2007/10/2, Andreas Raab <[hidden email]>:
> > Philippe Marschall wrote:
> > > 2007/10/2, Andreas Raab <[hidden email]>:
> > >> Help! This is completely unusable. If I hadn't added my little patch
> > >> that recovers the versions involved in the lock up there would be open
> > >> mutiny here by now. How can you run a Squeaksource installation with a
> > >> *bit* of stability?
> > >
> > > Hey, it's open source man! Either pay me a million dollars of fix it yourself.
> > >
> > > If you want something reliable use professional software like Apache +
> > > WebDAV or Gemstone.
> > >
> > > *just kidding*
> >
> > Well, I'm not. I'd be perfectly happy to pay my share if someone were to
> > offer a solution that actually works. Not quite a million dollars but a
> > couple of hundred for sure. After all it's our source repository we are
> > talking about.
>
> If you don't need the whole web frontend but only a Monticello
> repository nothing beats Apache.
>
> If you want the whole ACID stuff:
> http://seaside.gemstone.com/ss
> Runs on Gemstone/S 64. All commits are transactions. If something goes
> wrong, in the worst case you loose your last request. If your last
> request was a commit you'll get a Monticello error and have to
> republish. You are limited to a size of 4 GB though, for anything
> beyond that the free version doesn't work anymore. Additionally
> SqueakSource won't run with the latest Seaside version (there is a new
> one in the works though) and there is no upgrade path from your
> existing installation.
>
> Somewhere between this and your current solution is the Magma backend Bert did:
> http://source.impara.de/ss
> I don't know of any users of this.
>
> > > I don't want to point fingers but all the information we have so far
> > > hints that the bug is not in SqueakSource but in Squeak itself or the
> > > VM (Sockets, Delays, Semaphores, ....). If you have some evidence that
> > > points to SqueakSource I'll be happy to look into it.
> >
> > Thanks I'll try this. I hadn't applied these changes to that particular
> > image (mostly because I try to touch a working installation as little as
> > possible) but to the best of my understanding of the situation the
> > problem always occurs when it tries to save a snapshot of the data.
> > Which is really not that surprising if you consider what I wrote a while
> > back about saves being locked against each other but not serialized with
> > incoming modification requests.
>
> An option to try wout be to fork() the image with OSProcess before
> saving the model. This ensures there are no concurrent modifications.
> We don't do this because it doesn't work on the Mac VM.
>
> Cheers
> Philippe
>
>

> I haven't looked at what Bert did, but it's also interesting to note
> that, other than using Magma as a SS backend, it was designed to BE a
> Monticello repository.  In this way, just as objects are state plus
> behavior, a Magma repository is (or, should be) an object-model plus
> the code-base to operate those objects.
>
> After installing Magma, additional menu options appear on Monticello's
> "+Repository" menu for adding a Magma repository.  I've tested it but
> not heavily because the Right Thing To Do (tm) is store the MC object
> model directly, not copies of entire serialized packages of every
> version.  If you change three methods for a particular version, then
> Monticello should only create three new McMethod objects (whatever) in
> the new snapshot, all others should refer to the same identical ones
> of the previous verion.  This approach is a lot less wasteful than
> saving full copies every time, and would permit snappy performance
> from Magma given the small commits.
>
> Magma does also let you file-out individual classes and change sets
> along-side the MC packages in the same repository with convenience
> methods for browsing them before you install..
>
>
> On 10/3/07, Philippe Marschall <[hidden email]> wrote:
> > 2007/10/2, Andreas Raab <[hidden email]>:
> > > Philippe Marschall wrote:
> > > > 2007/10/2, Andreas Raab <[hidden email]>:
> > > >> Help! This is completely unusable. If I hadn't added my little patch
> > > >> that recovers the versions involved in the lock up there would be open
> > > >> mutiny here by now. How can you run a Squeaksource installation with a
> > > >> *bit* of stability?
> > > >
> > > > Hey, it's open source man! Either pay me a million dollars of fix it yourself.
> > > >
> > > > If you want something reliable use professional software like Apache +
> > > > WebDAV or Gemstone.
> > > >
> > > > *just kidding*
> > >
> > > Well, I'm not. I'd be perfectly happy to pay my share if someone were to
> > > offer a solution that actually works. Not quite a million dollars but a
> > > couple of hundred for sure. After all it's our source repository we are
> > > talking about.
> >
> > If you don't need the whole web frontend but only a Monticello
> > repository nothing beats Apache.
> >
> > If you want the whole ACID stuff:
> > http://seaside.gemstone.com/ss
> > Runs on Gemstone/S 64. All commits are transactions. If something goes
> > wrong, in the worst case you loose your last request. If your last
> > request was a commit you'll get a Monticello error and have to
> > republish. You are limited to a size of 4 GB though, for anything
> > beyond that the free version doesn't work anymore. Additionally
> > SqueakSource won't run with the latest Seaside version (there is a new
> > one in the works though) and there is no upgrade path from your
> > existing installation.
> >
> > Somewhere between this and your current solution is the Magma backend Bert did:
> > http://source.impara.de/ss
> > I don't know of any users of this.
> >
> > > > I don't want to point fingers but all the information we have so far
> > > > hints that the bug is not in SqueakSource but in Squeak itself or the
> > > > VM (Sockets, Delays, Semaphores, ....). If you have some evidence that
> > > > points to SqueakSource I'll be happy to look into it.
> > >
> > > Thanks I'll try this. I hadn't applied these changes to that particular
> > > image (mostly because I try to touch a working installation as little as
> > > possible) but to the best of my understanding of the situation the
> > > problem always occurs when it tries to save a snapshot of the data.
> > > Which is really not that surprising if you consider what I wrote a while
> > > back about saves being locked against each other but not serialized with
> > > incoming modification requests.
> >
> > An option to try wout be to fork() the image with OSProcess before
> > saving the model. This ensures there are no concurrent modifications.
> > We don't do this because it doesn't work on the Mac VM.
> >
> > Cheers
> > Philippe
> >
> >
>
>