ImageSegments as persistence

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 2/19/07, Sebastian Sastre <[hidden email]> wrote:

>     Avi did you have any progress on that primitive that you want for array
> allocation to disk?

No.  I'm still interested in it if anyone has time.

>     I understood is file based and not atomical (transactional) and that it
> is like a piece of the image in another file.

It's certainly atomic in that there is a single primitive that builds
the segment.  But you'll need to do a lot of work yourself to get the
ACID properties you might want.

>     How much can one rely on them? any experience in production
> applications?

Once a segment is successfully built, I've never had any trouble
loading one into a new image (and Dabble DB has produced god knows how
many hundreds of thousands of image segments by this point).  The
problems are in building them - you have to be *very* careful not to
have any outpointers, and the allocation of the buffer to build the
segment can cause low-memory conditions more often than I'd like
(hence the desire for the disk-based version).

>     I think, that if reliable, it is an interesting option for seaside
> applications that do not require transactions.

If you understand and can carefully manage them, they certainly are.
But they're hardly an "out of the box" persistence solution.

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

Agree. Do you save them in a #critical: ?

Very nice. Let me see if I understood you right.. by outpointers you mean
that the image segments must be "closed", I mean objects of one segment
should not have references to objets outside that segment right?

I think I saw the OS automatically giving more memory to a squeak in low
memory condition in windows but not in linux. How do you manage a low memory
condition?

>
> >     I think, that if reliable, it is an interesting option
> for seaside
> > applications that do not require transactions.
>
> If you understand and can carefully manage them, they certainly are.
> But they're hardly an "out of the box" persistence solution.

Right. When one use this to persist, in your opinion, how one should decide
to make a "save" on them? By an event? By time? How ofen?

Thanks,

Sebastian

>
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 2/20/07, Sebastian Sastre <[hidden email]> wrote:

> > It's certainly atomic in that there is a single primitive
> > that builds the segment.  But you'll need to do a lot of work
> > yourself to get the ACID properties you might want.
>
> Agree. Do you save them in a #critical: ?

Yes.

> Very nice. Let me see if I understood you right.. by outpointers you mean
> that the image segments must be "closed", I mean objects of one segment
> should not have references to objets outside that segment right?

Sort of.  Your image segment has a root - for me, this is usually
something like a global MyDatabase object, from which all of your data
is accessible.  The trick is that before you put out a segment, you
have to get rid of all references to any of your data that does not
stem from that root.  This includes your Seaside sessions (WARegistry
clearAllHandlers), any open debuggers/inspectors/workspaces that might
hold onto them, maybe DynamicVariables, etc.  You'll know when you've
got an outpointer because it will take an order of magnitude longer to
write out the image segment.

> I think I saw the OS automatically giving more memory to a squeak in low
> memory condition in windows but not in linux. How do you manage a low memory
> condition?

Badly. :)

> Right. When one use this to persist, in your opinion, how one should decide
> to make a "save" on them? By an event? By time? How ofen?

That's a trade off you'll have to figure out based on how your app
gets used.  You will probably want some mix of continuous transaction
logging for worst-case recovery (that might even be just logging the
HTTP requests that come in, since they'll have all of the data you
need), frequent full image snapshotting (which is faster than an image
segment and doesn't require you to clear outpointers, but which takes
up a lot of extra room, is somewhat less reliable [images can get in
wonky states] and harder to upgrade to a new codebase), and periodic
image segment building.  But what mix of those is right for you, and
how to trigger them, is hard for anyone else to say.

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

On 2/20/07, Romain Robbes <[hidden email]> wrote:

> Is there a size limit for ImageSegments, or a size beyond which they are not
> practical anymore?
> This is not related to Seaside actually, but I was wondering if saving an
> ImageSegment of say 100MB would work.

100MB is fine.  I've found that once you get much beyond that, you're
likely to run into low memory conditions when building the segment.

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside