Smalltalk › Squeak › Squeak - Dev

[squeak-dev] Upgrading running servers?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

17 messages Options

Andreas.Raab

[squeak-dev] Upgrading running servers?

Hi -

[Apologies for the cross-post but there may be answers to this question
coming from people who are subscribed to Seaside but not Squeak-dev]

What are the best practices for upgrading portions of your running
server? I'm not talking about replacing the main business logic of your
server application but rather relatively small things like an improved
log module, or an administrative report function etc. Basically parts
that you can assume aren't used while you upgrade them.

I was quite surprised that I wasn't able to find any recommendations on
how to do that. And I can't imagine that this problem hasn't come up in
Seaside or other Squeak or Smalltalk server deployments. So what do
people do in such a situation? Fire up VNC and load the latest MC
package? Is that feasible when you have hundreds of users pounding the
server? Any other (semi- or fully-) automated variants? Alternative
recommendations?

Thanks for any insights you can share.

Cheers,
- Andreas

Ken Causey-3

Re: [squeak-dev] Upgrading running servers?

Well, I can only say that I've updated (without restarting) a handful of
images that were running either Seaside or something else using Kom and
have not had any troubles. In most cases these were images that were
probably not getting but a few hits a minute, but I believe in at least
one case we made changes to Swiki (yes, wiki.squeak.org) and had no
trouble. Those were very minor changes though if I remember correctly
(just a method or two).

Ken

On Wed, 2008-04-02 at 09:28 -0700, Andreas Raab wrote:

> Hi -
>
> [Apologies for the cross-post but there may be answers to this question
> coming from people who are subscribed to Seaside but not Squeak-dev]
>
> What are the best practices for upgrading portions of your running
> server? I'm not talking about replacing the main business logic of your
> server application but rather relatively small things like an improved
> log module, or an administrative report function etc. Basically parts
> that you can assume aren't used while you upgrade them.
>
> I was quite surprised that I wasn't able to find any recommendations on
> how to do that. And I can't imagine that this problem hasn't come up in
> Seaside or other Squeak or Smalltalk server deployments. So what do
> people do in such a situation? Fire up VNC and load the latest MC
> package? Is that feasible when you have hundreds of users pounding the
> server? Any other (semi- or fully-) automated variants? Alternative
> recommendations?
>
> Thanks for any insights you can share.
>
> Cheers,
> - Andreas
>
>

signature.asc (196 bytes) Download Attachment

Ramon Leon-5

RE: [squeak-dev] Upgrading running servers?

In reply to this post by Andreas.Raab

> Hi -
>
> [Apologies for the cross-post but there may be answers to
> this question coming from people who are subscribed to
> Seaside but not Squeak-dev]
>
> What are the best practices for upgrading portions of your
> running server? I'm not talking about replacing the main
> business logic of your server application but rather
> relatively small things like an improved log module, or an
> administrative report function etc. Basically parts that you
> can assume aren't used while you upgrade them.
>
> I was quite surprised that I wasn't able to find any
> recommendations on how to do that. And I can't imagine that
> this problem hasn't come up in Seaside or other Squeak or
> Smalltalk server deployments. So what do people do in such a
> situation? Fire up VNC and load the latest MC package? Is
> that feasible when you have hundreds of users pounding the
> server? Any other (semi- or fully-) automated variants?
> Alternative recommendations?
>
> Thanks for any insights you can share.
>
> Cheers,
> - Andreas

Not that it's a perfect solution, but here's what I've been doing, it's been
working well and lets me just ftp up the new packages.

http://onsmalltalk.com/programming/smalltalk/upgrading-a-running-squeak-imag
e/

Ramon Leon
http://onsmalltalk.com

Rajeev Lochan

Re: [Seaside] Re: [squeak-dev] Upgrading running servers?

In reply to this post by Ken Causey-3

Hi,
Check out Ramon's Blog post, which can guide you, though I havent used them.

http://onsmalltalk.com/programming/smalltalk/upgrading-a-running-squeak-image/

http://onsmalltalk.com/programming/smalltalk/simple-file-based-application-configuration/

HTH,
Rajeev

On Wed, Apr 2, 2008 at 10:29 PM, Ken Causey <[hidden email]> wrote:

Well, I can only say that I've updated (without restarting) a handful of
images that were running either Seaside or something else using Kom and
have not had any troubles. In most cases these were images that were
probably not getting but a few hits a minute, but I believe in at least
one case we made changes to Swiki (yes, wiki.squeak.org) and had no
trouble. Those were very minor changes though if I remember correctly
(just a method or two).

Ken

On Wed, 2008-04-02 at 09:28 -0700, Andreas Raab wrote:
> Hi -
>
> [Apologies for the cross-post but there may be answers to this question
> coming from people who are subscribed to Seaside but not Squeak-dev]
>
> What are the best practices for upgrading portions of your running
> server? I'm not talking about replacing the main business logic of your
> server application but rather relatively small things like an improved
> log module, or an administrative report function etc. Basically parts
> that you can assume aren't used while you upgrade them.
>
> I was quite surprised that I wasn't able to find any recommendations on
> how to do that. And I can't imagine that this problem hasn't come up in
> Seaside or other Squeak or Smalltalk server deployments. So what do
> people do in such a situation? Fire up VNC and load the latest MC
> package? Is that feasible when you have hundreds of users pounding the
> server? Any other (semi- or fully-) automated variants? Alternative
> recommendations?
>
> Thanks for any insights you can share.
>
> Cheers,
> - Andreas
>
>

_______________________________________________
seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

--
Rajeev Lochan

Co-founder, AR-CAD.com

http://www.ar-cad.com
+91 9243468076 (Bangalore)
080 65355873

Andreas.Raab

Re: [Seaside] Re: [squeak-dev] Upgrading running servers?

Excellent links, thanks a lot (and to Ramon for writing the blog).
Interestingly, the issue of configuration management was one of the ones
that I've been thinking about recently too ;-)

Cheers,
- Andreas

Rajeev Lochan wrote:

> Hi,
> Check out Ramon's Blog post, which can guide you, though I havent used them.
>
> http://onsmalltalk.com/programming/smalltalk/upgrading-a-running-squeak-image/
>
> http://onsmalltalk.com/programming/smalltalk/simple-file-based-application-configuration/
>
> HTH,
> Rajeev
>
> On Wed, Apr 2, 2008 at 10:29 PM, Ken Causey <[hidden email]
> <mailto:[hidden email]>> wrote:
>
> Well, I can only say that I've updated (without restarting) a handful of
> images that were running either Seaside or something else using Kom and
> have not had any troubles. In most cases these were images that were
> probably not getting but a few hits a minute, but I believe in at least
> one case we made changes to Swiki (yes, wiki.squeak.org
> <http://wiki.squeak.org>) and had no
> trouble. Those were very minor changes though if I remember correctly
> (just a method or two).
>
> Ken
>
> On Wed, 2008-04-02 at 09:28 -0700, Andreas Raab wrote:
> > Hi -
> >
> > [Apologies for the cross-post but there may be answers to this
> question
> > coming from people who are subscribed to Seaside but not Squeak-dev]
> >
> > What are the best practices for upgrading portions of your running
> > server? I'm not talking about replacing the main business logic
> of your
> > server application but rather relatively small things like an
> improved
> > log module, or an administrative report function etc. Basically parts
> > that you can assume aren't used while you upgrade them.
> >
> > I was quite surprised that I wasn't able to find any
> recommendations on
> > how to do that. And I can't imagine that this problem hasn't come
> up in
> > Seaside or other Squeak or Smalltalk server deployments. So what do
> > people do in such a situation? Fire up VNC and load the latest MC
> > package? Is that feasible when you have hundreds of users
> pounding the
> > server? Any other (semi- or fully-) automated variants? Alternative
> > recommendations?
> >
> > Thanks for any insights you can share.
> >
> > Cheers,
> > - Andreas
> >
> >
>
> _______________________________________________
> seaside mailing list
> [hidden email]
> <mailto:[hidden email]>
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
>
>
> --
> Rajeev Lochan
>
> Co-founder, AR-CAD.com
>
> http://www.ar-cad.com
> +91 9243468076 (Bangalore)
> 080 65355873

Stephen Pair

Re: [Seaside] Re: [squeak-dev] Upgrading running servers?

If I understand what DabbleDB does, they essentially give each user their own squeak process to run in. When a session is first established, they start up a new image. Upgrading would be a matter of dropping a new image somewhere and as old sessions expire and new ones created, users are migrated over to the new code. With hydraVM, a scheme like that might be doable with just a small bit of session management in the main image.

(with swiki.net, I used to VNC in (I ran squeak headful in a regular VNC session...at the time, the squeak VNC server didn't exist), fix bugs or load code on the fly, but I was always a bit nervous about a slip of the mouse or a mistake taking the whole thing down...I would make changes in a way that they were captured and repeatable (using change sets (this was pre-MC)) and then re-apply the changes to a clean image on disk...I didn't like snapshotting with all the session related state bloating the image)

- Stephen

On Wed, Apr 2, 2008 at 9:23 PM, Andreas Raab <[hidden email]> wrote:

Excellent links, thanks a lot (and to Ramon for writing the blog). Interestingly, the issue of configuration management was one of the ones that I've been thinking about recently too ;-)

Cheers,
- Andreas

Rajeev Lochan wrote:

Hi,
Check out Ramon's Blog post, which can guide you, though I havent used them.

http://onsmalltalk.com/programming/smalltalk/upgrading-a-running-squeak-image/

http://onsmalltalk.com/programming/smalltalk/simple-file-based-application-configuration/

HTH,
Rajeev

On Wed, Apr 2, 2008 at 10:29 PM, Ken Causey <[hidden email] <mailto:[hidden email]>> wrote:

Well, I can only say that I've updated (without restarting) a handful of
images that were running either Seaside or something else using Kom and
have not had any troubles. In most cases these were images that were
probably not getting but a few hits a minute, but I believe in at least
one case we made changes to Swiki (yes, wiki.squeak.org
<http://wiki.squeak.org>) and had no

trouble. Those were very minor changes though if I remember correctly
(just a method or two).

Ken

On Wed, 2008-04-02 at 09:28 -0700, Andreas Raab wrote:
> Hi -
>
> [Apologies for the cross-post but there may be answers to this
question
> coming from people who are subscribed to Seaside but not Squeak-dev]
>
> What are the best practices for upgrading portions of your running
> server? I'm not talking about replacing the main business logic
of your
> server application but rather relatively small things like an
improved
> log module, or an administrative report function etc. Basically parts
> that you can assume aren't used while you upgrade them.
>
> I was quite surprised that I wasn't able to find any
recommendations on
> how to do that. And I can't imagine that this problem hasn't come
up in
> Seaside or other Squeak or Smalltalk server deployments. So what do
> people do in such a situation? Fire up VNC and load the latest MC
> package? Is that feasible when you have hundreds of users
pounding the
> server? Any other (semi- or fully-) automated variants? Alternative
> recommendations?
>
> Thanks for any insights you can share.
>
> Cheers,
> - Andreas
>
>

_______________________________________________
seaside mailing list
[hidden email]
<mailto:[hidden email]>

http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

--
Rajeev Lochan

Co-founder, AR-CAD.com

http://www.ar-cad.com
+91 9243468076 (Bangalore)
080 65355873

Nevin Pratt

Re: [Seaside] Re: [squeak-dev] Upgrading running servers?

> ...I didn't like snapshotting with all the session related state
> bloating the image)
>
> - Stephen

Other than bloating the image to a larger size, can anyone see any
dangers with snapshotting a live image like that?

Nevin

Andreas.Raab

Re: [squeak-dev] Upgrading running servers?

In reply to this post by Stephen Pair

Avi Bryant wrote:

> On Thu, Apr 3, 2008 at 12:00 PM, Stephen Pair <[hidden email]> wrote:
>> If I understand what DabbleDB does, they essentially give each user their
>> own squeak process to run in. When a session is first established, they
>> start up a new image. Upgrading would be a matter of dropping a new image
>> somewhere and as old sessions expire and new ones created, users are
>> migrated over to the new code. With hydraVM, a scheme like that might be
>> doable with just a small bit of session management in the main image.
>
> That's more or less right, yes. I've used variations on that strategy
> before - in general, having multiple images going where old sessions
> go to the old image but new sessions get routed to the new image makes
> for very smooth upgrades.

Yes, that's what we're doing too. When we need to upgrade a server we
simply don't let any new sessions go onto it and once everyone is off
the server is up for grabs. Unfortunately, some of the sessions last
quite a while (sometimes for several days because our notion of
"session" spans multiple users which need to overlap only for a short
amount of time) which I guess is good business-wise and we're not
running a lot of servers (which I guess is good too, since we scale) but
it makes for some rather long delays which could be mitigated by a "hot"
(or maybe just "warm") upgrade.

Cheers,
- Andreas

Avi Bryant-2

Re: [squeak-dev] Upgrading running servers?

On Thu, Apr 3, 2008 at 1:56 PM, Andreas Raab <[hidden email]> wrote:

> Yes, that's what we're doing too. When we need to upgrade a server we
> simply don't let any new sessions go onto it and once everyone is off the
> server is up for grabs.

Ah - so I think you understood this, but just for clarity: we run a
second image (or set of images) on the same server on a different port
and direct new sessions to it, rather than blocking new sessions from
that server entirely.

Avi

Andreas.Raab

[squeak-dev] Re: Upgrading running servers?

Avi Bryant wrote:

> On Thu, Apr 3, 2008 at 1:56 PM, Andreas Raab <[hidden email]> wrote:
>
>> Yes, that's what we're doing too. When we need to upgrade a server we
>> simply don't let any new sessions go onto it and once everyone is off the
>> server is up for grabs.
>
> Ah - so I think you understood this, but just for clarity: we run a
> second image (or set of images) on the same server on a different port
> and direct new sessions to it, rather than blocking new sessions from
> that server entirely.

We currently don't do this mostly because I'm unaware of a low-latency
proxy/forwarding solution that we could use (obviously we're more
concerned about latency than most web-based services). What do you use
to redirect the incoming connections?

Cheers,
- Andreas

Avi Bryant-2

Re: [squeak-dev] Re: Upgrading running servers?

On Thu, Apr 3, 2008 at 7:25 PM, Andreas Raab <[hidden email]> wrote:

> We currently don't do this mostly because I'm unaware of a low-latency
> proxy/forwarding solution that we could use (obviously we're more concerned
> about latency than most web-based services). What do you use to redirect the
> incoming connections?

We use Apache with mod_proxy. This is fine for HTTP (the extra
latency literally gets lost in the noise), but a chattier protocol
would be different, of course. I wonder if someone who knew their way
around ipfw could rig something up...

Avi

Bert Freudenberg

Re: [squeak-dev] Re: Upgrading running servers?

On 04.04.2008, at 08:42, Avi Bryant wrote:

> On Thu, Apr 3, 2008 at 7:25 PM, Andreas Raab <[hidden email]>
> wrote:
>
>> We currently don't do this mostly because I'm unaware of a low-
>> latency
>> proxy/forwarding solution that we could use (obviously we're more
>> concerned
>> about latency than most web-based services). What do you use to
>> redirect the
>> incoming connections?
>
> We use Apache with mod_proxy. This is fine for HTTP (the extra
> latency literally gets lost in the noise), but a chattier protocol
> would be different, of course. I wonder if someone who knew their way
> around ipfw could rig something up...

Yes, iptables magic came to my mind, too. This would have very low
latency overhead, I doubt it would be measurable at all.

- Bert -

RobertLJ

Re: [squeak-dev] Re: Upgrading running servers?

What about using something like nginx instead of Apache as a proxy to multiple images? Has anyone used nginx ?

http://nginx.net/

Bob

On Fri, Apr 4, 2008 at 6:14 AM, Bert Freudenberg <[hidden email]> wrote:

On 04.04.2008, at 08:42, Avi Bryant wrote:

On Thu, Apr 3, 2008 at 7:25 PM, Andreas Raab <[hidden email]> wrote:

We currently don't do this mostly because I'm unaware of a low-latency
proxy/forwarding solution that we could use (obviously we're more concerned
about latency than most web-based services). What do you use to redirect the
incoming connections?

We use Apache with mod_proxy. This is fine for HTTP (the extra
latency literally gets lost in the noise), but a chattier protocol
would be different, of course. I wonder if someone who knew their way
around ipfw could rig something up...

Yes, iptables magic came to my mind, too. This would have very low latency overhead, I doubt it would be measurable at all.

- Bert -

johnmci

Re: [squeak-dev] Re: Upgrading running servers?

In reply to this post by Andreas.Raab

On Apr 3, 2008, at 7:25 PM, Andreas Raab wrote:

> We currently don't do this mostly because I'm unaware of a low-
> latency proxy/forwarding solution that we could use (obviously we're
> more concerned about latency than most web-based services). What do
> you use to redirect the incoming connections?
>
> Cheers,
> - Andreas

In my past experiences as a network guy I always felt it was easier to
blame the network. Well like people blame the GC for a variety of
problems.
In reality some usage of a protocol analyser and trace program is
require: http://developer.apple.com/qa/qa2001/qa1176.html
to figure out what exactly is going on and where the millisecond
disappear.

As you know guessing is usually wrong.

--
=
=
=
========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com
=
=
=
========================================================================

Andreas.Raab

[squeak-dev] Re: Upgrading running servers?

In reply to this post by Bert Freudenberg

Bert Freudenberg wrote:

> On 04.04.2008, at 08:42, Avi Bryant wrote:
>> On Thu, Apr 3, 2008 at 7:25 PM, Andreas Raab <[hidden email]> wrote:
>>> We currently don't do this mostly because I'm unaware of a low-latency
>>> proxy/forwarding solution that we could use (obviously we're more
>>> concerned
>>> about latency than most web-based services). What do you use to
>>> redirect the
>>> incoming connections?
>>
>> We use Apache with mod_proxy. This is fine for HTTP (the extra
>> latency literally gets lost in the noise), but a chattier protocol
>> would be different, of course. I wonder if someone who knew their way
>> around ipfw could rig something up...
>
> Yes, iptables magic came to my mind, too. This would have very low
> latency overhead, I doubt it would be measurable at all.

Can iptables look at the content of a connection? The problem is that if
a user is in a forum and another one tries to join him, they need to end
up in the same image (at least right now). And we can't use different
ports for this - we run through firewalls and NATs and all sorts of
insane corporate IT setups so the choice of port is not completely ours.

Cheers,
- Andreas

Bert Freudenberg

Re: [squeak-dev] Re: Upgrading running servers?

On 04.04.2008, at 20:14, Andreas Raab wrote:

> Bert Freudenberg wrote:
>> On 04.04.2008, at 08:42, Avi Bryant wrote:
>>> On Thu, Apr 3, 2008 at 7:25 PM, Andreas Raab <[hidden email]>
>>> wrote:
>>>> We currently don't do this mostly because I'm unaware of a low-
>>>> latency
>>>> proxy/forwarding solution that we could use (obviously we're more
>>>> concerned
>>>> about latency than most web-based services). What do you use to
>>>> redirect the
>>>> incoming connections?
>>>
>>> We use Apache with mod_proxy. This is fine for HTTP (the extra
>>> latency literally gets lost in the noise), but a chattier protocol
>>> would be different, of course. I wonder if someone who knew their
>>> way
>>> around ipfw could rig something up...
>> Yes, iptables magic came to my mind, too. This would have very low
>> latency overhead, I doubt it would be measurable at all.
>
> Can iptables look at the content of a connection? The problem is
> that if a user is in a forum and another one tries to join him, they
> need to end up in the same image (at least right now). And we can't
> use different ports for this - we run through firewalls and NATs and
> all sorts of insane corporate IT setups so the choice of port is not
> completely ours.

Yes, iptables can look at the packet contents, for example using the
u32 module:

http://www.stearns.org/doc/iptables-u32.current.html

You should ask a real expert of course, I never did this myself.

- Bert -

Andreas.Raab

[squeak-dev] Re: Upgrading running servers?

Bert Freudenberg wrote:

>>> Yes, iptables magic came to my mind, too. This would have very low
>>> latency overhead, I doubt it would be measurable at all.
>>
>> Can iptables look at the content of a connection? The problem is that
>> if a user is in a forum and another one tries to join him, they need
>> to end up in the same image (at least right now). And we can't use
>> different ports for this - we run through firewalls and NATs and all
>> sorts of insane corporate IT setups so the choice of port is not
>> completely ours.
>
>
> Yes, iptables can look at the packet contents, for example using the u32
> module:
>
> http://www.stearns.org/doc/iptables-u32.current.html
>
> You should ask a real expert of course, I never did this myself.

Ooooohhhhh. How useful. I think you just solved that problem for me.

Thanks,
- Andreas