RAM recommendations for hosting

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

RAM recommendations for hosting

Nick
Hi,

I'm looking into hosting for when we're finally ready to go live with our GLASS app - do people have recommendations for a practical quantity of RAM? I'm happy to err on the cautious side so that if our PR works out and drives traffic to the site we won't be embarrassed.

I've scanned the old messages in the maillist about latency using Slicehost from Europe. Any recommendations for a European alternative and does the decreased latency make connecting to the remote server through Gemtools practical?

Thanks

Nick

 
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

SeanTAllen
In terms of RAM, the free version of gemstone/s is limited in the
amount that it can use.
You should be able to get by with 2-4 gigs w/o much of an issue while
maxing out the capabilities of
the free version. That assumes you have other applications as well (
webserver etc ).

On Tue, Aug 10, 2010 at 11:03 AM, Nick Ager <[hidden email]> wrote:

> Hi,
> I'm looking into hosting for when we're finally ready to go live with our
> GLASS app - do people have recommendations for a practical quantity of RAM?
> I'm happy to err on the cautious side so that if our PR works out and drives
> traffic to the site we won't be embarrassed.
> I've scanned the old messages in the maillist about latency using Slicehost
> from Europe. Any recommendations for a European alternative and does the
> decreased latency make connecting to the remote server through Gemtools
> practical?
> Thanks
> Nick
>
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

NorbertHartl
In reply to this post by Nick

On 10.08.2010, at 17:03, Nick Ager wrote:

> Hi,
>
> I'm looking into hosting for when we're finally ready to go live with our GLASS app - do people have recommendations for a practical quantity of RAM? I'm happy to err on the cautious side so that if our PR works out and drives traffic to the site we won't be embarrassed.
>
I think I saw some article about an installation in a 256M slice. I have openvz on my server and played a little with different settings. One of the more affecting parameters is the size of the shared page cache.I tried to give it 200M but this made a few things really slow. The default setting of 500M is quite ok. I'm using this and the virtual instance has 1G of RAM and runs well. If you want to raise performance then you need to increase the shared page cache although it is limited to 1G (and I guess you still could get away with 2G RAM). If you need more performance after that than you won't use something like slicehost but a dedicated machine with multiple disks for extents and tranlogs and such. Ahh, and of course a bigger gemstone edition that lets you use more than one CPU. But to be honest my advize would be to work with the installation defaults and see how far you can get. Server hosters upgrade you happily anytime and the gemstone installation is moved from machine to machine in no time.

> I've scanned the old messages in the maillist about latency using Slicehost from Europe. Any recommendations for a European alternative and does the decreased latency make connecting to the remote server through Gemtools practical?
>
We have our servers at http://www.hetzner.de/ and this is a really good one. We never had big problems with them.

Norbert


Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Nick
Hi

Thanks for the advice - much appreciated. Norbert with your servers hosted locally - does the latency allow you to use Gemtools across the Internet to manage your server?

Nick 

On 10 August 2010 17:19, Norbert Hartl <[hidden email]> wrote:

On 10.08.2010, at 17:03, Nick Ager wrote:

> Hi,
>
> I'm looking into hosting for when we're finally ready to go live with our GLASS app - do people have recommendations for a practical quantity of RAM? I'm happy to err on the cautious side so that if our PR works out and drives traffic to the site we won't be embarrassed.
>
I think I saw some article about an installation in a 256M slice. I have openvz on my server and played a little with different settings. One of the more affecting parameters is the size of the shared page cache.I tried to give it 200M but this made a few things really slow. The default setting of 500M is quite ok. I'm using this and the virtual instance has 1G of RAM and runs well. If you want to raise performance then you need to increase the shared page cache although it is limited to 1G (and I guess you still could get away with 2G RAM). If you need more performance after that than you won't use something like slicehost but a dedicated machine with multiple disks for extents and tranlogs and such. Ahh, and of course a bigger gemstone edition that lets you use more than one CPU. But to be honest my advize would be to work with the installation defaults and see how far you can get. Server hosters upgrade you happily anytime and the gemstone installation is moved from machine to machine in no time.

> I've scanned the old messages in the maillist about latency using Slicehost from Europe. Any recommendations for a European alternative and does the decreased latency make connecting to the remote server through Gemtools practical?
>
We have our servers at http://www.hetzner.de/ and this is a really good one. We never had big problems with them.

Norbert



Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Dale Henrichs
In reply to this post by Nick
Nick Ager wrote:
> Hi,
>
> I'm looking into hosting for when we're finally ready to go live with
> our GLASS app - do people have recommendations for a practical quantity
> of RAM? I'm happy to err on the cautious side so that if our PR works
> out and drives traffic to the site we won't be embarrassed.

For performance, your Shared Page Cache should be sized such that the
whole db fits...For practicality, your Shared Page Cache should be big
enough to house your working set ... which is application specific.

You need enough seaside gems to handle the number of _concurrent_
requests that you expect ... if you miss, then requests will be queued
up so it's not the end of the world. With a 50M tempObjCache I think
that translates to ~150M memory footprint. The default maintenance vm
runs at 200M tempObjCache but I think that could be much smaller in
practice (say 50M tempObjectCache).

These are the variables ... GLASS has been run on 256M slices, but keep
in mind that we do don't allocate the full 150M for a Gem (we size the
gem based on demand) so for production, you need to size the slice to
accommodate largest size you would expect ... which really will come
from experience/testing..

>
> I've scanned the old messages in the maillist about latency using
> Slicehost from Europe. Any recommendations for a European alternative
> and does the decreased latency make connecting to the remote server
> through Gemtools practical?

I haven't done the A/B tests between using vnc and "local gemtools over
the wire" for remote access, but I think those are your two options.

I would think that you'd want a local sandbox system for doing basic
development and testing, so that you don't have to do development
directly on slicehost.

I would also think that for routine maintenance activities, you should
use topaz scripts which are command line activities ... many if not most
of the admin scripts are easily done in topaz.

So that leaves debugging issues that are showing up in production. In
that case, my first line of defense is the ObjectLog (installed with
authorization ... see the Seaside30 workspace). With the object log you
can inspect each of the continuations that are dropped into the object
log and at lest get the a picture of the stack trace ... if you need to
debug the error, then I take a deep breath and and bring up the
continuation under the remote debugger ... alternativley for complex
issues, I've occasionally dropped object log entries from my application
code (like a transcript show: with live objects) and then used the
inspector from the ObjectLog to inspect things ...

BTW, once you get yourself an inspector, you do have a workspace that's
running in your browser, so you can execute arbitrary expresssions and
inspect the results without the delays you'd see from GemTools ....
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

SeanTAllen
On Tue, Aug 10, 2010 at 1:09 PM, Dale Henrichs <[hidden email]> wrote:

> Nick Ager wrote:
>>
>> Hi,
>>
>> I'm looking into hosting for when we're finally ready to go live with our
>> GLASS app - do people have recommendations for a practical quantity of RAM?
>> I'm happy to err on the cautious side so that if our PR works out and drives
>> traffic to the site we won't be embarrassed.
>
> For performance, your Shared Page Cache should be sized such that the whole
> db fits...For practicality, your Shared Page Cache should be big enough to
> house your working set ... which is application specific.
>
> You need enough seaside gems to handle the number of _concurrent_ requests
> that you expect ... if you miss, then requests will be queued up so it's not
> the end of the world. With a 50M tempObjCache I think that translates to
> ~150M memory footprint. The default maintenance vm runs at 200M tempObjCache
> but I think that could be much smaller in practice (say 50M
> tempObjectCache).
>
> These are the variables ... GLASS has been run on 256M slices, but keep in
> mind that we do don't allocate the full 150M for a Gem (we size the gem
> based on demand) so for production, you need to size the slice to
> accommodate largest size you would expect ... which really will come from
> experience/testing..
>
>>
>> I've scanned the old messages in the maillist about latency using
>> Slicehost from Europe. Any recommendations for a European alternative and
>> does the decreased latency make connecting to the remote server through
>> Gemtools practical?
>
> I haven't done the A/B tests between using vnc and "local gemtools over the
> wire" for remote access, but I think those are your two options.
>
> I would think that you'd want a local sandbox system for doing basic
> development and testing, so that you don't have to do development directly
> on slicehost.
>
> I would also think that for routine maintenance activities, you should use
> topaz scripts which are command line activities ... many if not most of the
> admin scripts are easily done in topaz.
>
> So that leaves debugging issues that are showing up in production. In that
> case, my first line of defense is the ObjectLog (installed with
> authorization ... see the Seaside30 workspace). With the object log you can
> inspect each of the continuations that are dropped into the object log and
> at lest get the a picture of the stack trace ... if you need to debug the
> error, then I take a deep breath and and bring up the continuation under the
> remote debugger ... alternativley for complex issues, I've occasionally
> dropped object log entries from my application code (like a transcript show:
> with live objects) and then used the inspector from the ObjectLog to inspect
> things ...
>
> BTW, once you get yourself an inspector, you do have a workspace that's
> running in your browser, so you can execute arbitrary expresssions and
> inspect the results without the delays you'd see from GemTools ....
>

We had serious issues with getting Gemtools to run in a fashion that
made working with a remote machine really feasible.
What we did and I highly recommend is creating a local vm that mirrors
your production setup as closely as possible.
When you need to debug server side issues, do a backup of the
database, download it to your local vm, restore it there
and debug locally using Gemtools. For anything other than minor
issues, it is a far more pleasurable experience.
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Reza Razavi
In reply to this post by Nick
At 17:03 10/08/2010, Nick Ager wrote:
Any recommendations for a European alternative

Hi Nick,

Linode offers a UK-based solution. Please check:
http://blog.linode.com/2009/12/

Regards,
Reza
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Nick
In reply to this post by Dale Henrichs
Hi Dale,


These are the variables ... GLASS has been run on 256M slices, but keep in mind that we do don't allocate the full 150M for a Gem (we size the gem based on demand) so for production, you need to size the slice to accommodate largest size you would expect ... which really will come from experience/testing..

Thanks for the comprehensive response - it's given me a good starting point and from there, I'll just have to measure.

I would think that you'd want a local sandbox system for doing basic development and testing, so that you don't have to do development directly on slicehost.
 
Yes I'm setting that up.
 
I would also think that for routine maintenance activities, you should use topaz scripts which are command line activities ... many if not most of the admin scripts are easily done in topaz.

Ok I'll start studying the topaz manual...
 

So that leaves debugging issues that are showing up in production. In that case, my first line of defense is the ObjectLog (installed with authorization ... see the Seaside30 workspace). With the object log you can inspect each of the continuations that are dropped into the object log and at lest get the a picture of the stack trace ... if you need to debug the error, then I take a deep breath and and bring up the continuation under the remote debugger ... alternativley for complex issues, I've occasionally dropped object log entries from my application code (like a transcript show: with live objects) and then used the inspector from the ObjectLog to inspect things ...

BTW, once you get yourself an inspector, you do have a workspace that's running in your browser, so you can execute arbitrary expresssions and inspect the results without the delays you'd see from GemTools ....

I've registered the ObjectLog and had a look - it's making more sense having read your mail; eg I found the inspector - not sure what I thought the priority link would do...

Thanks again, very helpful as ever

Nick
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Stephan Eggermont-3
In reply to this post by Nick

On 10 aug 2010, at 18:43, Nick Ager wrote:
> Thanks for the advice - much appreciated. Norbert with your servers hosted locally - does the latency allow you to use Gemtools across the Internet to manage your server?

I've run the gemtools on a slicehost, with a local X11 display accessed through

ssh -C -p4567 -X glass@slice GemTools-2.3.1.app/GemTools.sh

that was acceptable, though not fast.

Hetzner binds servers to a 100Mbps connection. That doesn't sound nice when you have lots
of pictures. They are cheap for root servers and co-location, though.
We are thinking about a slice at hosteurope.

>  If you need more performance after that than you won't use something like slicehost but a dedicated machine with multiple disks for extents and tranlogs and such.

A MacMini with two small SSDs I would suggest as next step.

Stephan Eggermont
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Nick
In reply to this post by Reza Razavi
Hi Raza,
 
Linode offers a UK-based solution. Please check:
http://blog.linode.com/2009/12/


Thanks for the tip - think that's the one I'll go with..

Nick 
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

Nick
In reply to this post by Stephan Eggermont-3
Hi Stephan,
 
I've run the gemtools on a slicehost, with a local X11 display accessed through

ssh -C -p4567 -X glass@slice GemTools-2.3.1.app/GemTools.sh

Thanks for the command line hints..
 
A MacMini with two small SSDs I would suggest as next step.

:-)

Thanks all the advice, really useful

Nick 
Reply | Threaded
Open this post in threaded view
|

Re: RAM recommendations for hosting

NorbertHartl
In reply to this post by Stephan Eggermont-3

On 10.08.2010, at 21:01, Stephan Eggermont wrote:


On 10 aug 2010, at 18:43, Nick Ager wrote:
Thanks for the advice - much appreciated. Norbert with your servers hosted locally - does the latency allow you to use Gemtools across the Internet to manage your server?

I've run the gemtools on a slicehost, with a local X11 display accessed through

ssh -C -p4567 -X glass@slice GemTools-2.3.1.app/GemTools.sh

that was acceptable, though not fast.

Never thought about exporting the client via X11 over a ssh connection :) X11 tends to be rather slow on updates. Did you try vnc? The last thing I tried in that direction was a headless Xvnc server that you can connect with any vnc client.

In the past I used ssh tunnels to connect to the server. http://selfish.org/blog/easy%20remote%20gemstone

Nowadays I use a openvpn connection to my server. OpenVPN in udp mode is a bit more reliable then tcp based ssh. Ok, for me there are several reasons. I have more than one virtual instance. I connect to the root host and route all traffic through the tunnel. So I can use the gemtools to any instance and I can access the /seaside/config app which isn't reachable from the outside. 

Hetzner binds servers to a 100Mbps connection. That doesn't sound nice when you have lots
of pictures. They are cheap for root servers and co-location, though.
We are thinking about a slice at hosteurope.

Ok, I never had problems with that. But then I don't have sites that are very successful :) We have mail server, dns and such. For this hetzner setup is ok because you can go self maintained. Hosteurope is pain because the don't allow to have your own nameservers and if there are troubles you're lost.

If you need more performance after that than you won't use something like slicehost but a dedicated machine with multiple disks for extents and tranlogs and such.

A MacMini with two small SSDs I would suggest as next step.

A mac mini installed at hosteurope? :)

Norbert