Amazon EC2

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Amazon EC2

Bob Houston
Hello,

I noticed, this morning, that Amazon reopened their "limited beta" of  
the Amazon Elastic Compute Cloud service.
Has anyone looked into using this services to host seaside applications?
Thanks.

http://www.amazon.com/gp/browse.html?node=201590011
http://overstimulate.com/articles/2006/08/24/amazon-does-it-again.html
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Avi  Bryant

On Sep 6, 2006, at 10:51 AM, Bob Houston wrote:

> Hello,
>
> I noticed, this morning, that Amazon reopened their "limited beta"  
> of the Amazon Elastic Compute Cloud service.
> Has anyone looked into using this services to host seaside  
> applications?

Yes, I think it would be a great platform for that.  That would be an  
interesting iteration on seasidehosting.st: a pre-built linux image  
for EC2 configured to pull a Squeak image and a directory of static  
content out of S3, then start up apache, mod_proxy, and the Squeak VM...

Unfortunately it still seems to be hard to get into the beta.  I  
tried signing up first thing this morning but no luck.

Cheers,
Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Darius Clarke
I really like the idea as well.

Creating a production application w/o a fixed IP address hosted on
their service would be the tricky part.

Cheers,
Darius

On 9/6/06, Avi Bryant <[hidden email]> wrote:

>
> On Sep 6, 2006, at 10:51 AM, Bob Houston wrote:
>
> > Hello,
> >
> > I noticed, this morning, that Amazon reopened their "limited beta"
> > of the Amazon Elastic Compute Cloud service.
> > Has anyone looked into using this services to host seaside
> > applications?
>
> Yes, I think it would be a great platform for that.  That would be an
> interesting iteration on seasidehosting.st: a pre-built linux image
> for EC2 configured to pull a Squeak image and a directory of static
> content out of S3, then start up apache, mod_proxy, and the Squeak VM...
>
> Unfortunately it still seems to be hard to get into the beta.  I
> tried signing up first thing this morning but no luck.
>
> Cheers,
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Avi  Bryant

On Sep 6, 2006, at 11:57 AM, Darius Clarke wrote:

> I really like the idea as well.
>
> Creating a production application w/o a fixed IP address hosted on
> their service would be the tricky part.

Yes.  So, say you had a server named control.seaside.st, with a fixed  
IP, and DNS and Apache + mod_proxy installed.  Say the domain name of  
your production app was myapp.seaside.st, and control.seaside.st was  
the authority for that domain.  To begin with, the myapp domain would  
be pointed to control.seaside.st itself.  The first time a request  
for myapp came in, control would provision an EC2 instance (from a  
waiting pool) with the myapp image.  It would then modify its DNS to  
point myapp to that new EC2 instance, and proxy the request through.  
While the DNS change propagated out (you could have a short TTL), it  
would have to continue to proxy those requests, but eventually  
everyone would be hitting the EC2 instance directly.

When you want to decommission myapp from that EC2 instance, you do  
the same thing in reverse: you switch the DNS back to control, and  
the EC2 instance has to stay up for a few minutes until the TTL is  
over and everyone's cache is updated.

This is a little complex, but has two advantages over the current  
seasidehosting.st setup:

1. There are no capacity problems - if it becomes wildly popular, it  
just means the EC2 bills go up a bit.
2. We get perfect sandboxing, rather than requiring a modified VM:  
each app is running in a separate virtual server so you can let  
people do whatever they want.

Thoughts, refinements?

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

johnmci
In reply to this post by Avi Bryant
As the fellow doing the Sophie storage subsystem I looked into it as  
a way to store Sophie Books. It seems feasible.
Right now we have classes to store a book as a zip file, a set of  
files in a folder, and a set of files on a http server.
I had not pursed this option yet, but hearing one can't get into the  
beta will push back our timing.

On 6-Sep-06, at 11:12 AM, Avi Bryant wrote:

>
> On Sep 6, 2006, at 10:51 AM, Bob Houston wrote:
>
>> Hello,
>>
>> I noticed, this morning, that Amazon reopened their "limited beta"  
>> of the Amazon Elastic Compute Cloud service.
>> Has anyone looked into using this services to host seaside  
>> applications?
>
> Yes, I think it would be a great platform for that.  That would be  
> an interesting iteration on seasidehosting.st: a pre-built linux  
> image for EC2 configured to pull a Squeak image and a directory of  
> static content out of S3, then start up apache, mod_proxy, and the  
> Squeak VM...
>
> Unfortunately it still seems to be hard to get into the beta.  I  
> tried signing up first thing this morning but no luck.
>
> Cheers,
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===


_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

radoslav hodnicak
In reply to this post by Avi Bryant

I was looking at this service (no I don't have beta access either), and
from what I understand your environment/data go poof if your instance is
shut down or crashes(!). I agree this will be neat for pure seaside clones
pumping html, but data persistence might cause some unnecessary headaches.

rado
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

RE: Amazon EC2

Boris Popov, DeepCove Labs (SNN)
Indeed, see the FAQ entry over at http://tinyurl.com/k8knk

"Q: What happens to my data when a system terminates?

The data stored on a specific instance persists only as long as that
instance is alive. You have several options to persist your data:

   1. Prior to terminating an instance, backup the data to persistent
storage, either over the Internet, or to Amazon S3.
   2. Run a redundant set of systems with replication of the data
between them.

We recommend you should not rely on a single instance to provide
reliability for your data."

-Boris

--
+1.604.689.0322
DeepCove Labs Ltd.
4th floor 595 Howe Street
Vancouver, Canada V6C 2T5

[hidden email]

CONFIDENTIALITY NOTICE

This email is intended only for the persons named in the message
header. Unless otherwise indicated, it contains information that is
private and confidential. If you have received it in error, please
notify the sender and delete the entire message including any
attachments.

Thank you.

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of
radoslav hodnicak
Sent: Wednesday, September 06, 2006 1:54 PM
To: The Squeak Enterprise Aubergines Server - general discussion.
Subject: Re: [Seaside] Amazon EC2


I was looking at this service (no I don't have beta access either), and
from what I understand your environment/data go poof if your instance is

shut down or crashes(!). I agree this will be neat for pure seaside
clones
pumping html, but data persistence might cause some unnecessary
headaches.

rado
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Avi  Bryant

On Sep 6, 2006, at 2:03 PM, Boris Popov wrote:

> Indeed, see the FAQ entry over at http://tinyurl.com/k8knk
>
> "Q: What happens to my data when a system terminates?
>
> The data stored on a specific instance persists only as long as that
> instance is alive. You have several options to persist your data:
>
>    1. Prior to terminating an instance, backup the data to persistent
> storage, either over the Internet, or to Amazon S3.
>    2. Run a redundant set of systems with replication of the data
> between them.
>
> We recommend you should not rely on a single instance to provide
> reliability for your data."

Sure, so you need to be connecting to some database system hosted  
elsewhere, or someone needs to build a database which can use S3 as a  
backend...

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Bob Houston
Yes, I agree.   A database hosted elsewhere might be the simplest  
solution.   Of course, an open-source object DB would be nice to  
use.  I'm curious if this might be an acceptable use for a GOODS  
database?  Is the GOODS "replication support" applicable here?  My  
thought is that we could use GOODS to run a primary server in our EC2  
instance, with a replicated server on a real host somewhere.  Or  
perhaps the other way around, with a real host running the primary  
server?   This way a real host machine will have a current copy of  
the data, and can be used to "seed" the EC2 instance at startup.  
Comments?

If GOODS is not the way to go, what other databases do you suggest.  
The goal is to have a local cache of the data in the EC2 instance  
(for performance), with a replica of the data on a real host (for  
persistence).


On Sep 6, 2006, at 5:39 PM, Avi Bryant wrote:

>
> On Sep 6, 2006, at 2:03 PM, Boris Popov wrote:
>
>> Indeed, see the FAQ entry over at http://tinyurl.com/k8knk
>>
>> "Q: What happens to my data when a system terminates?
>>
>> The data stored on a specific instance persists only as long as that
>> instance is alive. You have several options to persist your data:
>>
>>    1. Prior to terminating an instance, backup the data to persistent
>> storage, either over the Internet, or to Amazon S3.
>>    2. Run a redundant set of systems with replication of the data
>> between them.
>>
>> We recommend you should not rely on a single instance to provide
>> reliability for your data."
>
> Sure, so you need to be connecting to some database system hosted  
> elsewhere, or someone needs to build a database which can use S3 as  
> a backend...
>
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Avi  Bryant

On Sep 6, 2006, at 3:12 PM, Bob Houston wrote:

> Yes, I agree.   A database hosted elsewhere might be the simplest  
> solution.   Of course, an open-source object DB would be nice to  
> use.  I'm curious if this might be an acceptable use for a GOODS  
> database?  Is the GOODS "replication support" applicable here?  My  
> thought is that we could use GOODS to run a primary server in our  
> EC2 instance, with a replicated server on a real host somewhere.  
> Or perhaps the other way around, with a real host running the  
> primary server?   This way a real host machine will have a current  
> copy of the data, and can be used to "seed" the EC2 instance at  
> startup.  Comments?

I guess the question is how that "seed" process works... anyone have  
experience with GOODS replication, and know how you bring up a new  
instance?

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Yanni Chiu
In reply to this post by Avi Bryant
Avi Bryant wrote:
> Sure, so you need to be connecting to some database system hosted  
> elsewhere, or someone needs to build a database which can use S3 as a  
> backend...

Given the pricing, it seems more cost effective to set up one or more
dedicated (virtual) machines with a separate provider, to handle "base
load". Then EC2 images would be kicked in as needed. The equation might
change if there were a cost to having zero images running - maybe you
just need to fire one up for an hour once a month.

The base load machines would provide the database, so no need to deal
with backup to S3. However, any communication between base and EC2 would
be charged on the meter.

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

How to compose an HTTP Post command to external server?

Kurt Thams
In reply to this post by Avi Bryant
How does one compose an HTTP Post command to an external (non-Seaside)
server, (setting various variables of course)?

Should it be done using HTTPClient httpPostDocument:

Or is there a more "Seaside"ish way?

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Jason Johnson-3
In reply to this post by Avi Bryant
My only nit-pick would be that you're being a bit optimistic about the
DNS.  Afaik most or all of the ISP's hard code the TTL's to at least 48
hours, so whatever you
set isn't going to even be looked at.

But I don't think that hurts your picture much does it?

Avi Bryant wrote:

>
> On Sep 6, 2006, at 11:57 AM, Darius Clarke wrote:
>
>> I really like the idea as well.
>>
>> Creating a production application w/o a fixed IP address hosted on
>> their service would be the tricky part.
>
> Yes.  So, say you had a server named control.seaside.st, with a fixed
> IP, and DNS and Apache + mod_proxy installed.  Say the domain name of
> your production app was myapp.seaside.st, and control.seaside.st was
> the authority for that domain.  To begin with, the myapp domain would
> be pointed to control.seaside.st itself.  The first time a request for
> myapp came in, control would provision an EC2 instance (from a waiting
> pool) with the myapp image.  It would then modify its DNS to point
> myapp to that new EC2 instance, and proxy the request through.  While
> the DNS change propagated out (you could have a short TTL), it would
> have to continue to proxy those requests, but eventually everyone
> would be hitting the EC2 instance directly.
>
> When you want to decommission myapp from that EC2 instance, you do the
> same thing in reverse: you switch the DNS back to control, and the EC2
> instance has to stay up for a few minutes until the TTL is over and
> everyone's cache is updated.
>
> This is a little complex, but has two advantages over the current
> seasidehosting.st setup:
>
> 1. There are no capacity problems - if it becomes wildly popular, it
> just means the EC2 bills go up a bit.
> 2. We get perfect sandboxing, rather than requiring a modified VM:
> each app is running in a separate virtual server so you can let people
> do whatever they want.
>
> Thoughts, refinements?
>
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
> --No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.405 / Virus Database: 268.12.1/440 - Release Date: 9/6/2006
>
>

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Chris Double
On 9/7/06, Jason Johnson <[hidden email]> wrote:
> My only nit-pick would be that you're being a bit optimistic about the
> DNS.  Afaik most or all of the ISP's hard code the TTL's to at least 48
> hours, so whatever you set isn't going to even be looked at.

How do the various dynamic name services work if that's the case?
These are the services that give you a domain name and you run a
client program on your machine that notifies their servers that your
ip has changed. That seems to happen pretty much instantly.

I've just got on the EC2 beta - it's a very interesting service and
it'll be interesting to see what can be made of it.

Chris.
--
http://www.bluishcoder.co.nz
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Jason Johnson-3
Chris Double wrote:

> On 9/7/06, Jason Johnson <[hidden email]> wrote:
>> My only nit-pick would be that you're being a bit optimistic about the
>> DNS.  Afaik most or all of the ISP's hard code the TTL's to at least 48
>> hours, so whatever you set isn't going to even be looked at.
>
> How do the various dynamic name services work if that's the case?
> These are the services that give you a domain name and you run a
> client program on your machine that notifies their servers that your
> ip has changed. That seems to happen pretty much instantly.
>
> I've just got on the EC2 beta - it's a very interesting service and
> it'll be interesting to see what can be made of it.
>
> Chris.
Well, to be honest I'm not sure.  But the last time I had to change
where my DNS names point
to it was 48 hours before I could get to my site by name.  My company
always just kept serving the old names until our
statistics showed that no one was still using the old name.  I will look
into it when I get time, I have wondered what they were
doing for a while now. :)
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Avi  Bryant

On Sep 7, 2006, at 5:41 AM, Jason Johnson wrote:
>
> Well, to be honest I'm not sure.  But the last time I had to change  
> where my DNS names point
> to it was 48 hours before I could get to my site by name.  My  
> company always just kept serving the old names until our
> statistics showed that no one was still using the old name.  I will  
> look into it when I get time, I have wondered what they were
> doing for a while now. :)

If you run your own DNS server (which was part of the setup I  
described), you can set the expiry to whatever you want... certainly  
setting it to 5 minutes wouldn't be unreasonable (you don't want it  
*too* short or everyone ends up doing way too many DNS lookups).

Avi
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Jason Johnson-3
Avi Bryant wrote:

>
> On Sep 7, 2006, at 5:41 AM, Jason Johnson wrote:
>>
>> Well, to be honest I'm not sure.  But the last time I had to change
>> where my DNS names point
>> to it was 48 hours before I could get to my site by name.  My company
>> always just kept serving the old names until our
>> statistics showed that no one was still using the old name.  I will
>> look into it when I get time, I have wondered what they were
>> doing for a while now. :)
>
> If you run your own DNS server (which was part of the setup I
> described), you can set the expiry to whatever you want... certainly
> setting it to 5 minutes wouldn't be unreasonable (you don't want it
> *too* short or everyone ends up doing way too many DNS lookups).
>
> Avi
> _______________________________________________
> Seaside mailing list
> [hidden email]
> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
>
>
> --No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.405 / Virus Database: 268.12.1/440 - Release Date: 9/6/2006
>
>
Right, but what I am saying is, the users (e.g. AOL users) are using
some ISP for internet access.  When they try to hit your page, their PC
sends a DNS request to whoever it's configured for, which will be the
ISP (e.g. AOL) servers.  The ISP servers will ask the root servers, find
you and give the answer, but they (or they used to) ignore the TTL
field.  They just run a modified version of BIND or whatever with the
cache time hard coded to 2 days.  So for the next 2 days all users that
use the effected ISP server will hit that cache.  That wouldn't mean all
of AOL for example, but some percentage.

Now I don't know how systems that us Dynamic DNS are getting around
this, but I guess they are so it probably wont be a problem.  All I know
is I changed over my domain some months back and I couldn't get to my
site for 2 days by name because of it.

If this reminder is irrelevant for whatever reason, I apologize.  I was
trained for nearly a decade to point such things out. :)
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Amazon EC2

Chris Muller
In reply to this post by Bob Houston
S3, Turk and now EC2.  What a fascinating set of distributed computing
services!  Do we have a new species of computer here (low-cost virtual
computer)?  What a hopeful enablement for small players!

Wouldn't it be neat if, like Turk, anyone could contribute their own
computer to sell their CPU cycles for a few pennies.  Does annyone know
how it works?  How else can Amazon quickly procure enough hardware to
meet demand fluctuations, while still keeping prices fixed and so
reasonable, I dunno?

I think they'll still need a way to protect free services from huge
download run-ups..  Some way for downloaders to share that cost.

I'm imagining, a computer-network simulating a (super)computer, running
some AI program that, itself, running some compelling AI simulation
that has born its own "society" of objects.  Put a 3d world interface
on it and the Matrix is born!

_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Re: Amazon EC2

Darius Clarke
> Put a 3d world interface on it and the Matrix is born!
>

As long as the Matrix has a credit card number.
_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
Reply | Threaded
Open this post in threaded view
|

Re: Re: Amazon EC2

timrowledge
In reply to this post by Chris Muller

On 7-Sep-06, at 7:23 PM, Chris Muller wrote:


> that has born its own "society" of objects.  Put a 3d world interface
> on it and the Matrix is born!
Ick, I hope not. I'd look silly in a black leather coat running  
around saying 'excellent, dooood'.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: HCFI: Halt and Catch Fire Immediate


_______________________________________________
Seaside mailing list
[hidden email]
http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside
12