Cloud Foundry

Hi guys,

What is the current status of putting Gemstone on Cloud Foundry
(http://cloudfoudry.org)? Maglev as Ruby is probably close, and if
Maglev will be supported, there is not much steps to Smalltalk support
too, is this a right way of thinking?

And I love to have Aida hosting on Gemstone Cloud Foundry .. :)

Best regards
Janko


--
Janko Mivšek
Aida/Web
Smalltalk Web Application Server
http://www.aidaweb.si

Am 23.07.2011 um 14:21 schrieb Janko Mivšek:

> Hi guys,
>
> What is the current status of putting Gemstone on Cloud Foundry
> (http://cloudfoudry.org)? Maglev as Ruby is probably close, and if
> Maglev will be supported, there is not much steps to Smalltalk support
> too, is this a right way of thinking?
>
> And I love to have Aida hosting on Gemstone Cloud Foundry .. :)
>
Janko, do you know more about what cloud foundry is? I've checked the docs on their web site but must confess that I don't really get what it will be. I can imagine that they like to help developers by choosing a framework upfront so deployment can be assisted and mapped onto vmware vms. Looking at my projects in the last years there was always some edge case involved that would have prevented the usage of something like cloud foundry. That is because any help you get from such a framework is a restriction if you look at it from a different angle. If anyone of you has some insights (maybe the new members of the vmware family ;) ) that would be great.

At the moment I'm playing with veewee, vagrant, chef/puppet to automate building a complete appliance from scratch. And I'm working on having most of the steps done by jenkins. The outcome wil be a single command that builds a virtual machine that could be e.g. uploaded to amazons EC^2. That is most flexible but maintaining the stuff you need to do yourself.

I'm still at the beginning of figuring out the best solution. The stuff I'm using right now is virtualbox based. There will be more intercompatibility in the future but for now some has to decide which road to follow.

thanks in advance,

Norbert

S, Norbert Hartl piše:
>
> Am 23.07.2011 um 14:21 schrieb Janko Mivšek:

>> What is the current status of putting Gemstone on Cloud Foundry
>> (http://cloudfoudry.org)? Maglev as Ruby is probably close, and if
>> Maglev will be supported, there is not much steps to Smalltalk support
>> too, is this a right way of thinking?
>>
>> And I love to have Aida hosting on Gemstone Cloud Foundry .. :)

> Janko, do you know more about what cloud foundry is? I've checked the docs on their web site but must confess that I don't really get what it will be. I can imagine that they like to help developers by choosing a framework upfront so deployment can be assisted and mapped onto vmware vms. Looking at my projects in the last years there was always some edge case involved that would have prevented the usage of something like cloud foundry. That is because any help you get from such a framework is a restriction if you look at it from a different angle. If anyone of you has some insights (maybe the new members of the vmware family ;) ) that would be great.

Think Cloud Foundry as a preprepared GLASS VMware appliance, where you
can install and run your apps, but in the cloud. Gemstone is a database
service while the foundry take care about preparing and running your
app. Kind of starting and stopping GS Gems.

Instead in your appliance you can run it everywhere in the cloud. And
add resources like CPU and memory "elastically" when needed. And even
migrate it from one provider to another, because Cloud Foundry is
supposed to be a a standard.

> At the moment I'm playing with veewee, vagrant, chef/puppet to automate building a complete appliance from scratch. And I'm working on having most of the steps done by jenkins. The outcome wil be a single command that builds a virtual machine that could be e.g. uploaded to amazons EC^2. That is most flexible but maintaining the stuff you need to do yourself.
>
> I'm still at the beginning of figuring out the best solution. The stuff I'm using right now is virtualbox based. There will be more intercompatibility in the future but for now some has to decide which road to follow.
>
> thanks in advance,
>
> Norbert
>
>

--
Janko Mivšek
Aida/Web
Smalltalk Web Application Server
http://www.aidaweb.si

Janko,

We are busy working away at integrating MagLev and GemStone/S into the Cloud Foundry and we're hoping to have a private cloud up and running by ESUG.

For the Smalltalk language we plan to provide a GemStone/S 'service' (persistent data store) and support for deploying Seaside, Iliad, Aida and Pier applications. The basic idea is that you'd develop your application in Pharo/Squeak and then deploy into the cloud foundry...There are a bunch of technical details that have to be worked out and I should have more information for my ESUG talk[1].

Dale

[1] http://www.esug.org/wiki/pier/Conferences/2011/Schedule-And-Talks/Glass

----- Original Message -----
| From: "Janko Mivšek" <[hidden email]>
| To: "GemStonebeta discussion" <[hidden email]>
| Sent: Saturday, July 23, 2011 5:21:47 AM
| Subject: [GS/SS Beta] Cloud Foundry
|
| Hi guys,
|
| What is the current status of putting Gemstone on Cloud Foundry
| (http://cloudfoudry.org)? Maglev as Ruby is probably close, and if
| Maglev will be supported, there is not much steps to Smalltalk
| support
| too, is this a right way of thinking?
|
| And I love to have Aida hosting on Gemstone Cloud Foundry .. :)
|
| Best regards
| Janko
|
|
| --
| Janko Mivšek
| Aida/Web
| Smalltalk Web Application Server
| http://www.aidaweb.si
|

Norbert,

Janko did a good job of describing the cloud foundry in more detail ...

I am curious what types of things you consider to be restrictions ...

The basic model is that you will deploy a Metacello configuration for a particular web/application framework (Seaside, Aida, etc.) into the cloud. At the moment we are imagining that you will be assigned your own GemStone user id (providing data security) and your code will be loaded into the a SymbolDictionary for that user (Seaside, Aida, etc. will pre pre-deployed for that user). Gems and a shared page cache will be started on one of the os vms in the cloud and you'll be given an URL that can be used to connect to your gem(s). Once you've created your initial instance you will be able to hit your primary Seaside application over the web. For development, you will be using tODE[1] over the web.

The cloud will take care of managing the execution of the gems ... if the hardware it is running on fails or the process itself falls over for some reason, the cloud will restart the gem. You can request additional gems to be added or removed depending upon your needs ....

There will be one humongous stone running (probably several) to provide the persistent store.

At least that is what we are thinking at the moment...

In addition to resolving a bunch of technical details, we need to navigate the corporate waterways....But we've got folks working on all of these things....

The Cloud Foundry is in beta within VMware and as we get to the point where we have a functional GemStone/S service I'm sure we'll be asking you folks to join the Cloud Foundry beta program and take things for a spin...

I think that the big take away here is that if everything works out, VMware will be providing a persistent Smalltalk native object store in the cloud and you guys won't have to worry about managing a stone ....

With that said, there are no plans afoot to prevent folks from standing up your own stones in EC2 and running with the free license ...

Dale

[1] http://code.google.com/p/tode/

----- Original Message -----
| From: "Norbert Hartl" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Saturday, July 23, 2011 6:04:49 AM
| Subject: Re: [GS/SS Beta] Cloud Foundry
|
|
| Am 23.07.2011 um 14:21 schrieb Janko Mivšek:
|
| > Hi guys,
| >
| > What is the current status of putting Gemstone on Cloud Foundry
| > (http://cloudfoudry.org)? Maglev as Ruby is probably close, and if
| > Maglev will be supported, there is not much steps to Smalltalk
| > support
| > too, is this a right way of thinking?
| >
| > And I love to have Aida hosting on Gemstone Cloud Foundry .. :)
| >
| Janko, do you know more about what cloud foundry is? I've checked the
| docs on their web site but must confess that I don't really get what
| it will be. I can imagine that they like to help developers by
| choosing a framework upfront so deployment can be assisted and
| mapped onto vmware vms. Looking at my projects in the last years
| there was always some edge case involved that would have prevented
| the usage of something like cloud foundry. That is because any help
| you get from such a framework is a restriction if you look at it
| from a different angle. If anyone of you has some insights (maybe
| the new members of the vmware family ;) ) that would be great.
|
| At the moment I'm playing with veewee, vagrant, chef/puppet to
| automate building a complete appliance from scratch. And I'm working
| on having most of the steps done by jenkins. The outcome wil be a
| single command that builds a virtual machine that could be e.g.
| uploaded to amazons EC^2. That is most flexible but maintaining the
| stuff you need to do yourself.
|
| I'm still at the beginning of figuring out the best solution. The
| stuff I'm using right now is virtualbox based. There will be more
| intercompatibility in the future but for now some has to decide
| which road to follow.
|
| thanks in advance,
|
| Norbert
|
|

Am 23.07.2011 um 18:19 schrieb Dale Henrichs:

> Norbert,
>
> Janko did a good job of describing the cloud foundry in more detail ...
>
> I am curious what types of things you consider to be restrictions ...

I think cloud foundry is a good fit for web-only applications. There are standard ways of deploying applications and automated orchestration of services. So it is of big help. The logical consequence is that non-standard deployments and non off-the-shelf orchestration needs become restrictions pretty soon. A classical problem is access to the filesystem. Theoretically you don't need it when using gemstone and there is something humongous (learned a new word) in the background. But I think there will be size restrictions in service and after all it isn't feasible for gemstone to deliver a lot of binary data. If the existent deployment mechanisms don't allow filesystem access or don't have some tools to debug them (we know it is always a drag) then I would call it a restriction which will eat a lot of time in deployment.
In my projects I always have different needs in orchestration because I often need more than one service to collaborate. At the current project this is gemstone and node.js services that need to talk to each other. In custom orchestration I can build shortcuts for the services throughout the system that may avoid the need for additional security setups. If there is no help in orchestration you can have multiple services but you are forced to put all your services into public if you want them to collaborate (thus raise the need for an extra SSL setup or something similar). In my case the node.js service is only used by gemstone and it shouldn't be public.
Well, I would hope I would have projects that are sufficiently covered by a web-only approach but I'm not too lucky :)
>
> The basic model is that you will deploy a Metacello configuration for a particular web/application framework (Seaside, Aida, etc.) into the cloud. At the moment we are imagining that you will be assigned your own GemStone user id (providing data security) and your code will be loaded into the a SymbolDictionary for that user (Seaside, Aida, etc. will pre pre-deployed for that user). Gems and a shared page cache will be started on one of the os vms in the cloud and you'll be given an URL that can be used to connect to your gem(s). Once you've created your initial instance you will be able to hit your primary Seaside application over the web. For development, you will be using tODE[1] over the web.
>
That is great. I assume it is still possible to use gemtools. Or will the object log integrated somewhere in the tODE? A handler that can query the object log would be a good thing. Then you don't need this in cloud foundry because you can query the non-emptiness of the object log from remote.
Will there be anything like the sandboxing we talked in barcelona about? I mean that you can upload your new version that is loaded into a sandbox to try before release? That would be ober cool! (Can I say it that way? It is really funny struggeling to find the right way how other countries embed your language into theirs :) )

Norbert

Embedded reply...

----- Original Message -----
| From: "Norbert Hartl" <[hidden email]>
| To: "GemStone Seaside beta discussion" <[hidden email]>
| Sent: Monday, July 25, 2011 4:00:29 AM
| Subject: Re: [GS/SS Beta] Cloud Foundry
|
|
| Am 23.07.2011 um 18:19 schrieb Dale Henrichs:
|
| > Norbert,
| >
| > Janko did a good job of describing the cloud foundry in more detail
| > ...
| >
| > I am curious what types of things you consider to be restrictions
| > ...
|
| I think cloud foundry is a good fit for web-only applications. There
| are standard ways of deploying applications and automated
| orchestration of services. So it is of big help. The logical
| consequence is that non-standard deployments and non off-the-shelf
| orchestration needs become restrictions pretty soon. A classical
| problem is access to the filesystem. Theoretically you don't need it
| when using gemstone and there is something humongous (learned a new
| word) in the background. But I think there will be size restrictions
| in service and after all it isn't feasible for gemstone to deliver a
| lot of binary data. If the existent deployment mechanisms don't
| allow filesystem access or don't have some tools to debug them (we
| know it is always a drag) then I would call it a restriction which
| will eat a lot of time in deployment.

In the case of the Cloud Foundry, you do have access to the file system (they sandbox the unix user for your account), but you cannot depend upon the contents of the file system surviving between GemStone gem invocations ... With that said, files can be deployed along with your source code (the Cloud Foundry term is `droplet`). Part of starting up the GemStone gem is to "load source" which may read data from the files, etc. When means that a certain amount of static binary data can be specified as part of your droplet.

Since we will be inventing the deployment process we should be able to make adjustments to just what gets deployed and how. For example, once you've loaded your mcz files into GemStone (which are part of the droplet) you shouldn't have to load them again ...


| In my projects I always have different needs in orchestration because
| I often need more than one service to collaborate. At the current
| project this is gemstone and node.js services that need to talk to
| each other. In custom orchestration I can build shortcuts for the
| services throughout the system that may avoid the need for
| additional security setups. If there is no help in orchestration you
| can have multiple services but you are forced to put all your
| services into public if you want them to collaborate (thus raise the
| need for an extra SSL setup or something similar). In my case the
| node.js service is only used by gemstone and it shouldn't be public.
| Well, I would hope I would have projects that are sufficiently
| covered by a web-only approach but I'm not too lucky :)

I think that the node.js is a service within CloudFoundry so in theory you should be able to connect to the node.js and GemStone/S services ...
 
| >
| > The basic model is that you will deploy a Metacello configuration
| > for a particular web/application framework (Seaside, Aida, etc.)
| > into the cloud. At the moment we are imagining that you will be
| > assigned your own GemStone user id (providing data security) and
| > your code will be loaded into the a SymbolDictionary for that user
| > (Seaside, Aida, etc. will pre pre-deployed for that user). Gems
| > and a shared page cache will be started on one of the os vms in
| > the cloud and you'll be given an URL that can be used to connect
| > to your gem(s). Once you've created your initial instance you will
| > be able to hit your primary Seaside application over the web. For
| > development, you will be using tODE[1] over the web.
| >
| That is great. I assume it is still possible to use gemtools. Or will
| the object log integrated somewhere in the tODE? A handler that can
| query the object log would be a good thing. Then you don't need this
| in cloud foundry because you can query the non-emptiness of the
| object log from remote.

Within Cloud Foundry there are limits to what external connections you are allowed to accept connections on (basically HTTP only), so GemTools wopn't work in the CloudFoundry ... nor will topaz. tODE is HTTP-based so the intent is to provide all of the development access you need through tODE ... the current incarnation of tODE give you access to the ObjectLog and you can bring up a debugger (more of a stack debugger) on the continuations in the Object Log.

|
| > The cloud will take care of managing the execution of the gems ...
| > if the hardware it is running on fails or the process itself falls
| > over for some reason, the cloud will restart the gem. You can
| > request additional gems to be added or removed depending upon your
| > needs ....
| >
| > There will be one humongous stone running (probably several) to
| > provide the persistent store.
| >
| > At least that is what we are thinking at the moment...
| >
| > In addition to resolving a bunch of technical details, we need to
| > navigate the corporate waterways....But we've got folks working on
| > all of these things....
| >
| > The Cloud Foundry is in beta within VMware and as we get to the
| > point where we have a functional GemStone/S service I'm sure we'll
| > be asking you folks to join the Cloud Foundry beta program and
| > take things for a spin...
| >
| > I think that the big take away here is that if everything works
| > out, VMware will be providing a persistent Smalltalk native object
| > store in the cloud and you guys won't have to worry about managing
| > a stone ....
| >
| > With that said, there are no plans afoot to prevent folks from
| > standing up your own stones in EC2 and running with the free
| > license ...
| >
| Will there be anything like the sandboxing we talked in barcelona
| about? I mean that you can upload your new version that is loaded
| into a sandbox to try before release? That would be ober cool! (Can
| I say it that way? It is really funny struggeling to find the right
| way how other countries embed your language into theirs :) )

It is looking like we might have those sandboxing capabilities sooner than I thought ... There are some things that I'm working with right now that include some of the fundamental technology that is required to do sandboxing:) über cool, ya sure, ya betcha!

|
| Norbert
|
| > Dale
| >
| > [1] http://code.google.com/p/tode/
| >
| > ----- Original Message -----
| > | From: "Norbert Hartl" <[hidden email]>
| > | To: "GemStone Seaside beta discussion"
| > | <[hidden email]>
| > | Sent: Saturday, July 23, 2011 6:04:49 AM
| > | Subject: Re: [GS/SS Beta] Cloud Foundry
| > |
| > |
| > | Am 23.07.2011 um 14:21 schrieb Janko Mivšek:
| > |
| > | > Hi guys,
| > | >
| > | > What is the current status of putting Gemstone on Cloud Foundry
| > | > (http://cloudfoudry.org)? Maglev as Ruby is probably close, and
| > | > if
| > | > Maglev will be supported, there is not much steps to Smalltalk
| > | > support
| > | > too, is this a right way of thinking?
| > | >
| > | > And I love to have Aida hosting on Gemstone Cloud Foundry .. :)
| > | >
| > | Janko, do you know more about what cloud foundry is? I've checked
| > | the
| > | docs on their web site but must confess that I don't really get
| > | what
| > | it will be. I can imagine that they like to help developers by
| > | choosing a framework upfront so deployment can be assisted and
| > | mapped onto vmware vms. Looking at my projects in the last years
| > | there was always some edge case involved that would have
| > | prevented
| > | the usage of something like cloud foundry. That is because any
| > | help
| > | you get from such a framework is a restriction if you look at it
| > | from a different angle. If anyone of you has some insights (maybe
| > | the new members of the vmware family ;) ) that would be great.
| > |
| > | At the moment I'm playing with veewee, vagrant, chef/puppet to
| > | automate building a complete appliance from scratch. And I'm
| > | working
| > | on having most of the steps done by jenkins. The outcome wil be a
| > | single command that builds a virtual machine that could be e.g.
| > | uploaded to amazons EC^2. That is most flexible but maintaining
| > | the
| > | stuff you need to do yourself.
| > |
| > | I'm still at the beginning of figuring out the best solution. The
| > | stuff I'm using right now is virtualbox based. There will be more
| > | intercompatibility in the future but for now some has to decide
| > | which road to follow.
| > |
| > | thanks in advance,
| > |
| > | Norbert
| > |
| > |
|
|