Smalltalk › Frameworks & Tools › Magma

depoyment

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

14 messages Options

mmimica

depoyment

Hello List!

I'm developing a small website in Seaside and using Magma for
persistence. Now it's time to deploy the project to a server where
development would continue in a more real environment. I'm not sure how
to do it, since I've never deployed a smalltalk image.

I was going to run the Magma server in a separate VM, containing not a
lot more than needed by Magma. Let's say a plain Pharo image with Magma
installed. Seems reasonable to me, but is it a good idea to have it
separated from the WEB server and the only client?

The other thing I'm concerned about is stopping the server. How do I
shut it down gracefully when running in headless mode?

--
Milan Mimica
http://sparklet.sf.net

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Amir Ansari

Re: depoyment

This is my setup:

A web server (FreeBSD + nginx), and a second server with Seaside VM + Magma VM. The Magma VM is just Pharo with the Magma server package. Both VMs are running headless, but have RFB (RemoteFrameBuffer) installed, so I can manage them remotely via VNC viewer. Monticello works nicely for package updates, too.

Amir

On Tue, 09 Aug 2011 20:48:02 +0200
Milan Mimica <[hidden email]> wrote:

> Hello List!
>
> I'm developing a small website in Seaside and using Magma for
> persistence. Now it's time to deploy the project to a server where
> development would continue in a more real environment. I'm not sure how
> to do it, since I've never deployed a smalltalk image.
>
> I was going to run the Magma server in a separate VM, containing not a
> lot more than needed by Magma. Let's say a plain Pharo image with Magma
> installed. Seems reasonable to me, but is it a good idea to have it
> separated from the WEB server and the only client?
>
> The other thing I'm concerned about is stopping the server. How do I
> shut it down gracefully when running in headless mode?

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Chris Muller-3

Re: depoyment

In reply to this post by mmimica

Hi Milan,

> I'm developing a small website in Seaside and using Magma for persistence.
> Now it's time to deploy the project to a server where development would
> continue in a more real environment. I'm not sure how to do it, since I've
> never deployed a smalltalk image.

Congratulations. Deploying is fun because you get to see your
hard-work put to action. Some suggestions:

- Write deployment "make" scripts. I like to write mine in
Smalltalk and put it right in the same MC package as the app. It
assembles all the files, etc. into a zip. It's a nice way to document
how to deploy the system since it also provides the convenience of
producing it in one-click. One nice feature is to have a single-file
deployment (i.e., zip, tgz, etc.), if possible, since that makes it
easy to scp to remote servers.

- I also recommend, for the above, to include the Squeak VM you want
to use to run it. Include it in the app deployment directory and then
all of your OS utility scripts point, relative-path, to that
executable. That way it's self-contained and can run anywhere since
no specific permissions are required.

- A set of OS utility scripts - "start", "stop" of course, but I
also have, "upgrade" since I seem to be upgrading my magma server code
every few months (to, ahem, eat my own dog food). Also, perhaps,
"backup", although I also perform backups remotely on-demand.

- I don't know what OS you're using but.. since you are using Magma,
you already have MaCommandLineProcessor in your image. It is +very
convenient+ for interfacing with Squeak from the Linux command-line.

> I was going to run the Magma server in a separate VM, containing not a lot
> more than needed by Magma.

I like to run my own Magma servers in their own image too. I use a
pre-built image and it's not saved by my shutdown script. That way, I
always know what I'm running.

> Let's say a plain Pharo image with Magma
> installed. Seems reasonable to me, but is it a good idea to have it
> separated from the WEB server and the only client?

Yes, this approach provides upward scalability.

> The other thing I'm concerned about is stopping the server. How do I shut it
> down gracefully when running in headless mode?

I've attached working start and stop scripts (one line each!). See
stops.st, which opens a "admin MagmaSession" and instructs the server
to shut down gracefully.

- Chris

>
> --
> Milan Mimica
> http://sparklet.sf.net
>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

magma-linux-scripts.zip (1K) Download Attachment

mmimica

Re: depoyment

Chris Muller wrote:
> - I don't know what OS you're using but.. since you are using Magma,
> you already have MaCommandLineProcessor in your image. It is +very
> convenient+ for interfacing with Squeak from the Linux command-line.
>

I'm getting:
MessageNotUnderstood: StandardFileStream class>>stdout

at CommandLineProcessor class >> do

Using Pharo 1.2

--
Milan Mimica
http://sparklet.sf.net

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Chris Muller-4

Re: depoyment

Ok, would you mind finding out whether Pharo supports writing to
stdout? It may need to be upgraded to have the same capabilities as
Squeak.

You may want to consider just deploying your Magma servers on Squeak.
You could still use Pharo for your web-server if you want..

Another option, if you feel you must run Pharo, you could comment out
that code which accesses stdout and run "blind" - no console
messages...

- Chris

On Wed, Aug 10, 2011 at 3:25 PM, Milan Mimica <[hidden email]> wrote:

> Chris Muller wrote:
>>
>> - I don't know what OS you're using but.. since you are using Magma,
>> you already have MaCommandLineProcessor in your image. It is +very
>> convenient+ for interfacing with Squeak from the Linux command-line.
>>
>
> I'm getting:
> MessageNotUnderstood: StandardFileStream class>>stdout
>
> at CommandLineProcessor class >> do
>
> Using Pharo 1.2
>
>
> --
> Milan Mimica
> http://sparklet.sf.net
>
>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Igor Stasenko

Re: depoyment

I think we're added stdout support starting from 1.3.

But it is worse than that: some VMs are still not supporting stdout :(

On 10 August 2011 22:48, Chris Muller <[hidden email]> wrote:

> Ok, would you mind finding out whether Pharo supports writing to
> stdout? It may need to be upgraded to have the same capabilities as
> Squeak.
>
> You may want to consider just deploying your Magma servers on Squeak.
> You could still use Pharo for your web-server if you want..
>
> Another option, if you feel you must run Pharo, you could comment out
> that code which accesses stdout and run "blind" - no console
> messages...
>
> - Chris
>
> On Wed, Aug 10, 2011 at 3:25 PM, Milan Mimica <[hidden email]> wrote:
>> Chris Muller wrote:
>>>
>>> - I don't know what OS you're using but.. since you are using Magma,
>>> you already have MaCommandLineProcessor in your image. It is +very
>>> convenient+ for interfacing with Squeak from the Linux command-line.
>>>
>>
>> I'm getting:
>> MessageNotUnderstood: StandardFileStream class>>stdout
>>
>> at CommandLineProcessor class >> do
>>
>> Using Pharo 1.2
>>
>>
>> --
>> Milan Mimica
>> http://sparklet.sf.net
>>
>>
> _______________________________________________
> Magma mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/magma
>

--
Best regards,
Igor Stasenko AKA sig.
_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

mmimica

Re: depoyment

In reply to this post by Chris Muller-4

Yes, I figured I can run it blind and that's fine. But I've got another problem I just can't make sense out of it.
Running this code on my machine:
| mySession | mySession := (MagmaRemoteLocation host: 'my.server.ip.addr' port: 51001) newSession. mySession connectAs: 'chris'. mySession root. mySession disconnect.

I am getting this error:
ConnectionTimedOut: Cannot connect to 127.0.0.1:51001

What confuses hell out of me is that it tries to connect to localhost when sending #root to session object. Why? How?

Chris Muller wrote:

Ok, would you mind finding out whether Pharo supports writing to
stdout?  It may need to be upgraded to have the same capabilities as
Squeak.

You may want to consider just deploying your Magma servers on Squeak.
You could still use Pharo for your web-server if you want..

Another option, if you feel you must run Pharo, you could comment out
that code which accesses stdout and run "blind" - no console
messages...

 - Chris

On Wed, Aug 10, 2011 at 3:25 PM, Milan Mimica [hidden email] wrote:

Chris Muller wrote:

 - I don't know what OS you're using but.. since you are using Magma,
you already have MaCommandLineProcessor in your image.  It is +very
convenient+ for interfacing with Squeak from the Linux command-line.

I'm getting:
MessageNotUnderstood: StandardFileStream class>>stdout

at CommandLineProcessor class >> do

Using Pharo 1.2


--
Milan Mimica
http://sparklet.sf.net

-- 
Milan Mimica
http://sparklet.sf.net

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Chris Muller-4

Re: depoyment

Magma uses NetNameResolver to convert the hostname you provide to an IP.

On Wed, Aug 10, 2011 at 4:35 PM, Milan Mimica <[hidden email]> wrote:

> Yes, I figured I can run it blind and that's fine. But I've got another
> problem I just can't make sense out of it.
> Running this code on my machine:
> | mySession |
> mySession :=
>     (MagmaRemoteLocation
>       host: 'my.server.ip.addr'
>       port: 51001) newSession.
> mySession connectAs: 'chris'.
> mySession root.
> mySession disconnect.
>
> I am getting this error:
> ConnectionTimedOut: Cannot connect to 127.0.0.1:51001
>
> What confuses hell out of me is that it tries to connect to localhost when
> sending #root to session object. Why? How?
>
>
> Chris Muller wrote:
>
> Ok, would you mind finding out whether Pharo supports writing to
> stdout? It may need to be upgraded to have the same capabilities as
> Squeak.
>
> You may want to consider just deploying your Magma servers on Squeak.
> You could still use Pharo for your web-server if you want..
>
> Another option, if you feel you must run Pharo, you could comment out
> that code which accesses stdout and run "blind" - no console
> messages...
>
> - Chris
>
> On Wed, Aug 10, 2011 at 3:25 PM, Milan Mimica <[hidden email]>
> wrote:
>
>
> Chris Muller wrote:
>
>
> - I don't know what OS you're using but.. since you are using Magma,
> you already have MaCommandLineProcessor in your image. It is +very
> convenient+ for interfacing with Squeak from the Linux command-line.
>
>
>
> I'm getting:
> MessageNotUnderstood: StandardFileStream class>>stdout
>
> at CommandLineProcessor class >> do
>
> Using Pharo 1.2
>
>
> --
> Milan Mimica
> http://sparklet.sf.net
>
>
>
>
>
>
> --
> Milan Mimica
> http://sparklet.sf.net
>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

mmimica

Re: depoyment

Would you please stop assuming I'm a moron and read again what I wrote? Here, I'll write it down again, perhaps more clear this time:

| mySession | mySession := (MagmaRemoteLocation host: '217.172.181.118' port: 51001) newSession. Transcript show: mySession primaryLocation; cr. " --> says a MagmaRemoteLocation magma:217.172.181.118:51001" mySession connectAs: 'chris'. Transcript show: mySession primaryLocation; cr. " --> says a MagmaRemoteLocation magma:localhost:51001" mySession root. " --> fails because it's trying to connect to localhost " mySession disconnect.
I just doesn't look right, does it?
I did some debugging, and what happens is that in MagmaSession>>primConnect repository definition is overridden by something which is, I assume, returned by the server.

Chris Muller wrote:

Magma uses NetNameResolver to convert the hostname you provide to an IP.

On Wed, Aug 10, 2011 at 4:35 PM, Milan Mimica [hidden email] wrote:

Yes, I figured I can run it blind and that's fine. But I've got another
problem I just can't make sense out of it.
Running this code on my machine:
| mySession |
  mySession :=
    (MagmaRemoteLocation
      host: 'my.server.ip.addr'
      port: 51001) newSession.
  mySession connectAs: 'chris'.
  mySession root.
  mySession disconnect.

I am getting this error:
ConnectionTimedOut: Cannot connect to 127.0.0.1:51001

What confuses hell out of me is that it tries to connect to localhost when
sending #root to session object. Why? How?


Chris Muller wrote:

Ok, would you mind finding out whether Pharo supports writing to
stdout?  It may need to be upgraded to have the same capabilities as
Squeak.

You may want to consider just deploying your Magma servers on Squeak.
You could still use Pharo for your web-server if you want..

Another option, if you feel you must run Pharo, you could comment out
that code which accesses stdout and run "blind" - no console
messages...

 - Chris

On Wed, Aug 10, 2011 at 3:25 PM, Milan Mimica [hidden email]
wrote:


Chris Muller wrote:


 - I don't know what OS you're using but.. since you are using Magma,
you already have MaCommandLineProcessor in your image.  It is +very
convenient+ for interfacing with Squeak from the Linux command-line.



I'm getting:
MessageNotUnderstood: StandardFileStream class>>stdout

at CommandLineProcessor class >> do

Using Pharo 1.2


--
Milan Mimica
http://sparklet.sf.net






--
Milan Mimica
http://sparklet.sf.net

-- 
Milan Mimica
http://sparklet.sf.net

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Chris Muller-4

Re: depoyment

Hey Milan, I never thought you were a "moron" - why would I from just
two e-mails? You asked a question, and I spent my precious time
trying to give you a helpful answer, so I didn't appreciate that
attitude.

Yes, the Node is a persistent object in the database and the server
commits its hostname - using NetNameResolver localHostName. See
MagmaServerConsole>>#thisLocation and you can browse senders from
there if you're interested. It sounds like you need to set up a
hostname for your machine.

- Chris

On Thu, Aug 11, 2011 at 12:17 PM, Milan Mimica <[hidden email]> wrote:

> Would you please stop assuming I'm a moron and read again what I wrote?
> Here, I'll write it down again, perhaps more clear this time:
>
> | mySession |
> mySession :=
>     (MagmaRemoteLocation
>       host: '217.172.181.118'
>       port: 51001) newSession.
> Transcript show: mySession primaryLocation; cr. " --> says a
> MagmaRemoteLocation magma:217.172.181.118:51001"
> mySession connectAs: 'chris'.
> Transcript show: mySession primaryLocation; cr. " --> says a
> MagmaRemoteLocation magma:localhost:51001"
> mySession root. " --> fails because it's trying to connect to localhost "
> mySession disconnect.
>
> I just doesn't look right, does it?
> I did some debugging, and what happens is that in MagmaSession>>primConnect
> repository definition is overridden by something which is, I assume,
> returned by the server.
>
>
> Chris Muller wrote:
>
> Magma uses NetNameResolver to convert the hostname you provide to an IP.
>
> On Wed, Aug 10, 2011 at 4:35 PM, Milan Mimica <[hidden email]>
> wrote:
>
>
> Yes, I figured I can run it blind and that's fine. But I've got another
> problem I just can't make sense out of it.
> Running this code on my machine:
> | mySession |
> mySession :=
>     (MagmaRemoteLocation
>       host: 'my.server.ip.addr'
>       port: 51001) newSession.
> mySession connectAs: 'chris'.
> mySession root.
> mySession disconnect.
>
> I am getting this error:
> ConnectionTimedOut: Cannot connect to 127.0.0.1:51001
>
> What confuses hell out of me is that it tries to connect to localhost when
> sending #root to session object. Why? How?
>
>
> Chris Muller wrote:
>
> Ok, would you mind finding out whether Pharo supports writing to
> stdout? It may need to be upgraded to have the same capabilities as
> Squeak.
>
> You may want to consider just deploying your Magma servers on Squeak.
> You could still use Pharo for your web-server if you want..
>
> Another option, if you feel you must run Pharo, you could comment out
> that code which accesses stdout and run "blind" - no console
> messages...
>
> - Chris
>
> On Wed, Aug 10, 2011 at 3:25 PM, Milan Mimica <[hidden email]>
> wrote:
>
>
> Chris Muller wrote:
>
>
> - I don't know what OS you're using but.. since you are using Magma,
> you already have MaCommandLineProcessor in your image. It is +very
> convenient+ for interfacing with Squeak from the Linux command-line.
>
>
>
> I'm getting:
> MessageNotUnderstood: StandardFileStream class>>stdout
>
> at CommandLineProcessor class >> do
>
> Using Pharo 1.2
>
>
> --
> Milan Mimica
> http://sparklet.sf.net
>
>
>
>
>
>
> --
> Milan Mimica
> http://sparklet.sf.net
>
>
>
>
>
> --
> Milan Mimica
> http://sparklet.sf.net
>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

mmimica

Re: depoyment

Chris Muller wrote:
> Yes, the Node is a persistent object in the database and the server
> commits its hostname - using NetNameResolver localHostName. See
> MagmaServerConsole>>#thisLocation and you can browse senders from
> there if you're interested. It sounds like you need to set up a
> hostname for your machine.
>

Now that's some useful info. I have transfered the database from my
machine to a dedicated server. By transfered I mean plain file copy. So
the remote database now has the wrong Node written, with "localhost" in
it. How do I fix?

--
Milan Mimica
http://sparklet.sf.net

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Chris Muller-4

Re: depoyment

It's been a while since I've "moved" a repository from one host to
another. IIRC, the Node must be updated manually in that case.

Just do it through a normal transaction. Open the repository with a
local session, #begin, update the node the hostname and port the
primary will be running on, then #commit. If you want to do HA, make
a #fullBackup AFTER you start primary and copy those backup files to
your secondary host and start it up (you may rename the repository
directory in the target if you wish - Magma doesn't care). That
secondary will look into the persistent Node to know where to connect
to the primary to receive updates.

I run all my my repositories HA, but my environment and processes have
been stable for a couple of years now - so it's been a while since
I've been in the HA code - so please forgive my going by memory here.
Let me know if you have any troubles.

- Chris

On Fri, Aug 12, 2011 at 10:05 AM, Milan Mimica <[hidden email]> wrote:

> Chris Muller wrote:
>>
>> Yes, the Node is a persistent object in the database and the server
>> commits its hostname - using NetNameResolver localHostName. See
>> MagmaServerConsole>>#thisLocation and you can browse senders from
>> there if you're interested. It sounds like you need to set up a
>> hostname for your machine.
>>
>
> Now that's some useful info. I have transfered the database from my machine
> to a dedicated server. By transfered I mean plain file copy. So the remote
> database now has the wrong Node written, with "localhost" in it. How do I
> fix?
>
> --
> Milan Mimica
> http://sparklet.sf.net
>
>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

mmimica

Re: depoyment

Chris Muller wrote:
> It's been a while since I've "moved" a repository from one host to
> another. IIRC, the Node must be updated manually in that case.
>

I think I'm just going to recreate the database. There's only a few test
data in it.
But sooner or later there is likely going to be some moving around. Can
I rebuild the database from the commit logs maybe?

--
Milan Mimica
http://sparklet.sf.net

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma

Chris Muller-4

Re: depoyment

Mmm, replaying commit-logs doesn't really assist the process of moving
it to a new server.

Moving to a new server can be done one of two ways; either by the
approach I mentioned in my earlier e-mail (shutting it down and
updating the Node manually) or by starting a warm-backup on the server
you want to become the new primary and then invoking a deliberate swap
of primary duty with the current primary. This latter approach allows
Magma repositories to move between servers without stopping service
and is demonstrated by the test-suite, HA Test Case 4,
#verifyQuickSwapFunction.

HTH,
Chris

On Sun, Aug 14, 2011 at 4:50 AM, Milan Mimica <[hidden email]> wrote:

> Chris Muller wrote:
>>
>> It's been a while since I've "moved" a repository from one host to
>> another. IIRC, the Node must be updated manually in that case.
>>
>
> I think I'm just going to recreate the database. There's only a few test
> data in it.
> But sooner or later there is likely going to be some moving around. Can I
> rebuild the database from the commit logs maybe?
>
> --
> Milan Mimica
> http://sparklet.sf.net
>
>

_______________________________________________
Magma mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/magma