Smalltalk › Instantiations

headless server forking new VA process and telling it what to do via SST

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

5 messages Options

Wayne Johnston

headless server forking new VA process and telling it what to do via SST

Has anyone done this and/or have advice?

If your server function is handled as a Windows service, one way of handling more work is to have multiple of those services running the same VA image. But how about instead you have just one service, and it dynamically creates a new process running the same packaged image, say with an argument specifying the port to use, and then the main service can communicate to the new image via SST over that port?

You could imagine many such "worker bee" processes, maybe one per CPU or something...

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Marten Feldtmann-4

Re: headless server forking new VA process and telling it what to do via SST

As a general answer I would say: yes.

If the pure communication becomes a point to consider one should really thing about using frameworks like ZeroMQ together with VASmalltalk. By using frameworks like this VASmalltalk does not have to do the communication task, which might be pretty intensive for VA (we had a pretty ip centric application doing much, much data transfer and more that 30-40 clients is difficult for one VA image to handle) - but sending /receiving is done in a different C task and VASmalltalk does the logic of answering the requests. By going this way, VA scales much, much better ... and ZeroMQ encourages the usages of server-worker processes.

Douglas Swartz

Re: headless server forking new VA process and telling it what to do via SST

In reply to this post by Wayne Johnston

Wayne,

I've worked on systems that used this approach. A few years ago we had
a system serving web requests with the Abt HTML classes rather than
SST. It worked in a very similar manner. I think we sometimes had up to 100
images running concurrently on a good sized AIX machine. Each listened
on a separate port, and a front end balanced requests to the
processes.

20 years ago we had a system running on Solaris that worked
similarly. That one used 30-40 images which picked up their work from
MQ-Series queues rather than off an IP port. That system effectively
handled lots of background/batch requests in the same images that
handled interactive requests by using two different MQ queues.

The nice thing about different images and OS processes to handle large
amounts of concurrent work is that you don't have to handle
multi-process switching and coordination in the image. This simplifies
transaction coding. Obviously you still have to ensure you don't
create database deadlocks and such.

Doug

Thursday, October 29, 2015, 12:32:45 PM, you wrote:

> Has anyone done this and/or have advice?

> If your server function is handled as a Windows service, one way of
> handling more work is to have multiple of those services running the
> same VA image. But how about instead you have just one service, and
> it dynamically creates a new process running the same packaged
> image, say with an argument specifying the port to use, and then the
> main service can communicate to the new image via SST over that port?

> You could imagine many such "worker bee" processes, maybe one per CPU or something...

--
Best regards,
Douglas mailto:[hidden email]

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Instantiations mailing list

Re: [Bulk] headless server forking new VA process and telling it what to do via SST

In reply to this post by Wayne Johnston

Should work,.....

....but....

it takes quite a lot of work to implement a framework that can deal with all the different errors that may occur.
Who closes the worker bee images if the server image is down due to an error?
How do you find out if a woker bee is just busy or is crashed? ... etc ...pp...

Debugging such application can also be painful but having worker a bees and server on different VMs or workstations/servers helps to keep oversight.

Marten is right. A framework based on ZeroMQ might take a lot of that burden from the developers.

In your case you are saying that you will have one and the same job scaled. I guess in that case it might not be that difficult to implement this.
Taking one and the same image for this kind of scaling might run into issues when it comes to the usage of shared libraries.

Are VAST dlls multi-threaded/thread-safe?

One problem that I could imagine is the windows security policies. I know that a headless windows service must be registered as such and I am not sure if the service user is allowed to fire off additional applications.
To be save you might consider to have an individual path including all VAST deployment files, external resource files and a copy of your server image for each of your potential workerbee instances. That will enable you to register each image with an individual name to log into the windows event registry. That makes debugging based on logs much easier.
The problem here is, that you will have to register all your potential headless server images as services and that can't be done during runtime....
and you might have to start them all at the same time no matter if needed or not...

Sebastian

Am 29.10.2015 um 10:32 schrieb Wayne Johnston:

Has anyone done this and/or have advice?

If your server function is handled as a Windows service, one way of handling more work is to have multiple of those services running the same VA image. But how about instead you have just one service, and it dynamically creates a new process running the same packaged image, say with an argument specifying the port to use, and then the main service can communicate to the new image via SST over that port?

You could imagine many such "worker bee" processes, maybe one per CPU or something...

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at http://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Wayne Johnston

Re: headless server forking new VA process and telling it what to do via SST

In reply to this post by Wayne Johnston

Thanks everyone.

On what Sebastian wrote: "Are VAST dlls multi-threaded/thread-safe?"

They must be. For several years we've had some applications where multiple Windows services use a single installation of the DLLs and one image.