OPentalk question

Hello everyone;

Does anyone use an Opentalk in the real production environment ?

Can you share your experiences as far as pitfalls to watch out for,
maybe some suggestions, etc.

Had anybody built an Opentalk server running  headless as service on NT
or Linux environments?


I am now designing distributed system, and I am thinking of building
middle tear with Opentalk VW.  From the description , it looks like
Opentalk takes care of such mundane things as communications, sessions,
connections, object marshaling and so on. my concern is how easy it is
to program in it and how stable the server might be...

Any information is greatly appreciated.

--Mark

Well, maybe a bit of biased feedback won't hurt either :-). Personally I think Opentalk is a good choice. It is used in production by our users, although I don't have a good feeling about the actual level of abuse it's handling there. However, for example in our own tests an Opentalk based HTTP server seemed to handle decent amounts of load just fine. And all the Opentalk based protocol stacks share lots of the same infrastructure support.

As far as ease of use goes, I don't think it can get much easier than it already is with Opentalk-STST. I'm more concerned about it being deceivingly easy. The web of remote references can explode exponentially when not kept in check and you quickly end up exchanging tens of thousands of remote messages and creating all kinds of impossible dependencies between the images. So it won't solve the difficult problems of distributed computing for you, it will just enable it. It's still up to you to control and optimize the remote interactions.

From your short list below I should note that we don't have a built in session support in Opentalk, which certainly is a limitation, but the rest is ready to use. So you may need to extend the framework further to meet your needs, but extensibility was one of the main goals with Opentalk, so I sure hope that you'll find it acceptable in that regard. If not I'd like to hear about it too.

HTH,

Martin

Mark Pirogovsky wrote:

Hi Mark;

I was using Opentalk in a real production environment since around 2000.  I
think my shop was one of the first using it in production.  It was pretty
easy to set up, and after that, just worked with no problems.  It was pretty
much a Windows shop (Win2K).

See
http://www.cincomsmalltalk.com/userblogs/arden/blogView?showComments=true&pr
intTitle=Discovering_things_on_your_way_there_...&entry=3339067620


Relevant part:

Second, we had a problem with certain situations, where all the users would
refresh a large volume of complex data, (of usually the same or similar
data), all at the same time. This swamped the database so badly that we were
seeing unacceptably long user delays. Really unacceptable. We solved this
problem using Opentalk. These particular large scale refresh queries were
re-directed to an Opentalk enabled server. Queries came in on a queue. The
first one retrieved the data from the relational db and cached it. Any
additional queries, that were looking for the same data, were returned the
results from the cache. Problem solved.

- Arden Thomas
[hidden email]

-----Original Message-----
From: Mark Pirogovsky [mailto:[hidden email]]
Sent: Wednesday, May 16, 2007 9:31 AM
To: [hidden email]
Subject: OPentalk question

Hello everyone;

Does anyone use an Opentalk in the real production environment ?

Can you share your experiences as far as pitfalls to watch out for,
maybe some suggestions, etc.

Had anybody built an Opentalk server running  headless as service on NT
or Linux environments?


I am now designing distributed system, and I am thinking of building
middle tear with Opentalk VW.  From the description , it looks like
Opentalk takes care of such mundane things as communications, sessions,
connections, object marshaling and so on. my concern is how easy it is
to program in it and how stable the server might be...

Any information is greatly appreciated.

--Mark

No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.5.467 / Virus Database: 269.7.1/805 - Release Date: 5/15/2007
10:47 AM
 

No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.467 / Virus Database: 269.7.1/807 - Release Date: 5/16/2007
6:05 PM
 

Martin,
Thanks for the information.

The problem I am trying to solve is how to build middle tier which would
respond few simple requests and return few simple objects, and I want to
keep it simple as well as I want to stay away from programming server
type things (sessions, connections, processes, etc.), object
serializations and such.  So far it looks like the OpenTalk is a good
match, with simple API and the plumbing hidden.

One concern though -- the only answer to the group came from you, one of
the authors, it is a bit like to ask Ford people which car is better
Ford or Toyota :-)

So is it possible to encounter Opentalk based things "in the wild",
outside the lab?  And more - if deployed how much care and feeding the
opentalk application server would need ?

And as far as STST communication: is it wide open or it is possible to
use some kind of encryption, to protect it ?

TIA

--Mark

Martin Kobetic wrote:

I hoped that someone from Cincom would comment but one use of opentalk in production is the team's own bug tracking database MARS.  This is a client/server app written in VW.  The code's not pretty because its evolved over the years, but when Opentalk was used to free it from a Wave-baed Web interface it became very nice to use.  Martin and Bob might say more.

Mark:

For what its worth I played with OT for a while and its quite pluggable,  
adding your own marshallers is straightforward. As far as to its use I  
have not used it for anything commercial though.  We do plan to use it.  
One of the biggest problems with Opentalk both personally in my hobby  
frameworks but also for real production , at least for us, was its issues  
with navigating across NATs. Keep in mind though , that I want to be able  
to use full transparency i.e. full support of pass by reference semantics.  
I understand that pass by value semantics could have been used. To be  
clear I'm talking about the use of Opentalk ST-ST i.e. distributed  
Smalltalk to Smalltalk messaging.

In anycase, I believe that I was told that those issues are now fixed in  
7.5. I have not had a chance to confirm.

hth,


-Charles
-

-

On Thu, 17 May 2007 11:15:29 -0400, Mark Pirogovsky  
<[hidden email]> wrote:



--
Charles A. Monteiro
http://wiki.nycsmalltalk.org
http://www.monteirosfusion.com
http://monteirofusion.blogspot.com

Thank you Charles,

Could anybody elaborate more on the NAT traversal business ?

Does it mean that I can not use the OPenTalk over the internet, where
different type of the connection might be present ?  In my case I am
talking more of the RPC type communication where the client requests
the service and receives response from the server...similar to the HTTP
GET:.

TIA.

--Mark

Charles A. Monteiro wrote:

Here's a copy of the 7.5 release note dealing with the topic of Firewalls and such:

STST and Firewalls

Previously there were severe limitations imposed on the STST protocol
when sending requests to a broker behind a firewall. With properly
configured localityTest (available as of VW 7.4) the broker is able to
accept the request; however, it wasn't able to export new objects with a
valid, publicly accessible address. So any communication would have to
be restricted to passing all arguments by value (as opposed to passing by
reference). This is functionally equivalent to capabilities provided by
protocols like SOAP, but it severely restricts the transparency and
freedom that the STST protocol is capable of.

In order to enable usable pass-by-reference semantics, the broker has to
advertise its external (firewall) address in the object references that it
generates. This will provide the clients with a valid (firewall) address to
connect to. The clients brokers will then connect to the firewall and the
firewall will forward those connection requests to the server broker
behind it. Of course, the firewall has to be properly configured to do so (it
is the same setup as with any other server that is exposed to the
Internet).

It is now possible to configure the broker for this type of setup. All that
needs to be done is, when the broker is being created via
BrokerConfiguration newAt: anAddress
the address parameter must be the external (firewall) address through
which it can be accessed. The same applies to the various instance
creation methods on the class side of BasicRequestBroker.
Note that if the server broker needs to send a request back to client, the
roles are reversed. For this to happen the client has to export an object
and provide accessible address in the corresponding object reference. If
the client is sitting behind a firewall, it will have to be configured similarly
as the server, with an external access address and with the firewall
configured to forward to the client. Otherwise the server won't be able to
establish a connection back to the client. It is not yet possible for the
server to reuse the previously established connection from the client to
deliver requests from the server.

Another limitation is that currently the port number that is opened on the
firewall for the purpose of forwarding traffic to an STST broker must be
the same as the port number used by the broker on the physical host that
it runs on. So if the open port on the firewall is 4242 the broker will also
bind to port 4242 on its host machine. Consequently this port number
must be available on the host.

Finally, note that there is a slight overlap of this new functionality with
previously available support for binding to specific network interfaces.
Previously if the address parameter to the broker creation methods
wasn't a wildcard address (0.0.0.0) or completely unspecified (via
#newAtPort:), it was expected to be an address of one of the network
interfaces of the broker's host or localhost (127.0.0.1). The broker would
then bind to that specific interface instead of all available interfaces.
Specifying any other address would result in broker's failure to start. Now
it is possible to specify some other address in which case the broker will
expect it to be an external (firewall) address. In this case the broker will
bind to all local interfaces and advertise the external address in all object
references that it generates. For this the broker has to be able to
distinguish a local address from an external one. However, there's
currently no platform independent way to query for all the host interfaces
and to obtain their IP addresses. Opentalk attempts to work around this
limitation by obtaining the host name from the operating system and
converting it to an IP address using reverse DNS lookup (see
GenericProtocol class>>hostAddress). That yields satisfactory results in
many cases, provided that the local DNS is configured properly and the
host has single network interface. However it cannot handle multiple
network interfaces or misconfigured DNS. Therefore we've added
GenericProtocol class>>hostAddresses, which is an explicit, manually
configured list of local network interface addresses. If the dynamic IP
address discovery fails in some configuration, the user will have to
configure this list manually. In most cases this shouldn't be necessary.

Bidirectional Connection Support (STST)

Bidirectional connection is a new capability that will be useful for
Opentalk clients attempting to access a server from behind a firewall.
Previously the server was unable to send requests back to the client
unless that client was exposed through the firewall to incoming
connections (which is usually blocked by client firewalls). This new
capability allows the server to reuse a previously established connection
initiated by the client (which is usually allowed by client firewalls) for
requests sent from the server to the client (e.g. callbacks to client
objects). It also allows for better management of networking resources on
heavily loaded servers, as there won't be a need for two separate
connections per client when there are requests flowing in both directions.
Bidirectional support is controlled by picking the right type of adaptor for
broker configuration. ConnectionAdaptor means the broker is bidirection
capable, and AsymmetricConnectionAdaptor means it is not.

Currently only STST provides bidirection support. Choosing the
ConnectionAdaptor for any other protocol will make no difference, the
connections will still be asymmetric (Note that many of the protocols,
such as HTTP, require that anyway). It is safe to mix bidirection capable
and incapable brokers arbitrarily; they will simply fall back into the non
bidirectional connection mode. Only two bidirection capable brokers can
take advantage of bidirectional connections.

CAUTION! There is an issues that maintainers of bidirection enabled
public servers need to be aware of. If the client brokers aren't explicitly
set up with distinct access points, then they will most likely pick up their
local host IP. The usual case is going to be that the IP will be from one of
the private network ranges, e.g. 192.168.1/24. If several clients are in this
situation, it may happen that they end up advertising the same
address/port for their objects. The server will not be able to distinguish
them. It will be most likely be the first client broker connecting the server
that will get associated with that address under the bidirectional mode
and all requests to that address/port will then be routed to that broker.
This means that messages potentially meant for other brokers with that
address will be routed to the wrong broker. Without bidirection support,
the request attempts from the server will simply fail in these cases, but
with bidirection mode they may just happen to work with unpredictable,
possibly catastrophic results. So it is important to think about this when
enabling the bidirection mode for a server. Probably the simplest workaround
is simply requiring the client brokers to be configured with their
external firewall address. That will guarantee uniqueness. There's no
need to make any holes in the client firewalls, because requests will take
advantage of bidirectional connections, it's just to make sure all the
clients can be distinguished on the server side.

HTH,

Martin

Mark Pirogovsky wrote:

Hey Eliot,

Thanks for the nudge :-). I felt that Mark was more after experiences from the field rather than a Cincom comment. Also I'm not sure that this is really a typical example of an Opentalk application. It really is just a GUI client attached to what was formerly a web app  via a thin layer of Opentalk-STST. So the remote API still has a web-servlet feel to it to some extent, largely since this was really a spare time project so the emphasis was on doing as little as possible. However I think I can honestly say that thanks to Opentalk there's very little code that deals with network communication, almost none in fact compared to the rest. From the Opentalk point of view it sort of felt like Marvin's "Here I am, brain the size of a planet, and they ask me to take you to the bridge.".

Anyway, one possibly interesting curiosity is that for querying in the database we're passing the actual BlockClosure instances that are composed on the client and executed on the server. With some limitations of course, but I think it's still kinda neat. I don't think that the usual load patterns are very significant here so it's probably not much of a testament in that regard. One issue that it did expose is currently limited support for large byte/character objects. So passing for example a smalltalk image attached to a bug report could put a significant strain on both sides of the connection. However there is a number of ways to work around that fairly easily. Ideally I'd like to add optimized support for streaming straight to/from  external streams built directly into the STST protocol. But that's not there yet.

HTH,

Martin

Eliot Miranda wrote:
> I hoped that someone from Cincom would comment but one use of opentalk
> in production is the team's own bug tracking database MARS.  This is a
> client/server app written in VW.  The code's not pretty because its
> evolved over the years, but when Opentalk was used to free it from a
> Wave-baed Web interface it became very nice to use.  Martin and Bob
> might say more.

Mark:

Again, its my understanding the given your requirmeents you could use pass  
by value semantics but the question is how transparent do you want to keep  
things? Maybe you don't need ST-ST, but could indeed use an HTTP based  
marshaller.

Anyhow, the NAT thing with regards to Opentalk ST-ST is supposed to be  
fixed :), we put off doing stuff because of that issue and now hope to  
ramp up those plans again after we get past a couple short term fires.

BTW, there were also other issues such as Opentalk ST-ST being able to  
traverse thru a secure tunnel i.e. like STunnel, again hope those were  
fixed , basically related to an extent to the general issue of  
understanding the correct remote references.

FYI, I have also used it internally within our network to proxy db  
connections i.e. an image with Opentalk, no database layer which includes  
no db client installation, no dlls etc which in turn talks to an app  
server hooked to an rdms. I was surprised that I did not noticed any  
discernible performance difference in updating our UIs. Mind you , I never  
tested it for scaling i.e. I have everything within access of our local  
lan.

If Opentalk gets past these glaring issues it will be a great and enabling  
framework. Again, is supposedly has :)

-Charles

On Thu, 17 May 2007 15:00:15 -0400, Mark Pirogovsky  
<[hidden email]> wrote:



--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/