Updates

Hi all,

First of all, I wanted to send a late congratulations to the Dolphin
guys for building a SOAP server under Swazoo.  I think it's awesome that
Swazoo's framework was flexible enough to allow that.  Nice work!

Although this list has seemed kind of quiet, Swazoo has been far from
dead for me.  I've actually got enough sites hosting with me now that my
DSL line pays me each month -- and there are more to come -- so I've had
good reason to keep Swazoo development alive.  As such, I've attached my
latest version of Swazoo for VisualWorks 5i.3 (it'll probably work under
5i.2 as well).

If anyone else has been actively working on Swazoo, I'd be very
interested in hearing about what's going on.  I never meant for this
project to turn into a sort of benevolent dictatorship, but that might
be the code model that works best for our sort of small project.  Or
maybe we go for branch-and-merge development cycles (that is, if anyone
else is actively developing under Swazoo).  I'm open to ideas here; I
just know how poorly CVS has worked for us.

In this version, there are several things that I'm excited about:

- There's a class called SwazooURI that encapsulates the aspects of a
HTTPRequest's URI much better than the simple collection of path
elements did.  Most significantly (for me) is that it can tell you if a
URI ends in a $/ or not.  This makes it possible for the FileResource to
send #movedPermanently when somebody requests a directory without the
trailing slash.

- The CookieSessionResource, MozillaSidebarResource, and ACLResource are
all included.

- Tests are split out into a separate Parcel.

- The new URI Resolution framework (from quite a while ago) is included.
 It supports such things as "transparent" resources -- resources that
have no URI identifier but are consulted during resolution -- and uses a
double-dispatch to get its work done (instead of advancing/retreating a
stream).  The transparent resources make for some exciting
possibilities: things like counters and logs can be implemented as
resources.  There was an elegance to this that I thought was really
cool.  If you want a counter, put the counted resources under a
CounterResource.  Want web access logged?  Put the resources under an
AccessLogResource.

- This release has the starts of a platform-independent stream
implementation.  In theory, all that should be required to get Swazoo
running on a new dialect of ST is to implement a subclass of
AbstractSwazooSocket called SwazooSocket.  It needs to respond to 8
messages, and there are tests for almost all of them.  I'm curious what
the Dolphin guys think of this.

I also realize that VW5i parcels -- especially XML parcels -- probably
aren't the best distribution format for a cross-platform project.  What
would people prefer?  I'm open to suggestions on this one, too.

Please give feedback, any of you who are still interested in this
project.  I still think there's a lot of potential in Swazoo, and I'm
excited to be a part of this project.


Ken Treis
[hidden email]

Hi Ken,

> First of all, I wanted to send a late congratulations to the Dolphin
> guys for building a SOAP server under Swazoo.  I think it's awesome that
> Swazoo's framework was flexible enough to allow that.  Nice work!

Thanks, Swazoo has been great fun to work with.

> Although this list has seemed kind of quiet, Swazoo has been far from
> dead for me.

Swazoo is also far from dead for me. Jerry and I have made a few changes to
get it to work more smoothly in the Dolphin environment, and I have found
that I have just enjoyed working with it, and as a result learning about
HTTP Servers. I have just bought the book "Web Protocols and Practice;
HTTP/1.1 Networking Protocols, Caching and Traffic Management"
(ISBN:0201710889), and it appears to have some good content. I have been
working with RFC2616 as my main reference, and I have often had to guess at
what the "best practice" is in certain situations.

> good reason to keep Swazoo development alive.  As such, I've attached my
> latest version of Swazoo for VisualWorks 5i.3 (it'll probably work under
> 5i.2 as well).

Thank-you, I have a version of 5i.2 and I was able to install the parcels,
and browse the code.

> If anyone else has been actively working on Swazoo, I'd be very
> interested in hearing about what's going on.  I never meant for this
> project to turn into a sort of benevolent dictatorship, but that might
> be the code model that works best for our sort of small project.  Or
> maybe we go for branch-and-merge development cycles (that is, if anyone
> else is actively developing under Swazoo).  I'm open to ideas here; I
> just know how poorly CVS has worked for us.

This is a tough question, and one that I and Jerry have struggled with.
Jerry's original port had the goal of staying code compatible with the VW
version, and using a compatibility package of extra classes and loose
methods to make this work. As I worked with the code, I found this difficult
to maintain. In certain area's it was impossible.

The exceptions that are signaled, and subtle differences in exception
handling, and what unwind blocks do when a process is terminated, lead to
most of the changes I made in the connection code.

Hi everyone,

It's great to see a little activity on the list.

I think the branch-and-merge model will work fine- it allows everyone to try
out new stuff, and bring the best bits to be merged back in to the main
release.

Those transparent resources DO sound pretty nice!

Jerry Bell
[hidden email]

 

Steve Waring wrote:

If anyone else has been actively working on Swazoo, I'd be very
interested in hearing about what's going on.  I never meant for this
project to turn into a sort of benevolent dictatorship, but that might
be the code model that works best for our sort of small project.  Or
maybe we go for branch-and-merge development cycles (that is, if anyone
else is actively developing under Swazoo).  I'm open to ideas here; I
just know how poorly CVS has worked for us.

This is a tough question, and one that I and Jerry have struggled with.
Jerry's original port had the goal of staying code compatible with the VW
version, and using a compatibility package of extra classes and loose
methods to make this work. As I worked with the code, I found this difficult
to maintain. In certain area's it was impossible.

The exceptions that are signaled, and subtle differences in exception
handling, and what unwind blocks do when a process is terminated, lead to
most of the changes I made in the connection code.

Let me know what you think of the SwazooStream framework -- would it work for Dolphin?  I understand that your HTTPConnection class still might have to differ a bit, so maybe the tradeoffs wouldn't be worth it.

From this perspective, the Dolphin port realistically could not fully merge
back, however I see Swazoo as five distinct "systems" ...
connections,messages,resolution, resources and tests. While merging
connection code would take careful analysis as to the intent of the changes,
we could easily merge resources etc. With both the VW and Dolphin ports
having their own sets of tests, I think it would be relatively easy for me
to integrate changes into Dolphin, and for you to integrate changes into VW.

I don't have Dolphin (I'm running Linux 2.4), but I'll look at your pac files.

The only structural changes I have made are;
-HTTPConnection uses a state pattern, the state being what we believe the
error status of the connection is. Each state responds to four operations;
#readRequest, #getResponse, #writeResponse, and #cleanConnection. The states
themselves are responsible for their own exception handling, and this has
made writing tests for connections easier. Around each operation, events are
signaled, which can be used for secondary processing (logging at the moment)

-Dolphin's HTTPServer has a second loop process, which runs at a higher
priority than the server loop/interact loop, and wakes up every 30 seconds
to look for connections which need closing. (I could not figure out a clean
way to replicate the #interruptWith: in
HTTPConnection>>getAndDispatchMessages. In Dolphin, blocks are not true
closures, and that forked process would keep a reference to the connection
instance).<
/pre>
        

The #interruptWith: code is nasty; I don't like it at all.  The only reason it's there is to wait for a short amount of time before reading the next message.  If no new message arrives, we want to close the stream.

It seems like this could be done with a lower-level timeout (like readWait in VisualWorks) or something similar.  As long as data keeps coming, we want to keep the socket open.  Otherwise, we want to close 'em down.

This problem may have gone away with the new VW SwazooSocket; it seems to dutifully raise an error when the stream is closed.

Other smaller changes;
-Added HEAD method

Cool.  I'll merge it back.

-Added a Script class, which makes it easier to use the same script with
different sites.

Personally, I'd love it if the SiteRegistry went away entirely.  But I haven't figured out something better yet (short of storing Resources in GemStone).

-Added a chunked encoding response

I'll have a look at this, too.

-Added a some classes; a response/resource/identifier for a local proxy
(very non-compliant proxy:)

Sounds kind of cool, though.

-Added etags and a notModified response to FileResource

I'll grab these as well.

-I needed to give resources a pointer to their parent, mainly so that a
resource could work out its address location.

I've been holding off on this one because it didn't seem necessary, philosophically.  But then I found myself passing the request object all over the place while assembling a reply, just so I could have access to the URI.

-The content I have worked on; Spray and QaDResources, are both "namespace"
resources, in that they take over the rest of their child namespace.

Spray is a WebServices package, its goal is to analyze a class, and extract
(from comments and method signatures) the necessary metadata needed to build
a WebServices Description Language (WSDL) file for the class, so that an
instance can expose its methods as WebService operations. The metadata is
then used to decode the soap xml messages into smalltalk objects, and then
encode answers into xml messages.

The QaDResources is a "spike" package, exploring some ideas that myself and
Jerry are interested in. The basic idea being that every instance of a class
in the QaDResources framework has a url, which gives the resource its
identity. Probably the bulk of this package at the moment is a set of
resources that act as "transforms", transforming some object int
o xhtml.

All the code is available at;
http://www.chartexplorer.com/dolphin/SWSpray/spraycode.html

I have not attached a license to this code as yet, because I have trouble
understanding exactly what the LGPL is. My intent is to release Spray free
of any restrictions. This is a learning project for me, and I want to make
it as easy as possible for anyone to look at or use the code. The download
contains everything but the kitchen sink at the moment, but the Dolphin pac
files are in an ASCII chunk format.

I have spent most of the past couple of months on the Spray package, and the
first version is now essentially finished. I have been planning that on
spend time, just working on Swazoo.

I will certainly use that opportunity to add in some of the changes you have
made, and I will try to organise things so that a merge of 
code back to VW
could be made easier. I had assumed that VW development had stopped on
Swazoo, but your message does change that assumption, and that is a good
thing. I would like to work on Swazoo as a group project, even if that only
means sharing ideas and a fork and merge process as you have suggested.

At this point, it seems that the fork-and-merge form of development is the only thing that's really practical.  Of course, I'd love to have a unified base at some point.  Between now and (the hypothetical) then, let's just make a point of building things for portability.  If you see me doing things that really hinder a port (e.g. #interruptWith:), yell.

Thanks for the feedback, and keep up the great work.  This is cool!


Ken

Hi Ken,

> Let me know what you think of the SwazooStream framework -- would it
> work for Dolphin?  I understand that your HTTPConnection class still
> might have to differ a bit, so maybe the tradeoffs wouldn't be worth it.

Definately, we already have a SwazooSocketStream which has almost the same
protocol as SwazooStream. It doesnt have the *Byte* methods, but these
methods would be a good addition as it will eliminate some of the String to
ByteArray conversions that go on.

I also added a subclass; SwazooSocketLoggingStream. This has a second stream
which is replaced by the HTTPConnection each message read. The class writes
everything to the second stream, that is written to, or read from the
socket, before super sending the method to SwazooSocketStream. We dont use
this stream on dolphinharbor, but it can be handy in tracking down problems,
and being as close to the "wire" as possible, it doesnt complicate the
message/connection classes.


> The #interruptWith: code is nasty; I don't like it at all.  The only
> reason it's there is to wait for a short amount of time before reading
> the next message.  If no new message arrives, we want to close the stream.
>
> It seems like this could be done with a lower-level timeout (like
> readWait in VisualWorks) or something similar.  As long as data keeps
> coming, we want to keep the socket open.  Otherwise, we want to close
> 'em down.

Yes, this is a grey area. I came across this paper;
http://www.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt

which discusses some alternatives. I just looked at the code and we are
shutting down connections that are still reading the request after 70
seconds. By this time IE will have shut an inactive connection, and as I
understand it, it is better to let the client close the connection, as they
then go through the TIME_WAIT state. Also from the server perspective, if
the server is not under load, it is cleaner to resuse the connection, rather
than set up a new process.

> This problem may have gone away with the new VW SwazooSocket; it seems
> to dutifully raise an error when the stream is closed.

Yes we get this as well via a RemoteSocketClosed exception. The Dolphin
sockets use a windows message model, and this comes through as an event,
which the Dolphin code then uses to signal an exception.

I still think an "interuptWith"/"connectionKiller" is needed to stop badly
behaved opening multiple connections and not closing them.

> Personally, I'd love it if the SiteRegistry went away entirely.  But I
> haven't figured out something better yet (short of storing Resources in
> GemStone).

I would love to work with GemStone, but it is a bit out of my price range :)

The configuration script format has growen on me. I find it to be very
flexible, and I have been able to use it for everything I have needed. The
SiteScript class is handy, because I can use the same scripts locally and on
dolphinharbor. I also added a quick user interface, which displays the
default SiteScripts as menu choices, and makes it easy to load/reload a
site.

The other change I made to the configuration process was that a
CompositeResource sends its children an #onResourceCreated event, where I
can do further initialization.

> >-Added a chunked encoding response
> >
> I'll have a look at this, too.

This is proof of concept code at the moment, but I have an idea for a uses
on dolphinharbor. I think this kind of transfer will become more important,
so it could be a good idea to get a standard Swazoo protocol for it. What I
did was; a ChunkedTransferResource, answers a ChunkedResponse with itself
set into an instance variable. When the response is sent the #writeEntityTo:
message, it then sends the message #chunkTransfer: to the resource, with
itself as the argument. The resource then treats the response as a normal
stream; the response's stream methods wrapping each argument as a "chunk".

> I've been holding off on this one because it didn't seem necessary,
> philosophically.  But then I found myself passing the request object all
> over the place while assembling a reply, just so I could have access to
> the URI.

The main reason I added it was that each WebService needs to publish a WSDL
file which describes the types/operations in the service. I need to generate
namespaces for the types, and the specification does not allow relative
uris, so I use relative uris until the webService is created, and then use
the url of the webService to make the relative uris absolute. Being xml
namespaces, they dont have to resolve to anything, but they do need to be
unique.

Since I added it, I have used it lots :) It has been very handy in being
able to use the same configuration information for my local work and
dolphinharbor. For example dolphinharbor uses a RedirectionResource from /
to /qad/welcome/index.html. I have this resource configured using the
relative uris, but it will use its ability to access its parent, to answer
absolute urls.

This may break the Site alias model, but I did not understand what that is
used for, and I have not needed it as yet. What are Site alais's used for?

> At this point, it seems that the fork-and-merge form of development is
> the only thing that's really practical.  Of course, I'd love to have a
> unified base at some point.  Between now and (the hypothetical) then,
> let's just make a point of building things for portability.  If you see
> me doing things that really hinder a port (e.g. #interruptWith:), yell.

I think you will probably have a harder time getting dolphin working on
Linux, than I will learning how to get the browser I want up in VW :) I have
been meaning to spend some time in VW, so I will make it a project to try
and port the code/tests back into VW form, so you can at least get a look at
them running. I wont be able to do this in the next couple of weeks, but I
am planning on working on Swazoo after that.

Steve