SiteRegistry renamed, SUnit tests

Hi,

Here is a copy of my conversation with Joe. Comments please
:)

Joseph Bacanskas wrote:

Off the top of my head, I think that a class' name should
describe
what it is/does.  The name SiteRegistry, describes what the
class is/does as it interacts with the rest of Swazoo.  It
seems that naming the Class Swazoo points new users to a
possible starting place when exploring the application.

I think that the tests served several functions; one of
which was to document how the system works.  The tests show
how each function needed for the app are
performed.  Without them, it's a lot harder to understand
how all of the pieces work together, and impossible to know
when a developer is finished.

I know I'm really pounding on the idea of SUnit tests, but
they have several functions which are now no longer being
performed.  We want the application to be
more robust and more understandable every time  we change
the code.  We want the tests to document how the system is
used.  We want the tests to show people who are trying to
port Swazoo to other Smalltalks that when their port passes
the tests, they are done.

BTW - I'm getting on a plane to GemStone soon, and won't see
this
account until Friday.
Please reply to [hidden email].

Also, should we move this to the mailing list so that others
can
participate?

Thanks again for working so hard on this project; I truely
appreciate you efforts.  You are
moving things forward at a great pace!

Janko Mivsek wrote:
SiteRegistry to will certainly miss them?  By my last made Swazoo
> > > > > and
> > > > >   restart with Swazoo stop/start. Only previously
started
> > > > > Sites
> > > > >   will start again
> > > > > - Automatic restart of Sites after snapshot or image
restart be
> > > > > easier to
> > > > >   understand/self documented
> > > > >
> > > > > Please do a test drive of this version and tell me what
you
> > > > > think :) I need quick responses because I'm starting to
make
> > > > > a Wiki integration.
> > > > >
> > > > > Regards
> > > > > Janko
> > > > >

--
Janko Mivsek
Systems Architect
EraNova d.o.o.
Pod hribom 55
SI-1000 Ljubljana
Slovenia
phone: +386 1 5816 232
fax: +386 1 5816 231
[hidden email]
www.eranova.si

Janko,
I can hardly contribute anything lately to the effort, but still, here are my 2 p...

Joseph has outlined his reservations very eloquently, and I agree with them all very strongly.

regarding renaming SiteRegistry to Swazoo.
you can still achieve your goal by using the Facade pattern. basically having the Swazoo class "fronting" for SiteRegistry and others.
since you know GemStone, look at the class System as an example of using Facade.

regards, and see you @ oopsla,
Benny

Hi,

Benny & Wendy Sadeh wrote:
>
> Janko,
> I can hardly contribute anything lately to the effort, but still, here are my 2 p...
>
> Joseph has outlined his reservations very eloquently, and I agree with them all very strongly.

My motivation to do rename was bad experiences with putting
WikiWorks in action. Me and Peter van Rooijen tried to test
WikiWorks and to study it for possible integration with
Swazoo. But, how to start this thing? Ok, let we explore
Wiki class first. Nothing. Where the heck are some examples?
After looking around all classes we discovered examples in
NetworkServer. This is IMHO completely inappropriate place
for examples.

Therefore I decided to care a lot in Swazoo to be as easy as
possible to use. Just load the parcel and call one obvious
method and your server is running! I think that we must care
a lot of usability issues here if we like Swazoo to become a
standard, well known server for a whole Smalltalk community.

That's also a reason that I propose to put SUnit test in a
separate parcel. For SUnit tests you need the SUnit parcels
and it is quite difficult to find it from first place. I had
this experience when I had a Swazoo-Aida presentation at
Gemstone. I had a Swazoo parcel with me, but not SUnit.
Thanks to Joe help providing me a SUnit I was able to make a
presentation. Therefore, Its better to have only one Swazoo
parcel, which can work without any more prerequisites from
plain VW image. If you like, you can load tests parcel
anytime later. Maybe we can tweak parcel loading to test if
SUnit is presented and load test parcel automatically then.


> regarding renaming SiteRegistry to Swazoo.
> you can still achieve your goal by using the Facade pattern. basically having the Swazoo class "fronting" for SiteRegistry and others.
> since you know GemStone, look at the class System as an example of using Facade.

Ok, let we do this way:

        Swazoo
          sites <SiteRegistry>
          servers <OrderedCollection>

        SiteRegistry
          sites

SiteRegistry will be responsible for site registration,
finding by hostname, configuring from sites.cfg etc.

Swazoo responsibilities will be start/stop of sites,
autostart after image startup and finding/starting the right
HTTPServers for each Site. All site specific behavior will
be delegated through Swazoo to SiteRegistry. Is that ok?


 
--
Janko Mivsek
Systems Architect
EraNova d.o.o.
Pod hribom 55
SI-1000 Ljubljana
Slovenia
phone: +386 1 5816 232
fax: +386 1 5816 231
[hidden email]
www.eranova.si

Hi:

Janko - Two quick observations, (I just got home and have over 100 emails!!!) firstly: The examples in WikiWorks are in HTTPServer, not the
abstact superclass, NetworkServer; secondly: in WikiWorks had tests the way Swazoo does, you would not have experienced any problems,
because you would have had example code.


Janko's experience points up what I was saying earlier, because Swazoo has tests, people unfamiliar with the code can get it ported and
running.  WikiWorks, without tests, gave you great difficulty.  The other problem you will have with wiki is that it will be very difficult
for you to know "when you are done" whereas, Jerry Bell knew he had a successful Dolphin port of Swazoo as soon as all of the tests ran.

I agree that the packaging of Swazoo could be improved.  I think that we packaged it the way we did because it was "the simplest thing that
could possibly work" at the time.  Clearly, that is no longer true.  I propose the following:

1. Put the tests in a separate parcel, which has SUnit as a prereq.
2. Janko, since you made the big changes, you should bring the tests up to date with your current code, so that they all run successfully
and that and new behavior is covered by new tests
3. We look at the result and decide whether we should write some short instructions.  For an example of what I mean, see the WikiWorksGS
instructions.

As to the class rename, I already made my comments and will abide by whatever the team decides.

Janko Mivsek wrote:

--
Thanks!!

Joseph Bacanskas [|]

--- I use Smalltalk.  My amp goes to eleven.

This is just a general discussion of an application type which I would like
to see built on top of Swazoo.   I hope that this is an appropriate topic
for the Swazoo list.

There is a product available from Xerox which I am very interested in - it's
called Docushare, and it's basically a web based document repository.  It
uses webdav to provide a web based, multi-user, versioning-capable document
repository which can hold any type of document.  It also has some very nice
windows based clients, including one that works with ODMA in programs such
as WordPerfect (so file..open goes directly to the document repository,
acting just as if the file were on a normal drive or share).  It also does
handy things like automatically creating thumbnails for image files stored
in the repository.

 I like the idea - it's a nice, web based interface, so you can access it
from anywhere over the net.  And, it holds real documents of any type so
you're not confined to HTML etc.  I'm interested in using this for both
simple shared document repositories, and also as a sort of ultimately
flexible object database (when Smalltalk objects are some of the document
types being stored).  Actually, I guess it is sort of a web interface onto
an object database which supports versioning?

 From what I can tell, the Xerox product is basically some python code and a
licensed search engine.  It's expensive as well- $5000 buys you a 50 user
repository.  Not 50 concurrent users, 50 users, period.  I've played with
the evaluation version, but we don't have the real thing- none of our
internal needs are great enough to justify the $5k at this point, and
besides it just doesn't seem to be complicated enough to be worth that much.

I envision something like this: Swazoo with extensions handles web server
stuff plus other webdav-type chores.  A pluggable object database on the
back end handles storing the actual stuff.  All collections and resources
are actually Smalltalk objects - they are either real Smalltalk objects
which conform to a specific protocol to support storage in the repository
directly, or they are wrapper objects around external documents, which
handle things like storing properties, versioning, etc.  The real Smalltalk
objects can be really interesting, instead of merely storing metadata
property strings and returning them when asked, a Smalltalk object stored in
this system might instead perform a method and return the result, so they
can be active instead of static.  Since I don't particularly want to write a
search engine, I would probably use an existing search engine to search the
server (true Smalltalk objects would be able to spit out a representation of
themselves, perhaps in XML, which the indexer could use).

I have downloaded some of the webdav documentation, and have begun reading
it.  I can already tell that it is more complicated than I expected, I will
probably try to aim for a subset at first with no support for things such as
concurrent check-out, merging, or multiple version paths.   Perhaps one day,
the native Smalltalk objects would be smart enough to handle these things by
themselves.   But, to be truly accepted, it would probably need to conform
to the webdav specs eventually.

I already have many questions for the group: is anyone else interested in
this?   How is this like/unlike the Wiki and other projects which are in
progress?  A useful system would require security via ACLs, and this seems
to be a requirement for many other kinds of  apps as well- is there some
sort of generic security system we can add to Swazoo?  What other parts of
the system would be useful for other types of applications, and therefore
should perhaps be broken out as separate projects?  Or perhaps this
functionality should become part of the base Swazoo system?

Thanks for your time,

Jerry Bell
[hidden email]

Jerry Bell wrote:

> This is just a general discussion of an application type which I would like
> to see built on top of Swazoo.   I hope that this is an appropriate topic
> for the Swazoo list.

It's a great topic for the list.  This is the place to discuss this kind of
thing.  I'm just sorry I haven't had time to respond until now.

Sounds pretty cool.  I presume there is also some form of authentication -- so
that only certain people have access to certain documents, certain people can
make modifications, etc.

>  From what I can tell, the Xerox product is basically some python code and a
> licensed search engine.  It's expensive as well- $5000 buys you a 50 user
> repository.  Not 50 concurrent users, 50 users, period.  I've played with
> the evaluation version, but we don't have the real thing- none of our
> internal needs are great enough to justify the $5k at this point, and
> besides it just doesn't seem to be complicated enough to be worth that much.

Ouch.  Maybe I've been playing with Linux for too long, because I'm always a
little shocked at software licensing costs.

Just as a tip, make sure that you start with the simplest thing that could
possibly work.  Swazoo can already pass data back and forth reliably using GETs
and POSTs.  Then, on the back end, you keep some sort of collection of objects
that contain the documents.  You can even just take the MimeObjects that you
receive in a multipart/form-data POST and wrap them.  Then, when it comes time
to serve the object back, you already have its MIME type (contentType) and its
data.  That makes it a snap to serve them back.

On Signal Ground (which has gotten a bunch of hits lately, thanks to Janko's
presentation and John McIntosh's report -- thanks guys!), I do exactly this.
Images can be uploaded via a POST (where the form's enctype is
multipart/form-data), and then they're stored as-is in an Article, which gets
persisted to disk.  That means that I don't have to care if the image is a
PNG (our preference), GIF, or JPEG.  As long as the browser knows (and
sends) the correct mime type, Signal Ground will spit it right back.

If you're versioning something like word documents, searching could be a real
problem.  It's tough to rip apart that format.  I'd hold off on the searching
until the store/retrieve versions works well.

> I have downloaded some of the webdav documentation, and have begun reading
> it.  I can already tell that it is more complicated than I expected, I will
> probably try to aim for a subset at first with no support for things such as
> concurrent check-out, merging, or multiple version paths.   Perhaps one day,
> the native Smalltalk objects would be smart enough to handle these things by
> themselves.   But, to be truly accepted, it would probably need to conform
> to the webdav specs eventually.

I don't know a thing about webdav, but I'd say that this is probably a
second-phase, as well.  Would it be that hard to spec out a protocol for this?
Something like

GET /resource/folder/document+name : gets the current version
GET /resource/folder/document+name?version=5 : gets version 5
GET /resource/folder/document+name?view=showVersions : shows available
versions, with summaries and links to fetch individual versions
POST /resource/folder/document+name : done from a form with input fields for
the file itself and a summary string.  This puts a new version into the
repository

> I already have many questions for the group: is anyone else interested in
> this?

Sounds interesting to me.  :)  Don't know how much time I'll have for it --
same old excuses -- but it sounds like a neat little application.

>   How is this like/unlike the Wiki and other projects which are in
> progress?

Wikis (esp. WikiWorks) are similar in that they keep versions of the pages, but
the content (currently) is text only.  There's no file versioning.

> A useful system would require security via ACLs, and this seems
> to be a requirement for many other kinds of  apps as well- is there some
> sort of generic security system we can add to Swazoo?

This was part of our initial vision, but it hasn't been talked about much
lately.  In effect, our tree of resources (the fundamental way that things are
organized on a Swazoo site) should have mechanisms for dealing with this.  It'd
look like this:

<Site>
 <SiteIdentifier ip: '127.0.0.1' port: 8080 host: 'localhost'>
 <CompositeResource stringUriPattern: '/'>
  <ReadOnlyDocumentResource stringUriPattern: 'documents'>
  <ACLResource stringUriPattern: 'edit' aclFile: 'docs.acl'>
   <FullControlDocumentResource stringUriPattern: 'documents'>
  </CompositeResource>
 </CompositeResource>
</Site>

The ACLResource makes sure that anybody who doesn't have permission to access
/edit/documents gets a "permission denied".  It filters access to the resources
below it.

This isn't very fine-grained, but a more complex version of the ACLResource
should be able to handle it on a per-document basis.  Then, you wouldn't need
the ReadOnly guy .. the ACLResource lets people read some, not read others,
etc.  This would be more complete, but the above is nice and simple.

> What other parts of
> the system would be useful for other types of applications, and therefore
> should perhaps be broken out as separate projects?  Or perhaps this
> functionality should become part of the base Swazoo system?

You'll probably see these as you go along.  I have a similar vision -- I've
seen that various aspects of the Wiki could be very useful when building other
Swazoo resources.  So I sat down and tried to write a Wiki toolkit, and I soon
got bogged down in details.  "What if I'll want to do this later?  What about
this possible situation?"  I broke the "You aren't gonna need it" principle all
over the place.  What I should have done is worked on porting the core of
WikiWorks instead, and then refactored as I needed it.  I want to return to
this project as soon as I can, and that's the approach I'll probably take.

Long-winded answer, but I'd basically suggest the same for you.  Build it, then
chop it up.

> Thanks for your time,

Thanks for yours.  :)


Ken Treis
[hidden email]