New SqueakMap on the air... and we got problems Houston!

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

New SqueakMap on the air... and we got problems Houston!

Göran Krampe
Howdy!

Ok, now its 2:12 AM and I gotta get up in about 4.5 hours. I have now
upgraded SM and AFAICT it seems to be working ok with 3.8 and 3.9
clients. It still (forgot to add code for that - perhaps we can still do
it) fails if you already have an old map in the "sm" dir and fires up a
fresh image and try to open the loader. But you can then either delete
the map-files in the sm dir (no need to nuke the whole dir and thus the
cache) and try again or simply execute manually:

        SMSqueakMap bootStrap

That is meant to install the latest SM into the image. :) Now, obviously
with a 3.6-7 image we stumble on ByteSymbol, ByteString when loading the
ImageSegment - this is why 3.6, 3.7 Squeaks can't cope with the new map.

Sigh.

So... anyone got a bright idea or should we just suck it up and declare
SM to be 3.8+ country? Time to move to a Magma solution? Though it still
seems awkward to demand Magma client in all SM enabled images. :) Or
perhaps we could just use a good ole SmartRefStream? I dunno, eyes are
dried up by now.

Over and out.

regards, Göran

PS. Feel free to post about more stuff to fix - now I have all the "hows
and whats" fresh in my head so making more tweaks is quite ok. And no, I
haven't bothered with SMLoader yet - but that is easy compared to the
rest.

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Andreas.Raab
[hidden email] wrote:
> So... anyone got a bright idea or should we just suck it up and declare
> SM to be 3.8+ country? Time to move to a Magma solution? Though it still
> seems awkward to demand Magma client in all SM enabled images. :) Or
> perhaps we could just use a good ole SmartRefStream? I dunno, eyes are
> dried up by now.

How about some sort of well-formed output, like (dare I use the word?)
XML? I've attached a quick hack (well, okay it's been about an hour of
work ;-) that allows you to write a complete map w/ Yaxo (no read back
yet). It's just a proof of concept right now but the results (about
10secs to write the whole thing in about 900k) are quite encouraging
(the current map takes about 500k but this version wastes lots of memory
-and time- for writing out duplicate information that could easily be
avoided).

Cheers,
   - Andreas

PS. I'm not sure if this will run out of the box with 3.8 or 3.9 since I
used a Croquet image which has a few extra things in it. Let me know if
there are any issues.

PPS. If you're interested in a solution like this I should be able to
spend another hour or two on the read back side as well.



SMXMLOutput.1.cs (24K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

stéphane ducasse-2
I have the impression that a human "readable" format as XML can be a  
good way to do that :)

On 4 avr. 06, at 05:25, Andreas Raab wrote:

> [hidden email] wrote:
>> So... anyone got a bright idea or should we just suck it up and  
>> declare
>> SM to be 3.8+ country? Time to move to a Magma solution? Though it  
>> still
>> seems awkward to demand Magma client in all SM enabled images. :) Or
>> perhaps we could just use a good ole SmartRefStream? I dunno, eyes  
>> are
>> dried up by now.
>
> How about some sort of well-formed output, like (dare I use the  
> word?) XML? I've attached a quick hack (well, okay it's been about  
> an hour of work ;-) that allows you to write a complete map w/ Yaxo  
> (no read back yet). It's just a proof of concept right now but the  
> results (about 10secs to write the whole thing in about 900k) are  
> quite encouraging (the current map takes about 500k but this  
> version wastes lots of memory -and time- for writing out duplicate  
> information that could easily be avoided).
>
> Cheers,
>   - Andreas
>
> PS. I'm not sure if this will run out of the box with 3.8 or 3.9  
> since I used a Croquet image which has a few extra things in it.  
> Let me know if there are any issues.
>
> PPS. If you're interested in a solution like this I should be able  
> to spend another hour or two on the read back side as well.
> <SMXMLOutput.1.cs>
>


Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Philippe Marschall
In reply to this post by Andreas.Raab
2006/4/4, Andreas Raab <[hidden email]>:

> [hidden email] wrote:
> > So... anyone got a bright idea or should we just suck it up and declare
> > SM to be 3.8+ country? Time to move to a Magma solution? Though it still
> > seems awkward to demand Magma client in all SM enabled images. :) Or
> > perhaps we could just use a good ole SmartRefStream? I dunno, eyes are
> > dried up by now.
>
> How about some sort of well-formed output, like (dare I use the word?)
> XML? I've attached a quick hack (well, okay it's been about an hour of
> work ;-) that allows you to write a complete map w/ Yaxo (no read back
> yet). It's just a proof of concept right now but the results (about
> 10secs to write the whole thing in about 900k) are quite encouraging
> (the current map takes about 500k but this version wastes lots of memory
> -and time- for writing out duplicate information that could easily be
> avoided).
>
> Cheers,
>    - Andreas
>
> PS. I'm not sure if this will run out of the box with 3.8 or 3.9 since I
> used a Croquet image which has a few extra things in it. Let me know if
> there are any issues.
>
> PPS. If you're interested in a solution like this I should be able to
> spend another hour or two on the read back side as well.

Well, this would be a good time, to fix open Yaxo serialization bugs:
http://bugs.impara.de/view.php?id=3082

Cheers
Philippe

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Göran Krampe
In reply to this post by Andreas.Raab
Hi Andreas!

Andreas Raab <[hidden email]> wrote:
> [hidden email] wrote:
> > So... anyone got a bright idea or should we just suck it up and declare
> > SM to be 3.8+ country? Time to move to a Magma solution? Though it still
> > seems awkward to demand Magma client in all SM enabled images. :) Or
> > perhaps we could just use a good ole SmartRefStream? I dunno, eyes are
> > dried up by now.
>
> How about some sort of well-formed output, like (dare I use the word?)
> XML?

Hehe, well, I did consider that (a year or two back) but avoided it
because I didn't want to take on the "luggage" that serialization code
is (to be maintained for shape changes etc when the model evolves). Back
in the beginning of SM I used Smalltalk as format and it was cumbersome
to maintain it - but of course, then I also used an incremental model
which made it even more complex.

> I've attached a quick hack (well, okay it's been about an hour of
> work ;-)

Don't you have better things to do? Like getting Crocodile out the door?
;) I can't help wondering why you ended up coding on this? Just curious.
:)

> that allows you to write a complete map w/ Yaxo (no read back
> yet). It's just a proof of concept right now but the results (about
> 10secs to write the whole thing in about 900k) are quite encouraging
> (the current map takes about 500k but this version wastes lots of memory
> -and time- for writing out duplicate information that could easily be
> avoided).
>
> Cheers,
>    - Andreas

Mmmm, let me say that I would like to hear more views on this issue
before deciding for the future. As I said XML has been on my radar but
my main idea had actually been to use SmartRefStreams and not just a
single one but instead "split" the model into accounts and enable them
to be stored on multiple servers (along the lines previously discussed
enabling personal, department, company and global wide maps etc).

But on the other hand - SmartRefStream and friends is probably just as
sensitive to these things as ImageSegment is. So...

...yes, we probably need to go the route of an explicit format unbound
to the domain classes and yes, then XML is "as fine as any".

> PS. I'm not sure if this will run out of the box with 3.8 or 3.9 since I
> used a Croquet image which has a few extra things in it. Let me know if
> there are any issues.

Sure.

> PPS. If you're interested in a solution like this I should be able to
> spend another hour or two on the read back side as well.

Let us see what others say in the community but yes, after typing this
email I am probably leaning towards this route - even though it "hurts"
going back to coding these things - Magma or ImageSegments etc are so
darn nice (when they work). ;)

And if I/we are going to do this I think we/I really should take a stab
at the "distributed approach" at the same time. If anyone is interested
in hearing a recap on that, just say "recap".

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Andreas.Raab
[hidden email] wrote:
>> How about some sort of well-formed output, like (dare I use the word?)
>> XML?
>
> Hehe, well, I did consider that (a year or two back) but avoided it
> because I didn't want to take on the "luggage" that serialization code
> is (to be maintained for shape changes etc when the model evolves). Back
> in the beginning of SM I used Smalltalk as format and it was cumbersome
> to maintain it - but of course, then I also used an incremental model
> which made it even more complex.

Interesting arguments. In my experience, maintaining shape changes with
image segments gets very hard very quickly. Utilizing well-defined
external formats has (again, in my experience) proven to be much more
robust when it comes to changes. Updating incrementally is certainly
something that's a bit non-trivial, but if you look at the data records
generated I think it'd be straightforward to merge them incrementally
(discounting deletions but there are ways to deal with that, too).

>> I've attached a quick hack (well, okay it's been about an hour of
>> work ;-)
>
> Don't you have better things to do? Like getting Crocodile out the door?
> ;) I can't help wondering why you ended up coding on this? Just curious.
> :)

Oh, I just needed an hour of clean, mindless fun to relax from some of
the harder things I'm working on ;-) Usually I play a game of mine
sweeper but writing some XML exporter code works just as well. And
besides, I *do* think this is an important issue and I don't think SM
should be 3.8+ only for a reason as simple to solve as this.

> Mmmm, let me say that I would like to hear more views on this issue
> before deciding for the future. As I said XML has been on my radar but
> my main idea had actually been to use SmartRefStreams and not just a
> single one but instead "split" the model into accounts and enable them
> to be stored on multiple servers (along the lines previously discussed
> enabling personal, department, company and global wide maps etc).
>
> But on the other hand - SmartRefStream and friends is probably just as
> sensitive to these things as ImageSegment is. So...

SmartRefStream is less sensitive to shape changes than image segments
but it would be affected by the Byte vs. WideString problem in the same
way (I think; with some retrofitting of the earlier Squeak versions you
also may be able to change it to read ByteString and do something
sensible about WideString but that seems a harder problem than just
using a well-defined format).

>> PPS. If you're interested in a solution like this I should be able to
>> spend another hour or two on the read back side as well.
>
> Let us see what others say in the community but yes, after typing this
> email I am probably leaning towards this route - even though it "hurts"
> going back to coding these things - Magma or ImageSegments etc are so
> darn nice (when they work). ;)

Sure. And I have no problem at all throwing that solution away. But I
thought it'd be interesting to see what the speed/space tradeoffs are
since the classic discussion goes along the lines of "oh, but it's so
large and so slow" and I wanted to be able to see if that's true (it is
not - with the three most obvious optimization applied space goes down
to 600k and speed to 3secs and that is roughly on par with the current
format).

Cheers,
   - Andreas


Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Göran Krampe
Hi Andreas!

Andreas Raab <[hidden email]> wrote:

> [hidden email] wrote:
> >> How about some sort of well-formed output, like (dare I use the word?)
> >> XML?
> >
> > Hehe, well, I did consider that (a year or two back) but avoided it
> > because I didn't want to take on the "luggage" that serialization code
> > is (to be maintained for shape changes etc when the model evolves). Back
> > in the beginning of SM I used Smalltalk as format and it was cumbersome
> > to maintain it - but of course, then I also used an incremental model
> > which made it even more complex.
>
> Interesting arguments. In my experience, maintaining shape changes with
> image segments gets very hard very quickly.

Yes, I agree - in *general*. But in the case of SM it has been different
because I didn't have the issue of having to be able read *old*
ImageSegments into new code. I only have the issue of being able to read
ImageSegments into the same code that produced it, *but* living in
different Squeak versions.

But if I use my own serialization format then I need to maintain that
code when the model evolves. Using ImageSegments I haven't had to do
that, since there is no code to maintain.

> Utilizing well-defined
> external formats has (again, in my experience) proven to be much more
> robust when it comes to changes.

Yes, again I agree *in general*, especially if you take care and design
the format so that it doesn't reflect the actual
implementation/representation you use in your code. But in this case,
given how it works, it doesn't matter. Until now of course when the
Squeak versions have diverged so much that the leaf base classes my
domain model uses have changed.

> Updating incrementally is certainly
> something that's a bit non-trivial, but if you look at the data records
> generated I think it'd be straightforward to merge them incrementally
> (discounting deletions but there are ways to deal with that, too).

Well, with the plan to actually store SMAccounts separately we will not
need to go the incremental route to get good performance (bandwidth
performance that is - there are quite a few people on modems that don't
like the ~1Mb download penalty when updating map) - when updating we
will only need to download those accounts that have changed their
content - and it will at a given point in time be very few.

> >> I've attached a quick hack (well, okay it's been about an hour of
> >> work ;-)
> >
> > Don't you have better things to do? Like getting Crocodile out the door?
> > ;) I can't help wondering why you ended up coding on this? Just curious.
> > :)
>
> Oh, I just needed an hour of clean, mindless fun to relax from some of
> the harder things I'm working on ;-) Usually I play a game of mine
> sweeper but writing some XML exporter code works just as well. And
> besides, I *do* think this is an important issue and I don't think SM
> should be 3.8+ only for a reason as simple to solve as this.

I agree.

> > Mmmm, let me say that I would like to hear more views on this issue
> > before deciding for the future. As I said XML has been on my radar but
> > my main idea had actually been to use SmartRefStreams and not just a
> > single one but instead "split" the model into accounts and enable them
> > to be stored on multiple servers (along the lines previously discussed
> > enabling personal, department, company and global wide maps etc).
> >
> > But on the other hand - SmartRefStream and friends is probably just as
> > sensitive to these things as ImageSegment is. So...
>
> SmartRefStream is less sensitive to shape changes than image segments
> but it would be affected by the Byte vs. WideString problem in the same
> way (I think; with some retrofitting of the earlier Squeak versions you
> also may be able to change it to read ByteString and do something
> sensible about WideString but that seems a harder problem than just
> using a well-defined format).

Yes, a harder problem when the base classes evolve - but it would still
be less pain to maintain otherwise. What I mean by that is that the
manually written XML saving/loading code needs to be maintained when the
SM model evolves. But using a non-manual serialization scheme would not
need maintenance for that.

But as you say, when the base classes evolve like this then a manual
approach is simpler to adapt. I wonder how the various automatic XML
seralizers we have could cope with this situation - if they have hooks
and/or if they use "reasonable code" to instantiate base classes then we
might be able to use one of those. This one comes to mind:

        http://map.squeak.org/packagebyname/sixx

Hmmm, well, no - not very goood out of the box at least. :) It looks
kinda neat but... it generated 19Mb of XML for the map, which compressed
to 900kb. ;) And no, it is too low level and verbose at the same time
for my taste.

> >> PPS. If you're interested in a solution like this I should be able to
> >> spend another hour or two on the read back side as well.
> >
> > Let us see what others say in the community but yes, after typing this
> > email I am probably leaning towards this route - even though it "hurts"
> > going back to coding these things - Magma or ImageSegments etc are so
> > darn nice (when they work). ;)
>
> Sure. And I have no problem at all throwing that solution away. But I
> thought it'd be interesting to see what the speed/space tradeoffs are
> since the classic discussion goes along the lines of "oh, but it's so
> large and so slow"

No, I have never considered "large and slow" to be a problem with XML
when *I* am in control of the XML. :) But as SIXX shows it *can* be
large and slow. I have built quite advanced XML export/imports in Java
(for saving loading newspaper pages in a DTP program similar to QXPress
written in Java) so I know how to do it.

> and I wanted to be able to see if that's true (it is
> not - with the three most obvious optimization applied space goes down
> to 600k and speed to 3secs and that is roughly on par with the current
> format).

Yes, good. Ok, send me your latest code and I will start working on
this. Having peeked at your code I might also want to generalize this a
bit and make it slightly more high level too (It looks to me there is a
bit of code redundancy going on when it comes to serailizing the
collections).
 
> Cheers,
>    - Andreas

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Avi  Bryant

On Apr 5, 2006, at 12:52 AM, [hidden email] wrote:
>
> Yes, I agree - in *general*. But in the case of SM it has been  
> different
> because I didn't have the issue of having to be able read *old*
> ImageSegments into new code. I only have the issue of being able to  
> read
> ImageSegments into the same code that produced it, *but* living in
> different Squeak versions.

I would say the most important property is being able to read *new*  
versions of the map into *old* versions of the code.  That way people  
aren't suddenly forced to upgrade all their images as soon as a new  
version gets released.  I work with a wide range of Squeak versions  
and SqueakMap is a common annoyance because of that.

That would also save you from having to worry about new versions of  
the SM code loading into old versions of Squeak - as long as people  
can keep using their old version of SM with new maps there shouldn't  
be too much grumbling.

Avi

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Göran Krampe
Hi!

Avi Bryant <[hidden email]> wrote:

>
> On Apr 5, 2006, at 12:52 AM, [hidden email] wrote:
> >
> > Yes, I agree - in *general*. But in the case of SM it has been  
> > different
> > because I didn't have the issue of having to be able read *old*
> > ImageSegments into new code. I only have the issue of being able to  
> > read
> > ImageSegments into the same code that produced it, *but* living in
> > different Squeak versions.
>
> I would say the most important property is being able to read *new*  
> versions of the map into *old* versions of the code.  That way people  
> aren't suddenly forced to upgrade all their images as soon as a new  
> version gets released.  I work with a wide range of Squeak versions  
> and SqueakMap is a common annoyance because of that.
>
> That would also save you from having to worry about new versions of  
> the SM code loading into old versions of Squeak - as long as people  
> can keep using their old version of SM with new maps there shouldn't  
> be too much grumbling.

Eh? Ok, this is how it works:

- SMLoader can be upgraded at leisure (mostly). Sure, it depends on
SMBase but not much so it probably "works":
- SMServer is the web UI for the server. Can evolve independently of
everything else.
- SMBase is the domain model and is used both locally and on the server.
If the server is upgraded it *might* work fine with loading such a map
into an old SMBase, but only if there haven't been any larger changes.

When the client connects and tries to fetch a new map it first asks if
the "protocol version" is ok. This means that if the server has a new
"protocol version" (which I only bump when I feel we have to) then it
will ask to upgrade the local SMBase. Otherwise it will not.

Now... what do you *mean* with "being able to read *new* versions of the
map into *old* versions of the code"?

IMHO that is... not possible. :) Unless we are talking of simple
behavior changes and no structural changes and that those behavioral
changes "aren't important".
 
> Avi

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Andreas.Raab
[hidden email] wrote:
> Now... what do you *mean* with "being able to read *new* versions of the
> map into *old* versions of the code"?

It simply means that older versions of the client are capable of dealing
with newer data sets coming from the server, so that the clients don't
need to be updated in sync with the server. There are many good reasons
for this; the most important one being that people like to be able to
decide when they want to upgrade instead of being forced to.

> IMHO that is... not possible. :) Unless we are talking of simple
> behavior changes and no structural changes and that those behavioral
> changes "aren't important".

Oh, it's perfectly possible. All it means is that the server sends
enough information so that older clients know how to make sense of it
(within reason; unsupported features are still unsupported features) and
that the clients *expect* to see information that they don't know how to
make sense of (e.g., they need to be designed to be robust in the face
of future changes). There is nothing magical about it.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Masashi UMEZAWA-2
In reply to this post by Göran Krampe
Hi,

> But as you say, when the base classes evolve like this then a manual
> approach is simpler to adapt. I wonder how the various automatic XML
> seralizers we have could cope with this situation - if they have hooks
> and/or if they use "reasonable code" to instantiate base classes then we
> might be able to use one of those. This one comes to mind:
>
> http://map.squeak.org/packagebyname/sixx
>
> Hmmm, well, no - not very goood out of the box at least. :) It looks
> kinda neat but... it generated 19Mb of XML for the map, which compressed
> to 900kb. ;) And no, it is too low level and verbose at the same time
> for my taste.
>

SIXX is designed to be generic, so the generated XML is sometimes too
verbose by default.
But you can also override a few methods for generating customized XML.
You can specify which fields will be stored, and even embed your
compact XML in SIXX.

In 'SIXX-Examples' category, there are some examples for custom serialization.

As for class migration, I have some plan for supporting class mapper in
SixxReadStream. But it will be not done in near future, I admit...

--
[:masashi | ^umezawa]

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Göran Krampe
In reply to this post by Andreas.Raab
Hi!

Andreas Raab <[hidden email]> wrote:
> [hidden email] wrote:
> > Now... what do you *mean* with "being able to read *new* versions of the
> > map into *old* versions of the code"?
>
> It simply means that older versions of the client are capable of dealing
> with newer data sets coming from the server, so that the clients don't
> need to be updated in sync with the server. There are many good reasons
> for this; the most important one being that people like to be able to
> decide when they want to upgrade instead of being forced to.

Well, I for one don't want to have multiple versions of SM running out
there trying to make sense of the model coming from the latest version
running on the server. It sounds way too risky for my taste.

> > IMHO that is... not possible. :) Unless we are talking of simple
> > behavior changes and no structural changes and that those behavioral
> > changes "aren't important".
>
> Oh, it's perfectly possible. All it means is that the server sends
> enough information so that older clients know how to make sense of it
> (within reason; unsupported features are still unsupported features) and
> that the clients *expect* to see information that they don't know how to
> make sense of (e.g., they need to be designed to be robust in the face
> of future changes). There is nothing magical about it.

Ok, I understand what you mean - but I still don't like the effects. If
we move towards an XML format (which I now really intend to do unless
someone comes up with a brilliant alternative) then yes, we would at
least have a sporting chance of making the client side "work" with a map
coming from a newer server but... it would also mean that different
images will behave differently. And it could mean that bugs will pop up
like "oh, right, darn, didn't think of that - it works in SM 2.31 and
2.33+ but you are right, it would barf in 2.32".

Sure, I can make the code do a "hey, this tag is odd - lets skip
it"-kinda thing, but I still will reserve the "right" for the client to
say - "nope, sorry, this model is too new for me, at leat the author
thinks so - press yes to go ahead, and have fun in the debugger when SM
trashes your image". :)

> Cheers,
>    - Andreas

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Andreas.Raab
Hi Goran,

>> It simply means that older versions of the client are capable of dealing
>> with newer data sets coming from the server, so that the clients don't
>> need to be updated in sync with the server. There are many good reasons
>> for this; the most important one being that people like to be able to
>> decide when they want to upgrade instead of being forced to.
>
> Well, I for one don't want to have multiple versions of SM running out
> there trying to make sense of the model coming from the latest version
> running on the server. It sounds way too risky for my taste.

Well, that's like saying it's way to risky that people use different
browsers for displaying web content and *force them* to install IE 7
when they want to look at a random page. And "oh, by the way" one of the
reasons why I'm in this discussion is that SM is currently just the
icing on the cake of all the issues that I'm dealing with in the Croquet
release. Its attempt to forcefully update itself in an environment it
knows relatively little about only leads to problems. I'm actually on
the verge of removing SM last minute; the way it's right now with the
2.1 client not working and 2.2 failing to install it's really not very
attractive to carry that much dead code around. (but I'll wait with that
decision simply because I had a *very bad* day today and perhaps I feel
different later).

> Ok, I understand what you mean - but I still don't like the effects. If
> we move towards an XML format (which I now really intend to do unless
> someone comes up with a brilliant alternative) then yes, we would at
> least have a sporting chance of making the client side "work" with a map
> coming from a newer server but... it would also mean that different
> images will behave differently. And it could mean that bugs will pop up
> like "oh, right, darn, didn't think of that - it works in SM 2.31 and
> 2.33+ but you are right, it would barf in 2.32".

Well, isn't that called testing?! ;-) Maybe a few tests would help to
automate the process?! ;-)) Actually, I'm only partly kidding - it seems
pretty clear that any version you want to support needs at least a
rudimentary amount of support and testing but if there are people out
there interested in a particular system I'm sure they'll give you the
feedback you need.

> Sure, I can make the code do a "hey, this tag is odd - lets skip
> it"-kinda thing, but I still will reserve the "right" for the client to
> say - "nope, sorry, this model is too new for me, at leat the author
> thinks so - press yes to go ahead, and have fun in the debugger when SM
> trashes your image". :)

And that's perfectly reasonable, though, in my experience fairly
unlikely if you start out with the goal of supporting future versions.
Like, for example, the interpreter proxy - it was designed to be able to
indicate to a plugin that it has changed beyound recognition by changing
the major version number. And yet, even though the VM has evolved since
the proxy was first invented, the very first plugins will still happily
work with the very latest VMs. In other words, it's really more a
question of *wanting* to support future versions - if you have a basic
understanding of what your system needs to provide to a client to work
reasonably well (and I think by now, you do have a fairly good
understanding of that for SM) you can design things so that they still
work tomorrow even if some parts change and evolve. The VM is proof of
that and I'm virtually certain that SM could be designed with a very
similar version scheme with very similar long-term properties.

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Göran Krampe
Hi Andreas!

Andreas Raab <[hidden email]> wrote:

> Hi Goran,
>
> >> It simply means that older versions of the client are capable of dealing
> >> with newer data sets coming from the server, so that the clients don't
> >> need to be updated in sync with the server. There are many good reasons
> >> for this; the most important one being that people like to be able to
> >> decide when they want to upgrade instead of being forced to.
> >
> > Well, I for one don't want to have multiple versions of SM running out
> > there trying to make sense of the model coming from the latest version
> > running on the server. It sounds way too risky for my taste.
>
> Well, that's like saying it's way to risky that people use different
> browsers for displaying web content and *force them* to install IE 7
> when they want to look at a random page.

Well, SM does more than "looking" (if IE 5 renders a page ugly it
doesn't hurt that much - if SM screws up installing things it hurts
more) and I am aiming for it to do even more in the future - actually
modifying the map (getting rid of the web UI for modifying the map and
instead letting client images upload modified SMObjects in XML form - at
least that is the plan this hour).

> And "oh, by the way" one of the
> reasons why I'm in this discussion is that SM is currently just the
> icing on the cake of all the issues that I'm dealing with in the Croquet
> release. Its attempt to forcefully update itself in an environment it
> knows relatively little about only leads to problems.

I can buy that. But doesn't it actually ask? Anyway, I know what you
mean.

> I'm actually on
> the verge of removing SM last minute; the way it's right now with the
> 2.1 client not working and 2.2 failing to install

AFAIK you haven't yet mentioned 2.2 failing to install. I just tried it
though (in Croquet) and I noted you are forbidding underscores as
assignments - that was at least one issue (and I now changed the
underscores in the load script). Hmmm, let me test again btw...

...yes, ok, it works fine (unless I have missed something) if you first
do:

        Preferences enable: #allowUnderscoreAssignment

:)

But I can clean those underscores up of course, anyone got a nice script
that *works* :) to do that?

> it's really not very
> attractive to carry that much dead code around. (but I'll wait with that
> decision simply because I had a *very bad* day today and perhaps I feel
> different later).

Sure.

> > Ok, I understand what you mean - but I still don't like the effects. If
> > we move towards an XML format (which I now really intend to do unless
> > someone comes up with a brilliant alternative) then yes, we would at
> > least have a sporting chance of making the client side "work" with a map
> > coming from a newer server but... it would also mean that different
> > images will behave differently. And it could mean that bugs will pop up
> > like "oh, right, darn, didn't think of that - it works in SM 2.31 and
> > 2.33+ but you are right, it would barf in 2.32".
>
> Well, isn't that called testing?! ;-) Maybe a few tests would help to
> automate the process?! ;-))

I know. :) Yes, I should write some tests for SM.

> Actually, I'm only partly kidding - it seems
> pretty clear that any version you want to support needs at least a
> rudimentary amount of support and testing but if there are people out
> there interested in a particular system I'm sure they'll give you the
> feedback you need.

Sure, but I agree - I should add tests.

> > Sure, I can make the code do a "hey, this tag is odd - lets skip
> > it"-kinda thing, but I still will reserve the "right" for the client to
> > say - "nope, sorry, this model is too new for me, at leat the author
> > thinks so - press yes to go ahead, and have fun in the debugger when SM
> > trashes your image". :)
>
> And that's perfectly reasonable, though, in my experience fairly
> unlikely if you start out with the goal of supporting future versions.
[SNIP]

Well, I am convinced to at least give it a shot. And it will be much
easier when we have moved to a "declarative" format like XML.

regards, Göran

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Andreas.Raab
Hi Goran -

[hidden email] wrote:
> Well, SM does more than "looking" (if IE 5 renders a page ugly it
> doesn't hurt that much - if SM screws up installing things it hurts
> more) and I am aiming for it to do even more in the future - actually
> modifying the map (getting rid of the web UI for modifying the map and
> instead letting client images upload modified SMObjects in XML form - at
> least that is the plan this hour).

I think that's a discussion for another day but I'd recommend to keep
the web UI up.

> AFAIK you haven't yet mentioned 2.2 failing to install. I just tried it
> though (in Croquet) and I noted you are forbidding underscores as
> assignments - that was at least one issue (and I now changed the
> underscores in the load script). Hmmm, let me test again btw...
>
> ...yes, ok, it works fine (unless I have missed something) if you first
> do:
>
> Preferences enable: #allowUnderscoreAssignment

Actually, yes I missed something. My problem wasn't really installing (I
just thought it was) but earlier in the process. There is a bug in 2.1
in SMSqueakMap>>reload stating, e.g.,

contents := (self directory oldFileNamed: fname) ascii upToEnd unzipped.

Since you're setting it to ascii, the stream installs a UTF8Converter
and stops reading at certain characters (this is another place that
really should raise an error to inform the user that something went
wrong instead of returning nil which informs the sender that the stream
is apparently at end). The above needs to be "binary upToEnd asString
unzipped" or somesuch. I got caught by that because I expected that
Squeakmap was already installing.

> But I can clean those underscores up of course, anyone got a nice script
> that *works* :) to do that?

Oh, I am *very* confident that I have thoroughly debugged Bert's script.
That exercise was a major part of my painful experiences yesterday
because in the process of noting where it had failed earlier I blew up
all of our repositories (and badly so). But actually it will work fine
if you do just one change - that is use #hasLiteralSuchThat: instead of
just looking at the top-level literals. That's because
hasLiteralSuchThat: will look inside Array literals and that will avoid
issues like here:

         version >= 1.1 ifTrue:[
             extensions addAll: #(
                 'GL:=EXT:=blend:=logic:=op'
                 'GL:=EXT:=copy:=texture'
                 'GL:=EXT:=polygon:=offset'
                 'GL:=EXT:=subtexture'
                 'GL:=EXT:=texture'
                 'GL:=EXT:=texture:=object'
                 'GL:=EXT:=vertex:=array'
             ).
         ].


Nice, eh?

Cheers,
   - Andreas

Reply | Threaded
Open this post in threaded view
|

Re: New SqueakMap on the air... and we got problems Houston!

Göran Krampe
Hi Andreas!

Andreas Raab <[hidden email]> wrote:

> Hi Goran -
>
> [hidden email] wrote:
> > Well, SM does more than "looking" (if IE 5 renders a page ugly it
> > doesn't hurt that much - if SM screws up installing things it hurts
> > more) and I am aiming for it to do even more in the future - actually
> > modifying the map (getting rid of the web UI for modifying the map and
> > instead letting client images upload modified SMObjects in XML form - at
> > least that is the plan this hour).
>
> I think that's a discussion for another day but I'd recommend to keep
> the web UI up.

Yes, I will never *drop* the web UI - but hopefully we can then move to
make Squeak tools to modify the map instead of being *forced* to use the
web UI.

> > AFAIK you haven't yet mentioned 2.2 failing to install. I just tried it
> > though (in Croquet) and I noted you are forbidding underscores as
> > assignments - that was at least one issue (and I now changed the
> > underscores in the load script). Hmmm, let me test again btw...
> >
> > ...yes, ok, it works fine (unless I have missed something) if you first
> > do:
> >
> > Preferences enable: #allowUnderscoreAssignment
>
> Actually, yes I missed something. My problem wasn't really installing (I
> just thought it was) but earlier in the process. There is a bug in 2.1
> in SMSqueakMap>>reload stating, e.g.,
>
> contents := (self directory oldFileNamed: fname) ascii upToEnd unzipped.
>
> Since you're setting it to ascii, the stream installs a UTF8Converter
> and stops reading at certain characters (this is another place that
> really should raise an error to inform the user that something went
> wrong instead of returning nil which informs the sender that the stream
> is apparently at end). The above needs to be "binary upToEnd asString
> unzipped" or somesuch. I got caught by that because I expected that
> Squeakmap was already installing.

Yes, I know. This code has been fixed in 2.2. I can't remember the exact
details but for some odd reason (I think I was trying to find how to
unzip the darn thing and didn't really understand at that moment that
ascii would start doing funky conversions - or perhaps it was that it
doesn't in 3.7 but does in 3.8, I dunno) I ended up with the above code
and didn't discover that it actually doesn't work. :)

Btw, yesterday I was staring at the MultiByteFileStream stuff and...
well, IMHO it would have been better *for me* (other users may have
other stories to tell) if the default was binary and not ascii. The
principle of least surprise. If I open a filestream and don't tell it
*anything*, then I would expect it to just feed me the bits and bytes -
as Strings or ByteArrays, but not doing any conversions or line end
mumbo jumbo or any other non expected "nice things". An example of this
is inspecting a file in the file list - I really appreciated the fact
that filelist didn't do *any* conversion on the stuff it showed me - now
it does. And I also wonder where the hex view went... anyway:

Yesterday my collegue wanted to save stuff with platform specific line
endings (wantsLineEndConversion: true etc) but NOT doing any other
conversions. We realized that you can't set converter to nil - it will
lazily set itself to the default platform converter (seems to me at
least). And if you tell the stream to be binary it will not do line end
conversions.

What I ended up doing was creating NullTextConverter (which does no
conversion at all) and then it worked fine. It seems to me that a
cleaner approach here would be to:

1. Do line end conversions or not regardless of the 2 choices below..
2. Binary or ascii - only decides if we use ByteArrays or Strings,
doesn't concern conversions or line ends.
3. Selection of converter where we also have a NullConverter that does
nothing.

IMHO (having not dissected this in total detail) the above three options
should be combinable. So for example, in our case we have utf8 strings
that we want to write out *as is* and use #cr to get platform specific
line endings.

> > But I can clean those underscores up of course, anyone got a nice script
> > that *works* :) to do that?
>
> Oh, I am *very* confident that I have thoroughly debugged Bert's script.

:)

> That exercise was a major part of my painful experiences yesterday
> because in the process of noting where it had failed earlier I blew up
> all of our repositories (and badly so). But actually it will work fine
> if you do just one change - that is use #hasLiteralSuchThat: instead of
> just looking at the top-level literals. That's because
> hasLiteralSuchThat: will look inside Array literals and that will avoid
> issues like here:
>
>          version >= 1.1 ifTrue:[
>              extensions addAll: #(
>                  'GL:=EXT:=blend:=logic:=op'
>                  'GL:=EXT:=copy:=texture'
>                  'GL:=EXT:=polygon:=offset'
>                  'GL:=EXT:=subtexture'
>                  'GL:=EXT:=texture'
>                  'GL:=EXT:=texture:=object'
>                  'GL:=EXT:=vertex:=array'
>              ).
>          ].
>
>
> Nice, eh?

Neato. Btw, the method in which I discovered that it had messed up my
literals (in an array) I also noted it had changed the underscores
inside the method comment. Or at least I think it had, can't swear on my
laptop though. ;)

> Cheers,
>    - Andreas

regards, Göran

Reply | Threaded
Open this post in threaded view
|

FixUnderscores fixed (was Re: New SqueakMap on the air... and we got problems Houston!)

Bert Freudenberg-3
Am 07.04.2006 um 09:00 schrieb [hidden email]:

> Andreas Raab <[hidden email]> wrote:
>> [hidden email] wrote:
>
>>> But I can clean those underscores up of course, anyone got a nice  
>>> script
>>> that *works* :) to do that?
>>
>> Oh, I am *very* confident that I have thoroughly debugged Bert's  
>> script.
>
> Neato. Btw, the method in which I discovered that it had messed up my
> literals (in an array) I also noted it had changed the underscores
> inside the method comment. Or at least I think it had, can't swear  
> on my
> laptop though. ;)

It simply does a text search-and-replace of '_' to ':=', skipping  
methods which have literal underscores. It failed to look deeply  
inside Array literals, though.

FixUnderscores-bf.8 from http://source.impara.de/underscore.html has  
the fix for that. It will still replace underscores in comments  
(unless there is a literal underscore in the method, too).

- Bert -


Reply | Threaded
Open this post in threaded view
|

Loading code containing underscore assignments using Monticello

Chris Becker-2
Hi all,

I'm trying to load code that contains underscore assignment operators  
using Monticello. It pops up a "syntax error" dialog for each method.  
Since I have hundreds of methods it would be a nightmare to replace  
them with ":=" manually.

Anyone solve this problem yet?

Thanks,

Chris Becker

Reply | Threaded
Open this post in threaded view
|

Re: Loading code containing underscore assignments using Monticello

Bert Freudenberg-3
Am 25.04.2006 um 18:05 schrieb Chris Becker:

> Hi all,
>
> I'm trying to load code that contains underscore assignment  
> operators using Monticello. It pops up a "syntax error" dialog for  
> each method. Since I have hundreds of methods it would be a  
> nightmare to replace them with ":=" manually.
>
> Anyone solve this problem yet?

Which image are you using?

- Bert -


Reply | Threaded
Open this post in threaded view
|

Re: Loading code containing underscore assignments using Monticello

Chris Becker-2
It's the Croquet 1.0.10 image.

The code I'm loading into it was created in Squeak 3.6.

Thanks,

Chris


On Apr 25, 2006, at 5:39 PM, Bert Freudenberg wrote:

> Am 25.04.2006 um 18:05 schrieb Chris Becker:
>
>> Hi all,
>>
>> I'm trying to load code that contains underscore assignment  
>> operators using Monticello. It pops up a "syntax error" dialog for  
>> each method. Since I have hundreds of methods it would be a  
>> nightmare to replace them with ":=" manually.
>>
>> Anyone solve this problem yet?
>
> Which image are you using?
>
> - Bert -
>


12