Cleaning up SqueakMap

After the recent discussions about revamping SqueakMap, I decided to attempt loading interesting packages into 4.1.  The first one I tried was "3D Facial Animation."

Current Status: there is no file to download and no way to contact the owner

Process:
1. Download link - broken
2. Link to user page - page does not exist
2.5. SqueakSource - project search "3D" and "facial" produce no results
3. Owner email - mail delivery failed
4. Google - no promising leads (name mentioned, but didn't find contact info)
5. Facebook - found person with same name; message, pending response

Now the only value I can see to having this project appear in SM is to let people know "hey this sort of thing is possible."  OTOH, it sucked quite a bit of time from my day tracking it down, when who knows how many previous detectives went through the same process.

It would be great it I could:
* indicate the status of this project on SqueakMap e.g. "No file to download - for historical value only"
* leave the process I took right there, so that someone who really wants it can pick up where I left off instead of starting from scratch.

Thanks.
Sean

Chris Muller wrote (and I deleted by accident - sorry Chris):
Web-sites come and go...  This is why I think SqueakMap should keep a
copy of the files in its own files area; they are generally small so
space shouldn't be an issue.

There are lots of packages on SqS that are in the same boat you describe. And those files are readily available. The won't work because Squeak has moved on. There is a difference between what the catalog or repository is responsible for. And what the package maintainer is responsible for. If the code is for 3.6, I don't suppose it would load in to 4.1 very readily in any case. Code decays over time. 

Moving on to Aconcagua...

SM Package Loader: nothing installed (seems there are no files mentioned in SM)
SS: latest version all-but-2 tests green in Squeak 4.1#10142

- S

Thanks for doing this!  What a great gift for 4.2 if many (all??) of
the SqueakMap packages could just "load-and-go".

One suggestion:  If you encounter trouble loading some of them into
Squeak 4.1, try loading them into the version of squeak they're
declared to support..

On Thu, May 13, 2010 at 1:44 PM, Sean P. DeNigris <[hidden email]> wrote:

Chris Cunnington-3 wrote

There are lots of packages on SqS that are in the same boat you describe.
And those files are readily available. The won't work because Squeak has
moved on.

I would say they don't work because... they don't work ;-)  "Moved on" seems too tidy a description for how the Squeak story occurs to me.  It's more of a labyrinth than a linear timeline.  The fact that packages are not current could reflect a whole lot of coincidental factors (e.g. owner had children) rather than not being useful.  What I'm saying is building on Bert's comments from "Re: SqueakMap soon working in 4.0/4.1!":

Bert Freudenberg wrote

There is one fundamental problem with both the SqueakMap and Universes model that has not been mentioned yet: It does not encourage participation. For a package author, maintaining a package entry is just an additional burden. And a package user cannot really do much about a broken package entry.

Contrast that with the Trunk Model: One reason it works is that it takes almost zero effort to participate. You publish a fix to the inbox and announce that on squeak-dev. And it's very simple for a core developer to take it and commit to the trunk.

IMHO we need something similarly simple for a package management system for Squeak. Something where it is easy to share. If one user figures out how to get a particular package to load, it must be trivial to share that method. And submitting one such "package loading instruction" must not sign up the user to be perpetual maintainer of the package.

So I think the private "ownership" model is flawed. We need a package management system that allows easy contribution. Maybe SqueakMap can be restructured for this, but currently it seems heavily geared towards single maintainers.

Let's empower people who have done legwork in finding out whether projects work, or to make them work, to share that with the community so we're not churning, doing the same work over and over in secret.

Chris Cunnington-3 wrote

If the code is for 3.6, I don't suppose it would load in to 4.1 very readily in any
case. Code decays over time.

Sure.  I'm looking from the user perspective - if I'm working on something that can enhanced with an existing package, whether there is slightly broken code, or severely broken code, or no code at all, will make a big difference in pursuing it; and if someone else already spent hours finding that out, I'd want to know.

Sean

Chris Muller-3 wrote

Thanks for doing this!  What a great gift for 4.2 if many (all??) of
the SqueakMap packages could just "load-and-go".

My pleasure - equal parts love for the community and selfish curiosity ;-)

Chris Muller-3 wrote

One suggestion:  If you encounter trouble loading some of them into
Squeak 4.1, try loading them into the version of squeak they're
declared to support..

I didn't get that far yet, but that definitely will come into play, esp. trying to get one of the key binding projects loadable (which is urgent for me).

Thanks.
Sean

Hi,


CM> I searched my computer for Avatar.cs.gz, but didn't find it.  Does
CM> anyone else still have Avatar.cs.gz in their cache or anywhere else on
CM> their computer?

mailed a copy to Chris. In case of interest I got a maybe 4 years old
copy of the swiki.

Cheers

Herbert

Applescript project
SM Package Loader: most samples ran with 1 change "AEDesc>>createFromText: aString ... (aString isKindOf: String) ifFalse:..." (which I later found was already fixed in the SqueakSource version)

SqSource: latest version seemed to run all samples OK

Keymapping (I'm excited about this one!)

SM: not listed
SS (latest version): all tests pass in 4.1#10142!!  I'm still playing with it, but it seems to work with minimal changes.  I'll keep y'all posted.

- S

The old Squeak CDs had full copies of SqueakMap, and you can download
them at no cost. Most likely you will find it on one of the CDs.

http://www.squeak.org/Download/SqueakCD/
http://ftp.squeak.org/CDROM/


Dave

On Thu, May 13, 2010 at 01:34:42PM -0500, Chris Muller wrote:

I looked for purposes of contributing to the process of freshening
SqueakMap.  I wasn't looking for it, myself.

Only 24K, attached, in case someone doesn't want to download and dig
through teh CD (but that might be a lot of fun!).

  - Chris

2010/5/13 Herbert König <[hidden email]>:

Hi!

On 05/13/2010 08:34 PM, Chris Muller wrote:
> Web-sites come and go...  This is why I think SqueakMap should keep a
> copy of the files in its own files area; they are generally small so
> space shouldn't be an issue.

Just so you all know - SM does have two features that might be good to
know about:

1. You *can* upload and host files on SM if you like to. Some packages
do. But you probably knew that.

2. SM *caches* the file pointed to by the release URL! This is not so
well known, but when you enter a release (or edit the URL IIRC) SM will
donwload that file, compute the SHA hash for it, and keep it on the server.

Now, when you download a file from the map the loader will compute the
SHA hash and compare it to the one computed originally - and if it does
*not* match - the loader will try to load from the SM server cache
instead, for example:


        http://map.squeak.org/package/1e5499c3-129e-4e56-a338-0a2c113bb2c4/autoversion/3/cache

which will give you the SM server copy of:

        http://www.squeaksource.com/Aida/Aida-5.6.mcz

Now... there is one problem, if the file pointed to by the URL is
changed "in place" then the SHA hash will fail and the cached version
will be used instead.

So, one way to "screw it up" is to register a release with a URL that
fails - say returns a 404. SM will then consider that result to be the
correct result and compute an SHA hash and store the 404.html file as
the cached "proper file". When the real file is placed on that URL it
will not work since the SHA hash will be different and SM will instead
rely on the cached file (which of course is a silly 404.html).

You can see if a download ends up using the cache by looking at transcript.

A bunch of years ago I published a doit that iterated over the map and
downloaded all releases, you can easily write such a script and also
find out which ones fail SHA checks etc. I think I did it for the Squeak
CD which was mentioned.

regards, Göran

On 5/14/10, Göran Krampe <[hidden email]> wrote:
> Hi!
>
> So, one way to "screw it up" is to register a release with a URL that
> fails - say returns a 404. SM will then consider that result to be the
> correct result and compute an SHA hash and store the 404.html file as
> the cached "proper file". When the real file is placed on that URL it
> will not work since the SHA hash will be different and SM will instead
> rely on the cached file (which of course is a silly 404.html).

This is the description of a bug. If an URL returns 404 and SqueakMap
still has a file cached it should not overwrite the cache version with
404.html

--Hannes

On 05/14/2010 10:08 AM, Hannes Hirzel wrote:
You did not read what I wrote (or I didn't describe it clear enough), I
was describing a procedure where you *first* register a URL/release and
only *afterwards* ensures that there is a file on that URL. That will
not work.

SM only updates the cache (AFAIK) if you:

        1. Enter a new release.
        2. Edit an existing release and *modify* its download URL.

...since the purpose of the cache is to actually *help* when a URL goes
bad/offline etc, it would otherwise be quite pointless and indeed a
major bug.

regards, Göran

Sean,

just wondering if this might be what you meant by the "interesting 3D Facial
Animation" package:

http://wiki.squeak.org/squeak/651

if so, it does actually work in 4.1 update 10143 - at least on windows... i got it to work exactly as it did way back when it 1st came out. 

it actually is quite interesting. but, in context, it seems more interesting that it does not load correctly even from the above...

i will send how i got it to work later 2day. but, i did not have to alter anything except how to load the code.

forgive if you meant a different "3D Facial Animation" package... this one is the only i have known of, or at least used...

ken






 


----- Original Message ----
From: Sean P. DeNigris <[hidden email]>
To: [hidden email]
Sent: Thu, May 13, 2010 11:03:21 AM
Subject: [squeak-dev] Cleaning up SqueakMap


After the recent discussions about revamping SqueakMap, I decided to attempt
loading interesting packages into 4.1.  The first one I tried was "3D Facial
Animation."

Current Status: there is no file to download and no way to contact the owner

Process:
1. Download link - broken
2. Link to user page - page does not exist
2.5. SqueakSource - project search "3D" and "facial" produce no results
3. Owner email - mail delivery failed
4. Google - no promising leads (name mentioned, but didn't find contact
info)
5. Facebook - found person with same name; message, pending response

Now the only value I can see to having this project appear in SM is to let
people know "hey this sort of thing is possible."  OTOH, it sucked quite a
bit of time from my day tracking it down, when who knows how many previous
detectives went through the same process.

It would be great it I could:
* indicate the status of this project on SqueakMap e.g. "No file to download
- for historical value only"
* leave the process I took right there, so that someone who really wants it
can pick up where I left off instead of starting from scratch.

Thanks.
Sean
--
View this message in context: http://forum.world.st/Cleaning-up-SqueakMap-tp2197965p2197965.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.

Squeak List wrote

just wondering if this might be what you meant by the "interesting 3D Facial
Animation" package:

http://wiki.squeak.org/squeak/651

It's a different project, but...

Squeak List wrote

if so, it does actually work in 4.1 update 10143 - at least on windows... i got it to work exactly as it did way back when it 1st came out. 

it actually is quite interesting. but, in context, it seems more interesting that it does not load correctly even from the above...

i will send how i got it to work later 2day. but, i did not have to alter anything except how to load the code.

Sure we'd like to know how to load it :)

Sean

> ...since the purpose of the cache is to actually *help* when a URL goes
> bad/offline etc, it would otherwise be quite pointless and indeed a major
> bug.

Thanks for that explanation.  After all those years of using SM and
pulling my hair due to forgetting about the "cache bug", I finally
know what the motivation for it was..

I suppose my only suggestion would be, then, that we change SM to
always remove the cache entry and re-download the file and recompute
the hash when a release is updated, even if the URL did not change.
The purpose of the cache is trying to keep the *contents* of the
downloaded file correct and preserved for us, so those *contents*
should be updated in the cache when the release is updated by the
author, wouldn't you say?


2010/5/14 Göran Krampe <[hidden email]>:

Hi!

On 05/14/2010 09:32 PM, Chris Muller wrote:
Sounds fair to me. :) We could even add a button somewhere saying
"Repopulate cache" or something perhaps slightly more explanatory.

regards, Göran

Sean,

"Sure we'd like to know how to load it :)"...

well, i spelled out exactly what i did in a message 2 the group.

and, it bounced back:

"
Your mail to 'Squeak-dev' with the subject
    Re: [squeak-dev] Re: Cleaning up SqueakMap
Is being held until the list moderator can review it for approval.
The reason it is being held:
    Message body is too big: 131288 bytes with a limit of 100 KB
"

which is (maybe) because i found the Avatar.cs.gz code and i attached it to the email. and i also attached a .txt version. yahoo mail says it is under 100kb...

yet, it "bounces back"...

"Message body is too big: 131288 bytes with a limit of 100 KB" .... uh....???? whatever...

+++++++

ok, so here is the same thing minus the  Avatar.cs.gz file: i will try with only the "text version" of  Avatar.cs.gz attached... :

Sean,

this is what i did on an amd dual processor machine running windows vista 32 bit:


From Scratch:

1)setup/install fresh Squeak 4.1.

2) tell it to update via trunk.

3) update - to update number: 10145 (2nite).

(btw, sidenote: updating a fresh install leaves what looks like a "rectangle just sitting there with no indication what it is"... but it is a a SystemProgressMorph... and i just delete it. but is that intentional, that there is "a blank white rectangle just sitting there with no indication what it is" after updating? i am guessing that it is not...)

4) purge undo records and collect garbage.

5) save and quit squeak - rename if desired...

6) download: http://wiki.squeak.org/squeak/uploads/651/Speech.sar ---- AS A .ZIP FILE. which happens automatically in my Windows Vista/IE8 system...
this isnt really important if you are familiar/comfortable working with .sar files, but it is how i got this to work 1st :) ---- download it into the same directory as where the freshly updated 4.1 Squeak is.

7) extract Speech.zip into the same folder that contains the squeak image, sources, etc... it will add 7 new .st files...

8) re-open the freshly updated squeak and open FileList.

9) highlight and "Install"

IN THIS ORDER:

Speech-TTS.st

(which again leaves what looks like a "rectangle just sitting there with no indication what it is"... same as above... just move it out of the way or delete it... same for below...)

Speech-Support.st

Speech-Phonetics.st

Speech-Phoneme Recognizer.st

Speech-Events.st

Speech-Gestures.st

Speech-Klatt.st

10) close FileList (and any remaining "rectangle(s) just sitting there with no indication what it is"...)...

11) purge undo records and collect garbage.

12) open a workspace and DO IT:  

Speaker manWithHead say: 'this code is ancient'

or for "closer to the avatar look" link below:

Speaker manWithEditor say: 'this looks more like the avatar link below'

check out the "test" and "new head" buttons...

13) in Browser, check out the Category:Speech-TTS / Class: Speaker / Class side "examples" and "examples-others"... or whatever else may be of interest :)

---

i found the following, but do not remember ever seeing it in squeak (it, the link and images, reminds me of JUN):
http://translate.googleusercontent.com/translate_c?hl=en&sl=ja&u=http://languagegame.org:8888/zoo/160.version%3Fid%3D116&prev=/search%3Fq%3DAvatarBuilder%2Bnew%2BopenInWorld%26hl%3Den%26sa%3DG&rurl=translate.google.com&usg=ALkJrhj5aBZ8N0sXMcEJapxwWGgi6MDnfw

1 - 13 above are more like South Park faces - imo...

---

re: 6 and 7 above, that was just the quickest way for "me" to get it to work.

actually, all of it was the quickest way for "me" to get it to work...

and this .sar is on squeakmap (under "Speech") - but it will not install correctly unless you do something similar to the above.

however, i am impressed that the examples work in 4.1 without altering the actual code.

yes, it seems that squeakmap will not install it correctly from what i have seen so far... and i did not find any tutorial that walked me through how to install code which i could not predict would work in the latest and greatest (4.1)... also, i have not made or ran any tests beyond the above: my only tests of interest were the above 2 examples. i like it very much when "examples" actually work.

if you look in the "install" folder (in the same directory as the fresh squeak install - if you did as above), and open up the "preamble": it does not even show all 7 classes... and i was using these Classes/code before 2003 - more like 2001 when i found them interesting and useful... i have no idea who: [ 3/19/2003 bkv ] is... but it (the SPEECH Classes and their "south park style") was working very well before 2003 (more like 2001 when i was using this) - which is interesting, since the attached avatar code begins with:
"
From Squeak3.2alpha of 10 October 2001 [latest update: #4599] on 21 December 2001 at 3:00:40 pm ....
"
- yet, at least from the tiny bit i have checked  is working in 2010.
---

i found: Avatar.cs.gz  - and am attaching it (and a text version).   

not particularly interested in trying to figure out what "its" install problems are ATM :)



that's it for now.

ken