Smalltalk › Squeak › Squeak - Dev

[squeak-dev] Closures in Trunk

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

20 messages Options

Andreas.Raab

[squeak-dev] Closures in Trunk

Folks -

As promised I spent some time to get closures into the trunk. This means
that you *must* use closure-enabled VMs in order to be able to stay
current. Current closure-enabled VMs can be obtained here:

http://squeakvm.org/win32/release/SqueakVM-Win32-3.11.3-bin.zip (win)
ftp://ftp.smalltalkconsulting.com/Squeak%204.1.1beta2U.app.zip (mac)
http://ftp.squeak.org/Exupery/vms/exupery-vm-0.15.1-linux.tz (linux)

In addition, I have done a round of (automatic) replacing underscores
with colon-equals to simplify diffing between Croquet/Cobalt/Pharo and
Squeak. Nicolas - if you want to try the diff again, it should make a
*lot* more sense now.

Finally, since the updates will take quite a while, I've prepared an
image that is up-to-date with the trunk as of today. You can download it
from here:

http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip

Cheers,
- Andreas

Antony Blakey-3

Re: [squeak-dev] Closures in Trunk

On 19/07/2009, at 3:46 PM, Andreas Raab wrote:

> http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip

That's actually at http://squeakvm.org/win32/release/Squeak-3.10.2-trunk.zip

Antony Blakey
--------------------------
CTO, Linkuistics Pty Ltd
Ph: 0438 840 787

Only two things are infinite, the universe and human stupidity, and
I'm not sure about the former.
-- Albert Einstein

Edgar J. De Cleene

Re: [squeak-dev] Closures in Trunk

In reply to this post by Andreas.Raab

On 7/19/09 3:16 AM, "Andreas Raab" <[hidden email]> wrote:

> Folks -
>
> As promised I spent some time to get closures into the trunk. This means
> that you *must* use closure-enabled VMs in order to be able to stay
> current. Current closure-enabled VMs can be obtained here:
>
> http://squeakvm.org/win32/release/SqueakVM-Win32-3.11.3-bin.zip (win)
> ftp://ftp.smalltalkconsulting.com/Squeak%204.1.1beta2U.app.zip (mac)
> http://ftp.squeak.org/Exupery/vms/exupery-vm-0.15.1-linux.tz (linux)
>
> In addition, I have done a round of (automatic) replacing underscores
> with colon-equals to simplify diffing between Croquet/Cobalt/Pharo and
> Squeak. Nicolas - if you want to try the diff again, it should make a
> *lot* more sense now.
>
> Finally, since the updates will take quite a while, I've prepared an
> image that is up-to-date with the trunk as of today. You can download it
> from here:
>
> http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip
>
> Cheers,
> - Andreas

Load processs don't complete.

See atached walkback
I on IRC and Skype now ...

Edgar

SqueakDebug.log (5K) Download Attachment

Ken Causey-3

Re: [squeak-dev] Closures in Trunk

On Sun, 2009-07-19 at 09:20 -0300, Edgar J. De Cleene wrote:

> On 7/19/09 3:16 AM, "Andreas Raab" <[hidden email]> wrote:
>
> > Folks -
> >
> > As promised I spent some time to get closures into the trunk. This means
> > that you *must* use closure-enabled VMs in order to be able to stay
> > current. Current closure-enabled VMs can be obtained here:
> >
> > http://squeakvm.org/win32/release/SqueakVM-Win32-3.11.3-bin.zip (win)
> > ftp://ftp.smalltalkconsulting.com/Squeak%204.1.1beta2U.app.zip (mac)
> > http://ftp.squeak.org/Exupery/vms/exupery-vm-0.15.1-linux.tz (linux)
> >
> > In addition, I have done a round of (automatic) replacing underscores
> > with colon-equals to simplify diffing between Croquet/Cobalt/Pharo and
> > Squeak. Nicolas - if you want to try the diff again, it should make a
> > *lot* more sense now.
> >
> > Finally, since the updates will take quite a while, I've prepared an
> > image that is up-to-date with the trunk as of today. You can download it
> > from here:
> >
> > http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip
> >
> > Cheers,
> > - Andreas
> Load processs don't complete.
> See atached walkback
> I on IRC and Skype now ...
>
> Edgar

Edgar,

It would help if you could be more specific. My first thought was that
you referred to taking an image that was at least a day or two out of
date with respect to the trunk and doing an update from server. So I
did that and I got an error (more about that in a minute) but one quite
different to yours. Your error is almost certainly regarding a bad zip
file, specifically it appears a malformed MCZ file, however I can't see
in the debug info a specific filename or other clue as to which file.

My error:

So I took my trusty working.image updates yesterday or the day before
and started it using the 0.15.1 exupery squeak vm and closed everything
but the Transcript (just to be safe) and did a load code updates from
server. It chokes on Kernel-ar.188 with an error "Method context cannot
be changed". Debug log attached.

Ken

SqueakDebug.log (5K) Download Attachment

signature.asc (196 bytes) Download Attachment

Michael van der Gulik-2

Re: [squeak-dev] Closures in Trunk

In reply to this post by Andreas.Raab

On Sun, Jul 19, 2009 at 6:16 PM, Andreas Raab <[hidden email]> wrote:

Folks -

As promised I spent some time to get closures into the trunk. This means that you *must* use closure-enabled VMs in order to be able to stay current. Current closure-enabled VMs can be obtained here:

http://squeakvm.org/win32/release/SqueakVM-Win32-3.11.3-bin.zip (win)
ftp://ftp.smalltalkconsulting.com/Squeak%204.1.1beta2U.app.zip (mac)
http://ftp.squeak.org/Exupery/vms/exupery-vm-0.15.1-linux.tz (linux)

In addition, I have done a round of (automatic) replacing underscores with colon-equals to simplify diffing between Croquet/Cobalt/Pharo and Squeak. Nicolas - if you want to try the diff again, it should make a *lot* more sense now.

Finally, since the updates will take quite a while, I've prepared an image that is up-to-date with the trunk as of today. You can download it from here:

http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip

What is the status of the "NewCompiler" with regard to this new closure VM? Will it still work if it ignores the new bytecodes?

I understand that Eliot modified the "old compiler". I'm going to make compiler modifications over the next year or two for my own project, and I'd prefer working on the NewCompiler.

Gulik.

--
http://gulik.pbwiki.com/

Andreas.Raab

[squeak-dev] Re: Closures in Trunk

In reply to this post by Ken Causey-3

Ken Causey wrote:
> So I took my trusty working.image updates yesterday or the day before
> and started it using the 0.15.1 exupery squeak vm and closed everything
> but the Transcript (just to be safe) and did a load code updates from
> server. It chokes on Kernel-ar.188 with an error "Method context cannot
> be changed". Debug log attached.

Bummer. So much for testing the bootstrap - I didn't have the previous
versions in the test setup and consequently no MCDs were generated, and
nobody ran into the problem. As usual, there is good news and bad news.
The good news: There is an easy workaround; just run this before you try
the update:

(MethodContext instVarNames includes: 'receiverMap') ifTrue:[
MethodContext instVarNames at: 2 put: 'closureOrNil'.
].

The bad news: It appears as if MCDs drop preambles. Plus, I am not sure
I understand the situations under which MCDs are used. For example, I
when (after the error) I tried loading that particular Kernel package,
it worked although I had somewhat expected it to use a diff, too. So
this might be another way to do it. If anyone can shed light on the
behavior of MCDs, when they are created etc. this would be greatly
appreciated.

And of course, if all else fails, download the image.

Cheers,
- Andreas

Bert Freudenberg

Re: [squeak-dev] Re: Closures in Trunk

On 20.07.2009, at 01:22, Andreas Raab wrote:

> Ken Causey wrote:
>> So I took my trusty working.image updates yesterday or the day before
>> and started it using the 0.15.1 exupery squeak vm and closed
>> everything
>> but the Transcript (just to be safe) and did a load code updates from
>> server. It chokes on Kernel-ar.188 with an error "Method context
>> cannot
>> be changed". Debug log attached.
>
> Bummer. So much for testing the bootstrap - I didn't have the
> previous versions in the test setup and consequently no MCDs were
> generated, and nobody ran into the problem. As usual, there is good
> news and bad news. The good news: There is an easy workaround; just
> run this before you try the update:
>
> (MethodContext instVarNames includes: 'receiverMap') ifTrue:[
> MethodContext instVarNames at: 2 put: 'closureOrNil'.
> ].
>
> The bad news: It appears as if MCDs drop preambles. Plus, I am not
> sure I understand the situations under which MCDs are used. For
> example, I when (after the error) I tried loading that particular
> Kernel package, it worked although I had somewhat expected it to use
> a diff, too. So this might be another way to do it. If anyone can
> shed light on the behavior of MCDs, when they are created etc. this
> would be greatly appreciated.

They are created on the squeaksource server when requested (and cached
there for future use). They get requested in
MCConfiguration>>versionNamed:for:from:, foremost to reduce download
size, resulting in an MCDiffyVersion rather than a regular MCVersion.
For this, a file name like MyPackage-ab.42(xy.39).mcd is constructed
if MyPackage-xy.39 is in the image and MyPackage-ab.42 is to be
loaded. The diffy version contains only those methods that changed
between the two versions.

IIRC, preambles are not handled specifically by DiffyVersions. So when
a preamble is requested of the diffy version and it has none, then it
would have to look in its base version. Not sure if this is working,
it probably has never been tested before. Alternatively, the server
could be modified to always include preambles in mcds.

(as an aside, mcd versions have a much greater advantage if the base
version in the image is unmodified, then they can be applied *much*
more efficiently than when loading a full mcz, incredibly speeding up
the upgrade process)

- Bert -

Andreas.Raab

[squeak-dev] Re: Closures in Trunk

Bert Freudenberg wrote:
> IIRC, preambles are not handled specifically by DiffyVersions. So when a
> preamble is requested of the diffy version and it has none, then it
> would have to look in its base version. Not sure if this is working, it
> probably has never been tested before. Alternatively, the server could
> be modified to always include preambles in mcds.

I think the issue might be related to the apparent problem that
preambles and postscripts do not appear to be included in the sources,
but only stored in binary form. This seems badly broken to say the least
- if the loader ever needs to fall back to source parsing, both
preambles and postscripts would be completely ignored. I could see how
that could cause problems along the way.

Cheers,
- Andreas

Bert Freudenberg

Re: [squeak-dev] Re: Closures in Trunk

On 21.07.2009, at 00:54, Andreas Raab wrote:

> Bert Freudenberg wrote:
>> IIRC, preambles are not handled specifically by DiffyVersions. So
>> when a preamble is requested of the diffy version and it has none,
>> then it would have to look in its base version. Not sure if this is
>> working, it probably has never been tested before. Alternatively,
>> the server could be modified to always include preambles in mcds.
>
> I think the issue might be related to the apparent problem that
> preambles and postscripts do not appear to be included in the
> sources, but only stored in binary form. This seems badly broken to
> say the least - if the loader ever needs to fall back to source
> parsing, both preambles and postscripts would be completely ignored.
> I could see how that could cause problems along the way.

Indeed, that is a problem. I never looked how they were actually
implemented ... maybe newer MC versions do better?

- Bert -

bpi

[squeak-dev] Failing tests in Trunk (was: Closures in Trunk)

In reply to this post by Andreas.Raab

Am 19.07.2009 um 08:16 schrieb Andreas Raab:
> Finally, since the updates will take quite a while, I've prepared an
> image that is up-to-date with the trunk as of today. You can
> download it from here:
>
> http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip
Thanks, Andreas!

I have downloaded the new image, loaded the code updates and then ran
the tests. On a Mac with the VM 4.1.1beta2U I get 10 failures and 22
errors. Most of the errors are BlockClosureTests and
ClosureCompilerTests. Is that to be expected or is it just me somehow?

I note that in 3.10.2 there was only 1 failure on the Mac at least,
not quite but almost green. This leads me to another question about
the New Community Development Model: Should there be a rule that no
commit to the trunk is allowed to introduce new failures or errors? I
think that rule would be good. What do others think?

Cheers,
Bernhard

Eliot Miranda-2

Re: [squeak-dev] Failing tests in Trunk (was: Closures in Trunk)

On Thu, Jul 23, 2009 at 2:21 PM, Bernhard Pieber <[hidden email]> wrote:

Am 19.07.2009 um 08:16 schrieb Andreas Raab:

Finally, since the updates will take quite a while, I've prepared an image that is up-to-date with the trunk as of today. You can download it from here:

http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip

Thanks, Andreas!

I have downloaded the new image, loaded the code updates and then ran the tests. On a Mac with the VM 4.1.1beta2U I get 10 failures and 22 errors. Most of the errors are BlockClosureTests and ClosureCompilerTests. Is that to be expected or is it just me somehow?

Yes. My first attempt at the closure compiler did not fix the decompiler, did not implement copying methods with temp names correctly and had a minor compiler bug with optimized blocks.

The issue with the decompiler and with copying methods with temp names is that with BlueBook blocks temporaries are simply an array but with closures they are a tree, since temporaries in blocks live in those blocks, not on the stack of the home context. Since the decompiler and discarding sources can be lived without I left them for later.

I now have a functional decompiler, have implemented copy with temp names and have an unintegrated fix for the compiler bug (the failing test is testInlineBlockCollectionLR3). Once Andreas' new Monticello loader is more generally available I can make these fixes available. The problem is that the fixes are integrated into packages in Qwaq's repository and being compiler changes they don't "just load".

I note that in 3.10.2 there was only 1 failure on the Mac at least, not quite but almost green. This leads me to another question about the New Community Development Model: Should there be a rule that no commit to the trunk is allowed to introduce new failures or errors? I think that rule would be good. What do others think?

Apologies but the failures with closures could not be avoided as there was a lot to do. I had to release incrementally if I was going to get anything out at all. So in certain cases such as this I would beg an exception.

Cheers,
Bernhard

Nicolas Cellier

Re: [squeak-dev] Failing tests in Trunk (was: Closures in Trunk)

In reply to this post by bpi

2009/7/23 Bernhard Pieber <[hidden email]>:

> Am 19.07.2009 um 08:16 schrieb Andreas Raab:
>>
>> Finally, since the updates will take quite a while, I've prepared an image
>> that is up-to-date with the trunk as of today. You can download it from
>> here:
>>
>> http://squeakvm.org/win32/release/Squeak3.10.2-trunk.zip
>
> Thanks, Andreas!
>
> I have downloaded the new image, loaded the code updates and then ran the
> tests. On a Mac with the VM 4.1.1beta2U I get 10 failures and 22 errors.
> Most of the errors are BlockClosureTests and ClosureCompilerTests. Is that
> to be expected or is it just me somehow?
>

No, the closure changes are not 100% polished, this is the same in Pharo.

> I note that in 3.10.2 there was only 1 failure on the Mac at least, not
> quite but almost green. This leads me to another question about the New
> Community Development Model: Should there be a rule that no commit to the
> trunk is allowed to introduce new failures or errors? I think that rule
> would be good. What do others think?
>

I have 3 remarks:
1) For some valuable big changes like closure, I think we should
tolerate a few failures/errors on minor API. It can be a way to
accelerate resolution of these failures/errors (that did not really
happen here because this is very technical).
2) Publishing some new tests might lead to new failure/error reports
for old bugs, so the rule should rather be: no regression.
3) Running all tests is quite long: this is a work for an automaton...

> Cheers,
> Bernhard
>
>

Ken Causey-3

Re: [squeak-dev] Re: Closures in Trunk

In reply to this post by Bert Freudenberg

So back to the original problem with Kernel-ar.188... I hadn't updated
in a few days (ok, maybe more than a week) and finally did so today
trying out the 3.10-6 unix vm and ran into this bug. Having a terrible
memory I had to search through the mailing list to figure out what to
do. Are we just going to leave this issue at the current 'requires a
workaround' status? I hope not.

Ken

On Tue, 2009-07-21 at 08:43 -0300, Bert Freudenberg wrote:

> On 21.07.2009, at 00:54, Andreas Raab wrote:
>
> > Bert Freudenberg wrote:
> >> IIRC, preambles are not handled specifically by DiffyVersions. So
> >> when a preamble is requested of the diffy version and it has none,
> >> then it would have to look in its base version. Not sure if this is
> >> working, it probably has never been tested before. Alternatively,
> >> the server could be modified to always include preambles in mcds.
> >
> > I think the issue might be related to the apparent problem that
> > preambles and postscripts do not appear to be included in the
> > sources, but only stored in binary form. This seems badly broken to
> > say the least - if the loader ever needs to fall back to source
> > parsing, both preambles and postscripts would be completely ignored.
> > I could see how that could cause problems along the way.
>
>
> Indeed, that is a problem. I never looked how they were actually
> implemented ... maybe newer MC versions do better?
>
> - Bert -
>
>
>
>

signature.asc (196 bytes) Download Attachment

Bert Freudenberg

Re: [squeak-dev] Re: Closures in Trunk

I just had another look:

MCMcdReader versionFromFile: 'Kernel-ar.188(dtl.187).mcd'

Indeed there is no preamble in the patch. Now, why ...

*smacks head*

The problem is the version of Monticello on the source server, which
creates the diffs but does not know preambles.

- Bert -

On 12.08.2009, at 18:53, Ken Causey wrote:

> So back to the original problem with Kernel-ar.188... I hadn't
> updated
> in a few days (ok, maybe more than a week) and finally did so today
> trying out the 3.10-6 unix vm and ran into this bug. Having a
> terrible
> memory I had to search through the mailing list to figure out what to
> do. Are we just going to leave this issue at the current 'requires a
> workaround' status? I hope not.
>
> Ken
>
> On Tue, 2009-07-21 at 08:43 -0300, Bert Freudenberg wrote:
>> On 21.07.2009, at 00:54, Andreas Raab wrote:
>>
>>> Bert Freudenberg wrote:
>>>> IIRC, preambles are not handled specifically by DiffyVersions. So
>>>> when a preamble is requested of the diffy version and it has none,
>>>> then it would have to look in its base version. Not sure if this is
>>>> working, it probably has never been tested before. Alternatively,
>>>> the server could be modified to always include preambles in mcds.
>>>
>>> I think the issue might be related to the apparent problem that
>>> preambles and postscripts do not appear to be included in the
>>> sources, but only stored in binary form. This seems badly broken to
>>> say the least - if the loader ever needs to fall back to source
>>> parsing, both preambles and postscripts would be completely ignored.
>>> I could see how that could cause problems along the way.
>>
>>
>> Indeed, that is a problem. I never looked how they were actually
>> implemented ... maybe newer MC versions do better?
>>
>> - Bert -
>>
>>
>>
>>
>

Bert Freudenberg

Re: [squeak-dev] Re: Closures in Trunk

On 21.07.2009, at 00:54, Andreas Raab wrote:
>
> I think the issue might be related to the apparent problem that
> preambles and postscripts do not appear to be included in the
> sources, but only stored in binary form. This seems badly broken to
> say the least - if the loader ever needs to fall back to source
> parsing, both preambles and postscripts would be completely ignored.
> I could see how that could cause problems along the way.

As I wrote in my last message, this is not the actual problem here.

But I fixed it anyway =)

And made the scripts show up in a snapshot browser.

Odd that Morphic has a postscript ... but at least now it's easy to
see. Just update from trunk.

- Bert -

Eliot Miranda-2

Re: [squeak-dev] Re: Closures in Trunk

On Wed, Aug 12, 2009 at 4:01 PM, Bert Freudenberg <[hidden email]> wrote:

On 21.07.2009, at 00:54, Andreas Raab wrote:

I think the issue might be related to the apparent problem that
preambles and postscripts do not appear to be included in the
sources, but only stored in binary form. This seems badly broken to
say the least - if the loader ever needs to fall back to source
parsing, both preambles and postscripts would be completely ignored.
I could see how that could cause problems along the way.

As I wrote in my last message, this is not the actual problem here.

But I fixed it anyway =)

I don't understand your banter^h^h^h^h^h^hemoticon!

`D: D=`	horror, disgust ^[1]^[3]^[4]

but =) ?? explain!

And made the scripts show up in a snapshot browser.

Odd that Morphic has a postscript ... but at least now it's easy to see. Just update from trunk.

- Bert -

Bert Freudenberg

[squeak-dev] [OT] Emoticons

On 13.08.2009, at 01:14, Eliot Miranda wrote:

> On Wed, Aug 12, 2009 at 4:01 PM, Bert Freudenberg <[hidden email]
> > wrote:
>> But I fixed it anyway =)
>
>
> I don't understand your banter^h^h^h^h^h^hemoticon!
> D: D=
> horror, disgust [1][3][4]
> but =) ?? explain!

http://en.wikipedia.org/wiki/Emoticon#Western_style

- Bert -

Eliot Miranda-2

Re: [squeak-dev] [OT] Emoticons

On Wed, Aug 12, 2009 at 4:21 PM, Bert Freudenberg <[hidden email]> wrote:

On 13.08.2009, at 01:14, Eliot Miranda wrote:

On Wed, Aug 12, 2009 at 4:01 PM, Bert Freudenberg <[hidden email]> wrote:

But I fixed it anyway =)

I don't understand your banter^h^h^h^h^h^hemoticon!
D: D=
horror, disgust [1][3][4]
but =) ?? explain!

http://en.wikipedia.org/wiki/Emoticon#Western_style

I looked up =) in http://en.wikipedia.org/wiki/List_of_emoticons but it wasn't there (o.0)

__,,,^..^,,,__

- Bert -

Bert Freudenberg

Re: [squeak-dev] [OT] Emoticons

On 13.08.2009, at 01:33, Eliot Miranda wrote:

On Wed, Aug 12, 2009 at 4:21 PM, Bert Freudenberg <[hidden email]> wrote:

On 13.08.2009, at 01:14, Eliot Miranda wrote:

On Wed, Aug 12, 2009 at 4:01 PM, Bert Freudenberg <[hidden email]> wrote:

But I fixed it anyway =)

I don't understand your banter^h^h^h^h^h^hemoticon!
D: D=
horror, disgust [1][3][4]
but =) ?? explain!

http://en.wikipedia.org/wiki/Emoticon#Western_style

I looked up =) in http://en.wikipedia.org/wiki/List_of_emoticons but it wasn't there (o.0)

Look in the link I gave. Also:

- Bert -

Eliot Miranda-2

Re: [squeak-dev] [OT] Emoticons

On Wed, Aug 12, 2009 at 4:39 PM, Bert Freudenberg <[hidden email]> wrote:

On 13.08.2009, at 01:33, Eliot Miranda wrote:

On Wed, Aug 12, 2009 at 4:21 PM, Bert Freudenberg <[hidden email]> wrote:

On 13.08.2009, at 01:14, Eliot Miranda wrote:

On Wed, Aug 12, 2009 at 4:01 PM, Bert Freudenberg <[hidden email]> wrote:

But I fixed it anyway =)

I don't understand your banter^h^h^h^h^h^hemoticon!
D: D=
horror, disgust [1][3][4]
but =) ?? explain!

http://en.wikipedia.org/wiki/Emoticon#Western_style

I looked up =) in http://en.wikipedia.org/wiki/List_of_emoticons but it wasn't there (o.0)

Look in the link I gave.

yes, I followed it and found =) listed. But that two different pages on Wikipedia on emoticons exist and have different contents is cause for concern, hein?? (Hence (o.0), http://en.wikipedia.org/wiki/Emoticons#Common_eastern_examples ;) )

Also:

- Bert -

=) still makes me think of snowy on alcohol...