Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Chris Muller-3
I've definitely experienced the exact same problem, and got around it
the same way, restarting the method and proceeding.  Very strange.


On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]> wrote:

>
> Moving from vm-dev to squeak-dev because it is a trunk update stream question.
> Follow ups to squeak-dev please.
>
> Does anyone recognize this problem?
>
> We have an issue in trunk update processing that prevents a full update
> without manual intervention. This does not affect the 5.1 release, but
> it would be good to fix it so that the update from 5.0 to 5.1 can be
> reproduced in a continuous integration test.
>
> To reproduce: Start with Squeak5.0-15113.image, set the update URL preference
> to http://source.squeak.org/trunk, and do an "update from server".
>
> The error first appears when beginning to process the update-mt.378.mcm
> update map. The problem is related to updating the progress bar with
> ProgressInitiationException.
>
> When the failure occurs, the debugger (see attached image) seems to
> show an array access being interpreted as if the array was an instance
> of StrikeFont. Per Nicolas' note below, it may be related to showing
> the progress bar while updating it.
>
> I can restart the update process from the debugger from
> SystemProgressMorph>>position:label:min:max: in the stack frame, and the
> update proceeds. The error happens several more times, then at some point
> it seems to go away, so apparently the problem was fixed at some later
> point in the update stream.
>
> Does anyone recognize this problem? Can we work around it by changing one
> or more of the update maps?
>
> Thanks,
> Dave
>
>
> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
>>
>> yes, this is a problem I encountered both in 32 and 64 bits squeak while
>> updating.
>>
>> I think it's related to showing the progress bar while updating the
>> progress bar. It's a Squeak trunk update problem, not a VM problem.
>>
>> If some class layout change, but some block is still active (yeah, did you
>> see how many blocks we traverse just to change display of a bar...), then
>> the old block has invalid inst var offsets (offsets to the old object that
>> now has become a new object with new layout).
>>
>> In an interactive image, just close the debugger and relaunch the update.
>> In an headless automated process, well... We should have fixed the update
>> or start from a newer artifact...
>>
>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
>> >
>> > I just tried running the vmmaker-build script on my iMac with weirdly bad
>> > results.
>> >
>> > The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst trying
>> > to do the update part of the process I had several *very* odd failures
>> > where an object in a block was mistaken for nil during a message send. For
>> > example in a progress bar update (which I can???t provide a log for since the
>> > log got overwritten later) a line
>> > (bars at: index)
>> > lead to a nil is not indexable halt. Yet in inspecting the prior
>> > stackframe the bars object was a perfectly normal array of progress bars. A
>> > similar but not identical problem happened during another attempt.
>> >
>> > Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed when
>> > the vm exploded with no visible log.
>> >
>> > Now I???m attempting to repeat that with a new 5.1rc1 vm & already up to
>> > date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It got
>> > to the final save and quit but when I start the supposedly prepared image
>> > there is a problem with it trying to filein that file. I can???t tell right
>> > now if it relates to the manual filein I started or if something got added
>> > as a deferred action? Or maybe it???s a side-effect of the filein having a
>> > save & quit? Not come across this before.
>> >
>> >
>> >
>> > So far as I can tell it did complete the filein, so maybe all is well.
>> >
>

Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Levente Uzonyi
Same here. I think the code changes during the update and the old code
on the stack causes the problem.

Levente

On Sun, 21 Aug 2016, Chris Muller wrote:

>
> I've definitely experienced the exact same problem, and got around it
> the same way, restarting the method and proceeding.  Very strange.
>
>
> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]> wrote:
>>
>> Moving from vm-dev to squeak-dev because it is a trunk update stream question.
>> Follow ups to squeak-dev please.
>>
>> Does anyone recognize this problem?
>>
>> We have an issue in trunk update processing that prevents a full update
>> without manual intervention. This does not affect the 5.1 release, but
>> it would be good to fix it so that the update from 5.0 to 5.1 can be
>> reproduced in a continuous integration test.
>>
>> To reproduce: Start with Squeak5.0-15113.image, set the update URL preference
>> to http://source.squeak.org/trunk, and do an "update from server".
>>
>> The error first appears when beginning to process the update-mt.378.mcm
>> update map. The problem is related to updating the progress bar with
>> ProgressInitiationException.
>>
>> When the failure occurs, the debugger (see attached image) seems to
>> show an array access being interpreted as if the array was an instance
>> of StrikeFont. Per Nicolas' note below, it may be related to showing
>> the progress bar while updating it.
>>
>> I can restart the update process from the debugger from
>> SystemProgressMorph>>position:label:min:max: in the stack frame, and the
>> update proceeds. The error happens several more times, then at some point
>> it seems to go away, so apparently the problem was fixed at some later
>> point in the update stream.
>>
>> Does anyone recognize this problem? Can we work around it by changing one
>> or more of the update maps?
>>
>> Thanks,
>> Dave
>>
>>
>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
>>>
>>> yes, this is a problem I encountered both in 32 and 64 bits squeak while
>>> updating.
>>>
>>> I think it's related to showing the progress bar while updating the
>>> progress bar. It's a Squeak trunk update problem, not a VM problem.
>>>
>>> If some class layout change, but some block is still active (yeah, did you
>>> see how many blocks we traverse just to change display of a bar...), then
>>> the old block has invalid inst var offsets (offsets to the old object that
>>> now has become a new object with new layout).
>>>
>>> In an interactive image, just close the debugger and relaunch the update.
>>> In an headless automated process, well... We should have fixed the update
>>> or start from a newer artifact...
>>>
>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
>>>>
>>>> I just tried running the vmmaker-build script on my iMac with weirdly bad
>>>> results.
>>>>
>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst trying
>>>> to do the update part of the process I had several *very* odd failures
>>>> where an object in a block was mistaken for nil during a message send. For
>>>> example in a progress bar update (which I can???t provide a log for since the
>>>> log got overwritten later) a line
>>>> (bars at: index)
>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
>>>> stackframe the bars object was a perfectly normal array of progress bars. A
>>>> similar but not identical problem happened during another attempt.
>>>>
>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed when
>>>> the vm exploded with no visible log.
>>>>
>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm & already up to
>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It got
>>>> to the final save and quit but when I start the supposedly prepared image
>>>> there is a problem with it trying to filein that file. I can???t tell right
>>>> now if it relates to the manual filein I started or if something got added
>>>> as a deferred action? Or maybe it???s a side-effect of the filein having a
>>>> save & quit? Not come across this before.
>>>>
>>>>
>>>>
>>>> So far as I can tell it did complete the filein, so maybe all is well.
>>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Bert Freudenberg
Seen it, discussed with Marcel but we don't have a good idea yet how to work around it.

The only thing I can think of is to walk up the sender chain and restart some context above the one referencing the old instance layout. Maybe an exception handler in the updater that restarts updating in this specific case?

- Bert -


On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi <[hidden email]> wrote:

Same here. I think the code changes during the update and the old code on the stack causes the problem.

Levente


On Sun, 21 Aug 2016, Chris Muller wrote:


I've definitely experienced the exact same problem, and got around it
the same way, restarting the method and proceeding.  Very strange.


On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]> wrote:

Moving from vm-dev to squeak-dev because it is a trunk update stream question.
Follow ups to squeak-dev please.

Does anyone recognize this problem?

We have an issue in trunk update processing that prevents a full update
without manual intervention. This does not affect the 5.1 release, but
it would be good to fix it so that the update from 5.0 to 5.1 can be
reproduced in a continuous integration test.

To reproduce: Start with Squeak5.0-15113.image, set the update URL preference
to http://source.squeak.org/trunk, and do an "update from server".

The error first appears when beginning to process the update-mt.378.mcm
update map. The problem is related to updating the progress bar with
ProgressInitiationException.

When the failure occurs, the debugger (see attached image) seems to
show an array access being interpreted as if the array was an instance
of StrikeFont. Per Nicolas' note below, it may be related to showing
the progress bar while updating it.

I can restart the update process from the debugger from
SystemProgressMorph>>position:label:min:max: in the stack frame, and the
update proceeds. The error happens several more times, then at some point
it seems to go away, so apparently the problem was fixed at some later
point in the update stream.

Does anyone recognize this problem? Can we work around it by changing one
or more of the update maps?

Thanks,
Dave


On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:

yes, this is a problem I encountered both in 32 and 64 bits squeak while
updating.

I think it's related to showing the progress bar while updating the
progress bar. It's a Squeak trunk update problem, not a VM problem.

If some class layout change, but some block is still active (yeah, did you
see how many blocks we traverse just to change display of a bar...), then
the old block has invalid inst var offsets (offsets to the old object that
now has become a new object with new layout).

In an interactive image, just close the debugger and relaunch the update.
In an headless automated process, well... We should have fixed the update
or start from a newer artifact...

2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:

I just tried running the vmmaker-build script on my iMac with weirdly bad
results.

The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst trying
to do the update part of the process I had several *very* odd failures
where an object in a block was mistaken for nil during a message send. For
example in a progress bar update (which I can???t provide a log for since the
log got overwritten later) a line
(bars at: index)
lead to a nil is not indexable halt. Yet in inspecting the prior
stackframe the bars object was a perfectly normal array of progress bars. A
similar but not identical problem happened during another attempt.

Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed when
the vm exploded with no visible log.

Now I???m attempting to repeat that with a new 5.1rc1 vm & already up to
date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It got
to the final save and quit but when I start the supposedly prepared image
there is a problem with it trying to filein that file. I can???t tell right
now if it relates to the manual filein I started or if something got added
as a deferred action? Or maybe it???s a side-effect of the filein having a
save & quit? Not come across this before.



So far as I can tell it did complete the filein, so maybe all is well.






Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Chris Muller-3
This isn't the first time our changes have caused update hiccups.  I
think it just comes with the territory.  Easiest to just save off
another image that is beyond the problem and move on from there.

On Mon, Aug 22, 2016 at 7:36 AM, Bert Freudenberg <[hidden email]> wrote:

> Seen it, discussed with Marcel but we don't have a good idea yet how to work
> around it.
>
> The only thing I can think of is to walk up the sender chain and restart
> some context above the one referencing the old instance layout. Maybe an
> exception handler in the updater that restarts updating in this specific
> case?
>
> - Bert -
>
>
> On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi <[hidden email]>
> wrote:
>>
>>
>> Same here. I think the code changes during the update and the old code on
>> the stack causes the problem.
>>
>> Levente
>>
>>
>> On Sun, 21 Aug 2016, Chris Muller wrote:
>>
>>>
>>> I've definitely experienced the exact same problem, and got around it
>>> the same way, restarting the method and proceeding.  Very strange.
>>>
>>>
>>> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]>
>>> wrote:
>>>>
>>>>
>>>> Moving from vm-dev to squeak-dev because it is a trunk update stream
>>>> question.
>>>> Follow ups to squeak-dev please.
>>>>
>>>> Does anyone recognize this problem?
>>>>
>>>> We have an issue in trunk update processing that prevents a full update
>>>> without manual intervention. This does not affect the 5.1 release, but
>>>> it would be good to fix it so that the update from 5.0 to 5.1 can be
>>>> reproduced in a continuous integration test.
>>>>
>>>> To reproduce: Start with Squeak5.0-15113.image, set the update URL
>>>> preference
>>>> to http://source.squeak.org/trunk, and do an "update from server".
>>>>
>>>> The error first appears when beginning to process the update-mt.378.mcm
>>>> update map. The problem is related to updating the progress bar with
>>>> ProgressInitiationException.
>>>>
>>>> When the failure occurs, the debugger (see attached image) seems to
>>>> show an array access being interpreted as if the array was an instance
>>>> of StrikeFont. Per Nicolas' note below, it may be related to showing
>>>> the progress bar while updating it.
>>>>
>>>> I can restart the update process from the debugger from
>>>> SystemProgressMorph>>position:label:min:max: in the stack frame, and the
>>>> update proceeds. The error happens several more times, then at some
>>>> point
>>>> it seems to go away, so apparently the problem was fixed at some later
>>>> point in the update stream.
>>>>
>>>> Does anyone recognize this problem? Can we work around it by changing
>>>> one
>>>> or more of the update maps?
>>>>
>>>> Thanks,
>>>> Dave
>>>>
>>>>
>>>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
>>>>>
>>>>>
>>>>> yes, this is a problem I encountered both in 32 and 64 bits squeak
>>>>> while
>>>>> updating.
>>>>>
>>>>> I think it's related to showing the progress bar while updating the
>>>>> progress bar. It's a Squeak trunk update problem, not a VM problem.
>>>>>
>>>>> If some class layout change, but some block is still active (yeah, did
>>>>> you
>>>>> see how many blocks we traverse just to change display of a bar...),
>>>>> then
>>>>> the old block has invalid inst var offsets (offsets to the old object
>>>>> that
>>>>> now has become a new object with new layout).
>>>>>
>>>>> In an interactive image, just close the debugger and relaunch the
>>>>> update.
>>>>> In an headless automated process, well... We should have fixed the
>>>>> update
>>>>> or start from a newer artifact...
>>>>>
>>>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
>>>>>>
>>>>>>
>>>>>> I just tried running the vmmaker-build script on my iMac with weirdly
>>>>>> bad
>>>>>> results.
>>>>>>
>>>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst
>>>>>> trying
>>>>>> to do the update part of the process I had several *very* odd failures
>>>>>> where an object in a block was mistaken for nil during a message send.
>>>>>> For
>>>>>> example in a progress bar update (which I can???t provide a log for
>>>>>> since the
>>>>>> log got overwritten later) a line
>>>>>> (bars at: index)
>>>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
>>>>>> stackframe the bars object was a perfectly normal array of progress
>>>>>> bars. A
>>>>>> similar but not identical problem happened during another attempt.
>>>>>>
>>>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed
>>>>>> when
>>>>>> the vm exploded with no visible log.
>>>>>>
>>>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm & already up
>>>>>> to
>>>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It
>>>>>> got
>>>>>> to the final save and quit but when I start the supposedly prepared
>>>>>> image
>>>>>> there is a problem with it trying to filein that file. I can???t tell
>>>>>> right
>>>>>> now if it relates to the manual filein I started or if something got
>>>>>> added
>>>>>> as a deferred action? Or maybe it???s a side-effect of the filein
>>>>>> having a
>>>>>> save & quit? Not come across this before.
>>>>>>
>>>>>>
>>>>>>
>>>>>> So far as I can tell it did complete the filein, so maybe all is well.
>>>>>>
>>>>
>>>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Nicolas Cellier
There might be another possibility: destroy the progress bar in the preamble.
How to detect the progress bar is left as an exercize (translate: I just don't know, maybe also by walking the stack).
Reopen the progress bar in postscript.

2016-08-22 17:57 GMT+02:00 Chris Muller <[hidden email]>:
This isn't the first time our changes have caused update hiccups.  I
think it just comes with the territory.  Easiest to just save off
another image that is beyond the problem and move on from there.

On Mon, Aug 22, 2016 at 7:36 AM, Bert Freudenberg <[hidden email]> wrote:
> Seen it, discussed with Marcel but we don't have a good idea yet how to work
> around it.
>
> The only thing I can think of is to walk up the sender chain and restart
> some context above the one referencing the old instance layout. Maybe an
> exception handler in the updater that restarts updating in this specific
> case?
>
> - Bert -
>
>
> On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi <[hidden email]>
> wrote:
>>
>>
>> Same here. I think the code changes during the update and the old code on
>> the stack causes the problem.
>>
>> Levente
>>
>>
>> On Sun, 21 Aug 2016, Chris Muller wrote:
>>
>>>
>>> I've definitely experienced the exact same problem, and got around it
>>> the same way, restarting the method and proceeding.  Very strange.
>>>
>>>
>>> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]>
>>> wrote:
>>>>
>>>>
>>>> Moving from vm-dev to squeak-dev because it is a trunk update stream
>>>> question.
>>>> Follow ups to squeak-dev please.
>>>>
>>>> Does anyone recognize this problem?
>>>>
>>>> We have an issue in trunk update processing that prevents a full update
>>>> without manual intervention. This does not affect the 5.1 release, but
>>>> it would be good to fix it so that the update from 5.0 to 5.1 can be
>>>> reproduced in a continuous integration test.
>>>>
>>>> To reproduce: Start with Squeak5.0-15113.image, set the update URL
>>>> preference
>>>> to http://source.squeak.org/trunk, and do an "update from server".
>>>>
>>>> The error first appears when beginning to process the update-mt.378.mcm
>>>> update map. The problem is related to updating the progress bar with
>>>> ProgressInitiationException.
>>>>
>>>> When the failure occurs, the debugger (see attached image) seems to
>>>> show an array access being interpreted as if the array was an instance
>>>> of StrikeFont. Per Nicolas' note below, it may be related to showing
>>>> the progress bar while updating it.
>>>>
>>>> I can restart the update process from the debugger from
>>>> SystemProgressMorph>>position:label:min:max: in the stack frame, and the
>>>> update proceeds. The error happens several more times, then at some
>>>> point
>>>> it seems to go away, so apparently the problem was fixed at some later
>>>> point in the update stream.
>>>>
>>>> Does anyone recognize this problem? Can we work around it by changing
>>>> one
>>>> or more of the update maps?
>>>>
>>>> Thanks,
>>>> Dave
>>>>
>>>>
>>>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
>>>>>
>>>>>
>>>>> yes, this is a problem I encountered both in 32 and 64 bits squeak
>>>>> while
>>>>> updating.
>>>>>
>>>>> I think it's related to showing the progress bar while updating the
>>>>> progress bar. It's a Squeak trunk update problem, not a VM problem.
>>>>>
>>>>> If some class layout change, but some block is still active (yeah, did
>>>>> you
>>>>> see how many blocks we traverse just to change display of a bar...),
>>>>> then
>>>>> the old block has invalid inst var offsets (offsets to the old object
>>>>> that
>>>>> now has become a new object with new layout).
>>>>>
>>>>> In an interactive image, just close the debugger and relaunch the
>>>>> update.
>>>>> In an headless automated process, well... We should have fixed the
>>>>> update
>>>>> or start from a newer artifact...
>>>>>
>>>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
>>>>>>
>>>>>>
>>>>>> I just tried running the vmmaker-build script on my iMac with weirdly
>>>>>> bad
>>>>>> results.
>>>>>>
>>>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst
>>>>>> trying
>>>>>> to do the update part of the process I had several *very* odd failures
>>>>>> where an object in a block was mistaken for nil during a message send.
>>>>>> For
>>>>>> example in a progress bar update (which I can???t provide a log for
>>>>>> since the
>>>>>> log got overwritten later) a line
>>>>>> (bars at: index)
>>>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
>>>>>> stackframe the bars object was a perfectly normal array of progress
>>>>>> bars. A
>>>>>> similar but not identical problem happened during another attempt.
>>>>>>
>>>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed
>>>>>> when
>>>>>> the vm exploded with no visible log.
>>>>>>
>>>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm & already up
>>>>>> to
>>>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It
>>>>>> got
>>>>>> to the final save and quit but when I start the supposedly prepared
>>>>>> image
>>>>>> there is a problem with it trying to filein that file. I can???t tell
>>>>>> right
>>>>>> now if it relates to the manual filein I started or if something got
>>>>>> added
>>>>>> as a deferred action? Or maybe it???s a side-effect of the filein
>>>>>> having a
>>>>>> save & quit? Not come across this before.
>>>>>>
>>>>>>
>>>>>>
>>>>>> So far as I can tell it did complete the filein, so maybe all is well.
>>>>>>
>>>>
>>>
>
>
>
>




Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

David T. Lewis
In reply to this post by Bert Freudenberg
On Mon, Aug 22, 2016 at 02:36:09PM +0200, Bert Freudenberg wrote:

> Seen it, discussed with Marcel but we don't have a good idea yet how to
> work around it.
>
> The only thing I can think of is to walk up the sender chain and restart
> some context above the one referencing the old instance layout. Maybe an
> exception handler in the updater that restarts updating in this specific
> case?
>
> - Bert -
>

I don't see a good place to put an exception handler, at least not without
making things even more complicated than they already are.

It looks like the the direct reference to instance var 'bars' is where we
encounter the problem. Presumably the old code in a block is refering to
an ivar slot in SystemProgressMorph that used to point to the 'vars' array,
but that is now pointing to the 'font' ivar.

And indeed I see now that Marcel found and fixed this issue in Morphic-mt.1198,
so that accessor methods will now be used instead of direct references to
the variables.

But we still have the problem of a SystemProgressMorph evaluating an older
block that was created with an earlier version of the
SystemProgressMorph>>position:label:min:max: method, so the update stream
processing still fails if the update processing was started before
Morphic-mt.1198 was loaded.

A partial workaround is to modify #position:label:min:max: to use an accessor
for 'bars' prior to starting the update processing. This prevents the error
condition that we are discussing here, but unfortunately it also results
in a couple of merge dialogs for the modified methods, so the update still
requires manual intervention.

Dave


>
> On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi <[hidden email]>
> wrote:
>
> >
> > Same here. I think the code changes during the update and the old code on
> > the stack causes the problem.
> >
> > Levente
> >
> >
> > On Sun, 21 Aug 2016, Chris Muller wrote:
> >
> >
> >> I've definitely experienced the exact same problem, and got around it
> >> the same way, restarting the method and proceeding.  Very strange.
> >>
> >>
> >> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]>
> >> wrote:
> >>
> >>>
> >>> Moving from vm-dev to squeak-dev because it is a trunk update stream
> >>> question.
> >>> Follow ups to squeak-dev please.
> >>>
> >>> Does anyone recognize this problem?
> >>>
> >>> We have an issue in trunk update processing that prevents a full update
> >>> without manual intervention. This does not affect the 5.1 release, but
> >>> it would be good to fix it so that the update from 5.0 to 5.1 can be
> >>> reproduced in a continuous integration test.
> >>>
> >>> To reproduce: Start with Squeak5.0-15113.image, set the update URL
> >>> preference
> >>> to http://source.squeak.org/trunk, and do an "update from server".
> >>>
> >>> The error first appears when beginning to process the update-mt.378.mcm
> >>> update map. The problem is related to updating the progress bar with
> >>> ProgressInitiationException.
> >>>
> >>> When the failure occurs, the debugger (see attached image) seems to
> >>> show an array access being interpreted as if the array was an instance
> >>> of StrikeFont. Per Nicolas' note below, it may be related to showing
> >>> the progress bar while updating it.
> >>>
> >>> I can restart the update process from the debugger from
> >>> SystemProgressMorph>>position:label:min:max: in the stack frame, and the
> >>> update proceeds. The error happens several more times, then at some point
> >>> it seems to go away, so apparently the problem was fixed at some later
> >>> point in the update stream.
> >>>
> >>> Does anyone recognize this problem? Can we work around it by changing one
> >>> or more of the update maps?
> >>>
> >>> Thanks,
> >>> Dave
> >>>
> >>>
> >>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
> >>>
> >>>>
> >>>> yes, this is a problem I encountered both in 32 and 64 bits squeak while
> >>>> updating.
> >>>>
> >>>> I think it's related to showing the progress bar while updating the
> >>>> progress bar. It's a Squeak trunk update problem, not a VM problem.
> >>>>
> >>>> If some class layout change, but some block is still active (yeah, did
> >>>> you
> >>>> see how many blocks we traverse just to change display of a bar...),
> >>>> then
> >>>> the old block has invalid inst var offsets (offsets to the old object
> >>>> that
> >>>> now has become a new object with new layout).
> >>>>
> >>>> In an interactive image, just close the debugger and relaunch the
> >>>> update.
> >>>> In an headless automated process, well... We should have fixed the
> >>>> update
> >>>> or start from a newer artifact...
> >>>>
> >>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
> >>>>
> >>>>>
> >>>>> I just tried running the vmmaker-build script on my iMac with weirdly
> >>>>> bad
> >>>>> results.
> >>>>>
> >>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst
> >>>>> trying
> >>>>> to do the update part of the process I had several *very* odd failures
> >>>>> where an object in a block was mistaken for nil during a message send.
> >>>>> For
> >>>>> example in a progress bar update (which I can???t provide a log for
> >>>>> since the
> >>>>> log got overwritten later) a line
> >>>>> (bars at: index)
> >>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
> >>>>> stackframe the bars object was a perfectly normal array of progress
> >>>>> bars. A
> >>>>> similar but not identical problem happened during another attempt.
> >>>>>
> >>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed
> >>>>> when
> >>>>> the vm exploded with no visible log.
> >>>>>
> >>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm & already up
> >>>>> to
> >>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It
> >>>>> got
> >>>>> to the final save and quit but when I start the supposedly prepared
> >>>>> image
> >>>>> there is a problem with it trying to filein that file. I can???t tell
> >>>>> right
> >>>>> now if it relates to the manual filein I started or if something got
> >>>>> added
> >>>>> as a deferred action? Or maybe it???s a side-effect of the filein
> >>>>> having a
> >>>>> save & quit? Not come across this before.
> >>>>>
> >>>>>
> >>>>>
> >>>>> So far as I can tell it did complete the filein, so maybe all is well.
> >>>>>
> >>>>>
> >>>
> >>

>


Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Nicolas Cellier
There's yet another possibility, in a preamble:
- clone the old SystemProgressMorph class,
- mutate all instances to the clone class

Don't use class rename and class removal, because the system will do too much clean up.
instead hack at a lower level

2016-08-23 4:03 GMT+02:00 David T. Lewis <[hidden email]>:
On Mon, Aug 22, 2016 at 02:36:09PM +0200, Bert Freudenberg wrote:
> Seen it, discussed with Marcel but we don't have a good idea yet how to
> work around it.
>
> The only thing I can think of is to walk up the sender chain and restart
> some context above the one referencing the old instance layout. Maybe an
> exception handler in the updater that restarts updating in this specific
> case?
>
> - Bert -
>

I don't see a good place to put an exception handler, at least not without
making things even more complicated than they already are.

It looks like the the direct reference to instance var 'bars' is where we
encounter the problem. Presumably the old code in a block is refering to
an ivar slot in SystemProgressMorph that used to point to the 'vars' array,
but that is now pointing to the 'font' ivar.

And indeed I see now that Marcel found and fixed this issue in Morphic-mt.1198,
so that accessor methods will now be used instead of direct references to
the variables.

But we still have the problem of a SystemProgressMorph evaluating an older
block that was created with an earlier version of the
SystemProgressMorph>>position:label:min:max: method, so the update stream
processing still fails if the update processing was started before
Morphic-mt.1198 was loaded.

A partial workaround is to modify #position:label:min:max: to use an accessor
for 'bars' prior to starting the update processing. This prevents the error
condition that we are discussing here, but unfortunately it also results
in a couple of merge dialogs for the modified methods, so the update still
requires manual intervention.

Dave


>
> On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi <[hidden email]>
> wrote:
>
> >
> > Same here. I think the code changes during the update and the old code on
> > the stack causes the problem.
> >
> > Levente
> >
> >
> > On Sun, 21 Aug 2016, Chris Muller wrote:
> >
> >
> >> I've definitely experienced the exact same problem, and got around it
> >> the same way, restarting the method and proceeding.  Very strange.
> >>
> >>
> >> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]>
> >> wrote:
> >>
> >>>
> >>> Moving from vm-dev to squeak-dev because it is a trunk update stream
> >>> question.
> >>> Follow ups to squeak-dev please.
> >>>
> >>> Does anyone recognize this problem?
> >>>
> >>> We have an issue in trunk update processing that prevents a full update
> >>> without manual intervention. This does not affect the 5.1 release, but
> >>> it would be good to fix it so that the update from 5.0 to 5.1 can be
> >>> reproduced in a continuous integration test.
> >>>
> >>> To reproduce: Start with Squeak5.0-15113.image, set the update URL
> >>> preference
> >>> to http://source.squeak.org/trunk, and do an "update from server".
> >>>
> >>> The error first appears when beginning to process the update-mt.378.mcm
> >>> update map. The problem is related to updating the progress bar with
> >>> ProgressInitiationException.
> >>>
> >>> When the failure occurs, the debugger (see attached image) seems to
> >>> show an array access being interpreted as if the array was an instance
> >>> of StrikeFont. Per Nicolas' note below, it may be related to showing
> >>> the progress bar while updating it.
> >>>
> >>> I can restart the update process from the debugger from
> >>> SystemProgressMorph>>position:label:min:max: in the stack frame, and the
> >>> update proceeds. The error happens several more times, then at some point
> >>> it seems to go away, so apparently the problem was fixed at some later
> >>> point in the update stream.
> >>>
> >>> Does anyone recognize this problem? Can we work around it by changing one
> >>> or more of the update maps?
> >>>
> >>> Thanks,
> >>> Dave
> >>>
> >>>
> >>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
> >>>
> >>>>
> >>>> yes, this is a problem I encountered both in 32 and 64 bits squeak while
> >>>> updating.
> >>>>
> >>>> I think it's related to showing the progress bar while updating the
> >>>> progress bar. It's a Squeak trunk update problem, not a VM problem.
> >>>>
> >>>> If some class layout change, but some block is still active (yeah, did
> >>>> you
> >>>> see how many blocks we traverse just to change display of a bar...),
> >>>> then
> >>>> the old block has invalid inst var offsets (offsets to the old object
> >>>> that
> >>>> now has become a new object with new layout).
> >>>>
> >>>> In an interactive image, just close the debugger and relaunch the
> >>>> update.
> >>>> In an headless automated process, well... We should have fixed the
> >>>> update
> >>>> or start from a newer artifact...
> >>>>
> >>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
> >>>>
> >>>>>
> >>>>> I just tried running the vmmaker-build script on my iMac with weirdly
> >>>>> bad
> >>>>> results.
> >>>>>
> >>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst
> >>>>> trying
> >>>>> to do the update part of the process I had several *very* odd failures
> >>>>> where an object in a block was mistaken for nil during a message send.
> >>>>> For
> >>>>> example in a progress bar update (which I can???t provide a log for
> >>>>> since the
> >>>>> log got overwritten later) a line
> >>>>> (bars at: index)
> >>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
> >>>>> stackframe the bars object was a perfectly normal array of progress
> >>>>> bars. A
> >>>>> similar but not identical problem happened during another attempt.
> >>>>>
> >>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed
> >>>>> when
> >>>>> the vm exploded with no visible log.
> >>>>>
> >>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm & already up
> >>>>> to
> >>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd manner. It
> >>>>> got
> >>>>> to the final save and quit but when I start the supposedly prepared
> >>>>> image
> >>>>> there is a problem with it trying to filein that file. I can???t tell
> >>>>> right
> >>>>> now if it relates to the manual filein I started or if something got
> >>>>> added
> >>>>> as a deferred action? Or maybe it???s a side-effect of the filein
> >>>>> having a
> >>>>> save & quit? Not come across this before.
> >>>>>
> >>>>>
> >>>>>
> >>>>> So far as I can tell it did complete the filein, so maybe all is well.
> >>>>>
> >>>>>
> >>>
> >>

>





Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

marcel.taeumel
Nicolas Cellier wrote
There's yet another possibility, in a preamble:
- clone the old SystemProgressMorph class,
- mutate all instances to the clone class

Don't use class rename and class removal, because the system will do too
much clean up.
instead hack at a lower level

2016-08-23 4:03 GMT+02:00 David T. Lewis <[hidden email]>:

> On Mon, Aug 22, 2016 at 02:36:09PM +0200, Bert Freudenberg wrote:
> > Seen it, discussed with Marcel but we don't have a good idea yet how to
> > work around it.
> >
> > The only thing I can think of is to walk up the sender chain and restart
> > some context above the one referencing the old instance layout. Maybe an
> > exception handler in the updater that restarts updating in this specific
> > case?
> >
> > - Bert -
> >
>
> I don't see a good place to put an exception handler, at least not without
> making things even more complicated than they already are.
>
> It looks like the the direct reference to instance var 'bars' is where we
> encounter the problem. Presumably the old code in a block is refering to
> an ivar slot in SystemProgressMorph that used to point to the 'vars' array,
> but that is now pointing to the 'font' ivar.
>
> And indeed I see now that Marcel found and fixed this issue in
> Morphic-mt.1198,
> so that accessor methods will now be used instead of direct references to
> the variables.
>
> But we still have the problem of a SystemProgressMorph evaluating an older
> block that was created with an earlier version of the
> SystemProgressMorph>>position:label:min:max: method, so the update stream
> processing still fails if the update processing was started before
> Morphic-mt.1198 was loaded.
>
> A partial workaround is to modify #position:label:min:max: to use an
> accessor
> for 'bars' prior to starting the update processing. This prevents the error
> condition that we are discussing here, but unfortunately it also results
> in a couple of merge dialogs for the modified methods, so the update still
> requires manual intervention.
>
> Dave
>
>
> >
> > On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi <[hidden email]>
> > wrote:
> >
> > >
> > > Same here. I think the code changes during the update and the old code
> on
> > > the stack causes the problem.
> > >
> > > Levente
> > >
> > >
> > > On Sun, 21 Aug 2016, Chris Muller wrote:
> > >
> > >
> > >> I've definitely experienced the exact same problem, and got around it
> > >> the same way, restarting the method and proceeding.  Very strange.
> > >>
> > >>
> > >> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis <[hidden email]>
> > >> wrote:
> > >>
> > >>>
> > >>> Moving from vm-dev to squeak-dev because it is a trunk update stream
> > >>> question.
> > >>> Follow ups to squeak-dev please.
> > >>>
> > >>> Does anyone recognize this problem?
> > >>>
> > >>> We have an issue in trunk update processing that prevents a full
> update
> > >>> without manual intervention. This does not affect the 5.1 release,
> but
> > >>> it would be good to fix it so that the update from 5.0 to 5.1 can be
> > >>> reproduced in a continuous integration test.
> > >>>
> > >>> To reproduce: Start with Squeak5.0-15113.image, set the update URL
> > >>> preference
> > >>> to http://source.squeak.org/trunk, and do an "update from server".
> > >>>
> > >>> The error first appears when beginning to process the
> update-mt.378.mcm
> > >>> update map. The problem is related to updating the progress bar with
> > >>> ProgressInitiationException.
> > >>>
> > >>> When the failure occurs, the debugger (see attached image) seems to
> > >>> show an array access being interpreted as if the array was an
> instance
> > >>> of StrikeFont. Per Nicolas' note below, it may be related to showing
> > >>> the progress bar while updating it.
> > >>>
> > >>> I can restart the update process from the debugger from
> > >>> SystemProgressMorph>>position:label:min:max: in the stack frame,
> and the
> > >>> update proceeds. The error happens several more times, then at some
> point
> > >>> it seems to go away, so apparently the problem was fixed at some
> later
> > >>> point in the update stream.
> > >>>
> > >>> Does anyone recognize this problem? Can we work around it by
> changing one
> > >>> or more of the update maps?
> > >>>
> > >>> Thanks,
> > >>> Dave
> > >>>
> > >>>
> > >>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
> > >>>
> > >>>>
> > >>>> yes, this is a problem I encountered both in 32 and 64 bits squeak
> while
> > >>>> updating.
> > >>>>
> > >>>> I think it's related to showing the progress bar while updating the
> > >>>> progress bar. It's a Squeak trunk update problem, not a VM problem.
> > >>>>
> > >>>> If some class layout change, but some block is still active (yeah,
> did
> > >>>> you
> > >>>> see how many blocks we traverse just to change display of a bar...),
> > >>>> then
> > >>>> the old block has invalid inst var offsets (offsets to the old
> object
> > >>>> that
> > >>>> now has become a new object with new layout).
> > >>>>
> > >>>> In an interactive image, just close the debugger and relaunch the
> > >>>> update.
> > >>>> In an headless automated process, well... We should have fixed the
> > >>>> update
> > >>>> or start from a newer artifact...
> > >>>>
> > >>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge <[hidden email]>:
> > >>>>
> > >>>>>
> > >>>>> I just tried running the vmmaker-build script on my iMac with
> weirdly
> > >>>>> bad
> > >>>>> results.
> > >>>>>
> > >>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst
> > >>>>> trying
> > >>>>> to do the update part of the process I had several *very* odd
> failures
> > >>>>> where an object in a block was mistaken for nil during a message
> send.
> > >>>>> For
> > >>>>> example in a progress bar update (which I can???t provide a log for
> > >>>>> since the
> > >>>>> log got overwritten later) a line
> > >>>>> (bars at: index)
> > >>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
> > >>>>> stackframe the bars object was a perfectly normal array of progress
> > >>>>> bars. A
> > >>>>> similar but not identical problem happened during another attempt.
> > >>>>>
> > >>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file failed
> > >>>>> when
> > >>>>> the vm exploded with no visible log.
> > >>>>>
> > >>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm & already
> up
> > >>>>> to
> > >>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd manner.
> It
> > >>>>> got
> > >>>>> to the final save and quit but when I start the supposedly prepared
> > >>>>> image
> > >>>>> there is a problem with it trying to filein that file. I can???t
> tell
> > >>>>> right
> > >>>>> now if it relates to the manual filein I started or if something
> got
> > >>>>> added
> > >>>>> as a deferred action? Or maybe it???s a side-effect of the filein
> > >>>>> having a
> > >>>>> save & quit? Not come across this before.
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> So far as I can tell it did complete the filein, so maybe all is
> well.
> > >>>>>
> > >>>>>
> > >>>
> > >>
>
> >
>
>
>
Hi, there.

It is, unfortunately, not possible to turn back time and rewrite history. ;-) There is several important code that makes excessive use of instVar accesses such as in SystemProgressMorph or HandMorph. Making any change in their class layout while using some of their methods will provoke a strange error. This could only be changed by a VM that keeps old class layouts/schemata around for some longer time until there is no old compiled method on any stack anymore.

Usually, restarting the update process helps here. There is no way to fix that in order to interactively update from 5.0 to 5.1 without any errors.

However, we have an update script that updates build 15113, which is effectively Squeak 5.0 to the latest trunk. Here is the script:
https://github.com/squeak-smalltalk/squeak-app/blob/master/prepare_image.st
And here is the image build 15113:
http://files.squeak.org/base/Squeak-trunk/base.zip

We use that the create bundles with the latest alpha, beta, rc versions.

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Levente Uzonyi
It would be possible to start the update with the required error handlers
around it, but unless you really had to do it, you're better off using a
fixed image.

Levente

On Tue, 23 Aug 2016, marcel.taeumel wrote:

> Nicolas Cellier wrote
>> There's yet another possibility, in a preamble:
>> - clone the old SystemProgressMorph class,
>> - mutate all instances to the clone class
>>
>> Don't use class rename and class removal, because the system will do too
>> much clean up.
>> instead hack at a lower level
>>
>> 2016-08-23 4:03 GMT+02:00 David T. Lewis &lt;
>
>> lewis@.msen
>
>> &gt;:
>>
>>> On Mon, Aug 22, 2016 at 02:36:09PM +0200, Bert Freudenberg wrote:
>>>> Seen it, discussed with Marcel but we don't have a good idea yet how to
>>>> work around it.
>>>>
>>>> The only thing I can think of is to walk up the sender chain and
>>> restart
>>>> some context above the one referencing the old instance layout. Maybe
>>> an
>>>> exception handler in the updater that restarts updating in this
>>> specific
>>>> case?
>>>>
>>>> - Bert -
>>>>
>>>
>>> I don't see a good place to put an exception handler, at least not
>>> without
>>> making things even more complicated than they already are.
>>>
>>> It looks like the the direct reference to instance var 'bars' is where we
>>> encounter the problem. Presumably the old code in a block is refering to
>>> an ivar slot in SystemProgressMorph that used to point to the 'vars'
>>> array,
>>> but that is now pointing to the 'font' ivar.
>>>
>>> And indeed I see now that Marcel found and fixed this issue in
>>> Morphic-mt.1198,
>>> so that accessor methods will now be used instead of direct references to
>>> the variables.
>>>
>>> But we still have the problem of a SystemProgressMorph evaluating an
>>> older
>>> block that was created with an earlier version of the
>>> SystemProgressMorph>>position:label:min:max: method, so the update stream
>>> processing still fails if the update processing was started before
>>> Morphic-mt.1198 was loaded.
>>>
>>> A partial workaround is to modify #position:label:min:max: to use an
>>> accessor
>>> for 'bars' prior to starting the update processing. This prevents the
>>> error
>>> condition that we are discussing here, but unfortunately it also results
>>> in a couple of merge dialogs for the modified methods, so the update
>>> still
>>> requires manual intervention.
>>>
>>> Dave
>>>
>>>
>>>>
>>>> On Sun, Aug 21, 2016 at 10:55 PM, Levente Uzonyi &lt;
>
>> leves@.elte
>
>> &gt;
>>>> wrote:
>>>>
>>>>>
>>>>> Same here. I think the code changes during the update and the old
>>> code
>>> on
>>>>> the stack causes the problem.
>>>>>
>>>>> Levente
>>>>>
>>>>>
>>>>> On Sun, 21 Aug 2016, Chris Muller wrote:
>>>>>
>>>>>
>>>>>> I've definitely experienced the exact same problem, and got around
>>> it
>>>>>> the same way, restarting the method and proceeding.  Very strange.
>>>>>>
>>>>>>
>>>>>> On Sun, Aug 21, 2016 at 3:13 PM, David T. Lewis &lt;
>
>> lewis@.msen
>
>> &gt;
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>> Moving from vm-dev to squeak-dev because it is a trunk update
>>> stream
>>>>>>> question.
>>>>>>> Follow ups to squeak-dev please.
>>>>>>>
>>>>>>> Does anyone recognize this problem?
>>>>>>>
>>>>>>> We have an issue in trunk update processing that prevents a full
>>> update
>>>>>>> without manual intervention. This does not affect the 5.1 release,
>>> but
>>>>>>> it would be good to fix it so that the update from 5.0 to 5.1 can
>>> be
>>>>>>> reproduced in a continuous integration test.
>>>>>>>
>>>>>>> To reproduce: Start with Squeak5.0-15113.image, set the update URL
>>>>>>> preference
>>>>>>> to http://source.squeak.org/trunk, and do an "update from server".
>>>>>>>
>>>>>>> The error first appears when beginning to process the
>>> update-mt.378.mcm
>>>>>>> update map. The problem is related to updating the progress bar
>>> with
>>>>>>> ProgressInitiationException.
>>>>>>>
>>>>>>> When the failure occurs, the debugger (see attached image) seems to
>>>>>>> show an array access being interpreted as if the array was an
>>> instance
>>>>>>> of StrikeFont. Per Nicolas' note below, it may be related to
>>> showing
>>>>>>> the progress bar while updating it.
>>>>>>>
>>>>>>> I can restart the update process from the debugger from
>>>>>>> SystemProgressMorph>>position:label:min:max: in the stack frame,
>>> and the
>>>>>>> update proceeds. The error happens several more times, then at some
>>> point
>>>>>>> it seems to go away, so apparently the problem was fixed at some
>>> later
>>>>>>> point in the update stream.
>>>>>>>
>>>>>>> Does anyone recognize this problem? Can we work around it by
>>> changing one
>>>>>>> or more of the update maps?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Dave
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Aug 19, 2016 at 10:30:30PM +0200, Nicolas Cellier wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> yes, this is a problem I encountered both in 32 and 64 bits squeak
>>> while
>>>>>>>> updating.
>>>>>>>>
>>>>>>>> I think it's related to showing the progress bar while updating
>>> the
>>>>>>>> progress bar. It's a Squeak trunk update problem, not a VM
>>> problem.
>>>>>>>>
>>>>>>>> If some class layout change, but some block is still active (yeah,
>>> did
>>>>>>>> you
>>>>>>>> see how many blocks we traverse just to change display of a
>>> bar...),
>>>>>>>> then
>>>>>>>> the old block has invalid inst var offsets (offsets to the old
>>> object
>>>>>>>> that
>>>>>>>> now has become a new object with new layout).
>>>>>>>>
>>>>>>>> In an interactive image, just close the debugger and relaunch the
>>>>>>>> update.
>>>>>>>> In an headless automated process, well... We should have fixed the
>>>>>>>> update
>>>>>>>> or start from a newer artifact...
>>>>>>>>
>>>>>>>> 2016-08-19 21:32 GMT+02:00 tim Rowledge &lt;
>
>> tim@
>
>> &gt;:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I just tried running the vmmaker-build script on my iMac with
>>> weirdly
>>>>>>>>> bad
>>>>>>>>> results.
>>>>>>>>>
>>>>>>>>> The vm it downloads is the CogSpur.app-16.18.3692 version. Whilst
>>>>>>>>> trying
>>>>>>>>> to do the update part of the process I had several *very* odd
>>> failures
>>>>>>>>> where an object in a block was mistaken for nil during a message
>>> send.
>>>>>>>>> For
>>>>>>>>> example in a progress bar update (which I can???t provide a log
>>> for
>>>>>>>>> since the
>>>>>>>>> log got overwritten later) a line
>>>>>>>>> (bars at: index)
>>>>>>>>> lead to a nil is not indexable halt. Yet in inspecting the prior
>>>>>>>>> stackframe the bars object was a perfectly normal array of
>>> progress
>>>>>>>>> bars. A
>>>>>>>>> similar but not identical problem happened during another
>>> attempt.
>>>>>>>>>
>>>>>>>>> Attempting to filein the BuildSqueakSpurTrunkVMMaker.st file
>>> failed
>>>>>>>>> when
>>>>>>>>> the vm exploded with no visible log.
>>>>>>>>>
>>>>>>>>> Now I???m attempting to repeat that with a new 5.1rc1 vm &
>>> already
>>> up
>>>>>>>>> to
>>>>>>>>> date 5.1rc1 image. Which has sorta-worked, in a rather odd
>>> manner.
>>> It
>>>>>>>>> got
>>>>>>>>> to the final save and quit but when I start the supposedly
>>> prepared
>>>>>>>>> image
>>>>>>>>> there is a problem with it trying to filein that file. I can???t
>>> tell
>>>>>>>>> right
>>>>>>>>> now if it relates to the manual filein I started or if something
>>> got
>>>>>>>>> added
>>>>>>>>> as a deferred action? Or maybe it???s a side-effect of the filein
>>>>>>>>> having a
>>>>>>>>> save & quit? Not come across this before.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So far as I can tell it did complete the filein, so maybe all is
>>> well.
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>
>>>>
>>>
>>>
>>>
>
> Hi, there.
>
> It is, unfortunately, not possible to turn back time and rewrite history.
> ;-) There is several important code that makes excessive use of instVar
> accesses such as in SystemProgressMorph or HandMorph. Making any change in
> their class layout while using some of their methods will provoke a strange
> error. This could only be changed by a VM that keeps old class
> layouts/schemata around for some longer time until there is no old compiled
> method on any stack anymore.
>
> Usually, restarting the update process helps here. There is no way to fix
> that in order to interactively update from 5.0 to 5.1 without any errors.
>
> However, we have an update script that updates build 15113, which is
> effectively Squeak 5.0 to the latest trunk. Here is the script:
> https://github.com/squeak-smalltalk/squeak-app/blob/master/prepare_image.st
> And here is the image build 15113:
> http://files.squeak.org/base/Squeak-trunk/base.zip
>
> We use that the create bundles with the latest alpha, beta, rc versions.
>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Re-Does-anyone-recognize-this-glitch-in-our-trunk-update-processing-was-Vm-dev-buildspurtrunkvmmaker-tp4912130p4912305.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Bert Freudenberg
In reply to this post by marcel.taeumel
On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel <[hidden email]> wrote:

Hi, there.

It is, unfortunately, not possible to turn back time and rewrite history. ;-)

... unless we're dealing with software. Oh wait, we are ;)

There is several important code that makes excessive use of instVar
accesses such as in SystemProgressMorph or HandMorph. Making any change in
their class layout while using some of their methods will provoke a strange
error. This could only be changed by a VM that keeps old class
layouts/schemata around for some longer time until there is no old compiled
method on any stack anymore.

Class layout changes and instance migration is handled fully in the image by class ClassBuilder. The VM is not at fault here.
 
Usually, restarting the update process helps here. There is no way to fix
that in order to interactively update from 5.0 to 5.1 without any errors.

We can retroactively change the config map that introduced the problem. I'm certain it would be possible to fix, but I am not entirely sure it's worth the trouble.

- Bert - 


Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

marcel.taeumel
Bert Freudenberg wrote
On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel <[hidden email]>
wrote:

>
> Hi, there.
>
> It is, unfortunately, not possible to turn back time and rewrite history.
> ;-)
>

... unless we're dealing with software. Oh wait, we are ;)

There is several important code that makes excessive use of instVar
> accesses such as in SystemProgressMorph or HandMorph. Making any change in
> their class layout while using some of their methods will provoke a strange
> error. This could only be changed by a VM that keeps old class
> layouts/schemata around for some longer time until there is no old compiled
> method on any stack anymore.


Class layout changes and instance migration is handled fully in the image
by class ClassBuilder. The VM is not at fault here.


> Usually, restarting the update process helps here. There is no way to fix
> that in order to interactively update from 5.0 to 5.1 without any errors.
>

We can retroactively change the config map that introduced the problem. I'm
certain it would be possible to fix, but I am not entirely sure it's worth
the trouble.

- Bert -
Hi Bert,

the config map will not help, we would have to change numerous MCZs since point X in time to rewrite history. :) "Oh, would we have always been using accessors here instead of instVar accessess..."

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

marcel.taeumel
marcel.taeumel wrote
Bert Freudenberg wrote
On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel <[hidden email]>
wrote:

>
> Hi, there.
>
> It is, unfortunately, not possible to turn back time and rewrite history.
> ;-)
>

... unless we're dealing with software. Oh wait, we are ;)

There is several important code that makes excessive use of instVar
> accesses such as in SystemProgressMorph or HandMorph. Making any change in
> their class layout while using some of their methods will provoke a strange
> error. This could only be changed by a VM that keeps old class
> layouts/schemata around for some longer time until there is no old compiled
> method on any stack anymore.


Class layout changes and instance migration is handled fully in the image
by class ClassBuilder. The VM is not at fault here.


> Usually, restarting the update process helps here. There is no way to fix
> that in order to interactively update from 5.0 to 5.1 without any errors.
>

We can retroactively change the config map that introduced the problem. I'm
certain it would be possible to fix, but I am not entirely sure it's worth
the trouble.

- Bert -
Hi Bert,

the config map will not help, we would have to change numerous MCZs since point X in time to rewrite history. :) "Oh, would we have always been using accessors here instead of instVar accessess..."

Best,
Marcel
...even then, how do you update the code in image Y, which is going to be updated before it is updated? This is not even possible for every user's images out there... :-/

The Java Hotspot VM has a mapping table for changed instVar accesses. Maybe we could use this trick, too? Still, would only applies to future versions. We cannot rewrite history because it is not only one piece of software in a single version control system with a single version checked out. :)

Best,
Marcel
Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Nicolas Cellier
Or would this even more simple hack work? Put in the preamble something like:

Smalltalk at: SystemProgressMorph put: SystemProgressMorph copy.

This way, the old instances should not be mutated...

2016-08-24 7:06 GMT+02:00 marcel.taeumel <[hidden email]>:
marcel.taeumel wrote
>
> Bert Freudenberg wrote
>> On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel &lt;

>> Marcel.Taeumel@

>> &gt;
>> wrote:
>>
>>>
>>> Hi, there.
>>>
>>> It is, unfortunately, not possible to turn back time and rewrite
>>> history.
>>> ;-)
>>>
>>
>> ... unless we're dealing with software. Oh wait, we are ;)
>>
>> There is several important code that makes excessive use of instVar
>>> accesses such as in SystemProgressMorph or HandMorph. Making any change
>>> in
>>> their class layout while using some of their methods will provoke a
>>> strange
>>> error. This could only be changed by a VM that keeps old class
>>> layouts/schemata around for some longer time until there is no old
>>> compiled
>>> method on any stack anymore.
>>
>>
>> Class layout changes and instance migration is handled fully in the image
>> by class ClassBuilder. The VM is not at fault here.
>>
>>
>>> Usually, restarting the update process helps here. There is no way to
>>> fix
>>> that in order to interactively update from 5.0 to 5.1 without any
>>> errors.
>>>
>>
>> We can retroactively change the config map that introduced the problem.
>> I'm
>> certain it would be possible to fix, but I am not entirely sure it's
>> worth
>> the trouble.
>>
>> - Bert -
> Hi Bert,
>
> the config map will not help, we would have to change numerous MCZs since
> point X in time to rewrite history. :) "Oh, would we have always been
> using accessors here instead of instVar accessess..."
>
> Best,
> Marcel

...even then, how do you update the code in image Y, which is going to be
updated before it is updated? This is not even possible for every user's
images out there... :-/

The Java Hotspot VM has a mapping table for changed instVar accesses. Maybe
we could use this trick, too? Still, would only applies to future versions.
We cannot rewrite history because it is not only one piece of software in a
single version control system with a single version checked out. :)

Best,
Marcel




--
View this message in context: http://forum.world.st/Re-Does-anyone-recognize-this-glitch-in-our-trunk-update-processing-was-Vm-dev-buildspurtrunkvmmaker-tp4912130p4912403.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.




Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Nicolas Cellier
Due to read only ClassBinding it would be just a little bit more hackish:

(Smalltalk globals associationAt: #SystemProgressMorph) instVarAt: 2 put: SystemProgressMorph copy.

2016-09-01 1:12 GMT+02:00 Nicolas Cellier <[hidden email]>:
Or would this even more simple hack work? Put in the preamble something like:

Smalltalk at: SystemProgressMorph put: SystemProgressMorph copy.

This way, the old instances should not be mutated...

2016-08-24 7:06 GMT+02:00 marcel.taeumel <[hidden email]>:
marcel.taeumel wrote
>
> Bert Freudenberg wrote
>> On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel &lt;

>> Marcel.Taeumel@

>> &gt;
>> wrote:
>>
>>>
>>> Hi, there.
>>>
>>> It is, unfortunately, not possible to turn back time and rewrite
>>> history.
>>> ;-)
>>>
>>
>> ... unless we're dealing with software. Oh wait, we are ;)
>>
>> There is several important code that makes excessive use of instVar
>>> accesses such as in SystemProgressMorph or HandMorph. Making any change
>>> in
>>> their class layout while using some of their methods will provoke a
>>> strange
>>> error. This could only be changed by a VM that keeps old class
>>> layouts/schemata around for some longer time until there is no old
>>> compiled
>>> method on any stack anymore.
>>
>>
>> Class layout changes and instance migration is handled fully in the image
>> by class ClassBuilder. The VM is not at fault here.
>>
>>
>>> Usually, restarting the update process helps here. There is no way to
>>> fix
>>> that in order to interactively update from 5.0 to 5.1 without any
>>> errors.
>>>
>>
>> We can retroactively change the config map that introduced the problem.
>> I'm
>> certain it would be possible to fix, but I am not entirely sure it's
>> worth
>> the trouble.
>>
>> - Bert -
> Hi Bert,
>
> the config map will not help, we would have to change numerous MCZs since
> point X in time to rewrite history. :) "Oh, would we have always been
> using accessors here instead of instVar accessess..."
>
> Best,
> Marcel

...even then, how do you update the code in image Y, which is going to be
updated before it is updated? This is not even possible for every user's
images out there... :-/

The Java Hotspot VM has a mapping table for changed instVar accesses. Maybe
we could use this trick, too? Still, would only applies to future versions.
We cannot rewrite history because it is not only one piece of software in a
single version control system with a single version checked out. :)

Best,
Marcel




--
View this message in context: http://forum.world.st/Re-Does-anyone-recognize-this-glitch-in-our-trunk-update-processing-was-Vm-dev-buildspurtrunkvmmaker-tp4912130p4912403.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.





Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Nicolas Cellier
The copy is not completely clean because this assertion then fails:

(SystemProgressMorph superclass subclasses detect: [:e | e name = #SystemProgressMorph]) == SystemProgressMorph

IOW, the superclass is still pointing to the old class, so any change to superclass layout won't propagate to the new subclass... Bad.

But there's also this possibility:

| oldClass |
oldClass := SystemProgressMorph copy become: SystemProgressMorph.
SystemProgressMorph allInstancesDo: [:e | oldClass adoptInstance: e].

We mutate all the instances to a class detached from Smalltalk and immune to some changes...
Beware: don't you modify the layout of one superclass...

2016-09-01 1:17 GMT+02:00 Nicolas Cellier <[hidden email]>:
Due to read only ClassBinding it would be just a little bit more hackish:

(Smalltalk globals associationAt: #SystemProgressMorph) instVarAt: 2 put: SystemProgressMorph copy.

2016-09-01 1:12 GMT+02:00 Nicolas Cellier <[hidden email]>:
Or would this even more simple hack work? Put in the preamble something like:

Smalltalk at: SystemProgressMorph put: SystemProgressMorph copy.

This way, the old instances should not be mutated...

2016-08-24 7:06 GMT+02:00 marcel.taeumel <[hidden email]>:
marcel.taeumel wrote
>
> Bert Freudenberg wrote
>> On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel &lt;

>> Marcel.Taeumel@

>> &gt;
>> wrote:
>>
>>>
>>> Hi, there.
>>>
>>> It is, unfortunately, not possible to turn back time and rewrite
>>> history.
>>> ;-)
>>>
>>
>> ... unless we're dealing with software. Oh wait, we are ;)
>>
>> There is several important code that makes excessive use of instVar
>>> accesses such as in SystemProgressMorph or HandMorph. Making any change
>>> in
>>> their class layout while using some of their methods will provoke a
>>> strange
>>> error. This could only be changed by a VM that keeps old class
>>> layouts/schemata around for some longer time until there is no old
>>> compiled
>>> method on any stack anymore.
>>
>>
>> Class layout changes and instance migration is handled fully in the image
>> by class ClassBuilder. The VM is not at fault here.
>>
>>
>>> Usually, restarting the update process helps here. There is no way to
>>> fix
>>> that in order to interactively update from 5.0 to 5.1 without any
>>> errors.
>>>
>>
>> We can retroactively change the config map that introduced the problem.
>> I'm
>> certain it would be possible to fix, but I am not entirely sure it's
>> worth
>> the trouble.
>>
>> - Bert -
> Hi Bert,
>
> the config map will not help, we would have to change numerous MCZs since
> point X in time to rewrite history. :) "Oh, would we have always been
> using accessors here instead of instVar accessess..."
>
> Best,
> Marcel

...even then, how do you update the code in image Y, which is going to be
updated before it is updated? This is not even possible for every user's
images out there... :-/

The Java Hotspot VM has a mapping table for changed instVar accesses. Maybe
we could use this trick, too? Still, would only applies to future versions.
We cannot rewrite history because it is not only one piece of software in a
single version control system with a single version checked out. :)

Best,
Marcel




--
View this message in context: http://forum.world.st/Re-Does-anyone-recognize-this-glitch-in-our-trunk-update-processing-was-Vm-dev-buildspurtrunkvmmaker-tp4912130p4912403.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.






Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Eliot Miranda-2


On Thu, Sep 1, 2016 at 12:33 AM, Nicolas Cellier <[hidden email]> wrote:
The copy is not completely clean because this assertion then fails:

(SystemProgressMorph superclass subclasses detect: [:e | e name = #SystemProgressMorph]) == SystemProgressMorph

IOW, the superclass is still pointing to the old class, so any change to superclass layout won't propagate to the new subclass... Bad.

But there's also this possibility:

| oldClass |
oldClass := SystemProgressMorph copy become: SystemProgressMorph.
SystemProgressMorph allInstancesDo: [:e | oldClass adoptInstance: e].

We mutate all the instances to a class detached from Smalltalk and immune to some changes...
Beware: don't you modify the layout of one superclass...

+1
 

2016-09-01 1:17 GMT+02:00 Nicolas Cellier <[hidden email]>:
Due to read only ClassBinding it would be just a little bit more hackish:

(Smalltalk globals associationAt: #SystemProgressMorph) instVarAt: 2 put: SystemProgressMorph copy.

2016-09-01 1:12 GMT+02:00 Nicolas Cellier <[hidden email]>:
Or would this even more simple hack work? Put in the preamble something like:

Smalltalk at: SystemProgressMorph put: SystemProgressMorph copy.

This way, the old instances should not be mutated...

2016-08-24 7:06 GMT+02:00 marcel.taeumel <[hidden email]>:
marcel.taeumel wrote
>
> Bert Freudenberg wrote
>> On Tue, Aug 23, 2016 at 11:21 AM, marcel.taeumel &lt;

>> Marcel.Taeumel@

>> &gt;
>> wrote:
>>
>>>
>>> Hi, there.
>>>
>>> It is, unfortunately, not possible to turn back time and rewrite
>>> history.
>>> ;-)
>>>
>>
>> ... unless we're dealing with software. Oh wait, we are ;)
>>
>> There is several important code that makes excessive use of instVar
>>> accesses such as in SystemProgressMorph or HandMorph. Making any change
>>> in
>>> their class layout while using some of their methods will provoke a
>>> strange
>>> error. This could only be changed by a VM that keeps old class
>>> layouts/schemata around for some longer time until there is no old
>>> compiled
>>> method on any stack anymore.
>>
>>
>> Class layout changes and instance migration is handled fully in the image
>> by class ClassBuilder. The VM is not at fault here.
>>
>>
>>> Usually, restarting the update process helps here. There is no way to
>>> fix
>>> that in order to interactively update from 5.0 to 5.1 without any
>>> errors.
>>>
>>
>> We can retroactively change the config map that introduced the problem.
>> I'm
>> certain it would be possible to fix, but I am not entirely sure it's
>> worth
>> the trouble.
>>
>> - Bert -
> Hi Bert,
>
> the config map will not help, we would have to change numerous MCZs since
> point X in time to rewrite history. :) "Oh, would we have always been
> using accessors here instead of instVar accessess..."
>
> Best,
> Marcel

...even then, how do you update the code in image Y, which is going to be
updated before it is updated? This is not even possible for every user's
images out there... :-/

The Java Hotspot VM has a mapping table for changed instVar accesses. Maybe
we could use this trick, too? Still, would only applies to future versions.
We cannot rewrite history because it is not only one piece of software in a
single version control system with a single version checked out. :)

Putting more brittle stuff into he VM is not the solution.  We have all that we need above the VM that can be dynamically tailored to suit our needs.

Can we not put the progress morph in its own process, making it easier to tear down and restart?

 

Best,
Marcel




--
View this message in context: http://forum.world.st/Re-Does-anyone-recognize-this-glitch-in-our-trunk-update-processing-was-Vm-dev-buildspurtrunkvmmaker-tp4912130p4912403.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
 
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Does anyone recognize this glitch in our trunk update processing? (was: [Vm-dev] buildspurtrunkvmmakerimage.sh fails on Mac)

Bert Freudenberg
In reply to this post by Nicolas Cellier
On Thursday, 1 September 2016, Nicolas Cellier <[hidden email]> wrote:
The copy is not completely clean because this assertion then fails:

(SystemProgressMorph superclass subclasses detect: [:e | e name = #SystemProgressMorph]) == SystemProgressMorph

IOW, the superclass is still pointing to the old class, so any change to superclass layout won't propagate to the new subclass... Bad.

But there's also this possibility:

| oldClass |
oldClass := SystemProgressMorph copy become: SystemProgressMorph.
SystemProgressMorph allInstancesDo: [:e | oldClass adoptInstance: e].

We mutate all the instances to a class detached from Smalltalk and immune to some changes...

This is an interesting idea.

Basically we want to prevent the existing instances to be migrated to the new class layout when we change the instance variables. One way is what Nicolas suggests, but another would be to just keep the migration from happening in the first place, in ClassBuilder. 

Could this be a general approach? To never migrate instances that are receivers of a context? They  would still be instances of the old (now obsolete) class version. They could even be kept in a weak array to be migrated later, once the contexts are no longer active. The processes could even be restarted
to get rid of these contexts.

There are other Smalltalks (e.g. GemStone) that even allow different class versions to exist side-by-side. We do have class versions too, but they have only been used for migrating serialized instances when the instance variables did *not* change (which is how the need for migration is normally detected). Hmm, maybe that is actually irrelevant, just ran through my mind ;)

So what do you think about delayed migration of instances on the stack? 

- Bert -