Failing tests on CI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Failing tests on CI

Nicolas Cellier
It seems that this one would pass after a Compiler recompileAll, maybe
we should add some postInstall action somewhere after environments
changes...

http://build.squeak.org/job/SqueakTrunk/223/testReport/Tests.Compiler/MethodHighlightingTests/testMethodHighlighting/

Nicolas

Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Eliot Miranda-2


On Tue, Mar 12, 2013 at 2:13 PM, Nicolas Cellier <[hidden email]> wrote:
It seems that this one would pass after a Compiler recompileAll, maybe
we should add some postInstall action somewhere after environments
changes...

Yes, but a simple recompile will result in lots of unbound methods.  We need to do something more principled that I thought ReleaseBuilder did.  Frank, you were working on this right?
 

http://build.squeak.org/job/SqueakTrunk/223/testReport/Tests.Compiler/MethodHighlightingTests/testMethodHighlighting/

Nicolas




--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Frank Shearar-3
On 13 March 2013 22:41, Eliot Miranda <[hidden email]> wrote:

>
>
> On Tue, Mar 12, 2013 at 2:13 PM, Nicolas Cellier
> <[hidden email]> wrote:
>>
>> It seems that this one would pass after a Compiler recompileAll, maybe
>> we should add some postInstall action somewhere after environments
>> changes...
>
>
> Yes, but a simple recompile will result in lots of unbound methods.  We need
> to do something more principled that I thought ReleaseBuilder did.  Frank,
> you were working on this right?

Yes, but this is _before_ the ReleaseBuilderFor4dot5 kicks in. The
tests are failing in the SqueakTrunk job, and it's only
ReleaseSqueakTrunk that does all this (specifically, `Compile
recompileAll`). And of course it's a one-way trip;
ReleaseSqueakTrunk's artifact isn't fed back into SqueakTrunk. (And it
shouldn't, because ReleaseSqueakTrunk is supposed to take an
ever-shrinking trunk core and add expected goodies to produce a
standard image.)

Er, unless I miss the point: are you asking if we do any better than
`Compiler recompileAll`?

frank

Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Colin Putney-3
On 2013-03-14, at 3:00 AM, Frank Shearar <[hidden email]> wrote:

> Er, unless I miss the point: are you asking if we do any better than
> `Compiler recompileAll`?

I have a few more cleanups to do, and I'll include this. I don't think
we need a full recompile, it should be enough to update bindings to
the right class.

Sorry for the delay, dealing with the usual sequencing issues.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Eliot Miranda-2
In reply to this post by Frank Shearar-3


On Thu, Mar 14, 2013 at 3:00 AM, Frank Shearar <[hidden email]> wrote:
On 13 March 2013 22:41, Eliot Miranda <[hidden email]> wrote:
>
>
> On Tue, Mar 12, 2013 at 2:13 PM, Nicolas Cellier
> <[hidden email]> wrote:
>>
>> It seems that this one would pass after a Compiler recompileAll, maybe
>> we should add some postInstall action somewhere after environments
>> changes...
>
>
> Yes, but a simple recompile will result in lots of unbound methods.  We need
> to do something more principled that I thought ReleaseBuilder did.  Frank,
> you were working on this right?

Yes, but this is _before_ the ReleaseBuilderFor4dot5 kicks in. The
tests are failing in the SqueakTrunk job, and it's only
ReleaseSqueakTrunk that does all this (specifically, `Compile
recompileAll`). And of course it's a one-way trip;
ReleaseSqueakTrunk's artifact isn't fed back into SqueakTrunk. (And it
shouldn't, because ReleaseSqueakTrunk is supposed to take an
ever-shrinking trunk core and add expected goodies to produce a
standard image.)

Er, unless I miss the point: are you asking if we do any better than
`Compiler recompileAll`?

No.  I'm saying that if an update causes the need for a recompilation then we need to do better than Compiler recompileAll because the naive Compiler recompileAll creates lots of unbound methods.  We need to follow Compiler recompileAll with the relevant release magic to restart processes holding onto now obsolete methods etc.  So that release magic ought to be in the base image as a facility.  e.g. Smalltalk recompileAll might be much smarter than Compiler recompileAll. 

frank



--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Nicolas Cellier
2013/3/14 Eliot Miranda <[hidden email]>:

>
>
> On Thu, Mar 14, 2013 at 3:00 AM, Frank Shearar <[hidden email]>
> wrote:
>>
>> On 13 March 2013 22:41, Eliot Miranda <[hidden email]> wrote:
>> >
>> >
>> > On Tue, Mar 12, 2013 at 2:13 PM, Nicolas Cellier
>> > <[hidden email]> wrote:
>> >>
>> >> It seems that this one would pass after a Compiler recompileAll, maybe
>> >> we should add some postInstall action somewhere after environments
>> >> changes...
>> >
>> >
>> > Yes, but a simple recompile will result in lots of unbound methods.  We
>> > need
>> > to do something more principled that I thought ReleaseBuilder did.
>> > Frank,
>> > you were working on this right?
>>
>> Yes, but this is _before_ the ReleaseBuilderFor4dot5 kicks in. The
>> tests are failing in the SqueakTrunk job, and it's only
>> ReleaseSqueakTrunk that does all this (specifically, `Compile
>> recompileAll`). And of course it's a one-way trip;
>> ReleaseSqueakTrunk's artifact isn't fed back into SqueakTrunk. (And it
>> shouldn't, because ReleaseSqueakTrunk is supposed to take an
>> ever-shrinking trunk core and add expected goodies to produce a
>> standard image.)
>>
>> Er, unless I miss the point: are you asking if we do any better than
>> `Compiler recompileAll`?
>
>
> No.  I'm saying that if an update causes the need for a recompilation then
> we need to do better than Compiler recompileAll because the naive Compiler
> recompileAll creates lots of unbound methods.  We need to follow Compiler
> recompileAll with the relevant release magic to restart processes holding
> onto now obsolete methods etc.  So that release magic ought to be in the
> base image as a facility.  e.g. Smalltalk recompileAll might be much smarter
> than Compiler recompileAll.

Note that not only processes might point to an unbound CompiledMethod,
every object pointing to a blockClosure does, and that may include
some GUI elements.

Nicolas

>>
>>
>> frank
>>
>> >>
>> >> http://build.squeak.org/job/SqueakTrunk/223/testReport/Tests.Compiler/MethodHighlightingTests/testMethodHighlighting/
>> >>
>> >> Nicolas
>> >>
>> >
>> >
>> >
>> > --
>> > best,
>> > Eliot
>> >
>> >
>> >
>>
>
>
>
> --
> best,
> Eliot
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Eliot Miranda-2


On Thu, Mar 14, 2013 at 10:46 AM, Nicolas Cellier <[hidden email]> wrote:
2013/3/14 Eliot Miranda <[hidden email]>:
>
>
> On Thu, Mar 14, 2013 at 3:00 AM, Frank Shearar <[hidden email]>
> wrote:
>>
>> On 13 March 2013 22:41, Eliot Miranda <[hidden email]> wrote:
>> >
>> >
>> > On Tue, Mar 12, 2013 at 2:13 PM, Nicolas Cellier
>> > <[hidden email]> wrote:
>> >>
>> >> It seems that this one would pass after a Compiler recompileAll, maybe
>> >> we should add some postInstall action somewhere after environments
>> >> changes...
>> >
>> >
>> > Yes, but a simple recompile will result in lots of unbound methods.  We
>> > need
>> > to do something more principled that I thought ReleaseBuilder did.
>> > Frank,
>> > you were working on this right?
>>
>> Yes, but this is _before_ the ReleaseBuilderFor4dot5 kicks in. The
>> tests are failing in the SqueakTrunk job, and it's only
>> ReleaseSqueakTrunk that does all this (specifically, `Compile
>> recompileAll`). And of course it's a one-way trip;
>> ReleaseSqueakTrunk's artifact isn't fed back into SqueakTrunk. (And it
>> shouldn't, because ReleaseSqueakTrunk is supposed to take an
>> ever-shrinking trunk core and add expected goodies to produce a
>> standard image.)
>>
>> Er, unless I miss the point: are you asking if we do any better than
>> `Compiler recompileAll`?
>
>
> No.  I'm saying that if an update causes the need for a recompilation then
> we need to do better than Compiler recompileAll because the naive Compiler
> recompileAll creates lots of unbound methods.  We need to follow Compiler
> recompileAll with the relevant release magic to restart processes holding
> onto now obsolete methods etc.  So that release magic ought to be in the
> base image as a facility.  e.g. Smalltalk recompileAll might be much smarter
> than Compiler recompileAll.

Note that not only processes might point to an unbound CompiledMethod,
every object pointing to a blockClosure does, and that may include
some GUI elements.

Exactly.  e.g. in the Squeak4.3 base image:

CompiledMethod allInstances select:
[:m|
m ~~ thisContext method
and: [m methodClass isNil
or: [m selector isNil
or: [m ~~ (m methodClass >> m selector)]]]]
=> #()

But

Compiler recompileAll.
Smalltalk garbageCollect.
((CompiledMethod allInstances select:
[:m|
m ~~ thisContext method
and: [m methodClass isNil
or: [m selector isNil
or: [m ~~ (m methodClass >> m selector)]]]]) asSortedCollection: [:m1 :m2| m1 methodClass name < m2 methodClass name or: [m1 methodClass = m2 methodClass and: [m1 selector < m2 selector]]]) asArray
=> {(BlockClosure>>#newProcess "a CompiledMethod(3143)") . (BlockClosure>>#on:do: "a CompiledMethod(2372)") . (Compiler>>#evaluate:in:to:notifying:ifFail:logged: "a CompiledMethod(3380)") . (Compiler>>#translate:noPattern:ifFail: "a CompiledMethod(718)") . (Delay class>>#runTimerEventLoop "a CompiledMethod(84)") . (Delay class>>#startTimerEventLoop "a CompiledMethod(3416)") . (EventSensor>>#eventTickler "a CompiledMethod(350)") . (EventSensor>>#installEventTickler "a CompiledMethod(3857)") . (ExternalDropHandler class>>#registerStandardExternalDropHandlers "a CompiledMethod(2523)") . (GradientFillStyle class>>#initPixelRampCache "a CompiledMethod(1023)") . (HandMorph>>#handleEvent: "a CompiledMethod(2487)") . (HandMorph>>#processEvents "a CompiledMethod(3865)") . (HandMorph>>#sendEvent:focus:clear: "a CompiledMethod(3217)") . (HandMorph>>#sendFocusEvent:to:clear: "a CompiledMethod(2182)") . (HandMorph>>#sendKeyboardEvent: "a CompiledMethod(1440)") . (InputSensor>>#installInterruptWatcher "a CompiledMethod(145)") . (InputSensor>>#userInterruptWatcher "a CompiledMethod(2885)") . (KeyboardEvent>>#sentTo: "a CompiledMethod(2261)") . (Morph>>#handleEvent: "a CompiledMethod(2284)") . (Morph>>#handleFocusEvent: "a CompiledMethod(2176)") . (PasteUpMorph>>#becomeActiveDuring: "a CompiledMethod(2232)") . (PasteUpMorph>>#doOneCycle "a CompiledMethod(2078)") . (ProcessBrowser class>>#registerWellKnownProcess:label:allowStop:allowDebug: "a CompiledMethod(3997)") . (ProcessBrowser class>>#registerWellKnownProcesses "a CompiledMethod(1889)") . (ProcessorScheduler class>>#idleProcess "a CompiledMethod(1512)") . (ProcessorScheduler class>>#startUp "a CompiledMethod(2171)") . (Project class>>#spawnNewProcess "a CompiledMethod(510)") . (ScrollBar class>>#createArrowImagesCache "a CompiledMethod(2003)") . (ScrollBar class>>#createBoxImagesCache "a CompiledMethod(2713)") . (SequenceableCollection>>#do: "a CompiledMethod(4002)") . (ServiceAction class>>#id:text:button:description:action: "a CompiledMethod(3545)") . (ServiceAction class>>#text:button:description:action: "a CompiledMethod(2391)") . (ServiceCategory class>>#text:button:description: "a CompiledMethod(3931)") . (ServiceShortcuts class>>#insertPrefShortcut: "a CompiledMethod(3566)") . (SmalltalkImage>>#installLowSpaceWatcher "a CompiledMethod(522)") . (SmalltalkImage>>#lowSpaceWatcher "a CompiledMethod(832)") . (StandardToolSet class>>#codeCompletionAround:textMorph:keyStroke: "a CompiledMethod(1053)") . (TextEditor>>#dispatchOnKeyboardEvent: "a CompiledMethod(3157)") . (TextEditor>>#evaluateSelectionAndDo: "a CompiledMethod(4000)") . (TextEditor>>#keyStroke: "a CompiledMethod(332)") . (TextEditor>>#printIt "a CompiledMethod(815)") . (TextEditor>>#printIt: "a CompiledMethod(1977)") . (TextMorph>>#handleInteraction:fromEvent: "a CompiledMethod(933)") . (TextMorph>>#handleKeystroke: "a CompiledMethod(2806)") . (TextMorph>>#keyStroke: "a CompiledMethod(3260)") . (TextMorphForEditView>>#handleInteraction:fromEvent: "a CompiledMethod(1848)") . (TextMorphForEditView>>#keyStroke: "a CompiledMethod(296)") . (TheWorldMainDockingBar>>#showWelcomeText:label:in: "a CompiledMethod(3390)") . (TheWorldMainDockingBar>>#toggleFullScreenOn: "a CompiledMethod(1194)") . (ToolSet class>>#codeCompletionAround:textMorph:keyStroke: "a CompiledMethod(3438)") . (VariableNode class>>#initialize "a CompiledMethod(2215)") . (Vocabulary class>>#newPublicVocabulary "a CompiledMethod(1353)") . (WeakArray class>>#finalizationProcess "a CompiledMethod(1063)") . (WeakArray class>>#restartFinalizationProcess "a CompiledMethod(3742)") . (WeakRegistry>>#installFinalizer "a CompiledMethod(78)") . (WorldMenuProvider>>#closeTopWindow "a CompiledMethod(2095)") . (WorldMenuProvider>>#createNewService "a CompiledMethod(3640)") . (WorldMenuProvider>>#helpOnServices "a CompiledMethod(2200)") . (WorldMenuProvider>>#nextWindow "a CompiledMethod(2644)") . (WorldMenuProvider>>#preferencesBrowser "a CompiledMethod(2193)") . (WorldMenuProvider>>#rebuildRegistry "a CompiledMethod(2191)") . (WorldMenuProvider>>#servicesBrowser "a CompiledMethod(1893)") . (WorldState>>#doOneCycleFor: "a CompiledMethod(817)") . (WorldState>>#doOneCycleNowFor: "a CompiledMethod(3386)") . (WorldState>>#handsDo: "a CompiledMethod(2792)") . (WorldState>>#stepListSortBlock "a CompiledMethod(2016)")}


We want e.g.

Smalltalk recompileAll.
(CompiledMethod allInstances select:
[:m|
m ~~ thisContext method
and: [m methodClass isNil
or: [m selector isNil
or: [m ~~ (m methodClass >> m selector)]]]]
=> #()

But I don't see how to get near this:
1. lots of GUI stuff can only be cleared by closing windows.  Can we export the morphs and bring them back in?
2. (quite easy) we need to save and restart to restart a number of processes


Nicolas

>>
>>
>> frank
>>
>> >>
>> >> http://build.squeak.org/job/SqueakTrunk/223/testReport/Tests.Compiler/MethodHighlightingTests/testMethodHighlighting/
>> >>
>> >> Nicolas
>> >>
>> >
>> >
>> >
>> > --
>> > best,
>> > Eliot
>> >
>> >
>> >
>>
>
>
>
> --
> best,
> Eliot
>
>
>




--
best,
Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Colin Putney-3



On Thu, Mar 14, 2013 at 11:20 AM, Eliot Miranda <[hidden email]> wrote:
 
We want e.g.

Smalltalk recompileAll.
(CompiledMethod allInstances select:
[:m|
m ~~ thisContext method
and: [m methodClass isNil
or: [m selector isNil
or: [m ~~ (m methodClass >> m selector)]]]]
=> #()

But I don't see how to get near this:
1. lots of GUI stuff can only be cleared by closing windows.  Can we export the morphs and bring them back in?
2. (quite easy) we need to save and restart to restart a number of processes

Is this what you'd like to see as a generally useful facility, or is it what you think we need to do to fix the failing tests?

Colin


Reply | Threaded
Open this post in threaded view
|

Re: Failing tests on CI

Eliot Miranda-2


On Thu, Mar 14, 2013 at 11:42 AM, Colin Putney <[hidden email]> wrote:



On Thu, Mar 14, 2013 at 11:20 AM, Eliot Miranda <[hidden email]> wrote:
 
We want e.g.

Smalltalk recompileAll.
(CompiledMethod allInstances select:
[:m|
m ~~ thisContext method
and: [m methodClass isNil
or: [m selector isNil
or: [m ~~ (m methodClass >> m selector)]]]]
=> #()

But I don't see how to get near this:
1. lots of GUI stuff can only be cleared by closing windows.  Can we export the morphs and bring them back in?
2. (quite easy) we need to save and restart to restart a number of processes

Is this what you'd like to see as a generally useful facility, or is it what you think we need to do to fix the failing tests?

As a generally useful facility.  If the system is to be continuously updated and it is to remain self-compiling then it needs to function properly across updates that change the compiler.  We've had a number of these recently, inlining of timesRepeat:, addition of optimized selectors ifTrue:. whileTrue: et al to literals, fixing the return value of optimized to:do: and now Environments.  That the system needs recompiling after certain changes is arcane, that this is tricky to do right is even more arcane.  Those who publish a package that modifies the compiler generally know that a recompilation is necessary but avoid it (I suspect) because... the recompilation is tricky.  If we had a reasonably effective recompile then we'd be less reticent about offering to recompile after load.  I think we should provide something like an ability to note that recompilation is necessary and then a facility to ask the user if they want to recompile.  e.g. a class var in SmalltalkImage which is a dirty flag marking the system as needing recompilation, plus a facility to prompt the user to recompile if they want to.  So then out post-load scripts can contain somethign like

Smalltalk noteRecompilationIsNecessary.
Smalltalk promptUserToRecompile

which, if interactive, pops up a dialog asking to recompile and then recompiles smartly if the user says so.  The recompilation necessary flag gets cleared by Compiler recompileAll, not just by Smalltalk recompileAll.  (If one uses Compiler recompileAll then recompilation has been done albeit messily). 


Colin






--
best,
Eliot