OSProcess and Cuis: successes and failures, cumulative changes

> I've attached all the changes I've made to OSProcess.  This includes new
> work that replaces #fork with #newProcess and #resume for those
> situations in which the return value might be used.
>
> It mostly does not have changes from #ifNotNilDo: to #ifNotNil:.  So to
> run it in Cuis requires implementing ifNotNilDo:, or making the
> remaining changes from ifNotNilDo: to ifNotNil:.  I think the
> changeset's one change to ifNotNil: is in a method that's activated
> during filein.
>
> Despite the work on fork, I'm still getting about the same number of
> test failures in Cuis 2.6 (recall OSProcess unaltered passed all tests
> in squeak 4.1).  Adding in UnixProcessWin32FileLockingTestCase, there
> are 167 tests with 2 failures and 24 errors.  All problems are in
> UnixProcess{Unix,Win32}FileLockingTestCase.
>
> I'm also starting to suspect that the apparent errors with the process p
> nil were artifacts of the debugging process.  Typically the error is in
> tearDown
>
> | d |
> OSProcessAccessor emulateWin32FileLocking: self initialCompatibilitySetting.
> d := Delay forMilliseconds: 50.
> self fileStream close.
> self remoteProcess ifNotNilDo:
> [:p | p terminate.
> [p isComplete] whileFalse: [d wait].
> self remoteProcess: nil]
>
> In p terminate, p is undefined and terminate is not understood.  This
> really doesn't make sense, since the block is only executed ifNotNilDo:
> and when I trace into it things seem OK.  The final step sets the remote
> process to nil; perhaps by the time I bring up the debugger the old
> remote process is gone.
>
> Here's the  code I put in ProtoObject
> ifNotNilDo: ifNotNilBlock
> "RB adds for back compatibility"
> "Evaluate the block, unless I'm == nil (q.v.)"
>
> ^ ifNotNilBlock valueWithPossibleArgs: {self}

> On Fri, 2010-09-03 at 16:22 -0400, David T. Lewis wrote:
> > > Here's the  code I put in ProtoObject
> > > ifNotNilDo: ifNotNilBlock
> > >       "RB adds for back compatibility"
> > >       "Evaluate the block, unless I'm == nil (q.v.)"
> > >
> > >       ^ ifNotNilBlock valueWithPossibleArgs: {self}
> >
> Duh.  I also need, in UndefinedObject in Cuis
> ifNotNilDo: aBlock
> "RB back compatibility"
> ^ self
>
> Of course the right way is to change the OSProcess code to use something
> more portable than ifNotNilDo:.

>
> With that change, and still more tests added, I have 3 failures out of
> 190 tests run.  They are in
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses0[12]
> and the comparable 02 (but not 01) method for Win32 version.
>
> Unfortunately, the debugger one can bring up on the failures isn't very
> informative about the exact source of the failure.
>
> Ross
>

> On Fri, Sep 03, 2010 at 02:35:53PM -0700, Ross Boylan wrote:
> > On Fri, 2010-09-03 at 16:22 -0400, David T. Lewis wrote:
> > > > Here's the  code I put in ProtoObject
> > > > ifNotNilDo: ifNotNilBlock
> > > >       "RB adds for back compatibility"
> > > >       "Evaluate the block, unless I'm == nil (q.v.)"
> > > >
> > > >       ^ ifNotNilBlock valueWithPossibleArgs: {self}
> > >
> > Duh.  I also need, in UndefinedObject in Cuis
> > ifNotNilDo: aBlock
> > "RB back compatibility"
> > ^ self
> >
> > Of course the right way is to change the OSProcess code to use something
> > more portable than ifNotNilDo:.
>
>
> Hi Ross,
>
> I have updated both OSProcess and CommandShell on SqueakSource to address
> all of the #ifNotNilDo: issues for Cuis, as well as the #fork issues that
> you previously addressed. I think that I've captured all of your fixes.
> Thank you *very* much for this.
>
> I did not actually load into Cuis to check, but a lot of the issues
> were in the unit tests so I expect they will run properly now (knock
> wood).
>
> And I have to give a grudging nod of appreciation to Juan for forcing
> me to address the #ifNotNilDo: concerns. At first I was annoyed that I
> had to rewrite a lot of code, but I have to admit that by the time I
> was done I had cleaned out a lot of ugly cruft from OSProcess, so
> I guess once again Juan has got it right :)
>
> Dave

> On Fri, 2010-09-03 at 19:55 -0400, David T. Lewis wrote:
> > On Fri, Sep 03, 2010 at 02:35:53PM -0700, Ross Boylan wrote:
> > > On Fri, 2010-09-03 at 16:22 -0400, David T. Lewis wrote:
> > > > > Here's the  code I put in ProtoObject
> > > > > ifNotNilDo: ifNotNilBlock
> > > > >       "RB adds for back compatibility"
> > > > >       "Evaluate the block, unless I'm == nil (q.v.)"
> > > > >
> > > > >       ^ ifNotNilBlock valueWithPossibleArgs: {self}
> > > >
> > > Duh.  I also need, in UndefinedObject in Cuis
> > > ifNotNilDo: aBlock
> > > "RB back compatibility"
> > > ^ self
> > >
> > > Of course the right way is to change the OSProcess code to use something
> > > more portable than ifNotNilDo:.
> >
> >
> > Hi Ross,
> >
> > I have updated both OSProcess and CommandShell on SqueakSource to address
> > all of the #ifNotNilDo: issues for Cuis, as well as the #fork issues that
> > you previously addressed. I think that I've captured all of your fixes.
> > Thank you *very* much for this.
> >
> > I did not actually load into Cuis to check, but a lot of the issues
> > were in the unit tests so I expect they will run properly now (knock
> > wood).
> >
> > And I have to give a grudging nod of appreciation to Juan for forcing
> > me to address the #ifNotNilDo: concerns. At first I was annoyed that I
> > had to rewrite a lot of code, but I have to admit that by the time I
> > was done I had cleaned out a lot of ugly cruft from OSProcess, so
> > I guess once again Juan has got it right :)
> >
> > Dave
> The latest changeset files in cleanly to a new Cuis2.7 image (in
> particular, I did not add any hacks for ifNotNilDo:).  All 190 tests
> passed the first 2 or 3 times I ran the tests, but, after running the
> 2.6 tests below, I got various failures in 2.7.
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses02 failed
> once, and then
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses04 the next
> time.
>
> The 2.6 and 2.7 tests were in different directories.  In the initial 2.7
> tests only that image was up, while in all later cases both were up.
> The 2.7 image seems to do somewhat better than 2.6.  When I start from a
> fresh Cuis2.6 image, I get non-deterministic failures.  The first test
> suite run hung up, from memory at UnixProcessTestCase>>testCatAFile.
> The last two runs failed in
> UnixProcessWin32FileLockingTestCase>>testCooperatingProcesses04.
>
> I shut down the 2.7 image and tried running just the 2.6.  There was one
> failure, in
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses04.
>
> Juan, do you have any ideas what changes between 2.6 and 2.7 might be
> relevant, or what might be causing these failure patterns?

> > Juan, do you have any ideas what changes between 2.6 and 2.7 might
> be
> > relevant, or what might be causing these failure patterns?
>
> It is very possible that the problems are either in my unit tests, or
> in OSProcess/CommandShell. Many of the tests involve cooperating OS
> processes and are timing dependent. It would be good to understand
> what
> has changed between 2.6 and 2.7, but my working assumption would be
> that any changes in Cuis might make bugs in OSProcess/CommandShell
> more probable, but are not necessarily the root cause. The file
> locking
> tests are an example of tests that rely on cooperating OS processes,
> and that could fail due to timing issues of one sort of another.
>
> I note that CommandShellTestCase in particular is a real torture
> test for process scheduling, and Pharo images have tended to choke
> on these tests. Again, I don't assume that Pharo is the root cause,
> just that the behavior seems to be different.

> On Fri, 2010-09-03 at 19:55 -0400, David T. Lewis wrote:
>  
>> On Fri, Sep 03, 2010 at 02:35:53PM -0700, Ross Boylan wrote:
>>    
>>> On Fri, 2010-09-03 at 16:22 -0400, David T. Lewis wrote:
>>>      
>>>>> Here's the  code I put in ProtoObject
>>>>> ifNotNilDo: ifNotNilBlock
>>>>>       "RB adds for back compatibility"
>>>>>       "Evaluate the block, unless I'm == nil (q.v.)"
>>>>>
>>>>>       ^ ifNotNilBlock valueWithPossibleArgs: {self}
>>>>>          
>>> Duh.  I also need, in UndefinedObject in Cuis
>>> ifNotNilDo: aBlock
>>> "RB back compatibility"
>>> ^ self
>>>
>>> Of course the right way is to change the OSProcess code to use something
>>> more portable than ifNotNilDo:.
>>>      
>> Hi Ross,
>>
>> I have updated both OSProcess and CommandShell on SqueakSource to address
>> all of the #ifNotNilDo: issues for Cuis, as well as the #fork issues that
>> you previously addressed. I think that I've captured all of your fixes.
>> Thank you *very* much for this.
>>
>> I did not actually load into Cuis to check, but a lot of the issues
>> were in the unit tests so I expect they will run properly now (knock
>> wood).
>>
>> And I have to give a grudging nod of appreciation to Juan for forcing
>> me to address the #ifNotNilDo: concerns. At first I was annoyed that I
>> had to rewrite a lot of code, but I have to admit that by the time I
>> was done I had cleaned out a lot of ugly cruft from OSProcess, so
>> I guess once again Juan has got it right :)
>>
>> Dave
>>    
> The latest changeset files in cleanly to a new Cuis2.7 image (in
> particular, I did not add any hacks for ifNotNilDo:).  All 190 tests
> passed the first 2 or 3 times I ran the tests, but, after running the
> 2.6 tests below, I got various failures in 2.7.
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses02 failed
> once, and then
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses04 the next
> time.
>
> The 2.6 and 2.7 tests were in different directories.  In the initial 2.7
> tests only that image was up, while in all later cases both were up.
> The 2.7 image seems to do somewhat better than 2.6.  When I start from a
> fresh Cuis2.6 image, I get non-deterministic failures.  The first test
> suite run hung up, from memory at UnixProcessTestCase>>testCatAFile.
> The last two runs failed in
> UnixProcessWin32FileLockingTestCase>>testCooperatingProcesses04.
>
> I shut down the 2.7 image and tried running just the 2.6.  There was one
> failure, in
> UnixProcessUnixFileLockingTestCase>>testCooperatingProcesses04.
>
> Juan, do you have any ideas what changes between 2.6 and 2.7 might be
> relevant, or what might be causing these failure patterns?
>
> Ross