From a mooc user: method source with it' take so long in Pharo 5

There is a method in SourceFileArray #localProcessReadStreamAtFileIndex:atPosition:ifPresent:ifAbsent: which uses a ProccessLocalVariable called ProccessAndSessionLocalSourcesFileArray (see #localProcessReadOnlyCopy). Changing the last line in #readStreamAt:ifPresent:ifAbsent: to use this local process one makes the time to run this snippet:

 

[ 1 systemNavigation browseMethodsWithSourceString: 'Morph' matchCase: false ] timeProfile

 

From ~ 10 seconds to ~ 3 seconds (Windows).

 

However I cannot see that the file handles created by this processLocalVariable to ever be closed, so I suspect those are leaked? In that case there might be the need to implement some “clean up” mechanism for ProccessLocalVariables before they are changed/nilled when a process changes.

 

Another approach could be to not use a ProccessLocalVariabe at all, but extend the SourceFilesArray class to also hold one read only handle for each of its files, and use these in readStreamAtFileIndex:::… . I guess that it is also necessary then to have semaphore protecting the two last lines such that setting the position in the stream and reading from it cannot be changed by other threads.

Fun analysis :)

Hi Henrik,

    cc'ing squeak-dev cuz this is relevant to both communities (if and until Pharo takes the unwise decision to double the size of the image by keeping sources in the image). 

On May 22, 2016, at 5:01 AM, Henrik Nergaard <[hidden email]> wrote:

There is a method in SourceFileArray #localProcessReadStreamAtFileIndex:atPosition:ifPresent:ifAbsent: which uses a ProccessLocalVariable called ProccessAndSessionLocalSourcesFileArray (see #localProcessReadOnlyCopy). Changing the last line in #readStreamAt:ifPresent:ifAbsent: to use this local process one makes the time to run this snippet:

 

[ 1 systemNavigation browseMethodsWithSourceString: 'Morph' matchCase: false ] timeProfile

 

From ~ 10 seconds to ~ 3 seconds (Windows).

 

However I cannot see that the file handles created by this processLocalVariable to ever be closed, so I suspect those are leaked? In that case there might be the need to implement some “clean up” mechanism for ProccessLocalVariables before they are changed/nilled when a process changes.

 

Another approach could be to not use a ProccessLocalVariabe at all, but extend the SourceFilesArray class to also hold one read only handle for each of its files, and use these in readStreamAtFileIndex:::… . I guess that it is also necessary then to have semaphore protecting the two last lines such that setting the position in the stream and reading from it cannot be changed by other threads.

I like this approach, but it has a fatal flaw.  If one is debugging file access, then as the debugger causes the source of methods to be fetched as the debugger displays them, so the file pointer in the read-only file the access of which one is trying to debug will change under one's feet, and the results will be completely wrong.

An approach which uses a special copy for the debugger doesn't work; one can't debug access to that file for the same reason.

An approach that /should/ work is for the debugger to install its own copy around file-access.  This should be recursive.  So around every file access in the debugger there would need to be something like

       SourceFiles substituteFreshReadOnlyCopiesDuring: [...file access...]

which would remember the current read-only files, evaluate the block using ensure: and have the ensure: block reinstall the previous file.

Does this make sense?

 

readStreamAtFileIndex: index atPosition: position ifPresent: presentBlock ifAbsent: absentBlock

 

               | stream  result|

               stream := self readOnlyFileAt: index.

               stream ifNil: [ ^ absentBlock value ].        "sources file not available"          

 

               position > stream size ifTrue: [ ^ absentBlock value ].

 

               readSema critical: [

                              stream position: position.

                              result := presentBlock value: stream

               ].

 

               ^ result

 

 

Best regards,

Henrik

 

 

From: Pharo-dev [[hidden email]] On Behalf Of Nicolai Hess
Sent: Thursday, May 19, 2016 9:31 AM
To: Pharo Development List <[hidden email]>
Subject: Re: [Pharo-dev] From a mooc user: method source with it' take so long in Pharo 5

 

Squeak caches the opened readonly file(handle). It does not have to reopen the file on every call for reading (readonly).

 

2016-05-18 19:12 GMT+02:00 stepharo <[hidden email]>:

I am wondering why does the search 'method source with it' take so long in Pharo 5? On my PC, When I select the text 'menu' and search for all 'method source with it', in Squeak 5 it takes 3 seconds. In Pharo 5 it takes 21 seconds.