Smalltalk › Squeak › Squeak - Beginners

CommandShell/OSPRocess primCreatePipe failing. How to debug?

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 5 messages Options
tty
Reply | Threaded
Open this post in threaded view
♦
♦
|

CommandShell/OSPRocess primCreatePipe failing. How to debug?

tty
In


ExternalPipe>>
makePipe
        | t1 |
        *t1 := OSProcess accessor makePipeHandles.*
        t1 isNil
                ifTrue: [self error: 'cannot create OS pipe']
                ifFalse: [self
                                reader: (AttachableFileStream
                                                name: 'pipeReader'
                                                attachTo: (t1 at: 1)
                                                writable: false).
                        self
                                writer: (AttachableFileStream
                                                name: 'pipeWriter'
                                                attachTo: (t1 at: 2)
                                                writable: true)]


OSProcess accessor is a  * 'a UnixOSProcessAccessor on pid 19790'*


Where *self makePipeHandles* returns and undefined object.

How should I think about this?

thnks






--
Sent from: http://forum.world.st/Squeak-Beginners-f107673.html
_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners
David T. Lewis
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CommandShell/OSPRocess primCreatePipe failing. How to debug?

David T. Lewis
Can you say something about how you get to the point of getting
this error, and what operating system you are using? The code
snippet looks odd to me because it appears to be a decompiled
version of the method in OSProcess. Maybe you do not have a
changes file to go with your image file?

A typical reason for failing to create pipes is that you have
somehow opened too many file handles for your VM, possibly by
opening a lot of pipes and not closing them completely.

Dave

On Fri, Aug 30, 2019 at 07:58:50AM -0500, tty wrote:

> In
>
>
> ExternalPipe>>
> makePipe
> | t1 |
> *t1 := OSProcess accessor makePipeHandles.*
> t1 isNil
> ifTrue: [self error: 'cannot create OS pipe']
> ifFalse: [self
> reader: (AttachableFileStream
> name: 'pipeReader'
> attachTo: (t1 at: 1)
> writable: false).
> self
> writer: (AttachableFileStream
> name: 'pipeWriter'
> attachTo: (t1 at: 2)
> writable: true)]
>
>
> OSProcess accessor is a  * 'a UnixOSProcessAccessor on pid 19790'*
>
>
> Where *self makePipeHandles* returns and undefined object.
>
> How should I think about this?
>
> thnks
>
>
>
>
>
>
> --
> Sent from: http://forum.world.st/Squeak-Beginners-f107673.html
> _______________________________________________
> Beginners mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners
tty
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CommandShell/OSPRocess primCreatePipe failing. How to        debug?

tty
Hi David.

I am looping over 3221 classes that represent Lua modules and asking OSProcess to create the directories where their content to be stored should go and using Filestream to create and write the file contents.

Transcript clear.
[(WikitextModule allSubclasses select:[:module  |  (module pagetitle indexOf:$/ ifAbsent:[0]) > 0])
   do:[:m | |ss f d cmd shell content  ios|
       ss := m pagetitle squeakToUtf8 findBetweenSubStrs: '/'.
      d := ss collect:[:each | each] from: 1 to: (ss size -1).
     ((d at:1) = 'Sandbox') | ((d at:1) beginsWith:'User:')     "handle Sandbox and User: here"
       ifTrue:[]
       ifFalse:[
                d := d inject:'' into:[:a :b | a, '/',  b].
                f := ('lua',d, '/', ss last, '.lua') squeakToUtf8  .    "ios writes to shared by default"
                cmd := 'mkdir -p shared/lua', d.   "commandshell writes to squeak root directory, so we prepende 'shared' to it"
               Transcript show: (m name), ' : ', cmd; cr.
               shell := PipeableOSProcess bash. 
               shell pipeFromOutput reader setNonBlocking. 
               shell exec: cmd.
               (Delay forMilliseconds: 400) wait. 
               Transcript show: shell upToEnd ; cr.
               shell close. 
               content := m lua squeakToUtf8.
               ios := FileStream newFileNamed: f.
               ios isNil
                  ifFalse:[
                        content storeOn: ios.
                         ios close.
       ]]]] ifError:[:x :y | Transcript show: x;cr. y close. thisContext resume]

The OSPRocess repeatedly executes the command 'mkdir -p path/stored/in/the/class' .

Since I posted my question, I have broken the loop into subsets of the 3221. I initially attempted 500 record blocks, and that worked up to about 1500 loops.
I then broke the 1500 to 2000 into increments of 100 and it just completed the 1500->2000 stretch. (eliminating the possibility that something within the modules were the cause of the error)

copyFrom: 501 to: 1000
copyFrom: 1001 to: 1500
copyFrom: 1501 to: 1600
copyFrom: 1601 to: 1700
copyFrom: 1701 to: 1800
copyFrom: 1801 to: 1900
copyFrom: 1901 to: 2000
copyFrom: 2001 to: 2500
copyFrom: 2501 to: 3000


My hunch is that pounding OSPRocess is causing the bug.

BTW, OSProcess and CommandShell are GREAT tools and I very much appreciate you having developed them. 

Thank you for your time.

tty





---- On Fri, 30 Aug 2019 10:04:09 -0400 David T. Lewis <[hidden email]> wrote ----

Can you say something about how you get to the point of getting
this error, and what operating system you are using? The code
snippet looks odd to me because it appears to be a decompiled
version of the method in OSProcess. Maybe you do not have a
changes file to go with your image file?

A typical reason for failing to create pipes is that you have
somehow opened too many file handles for your VM, possibly by
opening a lot of pipes and not closing them completely.

Dave

On Fri, Aug 30, 2019 at 07:58:50AM -0500, tty wrote:

> In
>
>
> ExternalPipe>>
> makePipe
>     | t1 |
>     *t1 := OSProcess accessor makePipeHandles.*
>     t1 isNil
>         ifTrue: [self error: 'cannot create OS pipe']
>         ifFalse: [self
>                 reader: (AttachableFileStream
>                         name: 'pipeReader'
>                         attachTo: (t1 at: 1)
>                         writable: false).
>             self
>                 writer: (AttachableFileStream
>                         name: 'pipeWriter'
>                         attachTo: (t1 at: 2)
>                         writable: true)]
>
>
> OSProcess accessor is a * 'a UnixOSProcessAccessor on pid 19790'*
>
>
> Where *self makePipeHandles* returns and undefined object.
>
> How should I think about this?
>
> thnks
>
>
>
>
>
>
> --
> Sent from: http://forum.world.st/Squeak-Beginners-f107673.html
> _______________________________________________
> Beginners mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners



_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners
tty
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CommandShell/OSPRocess primCreatePipe failing. How to&nbsp; &nbsp; &nbsp; &nbsp; debug?

tty
In reply to this post by David T. Lewis
Also...

I am on Linux.
The weirdness of the methods only happens during the error. I have seen it happen in my classes to.

If I close the image without saving the normal method text appears.

Also, loading a lot of classes into Smalltalk causes strange stuff.

I attempted to load over 1/2 million classes into my image and Squeak really struggled when I got over 20,000 classes. 

So, it could be related to the 3221 classes I have imported that are part of the problem.


_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners
tty
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: CommandShell/OSPRocess primCreatePipe failing. How to&nbsp; &nbsp; &nbsp; &nbsp; debug?

tty
In reply to this post by David T. Lewis
Hi David.



This got me through all the records:

Transcript clear.
{1->100
. 101->200
. 201->300
. 301->400
. 401->500
. 501->600
. 601->700
. 701->800
. 801->900
. 901->1000} do:[:a |
Transcript show: a key asString, '->', a value asString; cr.
WikitextParserProxyBuilder writeSubModulesToFiles: (a key) to: (a value).
(Delay forMilliseconds: 2000) wait.
]
Transcript clear.
{ 1001->1100
. 1101->1200
. 1201->1300
. 1301->1400
. 1401->1500
. 1501->1600
. 1601->1700
. 1701->1800
. 1801->1900
. 1901->2000 }do:[:a |
Transcript show: a key asString, '->', a value asString; cr.
WikitextParserProxyBuilder writeSubModulesToFiles: (a key) to: (a value).
(Delay forMilliseconds: 2000) wait.
]
Transcript clear.

{ 2001->2100
. 2101->2200
. 2201->2300
. 2301->2400
. 2401->2500
. 2501->2600
. 2601->2700
}  do:[:a |
Transcript show: a key asString, '->', a value asString; cr.
WikitextParserProxyBuilder writeSubModulesToFiles: (a key) to: (a value).
(Delay forMilliseconds: 2000) wait.
]

where "WikitextParserProxyBuilder writeSubModulesToFiles: (a key) to: (a value)." contains the block DoIt code from earlier in the thread.

Thanks for you help!

tty



_______________________________________________
Beginners mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/beginners
Free forum by Nabble Edit this page