OSProcess command - non responding image when calling often
Locked
12
12
OSProcess command - non responding image when calling often
 
|
Hi,
sorry but I have another issue with OSProcess. For my app, I want to show small .png previews from my .pdfs. For this reason, I use an external software, currently pdfbox. It works fine, but regularly, I get a hanging image. I tried to reproduce it and found out, that it is not a problem with pdfbox (http://pdfbox.apache.org), because from command line, I can call it often without problems. There can be (hopefully) many users in my app with many previews at one time. So I have to call it often. For reproduction of the OSProcess issue, I used another simple command (from my other OSProcess umlaut issue :-)) to test it. calling this, results in a non responding image: 20 timesRepeat: [OSProcess command: ('cp /Library/WebServer/Documents/reports/bar.pdf /Library/WebServer/Documents/reports/test-c.pdf').]. also this: 20 timesRepeat: [OSProcess command: ('echo "Hi there"').] My original code is similar to this but here only for information: OSProcess command: ('{1} -jar {2} PDFToImage -format PNG -page 1 -dpi 100 {3}' format: {'Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java'. '/Users/sabine/Desktop/pdfbox-app-2.0.1.jar'. '/Library/WebServer/Documents/reports/201411121510-112859865-3378921/20150605_170_IFA_Berlin_Reisekostenabrechnung.pdf'}). So, my question is: is it wrong, calling OSProcess like this? How to do it instead? I am sorry to ask again and hope that there is a simple solution. Or is it a bug in OSProcess? Regards Sabine |
Re: OSProcess command - non responding image when calling often
|
Hi Sabine, which version of Pharo are you using? I had similar issues, and I managed to reduce those occurrences to a reasonable level by changing the way I called OSProcess (and a lot of help from Dave). I do have sometime lockups with OSSubprocess as well, but those are also rare. (I do think this is a bug somewhere in the VM which makes it loose signals) The code I use is in GitFileTree: a low-level way of calling OSProcess which seems good at minimizing lockups. Thierry 2016-06-07 9:31 GMT+02:00 Sabine Manaa <[hidden email]>: Hi, |
Re: OSProcess command - non responding image when calling often
|
|
Hi Thierry, hi Pharo Team, Thierry, thank you! I switched to Pharo5 a few days ago. I am not yet productive/online. I had a look at your code in a Pharo 3 image. It is the code after >>runGitCommand:in:, right? Puh, I don't want to go so deep and implement that again for a small feature like my png previews.... So I am asking polite to the pharo developers team if there are plans to provide a stable solution for calling external programs as described. My production system will be a windows system, so I need this for windows, too. As far as I understand, the new OSSubprocess is (currently) only for unix. It is no problem for me to wait some time for this. But it would be nice to know what the plans are, to decide what to do (and not programming something which will be solved in future). Regards Sabine 2016-06-07 9:43 GMT+02:00 Thierry Goubier [via Smalltalk] <[hidden email]>:
|
Re: OSProcess command - non responding image when calling often
|
|
It's strange that I was using GitFileTree over OSProcess for a long time without issue and yet every time I tried to use OSProcess directly I had this locking up.
> So I am asking polite to the pharo developers team if there are plans to > provide a stable solution for calling external programs as described. > > My production system will be a windows system, so I need this for windows, > too. > As far as I understand, the new OSSubprocess is (currently) only for unix. Yes, there's ProcessWrapper for Windows. An alternative approach might be to use direct FFI calls, e.g. MyClass class>>system: system: command "Perform OS system() call." ^ self ffiCall: #(int system #(char * command)) module: LibC And then you can do MyClass system: 'cp a.pdf b.pdf'. Although it still locks up my image from time to time… but at least order of magnite less. Also if you are targeting Windows, I suggest to look at Torsten's 'OSWindows' project (available from the catalog browser). Peter > > It is no problem for me to wait some time for this. > But it would be nice to know what the plans are, to decide what to do (and > not programming something which will be solved in future). > > Regards > Sabine > > > > > 2016-06-07 9:43 GMT+02:00 Thierry Goubier [via Smalltalk] < > [hidden email]>: > > > Hi Sabine, > > > > which version of Pharo are you using? I had similar issues, and I managed > > to reduce those occurrences to a reasonable level by changing the way I > > called OSProcess (and a lot of help from Dave). I do have sometime lockups > > with OSSubprocess as well, but those are also rare. > > > > (I do think this is a bug somewhere in the VM which makes it loose signals) > > > > The code I use is in GitFileTree: a low-level way of calling OSProcess > > which seems good at minimizing lockups. > > > > Thierry > > > > > > > > 2016-06-07 9:31 GMT+02:00 Sabine Manaa <[hidden email] > > <http:///user/SendEmail.jtp?type=node&node=4899548&i=0>>: > > > >> Hi, > >> > >> sorry but I have another issue with OSProcess. > >> > >> For my app, I want to show small .png previews from my .pdfs. For this > >> reason, I use an external software, currently pdfbox. It works fine, but > >> regularly, I get a hanging image. I tried to reproduce it and found out, > >> that it is not a problem with pdfbox (http://pdfbox.apache.org), because > >> from command line, I can call it often without problems. There can be > >> (hopefully) many users in my app with many previews at one time. So I have > >> to call it often. > >> > >> For reproduction of the OSProcess issue, I used another simple command > >> (from > >> my other OSProcess umlaut issue :-)) to test it. > >> > >> calling this, results in a non responding image: > >> > >> 20 timesRepeat: [OSProcess command: ('cp > >> /Library/WebServer/Documents/reports/bar.pdf > >> /Library/WebServer/Documents/reports/test-c.pdf').]. > >> > >> also this: > >> 20 timesRepeat: [OSProcess command: ('echo "Hi there"').] > >> > >> My original code is similar to this but here only for information: > >> OSProcess > >> command: > >> ('{1} -jar {2} PDFToImage -format PNG -page 1 > >> -dpi 100 {3}' > >> format: > >> {'Library/Internet\ > >> Plug-Ins/JavaAppletPlugin.plugin/Contents/Home/bin/java'. > >> > >> '/Users/sabine/Desktop/pdfbox-app-2.0.1.jar'. > >> > >> > >> '/Library/WebServer/Documents/reports/201411121510-112859865-3378921/20150605_170_IFA_Berlin_Reisekostenabrechnung.pdf'}). > >> > >> > >> So, my question is: is it wrong, calling OSProcess like this? How to do it > >> instead? > >> > >> I am sorry to ask again and hope that there is a simple solution. > >> Or is it a bug in OSProcess? > >> > >> Regards > >> Sabine > >> > >> > >> > >> > >> -- > >> View this message in context: > >> http://forum.world.st/OSProcess-command-non-responding-image-when-calling-often-tp4899540.html > >> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. > >> > >> > > > > > > ------------------------------ > > If you reply to this email, your message will be added to the discussion > > below: > > > > http://forum.world.st/OSProcess-command-non-responding-image-when-calling-often-tp4899540p4899548.html > > To start a new topic under Pharo Smalltalk Users, email > > [hidden email] > > To unsubscribe from Pharo Smalltalk Users, click here > > < > > . > > NAML > > <http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > > > > -- > View this message in context: http://forum.world.st/OSProcess-command-non-responding-image-when-calling-often-tp4899540p4899580.html > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. |
Re: OSProcess command - non responding image when calling often
|
|
In reply to this post by Sabine Manaa
2016-06-07 10:44 GMT+02:00 Sabine Manaa <[hidden email]>:
Yes. I'm not sure I backported in Pharo3 all the changes there. I know I did spend a lot of time at one point fighting with random lock-ups (apparently triggered more frequently by an upgrade on the underlying linux...)
It's only one method, to be honest. Sometimes, you want just something that works... and do whatever is necessary.
I'm sure there is. The thing is, if the bug is somewhere deep, then it may takes time to wait for it. This is a luxury I often don't have, and, given the low cost of the alternative implementation, then I just do it.
ProcessWrapper works perfectly fine, as far as I was told. Took a while to get it right for GitFileTree, so maybe it is some code to look at (MCFileTreeGitRepository class>>#runProcessWrapperGitCommand:in:)
Just consider that we need that feature now, we have code for that, and that it works now while waiting for a better solution :) Regards, Thierry
|
Re: OSProcess command - non responding image when calling often
|
|
In reply to this post by Peter Uhnak
2016-06-07 11:47 GMT+02:00 Peter Uhnak <[hidden email]>: It's strange that I was using GitFileTree over OSProcess for a long time without issue and yet every time I tried to use OSProcess directly I had this locking up. Yes. This is a bit worrying; the low-level OSProcess call was a bit of a stopgap while waiting for either OSProcess or the vm to sort the underlying issue (which is more significant than just my use); that it locks up under normal use is not good and should be reported. I allways considered GitFileTree to be a heavy hitter on OSProcess, running hundreds of external commands to load a complex project, so that if I would lock up, it wouldn't be a normal pattern of use.
Than GitFileTree or OSProcess? Still, you get lockups... which isn't good. Thierry
|
Re: OSProcess command - non responding image when calling often
|
|
On Tue, Jun 07, 2016 at 01:22:04PM +0200, Thierry Goubier wrote:
> 2016-06-07 11:47 GMT+02:00 Peter Uhnak <[hidden email]>: > > > It's strange that I was using GitFileTree over OSProcess for a long time > > without issue and yet every time I tried to use OSProcess directly I had > > this locking up. > > > > > Yes. This is a bit worrying; the low-level OSProcess call was a bit of a > stopgap while waiting for either OSProcess or the vm to sort the underlying > issue (which is more significant than just my use); that it locks up under > normal use is not good and should be reported. > > I allways considered GitFileTree to be a heavy hitter on OSProcess, running > hundreds of external commands to load a complex project, so that if I would > lock up, it wouldn't be a normal pattern of use. > > > > > > > So I am asking polite to the pharo developers team if there are plans to > > > provide a stable solution for calling external programs as described. > > > > > > My production system will be a windows system, so I need this for > > windows, > > > too. > > > As far as I understand, the new OSSubprocess is (currently) only for > > unix. > > > > Yes, there's ProcessWrapper for Windows. > > > > An alternative approach might be to use direct FFI calls, e.g. > > > > MyClass class>>system: > > system: command > > "Perform OS system() call." > > > > ^ self ffiCall: #(int system #(char * command)) module: LibC > > > > And then you can do MyClass system: 'cp a.pdf b.pdf'. > > > > Although it still locks up my image from time to time… but at least order > > of magnite less. > > > > Than GitFileTree or OSProcess? Still, you get lockups... which isn't good. No, GitFileTree never locked up my image. This was always my use. Peter |
Re: OSProcess command - non responding image when calling often
|
|
Hi guys, I will try to answer many things at once: 1) Sabine, OSSubprocess works for Unix derived, not only Linux, that includes OSX as well. 2) Tierry,Sabine for the case of loosing signals, the most trustful solution (in OSSubprocess) is to rely on a delay pooling instead of on the child reaper (based on signals). I am talking exactly about this API: https://github.com/marianopeck/OSSubprocess#delay-based-polling-waiting 3) I found a problem/bug with waits and signals in Pharo 5.0 as reported here: https://pharo.fogbugz.com/f/cases/18359/Problem-with-DelayExperimentalSpinScheduler-and-delay. That should also affect OSProcess. What I did in OSSubprocess v0.2.4 (https://github.com/marianopeck/OSSubprocess/tree/v0.2.4) is automatically set the mentioned workaround in the issue tracker. This may fix your issue. Hope any of this helps. Let us know. Cheers, On Tue, Jun 7, 2016 at 10:21 AM, Peter Uhnak <[hidden email]> wrote:
|
Re: OSProcess command - non responding image when calling often
|
|
Hi Thierry, Peter and Mariano,
thank you for your feedback. I have to look at your suggestions and come back then. It is always great to get so immediate answers! Regards Sabine |
Re: OSProcess command - non responding image when calling often
|
|
In reply to this post by Mariano Martinez Peck
Hi Mariano,
2016-06-07 16:04 GMT+02:00 Mariano Martinez Peck <[hidden email]>:
Which is what GitFileTree uses. I remember asking you for a similar API :) Polling to avoid the child reaper is also what GitFileTree does on the OSProcess side.
I understand that. I had a few lock-ups prior to that with GitFileTree and OSSubprocess, even with the polling interface, and I noticed that discussion (and the fact someone else complained of lost signals when connected to a database as well). But I don't know what the status is exactly for each version of the Pharo VM. Thanks, Thierry
|
Re: OSProcess command - non responding image when calling often
|
|
In reply to this post by Peter Uhnak
2016-06-07 15:21 GMT+02:00 Peter Uhnak <[hidden email]>:
Ok. I did had lockups with GitFileTree... a lot. Rarely now. But I still consider a GitFileTree lockup a possible occurrence. Thierry
|
Re: OSProcess command - non responding image when calling often
|
|
Hi Thierry, all,
you were right, it was not so complicated to write my own version for calling an external program based on your Github code. It worked fine on Pharo3+4 but on Pharo5 there are the locking problems described, also in your code. Until there is a decision if supporting windows in OSSubprocess or not and until the locking problem is not solved, I go back to Pharo4. (I loose to much time which I don't have). I will miss the code critics :-) Thank you all for your help. Regards Sabine |
Re: OSProcess command - non responding image when calling often
|
|
Sabine, Did you test if the one-liner (Delay delaySchedulerClass: DelayMillisecondScheduler.) fixes these lookups? Thanks On Wed, Jun 8, 2016 at 5:31 AM, Sabine Manaa <[hidden email]> wrote: Hi Thierry, all, |
Re: OSProcess command - non responding image when calling often
|
|
Hi Mariano, yes, I tried this. It creates a non-responding image. I have to kill it. Regards Sabine 2016-06-08 15:26 GMT+02:00 Mariano Martinez Peck [via Smalltalk] <[hidden email]>:
|
Re: OSProcess command - non responding image when calling often
|
|
On Wed, Jun 8, 2016 at 10:33 AM, Sabine Manaa <[hidden email]> wrote:
Uhhh too bad :( I thought we were talking about the same issue.
|
Re: OSProcess command - non responding image when calling often
|
|
I dont understand
Am Mittwoch, 8. Juni 2016 schrieb Mariano Martinez Peck [via Smalltalk] :
|
Re: OSProcess command - non responding image when calling often
|
|
As I said in a previous email I found out a bug in Pharo 5.0 related to delays. This affected OSSubprocess but could affect OSProcess as well. The workaround is setting ANOTHER kind of delay scheduler than the one that has the bug. I thought that the "locks" you were having could have been because of this issue. But if the workaround does not solve it, then the lockups you are having are because of another reason :( Cheers, On Wed, Jun 8, 2016 at 10:49 AM, Sabine Manaa <[hidden email]> wrote: I dont understand |
Re: OSProcess command - non responding image when calling often
|
|
Hi Mariano, ok, now I understand. But it is not clear to me, why "Delay delaySchedulerClass: DelayMillisecondScheduler" creates the locked image in my case. With a new Pharo5 image, there is no lock. I have to find out, what is causing this locks and report this to you, I hope tomorrow. Sabine 2016-06-08 15:58 GMT+02:00 Mariano Martinez Peck [via Smalltalk] <[hidden email]>:
|
Re: OSProcess command - non responding image when calling often
|
|
Hi Mariano, if you take the download Image from the Pharo website, then load commandShell and then zinc from the catalog browser and then do Delay delaySchedulerClass: DelayMillisecondScheduler. and then 20 timesRepeat: [OSProcess command: ('echo "Hi there"').] Then you will get the lock. Does this help to find the reason for the problem? Regards Sabine |
Re: OSProcess command - non responding image when calling often
|
|
Sabine, I did not yet tested OSProcess but with OSSubprocess (in 5.0) I cannot reproduce the issue: Polling strategy works: Delay delaySchedulerClass: DelayMillisecondScheduler. 20 timesRepeat: [OSSUnixSubprocess new command: 'echo'; arguments: { ' }; redirectStdout; runAndWaitPollingEvery: (Delay forMilliseconds: 50) retrievingStreams: true onExitDo: [ :process :outString :errString | Transcript show: outString;cr . ]. ]. Semaphore-based (child watcher) works: Delay delaySchedulerClass: DelayMillisecondScheduler. 20 timesRepeat: [OSSUnixSubprocess new command: 'echo'; arguments: { 'Hi There' }; redirectStdout; runAndWaitOnExitDo: [ :process :outString :errString | Transcript show: outString;cr . ]. ]. Using shellCommand: (bash) instead of #command: and #arguments: does work too: Delay delaySchedulerClass: DelayMillisecondScheduler. 20 timesRepeat: [OSSUnixSubprocess new shellCommand: ('echo "Hi there"'); redirectStdout; runAndWaitOnExitDo: [ :process :outString :errString | Transcript show: outString;cr . ]. ]. On Wed, Jun 8, 2016 at 3:11 PM, Sabine Manaa <[hidden email]> wrote:
|
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |