reading from a named pipe in Pharo

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

reading from a named pipe in Pharo

Siemen Baader
Hi,

I'd like to use an external tool to edit HTML code and then store and version it in my Pharo image. I'd like to create a named pipe that the external application will save to, and then have Pharo watch for whenever the named pipe is written to and then execute code to handle the data from there.

But I can't find any place to register a callback on FileReference and similar classes that will notify Pharo when the named pipe is changed, and when the external editor will read from it. Is there any support for this?

I know there are workarounds like polling or reading the file when it is used, but I'd prefer to wire things together directly and avoid duplicating the data if possible.

thanks,
Siemen
Reply | Threaded
Open this post in threaded view
|

Re: reading from a named pipe in Pharo

Mariano Martinez Peck
Hi Siemen, 

You may want to check the pipe support code in OSSubprocess [1]. Note that the pipes should work outside of OSSubprocess. You can read the documentation as well as the unit tests for the pipes. 

Cheers, 


On Sat, May 5, 2018 at 8:29 AM, Siemen Baader <[hidden email]> wrote:
Hi,

I'd like to use an external tool to edit HTML code and then store and version it in my Pharo image. I'd like to create a named pipe that the external application will save to, and then have Pharo watch for whenever the named pipe is written to and then execute code to handle the data from there.

But I can't find any place to register a callback on FileReference and similar classes that will notify Pharo when the named pipe is changed, and when the external editor will read from it. Is there any support for this?

I know there are workarounds like polling or reading the file when it is used, but I'd prefer to wire things together directly and avoid duplicating the data if possible.

thanks,
Siemen



--
Reply | Threaded
Open this post in threaded view
|

Re: reading from a named pipe in Pharo

Siemen Baader
Thanks, I'll check it out!

-- Siemen

On Sat, May 5, 2018 at 4:26 PM, Mariano Martinez Peck <[hidden email]> wrote:
Hi Siemen, 

You may want to check the pipe support code in OSSubprocess [1]. Note that the pipes should work outside of OSSubprocess. You can read the documentation as well as the unit tests for the pipes. 

Cheers, 


On Sat, May 5, 2018 at 8:29 AM, Siemen Baader <[hidden email]> wrote:
Hi,

I'd like to use an external tool to edit HTML code and then store and version it in my Pharo image. I'd like to create a named pipe that the external application will save to, and then have Pharo watch for whenever the named pipe is written to and then execute code to handle the data from there.

But I can't find any place to register a callback on FileReference and similar classes that will notify Pharo when the named pipe is changed, and when the external editor will read from it. Is there any support for this?

I know there are workarounds like polling or reading the file when it is used, but I'd prefer to wire things together directly and avoid duplicating the data if possible.

thanks,
Siemen



--

Reply | Threaded
Open this post in threaded view
|

Re: reading from a named pipe in Pharo

Alistair Grant
In reply to this post by Mariano Martinez Peck
Hi Siemen & Mariano,

On 5 May 2018 at 16:26, Mariano Martinez Peck <[hidden email]> wrote:
> Hi Siemen,
>
> You may want to check the pipe support code in OSSubprocess [1]. Note that
> the pipes should work outside of OSSubprocess. You can read the
> documentation as well as the unit tests for the pipes.
>
> Cheers,
>
> [1] https://github.com/marianopeck/OSSubprocess

OSSPipe seems to assume that is controlling both ends of the pipe,
while it may be that Siemen just wants to read from the pipe.  Anyway,
Mariano mentioned that pipes should work outside of OSSubprocess, and
I've made some modifications to FilePlugin which should facilitate
that, so thought I'd have a go.  This still doesn't completely meet
Siemen's requirements as it polls the stream, but...  I've attached a
little demo class, tested only on Ubuntu 16.04.  Be warned, I've never
used Unix pipes before, so feel free to critique the code:


NamedPipeEcho provides sample code showing how to read from a Unix
named pipe and echo the contents to stdout.

Interesting attributes of the sample code include:

- It is just a sample, it isn't intended to be production ready.
- It marks the pipe non-blocking so that multiple pipes can be
processed without the image blocking when reading.
- It's polling, rather than event driven.
- It doesn't have any external dependencies outside the core Pharo
image and standard Pharo VM.
- It defines a method that directly calls a primitive - I consider
this bad practice and would like to make the primitive generally
available through FilePlugin.


The simplest way to run this is in one terminal (bash shell):

$ mkfifo /dev/shm/pharopipe
$ pharo --headless Pharo.image eval "(NamedPipeEcho on:
'/dev/shm/pharopipe') run"

and in a second terminal (bash shell):

$ pharo --headless Pharo.image eval "NamedPipeEcho sampleWriteTo:
'/dev/shm/pharopipe'"

You should see:

Hello World
Line 2

in the first terminal, with a 15 second delay between the lines and exit.


There's also AsyncFile, which should make it event driven instead of
polled, but I haven't had a chance to look at that yet.

This is tested with:


Pharo7.0alpha
Build information:
Pharo-7.0+alpha.build.839.sha.675decd24230e0ef7d1579af07c2112c104c6d7b
(64 Bit)


5.0-201804122226  Thu Apr 12 22:34:48 UTC 2018 gcc 4.8 [Production
Spur 64-bit VM]
CoInterpreter VMMaker.oscog-eem.2361 uuid:
7ca2f89a-de70-422f-b92b-54f91ac4e47b Apr 12 2018
StackToRegisterMappingCogit VMMaker.oscog-eem.2361 uuid:
7ca2f89a-de70-422f-b92b-54f91ac4e47b Apr 12 2018
VM: 201804122226 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
Date: Thu Apr 12 15:26:17 2018 -0700 $ CommitHash: 07c6dc3 $
Plugins: 201804122226 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
Linux travis-job-b95e67e4-9fb2-4d69-a59f-1fbf859b912c
4.4.0-101-generic #124~14.04.1-Ubuntu SMP Fri Nov 10 19:05:36 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux
plugin path: /home/alistair/pharo7/Issue21692/vm/pharo-vm/lib/pharo/5.0-201804122226
[default: /home/alistair/pharo7/Issue21692/vm/pharo-vm/lib/pharo/5.0-201804122226/]


Earlier VMs probably won't work as #atEnd will always return true or
false (I forget which).

Cheers,
Alistair

AKG-PipePlay.st (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: reading from a named pipe in Pharo

Siemen Baader
Wow, thanks for your help, Alistair! I'll try it out.

cheers,
Siemen


On Sun, May 6, 2018 at 7:47 PM, Alistair Grant <[hidden email]> wrote:
Hi Siemen & Mariano,

On 5 May 2018 at 16:26, Mariano Martinez Peck <[hidden email]> wrote:
> Hi Siemen,
>
> You may want to check the pipe support code in OSSubprocess [1]. Note that
> the pipes should work outside of OSSubprocess. You can read the
> documentation as well as the unit tests for the pipes.
>
> Cheers,
>
> [1] https://github.com/marianopeck/OSSubprocess

OSSPipe seems to assume that is controlling both ends of the pipe,
while it may be that Siemen just wants to read from the pipe.  Anyway,
Mariano mentioned that pipes should work outside of OSSubprocess, and
I've made some modifications to FilePlugin which should facilitate
that, so thought I'd have a go.  This still doesn't completely meet
Siemen's requirements as it polls the stream, but...  I've attached a
little demo class, tested only on Ubuntu 16.04.  Be warned, I've never
used Unix pipes before, so feel free to critique the code:


NamedPipeEcho provides sample code showing how to read from a Unix
named pipe and echo the contents to stdout.

Interesting attributes of the sample code include:

- It is just a sample, it isn't intended to be production ready.
- It marks the pipe non-blocking so that multiple pipes can be
processed without the image blocking when reading.
- It's polling, rather than event driven.
- It doesn't have any external dependencies outside the core Pharo
image and standard Pharo VM.
- It defines a method that directly calls a primitive - I consider
this bad practice and would like to make the primitive generally
available through FilePlugin.


The simplest way to run this is in one terminal (bash shell):

$ mkfifo /dev/shm/pharopipe
$ pharo --headless Pharo.image eval "(NamedPipeEcho on:
'/dev/shm/pharopipe') run"

and in a second terminal (bash shell):

$ pharo --headless Pharo.image eval "NamedPipeEcho sampleWriteTo:
'/dev/shm/pharopipe'"

You should see:

Hello World
Line 2

in the first terminal, with a 15 second delay between the lines and exit.


There's also AsyncFile, which should make it event driven instead of
polled, but I haven't had a chance to look at that yet.

This is tested with:


Pharo7.0alpha
Build information:
Pharo-7.0+alpha.build.839.sha.675decd24230e0ef7d1579af07c2112c104c6d7b
(64 Bit)


5.0-201804122226  Thu Apr 12 22:34:48 UTC 2018 gcc 4.8 [Production
Spur 64-bit VM]
CoInterpreter VMMaker.oscog-eem.2361 uuid:
7ca2f89a-de70-422f-b92b-54f91ac4e47b Apr 12 2018
StackToRegisterMappingCogit VMMaker.oscog-eem.2361 uuid:
7ca2f89a-de70-422f-b92b-54f91ac4e47b Apr 12 2018
VM: 201804122226 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
Date: Thu Apr 12 15:26:17 2018 -0700 $ CommitHash: 07c6dc3 $
Plugins: 201804122226 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $
Linux travis-job-b95e67e4-9fb2-4d69-a59f-1fbf859b912c
4.4.0-101-generic #124~14.04.1-Ubuntu SMP Fri Nov 10 19:05:36 UTC 2017
x86_64 x86_64 x86_64 GNU/Linux
plugin path: /home/alistair/pharo7/Issue21692/vm/pharo-vm/lib/pharo/5.0-201804122226
[default: /home/alistair/pharo7/Issue21692/vm/pharo-vm/lib/pharo/5.0-201804122226/]


Earlier VMs probably won't work as #atEnd will always return true or
false (I forget which).

Cheers,
Alistair