Smalltalk › Pharo › Pharo Smalltalk Developers

Pavel's ChangeLog week of 2018-02-26

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

Pavel Krivanek-3

Pavel's ChangeLog week of 2018-02-26

Hi,

Guille is working on the Iceberg improvements and he wanted to be able to open a terminal window on top of the repository and interact with it via the command line. So we looked at this issue because I already in December made some experiments with the terminal emulation in Pharo.

In past, the Squeak had a working terminal emulation that used PseudoTTYPlugin. The VM is not built with this code for a long time but I tried to replace it with a small C library and then wrote an FFI interface to it. Together with that, I ported most of the old code Squeak code to Pharo.

With Guille we tried to avoid usage of such external library and wrote an FFI interface to all the required LibC functions. We were successful but we realized that there are several issues that are limiting us.

When you want to execute a separate process for the program that you want to open in terminal (typically the Bash), you need to redirect the standard IO files, create a fork of your process, do some additional initialization in it and call 'exec' on it. In the parent process, you change redirected IO files back to the original values.

But the problem is that between the FFI calls from Smalltalk the VM can do a lot of things including garbage collection etc. On OS X the fork() function has the following limitation described in man:

"There are limits to what you can do in the child process. To be totally safe you should restrict your yourself to only executing async-signal safe operations until such time as one of the exec functions is called. All APIs, including global data symbols, in any framework or library should be assumed to be unsafe after a fork() unless explicitly documented to be safe or async-signal safe. If you need to use these frameworks in the child process, you must exec. In this
situation it is reasonable to exec your-self. yourself."

As the result in most cases (but not all) the fork() and exec() pair from the Smalltalk side fails on OS X. Linux does not have this limitation however even there we found an issue. It is bound to the fact that fork() makes a fork of all the parent process that uses the same resources. As soon as Pharo is opened in a window and X11 is involved (the window wants to be repainted), it can lead to the VM crash.

So we learned that unfortunately we currently cannot use image-only FFI code for this task. We need a C library or VM plugin.

The repository of the terminal emulator is here: https://github.com/pavel-krivanek/terminal

and can be loaded using the following code:

Metacello new
baseline: 'TerminalEmulator';
repository: 'github://pavel-krivanek/terminal/src';
load.

#TerminalEmulator asClass compileLibrary.

It compiles and links the small library with only one function using the GCC so the machine needs to have a proper development environment.

The terminal emulator is in very early stage and has a lot of issues like processes cleanup, drawing, keyboard input etc. etc. If you are interested in it, feel free to contribute.

Cheers,
-- Pavel

Stephane Ducasse-3

Re: Pavel's ChangeLog week of 2018-02-26

Thanks pavel such experiments are super interesting because they push us outside our confort zone.

On Mon, Mar 5, 2018 at 10:29 AM, Pavel Krivanek <[hidden email]> wrote:

Hi,

Guille is working on the Iceberg improvements and he wanted to be able to open a terminal window on top of the repository and interact with it via the command line. So we looked at this issue because I already in December made some experiments with the terminal emulation in Pharo.

In past, the Squeak had a working terminal emulation that used PseudoTTYPlugin. The VM is not built with this code for a long time but I tried to replace it with a small C library and then wrote an FFI interface to it. Together with that, I ported most of the old code Squeak code to Pharo.

With Guille we tried to avoid usage of such external library and wrote an FFI interface to all the required LibC functions. We were successful but we realized that there are several issues that are limiting us.

When you want to execute a separate process for the program that you want to open in terminal (typically the Bash), you need to redirect the standard IO files, create a fork of your process, do some additional initialization in it and call 'exec' on it. In the parent process, you change redirected IO files back to the original values.

But the problem is that between the FFI calls from Smalltalk the VM can do a lot of things including garbage collection etc. On OS X the fork() function has the following limitation described in man:

"There are limits to what you can do in the child process. To be totally safe you should restrict your yourself to only executing async-signal safe operations until such time as one of the exec functions is called. All APIs, including global data symbols, in any framework or library should be assumed to be unsafe after a fork() unless explicitly documented to be safe or async-signal safe. If you need to use these frameworks in the child process, you must exec. In this
situation it is reasonable to exec your-self. yourself."

As the result in most cases (but not all) the fork() and exec() pair from the Smalltalk side fails on OS X. Linux does not have this limitation however even there we found an issue. It is bound to the fact that fork() makes a fork of all the parent process that uses the same resources. As soon as Pharo is opened in a window and X11 is involved (the window wants to be repainted), it can lead to the VM crash.

So we learned that unfortunately we currently cannot use image-only FFI code for this task. We need a C library or VM plugin.

The repository of the terminal emulator is here: https://github.com/pavel-krivanek/terminal

and can be loaded using the following code:

Metacello new
baseline: 'TerminalEmulator';
repository: 'github://pavel-krivanek/terminal/src';
load.

#TerminalEmulator asClass compileLibrary.

It compiles and links the small library with only one function using the GCC so the machine needs to have a proper development environment.

The terminal emulator is in very early stage and has a lot of issues like processes cleanup, drawing, keyboard input etc. etc. If you are interested in it, feel free to contribute.

Cheers,
-- Pavel

alistairgrant

Re: Pavel's ChangeLog week of 2018-02-26

In reply to this post by Pavel Krivanek-3

Hi Pavel,

I tried to get this working on Ubuntu 16.04 and found the following:

- The compilation failed unless I added the '-fPIC' flag (as suggested by gcc).
- The example code in the TerminalEmulator class comment references
ProcessEndpoint, but the class doesn't exist.

Pharo 7.0
Build information:
Pharo-7.0+alpha.build.674.sha.57447e756fa6fcf1877f9d0d1cda235b3b63c807
(64 Bit)

Cheers,
Alistair

On 5 March 2018 at 10:29, Pavel Krivanek <[hidden email]> wrote:

> Hi,
>
> Guille is working on the Iceberg improvements and he wanted to be able to
> open a terminal window on top of the repository and interact with it via the
> command line. So we looked at this issue because I already in December made
> some experiments with the terminal emulation in Pharo.
>
> In past, the Squeak had a working terminal emulation that used
> PseudoTTYPlugin. The VM is not built with this code for a long time but I
> tried to replace it with a small C library and then wrote an FFI interface
> to it. Together with that, I ported most of the old code Squeak code to
> Pharo.
>
> With Guille we tried to avoid usage of such external library and wrote an
> FFI interface to all the required LibC functions. We were successful but we
> realized that there are several issues that are limiting us.
>
> When you want to execute a separate process for the program that you want to
> open in terminal (typically the Bash), you need to redirect the standard IO
> files, create a fork of your process, do some additional initialization in
> it and call 'exec' on it. In the parent process, you change redirected IO
> files back to the original values.
>
> But the problem is that between the FFI calls from Smalltalk the VM can do a
> lot of things including garbage collection etc. On OS X the fork() function
> has the following limitation described in man:
>
> "There are limits to what you can do in the child process. To be totally
> safe you should restrict your yourself to only executing async-signal safe
> operations until such time as one of the exec functions is called. All
> APIs, including global data symbols, in any framework or library should be
> assumed to be unsafe after a fork() unless explicitly documented to be safe
> or async-signal safe. If you need to use these frameworks in the child
> process, you must exec. In this
> situation it is reasonable to exec your-self. yourself."
>
>
> As the result in most cases (but not all) the fork() and exec() pair from
> the Smalltalk side fails on OS X. Linux does not have this limitation
> however even there we found an issue. It is bound to the fact that fork()
> makes a fork of all the parent process that uses the same resources. As soon
> as Pharo is opened in a window and X11 is involved (the window wants to be
> repainted), it can lead to the VM crash.
>
> So we learned that unfortunately we currently cannot use image-only FFI code
> for this task. We need a C library or VM plugin.
>
> The repository of the terminal emulator is here:
> https://github.com/pavel-krivanek/terminal
>
> and can be loaded using the following code:
>
> Metacello new
> baseline: 'TerminalEmulator';
> repository: 'github://pavel-krivanek/terminal/src';
> load.
>
> #TerminalEmulator asClass compileLibrary.
>
>
> It compiles and links the small library with only one function using the GCC
> so the machine needs to have a proper development environment.
>
> The terminal emulator is in very early stage and has a lot of issues like
> processes cleanup, drawing, keyboard input etc. etc. If you are interested
> in it, feel free to contribute.
>
> Cheers,
> -- Pavel
>
>
>
>
>
>
>

Pavel Krivanek-3

Re: Pavel's ChangeLog week of 2018-02-26

2018-03-09 16:02 GMT+01:00 Alistair Grant <[hidden email]>:
> Hi Pavel,
>
> I tried to get this working on Ubuntu 16.04 and found the following:
>
> - The compilation failed unless I added the '-fPIC' flag (as suggested by gcc).

is it on 64-bit system and VM?

> - The example code in the TerminalEmulator class comment references
> ProcessEndpoint, but the class doesn't exist.

Thanks, this class was renamed

Cheers,
-- Pavel

>
> Pharo 7.0
> Build information:
> Pharo-7.0+alpha.build.674.sha.57447e756fa6fcf1877f9d0d1cda235b3b63c807
> (64 Bit)
>
>
> Cheers,
> Alistair
>
>
>
>
>
>
> On 5 March 2018 at 10:29, Pavel Krivanek <[hidden email]> wrote:
>> Hi,
>>
>> Guille is working on the Iceberg improvements and he wanted to be able to
>> open a terminal window on top of the repository and interact with it via the
>> command line. So we looked at this issue because I already in December made
>> some experiments with the terminal emulation in Pharo.
>>
>> In past, the Squeak had a working terminal emulation that used
>> PseudoTTYPlugin. The VM is not built with this code for a long time but I
>> tried to replace it with a small C library and then wrote an FFI interface
>> to it. Together with that, I ported most of the old code Squeak code to
>> Pharo.
>>
>> With Guille we tried to avoid usage of such external library and wrote an
>> FFI interface to all the required LibC functions. We were successful but we
>> realized that there are several issues that are limiting us.
>>
>> When you want to execute a separate process for the program that you want to
>> open in terminal (typically the Bash), you need to redirect the standard IO
>> files, create a fork of your process, do some additional initialization in
>> it and call 'exec' on it. In the parent process, you change redirected IO
>> files back to the original values.
>>
>> But the problem is that between the FFI calls from Smalltalk the VM can do a
>> lot of things including garbage collection etc. On OS X the fork() function
>> has the following limitation described in man:
>>
>> "There are limits to what you can do in the child process. To be totally
>> safe you should restrict your yourself to only executing async-signal safe
>> operations until such time as one of the exec functions is called. All
>> APIs, including global data symbols, in any framework or library should be
>> assumed to be unsafe after a fork() unless explicitly documented to be safe
>> or async-signal safe. If you need to use these frameworks in the child
>> process, you must exec. In this
>> situation it is reasonable to exec your-self. yourself."
>>
>>
>> As the result in most cases (but not all) the fork() and exec() pair from
>> the Smalltalk side fails on OS X. Linux does not have this limitation
>> however even there we found an issue. It is bound to the fact that fork()
>> makes a fork of all the parent process that uses the same resources. As soon
>> as Pharo is opened in a window and X11 is involved (the window wants to be
>> repainted), it can lead to the VM crash.
>>
>> So we learned that unfortunately we currently cannot use image-only FFI code
>> for this task. We need a C library or VM plugin.
>>
>> The repository of the terminal emulator is here:
>> https://github.com/pavel-krivanek/terminal
>>
>> and can be loaded using the following code:
>>
>> Metacello new
>> baseline: 'TerminalEmulator';
>> repository: 'github://pavel-krivanek/terminal/src';
>> load.
>>
>> #TerminalEmulator asClass compileLibrary.
>>
>>
>> It compiles and links the small library with only one function using the GCC
>> so the machine needs to have a proper development environment.
>>
>> The terminal emulator is in very early stage and has a lot of issues like
>> processes cleanup, drawing, keyboard input etc. etc. If you are interested
>> in it, feel free to contribute.
>>
>> Cheers,
>> -- Pavel
>>
>>
>>
>>
>>
>>
>>
>

alistairgrant

Re: Pavel's ChangeLog week of 2018-02-26

Hi Pavel,

On 9 March 2018 at 16:12, Pavel Krivanek <[hidden email]> wrote:
> 2018-03-09 16:02 GMT+01:00 Alistair Grant <[hidden email]>:
>> Hi Pavel,
>>
>> I tried to get this working on Ubuntu 16.04 and found the following:
>>
>> - The compilation failed unless I added the '-fPIC' flag (as suggested by gcc).
>
> is it on 64-bit system and VM?

Yep.

>> - The example code in the TerminalEmulator class comment references
>> ProcessEndpoint, but the class doesn't exist.
>
> Thanks, this class was renamed

To PseudoTTYEndpoint, it looks like.

Cool! The interactive terminal is up and running.

Is it possible to have a cursor?

How does one pass more than one argument, for example, if I want to
execute "ls -l -h", what should be the format be (ignore that the two
arguments can be combined in this case, it's just as a simple
example)?

PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l -h')

produces:

ls: invalid option -- ' '
Try 'ls --help' for more information.

PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l' '-h')

ignores the second argument.

Thanks!
Alistair

> Cheers,
> -- Pavel
>
>>
>> Pharo 7.0
>> Build information:
>> Pharo-7.0+alpha.build.674.sha.57447e756fa6fcf1877f9d0d1cda235b3b63c807
>> (64 Bit)
>>
>>
>> Cheers,
>> Alistair
>>
>>
>>
>>
>>
>>
>> On 5 March 2018 at 10:29, Pavel Krivanek <[hidden email]> wrote:
>>> Hi,
>>>
>>> Guille is working on the Iceberg improvements and he wanted to be able to
>>> open a terminal window on top of the repository and interact with it via the
>>> command line. So we looked at this issue because I already in December made
>>> some experiments with the terminal emulation in Pharo.
>>>
>>> In past, the Squeak had a working terminal emulation that used
>>> PseudoTTYPlugin. The VM is not built with this code for a long time but I
>>> tried to replace it with a small C library and then wrote an FFI interface
>>> to it. Together with that, I ported most of the old code Squeak code to
>>> Pharo.
>>>
>>> With Guille we tried to avoid usage of such external library and wrote an
>>> FFI interface to all the required LibC functions. We were successful but we
>>> realized that there are several issues that are limiting us.
>>>
>>> When you want to execute a separate process for the program that you want to
>>> open in terminal (typically the Bash), you need to redirect the standard IO
>>> files, create a fork of your process, do some additional initialization in
>>> it and call 'exec' on it. In the parent process, you change redirected IO
>>> files back to the original values.
>>>
>>> But the problem is that between the FFI calls from Smalltalk the VM can do a
>>> lot of things including garbage collection etc. On OS X the fork() function
>>> has the following limitation described in man:
>>>
>>> "There are limits to what you can do in the child process. To be totally
>>> safe you should restrict your yourself to only executing async-signal safe
>>> operations until such time as one of the exec functions is called. All
>>> APIs, including global data symbols, in any framework or library should be
>>> assumed to be unsafe after a fork() unless explicitly documented to be safe
>>> or async-signal safe. If you need to use these frameworks in the child
>>> process, you must exec. In this
>>> situation it is reasonable to exec your-self. yourself."
>>>
>>>
>>> As the result in most cases (but not all) the fork() and exec() pair from
>>> the Smalltalk side fails on OS X. Linux does not have this limitation
>>> however even there we found an issue. It is bound to the fact that fork()
>>> makes a fork of all the parent process that uses the same resources. As soon
>>> as Pharo is opened in a window and X11 is involved (the window wants to be
>>> repainted), it can lead to the VM crash.
>>>
>>> So we learned that unfortunately we currently cannot use image-only FFI code
>>> for this task. We need a C library or VM plugin.
>>>
>>> The repository of the terminal emulator is here:
>>> https://github.com/pavel-krivanek/terminal
>>>
>>> and can be loaded using the following code:
>>>
>>> Metacello new
>>> baseline: 'TerminalEmulator';
>>> repository: 'github://pavel-krivanek/terminal/src';
>>> load.
>>>
>>> #TerminalEmulator asClass compileLibrary.
>>>
>>>
>>> It compiles and links the small library with only one function using the GCC
>>> so the machine needs to have a proper development environment.
>>>
>>> The terminal emulator is in very early stage and has a lot of issues like
>>> processes cleanup, drawing, keyboard input etc. etc. If you are interested
>>> in it, feel free to contribute.
>>>
>>> Cheers,
>>> -- Pavel
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>

Pavel Krivanek-3

Re: Pavel's ChangeLog week of 2018-02-26

2018-03-09 16:47 GMT+01:00 Alistair Grant <[hidden email]>:

> Hi Pavel,
>
> On 9 March 2018 at 16:12, Pavel Krivanek <[hidden email]> wrote:
>> 2018-03-09 16:02 GMT+01:00 Alistair Grant <[hidden email]>:
>>> Hi Pavel,
>>>
>>> I tried to get this working on Ubuntu 16.04 and found the following:
>>>
>>> - The compilation failed unless I added the '-fPIC' flag (as suggested by gcc).
>>
>> is it on 64-bit system and VM?
>
> Yep.

I will check that, I guess some package is missing in this Ubuntu
environment because I think I checked it on the same system without
this issue.

>
>
>>> - The example code in the TerminalEmulator class comment references
>>> ProcessEndpoint, but the class doesn't exist.
>>
>> Thanks, this class was renamed
>
> To PseudoTTYEndpoint, it looks like.
>
> Cool! The interactive terminal is up and running.
>
> Is it possible to have a cursor?

the original implementation had it, I need to only fix it ;-)

>
> How does one pass more than one argument, for example, if I want to
> execute "ls -l -h", what should be the format be (ignore that the two
> arguments can be combined in this case, it's just as a simple
> example)?
>
> PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l -h')
>
> produces:
>
> ls: invalid option -- ' '
> Try 'ls --help' for more information.
>
>
> PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l' '-h')
>
> ignores the second argument.

In this prototype implementation, only one argument is allowed because
to pass variable arguments list with FFI is not trivial. It will be
fixed soon.

>
> Thanks!
> Alistair
>
>
>> Cheers,
>> -- Pavel
>>
>>>
>>> Pharo 7.0
>>> Build information:
>>> Pharo-7.0+alpha.build.674.sha.57447e756fa6fcf1877f9d0d1cda235b3b63c807
>>> (64 Bit)
>>>
>>>
>>> Cheers,
>>> Alistair
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 5 March 2018 at 10:29, Pavel Krivanek <[hidden email]> wrote:
>>>> Hi,
>>>>
>>>> Guille is working on the Iceberg improvements and he wanted to be able to
>>>> open a terminal window on top of the repository and interact with it via the
>>>> command line. So we looked at this issue because I already in December made
>>>> some experiments with the terminal emulation in Pharo.
>>>>
>>>> In past, the Squeak had a working terminal emulation that used
>>>> PseudoTTYPlugin. The VM is not built with this code for a long time but I
>>>> tried to replace it with a small C library and then wrote an FFI interface
>>>> to it. Together with that, I ported most of the old code Squeak code to
>>>> Pharo.
>>>>
>>>> With Guille we tried to avoid usage of such external library and wrote an
>>>> FFI interface to all the required LibC functions. We were successful but we
>>>> realized that there are several issues that are limiting us.
>>>>
>>>> When you want to execute a separate process for the program that you want to
>>>> open in terminal (typically the Bash), you need to redirect the standard IO
>>>> files, create a fork of your process, do some additional initialization in
>>>> it and call 'exec' on it. In the parent process, you change redirected IO
>>>> files back to the original values.
>>>>
>>>> But the problem is that between the FFI calls from Smalltalk the VM can do a
>>>> lot of things including garbage collection etc. On OS X the fork() function
>>>> has the following limitation described in man:
>>>>
>>>> "There are limits to what you can do in the child process. To be totally
>>>> safe you should restrict your yourself to only executing async-signal safe
>>>> operations until such time as one of the exec functions is called. All
>>>> APIs, including global data symbols, in any framework or library should be
>>>> assumed to be unsafe after a fork() unless explicitly documented to be safe
>>>> or async-signal safe. If you need to use these frameworks in the child
>>>> process, you must exec. In this
>>>> situation it is reasonable to exec your-self. yourself."
>>>>
>>>>
>>>> As the result in most cases (but not all) the fork() and exec() pair from
>>>> the Smalltalk side fails on OS X. Linux does not have this limitation
>>>> however even there we found an issue. It is bound to the fact that fork()
>>>> makes a fork of all the parent process that uses the same resources. As soon
>>>> as Pharo is opened in a window and X11 is involved (the window wants to be
>>>> repainted), it can lead to the VM crash.
>>>>
>>>> So we learned that unfortunately we currently cannot use image-only FFI code
>>>> for this task. We need a C library or VM plugin.
>>>>
>>>> The repository of the terminal emulator is here:
>>>> https://github.com/pavel-krivanek/terminal
>>>>
>>>> and can be loaded using the following code:
>>>>
>>>> Metacello new
>>>> baseline: 'TerminalEmulator';
>>>> repository: 'github://pavel-krivanek/terminal/src';
>>>> load.
>>>>
>>>> #TerminalEmulator asClass compileLibrary.
>>>>
>>>>
>>>> It compiles and links the small library with only one function using the GCC
>>>> so the machine needs to have a proper development environment.
>>>>
>>>> The terminal emulator is in very early stage and has a lot of issues like
>>>> processes cleanup, drawing, keyboard input etc. etc. If you are interested
>>>> in it, feel free to contribute.
>>>>
>>>> Cheers,
>>>> -- Pavel
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>

alistairgrant

Re: Pavel's ChangeLog week of 2018-02-26

On 9 March 2018 at 16:56, Pavel Krivanek <[hidden email]> wrote:

> 2018-03-09 16:47 GMT+01:00 Alistair Grant <[hidden email]>:
>> Hi Pavel,
>>
>> On 9 March 2018 at 16:12, Pavel Krivanek <[hidden email]> wrote:
>>> 2018-03-09 16:02 GMT+01:00 Alistair Grant <[hidden email]>:
>>>> Hi Pavel,
>>>>
>>>> I tried to get this working on Ubuntu 16.04 and found the following:
>>>>
>>>> - The compilation failed unless I added the '-fPIC' flag (as suggested by gcc).
>>>
>>> is it on 64-bit system and VM?
>>
>> Yep.
>
> I will check that, I guess some package is missing in this Ubuntu
> environment because I think I checked it on the same system without
> this issue.
>
>>
>>
>>>> - The example code in the TerminalEmulator class comment references
>>>> ProcessEndpoint, but the class doesn't exist.
>>>
>>> Thanks, this class was renamed
>>
>> To PseudoTTYEndpoint, it looks like.
>>
>> Cool! The interactive terminal is up and running.
>>
>> Is it possible to have a cursor?
>
> the original implementation had it, I need to only fix it ;-)
>>
>> How does one pass more than one argument, for example, if I want to
>> execute "ls -l -h", what should be the format be (ignore that the two
>> arguments can be combined in this case, it's just as a simple
>> example)?
>>
>> PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l -h')
>>
>> produces:
>>
>> ls: invalid option -- ' '
>> Try 'ls --help' for more information.
>>
>>
>> PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l' '-h')
>>
>> ignores the second argument.
>
> In this prototype implementation, only one argument is allowed because
> to pass variable arguments list with FFI is not trivial. It will be
> fixed soon.

Great, thanks!

Cheers,
Alistair

Ben Coman

Re: Pavel's ChangeLog week of 2018-02-26

In reply to this post by Pavel Krivanek-3

On 9 March 2018 at 23:56, Pavel Krivanek <[hidden email]> wrote:

2018-03-09 16:47 GMT+01:00 Alistair Grant <[hidden email]>:
> Hi Pavel,
>
> On 9 March 2018 at 16:12, Pavel Krivanek <[hidden email]> wrote:
>> 2018-03-09 16:02 GMT+01:00 Alistair Grant <[hidden email]>:
>>> Hi Pavel,
>>>
>>> I tried to get this working on Ubuntu 16.04 and found the following:
>>>
>>> - The compilation failed unless I added the '-fPIC' flag (as suggested by gcc).
>>
>> is it on 64-bit system and VM?
>
> Yep.

I will check that, I guess some package is missing in this Ubuntu
environment because I think I checked it on the same system without
this issue.

I get the same...

$ gcc -c -m64 -o pseudoTTYLib.c.o /home/ben/Pharo/images/Pharo-7.0.0-alpha.build.674.sha.57447e7.arch.64bit/pharo-local/iceberg/pavel-krivanek/terminal/lib/pseudoTTYLib.c

$ gcc -shared -m64 -o pseudoTTYLib.so pseudoTTYLib.c.o -lutil

/usr/bin/ld: pseudoTTYLib.c.o: relocation R_X86_64_PC32 against undefined symbol `fork@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC

/usr/bin/ld: final link failed: Bad value

collect2: error: ld returned 1 exit status

and it works with -fPIC

$ gcc -fPIC -c -m64 -o pseudoTTYLib.c.o /home/ben/Pharo/images/Pharo-7.0.0-alpha.build.674.sha.57447e7.arch.64bit/pharo-local/iceberg/pavel-krivanek/terminal/lib/pseudoTTYLib.c

$ gcc -shared -m64 -o pseudoTTYLib.so pseudoTTYLib.c.o -lutil

Interestingly, 32-bit works okay without -fPIC...

$ gcc -c -m32 -o pseudoTTYLib.c.o /home/ben/Pharo/images/Pharo-7.0.0-alpha.build.674.sha.57447e7.arch.64bit/pharo-local/iceberg/pavel-krivanek/terminal/lib/pseudoTTYLib.c

$ gcc -shared -m32 -o pseudoTTYLib.so pseudoTTYLib.c.o -lutil

The reason being... "Not only does gcc encourage you to use PIC for shared libraries on x64, it requires it by default."

https://eli.thegreenplace.net/2011/11/11/position-independent-code-pic-in-shared-libraries-on-x64/

>>> - The example code in the TerminalEmulator class comment references
>>> ProcessEndpoint, but the class doesn't exist.
>>
>> Thanks, this class was renamed
>
> To PseudoTTYEndpoint, it looks like.

It took me a few minutes to catch up, so for others, the following works in Pharo7 build674...

win := TerminalEmulator open.

(PseudoTTYEndpoint command: '/bin/bash' arguments: {'-i'}) asProtocolStack

push: TerminalEmulatorXterm new;

push: win tty;

install;

run

> Cool! The interactive terminal is up and running.

Ohhh, that is very cool!!!

Would something l like this be possible later? (It would make a good promotion video)

1. Double-click an Ubuntu desktop icon that opens only this terminal window

2. From that terminal use some special syntax to issue commands to the Image running it,

e.g. to open the IDE.

3. Make some modifications to the terminal code (like how colours are displayed)

while the terminal is running.

cheers -ben

>
> Is it possible to have a cursor?

the original implementation had it, I need to only fix it ;-)
>
> How does one pass more than one argument, for example, if I want to
> execute "ls -l -h", what should be the format be (ignore that the two
> arguments can be combined in this case, it's just as a simple
> example)?
>
> PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l -h')
>
> produces:
>
> ls: invalid option -- ' '
> Try 'ls --help' for more information.
>
>
> PseudoTTYEndpoint command: '/bin/ls' arguments: #('-l' '-h')
>
> ignores the second argument.

In this prototype implementation, only one argument is allowed because
to pass variable arguments list with FFI is not trivial. It will be
fixed soon.

>
> Thanks!
> Alistair
>
>
>> Cheers,
>> -- Pavel
>>
>>>
>>> Pharo 7.0
>>> Build information:
>>> Pharo-7.0+alpha.build.674.sha.57447e756fa6fcf1877f9d0d1cda235b3b63c807
>>> (64 Bit)
>>>
>>>
>>> Cheers,
>>> Alistair
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 5 March 2018 at 10:29, Pavel Krivanek <[hidden email]> wrote:
>>>> Hi,
>>>>
>>>> Guille is working on the Iceberg improvements and he wanted to be able to
>>>> open a terminal window on top of the repository and interact with it via the
>>>> command line. So we looked at this issue because I already in December made
>>>> some experiments with the terminal emulation in Pharo.
>>>>
>>>> In past, the Squeak had a working terminal emulation that used
>>>> PseudoTTYPlugin. The VM is not built with this code for a long time but I
>>>> tried to replace it with a small C library and then wrote an FFI interface
>>>> to it. Together with that, I ported most of the old code Squeak code to
>>>> Pharo.
>>>>
>>>> With Guille we tried to avoid usage of such external library and wrote an
>>>> FFI interface to all the required LibC functions. We were successful but we
>>>> realized that there are several issues that are limiting us.
>>>>
>>>> When you want to execute a separate process for the program that you want to
>>>> open in terminal (typically the Bash), you need to redirect the standard IO
>>>> files, create a fork of your process, do some additional initialization in
>>>> it and call 'exec' on it. In the parent process, you change redirected IO
>>>> files back to the original values.
>>>>
>>>> But the problem is that between the FFI calls from Smalltalk the VM can do a
>>>> lot of things including garbage collection etc. On OS X the fork() function
>>>> has the following limitation described in man:
>>>>
>>>> "There are limits to what you can do in the child process. To be totally
>>>> safe you should restrict your yourself to only executing async-signal safe
>>>> operations until such time as one of the exec functions is called. All
>>>> APIs, including global data symbols, in any framework or library should be
>>>> assumed to be unsafe after a fork() unless explicitly documented to be safe
>>>> or async-signal safe. If you need to use these frameworks in the child
>>>> process, you must exec. In this
>>>> situation it is reasonable to exec your-self. yourself."
>>>>
>>>>
>>>> As the result in most cases (but not all) the fork() and exec() pair from
>>>> the Smalltalk side fails on OS X. Linux does not have this limitation
>>>> however even there we found an issue. It is bound to the fact that fork()
>>>> makes a fork of all the parent process that uses the same resources. As soon
>>>> as Pharo is opened in a window and X11 is involved (the window wants to be
>>>> repainted), it can lead to the VM crash.
>>>>
>>>> So we learned that unfortunately we currently cannot use image-only FFI code
>>>> for this task. We need a C library or VM plugin.
>>>>
>>>> The repository of the terminal emulator is here:
>>>> https://github.com/pavel-krivanek/terminal
>>>>
>>>> and can be loaded using the following code:
>>>>
>>>> Metacello new
>>>> baseline: 'TerminalEmulator';
>>>> repository: 'github://pavel-krivanek/terminal/src';
>>>> load.
>>>>
>>>> #TerminalEmulator asClass compileLibrary.
>>>>
>>>>
>>>> It compiles and links the small library with only one function using the GCC
>>>> so the machine needs to have a proper development environment.
>>>>
>>>> The terminal emulator is in very early stage and has a lot of issues like
>>>> processes cleanup, drawing, keyboard input etc. etc. If you are interested
>>>> in it, feel free to contribute.
>>>>
>>>> Cheers,
>>>> -- Pavel
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>
>