Proof of Concept of a pharo cog spur VM to OpenBSD

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Proof of Concept of a pharo cog spur VM to OpenBSD

Christian Kellermann
 
Dear List,

I have managed to successfully build a pharo cog spur VM on the
OpenBSD operating system (http://openbsd.org).  I have put up some
patches on a fork of the opensmalltalk-vm github repo:
https://github.com/ckeen/opensmalltalk-vm/tree/OpenBSD_PoC

I habe briefly started current Squeak with it also, but did not
build a specific VM for it.

I would consider this a proof of concept at the moment and would
like to discuss how to get this into the main repo.  OpenBSD needs
a couple of changes, all of them pretty minor. Mostly this is due
to different location of include files (errno) or unsupported linker
flags (-lrt) or not included by default search paths (/usr/X11R6/{include,
lib), /usr/local/{include,lib})/

However just hacking them into the mvm script for one machine does
not seem right, are these files generated by something else?

I have also disabled the building of third party components. I am
currently relying on them being installed already by the OS package
manager.  If this gets integrated in a real OpenBSD package these
dependencies would be made explicit in the package's meta information.

Also how do you test these VMs? I am a SmallTalk beginner so I am
not yet familiar with the available tooling that's out there.

To reproduce this build on OpenBSD, make sure you compile/install
the resulting binaries into a partition that is mounted with the
OpenBSD specific wxallowed flag (usually /usr/local). Otherwise the
JIT will just cause a segmentation fault as mapping memory writable
and executable at the same time is prohibited by default.

My next planned steps would be:

- Get the needed patches into your repo
- Build all VMs under OpenBSD, once I know how to test those properly
- Propose an update to the outdated OpenBSD squeak package
- Propose a port of Pharo to be included in the OpenBSD ports tree
- Write a plugin for OpenBSD's sndio sound system (Pulse seems to
  work but that is usually not installed)

Looking forward to hearing your opinion.

Kind regards,

Christian

P.S.: There is also still some issue with the FFI binding a ByteArray
from libc (i.e. the 'environ' Variable). This fails for two reasons:
LibC is looking for the wrong libc module name ('libc.so.6' instead
of 'libc.so' or more specific to OpenBSD 'libc.so.<major>.<minor>'
for example 'libc.so.90.0'). And then I get an External Allocation
error. I am not sure whether this is a bug in the VM handling 64bits
or in Pharo. I have not yet tested this behaviour with Squeak yet.

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.
Reply | Threaded
Open this post in threaded view
|

Re: Proof of Concept of a pharo cog spur VM to OpenBSD

Stuart Cassoff-2
 

Hi,

Maybe have a look at
https://github.com/aryler/ostoobsd

Stu

> ---------- Original Message ----------
> From: Christian Kellermann <[hidden email]>
> Date: November 1, 2017 at 4:46 AM
>
>
>  
> Dear List,
>
> I have managed to successfully build a pharo cog spur VM on the
> OpenBSD operating system (http://openbsd.org).  I have put up some
> patches on a fork of the opensmalltalk-vm github repo:
> https://github.com/ckeen/opensmalltalk-vm/tree/OpenBSD_PoC
>
> I habe briefly started current Squeak with it also, but did not
> build a specific VM for it.
>
> I would consider this a proof of concept at the moment and would
> like to discuss how to get this into the main repo.  OpenBSD needs
> a couple of changes, all of them pretty minor. Mostly this is due
> to different location of include files (errno) or unsupported linker
> flags (-lrt) or not included by default search paths (/usr/X11R6/{include,
> lib), /usr/local/{include,lib})/
>
> However just hacking them into the mvm script for one machine does
> not seem right, are these files generated by something else?
>
> I have also disabled the building of third party components. I am
> currently relying on them being installed already by the OS package
> manager.  If this gets integrated in a real OpenBSD package these
> dependencies would be made explicit in the package's meta information.
>
> Also how do you test these VMs? I am a SmallTalk beginner so I am
> not yet familiar with the available tooling that's out there.
>
> To reproduce this build on OpenBSD, make sure you compile/install
> the resulting binaries into a partition that is mounted with the
> OpenBSD specific wxallowed flag (usually /usr/local). Otherwise the
> JIT will just cause a segmentation fault as mapping memory writable
> and executable at the same time is prohibited by default.
>
> My next planned steps would be:
>
> - Get the needed patches into your repo
> - Build all VMs under OpenBSD, once I know how to test those properly
> - Propose an update to the outdated OpenBSD squeak package
> - Propose a port of Pharo to be included in the OpenBSD ports tree
> - Write a plugin for OpenBSD's sndio sound system (Pulse seems to
>   work but that is usually not installed)
>
> Looking forward to hearing your opinion.
>
> Kind regards,
>
> Christian
>
> P.S.: There is also still some issue with the FFI binding a ByteArray
> from libc (i.e. the 'environ' Variable). This fails for two reasons:
> LibC is looking for the wrong libc module name ('libc.so.6' instead
> of 'libc.so' or more specific to OpenBSD 'libc.so.<major>.<minor>'
> for example 'libc.so.90.0'). And then I get an External Allocation
> error. I am not sure whether this is a bug in the VM handling 64bits
> or in Pharo. I have not yet tested this behaviour with Squeak yet.
>
> --
> May you be peaceful, may you live in safety, may you be free from
> suffering, and may you live with ease.
Reply | Threaded
Open this post in threaded view
|

Re: Proof of Concept of a pharo cog spur VM to OpenBSD

Nicolas Cellier
In reply to this post by Christian Kellermann
 
Hi Christian,
Thanks!
Your work sounds almost ready for inclusion. We could easily cherrypick the patches to base files, just emit PR with those.
I see that you have committed some thirdparty sources in https://github.com/ckeen/opensmalltalk-vm/commit/227f059f587a35970ebc3072fd8dca45c4887de5
My understanding is that they are either fetched during the build process, or the libraries are replaced by those distributed with OS as you further decided.
I would thus remove them.

Concerning the right place for putting os specific options, I don't know.

For automating the build, you would have to hack https://github.com/ckeen/opensmalltalk-vm/blob/OpenBSD_PoC/.travis.yml and add openbsd to the matrix
and maybe other hacks in the others .travis*.sh

and also https://github.com/ckeen/opensmalltalk-vm/blob/OpenBSD_PoC/tests/smalltalkCI.sh
The principle is to run an image that will launch the SUnit tests as prescribed by specification in https://github.com/ckeen/opensmalltalk-vm/blob/OpenBSD_PoC/tests/smalltalk.ston
The source code enabling this is at https://github.com/hpi-swa/smalltalkCI

Nicolas


2017-11-01 9:46 GMT+01:00 Christian Kellermann <[hidden email]>:

Dear List,

I have managed to successfully build a pharo cog spur VM on the
OpenBSD operating system (http://openbsd.org).  I have put up some
patches on a fork of the opensmalltalk-vm github repo:
https://github.com/ckeen/opensmalltalk-vm/tree/OpenBSD_PoC

I habe briefly started current Squeak with it also, but did not
build a specific VM for it.

I would consider this a proof of concept at the moment and would
like to discuss how to get this into the main repo.  OpenBSD needs
a couple of changes, all of them pretty minor. Mostly this is due
to different location of include files (errno) or unsupported linker
flags (-lrt) or not included by default search paths (/usr/X11R6/{include,
lib), /usr/local/{include,lib})/

However just hacking them into the mvm script for one machine does
not seem right, are these files generated by something else?

I have also disabled the building of third party components. I am
currently relying on them being installed already by the OS package
manager.  If this gets integrated in a real OpenBSD package these
dependencies would be made explicit in the package's meta information.

Also how do you test these VMs? I am a SmallTalk beginner so I am
not yet familiar with the available tooling that's out there.

To reproduce this build on OpenBSD, make sure you compile/install
the resulting binaries into a partition that is mounted with the
OpenBSD specific wxallowed flag (usually /usr/local). Otherwise the
JIT will just cause a segmentation fault as mapping memory writable
and executable at the same time is prohibited by default.

My next planned steps would be:

- Get the needed patches into your repo
- Build all VMs under OpenBSD, once I know how to test those properly
- Propose an update to the outdated OpenBSD squeak package
- Propose a port of Pharo to be included in the OpenBSD ports tree
- Write a plugin for OpenBSD's sndio sound system (Pulse seems to
  work but that is usually not installed)

Looking forward to hearing your opinion.

Kind regards,

Christian

P.S.: There is also still some issue with the FFI binding a ByteArray
from libc (i.e. the 'environ' Variable). This fails for two reasons:
LibC is looking for the wrong libc module name ('libc.so.6' instead
of 'libc.so' or more specific to OpenBSD 'libc.so.<major>.<minor>'
for example 'libc.so.90.0'). And then I get an External Allocation
error. I am not sure whether this is a bug in the VM handling 64bits
or in Pharo. I have not yet tested this behaviour with Squeak yet.

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.

Reply | Threaded
Open this post in threaded view
|

Re: Proof of Concept of a pharo cog spur VM to OpenBSD

Christian Kellermann
 
Hi Nicolas,

* Nicolas Cellier <[hidden email]> [171102 23:50]:
> Your work sounds almost ready for inclusion. We could easily cherrypick the
> patches to base files, just emit PR with those.

I will submit these today.

> I see that you have committed some thirdparty sources in
> https://github.com/ckeen/opensmalltalk-vm/commit/227f059f587a35970ebc3072fd8dca45c4887de5
> My understanding is that they are either fetched during the build process,
> or the libraries are replaced by those distributed with OS as you further
> decided.
> I would thus remove them.
>

Agreed, this has been an oversight on my side.

> Concerning the right place for putting os specific options, I don't know.
 
The commits that modify the plugins will break other platforms, which in this case means Linux.
I'd rather refactor the Makefiles in a way that they will include
those linker flags from the LDFLAGS given in the respective mvm
file.  So one can add or omit them as needed.


> For automating the build, you would have to hack
> https://github.com/ckeen/opensmalltalk-vm/blob/OpenBSD_PoC/.travis.yml and

I don't know anything about travis, does it even run an OpenBSD
platform?  Or are you hinting at something else?  I can run the
SUnit tests myself for now if that's ok. I might need more guidance
in this though.

Kind regards,

Christian

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.
Reply | Threaded
Open this post in threaded view
|

Re: Proof of Concept of a pharo cog spur VM to OpenBSD

Christian Kellermann
In reply to this post by Stuart Cassoff-2
 
* Stuart Cassoff <[hidden email]> [171102 18:45]:
>  
>
> Hi,
>
> Maybe have a look at
> https://github.com/aryler/ostoobsd

Thanks Stu, I will take the i386 patches from there. However these
ports do nothing with the external dependencies it seems. So idk
how useful the resulting VMs really are...

Kind regards,

Christian

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.
Reply | Threaded
Open this post in threaded view
|

Re: Proof of Concept of a pharo cog spur VM to OpenBSD

Christian Kellermann
 
* Christian Kellermann <[hidden email]> [171107 11:15]:
> Thanks Stu, I will take the i386 patches from there. However these
> ports do nothing with the external dependencies it seems. So idk
> how useful the resulting VMs really are...

This came out probably sounding more snarky than I meant it to be.
Please accept my apologies.  Getting this to even compile has been
a lot of work and I am grateful for the hint towards the prior work.
But why not upstream the patches and make OpenBSD a better working
platform for smalltalk?

So thanks again, Stu!

 
Kind regards,
 
Christian

--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.
Reply | Threaded
Open this post in threaded view
|

Re: Proof of Concept of a pharo cog spur VM to OpenBSD

Nicolas Cellier
In reply to this post by Christian Kellermann
 


2017-11-07 11:10 GMT+01:00 Christian Kellermann <[hidden email]>:

Hi Nicolas,

* Nicolas Cellier <[hidden email]> [171102 23:50]:
> Your work sounds almost ready for inclusion. We could easily cherrypick the
> patches to base files, just emit PR with those.

I will submit these today.

> I see that you have committed some thirdparty sources in
> https://github.com/ckeen/opensmalltalk-vm/commit/227f059f587a35970ebc3072fd8dca45c4887de5
> My understanding is that they are either fetched during the build process,
> or the libraries are replaced by those distributed with OS as you further
> decided.
> I would thus remove them.
>

Agreed, this has been an oversight on my side.

> Concerning the right place for putting os specific options, I don't know.

The commits that modify the plugins will break other platforms, which in this case means Linux.
I'd rather refactor the Makefiles in a way that they will include
those linker flags from the LDFLAGS given in the respective mvm
file.  So one can add or omit them as needed.


> For automating the build, you would have to hack
> https://github.com/ckeen/opensmalltalk-vm/blob/OpenBSD_PoC/.travis.yml and

I don't know anything about travis, does it even run an OpenBSD
platform?  Or are you hinting at something else?  I can run the
SUnit tests myself for now if that's ok. I might need more guidance
in this though.

Kind regards,

Christian



Hmm my mistake, maybe I saw it on gitlab...
I would have thought this was integrated

but it seems that there is only support for linux and osx

So you would have to execute the shell scripts by yourself...


--
May you be peaceful, may you live in safety, may you be free from
suffering, and may you live with ease.