Cog Rump Xen unikernel

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Cog Rump Xen unikernel

Ben Coman
 
I got curious about following up these two threads making Cog a unikernel...

Cog in the cloud
http://forum.world.st/Cog-in-the-cloud-td4796514.html

Hosting the Pharo VM on MirageOS
http://forum.world.st/Hosting-the-Pharo-VM-on-MirageOS-td4799424.html#none


One path forward is to use rumpkernel, which "provides a means to run
existing POSIX applications as unikernels on Xen"
https://blog.xenproject.org/2015/08/06/on-rump-kernels-and-the-rumprun-unikernel/

Deploying real-world software today as unikernels on Xen with Rumprun
http://events.linuxfoundation.org/sites/events/files/slides/xdps15-talk-final_0.pdf


So I thought I would try the "Building Rumprun Unikernels" tutorial here...
https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels
which I report here in case it was interest.  The tutorial is based on
QEMU (and I think I saw a comment somewhere that Cog doesn't run so
well on QEMU and needs Bochs), so after that I adapted it from QEMU to
Xen.


64-BIT DEBIAN 8 JESSIE & XEN

So first I installed Debian 8 Jessie 64-bit on my new laptop
$ uname -a
>> ...x86_64

Then installed Xen and after a reboot could choose it from the boot menu.
$ apt-get update
$ apt get install xen-system-amd64 libxen-dev

REBOOTED

Tested manual selection from menu to boot into Xen dom0.
Checked Xen was operating with...
$ xl info


32-BIT DEBIAN 8 JESSIE & XEN

However later I hit some complications determining whether the rump
unikernel was being compiled 32-bit or 64-bit in order to mix in with
32-bit Pharo, so to simplify things I did a complete reinstall of my
system as 32-bit.
$ uname -a
>> ...686-pae

Then installed Xen again  Xen only has a 64-bit hypervisor, but this
was not immediately available under 32-bit OS.  I can't exactly
remember but I think I just needed to add multiarch before it would
find the Xen package.
$ apt-get update
$ apt-get install multiarch
$ apt get install xen-system-amd64 libxen-dev
Otherwise it may have needed something like "dpkg --add-architecture ..."

REBOOTED

First test, manually selected to boot into Xen dom0, after which as
root the following checked Xen was operating...
$ xl info

Then set it to automatically boot into Xen...
$ dpkg-divert --divert /etc/grub.d/08_linux_xen --rename
/etc/grub.d/20_linux_xen
$ update-grub
REBOOTED


====================================
DID THE QUICK START TUTORIAL
USING GENERIC HW ON QEMU

0. Installed support tools and libs

$ apt-get install build-essential git-core cmake

1. Built the rumprun platform

$ mkdir -p ~/Repos/rumprun-hw
$ cd ~/Repos/rumprun-hw
$ git clone http://repo.rumpkernel.org/rumprun
$ cd rumprun
$ git submodule update --init
$ CC=cc ./build-rr.sh hw
Took note of this output needed later...
>> toolchain tuple: i486-rumprun-netbsdelf
>> cc wrapper: i486-rumprun-netbsdelf-gcc

$ echo $(pwd)
    /Users/ben/Repos/rumpkernel-hw/rumprun
$ export PATH="${PATH}:$(pwd)/rumprun/bin"


2. Built a toy application

$ mkdir -p test
$ cd test
$ cat > helloer.c << EOF   ....
$ cc -o helloer helloer.c
$ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c
$ rumprun-bake hw_generic helloer-rumprun.bin helloer-rumprun


3. Ran the toy application

$ su

$ rumprun qemu -i helloer-rumprun.bin

In another terminal, check its running
$ xl list
>> Name                                    ID  Mem  ...etc
>> Domain-0                              0    7583
>> rumprun-hellor-rumprun.bin  1        64

The app takes over the terminal its run from, so from another terminal...
$ xl destroy rumprun-hellor-rumprun.bin


====================================
ADAPTED THE QUICK START TUTORIAL
FOR XEN PARAVIRTUALIZATION

0. Started a new terminal to get an environment with a clean PATH

1. Built the rumprun platform

$ mkdir -p ~/Repos/rumprun-xen
$ cd ~/Repos/rumprun-xen
$ git clone http://repo.rumpkernel.org/rumprun
$ cd rumprun
$ git submodule update --init
$ CC=cc ./build-rr.sh xen   #<<<<
Took note of this output needed later...
>> toolchain tuple: i486-rumprun-netbsdelf
>> cc wrapper: i486-rumprun-netbsdelf-gcc

$ echo $(pwd)
    /Users/ben/Repos/rumpkernel-xen/rumprun
$ export PATH="${PATH}:$(pwd)/rumprun/bin"


2. Built a toy application

$ mkdir -p test
$ cd test
$ cat > helloer.c << EOF   ....
$ cc -o helloer helloer.c
$ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c
$ rumprun-bake xen_pv helloer-rumprun.bin helloer-rumprun


3. Ran toy application

$ cd ..
$ su
$ export PATH="${PATH}:`pwd`/rumprun/bin
$ cd test
$ rumprun xen -i helloer-rumprun.bin
$ xl list
>> Name                                    ID  Mem  ...etc
>> Domain-0                              0    7583
>> rumprun-hellor-rumprun.bin  1        64

This took over that terminal, so from another terminal...
$ xl destroy rumprun-hellor-rumprun.bin


====================================
TRIED IT USING CMAKE

$ cat > CMakeLists.txt << EOF see this link for text...
https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels

$ mkdir host_bld
$ cd host_bld
$ cmake -G "Unix Makefiles" ..
$ make
$ ./helloer

$ cd ..
$ mkdir rumprun_bld
$ cd rumprun_bld

Now with the given cmake line I got an error message:
   Could not find toolchain file:
../../rumprun/rumprun/share/x86_64-rumprun-netbsd-toolchain.cmake

$ find ../.. -name x86_64-rumprun-netbsd-toolchain.cmake
found it, so that the following worked...

$ cmake -G "Unix Makefiles"
-DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake
..
$ make
$ rumprun-bake xen_pv helloer.bin helloer
$ rumprun-bake hw_generic helloer.bin helloer
$ rumprun xen -i helloer.bin

Next I'll report on initial steps to get Cog working on Xen as a Rump
unikernel...

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

Eliot Miranda-2
 
Ben,

    great to hear you're working on this!  I'm watching with bated breath!  BTW, you might find that Docker is easier and more general; I believe there's no Xen for ARM for example, and ARM is increasingly interesting with up coming extremely cheap 64-bit versions of Pi and PINE.  Good luck!!

On Fri, Dec 11, 2015 at 10:27 AM, Ben Coman <[hidden email]> wrote:

I got curious about following up these two threads making Cog a unikernel...

Cog in the cloud
http://forum.world.st/Cog-in-the-cloud-td4796514.html

Hosting the Pharo VM on MirageOS
http://forum.world.st/Hosting-the-Pharo-VM-on-MirageOS-td4799424.html#none


One path forward is to use rumpkernel, which "provides a means to run
existing POSIX applications as unikernels on Xen"
https://blog.xenproject.org/2015/08/06/on-rump-kernels-and-the-rumprun-unikernel/

Deploying real-world software today as unikernels on Xen with Rumprun
http://events.linuxfoundation.org/sites/events/files/slides/xdps15-talk-final_0.pdf


So I thought I would try the "Building Rumprun Unikernels" tutorial here...
https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels
which I report here in case it was interest.  The tutorial is based on
QEMU (and I think I saw a comment somewhere that Cog doesn't run so
well on QEMU and needs Bochs), so after that I adapted it from QEMU to
Xen.


64-BIT DEBIAN 8 JESSIE & XEN

So first I installed Debian 8 Jessie 64-bit on my new laptop
$ uname -a
>> ...x86_64

Then installed Xen and after a reboot could choose it from the boot menu.
$ apt-get update
$ apt get install xen-system-amd64 libxen-dev

REBOOTED

Tested manual selection from menu to boot into Xen dom0.
Checked Xen was operating with...
$ xl info


32-BIT DEBIAN 8 JESSIE & XEN

However later I hit some complications determining whether the rump
unikernel was being compiled 32-bit or 64-bit in order to mix in with
32-bit Pharo, so to simplify things I did a complete reinstall of my
system as 32-bit.
$ uname -a
>> ...686-pae

Then installed Xen again  Xen only has a 64-bit hypervisor, but this
was not immediately available under 32-bit OS.  I can't exactly
remember but I think I just needed to add multiarch before it would
find the Xen package.
$ apt-get update
$ apt-get install multiarch
$ apt get install xen-system-amd64 libxen-dev
Otherwise it may have needed something like "dpkg --add-architecture ..."

REBOOTED

First test, manually selected to boot into Xen dom0, after which as
root the following checked Xen was operating...
$ xl info

Then set it to automatically boot into Xen...
$ dpkg-divert --divert /etc/grub.d/08_linux_xen --rename
/etc/grub.d/20_linux_xen
$ update-grub
REBOOTED


====================================
DID THE QUICK START TUTORIAL
USING GENERIC HW ON QEMU

0. Installed support tools and libs

$ apt-get install build-essential git-core cmake

1. Built the rumprun platform

$ mkdir -p ~/Repos/rumprun-hw
$ cd ~/Repos/rumprun-hw
$ git clone http://repo.rumpkernel.org/rumprun
$ cd rumprun
$ git submodule update --init
$ CC=cc ./build-rr.sh hw
Took note of this output needed later...
>> toolchain tuple: i486-rumprun-netbsdelf
>> cc wrapper: i486-rumprun-netbsdelf-gcc

$ echo $(pwd)
    /Users/ben/Repos/rumpkernel-hw/rumprun
$ export PATH="${PATH}:$(pwd)/rumprun/bin"


2. Built a toy application

$ mkdir -p test
$ cd test
$ cat > helloer.c << EOF   ....
$ cc -o helloer helloer.c
$ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c
$ rumprun-bake hw_generic helloer-rumprun.bin helloer-rumprun


3. Ran the toy application

$ su

$ rumprun qemu -i helloer-rumprun.bin

In another terminal, check its running
$ xl list
>> Name                                    ID  Mem  ...etc
>> Domain-0                              0    7583
>> rumprun-hellor-rumprun.bin  1        64

The app takes over the terminal its run from, so from another terminal...
$ xl destroy rumprun-hellor-rumprun.bin


====================================
ADAPTED THE QUICK START TUTORIAL
FOR XEN PARAVIRTUALIZATION

0. Started a new terminal to get an environment with a clean PATH

1. Built the rumprun platform

$ mkdir -p ~/Repos/rumprun-xen
$ cd ~/Repos/rumprun-xen
$ git clone http://repo.rumpkernel.org/rumprun
$ cd rumprun
$ git submodule update --init
$ CC=cc ./build-rr.sh xen   #<<<<
Took note of this output needed later...
>> toolchain tuple: i486-rumprun-netbsdelf
>> cc wrapper: i486-rumprun-netbsdelf-gcc

$ echo $(pwd)
    /Users/ben/Repos/rumpkernel-xen/rumprun
$ export PATH="${PATH}:$(pwd)/rumprun/bin"


2. Built a toy application

$ mkdir -p test
$ cd test
$ cat > helloer.c << EOF   ....
$ cc -o helloer helloer.c
$ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c
$ rumprun-bake xen_pv helloer-rumprun.bin helloer-rumprun


3. Ran toy application

$ cd ..
$ su
$ export PATH="${PATH}:`pwd`/rumprun/bin
$ cd test
$ rumprun xen -i helloer-rumprun.bin
$ xl list
>> Name                                    ID  Mem  ...etc
>> Domain-0                              0    7583
>> rumprun-hellor-rumprun.bin  1        64

This took over that terminal, so from another terminal...
$ xl destroy rumprun-hellor-rumprun.bin


====================================
TRIED IT USING CMAKE

$ cat > CMakeLists.txt << EOF see this link for text...
https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels

$ mkdir host_bld
$ cd host_bld
$ cmake -G "Unix Makefiles" ..
$ make
$ ./helloer

$ cd ..
$ mkdir rumprun_bld
$ cd rumprun_bld

Now with the given cmake line I got an error message:
   Could not find toolchain file:
../../rumprun/rumprun/share/x86_64-rumprun-netbsd-toolchain.cmake

$ find ../.. -name x86_64-rumprun-netbsd-toolchain.cmake
found it, so that the following worked...

$ cmake -G "Unix Makefiles"
-DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake
..
$ make
$ rumprun-bake xen_pv helloer.bin helloer
$ rumprun-bake hw_generic helloer.bin helloer
$ rumprun xen -i helloer.bin

Next I'll report on initial steps to get Cog working on Xen as a Rump
unikernel...

cheers -ben



--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

Holger Freyther


> On 11 Dec 2015, at 20:09, Eliot Miranda <[hidden email]> wrote:
>
> Ben,
>
>     great to hear you're working on this!  I'm watching with bated breath!  BTW, you might find that Docker is easier and more general; I believe there's no Xen for ARM for example, and ARM is increasingly interesting with up coming extremely cheap 64-bit versions of Pi and PINE.  Good luck!!

@Eliot: There is Xen for ARMv8 (because ARM is interesting for multi-core servers)
@Ben: Awesome. Will be interesting how small one can get it (with still functioning TCP/IP)
Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

Colin Putney-3
In reply to this post by Ben Coman
 


On Fri, Dec 11, 2015 at 10:27 AM, Ben Coman <[hidden email]> wrote:
 
Next I'll report on initial steps to get Cog working on Xen as a Rump
unikernel...

Awesome! 
Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

timrowledge
In reply to this post by Holger Freyther


> On 11-12-2015, at 12:38 PM, Holger Freyther <[hidden email]> wrote:
>
>
>
>> On 11 Dec 2015, at 20:09, Eliot Miranda <[hidden email]> wrote:
>>
>> Ben,
>>
>>    great to hear you're working on this!  I'm watching with bated breath!  BTW, you might find that Docker is easier and more general; I believe there's no Xen for ARM for example, and ARM is increasingly interesting with up coming extremely cheap 64-bit versions of Pi and PINE.  Good luck!!
>
> @Eliot: There is Xen for ARMv8 (because ARM is interesting for multi-core servers)
> @Ben: Awesome. Will be interesting how small one can get it (with still functioning TCP/IP)
>
And as always, I am available for doing an ARMv8  COG.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: DMZ: Divide Memory by Zero


Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

Ben Coman
In reply to this post by Ben Coman
 
Whoops I notice an erroneous line in that recipe...

On Sat, Dec 12, 2015 at 2:27 AM, Ben Coman <[hidden email]> wrote:
> I got curious about following up these two threads making Cog a unikernel...
> So I thought I would try the "Building Rumprun Unikernels" tutorial here...
> https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikernels
> which I report here in case it was interest.  The tutorial is based on
> QEMU (and I think I saw a comment somewhere that Cog doesn't run so
> well on QEMU and needs Bochs), so after that I adapted it from QEMU to
> Xen.
...
> $ cmake -G "Unix Makefiles"
> -DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake
> ..
> $ make
> $ rumprun-bake xen_pv helloer.bin helloer
ERRONEOUS> $ rumprun-bake hw_generic helloer.bin helloer
> $ rumprun xen -i helloer.bin

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

Ben Coman
 
Here is the recipe for my first successful build and run through to an error...
         StackVM.unikernel: could not find any display driver

This is done under 32-bit Debian 8 Jessie.
Attached is StackRumpXenUnikernelConfig.st

# Built the Rumprun Platform

$ mkdir -p cogunikernel
$ git clone http://repo.rumpkernel.org/rumprun
$ cd rumprun
$ git submodule update --init
$ CC=cc ./build-rr.sh xen
>> toolchain tuple: i486-rumprun-netbsdelf
>> cc wrapper: i486-rumprun-netbsdelf-gcc
>> installed to "/home/ben/Repos/cogunikernel/rumprun/./rumprun"


# Generated Cog VM sources

$ cd ../../cogunikernel
$ git clone https://github.com/pharo-project/pharo-vm.git
$ cd pharo-vm/image
$ ./newImage.sh
$ ./pharo-ui generator.image
     Tools > File Browser
     Filed in StackRumpXenUnikernelConfig.st
     From class comment, evaluated...
          StackRumpXenUnikernelConfig new
               generateSources; generate.


# Built the StackVM and StackVM.unikernel

$ bash build.sh


# Tested them...

$ cd ../results
$ ./StackVM
   > bash: ./StackVM: cannot execute binary file: Exec format error
   Good. That was the expected result.

$ rumprun xen -i StackVM.unikernel

   > StackVM.unikernel: could not find any display driver


# Stop unikernel machine running, from another terminal...
$ xl list
$ xl destroy rumprun-StackVM.unikernel

================

Now I basically just poked, cut and slashed the config until it could
compile and boot up the unikernel, without regard for what that may
break.  Making the VM *actually* work is bigger step that for now is
beyond me, but I'll have a bite at it.   Really my motivation in doing
this after I stumbled on the rumprun tutorial, was I wanted to crack
the seal on it to interest others capable :).   I'd be interested if
the above recipe is successful for others.

Todo:
* incorporate rumprun as an auto-built third party library, rather
than outside, perhaps using git submodule like the rump kernel already
does (if these can be nested)
* determine how to deal with different rumprun toolchains (depending
on which platform the target)
* move the baking step from the build script to the generated CMake
configuration
* determine how to deal with lack of display
* determine how an image file might be passed to the VM - maybe baking
the VM and image into a ram disk (?)

cheers -ben

StackRumpXenUnikernelConfig.st (11K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Cog Rump Xen unikernel

Ben Coman
 
There were a couple of minor log errors below, so I attach a script
that does it all,
tested on 32-bit Debian Jessie.
cheers -ben

On Sat, Dec 12, 2015 at 2:57 PM, Ben Coman <[hidden email]> wrote:

> Here is the recipe for my first successful build and run through to an error...
>          StackVM.unikernel: could not find any display driver
>
> This is done under 32-bit Debian 8 Jessie.
> Attached is StackRumpXenUnikernelConfig.st
>
> # Built the Rumprun Platform
>
> $ mkdir -p cogunikernel
> $ git clone http://repo.rumpkernel.org/rumprun
> $ cd rumprun
> $ git submodule update --init
> $ CC=cc ./build-rr.sh xen
>>> toolchain tuple: i486-rumprun-netbsdelf
>>> cc wrapper: i486-rumprun-netbsdelf-gcc
>>> installed to "/home/ben/Repos/cogunikernel/rumprun/./rumprun"
>
>
> # Generated Cog VM sources
>
> $ cd ../../cogunikernel
> $ git clone https://github.com/pharo-project/pharo-vm.git
> $ cd pharo-vm/image
> $ ./newImage.sh
> $ ./pharo-ui generator.image
>      Tools > File Browser
>      Filed in StackRumpXenUnikernelConfig.st
>      From class comment, evaluated...
>           StackRumpXenUnikernelConfig new
>                generateSources; generate.
>
>
> # Built the StackVM and StackVM.unikernel
>
> $ bash build.sh
>
>
> # Tested them...
>
> $ cd ../results
> $ ./StackVM
>    > bash: ./StackVM: cannot execute binary file: Exec format error
>    Good. That was the expected result.
>
> $ rumprun xen -i StackVM.unikernel
>
>    > StackVM.unikernel: could not find any display driver
>
>
> # Stop unikernel machine running, from another terminal...
> $ xl list
> $ xl destroy rumprun-StackVM.unikernel
>
> ================
>
> Now I basically just poked, cut and slashed the config until it could
> compile and boot up the unikernel, without regard for what that may
> break.  Making the VM *actually* work is bigger step that for now is
> beyond me, but I'll have a bite at it.   Really my motivation in doing
> this after I stumbled on the rumprun tutorial, was I wanted to crack
> the seal on it to interest others capable :).   I'd be interested if
> the above recipe is successful for others.
>
> Todo:
> * incorporate rumprun as an auto-built third party library, rather
> than outside, perhaps using git submodule like the rump kernel already
> does (if these can be nested)
> * determine how to deal with different rumprun toolchains (depending
> on which platform the target)
> * move the baking step from the build script to the generated CMake
> configuration
> * determine how to deal with lack of display
> * determine how an image file might be passed to the VM - maybe baking
> the VM and image into a ram disk (?)
>
> cheers -ben

build-cogunikernel.sh (1K) Download Attachment