Pharo Shared Memory with C++ executable

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Pharo Shared Memory with C++ executable

kilon.alios
Hey fellow Pharoers

I have great news, I was able to finally !!!! understand how shared memory file works and actually do it

Why is that great news you may ask.

A shared memory file is a file that maps a specific area of memory, which means when you write to a specific part of the memory it writes directly to the file without you having to actually do this. Memory mapped files have the distinct characteristic of being able to share memory between processes. This actually is the fastest way to make two process communicate and exchange data. 

Its called Inter Process Communication or IPC for short , but is not technically communication because both executables share the memory. 

This means that its possible to make Pharo use C++ libraries from inside a C++ executable. Because basically the shared memory will allow Pharo and the C++ executables to share data, part of that data can be messages to execute specific functions. 

What is super cool about shared memory mapped files is not only that they are super fast(its what the OS uses to load shared libraries ie. DLLs) , its not that it automagically stores the shared memory to disk like a Pharo image (so we can store live data) but foremost that a great deal of programming languages can use Memory Mapped Files which basically means they have wrapped the mmap functions of the LibC library (C/C++ Standard Library). 

That means we can now use this to make Pharo able to use C++ libraries, python libraries and any library from any programming language. Its also a solution when you dont have access to a DLL or making one is very hard like in my case that I use Unreal Engine. As you can imagine this will replace/extend my socket bridge that I made for python. 

Of course all the above are no magic, you have to have some basic understanding of how C++ works and how memory mapped files work, funny thing is that shared memory is actually simpler to understand than C++. You also have to make the messaging system and manage the synchronisation. But those are details which I will implement anyway.

So my challenge right now is to move this to Pharo. I have done it with C++, I made two different executables which share a string with the value "hello". 

I need two functions to do this one is open, which is the classic way to open files in C/C++ in my case is

#define FILEPATH "mmapped.bin"
 fd = open(FILEPATH, O_RDWR | O_CREAT | O_TRUNC, (mode_t)0600);


second one is lseek which expand the size of the file to a desired size

result = lseek(fd, FILESIZE-1, SEEK_SET);


the third one is mmap which is doing the actually mapping of the shared memory

 map =(std::string*) mmap(0, FILESIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);


Now my challenges after reading the UFFI documentation are the following 

1) How I can pass things like O_CREAT , MAP_SHARED
2) How I cast a void pointer (its what mmap returns) to a C++ string or  C++ int

In case you want to take a look at the code is part of a project I named Atlas which can be found here


The file atlas-server.cpp creates the memory mapped file and writes to the shared memory the string "hello"

The file atlas-client.cpp gains access to the shared memory and retrieves the "hello" string

Atlas basically will be the library that will give Pharo full access to Unreal internals. 
Reply | Threaded
Open this post in threaded view
|

Re: Pharo Shared Memory with C++ executable

kilon.alios

"Now my challenges after reading the UFFI documentation are the following 

1) How I can pass things like O_CREAT , MAP_SHARED
2) How I cast a void pointer (its what mmap returns) to a C++ string or  C++ int"

After carefully re-reading the documentation its seems the answer to question 1 is that I will have to take the constant values from the header and follow the instructions of the section "passing variables"

 About question 2, I could fetch the void pointer as return type , I know that what returns is ExternalAdress pointer but have no idea how to fetch the contents that the pointer points to. Maybe ByteArray>>integerAt:size:signed: is what I am looking for ?

Will give it a try tomorrow. In the meanwhile I am open to suggestions.
Reply | Threaded
Open this post in threaded view
|

Re: Pharo Shared Memory with C++ executable

EstebanLM

> On 6 Nov 2016, at 22:17, Dimitris Chloupis <[hidden email]> wrote:
>
>
> "Now my challenges after reading the UFFI documentation are the following

oh crap… I forget I have to finish that :(
UFFO doc: back to my TODO list...

Esteban

>
> 1) How I can pass things like O_CREAT , MAP_SHARED
> 2) How I cast a void pointer (its what mmap returns) to a C++ string or  C++ int"
>
> After carefully re-reading the documentation its seems the answer to question 1 is that I will have to take the constant values from the header and follow the instructions of the section "passing variables"
>
>  About question 2, I could fetch the void pointer as return type , I know that what returns is ExternalAdress pointer but have no idea how to fetch the contents that the pointer points to. Maybe ByteArray>>integerAt:size:signed: is what I am looking for ?
>
> Will give it a try tomorrow. In the meanwhile I am open to suggestions.