Smalltalk › Pharo › Pharo Smalltalk Users

[ANN] Python3Generator and MatplotLibBridge

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

32 messages Options

SergeStinckwich

Re: [ANN] Python3Generator and MatplotLibBridge

On Wed, Jan 10, 2018 at 2:33 PM, Julien <[hidden email]> wrote:

I guess the hardest part will be to define the 1 - 1 mapping between Numpy and PolyMath APIs, no?

I mean, how can you ensure that features using Floats will behave exactly the same in Python and Smalltalk for example?

​Float are the same basically,​ but ​I don't want to ensure something.
I just want to a quick&dirty code transformation :-)

Like : 
np.square(x) <=> x squared
np.sum(x,1) <=> x sum

et donc : 
np.sum(np.square(x), 1) <=> (x squared) sum

Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)

"Programs must be written for people to read, and only incidentally for machines to execute."

http://www.doesnotunderstand.org/

SergeStinckwich

Re: [ANN] Python3Generator and MatplotLibBridge

In reply to this post by Thierry Goubier

On Wed, Jan 10, 2018 at 2:35 PM, Thierry Goubier <[hidden email]> wrote:

2018-01-10 14:26 GMT+01:00 Serge Stinckwich <[hidden email]>:

On Wed, Jan 10, 2018 at 2:23 PM, Julien <[hidden email]> wrote:
Hello Serge,

Do you mean pieces of code you wrote in Python using Numpy that you want to transform to Smalltalk code using PolyMath?

It’s kind of the other way around of this project. :-)

yes this is exactly the opposite :-(
I guess I need a Python parser and after that I can do AST transformation ...

Hi Serge,

there is a Python2.7 parser with AST that could be used.

​yes I might have a look ... where is the code ?

​

Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)

"Programs must be written for people to read, and only incidentally for machines to execute."

http://www.doesnotunderstand.org/

Julien Delplanque-2

Re: [ANN] Python3Generator and MatplotLibBridge

In reply to this post by SergeStinckwich

I see :-)

Another interesting project then. :-)

Julien

---

Julien Delplanque
Doctorant à l’Université de Lille 1

http://juliendelplanque.be/phd.html
Equipe Rmod, Inria

Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq

Numéro de téléphone: +333 59 35 86 40

Le 10 janv. 2018 à 14:39, Serge Stinckwich <[hidden email]> a écrit :

On Wed, Jan 10, 2018 at 2:33 PM, Julien <[hidden email]> wrote:
I guess the hardest part will be to define the 1 - 1 mapping between Numpy and PolyMath APIs, no?

I mean, how can you ensure that features using Floats will behave exactly the same in Python and Smalltalk for example?

Float are the same basically, but I don't want to ensure something.
I just want to a quick&dirty code transformation :-)

Like :
np.square(x) <=> x squared
np.sum(x,1) <=> x sum

et donc :
np.sum(np.square(x), 1) <=> (x squared) sum

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Thierry Goubier

Re: [ANN] Python3Generator and MatplotLibBridge

In reply to this post by SergeStinckwich

2018-01-10 14:41 GMT+01:00 Serge Stinckwich <[hidden email]>:

On Wed, Jan 10, 2018 at 2:35 PM, Thierry Goubier <[hidden email]> wrote:

2018-01-10 14:26 GMT+01:00 Serge Stinckwich <[hidden email]>:

On Wed, Jan 10, 2018 at 2:23 PM, Julien <[hidden email]> wrote:
Hello Serge,

Do you mean pieces of code you wrote in Python using Numpy that you want to transform to Smalltalk code using PolyMath?

It’s kind of the other way around of this project. :-)

yes this is exactly the opposite :-(
I guess I need a Python parser and after that I can do AST transformation ...

Hi Serge,

there is a Python2.7 parser with AST that could be used.

yes I might have a look ... where is the code ?

SmaCC on github. Look for SmaCC-Python and SmaCC-Python-Tests.If you're using Moose,

you should be able to import just SmaCC-Python from the github repository. There is even a

BaselineOfPythonParser, but I haven't used it.

Thierry

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

kilon.alios

Re: [ANN] Python3Generator and MatplotLibBridge

In reply to this post by Julien Delplanque-2

On Mon, Jan 8, 2018 at 5:50 PM Julien <[hidden email]> wrote:

Beware that no mechanism to get back values from Python is defined for now (except if you just want the String
representation of those objects, then you can get that if you use atlas).

I’d like to have that but it is not easy. I would like a way to describe how to map Python’s objects to Pharo’s objects
from Pharo and to have the code to do that in Python side generated automatically but some thinking is needed…

Julien

It won't be easy

I have noted in the past the similarities between Pharo and Python but here there is also a massive difference.

In Pharo we have the mentality of reinventing many stuffs ourselves so the systems is based to a very large extend on Pharo that runs on very efficient JIT VM. Almost everything is Pharo code.

Python on the other hand is the exact opposite, Python runs on an extremely slow VM but more than 50% of its code in written in C, Python itself or its third party libraries.

So the irony is that even though in theory and practice Python code is one of the slowest , in actual scenarios its one of the fastest able to outperform even Java to a very large extend because its libraries that are made for high performance like Numpy are predominately C code. Pretty much every C or C++ library has wrappers for Python which is what made it so popular. Hence the reason behind the insanity when it comes to top performance Python being second only to C++.

So you will have not only map the Python oobjects to Pharo object but also map the C code. Even though this may sound impossible the good news is that Python libraries, having the extension pyd , are basically DLLs made supporting a specific API which is very minimal in design. Similar to our UFFI , the only major difference is that library is responsible of keeping track of the reference count, which is used by Python GC to erase no longer used objects from memory. Which is a very simple increase and decrease.

So not only you will have to remap the Python objects but also DLLs as well.

To add salt to the wound , Python has something that Pharo does not. Multiple inheritance. Which of course makes your goal even harder. Python coder's generally avoid multiple inheritance as much as we avoid overriding doesNotUnderstand , but its a feature they use none the less.

Hence why I did not even consider trying something like that. Plus you will be gaining no advantage because C code is unportable to Pharo anyway. Sure we have Slang , but Slang is extra careful with types which normal Pharo code does not. You will be losing performance because yes porting to Pharo as much as JIT VM may be great , its no much for Python libraries that merely leverage C. Plus you will have to update it each time the library changes etc.

So my conclusion was that the most efficient way was to let Python execute the library and just manipulate Python.

It's much easier to do the exact opposite, which goes against our mentality , and port your Pharo objects to Python objects. Because a) your objects will always be far less than a complete library and system b) you wont have to worry about C code because you wont have any c) all the other negatives I mention , disappear.

It usual case of not being able to have your cake and eat it too.

Julien Delplanque-2

Re: [ANN] Python3Generator and MatplotLibBridge

Even if I learned things from your answer, I think you misunderstood what I was talking about. :p

What I want is a mechanism to describe python’s objects from Pharo and being able to:

1. Serialise those objects from Python

2. Send those object to Pharo (whatever the way it is done: socket, write in a file from python and read from Pharo, etc…)

3. Materialize those serialised Python’s objects as Pharo objects.

Basically, I want to be able to transfer data from Python to Pharo easily.

Something better than getting the values held by Python objects by parsing their String representation « by hand »... :-)

Julien

---

Julien Delplanque
Doctorant à l’Université de Lille 1

http://juliendelplanque.be/phd.html
Equipe Rmod, Inria

Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq

Numéro de téléphone: +333 59 35 86 40

Le 10 janv. 2018 à 16:43, Dimitris Chloupis <[hidden email]> a écrit :

On Mon, Jan 8, 2018 at 5:50 PM Julien <[hidden email]> wrote:
Beware that no mechanism to get back values from Python is defined for now (except if you just want the String
representation of those objects, then you can get that if you use atlas).

I’d like to have that but it is not easy. I would like a way to describe how to map Python’s objects to Pharo’s objects
from Pharo and to have the code to do that in Python side generated automatically but some thinking is needed…

Julien

It won't be easy

I have noted in the past the similarities between Pharo and Python but here there is also a massive difference.

In Pharo we have the mentality of reinventing many stuffs ourselves so the systems is based to a very large extend on Pharo that runs on very efficient JIT VM. Almost everything is Pharo code.

Python on the other hand is the exact opposite, Python runs on an extremely slow VM but more than 50% of its code in written in C, Python itself or its third party libraries.

So the irony is that even though in theory and practice Python code is one of the slowest , in actual scenarios its one of the fastest able to outperform even Java to a very large extend because its libraries that are made for high performance like Numpy are predominately C code. Pretty much every C or C++ library has wrappers for Python which is what made it so popular. Hence the reason behind the insanity when it comes to top performance Python being second only to C++.

So you will have not only map the Python oobjects to Pharo object but also map the C code. Even though this may sound impossible the good news is that Python libraries, having the extension pyd , are basically DLLs made supporting a specific API which is very minimal in design. Similar to our UFFI , the only major difference is that library is responsible of keeping track of the reference count, which is used by Python GC to erase no longer used objects from memory. Which is a very simple increase and decrease.

So not only you will have to remap the Python objects but also DLLs as well.

To add salt to the wound , Python has something that Pharo does not. Multiple inheritance. Which of course makes your goal even harder. Python coder's generally avoid multiple inheritance as much as we avoid overriding doesNotUnderstand , but its a feature they use none the less.

Hence why I did not even consider trying something like that. Plus you will be gaining no advantage because C code is unportable to Pharo anyway. Sure we have Slang , but Slang is extra careful with types which normal Pharo code does not. You will be losing performance because yes porting to Pharo as much as JIT VM may be great , its no much for Python libraries that merely leverage C. Plus you will have to update it each time the library changes etc.

So my conclusion was that the most efficient way was to let Python execute the library and just manipulate Python.

It's much easier to do the exact opposite, which goes against our mentality , and port your Pharo objects to Python objects. Because a) your objects will always be far less than a complete library and system b) you wont have to worry about C code because you wont have any c) all the other negatives I mention , disappear.

It usual case of not being able to have your cake and eat it too.

kilon.alios

Re: [ANN] Python3Generator and MatplotLibBridge

yeah I am afraid I understood exactly what you meant because at first that was my initial desire when I made Atlas.

Essentially you want auto mapping of Python objects to Pharo, so you wont have to deal with Python and do everything from inside the Pharo image. So essentially reading and writing Python objects as if they were Pharo objects.

Take a more careful look to my reply, you will see that is exactly what I am focusing on.

Unless you dont mean automatically and mean manually, which in that case is something I have already done.

I used SmaCC , as already pointed out it already maps python syntax to pharo objects. So lucky for you this work is done for you. But you still need to traverse the object tree to do the remapping and there lies the tricky part for all the reasons I already explained in the previous answer ,again assuming you want to do this automagically.

Ironically one would expect to be relative easy to convert Python objects to Pharo objects , while making Python able to live code , incredible hard.

After trying both I can tell you that turning Python to a live coding environment was a walk in the park. Mapping objects is making me have nightmares.

You will be lucky if the objects are pure data of primitive format, this means, floats, strings, integers and all the other usual suspects. In that case you dont even need to worry about conversion you just export to JSON and import back to Pharo and voila you have objects that are both Pharo and Python compatible. JSON is such a common format that can be used any way you like, you want to write it to a file, share it in memory, transmit it through socket, anything you like its possible.

But then moment the Python objects maps to some exotic object with dependencies to C code, welcome to dependency hell, fasten your seat belt its going to be a bumpy ride.

On Wed, Jan 10, 2018 at 5:59 PM Julien <[hidden email]> wrote:

Even if I learned things from your answer, I think you misunderstood what I was talking about. :p

What I want is a mechanism to describe python’s objects from Pharo and being able to:
1. Serialise those objects from Python
2. Send those object to Pharo (whatever the way it is done: socket, write in a file from python and read from Pharo, etc…)
3. Materialize those serialised Python’s objects as Pharo objects.

Basically, I want to be able to transfer data from Python to Pharo easily.

Something better than getting the values held by Python objects by parsing their String representation « by hand »... :-)

Julien

---
Julien Delplanque
Doctorant à l’Université de Lille 1
http://juliendelplanque.be/phd.html
Equipe Rmod, Inria
Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq
Numéro de téléphone: <a href="tel:+33%203%2059%2035%2086%2040" value="+33359358640" target="_blank">+333 59 35 86 40

Le 10 janv. 2018 à 16:43, Dimitris Chloupis <[hidden email]> a écrit :

On Mon, Jan 8, 2018 at 5:50 PM Julien <[hidden email]> wrote:
Beware that no mechanism to get back values from Python is defined for now (except if you just want the String
representation of those objects, then you can get that if you use atlas).

I’d like to have that but it is not easy. I would like a way to describe how to map Python’s objects to Pharo’s objects
from Pharo and to have the code to do that in Python side generated automatically but some thinking is needed…

Julien

It won't be easy

I have noted in the past the similarities between Pharo and Python but here there is also a massive difference.

In Pharo we have the mentality of reinventing many stuffs ourselves so the systems is based to a very large extend on Pharo that runs on very efficient JIT VM. Almost everything is Pharo code.

Python on the other hand is the exact opposite, Python runs on an extremely slow VM but more than 50% of its code in written in C, Python itself or its third party libraries.

So the irony is that even though in theory and practice Python code is one of the slowest , in actual scenarios its one of the fastest able to outperform even Java to a very large extend because its libraries that are made for high performance like Numpy are predominately C code. Pretty much every C or C++ library has wrappers for Python which is what made it so popular. Hence the reason behind the insanity when it comes to top performance Python being second only to C++.

So you will have not only map the Python oobjects to Pharo object but also map the C code. Even though this may sound impossible the good news is that Python libraries, having the extension pyd , are basically DLLs made supporting a specific API which is very minimal in design. Similar to our UFFI , the only major difference is that library is responsible of keeping track of the reference count, which is used by Python GC to erase no longer used objects from memory. Which is a very simple increase and decrease.

So not only you will have to remap the Python objects but also DLLs as well.

To add salt to the wound , Python has something that Pharo does not. Multiple inheritance. Which of course makes your goal even harder. Python coder's generally avoid multiple inheritance as much as we avoid overriding doesNotUnderstand , but its a feature they use none the less.

Hence why I did not even consider trying something like that. Plus you will be gaining no advantage because C code is unportable to Pharo anyway. Sure we have Slang , but Slang is extra careful with types which normal Pharo code does not. You will be losing performance because yes porting to Pharo as much as JIT VM may be great , its no much for Python libraries that merely leverage C. Plus you will have to update it each time the library changes etc.

So my conclusion was that the most efficient way was to let Python execute the library and just manipulate Python.

It's much easier to do the exact opposite, which goes against our mentality , and port your Pharo objects to Python objects. Because a) your objects will always be far less than a complete library and system b) you wont have to worry about C code because you wont have any c) all the other negatives I mention , disappear.

It usual case of not being able to have your cake and eat it too.

Stephane Ducasse-3

Re: [ANN] Python3Generator and MatplotLibBridge

In reply to this post by Julien Delplanque-2

Johan Brichau could execute java code from VisualWorks.
He had special metaclass whose dictionaries contains strange mirror to
java methods. So I'm interested to see how far you can get :)

On Wed, Jan 10, 2018 at 4:58 PM, Julien <[hidden email]> wrote:

> Even if I learned things from your answer, I think you misunderstood what I
> was talking about. :p
>
> What I want is a mechanism to describe python’s objects from Pharo and being
> able to:
> 1. Serialise those objects from Python
> 2. Send those object to Pharo (whatever the way it is done: socket, write in
> a file from python and read from Pharo, etc…)
> 3. Materialize those serialised Python’s objects as Pharo objects.
>
> Basically, I want to be able to transfer data from Python to Pharo easily.
>
> Something better than getting the values held by Python objects by parsing
> their String representation « by hand »... :-)
>
> Julien
>
> ---
> Julien Delplanque
> Doctorant à l’Université de Lille 1
> http://juliendelplanque.be/phd.html
> Equipe Rmod, Inria
> Bâtiment B 40, avenue Halley 59650 Villeneuve d'Ascq
> Numéro de téléphone: +333 59 35 86 40
>
> Le 10 janv. 2018 à 16:43, Dimitris Chloupis <[hidden email]> a écrit
> :
>
>
>
> On Mon, Jan 8, 2018 at 5:50 PM Julien <[hidden email]> wrote:
>>
>> Beware that no mechanism to get back values from Python is defined for now
>> (except if you just want the String
>> representation of those objects, then you can get that if you use atlas).
>>
>> I’d like to have that but it is not easy. I would like a way to describe
>> how to map Python’s objects to Pharo’s objects
>> from Pharo and to have the code to do that in Python side generated
>> automatically but some thinking is needed…
>>
>> Julien
>
>
> It won't be easy
>
> I have noted in the past the similarities between Pharo and Python but here
> there is also a massive difference.
>
> In Pharo we have the mentality of reinventing many stuffs ourselves so the
> systems is based to a very large extend on Pharo that runs on very efficient
> JIT VM. Almost everything is Pharo code.
>
> Python on the other hand is the exact opposite, Python runs on an extremely
> slow VM but more than 50% of its code in written in C, Python itself or its
> third party libraries.
>
> So the irony is that even though in theory and practice Python code is one
> of the slowest , in actual scenarios its one of the fastest able to
> outperform even Java to a very large extend because its libraries that are
> made for high performance like Numpy are predominately C code. Pretty much
> every C or C++ library has wrappers for Python which is what made it so
> popular. Hence the reason behind the insanity when it comes to top
> performance Python being second only to C++.
>
> So you will have not only map the Python oobjects to Pharo object but also
> map the C code. Even though this may sound impossible the good news is that
> Python libraries, having the extension pyd , are basically DLLs made
> supporting a specific API which is very minimal in design. Similar to our
> UFFI , the only major difference is that library is responsible of keeping
> track of the reference count, which is used by Python GC to erase no longer
> used objects from memory. Which is a very simple increase and decrease.
>
> So not only you will have to remap the Python objects but also DLLs as well.
>
> To add salt to the wound , Python has something that Pharo does not.
> Multiple inheritance. Which of course makes your goal even harder. Python
> coder's generally avoid multiple inheritance as much as we avoid overriding
> doesNotUnderstand , but its a feature they use none the less.
>
> Hence why I did not even consider trying something like that. Plus you will
> be gaining no advantage because C code is unportable to Pharo anyway. Sure
> we have Slang , but Slang is extra careful with types which normal Pharo
> code does not. You will be losing performance because yes porting to Pharo
> as much as JIT VM may be great , its no much for Python libraries that
> merely leverage C. Plus you will have to update it each time the library
> changes etc.
>
> So my conclusion was that the most efficient way was to let Python execute
> the library and just manipulate Python.
>
> It's much easier to do the exact opposite, which goes against our mentality
> , and port your Pharo objects to Python objects. Because a) your objects
> will always be far less than a complete library and system b) you wont have
> to worry about C code because you wont have any c) all the other negatives I
> mention , disappear.
>
> It usual case of not being able to have your cake and eat it too.
>
>
>

askoh

Re: [ANN] Python3Generator and MatplotLibBridge

Administrator

In reply to this post by Julien Delplanque-2

Good job. Is there something which translates Python to Smalltalk?

Thanks,
Aik-Siong Koh

--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Stephane Ducasse-3

Re: [ANN] Python3Generator and MatplotLibBridge

In reply to this post by Thierry Goubier

Hi Thierry

One of these days I think that I will have to have a look at it :)

Stef

On Wed, Jan 10, 2018 at 2:35 PM, Thierry Goubier <[hidden email]> wrote:

2018-01-10 14:26 GMT+01:00 Serge Stinckwich <[hidden email]>:

On Wed, Jan 10, 2018 at 2:23 PM, Julien <[hidden email]> wrote:
Hello Serge,

Do you mean pieces of code you wrote in Python using Numpy that you want to transform to Smalltalk code using PolyMath?

It’s kind of the other way around of this project. :-)

yes this is exactly the opposite :-(
I guess I need a Python parser and after that I can do AST transformation ...

Hi Serge,

there is a Python2.7 parser with AST that could be used.

Thierry

Or using regex I can do basic stuff ...

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Thierry Goubier

Re: [ANN] Python3Generator and MatplotLibBridge

Hi Stef,

it's the one done with Damien, 3 or 4 years ago.

Thierry

2018-01-11 8:23 GMT+01:00 Stephane Ducasse <[hidden email]>:

Hi Thierry

One of these days I think that I will have to have a look at it :)

Stef

On Wed, Jan 10, 2018 at 2:35 PM, Thierry Goubier <[hidden email]> wrote:

2018-01-10 14:26 GMT+01:00 Serge Stinckwich <[hidden email]>:

On Wed, Jan 10, 2018 at 2:23 PM, Julien <[hidden email]> wrote:
Hello Serge,

Do you mean pieces of code you wrote in Python using Numpy that you want to transform to Smalltalk code using PolyMath?

It’s kind of the other way around of this project. :-)

yes this is exactly the opposite :-(
I guess I need a Python parser and after that I can do AST transformation ...

Hi Serge,

there is a Python2.7 parser with AST that could be used.

Thierry

Or using regex I can do basic stuff ...

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/

Stephane Ducasse-3

Re: [ANN] Python3Generator and MatplotLibBridge

I thought so and this is good to have it around.

On Thu, Jan 11, 2018 at 10:13 AM, Thierry Goubier <[hidden email]> wrote:

Hi Stef,

it's the one done with Damien, 3 or 4 years ago.

Thierry

2018-01-11 8:23 GMT+01:00 Stephane Ducasse <[hidden email]>:
Hi Thierry

One of these days I think that I will have to have a look at it :)

Stef

On Wed, Jan 10, 2018 at 2:35 PM, Thierry Goubier <[hidden email]> wrote:

2018-01-10 14:26 GMT+01:00 Serge Stinckwich <[hidden email]>:

On Wed, Jan 10, 2018 at 2:23 PM, Julien <[hidden email]> wrote:
Hello Serge,

Do you mean pieces of code you wrote in Python using Numpy that you want to transform to Smalltalk code using PolyMath?

It’s kind of the other way around of this project. :-)

yes this is exactly the opposite :-(
I guess I need a Python parser and after that I can do AST transformation ...

Hi Serge,

there is a Python2.7 parser with AST that could be used.

Thierry

Or using regex I can do basic stuff ...

--
Serge Stinckwich
UMI UMMISCO 209 (IRD/UPMC/UY1)
"Programs must be written for people to read, and only incidentally for machines to execute."
http://www.doesnotunderstand.org/