Macros?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Macros?

Debiller 777
Well, I've already asked about adding new literals to pharo or Smalltalk in general, however this time I have a better idea:
macros. Can they be added? Because if I understand correctly they may be the only way to do that.
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Stephan Eggermont-3
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan



Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Ben Coman
In reply to this post by Debiller 777


On 25 May 2018 at 21:22, Debiller 777 <[hidden email]> wrote:
Well, I've already asked about adding new literals to pharo or Smalltalk in general, however this time I have a better idea:
macros. Can they be added? Because if I understand correctly they may be the only way to do that.

Can you provide an example how how you would use a macro?  

cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Clément Béra
In reply to this post by Stephan Eggermont-3
What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).
In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

EstebanLM


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--

Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Clément Béra


On Fri, May 25, 2018 at 10:44 PM, Esteban Lorenzano <[hidden email]> wrote:


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Can you turn asMethodConstant On and Off so you have the constant in production and not a development time ? Typically at development time I change the constants generated a lot and I don't want to waste time recompiling all the time.

But yeah, my project is from 2014. I guess instead of preprocessing you could do everything at runtime with AST manipulation/recompilation like in asMethodConstant.

The main point of preprocessing IMO is to control performance, it's just easier for me to just look at the generated bytecode and change the preprocessor until it gets what I want, it's not always easy to run code that will change your method at runtime quickly so you can look at the bytecode generated.

Anyway, I am not convinced at all something like that should be in the base image.
 

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--




--
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Clément Béra
Just mentioning another use-case:

getDatabaseInstance
    ^ (Production CifTrue: [Database] CifFalse: [MockDatabase]) new

Since I use conditional compilation more often than just precompiling constants.

I don't see an equivalent of asMethodConstant AST manipulation at runtime strategy in the image for this case right now but it's probably a couple lines of code with Reflectivity.

On Sat, May 26, 2018 at 7:28 AM, Clément Bera <[hidden email]> wrote:


On Fri, May 25, 2018 at 10:44 PM, Esteban Lorenzano <[hidden email]> wrote:


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Can you turn asMethodConstant On and Off so you have the constant in production and not a development time ? Typically at development time I change the constants generated a lot and I don't want to waste time recompiling all the time.

But yeah, my project is from 2014. I guess instead of preprocessing you could do everything at runtime with AST manipulation/recompilation like in asMethodConstant.

The main point of preprocessing IMO is to control performance, it's just easier for me to just look at the generated bytecode and change the preprocessor until it gets what I want, it's not always easy to run code that will change your method at runtime quickly so you can look at the bytecode generated.

Anyway, I am not convinced at all something like that should be in the base image.
 

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--




--



--
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Denis Kudriashov
Hi

2018-05-26 8:46 GMT+03:00 Clément Bera <[hidden email]>:
Just mentioning another use-case:

getDatabaseInstance
    ^ (Production CifTrue: [Database] CifFalse: [MockDatabase]) new

I think following code will work:

getDatabaseInstance
     ^ (Production ifTrue: [Database] ifFalse: [MockDatabase]) asMethodConstant new

 

Since I use conditional compilation more often than just precompiling constants.

I don't see an equivalent of asMethodConstant AST manipulation at runtime strategy in the image for this case right now but it's probably a couple lines of code with Reflectivity.

On Sat, May 26, 2018 at 7:28 AM, Clément Bera <[hidden email]> wrote:


On Fri, May 25, 2018 at 10:44 PM, Esteban Lorenzano <[hidden email]> wrote:


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Can you turn asMethodConstant On and Off so you have the constant in production and not a development time ? Typically at development time I change the constants generated a lot and I don't want to waste time recompiling all the time.

But yeah, my project is from 2014. I guess instead of preprocessing you could do everything at runtime with AST manipulation/recompilation like in asMethodConstant.

The main point of preprocessing IMO is to control performance, it's just easier for me to just look at the generated bytecode and change the preprocessor until it gets what I want, it's not always easy to run code that will change your method at runtime quickly so you can look at the bytecode generated.

Anyway, I am not convinced at all something like that should be in the base image.
 

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--




--



--

Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Clément Béra


On Sat, May 26, 2018 at 8:07 AM, Denis Kudriashov <[hidden email]> wrote:
Hi

2018-05-26 8:46 GMT+03:00 Clément Bera <[hidden email]>:
Just mentioning another use-case:

getDatabaseInstance
    ^ (Production CifTrue: [Database] CifFalse: [MockDatabase]) new

I think following code will work:

getDatabaseInstance
     ^ (Production ifTrue: [Database] ifFalse: [MockDatabase]) asMethodConstant new


Arf, obviously you try to work around the example. You can have anything in the block, not necessarily a constant.

foo
   ^ Condition  CifTrue: [self bar] CifFalse: [self baz]

And I want the condition to be replaced by the contents of one block or the other, not just by a constant.

 

Since I use conditional compilation more often than just precompiling constants.

I don't see an equivalent of asMethodConstant AST manipulation at runtime strategy in the image for this case right now but it's probably a couple lines of code with Reflectivity.

On Sat, May 26, 2018 at 7:28 AM, Clément Bera <[hidden email]> wrote:


On Fri, May 25, 2018 at 10:44 PM, Esteban Lorenzano <[hidden email]> wrote:


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Can you turn asMethodConstant On and Off so you have the constant in production and not a development time ? Typically at development time I change the constants generated a lot and I don't want to waste time recompiling all the time.

But yeah, my project is from 2014. I guess instead of preprocessing you could do everything at runtime with AST manipulation/recompilation like in asMethodConstant.

The main point of preprocessing IMO is to control performance, it's just easier for me to just look at the generated bytecode and change the preprocessor until it gets what I want, it's not always easy to run code that will change your method at runtime quickly so you can look at the bytecode generated.

Anyway, I am not convinced at all something like that should be in the base image.
 

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--




--



--




--
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Denis Kudriashov

2018-05-26 9:33 GMT+03:00 Clément Bera <[hidden email]>:


On Sat, May 26, 2018 at 8:07 AM, Denis Kudriashov <[hidden email]> wrote:
Hi

2018-05-26 8:46 GMT+03:00 Clément Bera <[hidden email]>:
Just mentioning another use-case:

getDatabaseInstance
    ^ (Production CifTrue: [Database] CifFalse: [MockDatabase]) new

I think following code will work:

getDatabaseInstance
     ^ (Production ifTrue: [Database] ifFalse: [MockDatabase]) asMethodConstant new


Arf, obviously you try to work around the example. You can have anything in the block, not necessarily a constant.

foo
   ^ Condition  CifTrue: [self bar] CifFalse: [self baz]

And I want the condition to be replaced by the contents of one block or the other, not just by a constant.

Yes, I see. It's different


 

Since I use conditional compilation more often than just precompiling constants.

I don't see an equivalent of asMethodConstant AST manipulation at runtime strategy in the image for this case right now but it's probably a couple lines of code with Reflectivity.

On Sat, May 26, 2018 at 7:28 AM, Clément Bera <[hidden email]> wrote:


On Fri, May 25, 2018 at 10:44 PM, Esteban Lorenzano <[hidden email]> wrote:


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Can you turn asMethodConstant On and Off so you have the constant in production and not a development time ? Typically at development time I change the constants generated a lot and I don't want to waste time recompiling all the time.

But yeah, my project is from 2014. I guess instead of preprocessing you could do everything at runtime with AST manipulation/recompilation like in asMethodConstant.

The main point of preprocessing IMO is to control performance, it's just easier for me to just look at the generated bytecode and change the preprocessor until it gets what I want, it's not always easy to run code that will change your method at runtime quickly so you can look at the bytecode generated.

Anyway, I am not convinced at all something like that should be in the base image.
 

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--




--



--


Reply | Threaded
Open this post in threaded view
|

Re: Macros?

NorbertHartl
In reply to this post by Clément Béra
I need to say that this is not a good use case. First and most important is that you should not have deployment switches in your code. Your system should be configured for a special deployment mode not detecting in code. Second but not least important you add references to test classes in your business logic. Usually you do not load test code in a production image. 

Couldn‘t resist, sorry

Norbert

Am 26.05.2018 um 07:46 schrieb Clément Bera <[hidden email]>:

Just mentioning another use-case:

getDatabaseInstance
    ^ (Production CifTrue: [Database] CifFalse: [MockDatabase]) new

Since I use conditional compilation more often than just precompiling constants.

I don't see an equivalent of asMethodConstant AST manipulation at runtime strategy in the image for this case right now but it's probably a couple lines of code with Reflectivity.

On Sat, May 26, 2018 at 7:28 AM, Clément Bera <[hidden email]> wrote:


On Fri, May 25, 2018 at 10:44 PM, Esteban Lorenzano <[hidden email]> wrote:


On 25 May 2018, at 17:30, Clément Bera <[hidden email]> wrote:

What about a preprocessor like the Java preprocessors ? The Truffle project relies heavily on that for high performance Java and it's quite nice. It's difficult to do that in Smalltalk right now.

I think if you want to do what are asking for you just need to write a bytecode compiler extension.

I did something similar in the past to have precomputed constants through AST manipulation at compilation time. You can find it here with examples: http://smalltalkhub.com/#!/~ClementBera/NeoCompiler. Once the code is loaded you need to recompile the examples (NeoCompilerExample withAllSubclassesDo: #compileAll.). With it you can write code such as:
[ Smalltalk vm class ] Cvalue    
And depending if the compiler allowsPrecompilation or not, the bytecode compiler generates:
Smalltalk vm class 
Or just a push literal with the precomputed value (the class VirtualMachine).

this is not what #asMethodConstant provides?

Can you turn asMethodConstant On and Off so you have the constant in production and not a development time ? Typically at development time I change the constants generated a lot and I don't want to waste time recompiling all the time.

But yeah, my project is from 2014. I guess instead of preprocessing you could do everything at runtime with AST manipulation/recompilation like in asMethodConstant.

The main point of preprocessing IMO is to control performance, it's just easier for me to just look at the generated bytecode and change the preprocessor until it gets what I want, it's not always easy to run code that will change your method at runtime quickly so you can look at the bytecode generated.

Anyway, I am not convinced at all something like that should be in the base image.
 

Esteban

In the end I decided not to use this, but you can use it and extend it to support more than just constants (any AST manipulation is possible).
Just checked it works in Pharo 6.

On Fri, May 25, 2018 at 4:26 PM, Stephan Eggermont <[hidden email]> wrote:
Debiller 777 <[hidden email]>
wrote:
> Well, I've already asked about adding new literals to pharo or
Smalltalk in
> general, however this time I have a better idea:
> macros. Can they be added? Because if I understand correctly they may be
> the only way to do that.

Why do you think they would be a good idea? We have powerful
meta-programming facilities that are well understood and somewhat supported
by tooling. How do we get value out of macros?

Stephan






--




--



--
Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Michael J. Forster
In reply to this post by Ben Coman
On Fri, May 25, 2018 at 10:12 AM, Ben Coman <[hidden email]> wrote:

>
>
> On 25 May 2018 at 21:22, Debiller 777 <[hidden email]> wrote:
>>
>> Well, I've already asked about adding new literals to pharo or Smalltalk
>> in general, however this time I have a better idea:
>> macros. Can they be added? Because if I understand correctly they may be
>> the only way to do that.
>
>
> Can you provide an example how how you would use a macro?
>
> cheers -ben



Late to the party, but I thought some might find it interesting to
look through the "macros vs. blocks" thread on the old Lightweight
Languages mailing list. Posts from Guy Steele Jr., Avi Bryant, Trevor
Blackwell (rewrote Graham's Yahoo Store in Smalltalk), and others:

https://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02060.html


Cheers,

Mike

Reply | Threaded
Open this post in threaded view
|

Re: Macros?

Richard O'Keefe
I've been reconstructing an old programming language.
It originally had a very simple but awkward macro
facility:  if an identifier was declared as a macro,
whenever it was called, it could inspect/consume as
much of the remaining tokens as it wished and could
splice other tokens in.  On top of this a very nice
"declarative" macro system was built, which I loved
when I used it.  Since my implementation is not yet
self-hosting, I've been using higher-order functions
instead, and the more I do this the less I want macros.

You could think of the rewrite rules in VisualWorks
as something resembling macros.  Come to think of it,
Lukas Renggli, Helvetia, ... found it!




On 12 June 2018 at 12:10, Michael Forster <[hidden email]> wrote:
On Fri, May 25, 2018 at 10:12 AM, Ben Coman <[hidden email]> wrote:
>
>
> On 25 May 2018 at 21:22, Debiller 777 <[hidden email]> wrote:
>>
>> Well, I've already asked about adding new literals to pharo or Smalltalk
>> in general, however this time I have a better idea:
>> macros. Can they be added? Because if I understand correctly they may be
>> the only way to do that.
>
>
> Can you provide an example how how you would use a macro?
>
> cheers -ben



Late to the party, but I thought some might find it interesting to
look through the "macros vs. blocks" thread on the old Lightweight
Languages mailing list. Posts from Guy Steele Jr., Avi Bryant, Trevor
Blackwell (rewrote Graham's Yahoo Store in Smalltalk), and others:

https://people.csail.mit.edu/gregs/ll1-discuss-archive-html/msg02060.html


Cheers,

Mike