||

It is obviously a compromise (or a continuum) between abstractions and performance.

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

 

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

Thierry

(*) I tried to get rid of them in some of my stuff, where it is used in the same way GTInspector and GTSpooter uses pragmas, and I had to stop because of that feature. All other uses of pragmas are replaceable (and often, as in the gt stuff, redundant).

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <[hidden email]> wrote:

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

 

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax).  They have been broadly used.  IME they have simplified and reduced code where ever they have been used.  They don't have to be there but they're a good thing.  Why do you want to get rid of them?

(and yes, I'm biassed)

2015-02-05 18:51 GMT+01:00 Eliot Miranda <[hidden email]>:

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <[hidden email]> wrote:

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

 

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax).  They have been broadly used.  IME they have simplified and reduced code where ever they have been used.  They don't have to be there but they're a good thing.  Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example:

One of the uses of pragmas is associating methods containing Gui commands or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)).

The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot).

Each method is one line longer ("the pragma").

Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma.

(There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: )

Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

 

(and yes, I'm biassed)

Then you're the right person to give me counter arguments...

(Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of script hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with)

Thierry

(Look. I started using Smalltalk in 1992... and up to your description, I wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

On Thu, Feb 5, 2015 at 11:34 AM, Thierry Goubier <[hidden email]> wrote:

2015-02-05 18:51 GMT+01:00 Eliot Miranda <[hidden email]>:

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <[hidden email]> wrote:

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

 

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax).  They have been broadly used.  IME they have simplified and reduced code where ever they have been used.  They don't have to be there but they're a good thing.  Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example:

OK, the claim is too strong.  But they /have/ simplified code in cases where they're appropriate.  And I can cite several examples.

 

One of the uses of pragmas is associating methods containing Gui commands or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)).

The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot).

Each method is one line longer ("the pragma").

Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma.

(There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: )

Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

But triggering in the background happens for maintaining change sets, notifying other clients too.  It's not as if pragmas introduced such triggering; that kind of triggering has been in use for a long time.  And being able to reshape the GUI automatically is very useful.

 

 

(and yes, I'm biassed)

Then you're the right person to give me counter arguments...

(Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of script hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with)

Why?  A good use of pragmas is to associate meta data with a particular method.  Having calls off to the side always introduces the need for book-keeping to keep those methods off to the side in sync with the methods they're describing.  Typically everyone rolls their own.  But here we're adding a level of triggering just to keep the metadata methods in sync.

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods.  Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

This is one idiom that covers a host of other cases.  That's why I claim that whenever I've seen it used it has reduced complexity.

Some history.  Steve Dahl, I developed pragmas at ParcPlace, with Vassili Bykov adding abstractions for accessing them.  The first step was to replace some ugly class-side code to set unwind bits in ensure: and ifCurtailed: by a pragma the compiler would recognise and set the bits itself.  The first real use was to make the VisualWorks launcher's menus extensible.  Before pragmas the launcher's menu was static and had lots of disabled entries for launching tools that were sold separately such as DLLAndCConnect.  With pragmas the launcher's menu was defined with the base system's tools and then extended as each tool package was loaded, or cut-back as each tool was unloaded.  So that decoupled the launcher from introducing new tools.  A nice result.  

We then started using it for the browser and one could plug-in a single tool without redefining the browser's menu methods, which decoupled each extension.  All this was done in the context of the parcel system, where we could rapidly load packages (parcels ~= Fuel).  Pragmas allowed us to decouple these tools where they collided in places like menu definition, tool registration.

Then Tami Lee, who was managing the COM connection that turned a VW image into a COM server, became the first "user" of pragmas outside of myself and Steve. She used it to replace a lot of class-side methods that defined the signatures of methods that comprised the server.  It was a lovely clean-up.  One could define the COM signature for a method in the method itself, and the class side lost about three separate methods that defined all that metadata.  One could read the server method itself and understand its semantics without having to consult the class-side methods.  One didn't have to know that there was metadata hidden on the class side because it was right there in your face.

Then Vassili used it for his cool inspector framework, Trippy, which was similar to Glamour in some ways, and was a huge improvement over the old Inspector framework, again resulting in a much more pluggable, decoupled and extensible system.  Vassili also added the abstractions for accessing pragmas in methods.

Then we added checking so that one could restrict the compiler to accept only legal pragmas for a given class.  But if we defined the legal pragmas in a class-side method, say legalPragmas, then this would be exactly the kind of single point for extensions that causes collisions between packages, each of which might want to add its own set of pragmas.  The solution... use a pragma to mark a class-side method as defining a set of legal pragmas for a class.  One could have more than one method defining a set of legal pragmas; packages wishing to add their own cool pragmas were decoupled.  Once the system because recursive, it had to be a good idea ;-).

There are other uses; you've seen them.  I used them in VMMaker to eliminate metadata that was embedded as sends to methods defined as ^self that Slang had to extract and analyse, and filter-out from generated code.  They simplified Slang's code anaylsis, made the simulator more efficient (since there were no longer sends to execute).  My point is that in all the cases I've seen, using pragmas has

- simplified the code

- made it obvious that methods have metadata associated with them

- replaced specialized ways of associating metadata with code by the general pragma mechanism

and in many of the cases it has

- provided a more decoupled system

- provided a more dynamic and extensible system

I've been meaning to write up the history of pragmas for ages, but Vassili, Steve or I have always been too busy.  I think a community paper on their use and history would be worth-while, and might go a long way to reduce antipathies like yours.  I will forever be in debt to anyone who wants to volunteer to help me write such a paper.

 

Thierry

(Look. I started using Smalltalk in 1992... and up to your description, I wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

They /have/ to follow literal message syntax.  t's all the compiler will accept.  That's why there needs to be a paper.

--

best,

Eliot

2015-02-05 21:28 GMT+01:00 Eliot Miranda <[hidden email]>:

On Thu, Feb 5, 2015 at 11:34 AM, Thierry Goubier <[hidden email]> wrote:

2015-02-05 18:51 GMT+01:00 Eliot Miranda <[hidden email]>:

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <[hidden email]> wrote:

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

 

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax).  They have been broadly used.  IME they have simplified and reduced code where ever they have been used.  They don't have to be there but they're a good thing.  Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example:

OK, the claim is too strong.  But they /have/ simplified code in cases where they're appropriate.  And I can cite several examples.

 

One of the uses of pragmas is associating methods containing Gui commands or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)).

The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot).

Each method is one line longer ("the pragma").

Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma.

(There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: )

Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

But triggering in the background happens for maintaining change sets, notifying other clients too.  It's not as if pragmas introduced such triggering; that kind of triggering has been in use for a long time.  And being able to reshape the GUI automatically is very useful.

I don't contest the possibilities, it's just that they add a significant layer of complexity when non mastered (how many Pharo developpers know which event you have to register to to receive all new methods notifications? Is that documented in one of the books?), and that, except for using them as <primitives> or for extensibility, I see other syntaxes and smalltalk code which are simpler.

A good example is that the pragma syntax is never included in the one page Smalltalk syntax description :)

(and yes, I'm biassed)

Then you're the right person to give me counter arguments...

(Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of script hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with)

Why?  A good use of pragmas is to associate meta data with a particular method.  Having calls off to the side always introduces the need for book-keeping to keep those methods off to the side in sync with the methods they're describing.  Typically everyone rolls their own.  But here we're adding a level of triggering just to keep the metadata methods in sync.

I agree with the "metadata", but I'd prefer a executable, evaluate that block as a medata literal than the pragma. Something that says "onceAndStoreAsMetadata", to a block, for example. An API to compiled methods which says add metadata.

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

 

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods.  Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

This is one idiom that covers a host of other cases.  That's why I claim that whenever I've seen it used it has reduced complexity.

Some history.  Steve Dahl, I developed pragmas at ParcPlace, with Vassili Bykov adding abstractions for accessing them.  The first step was to replace some ugly class-side code to set unwind bits in ensure: and ifCurtailed: by a pragma the compiler would recognise and set the bits itself.  The first real use was to make the VisualWorks launcher's menus extensible.  Before pragmas the launcher's menu was static and had lots of disabled entries for launching tools that were sold separately such as DLLAndCConnect.  With pragmas the launcher's menu was defined with the base system's tools and then extended as each tool package was loaded, or cut-back as each tool was unloaded.  So that decoupled the launcher from introducing new tools.  A nice result.  

We then started using it for the browser and one could plug-in a single tool without redefining the browser's menu methods, which decoupled each extension.  All this was done in the context of the parcel system, where we could rapidly load packages (parcels ~= Fuel).  Pragmas allowed us to decouple these tools where they collided in places like menu definition, tool registration.

Then Tami Lee, who was managing the COM connection that turned a VW image into a COM server, became the first "user" of pragmas outside of myself and Steve. She used it to replace a lot of class-side methods that defined the signatures of methods that comprised the server.  It was a lovely clean-up.  One could define the COM signature for a method in the method itself, and the class side lost about three separate methods that defined all that metadata.  One could read the server method itself and understand its semantics without having to consult the class-side methods.  One didn't have to know that there was metadata hidden on the class side because it was right there in your face.

Then Vassili used it for his cool inspector framework, Trippy, which was similar to Glamour in some ways, and was a huge improvement over the old Inspector framework, again resulting in a much more pluggable, decoupled and extensible system.  Vassili also added the abstractions for accessing pragmas in methods.

Then we added checking so that one could restrict the compiler to accept only legal pragmas for a given class.  But if we defined the legal pragmas in a class-side method, say legalPragmas, then this would be exactly the kind of single point for extensions that causes collisions between packages, each of which might want to add its own set of pragmas.  The solution... use a pragma to mark a class-side method as defining a set of legal pragmas for a class.  One could have more than one method defining a set of legal pragmas; packages wishing to add their own cool pragmas were decoupled.  Once the system because recursive, it had to be a good idea ;-).

Ok, I start to see where the abstraction wasn't working so well... since pragmas are not executed, when writing a method you can't know if the pragma is correct, because even executing the method may not trigger the pragma induced code. So you need the legalPragmas to give metadata on metadata for the compiler to do a bit of static checking, but it doesn't work for system-wide pragmas unless you extend Object :(

And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There are other uses; you've seen them.  I used them in VMMaker to eliminate metadata that was embedded as sends to methods defined as ^self that Slang had to extract and analyse, and filter-out from generated code.  They simplified Slang's code anaylsis, made the simulator more efficient (since there were no longer sends to execute).  My point is that in all the cases I've seen, using pragmas has

- simplified the code

- made it obvious that methods have metadata associated with them

- replaced specialized ways of associating metadata with code by the general pragma mechanism

and in many of the cases it has

- provided a more decoupled system

- provided a more dynamic and extensible system

Yes, and I can point out some of its shortcomings: it's non-obvious, it's limited, tools, even that many years later don't support them well (in Squeak or Pharo, at least), its redundant in quite a few variants.

Please, could we improve a bit? Methods belonging to multiple protocols would give us the same decoupling as pragmas, and I would be free to avoid them where I shouldn't have to use them :)

I've been meaning to write up the history of pragmas for ages, but Vassili, Steve or I have always been too busy.  I think a community paper on their use and history would be worth-while, and might go a long way to reduce antipathies like yours.  I will forever be in debt to anyone who wants to volunteer to help me write such a paper.

That would certainly be interesting :)

 

 

Thierry

(Look. I started using Smalltalk in 1992... and up to your description, I wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

They /have/ to follow literal message syntax.  t's all the compiler will accept.  That's why there needs to be a paper.

Yes!

Thanks for taking the time to argument,

Hi Eliot,

maybe I'll frame the core question a bit differently, thanks to your explanations.

I see pragmas used mainly for two things:

1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified.

2- categorizing things in a decoupled, package-extension compatible way.

I still think there is something in the design that bother me. 1- and 2- are convention-based, so as is most of Smalltalk anyway, so diregarding another approach to 2- as convention-based is a bit dishonest: pragmas get more authority because they look like they have been integrated in the language definition compared to competing solutions.

No, this is just because I start to wonder if methods shouldn't have a bit more in that "define, add and manipulate" metadata idea, i.e. make it more explicit and start to have tools which represent methods as more like objects and not only text items (methods having metadata, additional slots or instance variables, counters, statistics, ast, traces, break points, etc...). And, in a way, a need to have two levels of execution: one which happens at compile time (i.e. when one accept a method), one which happens at runtime.

For that, I think pragmas are a start, but they are not the complete answer. And I think we should solve the package-extension compatible thing in 2- to not have 2- (which is just for me a plain categorisation issue) polluting the design by introducing additional/unrelated requirements.

2015-02-05 22:47 GMT+01:00 Eliot Miranda <[hidden email]>:

Hi Thierry,

A block is far more difficult to deal with than a pragma.  A pragma has a simple key, its selector, and simple access to its arameters, the arguments.  A block is opaque.  Essentially it can only be executable.  Pragmas are both executable and function as annotations.

Pragmas are executable, but in a context which is not visible at the place they are written.

 

 

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

 

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods.  Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

Yes, but there end up being lots of naming conventions and they are non-obvious.  Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions.  They just need documenting ;-).

They are still conventions which needs documentation. Pragmas are, in allmost all uses, indirections (i.e. a tag meaning something to a far away object), and the meaning associated to a pragma is, in every case, a roll-your-own solution. And they are clearly "in your face" :)

 

That's not necessarily true.  Some pragmas do cause processing at compile time.  For example, an FFI signature pragma can be checked at compile-time.  But it's in keeping with smalltalk that type checking is not performed at compile time in most cases, isn't it?  Why should one require that pragmas be semantically checked at compile time when normal SMalltalk code isn't?  At least one knows that the message instance the pragma is compiled to is a valid object and *can* be performed.  So pone does know at least that the pragma is executable.  Obviously whether that ability to be executed only becomes potent with the right receiver.  So any compiled pragma as the potential to be usefully evaluated.

Ok, but its all fuzzy.

It may be executed, but it may be not.

It may send an error if incorrect, but it may fail silently.

It may be executed at compile time, it may be executed sometime later triggered by the compilation, or it may be executed at runtime, as if it was normal code. And of course it may not be executed at all ...

You see what I mean?

 

And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There is lots about Smalltalk that is non-obvious, about programming in general that is non-obvious.  I don't see that as a specific criticism of pragmas.  Once one knows the idiom it is easy to use; its wide-spread use is evidence of that.  And hopefully this conversation will help make it more obvious :-).  That it is limited is also not perhaps a useful criticism.  The issue is whether it is adequate.  I think it is.  A literal message goes a long way.  I'e not heard of complaints about limitations so far.  Do you have specific examples where pragmas are inadequate?  The tools /do/ support them.  One can do senders and implementors in the browser and see the methods that include them, and the implementors of pragmas.  browseAllSelect: will narrow down the search.  There is an API for programmatic use (pragmasDo: et al).

See [*][**] for the tools support. For me, pragmas, even in their definition, have serious semantic issues. If they were strictly, as you describe them, a metadata API access executed at compile time, then they would be a lot stronger.

Make them sent at compile-time on the method object, and that would probably solve all the true arguments I have against them :)

for example: <worldMenu> against <tagAndAnnounce: #worldMenu>; <tag: #gtInspectorExtension> instead of <gtInspectorExtension>, etc...

(<tagAndAnnounce:> would gain a bit of performance, as well).

Thierry

[*] Searching a bit for menu pragmas in a Pharo image seems to return a bit of a mess. There is so many of them, all differents, and interestingly, this does not return the code triggered (or using) them, which makes discovery of what the pragma is used for (and its parameters) an 'interesting' problem.

I suggest a task then: an illustrated sequence of how to search for the code triggering world menu addition of a new command, for someone which is new to Pharo ;)

[**] even when used for categorisation, the API is prone to bugs, such as having a method without pragma overriding a superclass method with pragma because they have the same name.

Hi Eliot,

maybe I'll frame the core question a bit differently, thanks to your explanations.

I see pragmas used mainly for two things:

1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified.

2- categorizing things in a decoupled, package-extension compatible way.

I still think there is something in the design that bother me. 1- and 2- are convention-based, so as is most of Smalltalk anyway, so diregarding another approach to 2- as convention-based is a bit dishonest: pragmas get more authority because they look like they have been integrated in the language definition compared to competing solutions.

No, this is just because I start to wonder if methods shouldn't have a bit more in that "define, add and manipulate" metadata idea, i.e. make it more explicit and start to have tools which represent methods as more like objects and not only text items (methods having metadata, additional slots or instance variables, counters, statistics, ast, traces, break points, etc...). And, in a way, a need to have two levels of execution: one which happens at compile time (i.e. when one accept a method), one which happens at runtime.

For that, I think pragmas are a start, but they are not the complete answer. And I think we should solve the package-extension compatible thing in 2- to not have 2- (which is just for me a plain categorisation issue) polluting the design by introducing additional/unrelated requirements.

2015-02-05 22:47 GMT+01:00 Eliot Miranda <[hidden email]>:

Hi Thierry,

A block is far more difficult to deal with than a pragma.  A pragma has a simple key, its selector, and simple access to its arameters, the arguments.  A block is opaque.  Essentially it can only be executable.  Pragmas are both executable and function as annotations.

Pragmas are executable, but in a context which is not visible at the place they are written.

 

 

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

 

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods.  Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

Yes, but there end up being lots of naming conventions and they are non-obvious.  Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions.  They just need documenting ;-).

They are still conventions which needs documentation. Pragmas are, in allmost all uses, indirections (i.e. a tag meaning something to a far away object), and the meaning associated to a pragma is, in every case, a roll-your-own solution. And they are clearly "in your face" :)

 

That's not necessarily true.  Some pragmas do cause processing at compile time.  For example, an FFI signature pragma can be checked at compile-time.  But it's in keeping with smalltalk that type checking is not performed at compile time in most cases, isn't it?  Why should one require that pragmas be semantically checked at compile time when normal SMalltalk code isn't?  At least one knows that the message instance the pragma is compiled to is a valid object and *can* be performed.  So pone does know at least that the pragma is executable.  Obviously whether that ability to be executed only becomes potent with the right receiver.  So any compiled pragma as the potential to be usefully evaluated.

Ok, but its all fuzzy.

It may be executed, but it may be not.

It may send an error if incorrect, but it may fail silently.

It may be executed at compile time, it may be executed sometime later triggered by the compilation, or it may be executed at runtime, as if it was normal code. And of course it may not be executed at all ...

You see what I mean?

 

And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There is lots about Smalltalk that is non-obvious, about programming in general that is non-obvious.  I don't see that as a specific criticism of pragmas.  Once one knows the idiom it is easy to use; its wide-spread use is evidence of that.  And hopefully this conversation will help make it more obvious :-).  That it is limited is also not perhaps a useful criticism.  The issue is whether it is adequate.  I think it is.  A literal message goes a long way.  I'e not heard of complaints about limitations so far.  Do you have specific examples where pragmas are inadequate?  The tools /do/ support them.  One can do senders and implementors in the browser and see the methods that include them, and the implementors of pragmas.  browseAllSelect: will narrow down the search.  There is an API for programmatic use (pragmasDo: et al).

See [*][**] for the tools support. For me, pragmas, even in their definition, have serious semantic issues. If they were strictly, as you describe them, a metadata API access executed at compile time, then they would be a lot stronger.

Make them sent at compile-time on the method object, and that would probably solve all the true arguments I have against them :)

for example: <worldMenu> against <tagAndAnnounce: #worldMenu>; <tag: #gtInspectorExtension> instead of <gtInspectorExtension>, etc...

(<tagAndAnnounce:> would gain a bit of performance, as well).

Thierry

[*] Searching a bit for menu pragmas in a Pharo image seems to return a bit of a mess. There is so many of them, all differents, and interestingly, this does not return the code triggered (or using) them, which makes discovery of what the pragma is used for (and its parameters) an 'interesting' problem.

I suggest a task then: an illustrated sequence of how to search for the code triggering world menu addition of a new command, for someone which is new to Pharo ;)

[**] even when used for categorisation, the API is prone to bugs, such as having a method without pragma overriding a superclass method with pragma because they have the same name.

On 5 February 2015 at 21:20, Thierry Goubier <[hidden email]> wrote:
>
>
> 2015-02-05 21:28 GMT+01:00 Eliot Miranda <[hidden email]>:
>>
>>
>>
>> On Thu, Feb 5, 2015 at 11:34 AM, Thierry Goubier
>> <[hidden email]> wrote:
>>>
>>>
>>>
>>> 2015-02-05 18:51 GMT+01:00 Eliot Miranda <[hidden email]>:
>>>>
>>>>
>>>>
>>>> On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier
>>>> <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>> 2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <[hidden email]>:
>>>>>>
>>>>>> It is obviously a compromise (or a continuum) between abstractions and
>>>>>> performance.
>>>>>
>>>>>
>>>>> I agree. With a special view in that we are in a sub domain where
>>>>> simple things well designed (Smalltalk, that is) are amazingly good at
>>>>> supporting complex designs.
>>>>>
>>>>>>
>>>>>>
>>>>>> But there should remain a focus on efficiency (not just speed but also
>>>>>> memory), it is hard to fix these things years later.
>>>>>
>>>>>
>>>>> And I like the fact that efficient code and design is often a pleasure
>>>>> to read and learn from :)
>>>>>
>>>>> Now, being radical: could we get rid of pragmas ? The only reason I see
>>>>> to them is that they allow extension by external packages, because we can't
>>>>> have methods which belong to two protocols (*).
>>>>
>>>>
>>>> They are a Smalltalk-centric way of adding arbitrary metadata to
>>>> methods; Smalltalk-centric in that a pragma is a Message instance, may be
>>>> queried for senders, performed, etc, and that it can be parsed using the
>>>> standard compiler (they add no new syntax).  They have been broadly used.
>>>> IME they have simplified and reduced code where ever they have been used.
>>>> They don't have to be there but they're a good thing.  Why do you want to
>>>> get rid of them?
>>>
>>>
>>> Because the "they have simplified and reduced code where ever they have
>>> been used" is wrong. I just have to give you a counter example:
>>
>>
>> OK, the claim is too strong.  But they /have/ simplified code in cases
>> where they're appropriate.  And I can cite several examples.
>>
>>>
>>>
>>> One of the uses of pragmas is associating methods containing Gui commands
>>> or settings to specific objects. Based on an object inspected or selected,
>>> you search among all its methods the ones containing a specific pragma (and
>>> you order them by a parameter to that pragma, if you want), and you execute
>>> that method to retrieve the objects you want (presentations, menu commands,
>>> shortcuts, you name it, I use it :)).
>>>
>>> The code to do that is exactly as long as the one which, on the same
>>> object, retrieve all methods under a certain protocol (the latter being
>>> faster than the pragma one, to boot).
>>>
>>> Each method is one line longer ("the pragma").
>>>
>>> Each such method usually has in its name a copy of the pragma
>>> (gtInspectorXXX methods, I'm looking at you), because of course this is far
>>> more user friendly to indicate its purpose in the method name than in only
>>> the pragma.
>>>
>>> (There are two more arguments for the use of pragmas in that context, one
>>> which has a direct counter-example, one which hasn't: )
>>>
>>> Moreover, the semantic of pragmas is "interesting" to describe, and in
>>> some cases, require a good amount of dark magic about a global object
>>> listening to all methods changes and capturing (and executing) certain
>>> methods in a vague relation about when this is going to happen, or being
>>> triggered on specific system events (main menu rebuilding, anyone?). The
>>> funny thing is to see that pattern visible on a profile when loading
>>> packages (talk of a scalable approach).
>>
>>
>> But triggering in the background happens for maintaining change sets,
>> notifying other clients too.  It's not as if pragmas introduced such
>> triggering; that kind of triggering has been in use for a long time.  And
>> being able to reshape the GUI automatically is very useful.
>
>
> I don't contest the possibilities, it's just that they add a significant
> layer of complexity when non mastered (how many Pharo developpers know which
> event you have to register to to receive all new methods notifications? Is
> that documented in one of the books?), and that, except for using them as
> <primitives> or for extensibility, I see other syntaxes and smalltalk code
> which are simpler.
>
> A good example is that the pragma syntax is never included in the one page
> Smalltalk syntax description :)
>
>>
>>
>>>
>>>
>>>>
>>>> (and yes, I'm biassed)
>>>
>>>
>>> Then you're the right person to give me counter arguments...
>>>
>>> (Now, I'd look differently at pragmas used for gradual typing and so
>>> on... But even for something like FFI, I'd seriously prefer to have
>>> Smalltalk calls to describe the call and its arguments than a kind of script
>>> hidden inside pragmas, just for the discoverability and because it makes one
>>> less idiom to deal with)
>>
>>
>> Why?  A good use of pragmas is to associate meta data with a particular
>> method.  Having calls off to the side always introduces the need for
>> book-keeping to keep those methods off to the side in sync with the methods
>> they're describing.  Typically everyone rolls their own.  But here we're
>> adding a level of triggering just to keep the metadata methods in sync.
>
>
> I agree with the "metadata", but I'd prefer a executable, evaluate that
> block as a medata literal than the pragma. Something that says
> "onceAndStoreAsMetadata", to a block, for example. An API to compiled
> methods which says add metadata.
>
> I strongly agree with your keep it in sync argument, still. Pragmas are
> better than nothing.
>
>>
>>
>> There is no such need with pragmas; they are always in sync with the
>> methods they describe because they are embedded in their methods.  Instead
>> we can use triggering to do useful things, adding a pane to open inspectors
>> as soon as we define the method that describes the pane, adding or removing
>> a menu entry, etc.
>
>
> Just a naming convention does just that perfectly fine, and with less lines
> (except for extensions by external packages) and faster code in many cases.
>
>>
>>
>> This is one idiom that covers a host of other cases.  That's why I claim
>> that whenever I've seen it used it has reduced complexity.
>>
>> Some history.  Steve Dahl, I developed pragmas at ParcPlace, with Vassili
>> Bykov adding abstractions for accessing them.  The first step was to replace
>> some ugly class-side code to set unwind bits in ensure: and ifCurtailed: by
>> a pragma the compiler would recognise and set the bits itself.  The first
>> real use was to make the VisualWorks launcher's menus extensible.  Before
>> pragmas the launcher's menu was static and had lots of disabled entries for
>> launching tools that were sold separately such as DLLAndCConnect.  With
>> pragmas the launcher's menu was defined with the base system's tools and
>> then extended as each tool package was loaded, or cut-back as each tool was
>> unloaded.  So that decoupled the launcher from introducing new tools.  A
>> nice result.
>>
>> We then started using it for the browser and one could plug-in a single
>> tool without redefining the browser's menu methods, which decoupled each
>> extension.  All this was done in the context of the parcel system, where we
>> could rapidly load packages (parcels ~= Fuel).  Pragmas allowed us to
>> decouple these tools where they collided in places like menu definition,
>> tool registration.
>>
>> Then Tami Lee, who was managing the COM connection that turned a VW image
>> into a COM server, became the first "user" of pragmas outside of myself and
>> Steve. She used it to replace a lot of class-side methods that defined the
>> signatures of methods that comprised the server.  It was a lovely clean-up.
>> One could define the COM signature for a method in the method itself, and
>> the class side lost about three separate methods that defined all that
>> metadata.  One could read the server method itself and understand its
>> semantics without having to consult the class-side methods.  One didn't have
>> to know that there was metadata hidden on the class side because it was
>> right there in your face.
>>
>> Then Vassili used it for his cool inspector framework, Trippy, which was
>> similar to Glamour in some ways, and was a huge improvement over the old
>> Inspector framework, again resulting in a much more pluggable, decoupled and
>> extensible system.  Vassili also added the abstractions for accessing
>> pragmas in methods.
>>
>> Then we added checking so that one could restrict the compiler to accept
>> only legal pragmas for a given class.  But if we defined the legal pragmas
>> in a class-side method, say legalPragmas, then this would be exactly the
>> kind of single point for extensions that causes collisions between packages,
>> each of which might want to add its own set of pragmas.  The solution... use
>> a pragma to mark a class-side method as defining a set of legal pragmas for
>> a class.  One could have more than one method defining a set of legal
>> pragmas; packages wishing to add their own cool pragmas were decoupled.
>> Once the system because recursive, it had to be a good idea ;-).
>
>
> Ok, I start to see where the abstraction wasn't working so well... since
> pragmas are not executed, when writing a method you can't know if the pragma
> is correct, because even executing the method may not trigger the pragma
> induced code. So you need the legalPragmas to give metadata on metadata for
> the compiler to do a bit of static checking, but it doesn't work for
> system-wide pragmas unless you extend Object :(
>
> And often it doesn't matter if the pragma reference a completely non
> existent method or api, since it is probably never executed by anybody (and
> if it is, it won't probably reify the error message properly as a
> compilation error as it should, because it may be triggered miles away from
> the system browser).

Pragmas don't execute. They're _data_. There is no "calls unknown
sender" because the don't execute.

Eliot's point is that pragmas _describe_, and then other systems act
on those descriptions.

They're just like Java or C# attributes, or Python decorators. Only
they're better, because Java/C# attributes can do anything, whereas
pragmas merely describe.

frank

>> There are other uses; you've seen them.  I used them in VMMaker to
>> eliminate metadata that was embedded as sends to methods defined as ^self
>> that Slang had to extract and analyse, and filter-out from generated code.
>> They simplified Slang's code anaylsis, made the simulator more efficient
>> (since there were no longer sends to execute).  My point is that in all the
>> cases I've seen, using pragmas has
>> - simplified the code
>> - made it obvious that methods have metadata associated with them
>> - replaced specialized ways of associating metadata with code by the
>> general pragma mechanism
>> and in many of the cases it has
>> - provided a more decoupled system
>> - provided a more dynamic and extensible system
>
>
> Yes, and I can point out some of its shortcomings: it's non-obvious, it's
> limited, tools, even that many years later don't support them well (in
> Squeak or Pharo, at least), its redundant in quite a few variants.
>
> Please, could we improve a bit? Methods belonging to multiple protocols
> would give us the same decoupling as pragmas, and I would be free to avoid
> them where I shouldn't have to use them :)
>
>>
>>
>> I've been meaning to write up the history of pragmas for ages, but
>> Vassili, Steve or I have always been too busy.  I think a community paper on
>> their use and history would be worth-while, and might go a long way to
>> reduce antipathies like yours.  I will forever be in debt to anyone who
>> wants to volunteer to help me write such a paper.
>
>
> That would certainly be interesting :)
>
>>
>>
>>>
>>>
>>> Thierry
>>>
>>> (Look. I started using Smalltalk in 1992... and up to your description, I
>>> wasn't aware pragmas were supposed to follow message syntax ;) Thanks for
>>> the explanation, by the way)
>>
>>
>> They /have/ to follow literal message syntax.  t's all the compiler will
>> accept.  That's why there needs to be a paper.
>
>
> Yes!
>
> Thanks for taking the time to argument,
>
> Thierry
>
>>
>>
>>
>> --
>> best,
>> Eliot

On Fri, Feb 6, 2015 at 1:58 AM, Thierry Goubier <[hidden email]> wrote:

Hi Eliot,

maybe I'll frame the core question a bit differently, thanks to your explanations.

I see pragmas used mainly for two things:

1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified.

There's a third use that is, I think, the most powerful.  Maybe you see this as the same as 1, but I don't.  The method is a component to be included in some larger structure, e.g. it is an action method on a menu, or it is an implementation of a pane in an inspector.  The pragma is the message to be sent to the object that manipulates that larger structure to add the method to it.  This is how menu pragmas work in VW.  There is an object called a MenuBuilder.  To add a method to a menu (and which menu is described by the pragma) the menu builder sets the method as its current method and then performs the pragma.  The parameters in the pragma allow the MenuBuilder to add the method in the right way to the menu.  But the execution of the pragma is what actually adds the mehtod to the menu.  So its a combination of specification and execution.

2- categorizing things in a decoupled, package-extension compatible way.

I still think there is something in the design that bother me. 1- and 2- are convention-based, so as is most of Smalltalk anyway, so diregarding another approach to 2- as convention-based is a bit dishonest: pragmas get more authority because they look like they have been integrated in the language definition compared to competing solutions.

Yes, it's right to say they have been integrated (but no documented).

No, this is just because I start to wonder if methods shouldn't have a bit more in that "define, add and manipulate" metadata idea, i.e. make it more explicit and start to have tools which represent methods as more like objects and not only text items (methods having metadata, additional slots or instance variables, counters, statistics, ast, traces, break points, etc...). And, in a way, a need to have two levels of execution: one which happens at compile time (i.e. when one accept a method), one which happens at runtime.

I think whether methods are objects or text is in the eye of the beholder.  From my perspective methods /are/ objects.  The textual display and compilation of them is only the conventional programmer view of them.  Bu from a system perspective methods are objects.  Look for example at all the debugger machinery.  That is largely concerned with bytecodes in method objects.

For that, I think pragmas are a start, but they are not the complete answer. And I think we should solve the package-extension compatible thing in 2- to not have 2- (which is just for me a plain categorisation issue) polluting the design by introducing additional/unrelated requirements.

I have no problem with your criticism of pragmas used for categorisation.  Method categories are already there and sometimes much more appropriate than pragmas.  For example, in the SPur bootstrap the methods to be added or modified in Spur are included in categories of SpurBootstrapPrototypes and its subclasses.  If there were pragmas in these methods then the pragmas would be copied into Spur, which is not at all what I want.  Instead I copy across the methods in the relevant categories.

But this "not a complete solution" feels to me an unfair criticism.  Pragmas are what they are and they can do a lot.  But no part of the system does everything.  A system is composed of components.

2015-02-05 22:47 GMT+01:00 Eliot Miranda <[hidden email]>:

Hi Thierry,

A block is far more difficult to deal with than a pragma.  A pragma has a simple key, its selector, and simple access to its arameters, the arguments.  A block is opaque.  Essentially it can only be executable.  Pragmas are both executable and function as annotations.

Pragmas are executable, but in a context which is not visible at the place they are written.

 

 

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

 

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods.  Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

Yes, but there end up being lots of naming conventions and they are non-obvious.  Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions.  They just need documenting ;-).

They are still conventions which needs documentation. Pragmas are, in allmost all uses, indirections (i.e. a tag meaning something to a far away object), and the meaning associated to a pragma is, in every case, a roll-your-own solution. And they are clearly "in your face" :)

 

That's not necessarily true.  Some pragmas do cause processing at compile time.  For example, an FFI signature pragma can be checked at compile-time.  But it's in keeping with smalltalk that type checking is not performed at compile time in most cases, isn't it?  Why should one require that pragmas be semantically checked at compile time when normal SMalltalk code isn't?  At least one knows that the message instance the pragma is compiled to is a valid object and *can* be performed.  So pone does know at least that the pragma is executable.  Obviously whether that ability to be executed only becomes potent with the right receiver.  So any compiled pragma as the potential to be usefully evaluated.

Ok, but its all fuzzy.

It may be executed, but it may be not.

It may send an error if incorrect, but it may fail silently.

It may be executed at compile time, it may be executed sometime later triggered by the compilation, or it may be executed at runtime, as if it was normal code. And of course it may not be executed at all ...

You see what I mean?

 

And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There is lots about Smalltalk that is non-obvious, about programming in general that is non-obvious.  I don't see that as a specific criticism of pragmas.  Once one knows the idiom it is easy to use; its wide-spread use is evidence of that.  And hopefully this conversation will help make it more obvious :-).  That it is limited is also not perhaps a useful criticism.  The issue is whether it is adequate.  I think it is.  A literal message goes a long way.  I'e not heard of complaints about limitations so far.  Do you have specific examples where pragmas are inadequate?  The tools /do/ support them.  One can do senders and implementors in the browser and see the methods that include them, and the implementors of pragmas.  browseAllSelect: will narrow down the search.  There is an API for programmatic use (pragmasDo: et al).

See [*][**] for the tools support. For me, pragmas, even in their definition, have serious semantic issues. If they were strictly, as you describe them, a metadata API access executed at compile time, then they would be a lot stronger.

Make them sent at compile-time on the method object, and that would probably solve all the true arguments I have against them :)

for example: <worldMenu> against <tagAndAnnounce: #worldMenu>; <tag: #gtInspectorExtension> instead of <gtInspectorExtension>, etc...

(<tagAndAnnounce:> would gain a bit of performance, as well).

Thierry

[*] Searching a bit for menu pragmas in a Pharo image seems to return a bit of a mess. There is so many of them, all differents, and interestingly, this does not return the code triggered (or using) them, which makes discovery of what the pragma is used for (and its parameters) an 'interesting' problem.

I suggest a task then: an illustrated sequence of how to search for the code triggering world menu addition of a new command, for someone which is new to Pharo ;)

[**] even when used for categorisation, the API is prone to bugs, such as having a method without pragma overriding a superclass method with pragma because they have the same name.

--

best,

Eliot