How to silently generate and remove code

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

How to silently generate and remove code

jfabry
Hi all,

for LRP I am auto generating classes and methods, and removing them when I no longer need them. This however produces a lot of entries in the change file that are not needed at all. So I am looking into generating and removing them silently, however with not much success. Can anybody help me out in how to do the following without having entries in the changes file?

- add a class (The standard subclass:instanceVariableNames:classVariableNames:category: message does not seem to have a ‘silent’ equivalent)
- remove a class ( aClass removeFromSystem: false.  still logs to changes, apparently)
- compile a method (aClass compileSilently: methodstring classified: #generated. still logs to changes, apparently)
- remove a method (removeSelector: does not seem to have a ‘silent’ option)
- add / remove class variables (again, no ‘silent’ option)
- run a RBCreateAccessorsForVariableRefactoring

Thanks in advance!

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Alejandro Infante
Hi,
As far as I know the source code is not stored in the image, but in the changes file. So from that perspective the changes files is more than just logs, it actually store our code, so not recording that entry would be similar as having a method without the actual code.

Maybe a possible solution would be a way to clean the change file, but I have no idea if something like that exist.

Cheers,
Alejandro

> On Mar 27, 2015, at 5:15 PM, Johan Fabry <[hidden email]> wrote:
>
> Hi all,
>
> for LRP I am auto generating classes and methods, and removing them when I no longer need them. This however produces a lot of entries in the change file that are not needed at all. So I am looking into generating and removing them silently, however with not much success. Can anybody help me out in how to do the following without having entries in the changes file?
>
> - add a class (The standard subclass:instanceVariableNames:classVariableNames:category: message does not seem to have a ‘silent’ equivalent)
> - remove a class ( aClass removeFromSystem: false.  still logs to changes, apparently)
> - compile a method (aClass compileSilently: methodstring classified: #generated. still logs to changes, apparently)
> - remove a method (removeSelector: does not seem to have a ‘silent’ option)
> - add / remove class variables (again, no ‘silent’ option)
> - run a RBCreateAccessorsForVariableRefactoring
>
> Thanks in advance!
>
> ---> Save our in-boxes! http://emailcharter.org <---
>
> Johan Fabry   -   http://pleiad.cl/~jfabry
> PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile
>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Marcus Denker-4
In reply to this post by jfabry
I think we have not yet any good API for this… one problem is that the .changes is not only a transaction log, but in addition
the place where the source of a method is stored.

I would like to

-> make transaction log + source storage independent.
-> add a kind of “virtual” package for these generated entities so that there is no package that gets dirty when generating code
    reflectively
-> have a good API so one can distinguishes “system notifications are send” from “change is logged” and “source is stored”.

> On 27 Mar 2015, at 21:15, Johan Fabry <[hidden email]> wrote:
>
> Hi all,
>
> for LRP I am auto generating classes and methods, and removing them when I no longer need them. This however produces a lot of entries in the change file that are not needed at all. So I am looking into generating and removing them silently, however with not much success. Can anybody help me out in how to do the following without having entries in the changes file?
>
> - add a class (The standard subclass:instanceVariableNames:classVariableNames:category: message does not seem to have a ‘silent’ equivalent)
> - remove a class ( aClass removeFromSystem: false.  still logs to changes, apparently)
> - compile a method (aClass compileSilently: methodstring classified: #generated. still logs to changes, apparently)
> - remove a method (removeSelector: does not seem to have a ‘silent’ option)
> - add / remove class variables (again, no ‘silent’ option)
> - run a RBCreateAccessorsForVariableRefactoring
>
> Thanks in advance!
>
> ---> Save our in-boxes! http://emailcharter.org <---
>
> Johan Fabry   -   http://pleiad.cl/~jfabry
> PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile
>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

stepharo


Le 28/3/15 08:53, Marcus Denker a écrit :
> I think we have not yet any good API for this… one problem is that the .changes is not only a transaction log, but in addition
> the place where the source of a method is stored.
>
> I would like to
>
> -> make transaction log + source storage independent.
> -> add a kind of “virtual” package for these generated entities so that there is no package that gets dirty when generating code
>      reflectively
> -> have a good API so one can distinguishes “system notifications are send” from “change is logged” and “source is stored”.

+ 1

>
>> On 27 Mar 2015, at 21:15, Johan Fabry <[hidden email]> wrote:
>>
>> Hi all,
>>
>> for LRP I am auto generating classes and methods, and removing them when I no longer need them. This however produces a lot of entries in the change file that are not needed at all. So I am looking into generating and removing them silently, however with not much success. Can anybody help me out in how to do the following without having entries in the changes file?
>>
>> - add a class (The standard subclass:instanceVariableNames:classVariableNames:category: message does not seem to have a ‘silent’ equivalent)
>> - remove a class ( aClass removeFromSystem: false.  still logs to changes, apparently)
>> - compile a method (aClass compileSilently: methodstring classified: #generated. still logs to changes, apparently)
>> - remove a method (removeSelector: does not seem to have a ‘silent’ option)
>> - add / remove class variables (again, no ‘silent’ option)
>> - run a RBCreateAccessorsForVariableRefactoring
>>
>> Thanks in advance!
>>
>> ---> Save our in-boxes! http://emailcharter.org <---
>>
>> Johan Fabry   -   http://pleiad.cl/~jfabry
>> PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Ben Coman
In reply to this post by jfabry
Not sure if it is silent enough for you, but perhaps have a look at use of ClassFactoryForTestCase.
cheers -ben

On Sat, Mar 28, 2015 at 4:15 AM, Johan Fabry <[hidden email]> wrote:
Hi all,

for LRP I am auto generating classes and methods, and removing them when I no longer need them. This however produces a lot of entries in the change file that are not needed at all. So I am looking into generating and removing them silently, however with not much success. Can anybody help me out in how to do the following without having entries in the changes file?

- add a class (The standard subclass:instanceVariableNames:classVariableNames:category: message does not seem to have a ‘silent’ equivalent)
- remove a class ( aClass removeFromSystem: false.  still logs to changes, apparently)
- compile a method (aClass compileSilently: methodstring classified: #generated. still logs to changes, apparently)
- remove a method (removeSelector: does not seem to have a ‘silent’ option)
- add / remove class variables (again, no ‘silent’ option)
- run a RBCreateAccessorsForVariableRefactoring

Thanks in advance!

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile



Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

jfabry
In reply to this post by stepharo

> On Mar 28, 2015, at 08:06, stepharo <[hidden email]> wrote:
>
>
>
> Le 28/3/15 08:53, Marcus Denker a écrit :
>> I think we have not yet any good API for this… one problem is that the .changes is not only a transaction log, but in addition
>> the place where the source of a method is stored.
>>
>> I would like to
>>
>> -> make transaction log + source storage independent.
>> -> add a kind of “virtual” package for these generated entities so that there is no package that gets dirty when generating code
>>     reflectively
>> -> have a good API so one can distinguishes “system notifications are send” from “change is logged” and “source is stored”.
>
> + 1

+1 for that also. Priority for Pharo 5 ?


---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

jfabry
In reply to this post by Ben Coman

Thanks Ben for the pointer! In the end it’s doing +- the same things I had already figured out don’t really work though. So I guess it’ll have to wait until thorough support is implemented.

On Mar 28, 2015, at 14:45, Ben Coman <[hidden email]> wrote:

Not sure if it is silent enough for you, but perhaps have a look at use of ClassFactoryForTestCase.
cheers -ben




---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile

Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Marcus Denker-4
In reply to this post by jfabry

> On 28 Mar 2015, at 20:05, Johan Fabry <[hidden email]> wrote:
>
>
>> On Mar 28, 2015, at 08:06, stepharo <[hidden email]> wrote:
>>
>>
>>
>> Le 28/3/15 08:53, Marcus Denker a écrit :
>>> I think we have not yet any good API for this… one problem is that the .changes is not only a transaction log, but in addition
>>> the place where the source of a method is stored.
>>>
>>> I would like to
>>>
>>> -> make transaction log + source storage independent.
>>> -> add a kind of “virtual” package for these generated entities so that there is no package that gets dirty when generating code
>>>    reflectively
>>> -> have a good API so one can distinguishes “system notifications are send” from “change is logged” and “source is stored”.
>>
>> + 1
>
> +1 for that also. Priority for Pharo 5 ?
>

Yes!

        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

jfabry

> On Mar 28, 2015, at 19:03, Marcus Denker <[hidden email]> wrote:
>
>
>> On 28 Mar 2015, at 20:05, Johan Fabry <[hidden email]> wrote:
>>
>>
>>> On Mar 28, 2015, at 08:06, stepharo <[hidden email]> wrote:
>>>
>>>
>>>
>>> Le 28/3/15 08:53, Marcus Denker a écrit :
>>>> I think we have not yet any good API for this… one problem is that the .changes is not only a transaction log, but in addition
>>>> the place where the source of a method is stored.
>>>>
>>>> I would like to
>>>>
>>>> -> make transaction log + source storage independent.
>>>> -> add a kind of “virtual” package for these generated entities so that there is no package that gets dirty when generating code
>>>>   reflectively
>>>> -> have a good API so one can distinguishes “system notifications are send” from “change is logged” and “source is stored”.
>>>
>>> + 1
>>
>> +1 for that also. Priority for Pharo 5 ?
>>
>
> Yes!

That sounds like an interesting challenge. Keep me up to date and tell me if I can help!

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

stepharo
In reply to this post by Marcus Denker-4
Before that I would like that the compiler outputs classes definition in
an environment (that can be the default one) but that can be the one
I want and specified from the outside of the compiler.
May be this is already the case


Stef

Le 28/3/15 23:03, Marcus Denker a écrit :

>> On 28 Mar 2015, at 20:05, Johan Fabry <[hidden email]> wrote:
>>
>>
>>> On Mar 28, 2015, at 08:06, stepharo <[hidden email]> wrote:
>>>
>>>
>>>
>>> Le 28/3/15 08:53, Marcus Denker a écrit :
>>>> I think we have not yet any good API for this… one problem is that the .changes is not only a transaction log, but in addition
>>>> the place where the source of a method is stored.
>>>>
>>>> I would like to
>>>>
>>>> -> make transaction log + source storage independent.
>>>> -> add a kind of “virtual” package for these generated entities so that there is no package that gets dirty when generating code
>>>>     reflectively
>>>> -> have a good API so one can distinguishes “system notifications are send” from “change is logged” and “source is stored”.
>>> + 1
>> +1 for that also. Priority for Pharo 5 ?
>>
> Yes!
>
> Marcus
>


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Marcus Denker-4

> On 29 Mar 2015, at 10:32, stepharo <[hidden email]> wrote:
>
> Before that I would like that the compiler outputs classes definition in an environment (that can be the default one) but that can be the one
> I want and specified from the outside of the compiler.

For installing, it is the ClassInstaller that does that (and yes, it has an environment).

What the compiler does not have is to look up references e.g. to symbols regarding to an environment, but that is because we
don’t have that concept right now in the system in general. globals/class vars are requested from the class (which defines its environment).

So in general, we first need a use case, else it is not clear what to do…


        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

demarey

Le 30 mars 2015 à 09:10, Marcus Denker a écrit :

>
>> On 29 Mar 2015, at 10:32, stepharo <[hidden email]> wrote:
>>
>> Before that I would like that the compiler outputs classes definition in an environment (that can be the default one) but that can be the one
>> I want and specified from the outside of the compiler.
>
> For installing, it is the ClassInstaller that does that (and yes, it has an environment).
>
> What the compiler does not have is to look up references e.g. to symbols regarding to an environment, but that is because we
> don’t have that concept right now in the system in general. globals/class vars are requested from the class (which defines its environment).
>
> So in general, we first need a use case, else it is not clear what to do…
A first use case could be to install new code in a sandbox, check if the loading is good, and then move the code to the default environment, else drop the sandbox.
It is also a part of the feature asked by Johan: install code in a sandbox for tests purposes and then drop it without perturbing the system.

smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

jfabry

> On Mar 30, 2015, at 09:23, Christophe Demarey <[hidden email]> wrote:
>
>
> Le 30 mars 2015 à 09:10, Marcus Denker a écrit :
>
>>
>>> On 29 Mar 2015, at 10:32, stepharo <[hidden email]> wrote:
>>>
>>> Before that I would like that the compiler outputs classes definition in an environment (that can be the default one) but that can be the one
>>> I want and specified from the outside of the compiler.
>>
>> For installing, it is the ClassInstaller that does that (and yes, it has an environment).
>>
>> What the compiler does not have is to look up references e.g. to symbols regarding to an environment, but that is because we
>> don’t have that concept right now in the system in general. globals/class vars are requested from the class (which defines its environment).
>>
>> So in general, we first need a use case, else it is not clear what to do…
>
> A first use case could be to install new code in a sandbox, check if the loading is good, and then move the code to the default environment, else drop the sandbox.
> It is also a part of the feature asked by Johan: install code in a sandbox for tests purposes and then drop it without perturbing the system.

To be clear: I want to be able to generate code (classes and methods in these classes), and remove them when needed without it being logged in the changes file.  This is because it may end up to be a lot of code that changes extremely frequently, so it will pollute the change file a lot. And for debugging, I want to be able to see the source for the methods, even though they are not in .changes. (I will never want to version this code nor do anything else ‘serious’ with it, it’s all generated stuff).

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

wernerk
In reply to this post by Marcus Denker-4
On 03/30/2015 09:10 AM, Marcus Denker wrote:
> What the compiler does not have is to look up references e.g. to symbols regarding to an environment, but that is because we
> don’t have that concept right now in the system in general. globals/class vars are requested from the class (which defines its environment).
>
> So in general, we first need a use case, else it is not clear what to do…

Hi Marcus,
i dont know whether this really applies to the things you are talking
about here, but it sounds a bit like it eventually could, hence i'll
mention it: i had a little program, that constructs an array of blocks
(no object methods) via opal from scratch (using strings). now this
worked of course without problems, but there are situations where it
would be nice if those blocks would have - in a _simple_ way - access to
common variables constructed by the program, iow if all those blocks
would live in the same 'space' as those variables - or however thats
called correctly. does this scenario apply as a possible use case
(obviously completely different from Johan's use case) ?
werner

Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Thierry Goubier
In reply to this post by Marcus Denker-4
A use case:

the ability to test and debug the code generation part of SmaCC.

SmaCC, when compiling a parser, generates dozens of classes and hundreds of methods, some of those very long, as a sequence of refactorings (and optimise them).

Then SmaCC apply them all in one step.

I'd dream of having a virtual environment where I could test-compile and check all those methods, via a system browser, before they are compiled.

Thierry

2015-03-30 9:10 GMT+02:00 Marcus Denker <[hidden email]>:

> On 29 Mar 2015, at 10:32, stepharo <[hidden email]> wrote:
>
> Before that I would like that the compiler outputs classes definition in an environment (that can be the default one) but that can be the one
> I want and specified from the outside of the compiler.

For installing, it is the ClassInstaller that does that (and yes, it has an environment).

What the compiler does not have is to look up references e.g. to symbols regarding to an environment, but that is because we
don’t have that concept right now in the system in general. globals/class vars are requested from the class (which defines its environment).

So in general, we first need a use case, else it is not clear what to do…


        Marcus

Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

jfabry
In reply to this post by wernerk

> On Mar 30, 2015, at 11:01, Werner Kassens <[hidden email]> wrote:
>
>  i had a little program, that constructs an array of blocks (no object methods) via opal from scratch (using strings). now this worked of course without problems, but there are situations where it would be nice if those blocks would have - in a _simple_ way - access to common variables constructed by the program, iow if all those blocks would live in the same 'space' as those variables - or however thats called correctly. does this scenario apply as a possible use case (obviously completely different from Johan's use case) ?

Werner, let me surprise you: what I am doing is in the end exactly that! :-)

I am compiling blocks, written by the user of the system, that need references to variables that are outside the block, and they are shared between the different blocks. (The blocks have been written in a plain text textfield). So how do I have them access the variables?

 Here is the setup:
- I generate a class (MyGeneratedClass) with as *class variables* all the variables that need to be accessed by the different blocks
- For each block I generate a string with method source code (myGeneratedMethod_timestamp), whose body is: return the block
- I compile the method and add it to MyGeneratedClass class (add it at class side)
- I execute each method (MyGeneratedClass myGeneratedMethod_timestamp) , which gives me a BlockContext that has captured the variables.
- sending value to that executes the block, and all blocks share the variables in MyGeneratedClass.

Simple :-)

And thanks to Marcus for discussing this with me a couple of years ago ...

---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

wernerk
Hi Johan,
<g> you did surprise me! thanks for that info, i can use that indeed.
werner

On 03/30/2015 04:47 PM, Johan Fabry wrote:
> Werner, let me surprise you: what I am doing is in the end exactly that! :-) ...

Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

Marcus Denker-4
In reply to this post by Thierry Goubier
Ok, I noted all the use cases. 

On 30 Mar 2015, at 16:25, Thierry Goubier <[hidden email]> wrote:

A use case:

the ability to test and debug the code generation part of SmaCC.

SmaCC, when compiling a parser, generates dozens of classes and hundreds of methods, some of those very long, as a sequence of refactorings (and optimise them).

Then SmaCC apply them all in one step.

I'd dream of having a virtual environment where I could test-compile and check all those methods, via a system browser, before they are compiled.

Thierry

2015-03-30 9:10 GMT+02:00 Marcus Denker <[hidden email]>:

> On 29 Mar 2015, at 10:32, stepharo <[hidden email]> wrote:
>
> Before that I would like that the compiler outputs classes definition in an environment (that can be the default one) but that can be the one
> I want and specified from the outside of the compiler.

For installing, it is the ClassInstaller that does that (and yes, it has an environment).

What the compiler does not have is to look up references e.g. to symbols regarding to an environment, but that is because we
don’t have that concept right now in the system in general. globals/class vars are requested from the class (which defines its environment).

So in general, we first need a use case, else it is not clear what to do…


        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

jfabry
In reply to this post by wernerk
Here is a class that can be of use for you then :-)
It’s a bit more complicated than what you need exactly, because I have lexical scoping and subscopes et cetera, but you can ignore that really. The relevant methods for you are generateNamedClass:withSuper: and compileBlock:forScope:

Success!





> On Mar 30, 2015, at 12:06, Werner Kassens <[hidden email]> wrote:
>
> Hi Johan,
> <g> you did surprise me! thanks for that info, i can use that indeed.
> werner
>
> On 03/30/2015 04:47 PM, Johan Fabry wrote:
>> Werner, let me surprise you: what I am doing is in the end exactly that! :-) ...
>
>


---> Save our in-boxes! http://emailcharter.org <---

Johan Fabry   -   http://pleiad.cl/~jfabry
PLEIAD lab  -  Computer Science Department (DCC)  -  University of Chile


LRPCodeGenerator.st (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to silently generate and remove code

wernerk
Hi Johan,
thanks, i see how it works and removes the object later.
werner

On 03/30/2015 06:42 PM, Johan Fabry wrote:
> Here is a class that can be of use for you then :-)

12