fast subclassing/class creation in tests

On 26 Oct 2015, at 13:14, Peter Uhnák <[hidden email]> wrote:

Thanks Nicolai, the suspension seems to have quite a significant performance impact.

Out of curiosity I did some benchmarking. Although the numbers seems to vary a lot based on something (the size of the image jumped from 25 to 40MB after I was done... and there seems to be some correlation with the performance). Repeatedly running a benchmark seems also detrimental as the performance can drop by over 30%... so probably it would be best to run each benchmark in a completely new image...

However the fact that the performance decreases over time is problematic, because that means that running the test suite will get worse over time.

I find it also peculiar to see that unlogged anonymous classes are faster when the announcer is NOT suspended.

And while for classes the improvement was ~40x, for methods it was over 200x faster... so I guess the best way is to modify performTest behavior to something like

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MyTestClass>>performTest

SystemAnnouncer uniqueInstance suspendAllWhile: [ super performTest ]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Benchmarks (I've removed decimal parts and thousand separators because they are always confusing me):

Class creation with announcer:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[ |cls|

cls := Object subclass: #Something.

cls removeFromSystem.

] bench. "'9 per second'"

[ |cls|

cls := Object subclass: #Something.

cls removeFromSystemUnlogged.

] bench. "'173 per second'"

[ |cls|

cls := Object newAnonymousSubclass.

cls removeFromSystem.

] bench. "'4 per second'"

[ |cls|

cls := Object newAnonymousSubclass.

cls removeFromSystemUnlogged.

] bench. "'512 per second'"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Class creation without announcer:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

[ SystemAnnouncer uniqueInstance suspendAllWhile: [ |cls|

cls := Object subclass: #Something.

cls removeFromSystem.

] ] bench. "'371 per second'"

[ SystemAnnouncer uniqueInstance suspendAllWhile: [ |cls|

cls := Object subclass: #Something.

cls removeFromSystemUnlogged.

] ] bench. "'351 per second'"

[ SystemAnnouncer uniqueInstance suspendAllWhile: [ |cls|

cls := Object newAnonymousSubclass.

cls removeFromSystem.

] ] bench. "'368 per second'"

[ SystemAnnouncer uniqueInstance suspendAllWhile: [ |cls|

cls := Object newAnonymousSubclass.

cls removeFromSystemUnlogged.

] ] bench. "'430 per second'"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Methods with announcer:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

cls := Object subclass: #Something.

[

cls compile: 'method ^ 1'.

] bench. "'9 per second'"

anon := Object newAnonymousSubclass.

[

anon compile: 'method ^ 1'.

] bench. "'9 per second'"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Methods without announcer

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

cls := Object subclass: #Something.

[ SystemAnnouncer uniqueInstance suspendAllWhile: [

cls compile: 'method ^ 1'.

] ] bench. "'1814 per second'"

anon := Object newAnonymousSubclass.

[ SystemAnnouncer uniqueInstance suspendAllWhile: [

anon compile: 'method ^ 1'.

] ] bench. "'2063 per second'"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Peter

On Sun, Oct 25, 2015 at 12:29 AM, Robert Withers <[hidden email]> wrote:

Extend Pharo byte codes to support Newspeak namespaces. #justsayin

---

robert

On Oct 24, 2015, at 6:22 PM, Nicolai Hess <[hidden email]> wrote:

You can wrap the subclassing and the \removeFromSystem with
SystemAnnouncer uniqueInstance  suspendAllWhile: [ ]

2015-10-24 20:43 GMT+02:00 Peter Uhnák <[hidden email]>:

Hi,

is it possible to do fast, non-system wide class subclassing in tests?

Currently I would do something like

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MyTest>>testSomething

    cls := SomeParent subclass: #Something.

    "... do some tests ..."

    cls removeFromSystem

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This is for example what ClassTest is doing.

However this is very slow, and even for small tests suite --- just ten (so far) test methods it takes over ten seconds to test it all, which is quite bad for TDD.

My guess is that since it has to write to class to the environment and disk and then remove it it takes a while:

~~~~~~~~~~~~~

[ Object subclass: #Something ] timeToRun. "0:00:00:00.021"

[ #Something asClass removeFromSystem ] timeToRun. "0:00:00:00.526"

~~~~~~~~~~~~~

is it possible to speed it up? Create classes that are fast to remove?

Thanks,

Peter

On 29 Oct 2015, at 19:14, Peter Uhnák <[hidden email]> wrote:

yes, I already felt that test runs involving unlogged code generation is very slow… can you add an entry on the issue tracker?

I can, but I am not sure for what. :)

a) speed up regular class operations (class creation, removal, compilation)

b) speed up unlogged class ops

c) modify code generation tests in Pharo (ClassTest, possibly others) to use unlogged/suspendAllWhile

d) all of the above (as separate issues)

plus bonus

e) create classes/methods without polluting the environment (.changes file) (because only removal is unlogged, not creation) (if this is even possible)

Peter