Magritte 3 Gemstone port

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Magritte 3 Gemstone port

Nick
Hi,

Magritte 3 is now available for Gemstone - with tests green. The configuration has been updated and I loaded Magritte 3 with:

Gofer it
squeaksource: 'MetacelloRepository';
package: 'ConfigurationOfMagritte3';
load.

MCPlatformSupport commitOnAlmostOutOfMemoryDuring: [
ConfigurationOfMagritte3 project stableVersion load: #('Seaside Tests') ].


However the tests run extremely slowly - in the test suite I've measured 20x slowdown between Magritte 2 and Magritte 3 - making the tests a pain to execute.

I've run a series of tests to characterise where the problem might be. The tests are as follows:

-----
1) Test-suite benchmark:
| testSuite |
testSuite := (OBClassCategoryNode on: 'Magritte-Tests-Model-Description') classes
inject: TestSuite new
into: [:suite :classNode | 
(classNode theClass inheritsFrom: TestCase)
ifTrue: [suite addTests: classNode theClass suite tests].
suite].
Time millisecondsToRun:  [ testSuite run ].

2) Description benchmark:


Magritte 2:

Time millisecondsToRun: 

  [100 timesRepeat: [ MABooleanDescription  description ] ].  


Magritte 3:

Time millisecondsToRun: 
  [100 timesRepeat: [ MABooleanDescription new magritteDescription ] ].

3) Micro-benchmark

Magritte 2:
Time millisecondsToRun: 
  [100 timesRepeat: [ MABooleanDescription class allSelectors select: [ :each | each isDescriptionSelector ] ] ].

Magritte 3:
Time millisecondsToRun: 
  [100 timesRepeat: [ Pragma allNamed: MAPragmaBuilder magritteDescriptionPragma from: MABooleanDescription to: Object ] ].

* The micro-benchmark is illustrative of the difference between Magritte 2 and Magritte 3 - Magritte 2 uses a naming convention to retrieve descriptions, whereas Magritte 3 uses Pragmas.

-----

Results:

Pharo Magritte 2: (1) 436 (2) 0 (3) 85
Pharo Magritte 3: (1) 743 (2) 58 (3) 12
Gemstone Magritte 2: (1) 988 (2) 30 (3) 2229
Gemstone Magritte 3: (1) 21663 (2) 1575 (3) 444

Cavet: Although Gemstone and Pharo are running on the same machine - Gemstone is running virtualised in a VMWare Linux installation, so the results are not directly comparable, although the 30x performance decrease between Pharo and Gemstone surprised me.

First the good news: 
For Magritte the pragma description look-up is more performant than the named-based lookup (micro-benchmark (3)).

Now the bad news: 
In Pharo Magritte 2 is faster then Magritte 3 when running a description lookup based test-suite containing 1416 tests. My feeling is that in the context of a real application performing a smaller number of lookups this is probably not significant.
However in Gemstone the performance degradation is more noticeable and if nothing else makes running the tests painful.

How to proceed:
Magritte 2's performance advantage is gained from caching the descriptions. However I think the benefits of having dynamic (non-caching) descriptions outweigh the disadvantages and I'd be reluctant to add a generic layer of caching. In Pharo I don't think this is an issue. However in Gemstone I feel that these numbers need some further investigation. 

Any thoughts how to proceed?

Nick