Poor class creation performance, horrible deletion performance

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hi Richard,

This may be a stupid question but allthough I can understant the requirement for 10000 instances of a class I don't see why you want to define 10000 different Classes within a system.

What is your case ?

 

regards,

 

@+Maarten,

 

 

> "Richard Sargent" <[hidden email]> |

I ran a test recently using VW 7.9.1 as the client. [The test was to determine how quickly GemBuilder for Smalltalk can connect client and server classes.]

The test creates a number of classes on the client, then creates the same classes on the server, and times the login process. It repeats with successive larger numbers of classes. It starts with 99 classes, then 199, 299, 999, 1999, 2999, 9999, 19999, and finally 29,999.

This excerpt is from the last batch of 10,000 classes. As you can see, it took close to 15 minutes to create 10,000 classes in the client. Less than 2 seconds to do the same on the GemStone server. Admittedly, having no user iinterface does lighten the burden involved.

14-05-23 18:26:40:838   Creating 10000 client classes
14-05-23 18:41:28:260   Creating 10000 server classes
14-05-23 18:41:29:991

At the end of the test, it deletes all the client classes it created. An hour and ten minutes!?!

14-05-23 18:41:35:086   Deleting 29,999 client classes
14-05-23 19:51:57:939   Done

Is there any known and supported way to create or delete a batch of classes without this extraordinary overhead?

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Hi Richard,

This may be a stupid question but allthough I can understant the requirement for 10000 instances of a class I don't see why you want to define 10000 different Classes within a system.

What is your case ?

 

regards,

 

@+Maarten,

 

 

> "Richard Sargent" <[hidden email]> |

I ran a test recently using VW 7.9.1 as the client. [The test was to determine how quickly GemBuilder for Smalltalk can connect client and server classes.]

The test creates a number of classes on the client, then creates the same classes on the server, and times the login process. It repeats with successive larger numbers of classes. It starts with 99 classes, then 199, 299, 999, 1999, 2999, 9999, 19999, and finally 29,999.

This excerpt is from the last batch of 10,000 classes. As you can see, it took close to 15 minutes to create 10,000 classes in the client. Less than 2 seconds to do the same on the GemStone server. Admittedly, having no user iinterface does lighten the burden involved.

14-05-23 18:26:40:838   Creating 10000 client classes
14-05-23 18:41:28:260   Creating 10000 server classes
14-05-23 18:41:29:991

At the end of the test, it deletes all the client classes it created. An hour and ten minutes!?!

14-05-23 18:41:35:086   Deleting 29,999 client classes
14-05-23 19:51:57:939   Done

Is there any known and supported way to create or delete a batch of classes without this extraordinary overhead?

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Richard Sargent wrote Monday, May 26, 2014 6:30 AM:
> I ran a test recently using VW 7.9.1 as the client.
> ...it took close to 15 minutes to create 10,000 classes
> Less than 2 seconds to do the same on the GemStone server.
> Is there any known and supported way to create or delete
> a batch of classes without this extraordinary overhead?

We have similar challenges. In our case the classes are all just subclasses
in a certain tree, they are instantiable, add ivs and have values for class
instance variables defined in the superclass, but they add no shareds or
civs, have no methods, and no code refers to them. The first steps we found:
avoid recompiling the system each time you create a class, and avoid writing
the class to the .cha changes file:

SystemUtils modifySystem:
   [ChangeSet ignoreChangesWhile:
      [ "create classes" ]]

for the .cha silencing you'll also need to implement #definition and
#xmlDefinition on the class side of the superclass of your classes:

   ^(#{ChangeSet.BroadcastChanges} valueOrDo: [true])
      ifTrue:  [super definition]
      ifFalse: ['']

Even if you can remove those as we can, there's still a surprisingly large
cost these days to creating a class. Do post back if you find some more
optimizations!

All the best,
Steve

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

Did you try wrapping it in
SystemUtils modifySystem:["create new classes"].
1000 class creations took 0.71 wrapped, or 53 seconds without.

TimeProfiler is your friend :)

That really did help. Class creation is now sufficiently fast. 10,000
classes in 6 seconds or so. The class removal still takes a long time, less
but way too long. From an hour and a quarter, it's down to 54 minutes to
remove 30,000 classes.

-----Original Message-----
From: Steven Kelly [mailto:[hidden email]]
Sent: May 26, 2014 00:28
To: Richard Sargent; [hidden email]
Subject: RE: [vwnc] Poor class creation performance,horrible deletion
performance

Richard Sargent wrote Monday, May 26, 2014 6:30 AM:
> I ran a test recently using VW 7.9.1 as the client.
> ...it took close to 15 minutes to create 10,000 classes Less than 2
> seconds to do the same on the GemStone server.
> Is there any known and supported way to create or delete a batch of
> classes without this extraordinary overhead?

We have similar challenges. In our case the classes are all just subclasses
in a certain tree, they are instantiable, add ivs and have values for class
instance variables defined in the superclass, but they add no shareds or
civs, have no methods, and no code refers to them. The first steps we found:
avoid recompiling the system each time you create a class, and avoid writing
the class to the .cha changes file:

SystemUtils modifySystem:
   [ChangeSet ignoreChangesWhile:
      [ "create classes" ]]

for the .cha silencing you'll also need to implement #definition and
#xmlDefinition on the class side of the superclass of your classes:

   ^(#{ChangeSet.BroadcastChanges} valueOrDo: [true])
      ifTrue:  [super definition]
      ifFalse: ['']

Even if you can remove those as we can, there's still a surprisingly large
cost these days to creating a class. Do post back if you find some more
optimizations!

All the best,
Steve

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

We modified Parcel>>cleanseParcelSystem to defer cleaning until all classes
were removed.

I think you will find that will speed up class removal.

Terry

===========================================================
Terry Raymond
Crafted Smalltalk
80 Lazywood Ln.
Tiverton, RI  02878
(401) 624-4517      [hidden email]
===========================================================


> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]] On
> Behalf Of Richard Sargent
> Sent: Wednesday, May 28, 2014 11:49 AM
> To: 'Steven Kelly'; [hidden email]
> Subject: Re: [vwnc] Poor class creation performance, horrible deletion
> performance
>
> That really did help. Class creation is now sufficiently fast. 10,000
classes in 6
> seconds or so. The class removal still takes a long time, less but way too
long. subclasses in a
> certain tree, they are instantiable, add ivs and have values for class
instance
> variables defined in the superclass, but they add no shareds or civs, have
no
> methods, and no code refers to them. The first steps we found:
> avoid recompiling the system each time you create a class, and avoid
writing cost


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc