[vwnc] ST vs. java 6 server
Locked
 
|
According to this link:
<A href="mhtml:{EA86BB10-4F84-474B-8200-BD830FA080BF}mid://00000506/!x-usc:http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=vw&lang2=java">http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=vw&lang2=java it seems like ST is loosing ground to Java 6. However, this note gives a suggestion to improve the competitiveness of ST: "Remember how HotSpot works. It starts by running your program with an interpreter. When it discovers that some method is "hot" -- that is, executed a lot, either because it is called a lot or because it contains loops that loop a lot -- it sends that method off to be compiled. After that one of two things will happen, either the next time the method is called the compiled version will be invoked (instead of the interpreted version) or the currently long running loop will be replaced, while still running, with the compiled method. The latter is known as "on stack replacement" and exists in the 1.3/1.4 HotSpot based systems." Any comments? Victor _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
And VW was doing that before Java was conceived of!
Gemstone also does it for the server side -- I don't know about VA and Squeak -- but seems to me there is something there. Victor wrote:
-- Dennis Smith +1 416.798.7948 Cherniak Software Development Corporation Fax: +1 416.798.0948 509-2001 Sheppard Avenue East [hidden email] Toronto, ON M2J 4Z8 <a class="moz-txt-link-freetext" href="sip:dennis@CherniakSoftware.com">sip:dennis@... Canada http://www.CherniakSoftware.com Entrance off Yorkland Blvd south of Sheppard Ave east of the DVP _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Victor-67
> And
VW was doing that before Java was conceived of!
I am glad this is the case.
BTW. I didn't check the meaning or validity of the
benchmarks. Only saw the results as published in the link.
How to explain the speed an memory footprint advantage
of Java 6 over ST, as published (between 5 and 15 times for speed, in most
cases)?
Victor
===========================================
_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
In reply to this post by Dennis smith-4
> And VW
was doing that before Java was conceived of!
I am glad this is the case.
BTW. I didn't check the meaning or validity
of the benchmarks. Only saw the results as published in the link.
How to explain the speed an memory footprint
advantage of Java 6 over ST, as published (between 5 and 15 times for
speed, in most cases)?
Victor
===========================================
_______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
In reply to this post by Dennis smith-4
Just a comment on k-nucleotide... from 7.5 to 7.6, I
measured a roughly 5x improvement in execution speed without using the
dnaSequenceHash method. I think it would be interesting to see what
happens when one lets the VM do the (much better) hashing
instead.
Andres. From: [hidden email] [mailto:[hidden email]] On Behalf Of Dennis Smith Sent: Wednesday, September 17, 2008 4:33 PM To: VWNC, Subject: Re: [vwnc] ST vs. java 6 server Gemstone also does it for the server side -- I don't know about VA and Squeak -- but seems to me there is something there. Victor wrote:
-- Dennis Smith +1 416.798.7948 Cherniak Software Development Corporation Fax: +1 416.798.0948 509-2001 Sheppard Avenue East [hidden email] Toronto, ON M2J 4Z8 <A class=moz-txt-link-freetext href="sip:dennis@CherniakSoftware.com">sip:dennis@... Canada http://www.CherniakSoftware.com Entrance off Yorkland Blvd south of Sheppard Ave east of the DVP _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
--- "Valloud, Andres" <[hidden email]> wrote: > Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a > roughly > 5x improvement in execution speed without using the dnaSequenceHash > method. I think it would be interesting to see what happens when one > lets the VM do the (much better) hashing instead. > > Andres. Contribute a better program by following the FAQ instructions: http://shootout.alioth.debian.org/u32q/faq.php#play _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
I just got an idea out of this. It will take a bit longer, but I think
the results will be significantly better. Stay tuned... the ETA is Q2-Q3/2009. Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Isaac Gouy Sent: Thursday, September 18, 2008 10:20 AM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server --- "Valloud, Andres" <[hidden email]> wrote: > Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a > roughly 5x improvement in execution speed without using the > dnaSequenceHash method. I think it would be interesting to see what > happens when one lets the VM do the (much better) hashing instead. > > Andres. Contribute a better program by following the FAQ instructions: http://shootout.alioth.debian.org/u32q/faq.php#play _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
1. Is the benchmark based on VW 7.6 or 7.5?
2. Can someone verify the results at Cincom or for Cincom (for Andres). This is a very dramatic benchmark. Andres, I saw your interesting talk In Reno, I wonder if these tests are impacted in part by some underlying tuning issue or something "simple" like the hashing algorithm kind of meta-problem. Sometimes it just takes an expert to do some profiling like you did to expose a profound performance issue. Can you comment on why we might have performance problems for each kind of benchmark mentioned? It might be useful for us to know if it a bad ST code, a simple vm fix or a hard vm fix or even to classify the difference (like they are exploiting MT in Java and therefore leaning on more of the underlying hardware). That way the community doesn't have to wait up to a year to save our behinds because we might even be able to help. Conor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Thursday, September 18, 2008 12:21 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server I just got an idea out of this. It will take a bit longer, but I think the results will be significantly better. Stay tuned... the ETA is Q2-Q3/2009. Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Isaac Gouy Sent: Thursday, September 18, 2008 10:20 AM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server --- "Valloud, Andres" <[hidden email]> wrote: > Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a > roughly 5x improvement in execution speed without using the > dnaSequenceHash method. I think it would be interesting to see what > happens when one lets the VM do the (much better) hashing instead. > > Andres. Contribute a better program by following the FAQ instructions: http://shootout.alioth.debian.org/u32q/faq.php#play _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Andres Valloud-6
--- "Valloud, Andres" <[hidden email]> wrote: > I just got an idea out of this. It will take a bit longer, but I > think > the results will be significantly better. Stay tuned... the ETA is > Q2-Q3/2009. What was it the OP asked - something about ST loosing ground to Java 6? _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
In reply to this post by Conor Shankey
Conor,
I do remember comparing the old VW 7.5 hashing vs the new VW 7.6 hashing mechanisms, and that alone sped up k-nucleotide by ~5x with the same Smalltalk code. I do not remember using dnaSequenceHash in either test, so I think it was just comparing String>>hash between one and the other (plus the new symbol table which can be considerably faster as well). For k-nucleotide, the performance issues were as follows (IIRC): * Symbol table changes. The old symbol table could send String>>hash up to three times to obtain the same results. The new symbol table only sends hash once. Also, the new symbol tables can be further tuned for different performance scenarios. Together with the new hash functions, they were designed with heavy use in mind (e.g.: a goal to efficiently handle in excess of 250k symbols in an image). To give an idea of how much more efficient the new symbol tables are, consider that calculation of string hashes without primitives is comparatively expensive to implement in plain Smalltalk as is. And yet, using the primitive failure code, the new symbol table was faster than the old one with the previous hash method. * String>>hash problems. String>>hash was a sampling implementation that did not consider all characters. This leads to collisions and slower sets/dictionaries. Also, it could run into large integer arithmetic. The new hash primitives hash every character and because they run in a primitive they are significantly faster. Also, the resulting hash value distribution is far better and this leads to faster hashed collection performance due to less collisions. Finally, the hash primitive returns small integers only. With regards to the rest of the problems, I'd have to take a look at them to be able to tell. Of course, having a faster object engine always helps, but without a close examination it would be hard for me to say to what extent the Smalltalk implementation of the solution could be changed to take advantage of what we already have (which is quite a bit). Something I thought I'd point out is that perhaps we could do them one by one here in the mailing list. Perhaps this makes it easier to get the process going, and also it would spread the knowledge of how to make faster Smalltalk programs in public. So... do we have a bundle in the public store repository with the solutions for each of the problems? If we do not have such a thing, perhaps what we could do first it to make a bundle like this: ProgrammingLanguageShootout [bundle] ProblemA [bundle] SolutionA1 [package] SolutionA2 [package] ProblemB [bundle] SolutionB1 [package] SolutionB2 [package] SolutionB3 [package] ... etc... Then we can all contribute and contribute the result of all our work. Andres. Conor Shankey wrote: > 1. Is the benchmark based on VW 7.6 or 7.5? > 2. Can someone verify the results at Cincom or for Cincom (for Andres). This > is a very dramatic benchmark. > > Andres, I saw your interesting talk In Reno, I wonder if these tests are > impacted in part by some underlying tuning issue or something "simple" like > the hashing algorithm kind of meta-problem. Sometimes it just takes an > expert to do some profiling like you did to expose a profound performance > issue. Can you comment on why we might have performance problems for each > kind of benchmark mentioned? It might be useful for us to know if it a bad > ST code, a simple vm fix or a hard vm fix or even to classify the difference > (like they are exploiting MT in Java and therefore leaning on more of the > underlying hardware). That way the community doesn't have to wait up to a > year to save our behinds because we might even be able to help. > > Conor > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On Behalf > Of Valloud, Andres > Sent: Thursday, September 18, 2008 12:21 PM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > I just got an idea out of this. It will take a bit longer, but I think > the results will be significantly better. Stay tuned... the ETA is > Q2-Q3/2009. > > Andres. > > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Isaac Gouy > Sent: Thursday, September 18, 2008 10:20 AM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > > --- "Valloud, Andres" <[hidden email]> wrote: > > >> Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a >> roughly 5x improvement in execution speed without using the >> dnaSequenceHash method. I think it would be interesting to see what >> happens when one lets the VM do the (much better) hashing instead. >> >> Andres. >> > > > Contribute a better program by following the FAQ instructions: > > http://shootout.alioth.debian.org/u32q/faq.php#play > > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
Cool,
Thanks for responding so quickly and in such a detailed way. Perhaps we can have these results updated soon so that we show how competitive VW/ST is. Conor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud Sent: Thursday, September 18, 2008 2:50 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server Conor, I do remember comparing the old VW 7.5 hashing vs the new VW 7.6 hashing mechanisms, and that alone sped up k-nucleotide by ~5x with the same Smalltalk code. I do not remember using dnaSequenceHash in either test, so I think it was just comparing String>>hash between one and the other (plus the new symbol table which can be considerably faster as well). For k-nucleotide, the performance issues were as follows (IIRC): * Symbol table changes. The old symbol table could send String>>hash up to three times to obtain the same results. The new symbol table only sends hash once. Also, the new symbol tables can be further tuned for different performance scenarios. Together with the new hash functions, they were designed with heavy use in mind (e.g.: a goal to efficiently handle in excess of 250k symbols in an image). To give an idea of how much more efficient the new symbol tables are, consider that calculation of string hashes without primitives is comparatively expensive to implement in plain Smalltalk as is. And yet, using the primitive failure code, the new symbol table was faster than the old one with the previous hash method. * String>>hash problems. String>>hash was a sampling implementation that did not consider all characters. This leads to collisions and slower sets/dictionaries. Also, it could run into large integer arithmetic. The new hash primitives hash every character and because they run in a primitive they are significantly faster. Also, the resulting hash value distribution is far better and this leads to faster hashed collection performance due to less collisions. Finally, the hash primitive returns small integers only. With regards to the rest of the problems, I'd have to take a look at them to be able to tell. Of course, having a faster object engine always helps, but without a close examination it would be hard for me to say to what extent the Smalltalk implementation of the solution could be changed to take advantage of what we already have (which is quite a bit). Something I thought I'd point out is that perhaps we could do them one by one here in the mailing list. Perhaps this makes it easier to get the process going, and also it would spread the knowledge of how to make faster Smalltalk programs in public. So... do we have a bundle in the public store repository with the solutions for each of the problems? If we do not have such a thing, perhaps what we could do first it to make a bundle like this: ProgrammingLanguageShootout [bundle] ProblemA [bundle] SolutionA1 [package] SolutionA2 [package] ProblemB [bundle] SolutionB1 [package] SolutionB2 [package] SolutionB3 [package] ... etc... Then we can all contribute and contribute the result of all our work. Andres. Conor Shankey wrote: > 1. Is the benchmark based on VW 7.6 or 7.5? > 2. Can someone verify the results at Cincom or for Cincom (for Andres). This > is a very dramatic benchmark. > > Andres, I saw your interesting talk In Reno, I wonder if these tests are > impacted in part by some underlying tuning issue or something "simple" like > the hashing algorithm kind of meta-problem. Sometimes it just takes an > expert to do some profiling like you did to expose a profound performance > issue. Can you comment on why we might have performance problems for each > kind of benchmark mentioned? It might be useful for us to know if it a bad > ST code, a simple vm fix or a hard vm fix or even to classify the difference > (like they are exploiting MT in Java and therefore leaning on more of the > underlying hardware). That way the community doesn't have to wait up to a > year to save our behinds because we might even be able to help. > > Conor > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On Behalf > Of Valloud, Andres > Sent: Thursday, September 18, 2008 12:21 PM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > I just got an idea out of this. It will take a bit longer, but I think > the results will be significantly better. Stay tuned... the ETA is > Q2-Q3/2009. > > Andres. > > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Isaac Gouy > Sent: Thursday, September 18, 2008 10:20 AM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > > --- "Valloud, Andres" <[hidden email]> wrote: > > >> Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a >> roughly 5x improvement in execution speed without using the >> dnaSequenceHash method. I think it would be interesting to see what >> happens when one lets the VM do the (much better) hashing instead. >> >> Andres. >> > > > Contribute a better program by following the FAQ instructions: > > http://shootout.alioth.debian.org/u32q/faq.php#play > > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
|
|
In reply to this post by Andres Valloud-3
--- Andres Valloud <[hidden email]> wrote: > Conor, > > I do remember comparing the old VW 7.5 hashing vs the new VW 7.6 > hashing > mechanisms, and that alone sped up k-nucleotide by ~5x with the same > Smalltalk code. I do not remember using dnaSequenceHash in either > test, > so I think it was just comparing String>>hash between one and the > other > (plus the new symbol table which can be considerably faster as well). > > For k-nucleotide, the performance issues were as follows (IIRC): > > * Symbol table changes. > > The old symbol table could send String>>hash up to three times to > obtain > the same results. The new symbol table only sends hash once. Also, > the > new symbol tables can be further tuned for different performance > scenarios. Together with the new hash functions, they were designed > with heavy use in mind (e.g.: a goal to efficiently handle in excess > of > 250k symbols in an image). > > To give an idea of how much more efficient the new symbol tables are, > > consider that calculation of string hashes without primitives is > comparatively expensive to implement in plain Smalltalk as is. And > yet, > using the primitive failure code, the new symbol table was faster > than > the old one with the previous hash method. > > * String>>hash problems. > > String>>hash was a sampling implementation that did not consider all > characters. This leads to collisions and slower sets/dictionaries. > Also, it could run into large integer arithmetic. > > The new hash primitives hash every character and because they run in > a > primitive they are significantly faster. Also, the resulting hash > value > distribution is far better and this leads to faster hashed collection > > performance due to less collisions. Finally, the hash primitive > returns > small integers only. > > With regards to the rest of the problems, I'd have to take a look at > them to be able to tell. Of course, having a faster object engine > always helps, but without a close examination it would be hard for me > to > say to what extent the Smalltalk implementation of the solution could > be > changed to take advantage of what we already have (which is quite a > bit). > > Something I thought I'd point out is that perhaps we could do them > one > by one here in the mailing list. Perhaps this makes it easier to get > > the process going, and also it would spread the knowledge of how to > make > faster Smalltalk programs in public. So... do we have a bundle in > the > public store repository with the solutions for each of the problems? > If > we do not have such a thing, perhaps what we could do first it to > make a > bundle like this: > > ProgrammingLanguageShootout [bundle] > ProblemA [bundle] > SolutionA1 [package] > SolutionA2 [package] > ProblemB [bundle] > SolutionB1 [package] > SolutionB2 [package] > SolutionB3 [package] > ... etc... > > Then we can all contribute and contribute the result of all our work. > > Andres. Incidentally, the reason the n-body and spectral-norm programs which work just fine on 32 bit Ubuntu are failing on 64 bit Ubuntu is that 100.0d0 asFixedPoint: 5 causes a Segmentation fault. Using vw7.6nc/bin/linuxx86_64/vwlinuxx86_64 and vw7.6nc/preview/64-bit/visual64.im _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
Isaac,
I think the issue relates to SmallDouble>>fractionPart causing the crash. In any case, the segfault has been fixed a while ago in internal VW7.7 builds: visual visual64.im -headless -evaluate "100.0d0 asFixedPoint: 5" VisualWorks(R) 7.7 beta2 Aug 29 2008 [ 64-bit thapi ephemerons immutability ] 100.00000s Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Isaac Gouy Sent: Thursday, September 18, 2008 5:21 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server --- Andres Valloud <[hidden email]> wrote: > Conor, > > I do remember comparing the old VW 7.5 hashing vs the new VW 7.6 > hashing mechanisms, and that alone sped up k-nucleotide by ~5x with > the same Smalltalk code. I do not remember using dnaSequenceHash in > either test, so I think it was just comparing String>>hash between one > and the other (plus the new symbol table which can be considerably > faster as well). > > For k-nucleotide, the performance issues were as follows (IIRC): > > * Symbol table changes. > > The old symbol table could send String>>hash up to three times to > obtain the same results. The new symbol table only sends hash once. > Also, the new symbol tables can be further tuned for different > performance scenarios. Together with the new hash functions, they > were designed with heavy use in mind (e.g.: a goal to efficiently > handle in excess of 250k symbols in an image). > > To give an idea of how much more efficient the new symbol tables are, > > consider that calculation of string hashes without primitives is > comparatively expensive to implement in plain Smalltalk as is. And > yet, using the primitive failure code, the new symbol table was faster > than the old one with the previous hash method. > > * String>>hash problems. > > String>>hash was a sampling implementation that did not consider all > characters. This leads to collisions and slower sets/dictionaries. > Also, it could run into large integer arithmetic. > > The new hash primitives hash every character and because they run in a > primitive they are significantly faster. Also, the resulting hash > value distribution is far better and this leads to faster hashed > collection > > performance due to less collisions. Finally, the hash primitive > returns small integers only. > > With regards to the rest of the problems, I'd have to take a look at > them to be able to tell. Of course, having a faster object engine > always helps, but without a close examination it would be hard for me > to say to what extent the Smalltalk implementation of the solution > could be changed to take advantage of what we already have (which is > quite a bit). > > Something I thought I'd point out is that perhaps we could do them one > by one here in the mailing list. Perhaps this makes it easier to get > > the process going, and also it would spread the knowledge of how to > make faster Smalltalk programs in public. So... do we have a bundle > in the public store repository with the solutions for each of the > problems? > If > we do not have such a thing, perhaps what we could do first it to make > a bundle like this: > > ProgrammingLanguageShootout [bundle] > ProblemA [bundle] > SolutionA1 [package] > SolutionA2 [package] > ProblemB [bundle] > SolutionB1 [package] > SolutionB2 [package] > SolutionB3 [package] > ... etc... > > Then we can all contribute and contribute the result of all our work. > > Andres. Incidentally, the reason the n-body and spectral-norm programs which work just fine on 32 bit Ubuntu are failing on 64 bit Ubuntu is that 100.0d0 asFixedPoint: 5 causes a Segmentation fault. Using vw7.6nc/bin/linuxx86_64/vwlinuxx86_64 and vw7.6nc/preview/64-bit/visual64.im _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
In reply to this post by Conor Shankey
I got Eliot's k-nucleotide code into a 7.6 image, and then compared it
with using 7.6's hashing facilities. Just by using the new hashing methods, exactly the same code runs 2x faster. I am not doing something right though, and the tests I made fail to produce the right answer as expected by the shootout people. I just published the code to the public repository. Perhaps somebody can figure out what's wrong? The bundle is called ComputerLanguageShootout. Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Conor Shankey Sent: Thursday, September 18, 2008 5:16 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server Cool, Thanks for responding so quickly and in such a detailed way. Perhaps we can have these results updated soon so that we show how competitive VW/ST is. Conor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud Sent: Thursday, September 18, 2008 2:50 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server Conor, I do remember comparing the old VW 7.5 hashing vs the new VW 7.6 hashing mechanisms, and that alone sped up k-nucleotide by ~5x with the same Smalltalk code. I do not remember using dnaSequenceHash in either test, so I think it was just comparing String>>hash between one and the other (plus the new symbol table which can be considerably faster as well). For k-nucleotide, the performance issues were as follows (IIRC): * Symbol table changes. The old symbol table could send String>>hash up to three times to obtain the same results. The new symbol table only sends hash once. Also, the new symbol tables can be further tuned for different performance scenarios. Together with the new hash functions, they were designed with heavy use in mind (e.g.: a goal to efficiently handle in excess of 250k symbols in an image). To give an idea of how much more efficient the new symbol tables are, consider that calculation of string hashes without primitives is comparatively expensive to implement in plain Smalltalk as is. And yet, using the primitive failure code, the new symbol table was faster than the old one with the previous hash method. * String>>hash problems. String>>hash was a sampling implementation that did not consider all characters. This leads to collisions and slower sets/dictionaries. Also, it could run into large integer arithmetic. The new hash primitives hash every character and because they run in a primitive they are significantly faster. Also, the resulting hash value distribution is far better and this leads to faster hashed collection performance due to less collisions. Finally, the hash primitive returns small integers only. With regards to the rest of the problems, I'd have to take a look at them to be able to tell. Of course, having a faster object engine always helps, but without a close examination it would be hard for me to say to what extent the Smalltalk implementation of the solution could be changed to take advantage of what we already have (which is quite a bit). Something I thought I'd point out is that perhaps we could do them one by one here in the mailing list. Perhaps this makes it easier to get the process going, and also it would spread the knowledge of how to make faster Smalltalk programs in public. So... do we have a bundle in the public store repository with the solutions for each of the problems? If we do not have such a thing, perhaps what we could do first it to make a bundle like this: ProgrammingLanguageShootout [bundle] ProblemA [bundle] SolutionA1 [package] SolutionA2 [package] ProblemB [bundle] SolutionB1 [package] SolutionB2 [package] SolutionB3 [package] ... etc... Then we can all contribute and contribute the result of all our work. Andres. Conor Shankey wrote: > 1. Is the benchmark based on VW 7.6 or 7.5? > 2. Can someone verify the results at Cincom or for Cincom (for Andres). This > is a very dramatic benchmark. > > Andres, I saw your interesting talk In Reno, I wonder if these tests > are impacted in part by some underlying tuning issue or something "simple" like > the hashing algorithm kind of meta-problem. Sometimes it just takes an > expert to do some profiling like you did to expose a profound > performance issue. Can you comment on why we might have performance > problems for each kind of benchmark mentioned? It might be useful for > us to know if it a bad ST code, a simple vm fix or a hard vm fix or > even to classify the difference > (like they are exploiting MT in Java and therefore leaning on more of > the underlying hardware). That way the community doesn't have to wait > up to a year to save our behinds because we might even be able to help. > > Conor > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Valloud, Andres > Sent: Thursday, September 18, 2008 12:21 PM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > I just got an idea out of this. It will take a bit longer, but I > think the results will be significantly better. Stay tuned... the ETA > is Q2-Q3/2009. > > Andres. > > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Isaac Gouy > Sent: Thursday, September 18, 2008 10:20 AM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > > --- "Valloud, Andres" <[hidden email]> wrote: > > >> Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a >> roughly 5x improvement in execution speed without using the >> dnaSequenceHash method. I think it would be interesting to see what >> happens when one lets the VM do the (much better) hashing instead. >> >> Andres. >> > > > Contribute a better program by following the FAQ instructions: > > http://shootout.alioth.debian.org/u32q/faq.php#play > > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
Re: [vwnc] ST vs. java 6 server
|
|
I just fixed the tests for k-nucleotide and republished
ComputerLanguageShootout. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Valloud, Andres Sent: Thursday, September 18, 2008 6:21 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server I got Eliot's k-nucleotide code into a 7.6 image, and then compared it with using 7.6's hashing facilities. Just by using the new hashing methods, exactly the same code runs 2x faster. I am not doing something right though, and the tests I made fail to produce the right answer as expected by the shootout people. I just published the code to the public repository. Perhaps somebody can figure out what's wrong? The bundle is called ComputerLanguageShootout. Andres. -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Conor Shankey Sent: Thursday, September 18, 2008 5:16 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server Cool, Thanks for responding so quickly and in such a detailed way. Perhaps we can have these results updated soon so that we show how competitive VW/ST is. Conor -----Original Message----- From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud Sent: Thursday, September 18, 2008 2:50 PM To: [hidden email] Subject: Re: [vwnc] ST vs. java 6 server Conor, I do remember comparing the old VW 7.5 hashing vs the new VW 7.6 hashing mechanisms, and that alone sped up k-nucleotide by ~5x with the same Smalltalk code. I do not remember using dnaSequenceHash in either test, so I think it was just comparing String>>hash between one and the other (plus the new symbol table which can be considerably faster as well). For k-nucleotide, the performance issues were as follows (IIRC): * Symbol table changes. The old symbol table could send String>>hash up to three times to obtain the same results. The new symbol table only sends hash once. Also, the new symbol tables can be further tuned for different performance scenarios. Together with the new hash functions, they were designed with heavy use in mind (e.g.: a goal to efficiently handle in excess of 250k symbols in an image). To give an idea of how much more efficient the new symbol tables are, consider that calculation of string hashes without primitives is comparatively expensive to implement in plain Smalltalk as is. And yet, using the primitive failure code, the new symbol table was faster than the old one with the previous hash method. * String>>hash problems. String>>hash was a sampling implementation that did not consider all characters. This leads to collisions and slower sets/dictionaries. Also, it could run into large integer arithmetic. The new hash primitives hash every character and because they run in a primitive they are significantly faster. Also, the resulting hash value distribution is far better and this leads to faster hashed collection performance due to less collisions. Finally, the hash primitive returns small integers only. With regards to the rest of the problems, I'd have to take a look at them to be able to tell. Of course, having a faster object engine always helps, but without a close examination it would be hard for me to say to what extent the Smalltalk implementation of the solution could be changed to take advantage of what we already have (which is quite a bit). Something I thought I'd point out is that perhaps we could do them one by one here in the mailing list. Perhaps this makes it easier to get the process going, and also it would spread the knowledge of how to make faster Smalltalk programs in public. So... do we have a bundle in the public store repository with the solutions for each of the problems? If we do not have such a thing, perhaps what we could do first it to make a bundle like this: ProgrammingLanguageShootout [bundle] ProblemA [bundle] SolutionA1 [package] SolutionA2 [package] ProblemB [bundle] SolutionB1 [package] SolutionB2 [package] SolutionB3 [package] ... etc... Then we can all contribute and contribute the result of all our work. Andres. Conor Shankey wrote: > 1. Is the benchmark based on VW 7.6 or 7.5? > 2. Can someone verify the results at Cincom or for Cincom (for Andres). This > is a very dramatic benchmark. > > Andres, I saw your interesting talk In Reno, I wonder if these tests > are impacted in part by some underlying tuning issue or something "simple" like > the hashing algorithm kind of meta-problem. Sometimes it just takes an > expert to do some profiling like you did to expose a profound > performance issue. Can you comment on why we might have performance > problems for each kind of benchmark mentioned? It might be useful for > us to know if it a bad ST code, a simple vm fix or a hard vm fix or > even to classify the difference > (like they are exploiting MT in Java and therefore leaning on more of > the underlying hardware). That way the community doesn't have to wait > up to a year to save our behinds because we might even be able to help. > > Conor > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Valloud, Andres > Sent: Thursday, September 18, 2008 12:21 PM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > I just got an idea out of this. It will take a bit longer, but I > think the results will be significantly better. Stay tuned... the ETA > is Q2-Q3/2009. > > Andres. > > > -----Original Message----- > From: [hidden email] [mailto:[hidden email]] On > Behalf Of Isaac Gouy > Sent: Thursday, September 18, 2008 10:20 AM > To: [hidden email] > Subject: Re: [vwnc] ST vs. java 6 server > > > --- "Valloud, Andres" <[hidden email]> wrote: > > >> Just a comment on k-nucleotide... from 7.5 to 7.6, I measured a >> roughly 5x improvement in execution speed without using the >> dnaSequenceHash method. I think it would be interesting to see what >> happens when one lets the VM do the (much better) hashing instead. >> >> Andres. >> > > > Contribute a better program by following the FAQ instructions: > > http://shootout.alioth.debian.org/u32q/faq.php#play > > > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ > vwnc mailing list > [hidden email] > http://lists.cs.uiuc.edu/mailman/listinfo/vwnc > > _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc _______________________________________________ vwnc mailing list [hidden email] http://lists.cs.uiuc.edu/mailman/listinfo/vwnc |
«
Return to VisualWorks
|
1 view|%1 views
| Free forum by Nabble | Edit this page |