0.39 version
Locked
 
|
Hi guys,
I try to fix the CI jobs of SciSmalltalk for Pharo 5.0 with the spur VM. The situation was strange, because the CI was green for dev version and yellow for stable version: https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/1050/ Dev version was more stable than the stable one ... Another problem, when I load the last version dev in pharo 5.0, there was some group loop problem related to Daliot code ... Sorry Daliot, I remove dependencies to your code. In order to solve these problems, I build a new baseline 0.27 and a new version 0.39. I decide to remove also code for Squeak and old Pharo 3.0, because we have no users. Now last version of SciSmalltalk 0.39 loads in Pharo 4.0 and Pharo 5,0, but we a failing test for: PoissonGeneratorTest-testSampleAverageConvergesToLambda.fuel Any idea why ? Regards, -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
The random failing test is this one:
PoissonGeneratorTest>>testSampleAverageConvergesToLambda "law of large numbers" | poisson samples | samples := OrderedCollection new. poisson := PoissonGenerator lambda: ((1000 atRandom: Random new) -1). 1000 timesRepeat: [samples add: poisson next]. self should: [samples average >= (poisson lambda * 0.8)]. self should: [samples average <= (poisson lambda * 1.2)]. I don't understand the intention of this test. Sometimes the first comparison fails. Werner ? Right now, I modify 1000 to 10000 in order to have less random failings. Regards, On Wed, Jan 13, 2016 at 12:09 PM, Serge Stinckwich <[hidden email]> wrote: > Hi guys, > > I try to fix the CI jobs of SciSmalltalk for Pharo 5.0 with the spur VM. > The situation was strange, because the CI was green for dev version > and yellow for stable version: > https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/1050/ > Dev version was more stable than the stable one ... > > Another problem, when I load the last version dev in pharo 5.0, there > was some group loop problem related to Daliot code ... Sorry Daliot, I > remove dependencies to your code. > > In order to solve these problems, I build a new baseline 0.27 and a > new version 0.39. > I decide to remove also code for Squeak and old Pharo 3.0, because we > have no users. > > Now last version of SciSmalltalk 0.39 loads in Pharo 4.0 and Pharo > 5,0, but we a failing test for: > PoissonGeneratorTest-testSampleAverageConvergesToLambda.fuel > > Any idea why ? > > Regards, > -- > Serge Stinckwich > UCBN & UMI UMMISCO 209 (IRD/UPMC) > Every DSL ends up being Smalltalk > http://www.doesnotunderstand.org/ -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
I skip the test for the moment, because even with 10000 samples number
sometimes it fails ... Werner maybe you can solve this issue. Now we have green tests for Pharo 4.0 and 5.0 (spur VM) ! https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/1056/ On Wed, Jan 13, 2016 at 12:33 PM, Serge Stinckwich <[hidden email]> wrote: > The random failing test is this one: > > > PoissonGeneratorTest>>testSampleAverageConvergesToLambda > > "law of large numbers" > | poisson samples | > samples := OrderedCollection new. > poisson := PoissonGenerator lambda: ((1000 atRandom: Random new) -1). > 1000 timesRepeat: [samples add: poisson next]. > self should: [samples average >= (poisson lambda * 0.8)]. > self should: [samples average <= (poisson lambda * 1.2)]. > > > I don't understand the intention of this test. Sometimes the first > comparison fails. > Werner ? > > Right now, I modify 1000 to 10000 in order to have less random failings. > > Regards, > > On Wed, Jan 13, 2016 at 12:09 PM, Serge Stinckwich > <[hidden email]> wrote: >> Hi guys, >> >> I try to fix the CI jobs of SciSmalltalk for Pharo 5.0 with the spur VM. >> The situation was strange, because the CI was green for dev version >> and yellow for stable version: >> https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/1050/ >> Dev version was more stable than the stable one ... >> >> Another problem, when I load the last version dev in pharo 5.0, there >> was some group loop problem related to Daliot code ... Sorry Daliot, I >> remove dependencies to your code. >> >> In order to solve these problems, I build a new baseline 0.27 and a >> new version 0.39. >> I decide to remove also code for Squeak and old Pharo 3.0, because we >> have no users. >> >> Now last version of SciSmalltalk 0.39 loads in Pharo 4.0 and Pharo >> 5,0, but we a failing test for: >> PoissonGeneratorTest-testSampleAverageConvergesToLambda.fuel >> >> Any idea why ? >> >> Regards, >> -- >> Serge Stinckwich >> UCBN & UMI UMMISCO 209 (IRD/UPMC) >> Every DSL ends up being Smalltalk >> http://www.doesnotunderstand.org/ > > > > -- > Serge Stinckwich > UCBN & UMI UMMISCO 209 (IRD/UPMC) > Every DSL ends up being Smalltalk > http://www.doesnotunderstand.org/ -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by SergeStinckwich
I try to fix the CI jobs of SciSmalltalk for Pharo 5.0 with the spur VM. Hi Serge, i agree <g>, it looked somewhat strange, i just thought i give Nicolas a week or so time to correct my change before i make that change official by making a new stable version and you were faster. I decide to remove also code for Squeak and old Pharo 3.0, because we fwiw i occasionally do use 3.0, but then so what, i have it on my computer, hence its no problem.
it has the same intention as the GaussianGeneratorTest that also occasionally fails, it samples poisson random numbers and tests whether the average falls in the correct interval. i know <g> that you dont like these random-tests. i often make those random tests (but i didnt make that one) because they already made me find some bugs, but i understand that they in a way go contrary to a strict test philosophy. of course one could make them non-random by setting the seed, eg: testSampleAverageConvergesToLambda "law of large numbers" | poisson samples | samples := OrderedCollection new. poisson := PoissonGenerator lambda: 42. "changed" poisson generator seed:41. "changed" 1000 timesRepeat: [samples add: poisson next]. self should: [samples average >= (poisson lambda * 0.8)]. self should: [samples average <= (poisson lambda * 1.2)]. this example makes me think, that perhaps it would make sense to implement a NumberGenerator>>seed: anInteger ^generator seed: anInteger or? a more complicated, more general solution would be to have a classvar 'trueRandomTest' somewhere, for example in 'NumberGenerator', or whereever, with accessors trueRandomTest ^trueRandomTest ifNil:[trueRandomTest:=false] trueRandomTest: aBoolean ^trueRandomTest:= aBoolean and then using eg NumberGenerator trueRandomTest ifFalse:[poisson generator seed:41] in the above example or so. this way one could also do random-tests simply by setting trueRandomTest to true, but by default the test would be non-random. disadvantage would obviously be that random-tests would look a bit more complicated and an additional dependency on Random would be introduced on some of my tests if i make them initially non-random. werner You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by SergeStinckwich
I don't understand the intention of this test. Sometimes the first perhaps (??) i understand your problem of understanding the test, this test is not really efficient, and in so far wrong (in a way). the same problem is in the test of the standard deviation in the GaussianGeneratorTest. if you want to test if a random number follows a certain distribution by checking whether the mean is in a certain interval, you would choose that interval in such a way that the mean falls in that interval for example in 95% of the cases and then the test would fail in 5%. you would then choose the smallest interval possible for the test to be as strong as possible. with assymmetric distributions the solution is not really obvious. i mean, in this test the mean cant be lower than 0, the distribution of the mean is bounded on one side (same is true for the distribution of the standard deviation in the other test). this test constructs an interval symmetric around the mean. now suppose you do that, but make that interval really big so that its lower side becomes negative. you could then move the lower side up to 0 and thus make the interval smaller without the test failing more often. in this extreme case the problem is obvious, but it does not really go away in the less extreme cases. and the solution is not really simple, at least in so far as the programming of the test would then get much longer & more complicated. first you'd need to know the distribution of the mean of a poisson distribution (and the distribution of the sd of a normal distribution in the other test). then you'd choose one side of the interval, calc the value of this second distribution at this point and choose the second side as the inverse of that value on the other side of the mode, or something like that, im not sure, have not really thought it through (Hi Nicolas, want to chime in please?). in any case, i guess you understand <big grin> why i never wanted (and still dont want) to touch these two tests. werner You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by SergeStinckwich
Hi. Did you execute benchmarks to compare SciSmalltalk in Cog vs Spur? 2016-01-13 12:09 GMT+01:00 Serge Stinckwich <[hidden email]>: Hi guys, You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
No, but if someone on the mailing-list want to do that, this could be great !
On Thu, Jan 14, 2016 at 1:53 PM, Denis Kudriashov <[hidden email]> wrote: > Hi. > > Did you execute benchmarks to compare SciSmalltalk in Cog vs Spur? > > 2016-01-13 12:09 GMT+01:00 Serge Stinckwich <[hidden email]>: >> >> Hi guys, >> >> I try to fix the CI jobs of SciSmalltalk for Pharo 5.0 with the spur VM. >> The situation was strange, because the CI was green for dev version >> and yellow for stable version: >> https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/1050/ >> Dev version was more stable than the stable one ... >> >> Another problem, when I load the last version dev in pharo 5.0, there >> was some group loop problem related to Daliot code ... Sorry Daliot, I >> remove dependencies to your code. >> >> In order to solve these problems, I build a new baseline 0.27 and a >> new version 0.39. >> I decide to remove also code for Squeak and old Pharo 3.0, because we >> have no users. >> >> Now last version of SciSmalltalk 0.39 loads in Pharo 4.0 and Pharo >> 5,0, but we a failing test for: >> PoissonGeneratorTest-testSampleAverageConvergesToLambda.fuel >> >> Any idea why ? >> >> Regards, >> -- >> Serge Stinckwich >> UCBN & UMI UMMISCO 209 (IRD/UPMC) >> Every DSL ends up being Smalltalk >> http://www.doesnotunderstand.org/ >> >> -- >> You received this message because you are subscribed to the Google Groups >> "SciSmalltalk" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [hidden email]. >> For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "SciSmalltalk" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [hidden email]. > For more options, visit https://groups.google.com/d/optout. -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by werner kassens-2
On Wed, Jan 13, 2016 at 11:51 PM, werner kassens <[hidden email]> wrote:
>> I try to fix the CI jobs of SciSmalltalk for Pharo 5.0 with the spur VM. >> The situation was strange, because the CI was green for dev version >> and yellow for stable version: >> https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/1050/ >> Dev version was more stable than the stable one ... > > > Hi Serge, > i agree <g>, it looked somewhat strange, i just thought i give Nicolas a > week or so time to correct my change before i make that change official by > making a new stable version and you were faster. > >> >> I decide to remove also code for Squeak and old Pharo 3.0, because we >> have no users. > > > fwiw i occasionally do use 3.0, but then so what, i have it on my computer, > hence its no problem. Ok, sorry about that. In fact, I only remove the Pharo 3.0 configuration parts. The code should still run ;-) -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by werner kassens-2
On Thu, Jan 14, 2016 at 12:44 PM, werner kassens <[hidden email]> wrote:
>> I don't understand the intention of this test. Sometimes the first >> comparison fails. >> Werner ? > > > Hi Serge, Hi Werner, > perhaps (??) i understand your problem of understanding the test, this test > is not really efficient, and in so far wrong (in a way). the same problem is > in the test of the standard deviation in the GaussianGeneratorTest. > > if you want to test if a random number follows a certain distribution by > checking whether the mean is in a certain interval, you would choose that > interval in such a way that the mean falls in that interval for example in > 95% of the cases and then the test would fail in 5%. you would then choose > the smallest interval possible for the test to be as strong as possible. > with assymmetric distributions the solution is not really obvious. i mean, > in this test the mean cant be lower than 0, the distribution of the mean is > bounded on one side (same is true for the distribution of the standard > deviation in the other test). this test constructs an interval symmetric > around the mean. now suppose you do that, but make that interval really big > so that its lower side becomes negative. you could then move the lower side > up to 0 and thus make the interval smaller without the test failing more > often. in this extreme case the problem is obvious, but it does not really > go away in the less extreme cases. and the solution is not really simple, at > least in so far as the programming of the test would then get much longer & > more complicated. > > first you'd need to know the distribution of the mean of a poisson > distribution (and the distribution of the sd of a normal distribution in the > other test). then you'd choose one side of the interval, calc the value of > this second distribution at this point and choose the second side as the > inverse of that value on the other side of the mode, or something like that, > im not sure, have not really thought it through (Hi Nicolas, want to chime > in please?). A test should be easy to read and understand because a test is also part of the documentation > in any case, i guess you understand <big grin> why i never wanted (and still > dont want) to touch these two tests. So what is the decision ? remove these tests ? Regards, -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by Denis Kudriashov
Hi Denis, i once did compare cog and spur by comparing the speed of the tests in jenkins: https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/PHARO=50,VERSION=stable,VM=vm/lastCompletedBuild/testReport/ and https://ci.inria.fr/pharo-contribution/job/SciSmalltalk/PHARO=40,VERSION=stable,VM=vm/lastCompletedBuild/testReport/ over different builds at different days the speed was very similar, hence it suffices to look at one build (like eg the one mentioned above). my conclusion was, that the new vm is at least 20% faster. the KDTree test is always slower in the new vm but that is misleading as further tests for example with the KDTreeBenchmark showed that KDTree is also at least 20% faster werner On Thu, Jan 14, 2016 at 1:53 PM, Denis Kudriashov <[hidden email]> wrote:
You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
In reply to this post by SergeStinckwich
>So what is the decision ? remove these tests ? Hi Serge, <g> of course not. i think their intention to test whether the mean an sd of a sample are, what one would expect, is easy to understand and makes sense to me. i made two proposals how to deal with that problem. you know, it can occasionally be a little bit difficult for me to decide what to do, if i try to make different proposals and dont get a response or a response like "So what is the decision ? remove these tests ?". if its ok with you i will make testSampleAverageConvergesToLambda nonrandom by fixing the seed. i will then watch jenkins and if he complains again i will then make GaussianGeneratorTest or whatever other random tests fail also nonrandom by fixing the seed. agreed? werner On Fri, Jan 15, 2016 at 8:57 AM, Serge Stinckwich <[hidden email]> wrote: On Thu, Jan 14, 2016 at 12:44 PM, werner kassens <[hidden email]> wrote: You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
On Friday, January 15, 2016 at 2:42:57 PM UTC+1, werner kassens wrote:
--
Hi Serge, that question remained also unanswered. i guess, that's my fault and the reason is probably that my email-style is slightly incompatible with scismalltalk. if i wanted to contribute to scismalltalk in a meaningful way i occasionaly needed to ask somebody here how something should work and - i can't help it - i write like i talk, and i won't change my writing style. and i notice i said relatively much here. i think i should keep my mouth shut in this forum in the future and simply update the code i uploaded if jenkins complains. then scismalltalk, which is a nice project, will certainly flourish better and faster. sorry for any inconvenience i produced. werner p.s. and git is a nightmare (you see <big grin> how difficult it is for me not to say something stupid, even if i try my best) You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
On Mon, Jan 18, 2016 at 6:18 PM, werner kassens <[hidden email]> wrote:
> On Friday, January 15, 2016 at 2:42:57 PM UTC+1, werner kassens wrote: >> >> if its ok with you i will make testSampleAverageConvergesToLambda >> nonrandom by fixing the seed. i will then watch jenkins and if he complains >> again i will then make GaussianGeneratorTest or whatever other random tests >> fail also nonrandom by fixing the seed. agreed? > > > Hi Serge, > that question remained also unanswered. i guess, that's my fault and the > reason is probably that my email-style is slightly incompatible with > scismalltalk. if i wanted to contribute to scismalltalk in a meaningful way > i occasionaly needed to ask somebody here how something should work and - i > can't help it - i write like i talk, and i won't change my writing style. > and i notice i said relatively much here. i think i should keep my mouth > shut in this forum in the future and simply update the code i uploaded if > jenkins complains. then scismalltalk, which is a nice project, will > certainly flourish better and faster. > sorry for any inconvenience i produced. > werner There is nothing wrong with your emails. Maybe try to be more concrete and write shorter emails so this easier to reply to them. You can use also the github issue tracker if you want to save issues: https://github.com/SergeStinckwich/SciSmalltalk/issues Regarding your message, you can try to fix the seed as you say. Thank you. Regards, -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/ -- You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
|
|
On Mon, Jan 18, 2016 at 7:36 PM, Serge Stinckwich <[hidden email]> wrote: Maybe try to be more concrete and write shorter emails so this easier ok, i'll try that. you can try to fix the seed as you say. done. You received this message because you are subscribed to the Google Groups "SciSmalltalk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email]. For more options, visit https://groups.google.com/d/optout. |
«
Return to SciSmalltalk
|
1 view|%1 views
| Free forum by Nabble | Edit this page |