Benchmark

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Benchmark

Natalia Moskovchuk
Hello.
I want to use the benchmarks in my GSoC project. Can somebody write or
suggest me some resources about how could I run written benchmarks on CI
server and get there information?

Best regards,
Natalia

Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Sebastian Tleye
Hi Natalia, 

Here you have an interesting video

and a repository



2013/8/30 Natalia Moskovchuk <[hidden email]>
Hello.
I want to use the benchmarks in my GSoC project. Can somebody write or suggest me some resources about how could I run written benchmarks on CI server and get there information?

Best regards,
Natalia


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Natalia Moskovchuk
Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Benjamin AREZKI


2013/8/30 Natalia Moskovchuk [via Smalltalk] <[hidden email]>
Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Benchmark-tp4705781p4705818.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Smalltalk, click here.
NAML

Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Benjamin AREZKI
In reply to this post by Natalia Moskovchuk


2013/8/30 Benjamin AREZKI <[hidden email]>


2013/8/30 Natalia Moskovchuk [via Smalltalk] <[hidden email]>

Thanks. I already saw this video. I cann't see configuration on https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?



If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Benchmark-tp4705781p4705818.html
To start a new topic under Pharo Smalltalk Users, email [hidden email]
To unsubscribe from Smalltalk, click here.
NAML


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Camillo Bruni-3
In reply to this post by Natalia Moskovchuk
That you just need to run it from the command line which is not related to the benchmark project.
Currently it seems like the ci has some troubles, so I cannot get to the configuration either.


On 2013-08-30, at 17:25, Natalia Moskovchuk <[hidden email]> wrote:

> Thanks. I already saw this video. I cann't see configuration on
> https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?


signature.asc (457 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Clément Béra
However benchmark on the CI are not on a dedicated machine so it is not very reliable ....


2013/8/30 Camillo Bruni <[hidden email]>
That you just need to run it from the command line which is not related to the benchmark project.
Currently it seems like the ci has some troubles, so I cannot get to the configuration either.


On 2013-08-30, at 17:25, Natalia Moskovchuk <[hidden email]> wrote:

> Thanks. I already saw this video. I cann't see configuration on
> https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

EstebanLM
yes, it was just a proof of concept, waiting for their own dedicated machine(s)

On Aug 30, 2013, at 7:03 PM, Clément Bera <[hidden email]> wrote:

However benchmark on the CI are not on a dedicated machine so it is not very reliable ....


2013/8/30 Camillo Bruni <[hidden email]>
That you just need to run it from the command line which is not related to the benchmark project.
Currently it seems like the ci has some troubles, so I cannot get to the configuration either.


On 2013-08-30, at 17:25, Natalia Moskovchuk <[hidden email]> wrote:

> Thanks. I already saw this video. I cann't see configuration on
> https://ci.inria.fr/rmod/job/FullTextSearch/configure . Can you share it?



Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Natalia Moskovchuk
Thank all of you for help. I have successfuly added benchmarks to my project.
And additionaly I wrote simple step by step tutorial about how to write benchmarks in Pharo for people like me)).
Best regards,
Natalia
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Natalia Moskovchuk
Forgot to give the link(. http://nataliatymchuk.blogspot.com/2013/09/hello.html

10.09.13 02:15, Natalia Moskovchuk написав(ла):
Thank all of you for help. I have successfuly added benchmarks to my project.
And additionaly I wrote simple step by step tutorial about how to write
benchmarks in Pharo for people like me)).
Best regards,
Natalia



--
View this message in context: http://forum.world.st/Benchmark-tp4705781p4707388.html
Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Ben Coman
Natalia Moskovchuk wrote:

> Forgot to give the link(.
> http://nataliatymchuk.blogspot.com/2013/09/hello.html
>
> 10.09.13 02:15, Natalia Moskovchuk ???????(??):
>> Thank all of you for help. I have successfuly added benchmarks to my
>> project.
>> And additionaly I wrote simple step by step tutorial about how to write
>> benchmarks in Pharo for people like me)).
>> Best regards,
>> Natalia
>>
>>
>>
>> --
>> View this message in context:
>> http://forum.world.st/Benchmark-tp4705781p4707388.html
>> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>>
>
>
Thanks Natalia. That was interesting to learn.

btw, I followed the link to Camillo's tutorial, and from there to
benchmarkgame.com.  It was interesting interesting was VW Smalltalk
being faster than C for the thread-ring [2] benchmark.  I'm guessing
that bodes well for web programs handling many many requests each doing
a small task, rather than numerical processing like many of the
benchmarks.

[1]
http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=threadring&lang=all&data=u32
[2]
http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=threadring&lang=all&data=u32

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Stéphane Ducasse
In reply to this post by Natalia Moskovchuk
Hello Natalia

I started to write a chapter on Smark for a future Pharo book. So may be we could write it together :)
I can give you access to it if you are interested

Stef

On Sep 10, 2013, at 1:15 AM, Natalia Moskovchuk <[hidden email]> wrote:

> Thank all of you for help. I have successfuly added benchmarks to my project.
> And additionaly I wrote simple step by step tutorial about how to write
> benchmarks in Pharo for people like me)).
> Best regards,
> Natalia
>
>
>
> --
> View this message in context: http://forum.world.st/Benchmark-tp4705781p4707388.html
> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
>


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Natalia Moskovchuk
Hello Stephane.
Yes, I'm interested in that and it will be an honor for me)). But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .
Best regards,
Natalia 
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Stéphane Ducasse

On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <[hidden email]> wrote:

> Hello Stephane.
> Yes, I'm interested in that and it will be an honor for me)).

Excellent.
If you have a github account let me know and I will add you as a contributor
Have a look at

        https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tree/master/Drafts/

> But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .

No stress :)
My english sucks too and we should not care for now.

> Best regards,
> Natalia


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Natalia Tymchuk
Hello.
I have a question about problemSize in benchmarking. I got the small
times from running benchmarks and that's why I try to use problemSize
like in the video http://vimeo.com/68494202. But the benchmark for which
I rewrote code and used problemSize has given me very big result. Maybe
it looks like there is no division on that problemSize. Am I wrong? What
can I do?
Best regards,
Natalia

12.09.13 15:20, Stéphane Ducasse написав(ла):

> On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <[hidden email]> wrote:
>
>> Hello Stephane.
>> Yes, I'm interested in that and it will be an honor for me)).
> Excellent.
> If you have a github account let me know and I will add you as a contributor
> Have a look at
>
> https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tree/master/Drafts/
>
>> But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .
> No stress :)
> My english sucks too and we should not care for now.
>
>> Best regards,
>> Natalia
>


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Camillo Bruni-3
Short Answer:
-------------
problemSize is used to calibrate your benchmark, so usually you adapt this value for
your machine. And yes you are right, the result is not divided by the problemSize.


Long Answer:
------------
The typical use case is like this:

MathBench >> benchLoopSinus
        1 to: self problemSize to: [ :i | i sin ]

Now you have two parameters to modifiy:
1. the number of samples you take (aka, how many times you measure the time of #benchLoopSinus)
2. the problem size (aka, how many times you run #sin inside #benchLoopSinus)

You increase (1) to get a more stable result:

MathBench run: 1 "for debugging".
MathBench run: 100 "will take a long time, but results are more accurate"

You modify (2) to change the duration of your benchmark, in my silly example above, the
method is quite small and the benchmark would finish too quickly. Rule of thumb is to
get the run time of your benchmark (here #benchLoopSinus) in the range of 10ms and more.
This way you don't have to worry about the timer resolution used to capture the duration
of your method.


Does this answer your question?

On 2013-09-14, at 08:06, Natalia Tymchuk <[hidden email]> wrote:

> Hello.
> I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do?
> Best regards,
> Natalia
>
> 12.09.13 15:20, Stéphane Ducasse написав(ла):
>> On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <[hidden email]> wrote:
>>
>>> Hello Stephane.
>>> Yes, I'm interested in that and it will be an honor for me)).
>> Excellent.
>> If you have a github account let me know and I will add you as a contributor
>> Have a look at
>>
>> https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tree/master/Drafts/
>>
>>> But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .
>> No stress :)
>> My english sucks too and we should not care for now.
>>
>>> Best regards,
>>> Natalia
>>
>
>


signature.asc (457 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Natalia Tymchuk
Thanks, Camillo, for that big and very useful letter. You helped me a lot.
Best regards,
Natalia
14.09.13 15:16, Camillo Bruni написав(ла):

> Short Answer:
> -------------
> problemSize is used to calibrate your benchmark, so usually you adapt this value for
> your machine. And yes you are right, the result is not divided by the problemSize.
>
>
> Long Answer:
> ------------
> The typical use case is like this:
>
> MathBench >> benchLoopSinus
> 1 to: self problemSize to: [ :i | i sin ]
>
> Now you have two parameters to modifiy:
> 1. the number of samples you take (aka, how many times you measure the time of #benchLoopSinus)
> 2. the problem size (aka, how many times you run #sin inside #benchLoopSinus)
>
> You increase (1) to get a more stable result:
>
> MathBench run: 1 "for debugging".
> MathBench run: 100 "will take a long time, but results are more accurate"
>
> You modify (2) to change the duration of your benchmark, in my silly example above, the
> method is quite small and the benchmark would finish too quickly. Rule of thumb is to
> get the run time of your benchmark (here #benchLoopSinus) in the range of 10ms and more.
> This way you don't have to worry about the timer resolution used to capture the duration
> of your method.
>
>
> Does this answer your question?
>
> On 2013-09-14, at 08:06, Natalia Tymchuk <[hidden email]> wrote:
>> Hello.
>> I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do?
>> Best regards,
>> Natalia
>>
>> 12.09.13 15:20, Stéphane Ducasse написав(ла):
>>> On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <[hidden email]> wrote:
>>>
>>>> Hello Stephane.
>>>> Yes, I'm interested in that and it will be an honor for me)).
>>> Excellent.
>>> If you have a github account let me know and I will add you as a contributor
>>> Have a look at
>>>
>>> https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tree/master/Drafts/
>>>
>>>> But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .
>>> No stress :)
>>> My english sucks too and we should not care for now.
>>>
>>>> Best regards,
>>>> Natalia
>>


Reply | Threaded
Open this post in threaded view
|

Re: Benchmark

Stéphane Ducasse
In reply to this post by Camillo Bruni-3
we should add that to the book chapter :)

On Sep 14, 2013, at 2:16 PM, Camillo Bruni <[hidden email]> wrote:

> Short Answer:
> -------------
> problemSize is used to calibrate your benchmark, so usually you adapt this value for
> your machine. And yes you are right, the result is not divided by the problemSize.
>
>
> Long Answer:
> ------------
> The typical use case is like this:
>
> MathBench >> benchLoopSinus
> 1 to: self problemSize to: [ :i | i sin ]
>
> Now you have two parameters to modifiy:
> 1. the number of samples you take (aka, how many times you measure the time of #benchLoopSinus)
> 2. the problem size (aka, how many times you run #sin inside #benchLoopSinus)
>
> You increase (1) to get a more stable result:
>
> MathBench run: 1 "for debugging".
> MathBench run: 100 "will take a long time, but results are more accurate"
>
> You modify (2) to change the duration of your benchmark, in my silly example above, the
> method is quite small and the benchmark would finish too quickly. Rule of thumb is to
> get the run time of your benchmark (here #benchLoopSinus) in the range of 10ms and more.
> This way you don't have to worry about the timer resolution used to capture the duration
> of your method.
>
>
> Does this answer your question?
>
> On 2013-09-14, at 08:06, Natalia Tymchuk <[hidden email]> wrote:
>> Hello.
>> I have a question about problemSize in benchmarking. I got the small times from running benchmarks and that's why I try to use problemSize like in the video http://vimeo.com/68494202. But the benchmark for which I rewrote code and used problemSize has given me very big result. Maybe it looks like there is no division on that problemSize. Am I wrong? What can I do?
>> Best regards,
>> Natalia
>>
>> 12.09.13 15:20, Stéphane Ducasse написав(ла):
>>> On Sep 11, 2013, at 9:22 PM, Natalia Tymchuk <[hidden email]> wrote:
>>>
>>>> Hello Stephane.
>>>> Yes, I'm interested in that and it will be an honor for me)).
>>> Excellent.
>>> If you have a github account let me know and I will add you as a contributor
>>> Have a look at
>>>
>>> https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tree/master/Drafts/
>>>
>>>> But I never wrote the books and my English is not at its best level. However I think that after a couple reviews my English will be better .
>>> No stress :)
>>> My english sucks too and we should not care for now.
>>>
>>>> Best regards,
>>>> Natalia
>>>
>>
>>
>