Fun with spock (NativeBoost FFI)

So, thanks to Igor Stasenko, I've managed to learn how to (sorta) use
the new NativeBoost FFI (spock). We (he) implemented a binding to a few
of the gmplib functions. gmp is the GNU Multi-Precision Library.

Here is a simple benchmark of the Pharo factorial method vs the gmplib
mpz_fac_ui() function. As might be expected, gmplib is  faster for any
test larger than 13 factorial, as LargePositiveInteger  starts to become
involved, and the speed differential becomes rather huge as larger
integers are tested:

1000 factorial time to run with:
squeak: 2 ms      gmplib: 0.004 ms   ratio: 50.0
11000 factorial time to run with:
squeak: 354 ms      gmplib: 0.191 ms   ratio: 185.34031413612567
[...]
91000 factorial time to run with:
squeak: 34598 ms      gmplib: 4.759 ms   ratio: 727.0014708972473



Running on a Mac OS X Snow Leopard 2.8Ghz, with NBCog and 1.4 Pharo image.

benchmark:

a := GMPzWrapper new.

1000 to: 100000 by: 10000 do:
[:i|
    squeakTime :=  [ i factorial] timeToRun.
    Transcript show:  i asString, ' factorial time to run with:';cr.
    Transcript show:  'squeak: ', (squeakTime) asString, ' ms '.
    gmpTime :=  [100 timesRepeat: [a prim_mpz_fac_ui: i]] timeToRun.
    Transcript show:  '     gmplib: ' , (gmpTime/1000.0) asString, ' ms  '.
    Transcript show: ' ratio: ', (squeakTime/(gmpTime/100.0))  
asString;cr;cr.
]


--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 5/13/12 11:02 AM, Lawson English wrote:

[results]

Ack, I printed the times for the gmplib off by a factor of 10. However,
the ratio is still correct.

gmplib is at least 700x faster than the built-in factorial method for
100000 factorial.

--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

Smalltalk cannot beat gmp, but the comparison is not fair, anything
can beat current implementation of factorial...
See http://smallissimo.blogspot.fr/2011/07/evaluating-factorial-from-its-prime.html
and some following posts which has a few links on how to compute it
efficiently.

Nicolas

2012/5/13 Lawson English <[hidden email]>:

On 13 May 2012 20:58, Nicolas Cellier
<[hidden email]> wrote:
> Smalltalk cannot beat gmp, but the comparison is not fair, anything
> can beat current implementation of factorial...

Right. It is not fair. Especially that we're talking about something
implemented in C
vs something implemented in smalltalk.
The comparison would be more fair if we could have a separate
primitive for factorial.



--
Best regards,
Igor Stasenko.

2012/5/13 Igor Stasenko <[hidden email]>:
[100000 primeSwingFactorial] timeToRun 3411
[100000 factorial] timeToRun 61219

So gmp is still 40x faster than a reasonably optimized Smalltalk factorial...
I think that's fair, because gmp is highly optimized.

Since cost is dominated by LargeInteger arithmetic, and since
Smalltalk LargeInteger arithmetic in primitives still operates on
bytes, I'm not so surprised of the gap...

I'd like to see a 64bit image operating on 32 bits positive integers
digits, that would already be a progress...

Nicolas

[pharo-project list added back in]

On 5/13/12 1:34 PM, Nicolas Cellier wrote:
> [100000 primeSwingFactorial] timeToRun 3411 [100000 factorial]
> timeToRun 61219 So gmp is still 40x faster than a reasonably optimized
> Smalltalk factorial... I think that's fair, because gmp is highly
> optimized. Since cost is dominated by LargeInteger arithmetic, and
> since Smalltalk LargeInteger arithmetic in primitives still operates
> on bytes, I'm not so surprised of the gap... I'd like to see a 64bit
> image operating on 32 bits positive integers digits, that would
> already be a progress... Nicolas

For me, the most interesting thing would be to figure out how to
integrate such external libraries more tightly into the language syntax.

Right now,   x :=  3.14159... always creates a Float, which is stored as
a double.

x class ===> Float.

What if I wanted to generate a gmplib floating point number (or rational
or large integer) automatically when the gmplib binding is available?

Could I implement something that makes x a GMPfpWrapper if extra digits
are supplied?
e.g.:

x := 3.14159265358979323846264338327950288419716939937510.

x class ===> GMPfpWrapper.

NativeBoost allows one to cascade calls to external libraries if you
wrap them in assembly language in the method. High speed,
aribrary-precision floating point complex number arithmetic, can be
accomplished that way (no examples yet sorry, still learning) as well as
more complicated stuff. I'm hoping to make a proof-of-concept mandlebrot
set Morph that can zoom in arbitrarily. 32 bit floats limit the
resolution terribly.

So...

What can be done to better integrate such libraries? I browsed the
Parser and other classes but didn't understand what I was seeing well
enough to figure out how to begin.



L

--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 14 May 2012 00:03, Lawson English <[hidden email]> wrote: See SqNumberParser.
this is the guy who used by Parser to parse number literals.



--
Best regards,
Igor Stasenko.

On 5/13/12 3:49 PM, Igor Stasenko wrote:
Thanks.

L


--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 14 May 2012 01:39, Lawson English <[hidden email]> wrote:
keep in mind that the instance of those numbers from GMP library is kept in
external memory. that means if you save an image with it, all of them
will become invalid.
that, in own turn, makes them pretty useless as literals, unless you
will be using some intermediate form
to keep them and convert on request, i.e..

pi
  ^ '3.14159265358979323846264338327950288419716939937510' asGMPFloat


--
Best regards,
Igor Stasenko.

There is also ArbitraryPrecisionFloat
(http://www.squeaksource.com/ArbitraryPrecisionFl) for image internal
representation.
It is using base 2 for internal implementation, not sure if it fits
gmp floats...
It could also provide a slooow replacement if gmp is absent.

Nicolas

2012/5/14 Igor Stasenko <[hidden email]>:

On May 14, 2012, at 12:49 AM, Igor Stasenko wrote:
For Floats, there is a GNU project maintained by some groups at Inria:
       
        http://www.mpfr.org/

The main goal of MPFR is to provide a library for multiple-precision floating-point computation which is both
efficient and has a well-defined semantics. It copies the good ideas from the ANSI/IEEE-754 standard for
double-precision floating-point arithmetic (53-bit significand).



--
Marcus Denker -- http://marcusdenker.de

On 5/14/12 12:06 AM, Marcus Denker wrote:
I saw  that mentioned on the gmp site. The gmp maintainers recommend it  
for more robust floating point libraries.




--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

I am trying to implement an asString method for gmp floats but I don't
quite see how.


Prototype for getting a string is:


"Function: char * mpf_get_str (char *str, mp_exp_t *expptr, int base,
size_t n_digits, mpf_t op)"

where mp_exp_t *expptr   is just a pointer to a long, and size_t is just
a long.  I don't get how to set up the primitive call and how to evoke
it in the asString method.



--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 15 May 2012 02:38, Lawson English <[hidden email]> wrote: just do as it says -
create a string, long enough to hold n_digits + 2 characters, i.e.

str := ByteString new: ndigits + 2.

self mpf_get_str: str exp:.. .....

since you passing the string, you can ignore return value (just change the
return type of function signature to be void).
As result, a function should fill your string with numbers.
And don't use 'String' as type, leave 'char*' , like that NB will pass
a pointer to first byte
of your preallocated string, without converting it to C string which
is temporary buffer on stack.

void mpf_get_str (char *str, mp_exp_t *expptr, int base, size_t
n_digits, mpf_t op)

>
>
>
> --
> Squeak from the very start (introduction to Squeak and Pharo Smalltalk for
> the (almost) complete and compleate beginner).
> https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all
>



--
Best regards,
Igor Stasenko.

The problem I am having is I don't know how to pass in mp_exp_t *expptr


I keep getting Error: coma expected

 >>prim___gmpf_get_strWithExp:exp

"Function: char * mpf_get_str (char *str, mp_exp_t *expptr, int base,
size_t n_digits, mpf_t op)"
<primitive: #primitiveNativeCall module: #NativeBoostPlugin error:
errorCode>

     ^ self nbCall: #( String __gmpf_get_str (ulong 0,  NBSignedLong *
exp, long 10, long 0, mpf_t gmpf))

I've tried passing in  4 byte arrays and such, but nothing seems to work.


L






On 5/14/12 6:09 PM, Igor Stasenko wrote:

--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 15 May 2012 03:49, Lawson English <[hidden email]> wrote:
don't denote constants with type IIRC it doesn't works that way.
don't use 'String' as result value type. If you use this function in
manner as you shown, you forming a memory leak. Check documentation,
it says that if you pass a string, then you allocate it
otherwise a library will allocate it. But then you will need to
manually free it.

> I've tried passing in  4 byte arrays and such, but nothing seems to work.
>
concerting mpexp_t * argument, you should pass a pointer to the buffer
big enough to fit mpexp_t
type value.
You can allocate a bytearray (sizeof(mp_exp_t) and pass it. But doc
says nothing about it..



--
Best regards,
Igor Stasenko.

OK, getting closer. However I'm still not quite getting what I expect:


f1 := GMPfWrapper new.
f2 := GMPfWrapper new.
f3 := GMPfWrapper new.
f1 set_ui: 50.
f2 set_ui: 50.
f3 set_ui:50.
(f3 +f2) asString ==> '1

On 5/14/12 8:02 PM, Lawson English wrote: hmm apparently my source code was deleted. Something to do with the '>>"


relevant code:

GMPfWrapper>>asString



     |exp aString|
     exp := ByteArray new:4.
     aString := ByteString new: 12.   "for 10 digits"
     self prim___gmpf_get_str: aString WithExp:exp.
     ^aString

GMPfWrapper>>prim___gmpf_get_str: aString WithExp:exp

"Function: char * mpf_get_str (char *str, mp_exp_t *expptr, int base,
size_t n_digits, mpf_t op)"
<primitive: #primitiveNativeCall module: #NativeBoostPlugin error:
errorCode>

     ^ self nbCall: #( void __gmpf_get_str ( char *aString,  
NBSignedLong * exp, 10,  10, mpf_t gmpf))

--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 5/14/12 8:06 PM, Lawson English wrote:
Doh, its working just fine. the function only returns the minimum digits
required and returns the exponent in the expptr, so 1 has an expptr
value of 3, making it 100.0...

Sigh.

L.




--
Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner).
https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all

On 15 May 2012 09:42, Lawson English <[hidden email]> wrote: Right. Yeah, it works just fine, except there is no description what
is exp stands for.
So you have to guess or look for example code :)

> L.
>
>


--
Best regards,
Igor Stasenko.