Issue 4997 in pharo: isPrime is broken

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue 4997 in pharo: isPrime is broken

pharo
Status: Accepted
Owner: [hidden email]
Labels: Milestone-1.4

New issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Hi,

There seems to be a problem with #isPrime in Pharo 1.4. The following  
example
runs apparently forever:

  100000000000000000000000000039 isPrime.

Unfortunately I used #isPrime to demonstrate multiprocessing with worker
images with RemoteTask. RemoteTask seems to work fine on Pharo, but the
examples run "forever" and thus appear to fail due to the #isPrime problem.

RemoteTask is described here
<http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167.html>


Dave



_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #1 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

The method in Squeak 3.9 (by Enrico Spinielli) worked, but seems to have  
been replaced by an incomplete copy of Levente's improved methods in Squeak  
trunk. Suggest either adopting the remaining LargePositiveInteger>>isPrime  
and Integer>>isProbablyPrime from Squeak trunk, or just revert to the  
original. I think there was some discussion of this on squeak-dev in early  
2010.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #2 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Pharo implements isPrime with a naive deterministic primality test which  
will not end for soon with some bad luck...
Squeak implements isPrime with a probabilistic primality test  
(isProbablyPrime which is a Miller Rabin test).

The choice is to
- either mimic Squeak and let the primality test be probabilist
- or implement a fast determinist test
See http://en.wikipedia.org/wiki/Primality_test


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #3 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

I also note that Integer>>atRandom is not uniformly distributed for huge  
integers
(huge here means > 53 bits which is not that huge!)

For example  100000000000000000000000000039 highBit -> 97
self shouldnt: [(1 to: 10000) detect: [:i | (100000000000000000000000000037  
atRandom + 1) odd]] raise: Error.

Or try this and see how poor is the random generator:
((1 to: 50000) collect: [:i | (100000000000000000000000000037 atRandom + 1)  
bitAnd: 1<<28-1] as: Set) -> aSet(2)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #4 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Not sure you can say the PRNG in itself is poor on those grounds (though it  
has been shown to have flaws), but it's API in Squeak/Pharo surely begs to  
be misused in the manner you describe.

1) The PRNG has a period of 2^32-1, any integer above that will have  
numbers which will never appear from atRandom.
2) Internally it stores the seed as a Double, so atRandom will at least  
multiply the number with the smallest possible Double (ie why you see so  
few variations of the low bits in the 97bit-number).
3) Assuming a perfectly bit-random algorithm of any size n, it is still  
impossible to get a *perfectly* uniform distribution of any range with size  
other than 2^n -1. In other words, however you divide the outcome, some  
numbers will be more heavily biased than others.
4) Using it for cryptography, as suggested by nextInt: comment, is a *very  
bad idea*. Both because of 1) and 2), but also because a single call to  
next will expose the entire seed to an attacker. (Just multiply with m)

TLDR; The Random API and implementation is a bad fit for mostly any use  
other than fast generation of numbers of a small range/value.




_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #5 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

How much this affects the quality of numbers chosen to do primality checks  
with is outside my expertise though :)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #6 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Yes, this is a tough subject...

Miller Rabin strong pseudoprime has a probability of false positive <= 1/4  
over all bases in [2,n[
Repeated 25 times with 25 different bases at random leads to a probability  
<= 1/(4 raisedTo: 25) < 1.0e-15.

If the bases are chosen in a different set, who knows...
Maybe this guy... http://www.pseudoprime.com/pseudo.html


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #7 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Hi guys

let us know what you think is the best and we will act :).
We are pretty bad in that domain :).

Stef


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #8 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Well... a very simple to implement speedup is to do
isPrime
^(self < someThresholdWhereNaiveIsFasterAnyways or: [self isProbablyPrime])  
and: ["Old slow naive method"]

(isProbablyPrime will only ever return false positives).

Then again, you might get unlucky with the next number you choose in a  
demonstration and it is actually prime :)

The fast deterministic approaches seem somewhat tricker to implement,  
AFAICT they follow the same structure as the probabilistic one, but with  
prime check functions and numbers to test against computed for each prime,  
instead of being constant/chosen at random.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo
Updates:
        Labels: Type-Bug

Comment #9 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

(No comment was entered for this change.)


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo
Updates:
        Status: NoSourcesAvailable

Comment #10 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Next Action: Please attach machine readable code.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #11 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

so guys do you have one solution?

Stef


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #12 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

I would suggest
1) Introduce a change to image as described in Comment 8, close this issue.
2) Create a new issue about commenting in isProbablyPrimeWithK:andQ: about  
how random not properly distributed for large ints, and that the impact of  
this on the probability for each call to this method to return a false  
positive has not been investigated.
3) Create a new issue regarding implementing a fast deterministic  
fallback-approach for when isProbablyPrime: returns a positive.
Include links to
http://en.wikipedia.org/wiki/Primality_test
and
http://www.math.dartmouth.edu/~carlp/PDF/complexity12.pdf
as the probably correct approach with currently known lowest runtime  
complexity.
Should be a nice weekend project for a mathematically interested person :)

Sound good?


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #13 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

for comment 8 I have no idea what I should **really** do.
This is why having full code is always better.



_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo

Comment #14 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Hi guys

can one of you tell me what code I can use? Now I really do not know  
anything about prime number implementation so all these discussions are  
nearly useless and does not help us. Sadly.
I would like to start putting 1.4 in beta and this one is important.




_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo
Updates:
        Cc: [hidden email]

Comment #15 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

henrik could you have a look at this entry because I would like to really  
close it in 1.4. Thanks


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo
Updates:
        Cc: [hidden email]

Comment #16 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

For 1.4 I would really like to be able to run OSProcess and I need from you  
a clear description.

Should I only do that:
isPrime
^(self < someThresholdWhereNaiveIsFasterAnyways or: [self isProbablyPrime])  
and: ["Old slow naive method"]


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker
Reply | Threaded
Open this post in threaded view
|

Re: Issue 4997 in pharo: isPrime is broken

pharo
Updates:
        Status: Closed

Comment #17 on issue 4997 by [hidden email]: isPrime is broken
http://code.google.com/p/pharo/issues/detail?id=4997

Ok I took isPrime and isPrimeProbably.... methods from squeak trunk.


_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-bugtracker