UUIDPrimitivesTest>>testCreationRandom

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

UUIDPrimitivesTest>>testCreationRandom

Thomas Koenig-2
John M McIntosh,
Hi, I think you created UUID and its test case UUIDPrimitivesTest? Do you
have time to help resolve the problem?

UUIDPrimitivesTest>>testCreationRandom fails on this assertion
 self should: [((UUID new at: 7) bitAnd: 16rF0) = 16r40].
Here's what I can tell you:
-The failure occurs in 3.9b 7029 and in 3.8 6665
-I'm testing on win xp
-the assertion appears to fail the very first time (it's in a repeats 1000)
-(self at: 7) bitAnd: 16rF0 ==> 32 (not 64)
-the other assertion passes always.

Thanks
Tom



Reply | Threaded
Open this post in threaded view
|

Re: UUIDPrimitivesTest>>testCreationRandom

johnmci
Well first some background

See
http://www.webdav.org/specs/draft-leach-uuids-guids-01.txt

Most  current Squeak VMs provide a primitive that asks the operating  
system to generate a UUID, I believe that is true for unix,  
macintosh, and windows now. If such a primitive does not exist we  
generate a UUID using the UUID version 4, which it is a randomly or  
pseudo-randomly generated 60 bit value generated by Smalltalk code.

Those 60 bits of random data come from looking at the results of  
Random which is is an adaptation of the Park-Miller RNG.

See UUIDGenerator>>generateOneOrZero
In this routine we reset the generator to a random starting point on  
unix based or macintosh machines using /usr/urandom, or by using a  
formula based on the millisecond clock every 100,000 iterations. Mmm  
in looking at that I wonder what the probability of getting a seed is  
that is a repeat of a previously used seed.

Perhaps that if clause should be removed? Thoughts?

I'll note the first bug discovered here was that we reset the Random  
number to a static value at image startup time, that caused the UUID  
generation using UUIDGenerator to repeat on image startups that did  
not involve saves, hence a startup method was added to reset the  
generator at image startup time.

But back to the question.
You see originally the type of UUID generated by the operating system  
would have the last twelve bytes the same since that was the bits  
from the ethernet interface card. The testCreationRandom checks to  
see if those bytes are the same, if not then it assumes the UUID is a  
randomly generated UUID and then does a simple pass to see if there  
are any duplicates in a short sequence and checks flags to see if  
this is a random UUID, that as you noticed fails.

So why.

Ah, well because someone noticed that if you were tagging things with  
UUIDs, a thing that I believe Microsoft office does? Then why it's  
possible to determine which Windows computer generated that UUID, or  
at least which ethernet card was in use at the time of editing. Since  
that was a privacy concern Microsoft changed the algorithm to one-way  
(right, I"m sure) hash the entire UUID.

Therefore the testCreationRandom test likely should be deleted unless  
we are using the UUIDGenerator because the UUID has been hashed and  
none of the should:[] are valid.

Lastly
See earlier notes on this subject

http://lists.squeakfoundation.org/pipermail/squeak-dev/2003-January/ 
051248.html

I'll note the test

        Smalltalk platformName asLowercase = 'win32' ifTrue:
                ["Some microsoft platforms one-way hash the UUID so this test gets  
invoked
                however it will fail because the UUID could be node-based, only  
solution is to
                terminate the test early"
                ^self].

is now not complete, since in current versions of OS-X apple also  
decided to hash the uuid values they generate.
I'm not sure at what version of os-x they started doing this.
Therefore someone would need to add a test to see if platform is
SmalltalkImage current platformName asLowercase = 'mac os'
and
consider
SmalltalkImage current osVersion asNumber < ????

to check os-x at a paricular version cutoff point.


On 7-May-06, at 10:54 AM, Thomas Koenig wrote:

> John M McIntosh,
> Hi, I think you created UUID and its test case UUIDPrimitivesTest?  
> Do you
> have time to help resolve the problem?
>
> UUIDPrimitivesTest>>testCreationRandom fails on this assertion
>  self should: [((UUID new at: 7) bitAnd: 16rF0) = 16r40].
> Here's what I can tell you:
> -The failure occurs in 3.9b 7029 and in 3.8 6665
> -I'm testing on win xp
> -the assertion appears to fail the very first time (it's in a  
> repeats 1000)
> -(self at: 7) bitAnd: 16rF0 ==> 32 (not 64)
> -the other assertion passes always.
>
> Thanks
> Tom
>
>
>

--
========================================================================
===
John M. McIntosh <[hidden email]> 1-800-477-2659
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===


Reply | Threaded
Open this post in threaded view
|

RE: UUIDPrimitivesTest>>testCreationRandom

Thomas Koenig-2
John, Wow. Thanks for your quick and clear response.  Three things. 1. I
will submit the FIX to remove the original assertion since it's not longer
valid.  2.  I'll park the incomplete test issue as a Mantis BUG item unless
you already have and until someone provides the release number.  3.  I'll
defer to more skilled randomologists on the issue of repeating seeds.
Tom


> -----Original Message-----
> From: [hidden email] [mailto:squeak-dev-
> [hidden email]] On Behalf Of John M McIntosh
> Sent: Sunday, May 07, 2006 3:05 PM
> To: The general-purpose Squeak developers list
> Subject: Re: UUIDPrimitivesTest>>testCreationRandom
>
> Well first some background
>
> See
> http://www.webdav.org/specs/draft-leach-uuids-guids-01.txt
>
> Most  current Squeak VMs provide a primitive that asks the operating
> system to generate a UUID, I believe that is true for unix,
> macintosh, and windows now. If such a primitive does not exist we
> generate a UUID using the UUID version 4, which it is a randomly or
> pseudo-randomly generated 60 bit value generated by Smalltalk code.
>
> Those 60 bits of random data come from looking at the results of
> Random which is is an adaptation of the Park-Miller RNG.
>
> See UUIDGenerator>>generateOneOrZero
> In this routine we reset the generator to a random starting point on
> unix based or macintosh machines using /usr/urandom, or by using a
> formula based on the millisecond clock every 100,000 iterations. Mmm
> in looking at that I wonder what the probability of getting a seed is
> that is a repeat of a previously used seed.
>
> Perhaps that if clause should be removed? Thoughts?
>
> I'll note the first bug discovered here was that we reset the Random
> number to a static value at image startup time, that caused the UUID
> generation using UUIDGenerator to repeat on image startups that did
> not involve saves, hence a startup method was added to reset the
> generator at image startup time.
>
> But back to the question.
> You see originally the type of UUID generated by the operating system
> would have the last twelve bytes the same since that was the bits
> from the ethernet interface card. The testCreationRandom checks to
> see if those bytes are the same, if not then it assumes the UUID is a
> randomly generated UUID and then does a simple pass to see if there
> are any duplicates in a short sequence and checks flags to see if
> this is a random UUID, that as you noticed fails.
>
> So why.
>
> Ah, well because someone noticed that if you were tagging things with
> UUIDs, a thing that I believe Microsoft office does? Then why it's
> possible to determine which Windows computer generated that UUID, or
> at least which ethernet card was in use at the time of editing. Since
> that was a privacy concern Microsoft changed the algorithm to one-way
> (right, I"m sure) hash the entire UUID.
>
> Therefore the testCreationRandom test likely should be deleted unless
> we are using the UUIDGenerator because the UUID has been hashed and
> none of the should:[] are valid.
>
> Lastly
> See earlier notes on this subject
>
> http://lists.squeakfoundation.org/pipermail/squeak-dev/2003-January/
> 051248.html
>
> I'll note the test
>
> Smalltalk platformName asLowercase = 'win32' ifTrue:
> ["Some microsoft platforms one-way hash the UUID so this
test

> gets
> invoked
> however it will fail because the UUID could be node-based,
> only
> solution is to
> terminate the test early"
> ^self].
>
> is now not complete, since in current versions of OS-X apple also
> decided to hash the uuid values they generate.
> I'm not sure at what version of os-x they started doing this.
> Therefore someone would need to add a test to see if platform is
> SmalltalkImage current platformName asLowercase = 'mac os'
> and
> consider
> SmalltalkImage current osVersion asNumber < ????
>
> to check os-x at a paricular version cutoff point.
>
>
> On 7-May-06, at 10:54 AM, Thomas Koenig wrote:
>
> > John M McIntosh,
> > Hi, I think you created UUID and its test case UUIDPrimitivesTest?
> > Do you
> > have time to help resolve the problem?
> >
> > UUIDPrimitivesTest>>testCreationRandom fails on this assertion
> >  self should: [((UUID new at: 7) bitAnd: 16rF0) = 16r40].
> > Here's what I can tell you:
> > -The failure occurs in 3.9b 7029 and in 3.8 6665
> > -I'm testing on win xp
> > -the assertion appears to fail the very first time (it's in a
> > repeats 1000)
> > -(self at: 7) bitAnd: 16rF0 ==> 32 (not 64)
> > -the other assertion passes always.
> >
> > Thanks
> > Tom
> >
> >
> >
>
> --
> ========================================================================
> ===
> John M. McIntosh <[hidden email]> 1-800-477-2659
> Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
> ========================================================================
> ===
>