Smalltalk › Pharo › Pharo Smalltalk Users

STON materialization corrupts a dictionary if the keys are references

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

5 messages Options

Peter Uhnak

STON materialization corrupts a dictionary if the keys are references

Hi,

I ran into a problem with STON that seems to corrupt on materialization.

Consider the following script:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

c := OrderedCollection with: UUID new.

d := Dictionary with: c first -> nil.

o := { c . d }.

s := STON toStringPretty: o.

o2 := STON reader

allowComplexMapKeys: true;

on: s readStream;

next.

o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))"

o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary"

o2 second at: o2 second keys first. "the same error"

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

So even though it contains the key it fails to access it.

Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.

Thanks,

Peter

Sven Van Caekenberghe-2

Re: STON materialization corrupts a dictionary if the keys are references

Hi Peter,

Strange indeed. It did not fail for me in 5.0 with the standard STON version, but it does fail in 4.0 with the latest STON. Asking the bogus dictionary to #rehash seems to fix the problem. I will investigate and turn you example into a unit test.

Sven

> On 05 Apr 2016, at 11:14, Peter Uhnák <[hidden email]> wrote:
>
> Hi,
>
> I ran into a problem with STON that seems to corrupt on materialization.
>
> Consider the following script:
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> c := OrderedCollection with: UUID new.
> d := Dictionary with: c first -> nil.
> o := { c . d }.
>
> s := STON toStringPretty: o.
>
> o2 := STON reader
> allowComplexMapKeys: true;
> on: s readStream;
> next.
>
> o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))"
> o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary"
> o2 second at: o2 second keys first. "the same error"
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> So even though it contains the key it fails to access it.
>
> Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.
>
> Thanks,
> Peter
>
>
>

Peter Uhnak

Re: [Pharo-dev] STON materialization corrupts a dictionary if the keys are references

This is

Pharo #50666

Moose 6.0 build #1437

STON-Core-SvenVanCaekenberghe.67

(I also loaded STON-Core-SvenVanCaekenberghe.70 but the result is the same).

One thing to note: it doesn't fail always… for me it seems that about every ten or so attempts it succeeds.

Peter

On Tue, Apr 5, 2016 at 11:41 AM, Sven Van Caekenberghe <[hidden email]> wrote:

Hi Peter,

Strange indeed. It did not fail for me in 5.0 with the standard STON version, but it does fail in 4.0 with the latest STON. Asking the bogus dictionary to #rehash seems to fix the problem. I will investigate and turn you example into a unit test.

Sven

> On 05 Apr 2016, at 11:14, Peter Uhnák <[hidden email]> wrote:
>
> Hi,
>
> I ran into a problem with STON that seems to corrupt on materialization.
>
> Consider the following script:
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> c := OrderedCollection with: UUID new.
> d := Dictionary with: c first -> nil.
> o := { c . d }.
>
> s := STON toStringPretty: o.
>
> o2 := STON reader
> allowComplexMapKeys: true;
> on: s readStream;
> next.
>
> o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))"
> o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary"
> o2 second at: o2 second keys first. "the same error"
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> So even though it contains the key it fails to access it.
>
> Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.
>
> Thanks,
> Peter
>
>
>

Sven Van Caekenberghe-2

Re: [Pharo-dev] STON materialization corrupts a dictionary if the keys are references

> On 05 Apr 2016, at 11:50, Peter Uhnák <[hidden email]> wrote:
>
> This is
> Pharo #50666
> Moose 6.0 build #1437
> STON-Core-SvenVanCaekenberghe.67
>
> (I also loaded STON-Core-SvenVanCaekenberghe.70 but the result is the same).
>
> One thing to note: it doesn't fail always… for me it seems that about every ten or so attempts it succeeds.

Like I said, it is a hashing issue, sometimes it will be correct by accident.

I hope you did not have to much trouble with this bug, I guess it must have been hard to chase.

Is it urgent ?

I probably can give you a quick fix, but I would like to think a bit more about this, since rehashing each materialised dictionary seems expensive.

> Peter
>
> On Tue, Apr 5, 2016 at 11:41 AM, Sven Van Caekenberghe <[hidden email]> wrote:
> Hi Peter,
>
> Strange indeed. It did not fail for me in 5.0 with the standard STON version, but it does fail in 4.0 with the latest STON. Asking the bogus dictionary to #rehash seems to fix the problem. I will investigate and turn you example into a unit test.
>
> Sven
>
> > On 05 Apr 2016, at 11:14, Peter Uhnák <[hidden email]> wrote:
> >
> > Hi,
> >
> > I ran into a problem with STON that seems to corrupt on materialization.
> >
> > Consider the following script:
> >
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > c := OrderedCollection with: UUID new.
> > d := Dictionary with: c first -> nil.
> > o := { c . d }.
> >
> > s := STON toStringPretty: o.
> >
> > o2 := STON reader
> > allowComplexMapKeys: true;
> > on: s readStream;
> > next.
> >
> > o2 second keys. "an Array(an UUID('571bbfb7-78b9-430c-9aad-3e0088a25f4c'))"
> > o2 second printString. "KeyNotFound: key an UUID(''571bbfb7-78b9-430c-9aad-3e0088a25f4c'') not found in Dictionary"
> > o2 second at: o2 second keys first. "the same error"
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >
> > So even though it contains the key it fails to access it.
> >
> > Interestingly enough if I serialize and materialize the "broken" dictionary again it suddenly works again.
> >
> > Thanks,
> > Peter
> >
> >
> >
>
>
>

Peter Uhnak

Re: [Pharo-dev] STON materialization corrupts a dictionary if the keys are references

Like I said, it is a hashing issue, sometimes it will be correct by accident.

I hope you did not have to much trouble with this bug, I guess it must have been hard to chase.

The hardest thing often is to suspect the system or external libraries, but it wasn't that bad.

Is it urgent ?

I can rehash it for now (I have a trivial case), but it should be fixed eventually (it wouldn't be so easy to rehash more complex structures).

Thanks,

Peter