Smalltalk › Squeak › Squeak - Dev

Trunk update puts corrupt Monticello files into package-cache

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

6 messages Options

bpi

Trunk update puts corrupt Monticello files into package-cache

One other thing that I found out is that the trunk update from Squeak4.5-13680.image seems to create corrupt files in the package-cache. I use the following batch script to verify:

#!/bin/bash
cd trunk/package-cache
for f in *.mcd *.mcz
do
unzip -t $f | grep "No errors"
done

Most but not all of the files in the package-cache seem to be corrupt.

I would be interested if others see that also.

Cheers,
Bernhard

bpi

Re: Trunk update puts corrupt Monticello files into package-cache

I forgot to mention that I checked for some of the corrupt files if they are corrupt on source.squeak.org and it seems they are not, e.g.

curl -O http://source.squeak.org/trunk/Collections-eem.571$nice.569$.mcd
unzip -t Collections-eem.571$nice.569$.mcd
Archive: Collections-eem.571(nice.569).mcd
testing: package OK
testing: version OK
testing: base OK
testing: old/source.st OK
testing: new/source.st OK
testing: patch.bin OK
No errors detected in compressed data of Collections-eem.571(nice.569).mcd.

It seems to follow that the corruptions happen locally because of broken (intermediate?) code which is executed during the trunk update.

Cheers,
Bernhard

Am 30.08.2014 um 15:50 schrieb Bernhard Pieber <[hidden email]>:

> One other thing that I found out is that the trunk update from Squeak4.5-13680.image seems to create corrupt files in the package-cache. I use the following batch script to verify:
>
> #!/bin/bash
> cd trunk/package-cache
> for f in *.mcd *.mcz
> do
> unzip -t $f | grep "No errors"
> done
>
> Most but not all of the files in the package-cache seem to be corrupt.
>
> I would be interested if others see that also.
>
> Cheers,
> Bernhard

bpi

Re: Trunk update puts corrupt Monticello files into package-cache

I made another experiment. As all corrupt Monticello files happened to be .mcd files I tried Bert's suggention and did the trunk update without using .mcd but only .mcz files. To do this I commented out the diffy part in MCConfiguration>>versionNamed:for:from: and did the trunk update.

This results in the infamous "Error: Bits size mismatch" for Kernel-eem.851.mcz. I checked the package-cache and this is the only one corrupt .mcz file.

unzip -t Kernel-eem.851.mcz
Archive: Kernel-eem.851.mcz
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of Kernel-eem.851.mcz or
Kernel-eem.851.mcz.zip, and cannot find Kernel-eem.851.mcz.ZIP, period.

Again it seems not corrupt on source.squeak.org:

curl -O http://source.squeak.org/trunk/Kernel-eem.851.mcz
unzip -t Kernel-eem.851.mcz
Archive: Kernel-eem.851.mcz
testing: package OK
testing: version OK
testing: snapshot/source.st OK
testing: snapshot.bin OK
No errors detected in compressed data of Kernel-eem.851.mcz.

Cheers,
Bernhard

Am 30.08.2014 um 16:03 schrieb Bernhard Pieber <[hidden email]>:

> I forgot to mention that I checked for some of the corrupt files if they are corrupt on source.squeak.org and it seems they are not, e.g.
>
> curl -O http://source.squeak.org/trunk/Collections-eem.571$nice.569$.mcd
> unzip -t Collections-eem.571$nice.569$.mcd
> Archive: Collections-eem.571(nice.569).mcd
> testing: package OK
> testing: version OK
> testing: base OK
> testing: old/source.st OK
> testing: new/source.st OK
> testing: patch.bin OK
> No errors detected in compressed data of Collections-eem.571(nice.569).mcd.
>
> It seems to follow that the corruptions happen locally because of broken (intermediate?) code which is executed during the trunk update.
>
> Cheers,
> Bernhard
>
> Am 30.08.2014 um 15:50 schrieb Bernhard Pieber <[hidden email]>:
>
>> One other thing that I found out is that the trunk update from Squeak4.5-13680.image seems to create corrupt files in the package-cache. I use the following batch script to verify:
>>
>> #!/bin/bash
>> cd trunk/package-cache
>> for f in *.mcd *.mcz
>> do
>> unzip -t $f | grep "No errors"
>> done
>>
>> Most but not all of the files in the package-cache seem to be corrupt.
>>
>> I would be interested if others see that also.
>>
>> Cheers,
>> Bernhard
>

David T. Lewis

Re: Trunk update puts corrupt Monticello files into package-cache

In reply to this post by bpi

On Sat, Aug 30, 2014 at 04:03:23PM +0200, Bernhard Pieber wrote:

One of the updates in the update stream (Collections-eem.567) exposes a bug
that affects zip compression, and that therefore messes up MCZ processing.

The only available workaround is to use a very recently compiled VM, built
from sources circa July or later. Without that, it may not be possible to
successfully update the trunk stream. The VMs distributed with Squeak 4.5 or
from squeakvm.org probably will not work.

This information is not on Mantis, but it should be. Even after sorting out
the VM related issues, I think we still have issues remaining in the Smalltalk
zip processing (the fallback code for the primitives), so it may take some
time to get it all straightened out.

Dave

bpi

Re: Trunk update puts corrupt Monticello files into package-cache

Hi David,

Thanks for the answer!

I am aware of this. Therefore I use Eliot's latest Cog VM 6063 for all my experiments. Here is how I get it:
curl -O http://www.mirandabanda.org/files/Cog/VM/VM.r3063/Cog.app-14.32.3063.tgz
gunzip -c Cog.app-14.32.3063.tgz | tar xopf - && rm Cog.app-14.32.3063.tgz

The newest VM should not create corrupt zip files, right?

Cheers,
Bernhard

Am 30.08.2014 um 16:24 schrieb David T. Lewis <[hidden email]>:

> On Sat, Aug 30, 2014 at 04:03:23PM +0200, Bernhard Pieber wrote:
>> I forgot to mention that I checked for some of the corrupt files if they are corrupt on source.squeak.org and it seems they are not, e.g.
>>
>> curl -O http://source.squeak.org/trunk/Collections-eem.571$nice.569$.mcd
>> unzip -t Collections-eem.571$nice.569$.mcd
>> Archive: Collections-eem.571(nice.569).mcd
>> testing: package OK
>> testing: version OK
>> testing: base OK
>> testing: old/source.st OK
>> testing: new/source.st OK
>> testing: patch.bin OK
>> No errors detected in compressed data of Collections-eem.571(nice.569).mcd.
>>
>> It seems to follow that the corruptions happen locally because of broken (intermediate?) code which is executed during the trunk update.
>
> One of the updates in the update stream (Collections-eem.567) exposes a bug
> that affects zip compression, and that therefore messes up MCZ processing.
>
> The only available workaround is to use a very recently compiled VM, built
> from sources circa July or later. Without that, it may not be possible to
> successfully update the trunk stream. The VMs distributed with Squeak 4.5 or
> from squeakvm.org probably will not work.
>
> This information is not on Mantis, but it should be. Even after sorting out
> the VM related issues, I think we still have issues remaining in the Smalltalk
> zip processing (the fallback code for the primitives), so it may take some
> time to get it all straightened out.
>
> Dave
>
>

David T. Lewis

Re: Trunk update puts corrupt Monticello files into package-cache

On Sat, Aug 30, 2014 at 05:42:55PM +0200, Bernhard Pieber wrote:
> Hi David,
>
> Thanks for the answer!
>
> I am aware of this. Therefore I use Eliot's latest Cog VM 6063 for all my experiments. Here is how I get it:
> curl -O http://www.mirandabanda.org/files/Cog/VM/VM.r3063/Cog.app-14.32.3063.tgz
> gunzip -c Cog.app-14.32.3063.tgz | tar xopf - && rm Cog.app-14.32.3063.tgz
>
> The newest VM should not create corrupt zip files, right?

Yes, that VM should be good.

Dave

>
> Cheers,
> Bernhard
>
> Am 30.08.2014 um 16:24 schrieb David T. Lewis <[hidden email]>:
> > On Sat, Aug 30, 2014 at 04:03:23PM +0200, Bernhard Pieber wrote:
> >> I forgot to mention that I checked for some of the corrupt files if they are corrupt on source.squeak.org and it seems they are not, e.g.
> >>
> >> curl -O http://source.squeak.org/trunk/Collections-eem.571$nice.569$.mcd
> >> unzip -t Collections-eem.571$nice.569$.mcd
> >> Archive: Collections-eem.571(nice.569).mcd
> >> testing: package OK
> >> testing: version OK
> >> testing: base OK
> >> testing: old/source.st OK
> >> testing: new/source.st OK
> >> testing: patch.bin OK
> >> No errors detected in compressed data of Collections-eem.571(nice.569).mcd.
> >>
> >> It seems to follow that the corruptions happen locally because of broken (intermediate?) code which is executed during the trunk update.
> >
> > One of the updates in the update stream (Collections-eem.567) exposes a bug
> > that affects zip compression, and that therefore messes up MCZ processing.
> >
> > The only available workaround is to use a very recently compiled VM, built
> > from sources circa July or later. Without that, it may not be possible to
> > successfully update the trunk stream. The VMs distributed with Squeak 4.5 or
> > from squeakvm.org probably will not work.
> >
> > This information is not on Mantis, but it should be. Even after sorting out
> > the VM related issues, I think we still have issues remaining in the Smalltalk
> > zip processing (the fallback code for the primitives), so it may take some
> > time to get it all straightened out.
> >
> > Dave
> >
> >
>