Smalltalk › Squeak › Squeak - Dev

trunk update broken cuz there's an empty diffy version

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

13 messages Options

Eliot Miranda-2

trunk update broken cuz there's an empty diffy version

Hi Guys,

source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How do we fix it?

On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?
--
best,

Eliot

Bert Freudenberg

Re: trunk update broken cuz there's an empty diffy version

On 07.08.2014, at 19:52, Eliot Miranda <[hidden email]> wrote:

> Hi Guys,
>
> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How do we fix it?

Not empty, but broken:

bert$ unzip -t Morphic-eem.740$cmm.739$.mcd
Archive: Morphic-eem.740(cmm.739).mcd
testing: package OK
testing: version OK
testing: base OK
testing: old/source.st
error: invalid compressed data to inflate
testing: new/source.st
error: invalid compressed data to inflate
testing: patch.bin OK

Since the meta data and patch.bin are okay I'd expect this to load anyway ...

I remember seeing messages about broken ZIP files. Was there a fix? Did someone fix the squeaksource image, too? The MCDs are generated on the server.

> On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?

No pref, but you can comment out the diffy part in MCConfiguration>>versionNamed:for:from:.

- Bert -

smime.p7s (5K) Download Attachment

Eliot Miranda-2

Re: trunk update broken cuz there's an empty diffy version

On Mon, Aug 11, 2014 at 8:49 AM, Bert Freudenberg <[hidden email]> wrote:

On 07.08.2014, at 19:52, Eliot Miranda <[hidden email]> wrote:

> Hi Guys,
>
> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How do we fix it?

Not empty, but broken:

bert$ unzip -t Morphic-eem.740$cmm.739$.mcd
Archive: Morphic-eem.740(cmm.739).mcd
testing: package OK
testing: version OK
testing: base OK
testing: old/source.st
error: invalid compressed data to inflate
testing: new/source.st
error: invalid compressed data to inflate
testing: patch.bin OK

Since the meta data and patch.bin are okay I'd expect this to load anyway ...

It does now. I don't understand this stuff well enough to know quite what has happened :-(.

I remember seeing messages about broken ZIP files. Was there a fix? Did someone fix the squeaksource image, too? The MCDs are generated on the server.

There was an issue with the plugin that meant that adding inst vars to WriteStream and/or ReadStream would cause the plugin primitives to fail, falling back on the image code. There is an as-yet-undiagnosed bug that we suspect is to do with wide characters in the image code that causes it to fail to compress some sources (I suspect long sources that contain wide characters). Hence when I fixed (aStream on: coll from: nGreaterThanOne to: limit) contents by adding inst vars to WriteStream and ReadStream the image-level bug was provoked.

I've fixed the plugin in recent Cog VMs and IIRC David has folded the fix into recent interpreter VMs. But that still leaves the image code to debug.

> On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?

No pref, but you can comment out the diffy part in MCConfiguration>>versionNamed:for:from:.

Ah, OK. Mahalo! Any objections if I add a preference?

--
best,

Eliot

Bert Freudenberg

Re: trunk update broken cuz there's an empty diffy version

On 11.08.2014, at 20:06, Eliot Miranda <[hidden email]> wrote:

> On Mon, Aug 11, 2014 at 8:49 AM, Bert Freudenberg <[hidden email]> wrote:
>> > On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?
>>
>> No pref, but you can comment out the diffy part in MCConfiguration>>versionNamed:for:from:.
>>
> Ah, OK. Mahalo! Any objections if I add a preference?

Well, I generally dislike proliferating preferences. The code is supposed to automatically fall back to full versions if the more efficient diffy version doesn't load. If that fallback doesn't work I'd rather fix that problem rather than avoiding it with a preference. The diffs are much smaller than full MCDs, so for server performance and conserving bandwidth they are preferable.

- Bert -

smime.p7s (5K) Download Attachment

Eliot Miranda-2

Re: trunk update broken cuz there's an empty diffy version

On Mon, Aug 11, 2014 at 11:41 AM, Bert Freudenberg <[hidden email]> wrote:

On 11.08.2014, at 20:06, Eliot Miranda <[hidden email]> wrote:

> On Mon, Aug 11, 2014 at 8:49 AM, Bert Freudenberg <[hidden email]> wrote:

>> > On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?
>>
>> No pref, but you can comment out the diffy part in MCConfiguration>>versionNamed:for:from:.
>>
> Ah, OK. Mahalo! Any objections if I add a preference?

Well, I generally dislike proliferating preferences. The code is supposed to automatically fall back to full versions if the more efficient diffy version doesn't load. If that fallback doesn't work I'd rather fix that problem rather than avoiding it with a preference. The diffs are much smaller than full MCDs, so for server performance and conserving bandwidth they are preferable.

OK, makes sense.

--
best,

Eliot

Chris Muller-4

Re: trunk update broken cuz there's an empty diffy version

In reply to this post by Eliot Miranda-2

>> > On a related note is there any way to get the updater to download full
>> > packages, avoiding the diffy versions? e.g a preference?
>>
>> No pref, but you can comment out the diffy part in
>> MCConfiguration>>versionNamed:for:from:.
>
>
> Ah, OK. Mahalo! Any objections if I add a preference?

Wouldn't it be better for mcd's to just work than add a preference
just to get around a bug?

Eliot Miranda-2

Re: trunk update broken cuz there's an empty diffy version

On Mon, Aug 11, 2014 at 12:36 PM, Chris Muller <[hidden email]> wrote:

>> > On a related note is there any way to get the updater to download full
>> > packages, avoiding the diffy versions? e.g a preference?
>>
>> No pref, but you can comment out the diffy part in
>> MCConfiguration>>versionNamed:for:from:.
>
>
> Ah, OK. Mahalo! Any objections if I add a preference?

Wouldn't it be better for mcd's to just work than add a preference
just to get around a bug?

Sure, but when one is struggling to get a bootstrap working fixing bugs from elsewhere that just get in the way is not a viable option. I just wanted to rescue the Spur bootstrap. Anyway, looks like nothing needs to be done. It appears to be working now, and I'm happy to leave that a mystery for now.

best,

Eliot

Chris Muller-3

Re: trunk update broken cuz there's an empty diffy version

I wasn't expecting you to fix the bug, just simply thought Bert's solution (code tweak) would be sufficient for you until we can track it down. I assume you already have several code tweaks you make to your images to customize them for your own purpose (doesn't everyone?). so just one more temporarily.. I, for one, hope it does not remain a mystery much longer.

On Mon, Aug 11, 2014 at 4:08 PM, Eliot Miranda <[hidden email]> wrote:

On Mon, Aug 11, 2014 at 12:36 PM, Chris Muller <[hidden email]> wrote:

>> > On a related note is there any way to get the updater to download full
>> > packages, avoiding the diffy versions? e.g a preference?
>>
>> No pref, but you can comment out the diffy part in
>> MCConfiguration>>versionNamed:for:from:.
>
>
> Ah, OK. Mahalo! Any objections if I add a preference?

Wouldn't it be better for mcd's to just work than add a preference
just to get around a bug?

Sure, but when one is struggling to get a bootstrap working fixing bugs from elsewhere that just get in the way is not a viable option. I just wanted to rescue the Spur bootstrap. Anyway, looks like nothing needs to be done. It appears to be working now, and I'm happy to leave that a mystery for now.

--
best,
Eliot

bpi

Re: trunk update broken cuz there's an empty diffy version

In reply to this post by Eliot Miranda-2

Dear fellow Squeakers,

Here we go again. When I try to update the latest Squeak5.1-15113.image from http://ftp.squeak.org/5.1alpha using the latest CogSpur.r3684 on OS X with an empty package-cache I run into the Warning "About to serialize an empty diffy version.“ This time with SMLoader-cmm.85(ul.83).mcd. I wonder if I am the only one.

Cheers,
Bernhard

> Am 07.08.2014 um 19:52 schrieb Eliot Miranda <[hidden email]>:
>
> Hi Guys,
>
> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How do we fix it?
>
> On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?
> --
> best,
> Eliot
>

Levente Uzonyi

Re: trunk update broken cuz there's an empty diffy version

In this case the code in ul-83 and cmm-85 is the same, so the diff is
expected to be empty.
I wonder why is there a warning in such cases.

Levente

On Sun, 24 Apr 2016, Bernhard Pieber wrote:

> Dear fellow Squeakers,
>
> Here we go again. When I try to update the latest Squeak5.1-15113.image from http://ftp.squeak.org/5.1alpha using the latest CogSpur.r3684 on OS X with an empty package-cache I run into the Warning "About to serialize an empty diffy version.“ This time with SMLoader-cmm.85(ul.83).mcd. I wonder if I am the only one.
>
> Cheers,
> Bernhard
>
>> Am 07.08.2014 um 19:52 schrieb Eliot Miranda <[hidden email]>:
>>
>> Hi Guys,
>>
>> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How do we fix it?
>>
>> On a related note is there any way to get the updater to download full packages, avoiding the diffy versions? e.g a preference?
>> --
>> best,
>> Eliot
>>
>
>
>

Chris Muller-3

Re: trunk update broken cuz there's an empty diffy version

ul.83 is not a direct ancestor of cmm.85, so I would like to know what
code is creating a mcd between those two and why..

The warning about serializing empty mcz is there because we never
found the bug which was serializing empty mcz's. This warning is help
us identify when and why it occurs, and also to prevent the bug from
silently storing an empty mcz into the repository, possibly losing
code and/or causing harm to trunk.

On Sun, Apr 24, 2016 at 10:12 AM, Levente Uzonyi <[hidden email]> wrote:

> In this case the code in ul-83 and cmm-85 is the same, so the diff is
> expected to be empty.
> I wonder why is there a warning in such cases.
>
> Levente
>
>
> On Sun, 24 Apr 2016, Bernhard Pieber wrote:
>
>> Dear fellow Squeakers,
>>
>> Here we go again. When I try to update the latest Squeak5.1-15113.image
>> from http://ftp.squeak.org/5.1alpha using the latest CogSpur.r3684 on OS X
>> with an empty package-cache I run into the Warning "About to serialize an
>> empty diffy version.“ This time with SMLoader-cmm.85(ul.83).mcd. I wonder if
>> I am the only one.
>>
>> Cheers,
>> Bernhard
>>
>>> Am 07.08.2014 um 19:52 schrieb Eliot Miranda <[hidden email]>:
>>>
>>> Hi Guys,
>>>
>>> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How
>>> do we fix it?
>>>
>>> On a related note is there any way to get the updater to download full
>>> packages, avoiding the diffy versions? e.g a preference?
>>> --
>>> best,
>>> Eliot
>>>
>>
>>
>
>
>

Levente Uzonyi

Re: trunk update broken cuz there's an empty diffy version

On Sun, 24 Apr 2016, Chris Muller wrote:

> ul.83 is not a direct ancestor of cmm.85, so I would like to know what
> code is creating a mcd between those two and why..

It's like that because you reverted the changes of dtl.84.

>
> The warning about serializing empty mcz is there because we never
> found the bug which was serializing empty mcz's. This warning is help
> us identify when and why it occurs, and also to prevent the bug from
> silently storing an empty mcz into the repository, possibly losing
> code and/or causing harm to trunk.

But in this case the mcd has to be empty, because the code is identical,
so the warning is nothing but noise and a thing to block the automatic
update.

Levente

>
> On Sun, Apr 24, 2016 at 10:12 AM, Levente Uzonyi <[hidden email]> wrote:
>> In this case the code in ul-83 and cmm-85 is the same, so the diff is
>> expected to be empty.
>> I wonder why is there a warning in such cases.
>>
>> Levente
>>
>>
>> On Sun, 24 Apr 2016, Bernhard Pieber wrote:
>>
>>> Dear fellow Squeakers,
>>>
>>> Here we go again. When I try to update the latest Squeak5.1-15113.image
>>> from http://ftp.squeak.org/5.1alpha using the latest CogSpur.r3684 on OS X
>>> with an empty package-cache I run into the Warning "About to serialize an
>>> empty diffy version.“ This time with SMLoader-cmm.85(ul.83).mcd. I wonder if
>>> I am the only one.
>>>
>>> Cheers,
>>> Bernhard
>>>
>>>> Am 07.08.2014 um 19:52 schrieb Eliot Miranda <[hidden email]>:
>>>>
>>>> Hi Guys,
>>>>
>>>> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty. How
>>>> do we fix it?
>>>>
>>>> On a related note is there any way to get the updater to download full
>>>> packages, avoiding the diffy versions? e.g a preference?
>>>> --
>>>> best,
>>>> Eliot
>>>>
>>>
>>>
>>
>>
>>
>
>

Chris Muller-3

Re: trunk update broken cuz there's an empty diffy version

I fixed this case by replacing update-eem.333 with update-cmm.333
which loads dtl.84 in the interim. This should resolve Bernards
issue.

On Sun, Apr 24, 2016 at 1:36 PM, Levente Uzonyi <[hidden email]> wrote:

> On Sun, 24 Apr 2016, Chris Muller wrote:
>
>> ul.83 is not a direct ancestor of cmm.85, so I would like to know what
>> code is creating a mcd between those two and why..
>
>
> It's like that because you reverted the changes of dtl.84.
>
>>
>> The warning about serializing empty mcz is there because we never
>> found the bug which was serializing empty mcz's. This warning is help
>> us identify when and why it occurs, and also to prevent the bug from
>> silently storing an empty mcz into the repository, possibly losing
>> code and/or causing harm to trunk.
>
>
> But in this case the mcd has to be empty, because the code is identical, so
> the warning is nothing but noise and a thing to block the automatic update.
>
> Levente
>
>
>>
>> On Sun, Apr 24, 2016 at 10:12 AM, Levente Uzonyi <[hidden email]>
>> wrote:
>>>
>>> In this case the code in ul-83 and cmm-85 is the same, so the diff is
>>> expected to be empty.
>>> I wonder why is there a warning in such cases.
>>>
>>> Levente
>>>
>>>
>>> On Sun, 24 Apr 2016, Bernhard Pieber wrote:
>>>
>>>> Dear fellow Squeakers,
>>>>
>>>> Here we go again. When I try to update the latest Squeak5.1-15113.image
>>>> from http://ftp.squeak.org/5.1alpha using the latest CogSpur.r3684 on OS
>>>> X
>>>> with an empty package-cache I run into the Warning "About to serialize
>>>> an
>>>> empty diffy version.“ This time with SMLoader-cmm.85(ul.83).mcd. I
>>>> wonder if
>>>> I am the only one.
>>>>
>>>> Cheers,
>>>> Bernhard
>>>>
>>>>> Am 07.08.2014 um 19:52 schrieb Eliot Miranda <[hidden email]>:
>>>>>
>>>>> Hi Guys,
>>>>>
>>>>> source.squeak.org/trunk's Morphic-eem.740(cmm.739).mcd is empty.
>>>>> How
>>>>> do we fix it?
>>>>>
>>>>> On a related note is there any way to get the updater to download full
>>>>> packages, avoiding the diffy versions? e.g a preference?
>>>>> --
>>>>> best,
>>>>> Eliot
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>
>
>