Reproducible crashing the VM with a zipped fuel serialization

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
 
I revisited my MonticelloProjects experiment yesterday, and am still crashing
the vm when serializing to a compressed fuel file. I wrote about it at the time,
and then gave up as the vm was less stable during that period.

http://forum.world.st/Working-with-a-compressed-Fuel-file-td4869105.html

and it looks like the issue referenced:

http://forum.world.st/Re-Pharo-project-GZipWriteStream-crashing-my-VM-when-serializing-td4177725.html

To reproduce in a pharo6 image:

|project|
Gofer it
        smalltalkhubUser: 'StephanEggermont' project: 'MonticelloProjects';
        package: 'MonticelloProjects';
        load.

project := (Smalltalk at: #MCProject) new location:
'http://smalltalkhub.com/mc/StephanEggermont/Documentation/main'.
project read.
project saveToFile.

That reliably crashes in saveToFile.

It seems to be independent of 32/64 bit and mac/linux, as I tried some variations.

Stephan




crash.dmp (18K) Download Attachment
crash2.dmp (18K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
 
Any more information needed?

Stephan

Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Eliot Miranda-2
 



> On Jul 2, 2017, at 5:38 AM, stephan <[hidden email]> wrote:
>
> Any more information needed?

No, just time :-/

>
> Stephan
>
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
 

> Op 3 jul. 2017 om 05:20 heeft Eliot Miranda <[hidden email]> geschreven:
>
> No, just time :-/

No problem, if I find the time I'll try to create a smaller example.

Stephan
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
 
Still reproducible with current versions of Pharo7 and vm

Stephan



crash.dmp (16K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
 
On 21-09-17 14:51, stephan wrote:
> Still reproducible with current versions of Pharo7 and vm
I tried again after Pharo build #420

The Pull Request #654 was integrated: "Updated Fuel to 2.2.0 with the 64
bit adaptations"

Ubuntu 16.04LTS

vm: opensmalltalk / vm / cog / 201801121936 /
cog_linux64x64_pharo.cog.spur_201801121936

image: Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image





crash.dmp (11K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Stephan Eggermont-3
 
Hi Stephan,

I've opened an issue here: https://github.com/theseion/Fuel/issues/229.
I'll try to reproduce and see what could be the problem.

Cheers,
Max

On 13 January 2018 at 17:13:23, [hidden email] ([hidden email]) wrote:
<a href="http://airmail.calendar/2017-09-21 14:51:00 CEST" style="font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">On 21-09-17 14:51, stephan wrote:
> Still reproducible with current versions of Pharo7 and vm
I tried again after Pharo build #420

The Pull Request #654 was integrated: "Updated Fuel to 2.2.0 with the 64 
bit adaptations"

Ubuntu 16.04LTS

vm: opensmalltalk / vm / cog / <a href="tel://201801121936" style="font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">201801121936 / 
cog_linux64x64_pharo.cog.spur_201801121936

image: Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
 
Max Leske <[hidden email]> wrote:
> Hi Stephan,
>
> I've opened an issue here: https://github.com/theseion/Fuel/issues/229.
> I'll try to reproduce and see what could be the problem.

Thanks

Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Max Leske
 
So, I seem to have identified the problem. According to the RFC on Deflate (https://tools.ietf.org/html/rfc1951#page-5) the maximum number of non-compressible blocks is limited to (2^16) - 1:

"A compressed data set consists of a series of blocks, corresponding
   to successive blocks of input data.  The block sizes are arbitrary,
   except that non-compressible blocks are limited to 65,535 bytes."

Stephan's case exceeds this limit, which leads to an error when validating the match in DeflateStream>>validateMatchAt:from:to:. I assume that the compiler optimization removes the trap for the validation failure in this case (note that I simply used the Smalltalk implementation to trace the problem by replacing the primitive send with a super send).

With Fuel loaded (Pharo 7, 64-bit) the following produces a reproducible validation error, which is misleading, the problem is actually that the number of blocks was exceeded (I think that's because of the 16 bit bit mask that is being used in the implementation):

(FLGZipStrategy newWithTarget: FLByteArrayStreamStrategy new) writeStreamDo: [ :stream |
FLSerializer newDefault
serialize: ((1 to: (2 raisedTo: 16) - 82) collect: [ :i | 1 ]) asByteArray
on: stream ]

I'm not sure what should be done in this case but it appears to me that all remaining data could simply be appended to the last block, at least in the implementation I found in GCC (https://gcc.gnu.org/ml/gcc/2009-09/txt00001.txt):

/* Stored blocks are limited to 0xffff bytes, pending_buf is limited

     * to pending_buf_size, and each stored block has a 5 byte header:

     */

    ulg max_block_size = 0xffff;

    ulg max_start;



    if (max_block_size > s->pending_buf_size - 5) {

        max_block_size = s->pending_buf_size - 5;

    }


Where should we track this issue? Can someone take care to open an issue or should I do it?

In any case, I will close the Fuel issue, as this is clearly problem of the Deflate / Zip implementation.



Cheers,
Max

 

On 13 January 2018 at 18:31:09, Max Leske ([hidden email]) wrote:

Hi Stephan,

I've opened an issue here: https://github.com/theseion/Fuel/issues/229.
I'll try to reproduce and see what could be the problem.

Cheers,
Max

On 13 January 2018 at 17:13:23, [hidden email] ([hidden email]) wrote:
On 21-09-17 14:51, stephan wrote:
> Still reproducible with current versions of Pharo7 and vm
I tried again after Pharo build #420

The Pull Request #654 was integrated: "Updated Fuel to 2.2.0 with the 64 
bit adaptations"

Ubuntu 16.04LTS

vm: opensmalltalk / vm / cog / <a href="tel://201801121936" style="font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">201801121936 / 
cog_linux64x64_pharo.cog.spur_201801121936

image: Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Mariano Martinez Peck
 
Hi Max,

Great debugging capabilities! Since you are already in deep waters....could you check if it's the same problem I reported in the email "Problem with ZipArchive (probably bug due to large file)"?

Thanks a lot in advance,



On Sun, Jan 14, 2018 at 4:52 PM, Max Leske <[hidden email]> wrote:
 
So, I seem to have identified the problem. According to the RFC on Deflate (https://tools.ietf.org/html/rfc1951#page-5) the maximum number of non-compressible blocks is limited to (2^16) - 1:

"A compressed data set consists of a series of blocks, corresponding
   to successive blocks of input data.  The block sizes are arbitrary,

(FLGZipStrategy newWithTarget: FLByteArrayStreamStrategy new) writeStreamDo: [ :stream |
FLSerializer newDefault
serialize: ((1 to: (2 raisedTo: 16) - 82) collect: [ :i | 1 ]) asByteArray
on: stream ]

I'm not sure what should be done in this case but it appears to me that all remaining data could simply be appended to the last block, at least in the implementation I found in GCC (https://gcc.gnu.org/ml/gcc/2009-09/txt00001.txt):

/* Stored blocks are limited to 0xffff bytes, pending_buf is limited

     * to pending_buf_size, and each stored block has a 5 byte header:

     */

    ulg max_block_size = 0xffff;

    ulg max_start;



    if (max_block_size > s->pending_buf_size - 5) {

        max_block_size = s->pending_buf_size - 5;

    }


Where should we track this issue? Can someone take care to open an issue or should I do it?

In any case, I will close the Fuel issue, as this is clearly problem of the Deflate / Zip implementation.



Cheers,
Max

 

On 13 January 2018 at 18:31:09, Max Leske ([hidden email]) wrote:

Hi Stephan,

[hidden email]) wrote:

Hi Stephan,

I've opened an issue here: https://github.com/theseion/Fuel/issues/229.
I'll try to reproduce and see what could be the problem.

Cheers,
Max

On 13 January 2018 at 17:13:23, [hidden email] ([hidden email]) wrote:
On 21-09-17 14:51, stephan wrote:
> Still reproducible with current versions of Pharo7 and vm
I tried again after Pharo build #420

The Pull Request #654 was integrated: "Updated Fuel to 2.2.0 with the 64 
bit adaptations"

Ubuntu 16.04LTS

vm: opensmalltalk / vm / cog / <a href="tel://201801121936" style="font-family:helvetica;font-size:13px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank">201801121936 / 
cog_linux64x64_pharo.cog.spur_201801121936

image: Pharo-7.0.0-alpha.build.425.sha.eb0a6fb.arch.64bit.image




--
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Stephan Eggermont-3
In reply to this post by Max Leske
 
Max Leske <[hidden email]> wrote:
> So, I seem to have identified the problem.

Great!

> According to the RFC on Deflate (
> https://tools.ietf.org/html/rfc1951#page-5) the maximum number of
> non-compressible blocks is limited to (2^16) - 1:

Uhm, the size of each individual block or the number of blocks?

Stephan

Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Stephan Eggermont-3
 

On 15 January 2018 at 13:00:03, [hidden email] ([hidden email]) wrote:

Max Leske <[hidden email]> wrote:
> So, I seem to have identified the problem. 

Great!

> According to the RFC on Deflate (
> https://tools.ietf.org/html/rfc1951#page-5) the maximum number of
> non-compressible blocks is limited to (2^16) - 1:

Uhm, the size of each individual block or the number of blocks?

Yeah, my face looked the same when I read that :D. It's really the number of blocks.

Max



Stephan
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Stephan Eggermont-3
 
Hi Mariano,

I'll check that.

On 14 January 2018 at 21:10:17, [hidden email] ([hidden email]) wrote:

Hi Max,

Great debugging capabilities! Since you are already in deep waters....could
you check if it's the same problem I reported in the email "Problem with
ZipArchive (probably bug due to large file)"?

Thanks a lot in advance,
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Stephan Eggermont-3
 
I've taken the liberty of opening issues for the problem with Deflate.

Cheers,
Max

 

On 15 January 2018 at 19:21:20, [hidden email] ([hidden email]) wrote:

Send Vm-dev mailing list submissions to
[hidden email]

To subscribe or unsubscribe via the World Wide Web, visit
http://lists.squeakfoundation.org/mailman/listinfo/vm-dev
or, via email, send a message with subject or body 'help' to
[hidden email]

You can reach the person managing the list at
[hidden email]

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Vm-dev digest..."


Today's Topics:

1. Re: Reproducible crashing the VM with a zipped fuel
serialization (Max Leske)
2. [OpenSmalltalk/opensmalltalk-vm] 947faa: Clean up deployment
artifacts (GitHub)
3. Re: [OpenSmalltalk/opensmalltalk-vm] Build
FileAttributesPlugin on Linux (32 & 64 bit) (#191) (akgrant43)
4. [OpenSmalltalk/opensmalltalk-vm] bb1078: Make move Windows
compatible (GitHub)
5. [OpenSmalltalk/opensmalltalk-vm] f871a8: Minor
deployment-related fixes (GitHub)
6. [OpenSmalltalk/opensmalltalk-vm] 8b2701: Fix itimer naming
logic (GitHub)


----------------------------------------------------------------------

Message: 1
Date: Mon, 15 Jan 2018 13:38:51 +0100
From: Max Leske <[hidden email]>
To: [hidden email]
Subject: Re: [Vm-dev] Reproducible crashing the VM with a zipped fuel
serialization
Message-ID:
<[hidden email]>
Content-Type: text/plain; charset="utf-8"

On 15 January 2018 at 13:00:03, [hidden email] (
[hidden email]) wrote:

Max Leske <[hidden email]> wrote:
> So, I seem to have identified the problem.

Great!

> According to the RFC on Deflate (
> https://tools.ietf.org/html/rfc1951#page-5) the maximum number of
> non-compressible blocks is limited to (2^16) - 1:

Uhm, the size of each individual block or the number of blocks?

Yeah, my face looked the same when I read that :D. It's really the number
of blocks.

Max



Stephan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180115/00c3cfb9/attachment-0001.html>

------------------------------

Message: 2
Date: Mon, 15 Jan 2018 07:21:09 -0800
From: GitHub <[hidden email]>
To: [hidden email]
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 947faa: Clean up
deployment artifacts
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="utf-8"

Branch: refs/heads/fniephaus/artifacts
Home: https://github.com/OpenSmalltalk/opensmalltalk-vm
Commit: 947faacae15c502ddaa5afdc2f9ae1071299e73f
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/947faacae15c502ddaa5afdc2f9ae1071299e73f
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M .bintray.json
M .travis_build.sh
M .travis_deploy.sh
A deploy/generate-artifacts.sh

Log Message:
-----------
Clean up deployment artifacts

- Ship macOS VMs in a DMG file (#170)
- Clean up build junk that came with Windows VMs (#171)



------------------------------

Message: 3
Date: Mon, 15 Jan 2018 08:26:36 -0800
From: akgrant43 <[hidden email]>
To: OpenSmalltalk/opensmalltalk-vm
<[hidden email]>
Cc: Push <[hidden email]>
Subject: Re: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] Build
FileAttributesPlugin on Linux (32 & 64 bit) (#191)
Message-ID:
<OpenSmalltalk/opensmalltalk-vm/pull/191/push/[hidden email]>
Content-Type: text/plain; charset="utf-8"

@akgrant43 pushed 1 commit.

8050927 20294 Add FileAttributesPlugin to build


--
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/191/files/05396ec7c9ed5dac9c2faf6e6884086875421018..8050927e97b855100d699fa3f55f0712b664638a
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180115/58ba439e/attachment-0001.html>

------------------------------

Message: 4
Date: Mon, 15 Jan 2018 09:48:01 -0800
From: GitHub <[hidden email]>
To: [hidden email]
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] bb1078: Make move
Windows compatible
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="utf-8"

Branch: refs/heads/fniephaus/artifacts
Home: https://github.com/OpenSmalltalk/opensmalltalk-vm
Commit: bb107833638c6a55ee242ac82099cf765ff9c037
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/bb107833638c6a55ee242ac82099cf765ff9c037
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M .travis_build.sh

Log Message:
-----------
Make move Windows compatible


Commit: e21a70fa74930972e857ef6fb8c5d5c505883174
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/e21a70fa74930972e857ef6fb8c5d5c505883174
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M .travis_deploy.sh

Log Message:
-----------
Temporarily ignore deployment branches


Commit: 5c051b8d1fb8780085dc923cae9a70c3f45c2718
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/5c051b8d1fb8780085dc923cae9a70c3f45c2718
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M .travis_build.sh
M deploy/generate-artifacts.sh

Log Message:
-----------
No need to move folders around


Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/947faacae15c...5c051b8d1fb8

------------------------------

Message: 5
Date: Mon, 15 Jan 2018 09:57:57 -0800
From: GitHub <[hidden email]>
To: [hidden email]
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] f871a8: Minor
deployment-related fixes
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="utf-8"

Branch: refs/heads/fniephaus/artifacts
Home: https://github.com/OpenSmalltalk/opensmalltalk-vm
Commit: f871a824dcd4dfb241591859743b6be301c6e86a
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/f871a824dcd4dfb241591859743b6be301c6e86a
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M .travis_build.sh
M deploy/generate-artifacts.sh

Log Message:
-----------
Minor deployment-related fixes



------------------------------

Message: 6
Date: Mon, 15 Jan 2018 10:21:12 -0800
From: GitHub <[hidden email]>
To: [hidden email]
Subject: [Vm-dev] [OpenSmalltalk/opensmalltalk-vm] 8b2701: Fix itimer
naming logic
Message-ID:
<[hidden email]>

Content-Type: text/plain; charset="utf-8"

Branch: refs/heads/fniephaus/artifacts
Home: https://github.com/OpenSmalltalk/opensmalltalk-vm
Commit: 8b2701866da8c39ef4c0d9f622571c6bb0ceea2b
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/8b2701866da8c39ef4c0d9f622571c6bb0ceea2b
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M deploy/generate-artifacts.sh

Log Message:
-----------
Fix itimer naming logic


Commit: 7fa7d45ccae4d27ce314fca278ec01b2fd4a666d
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/7fa7d45ccae4d27ce314fca278ec01b2fd4a666d
Author: Fabio Niephaus <[hidden email]>
Date: 2018-01-15 (Mon, 15 Jan 2018)

Changed paths:
M .travis_build.sh

Log Message:
-----------
Restore slash for Windows


Compare: https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/f871a824dcd4...7fa7d45ccae4

------------------------------

Subject: Digest Footer

_______________________________________________
Vm-dev mailing list
[hidden email]
http://lists.squeakfoundation.org/mailman/listinfo/vm-dev


------------------------------

End of Vm-dev Digest, Vol 139, Issue 58
***************************************
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Max Leske
 
Hi Mariano,

The archive is in Zip64 format. The Zip format documentation (https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) states:

 4.4.21 total number of entries in the central dir on 
          this disk: (2 bytes)

      The number of central directory entries on this disk.
      If an archive is in ZIP64 format and the value in 
      this field is 0xFFFF, the size will be in the 
      corresponding 8 byte zip64 end of central 
      directory field.

That's exactly what I see, when I look at the data extracted from the central directory. As was pointed out, the implementation in Pharo doesn't support Zip64, although it would be nice to have.

Cheers,
Max

 

On 16 January 2018 at 07:40:05, Max Leske ([hidden email]) wrote:

Hi Mariano,

I'll check that.

On 14 January 2018 at 21:10:17, [hidden email] ([hidden email]) wrote:

Hi Max,

Great debugging capabilities! Since you are already in deep waters....could
you check if it's the same problem I reported in the email "Problem with
ZipArchive (probably bug due to large file)"?

Thanks a lot in advance,
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Mariano Martinez Peck
 


On Wed, Jan 17, 2018 at 6:41 PM, Max Leske <[hidden email]> wrote:
 
Hi Mariano,

The archive is in Zip64 format. The Zip format documentation (https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) states:


Hi Max,


thanks for the pointer. So what you are saying is the generated zip that github offers for downloading is a Zip64 bit format because otherwise it would haven't fit in the regular 32, right?
And of course, then we don't have Zip64 support.

Is that it?



 
 4.4.21 total number of entries in the central dir on 
          this disk: (2 bytes)

      The number of central directory entries on this disk.
      If an archive is in ZIP64 format and the value in 
      this field is 0xFFFF, the size will be in the 
      corresponding 8 byte zip64 end of central 
      directory field.

That's exactly what I see, when I look at the data extracted from the central directory. As was pointed out, the implementation in Pharo doesn't support Zip64, although it would be nice to have.

Cheers,
Max

 

On 16 January 2018 at 07:40:05, Max Leske ([hidden email]) wrote:

Hi Mariano,

I'll check that.

On 14 January 2018 at 21:10:17, [hidden email] ([hidden email]) wrote:

Hi Max,

Great debugging capabilities! Since you are already in deep waters....could
you check if it's the same problem I reported in the email "Problem with
ZipArchive (probably bug due to large file)"?

Thanks a lot in advance,




--
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible crashing the VM with a zipped fuel serialization

Max Leske
In reply to this post by Stephan Eggermont-3
 
On 19 January 2018 at 13:00:04, [hidden email] ([hidden email]) wrote:
Hi Max,


thanks for the pointer. So what you are saying is the generated zip that
github offers for downloading is a Zip64 bit format because otherwise it
would haven't fit in the regular 32, right?

Probably, yes. According to Wikipedia (https://en.wikipedia.org/wiki/Zip_(file_format)#ZIP64) Zip64 overcomes the 4 GiB limit as well as the limit of (2^16) - 1 members. In the case of this particular archive I'd say the number of members could possibly exceed the limit, although Zip64 can also be used without the limits being exceeded.

A quick test with a small Zip archive downloaded from Github shows that it can be extracted in Pharo, suggesting that Zip64 is only used by Github when either limit is exceeded.


Cheers,

Max


And of course, then we don't have Zip64 support.

Is that it?