[ANN] sha256 checksum for Pharo6 downloads

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] sha256 checksum for Pharo6 downloads

Marcus Denker-4
Hi,

A tiny first step: I added sha256 chechsums for all downloads created by the Pharo6 build process


This step:


now creates .sha256.txt files, e.g for the mac:


This allows to check that downloads from the file server are indeed the same files that the build server created.


As I said, just a very first step.

TODO:
- pgp signatures 
- insert into website
- SSL for files.pharo.org
- do it Pharo7
- ….

So: more to come!

Marcus
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

henry
Hi Marcus,

Are you using SHA256 inside Pharo to generate the checksum files? If so were you planning to use the OpenPGP by hmmosner, in the Cryptography repository for PGP signatures?

I am curious as to integration of the newest Cryptography package there into Pharo. There are two Undeclared: SoundRecorder and FillInTheBlank, though I do not know if such would be a part of a Pharo-Squeak compatibility layer.

More pertinent would be what to do with the HashFunctions that already reside in the Pharo image. If the decision was to include the entire Cryptography into base, then making the HashFunctions not go Obselete would be the objective.

Currently, SHA1, SHA256 and MD5 all get redefined by Cryptography-zzz.111.mcz loading.

What do you think about bringing Cryptography up to date for Pharo and what would be needed. It would help to seek common ground between Squeak and Pharo, yes?

HH


On Tue, Oct 24, 2017 at 11:34, Marcus Denker <marcus.denker@...> wrote:
Hi,

A tiny first step: I added sha256 chechsums for all downloads created by the Pharo6 build process


This step:


now creates .sha256.txt files, e.g for the mac:


This allows to check that downloads from the file server are indeed the same files that the build server created.


As I said, just a very first step.

TODO:
- pgp signatures 
- insert into website
- SSL for files.pharo.org
- do it Pharo7
- ….

So: more to come!

Marcus
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Marcus Denker-4


> On 24 Oct 2017, at 17:56, henry <[hidden email]> wrote:
>
> Hi Marcus,
>
> Are you using SHA256 inside Pharo to generate the checksum files? If so were you planning to use the OpenPGP by hmmosner, in the Cryptography repository for PGP signatures?
>
No, this just uses sha56sum on Linux.


        Marcus
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Marcus Denker-4
In reply to this post by Marcus Denker-4
How to validate a Pharo6 download with the example of the mac download:

1) get the checksum file (note: uses SSL):
        https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt

2) download Pharo:
        http://files.pharo.org/platform/Pharo6.1-mac.zip

with sha256sum installed, you can do:

        sha256sum -c Pharo6.1-mac.zip.sha256.txt

and it prints:

Pharo6.1-mac.zip: OK

        Marcus


> On 24 Oct 2017, at 17:34, Marcus Denker <[hidden email]> wrote:
>
> Hi,
>
> A tiny first step: I added sha256 chechsums for all downloads created by the Pharo6 build process
>
> https://ci.inria.fr/pharo/
>
> This step:
>
> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/
>
> now creates .sha256.txt files, e.g for the mac:
>
> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt
>
> This allows to check that downloads from the file server are indeed the same files that the build server created.
> http://files.pharo.org/platform/
> http://files.pharo.org/image/60/
>
>
> As I said, just a very first step.
>
> TODO:
> - pgp signatures
> - insert into website
> - SSL for files.pharo.org
> - do it Pharo7
> - ….
>
> So: more to come!
>
> Marcus


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Sven Van Caekenberghe-2
Great!

And here is how to do it in Pharo:

signature := 'https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt' asUrl retrieveContents findTokens: Character separators.
hash := signature first.
signedFile := signature second.
url := 'http://files.pharo.org/platform/Pharo6.1-mac.zip' asUrl.
ZnClient new url: url; downloadTo: FileLocator temp. "somewhat slow"
file := FileLocator temp / url file.
self assert: file exists.
self assert: (signedFile match: url file).
file readStreamDo: [ :in | sha256 := SHA256 hashStream: in ]. "very slow"
self assert: (hash sameAs: sha256 hex).

Would it not be cleaner if the signature was next to the resource ? Like

http://files.pharo.org/platform/Pharo6.1-mac.zip.sha256.txt

Or is that the next step ?

> On 25 Oct 2017, at 09:53, Marcus Denker <[hidden email]> wrote:
>
> How to validate a Pharo6 download with the example of the mac download:
>
> 1) get the checksum file (note: uses SSL):
> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt
>
> 2) download Pharo:
> http://files.pharo.org/platform/Pharo6.1-mac.zip
>
> with sha256sum installed, you can do:
>
> sha256sum -c Pharo6.1-mac.zip.sha256.txt
>
> and it prints:
>
> Pharo6.1-mac.zip: OK
>
> Marcus
>
>
>> On 24 Oct 2017, at 17:34, Marcus Denker <[hidden email]> wrote:
>>
>> Hi,
>>
>> A tiny first step: I added sha256 chechsums for all downloads created by the Pharo6 build process
>>
>> https://ci.inria.fr/pharo/
>>
>> This step:
>>
>> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/
>>
>> now creates .sha256.txt files, e.g for the mac:
>>
>> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt
>>
>> This allows to check that downloads from the file server are indeed the same files that the build server created.
>> http://files.pharo.org/platform/
>> http://files.pharo.org/image/60/
>>
>>
>> As I said, just a very first step.
>>
>> TODO:
>> - pgp signatures
>> - insert into website
>> - SSL for files.pharo.org
>> - do it Pharo7
>> - ….
>>
>> So: more to come!
>>
>> Marcus
>
>


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Marcus Denker-4


> On 25 Oct 2017, at 10:23, Sven Van Caekenberghe <[hidden email]> wrote:
>
> Great!
>
> And here is how to do it in Pharo:
>
> signature := 'https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt' asUrl retrieveContents findTokens: Character separators.
> hash := signature first.
> signedFile := signature second.
> url := 'http://files.pharo.org/platform/Pharo6.1-mac.zip' asUrl.
> ZnClient new url: url; downloadTo: FileLocator temp. "somewhat slow"
> file := FileLocator temp / url file.
> self assert: file exists.
> self assert: (signedFile match: url file).
> file readStreamDo: [ :in | sha256 := SHA256 hashStream: in ]. "very slow"
> self assert: (hash sameAs: sha256 hex).
>
Nice!

> Would it not be cleaner if the signature was next to the resource ? Like
>
> http://files.pharo.org/platform/Pharo6.1-mac.zip.sha256.txt
>
> Or is that the next step ?
>

Already there. But a signature like that is not a guarantee if it is downloaded from the same server… especially of that server does not
use SSL…

The “stack vector” that a checksum protects against is the compromise of a download server, especially untrusted mirrors. For that,
the checksum needs to come from some other (trusted) source. E.g. normally it is inlined on the download website.

But of course these things are never 100% guarantees, they just make it harder to do bad things.

        Marcus


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Marcus Denker-4


> On 25 Oct 2017, at 10:33, Marcus Denker <[hidden email]> wrote:
>
>
>
>> On 25 Oct 2017, at 10:23, Sven Van Caekenberghe <[hidden email]> wrote:
>>
>> Great!
>>
>> And here is how to do it in Pharo:
>>
>> signature := 'https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt' asUrl retrieveContents findTokens: Character separators.
>> hash := signature first.
>> signedFile := signature second.
>> url := 'http://files.pharo.org/platform/Pharo6.1-mac.zip' asUrl.
>> ZnClient new url: url; downloadTo: FileLocator temp. "somewhat slow"
>> file := FileLocator temp / url file.
>> self assert: file exists.
>> self assert: (signedFile match: url file).
>> file readStreamDo: [ :in | sha256 := SHA256 hashStream: in ]. "very slow"
>> self assert: (hash sameAs: sha256 hex).
>>
> Nice!
>
>> Would it not be cleaner if the signature was next to the resource ? Like
>>
>> http://files.pharo.org/platform/Pharo6.1-mac.zip.sha256.txt
>>
>> Or is that the next step ?
>>
>
> Already there. But a signature like that is not a guarantee if it is downloaded from the same server… especially of that server does not
> use SSL…
>
> The “stack vector”
      ^^^^ Attack vector
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Sven Van Caekenberghe-2
In reply to this post by Marcus Denker-4


> On 25 Oct 2017, at 10:33, Marcus Denker <[hidden email]> wrote:
>
>
>
>> On 25 Oct 2017, at 10:23, Sven Van Caekenberghe <[hidden email]> wrote:
>>
>> Great!
>>
>> And here is how to do it in Pharo:
>>
>> signature := 'https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-5-Publish/lastSuccessfulBuild/artifact/Pharo6.1-mac.zip.sha256.txt' asUrl retrieveContents findTokens: Character separators.
>> hash := signature first.
>> signedFile := signature second.
>> url := 'http://files.pharo.org/platform/Pharo6.1-mac.zip' asUrl.
>> ZnClient new url: url; downloadTo: FileLocator temp. "somewhat slow"
>> file := FileLocator temp / url file.
>> self assert: file exists.
>> self assert: (signedFile match: url file).
>> file readStreamDo: [ :in | sha256 := SHA256 hashStream: in ]. "very slow"
>> self assert: (hash sameAs: sha256 hex).
>>
> Nice!
>
>> Would it not be cleaner if the signature was next to the resource ? Like
>>
>> http://files.pharo.org/platform/Pharo6.1-mac.zip.sha256.txt
>>
>> Or is that the next step ?
>>
>
> Already there. But a signature like that is not a guarantee if it is downloaded from the same server… especially of that server does not
> use SSL…
>
> The “stack vector” that a checksum protects against is the compromise of a download server, especially untrusted mirrors. For that,
> the checksum needs to come from some other (trusted) source. E.g. normally it is inlined on the download website.
>
> But of course these things are never 100% guarantees, they just make it harder to do bad things.

Ah, OK, I understand, I just think that a shorter/simpler/easier-to-remember URL for the signature would be better.

> Marcus


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Marcus Denker-4


Would it not be cleaner if the signature was next to the resource ? Like

http://files.pharo.org/platform/Pharo6.1-mac.zip.sha256.txt

Or is that the next step ?


Already there. But a signature like that is not a guarantee if it is downloaded from the same server… especially of that server does not
use SSL…

The “stack vector” that a checksum protects against is the compromise of a download server, especially untrusted mirrors. For that,
the checksum needs to come from some other (trusted) source. E.g. normally it is inlined on the download website.

But of course these things are never 100% guarantees, they just make it harder to do bad things.

Ah, OK, I understand, I just think that a shorter/simpler/easier-to-remember URL for the signature would be better.

I will put them on pharo.org later, too (in a dedicated directory). And link them from the download page.

Marcus

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Sean P. DeNigris
Administrator
In reply to this post by Sven Van Caekenberghe-2
Sven Van Caekenberghe-2 wrote
> And here is how to do it in Pharo…

It would be great to add something like this to Launcher



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] sha256 checksum for Pharo6 downloads

Sven Van Caekenberghe-2


> On 25 Oct 2017, at 14:02, Sean P. DeNigris <[hidden email]> wrote:
>
> Sven Van Caekenberghe-2 wrote
>> And here is how to do it in Pharo…
>
> It would be great to add something like this to Launcher

Yeah, but

 file readStreamDo: [ :in | sha256 := SHA256 hashStream: in ].

is very slow (done completely in Pharo, large file), so it would need a good progress bar.

We would also need a canonical place to get the signatures from (like Marcus explained, best another, secure server).

Sven