SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other Smalltalk implementations

> Try this, after loading the Crypto packages.  I haven't tried it myself, yet, as I only got on this list and loaded Pharo a few days ago, but I will probably dig into it over the weekend.  I did load Crypto, though.
>
> http://www.squeaksource.com/Cryptography/SSL-mtf.16.mcz
>
> Is there a Pharo repository I could drop ported code into?
>
> Regards,
> Rob
>
>
>
>
>
> -----Original Message----- From: Sven Van Caekenberghe
> Sent: Tuesday, May 10, 2011 1:42 PM
> To: Pharo Project
> Subject: [Pharo-project] SSL/HTTPS - SecureSocketStream/SSLSession forPharo/Squeak and other Smalltalk implementations
>
> SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other Smalltalk implementations
>
>
> HTTP client and server functionality has become an essential part of any Smalltalk implementation. There exist various open source solutions delivering this functionality.
>
> Users quickly find out that HTTPS is not as readily available as HTTP. Although a solution exists in the form of SqueakSSL, it is not easily portable to Pharo and certainly not to other Smalltalk implementations. Furthermore, SqueakSSL is incomplete and more a working proof of concept than a high quality, high performance, maintainable piece of code, partially because of its dependence on the internals of SocketStream.
>
> Hence, some people wanting HTTPS on Pharo as well as one organisation have come foreword with a bounty to force a solution. Here are the requirements, a plan of action and a number of steps to build such a solution.
>
>
> First of all, it would be stupid not to start from the work already done for SqueakSSL. Any new solution should start from the existing plugin, from the existing set of primitives and from the current implementation of these primitives for the Windows, Mac OS X and Linux platforms.
>
> The Smalltalk part of the solution should be as platform independent as possible. We therefor need a new, clean implementation of SocketStream, with proper internal buffering and optimised versions of the bulk input and output primitives. The code should be time and space efficient. At first, only binary I/O is needed. Encoding/decoding is best done by wrapping the stream. Maybe a bivalent stream could be an option. The current ASCII support in quite limited.
>
> Next a SecureSocketStream can be implemented using this clean base combined with the SSLSession primitives. Here too, time and space efficiency are important although SSL will by definition be less efficient.
>
> Eventually, HTTP frameworks can then use this SecureSocketStream to actually implement HTTPS. Although the primary requirement is client functionality, server functionality should be provided as well.
>
> Next the existing issues with certificates need to be resolved, on all 3 platforms.
>
> Finally, the C code of the plugin for each of the 3 platforms needs to be cleaned up and improved where necessary. This requires contributions from developers who know and understand a platform as well as how SSL is supposed to work. Furthermore they have to understand and be motivated to work on Smalltalk plugins. Up to 3 different developers might be needed.
>
> The current implementation on Mac OS X does not seem to (directly) use the standard OpenSSL library from Linux although this library exists on Mac OS X. This could be an opportunity to reduce the number of implementations of the plugin to 2 API's.
>
> Needless to say the whole implementation should be open source licensed (MIT), be documented and have a number of units tests to cover it's main functionality as well edge cases encountered in the field.
>
> Friendly cooperation with other open source projects in the Smalltalk world is a plus. This project should benefit the whole community.
>
>
> The bounty should be split in different parts.
>
> The first part is setting up the new project, delivering the new SocketStream, SSLSession and SecureSocketStream, including the HTTPS client proof of concept. Traction within the community has to be developed.
>
> Next the issues with the plugin have to be identified. Then the work on the plugin code for each platform has to be defined and 3 more bounties could be offered.
>
> The goal should be to include the plugin in VM's by default.
>
> With a proper solution to the plugin issues, SSL server functionality should be no problem.
>
>
> I offer to take on part one and help manage the coordination of this project.
>
> So is there anyone capable and willing to help out on the plugin on on one or more platforms for part of the bounty ?
>
> Are there others who wish to raise the bounty ?
>
> We will need a couple of good beta testers who understand this field as well.
>
>
> Sven
>
>
>

> Try this, after loading the Crypto packages.  I haven't tried it myself, yet, as I only got on this list and loaded Pharo a few days ago, but I will probably dig into it over the weekend.  I did load Crypto, though.
>
> http://www.squeaksource.com/Cryptography/SSL-mtf.16.mcz
>
> Is there a Pharo repository I could drop ported code into?
>
> Regards,
> Rob
>
>
>
>
>
> -----Original Message----- From: Sven Van Caekenberghe
> Sent: Tuesday, May 10, 2011 1:42 PM
> To: Pharo Project
> Subject: [Pharo-project] SSL/HTTPS - SecureSocketStream/SSLSession forPharo/Squeak and other Smalltalk implementations
>
> SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other Smalltalk implementations
>
>
> HTTP client and server functionality has become an essential part of any Smalltalk implementation. There exist various open source solutions delivering this functionality.
>
> Users quickly find out that HTTPS is not as readily available as HTTP. Although a solution exists in the form of SqueakSSL, it is not easily portable to Pharo and certainly not to other Smalltalk implementations. Furthermore, SqueakSSL is incomplete and more a working proof of concept than a high quality, high performance, maintainable piece of code, partially because of its dependence on the internals of SocketStream.
>
> Hence, some people wanting HTTPS on Pharo as well as one organisation have come foreword with a bounty to force a solution. Here are the requirements, a plan of action and a number of steps to build such a solution.
>
>
> First of all, it would be stupid not to start from the work already done for SqueakSSL. Any new solution should start from the existing plugin, from the existing set of primitives and from the current implementation of these primitives for the Windows, Mac OS X and Linux platforms.
>
> The Smalltalk part of the solution should be as platform independent as possible. We therefor need a new, clean implementation of SocketStream, with proper internal buffering and optimised versions of the bulk input and output primitives. The code should be time and space efficient. At first, only binary I/O is needed. Encoding/decoding is best done by wrapping the stream. Maybe a bivalent stream could be an option. The current ASCII support in quite limited.
>
> Next a SecureSocketStream can be implemented using this clean base combined with the SSLSession primitives. Here too, time and space efficiency are important although SSL will by definition be less efficient.
>
> Eventually, HTTP frameworks can then use this SecureSocketStream to actually implement HTTPS. Although the primary requirement is client functionality, server functionality should be provided as well.
>
> Next the existing issues with certificates need to be resolved, on all 3 platforms.
>
> Finally, the C code of the plugin for each of the 3 platforms needs to be cleaned up and improved where necessary. This requires contributions from developers who know and understand a platform as well as how SSL is supposed to work. Furthermore they have to understand and be motivated to work on Smalltalk plugins. Up to 3 different developers might be needed.
>
> The current implementation on Mac OS X does not seem to (directly) use the standard OpenSSL library from Linux although this library exists on Mac OS X. This could be an opportunity to reduce the number of implementations of the plugin to 2 API's.
>
> Needless to say the whole implementation should be open source licensed (MIT), be documented and have a number of units tests to cover it's main functionality as well edge cases encountered in the field.
>
> Friendly cooperation with other open source projects in the Smalltalk world is a plus. This project should benefit the whole community.
>
>
> The bounty should be split in different parts.
>
> The first part is setting up the new project, delivering the new SocketStream, SSLSession and SecureSocketStream, including the HTTPS client proof of concept. Traction within the community has to be developed.
>
> Next the issues with the plugin have to be identified. Then the work on the plugin code for each platform has to be defined and 3 more bounties could be offered.
>
> The goal should be to include the plugin in VM's by default.
>
> With a proper solution to the plugin issues, SSL server functionality should be no problem.
>
>
> I offer to take on part one and help manage the coordination of this project.
>
> So is there anyone capable and willing to help out on the plugin on on one or more platforms for part of the bounty ?
>
> Are there others who wish to raise the bounty ?
>
> We will need a couple of good beta testers who understand this field as well.
>
>
> Sven
>
>
>

> Try this, after loading the Crypto packages.  I haven't tried it myself,
> yet, as I only got on this list and loaded Pharo a few days ago, but I
> will probably dig into it over the weekend.  I did load Crypto, though.
>
> http://www.squeaksource.com/Cryptography/SSL-mtf.16.mcz
>
> Is there a Pharo repository I could drop ported code into?
>
> Regards,
> Rob
>
>
>
>
>
> -----Original Message----- From: Sven Van Caekenberghe
> Sent: Tuesday, May 10, 2011 1:42 PM
> To: Pharo Project
> Subject: [Pharo-project] SSL/HTTPS - SecureSocketStream/SSLSession
> forPharo/Squeak and other Smalltalk implementations
>
> SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other
> Smalltalk implementations
>
>
> HTTP client and server functionality has become an essential part of any
> Smalltalk implementation. There exist various open source solutions
> delivering this functionality.
>
> Users quickly find out that HTTPS is not as readily available as HTTP.
> Although a solution exists in the form of SqueakSSL, it is not easily
> portable to Pharo and certainly not to other Smalltalk implementations.
> Furthermore, SqueakSSL is incomplete and more a working proof of concept
> than a high quality, high performance, maintainable piece of code,
> partially because of its dependence on the internals of SocketStream.
>
> Hence, some people wanting HTTPS on Pharo as well as one organisation have
> come foreword with a bounty to force a solution. Here are the
> requirements, a plan of action and a number of steps to build such a
> solution.
>
>
> First of all, it would be stupid not to start from the work already done
> for SqueakSSL. Any new solution should start from the existing plugin,
> from the existing set of primitives and from the current implementation of
> these primitives for the Windows, Mac OS X and Linux platforms.
>
> The Smalltalk part of the solution should be as platform independent as
> possible. We therefor need a new, clean implementation of SocketStream,
> with proper internal buffering and optimised versions of the bulk input
> and output primitives. The code should be time and space efficient. At
> first, only binary I/O is needed. Encoding/decoding is best done by
> wrapping the stream. Maybe a bivalent stream could be an option. The
> current ASCII support in quite limited.
>
> Next a SecureSocketStream can be implemented using this clean base
> combined with the SSLSession primitives. Here too, time and space
> efficiency are important although SSL will by definition be less
> efficient.
>
> Eventually, HTTP frameworks can then use this SecureSocketStream to
> actually implement HTTPS. Although the primary requirement is client
> functionality, server functionality should be provided as well.
>
> Next the existing issues with certificates need to be resolved, on all 3
> platforms.
>
> Finally, the C code of the plugin for each of the 3 platforms needs to be
> cleaned up and improved where necessary. This requires contributions from
> developers who know and understand a platform as well as how SSL is
> supposed to work. Furthermore they have to understand and be motivated to
> work on Smalltalk plugins. Up to 3 different developers might be needed.
>
> The current implementation on Mac OS X does not seem to (directly) use the
> standard OpenSSL library from Linux although this library exists on Mac OS
> X. This could be an opportunity to reduce the number of implementations of
> the plugin to 2 API's.
>
> Needless to say the whole implementation should be open source licensed
> (MIT), be documented and have a number of units tests to cover it's main
> functionality as well edge cases encountered in the field.
>
> Friendly cooperation with other open source projects in the Smalltalk
> world is a plus. This project should benefit the whole community.
>
>
> The bounty should be split in different parts.
>
> The first part is setting up the new project, delivering the new
> SocketStream, SSLSession and SecureSocketStream, including the HTTPS
> client proof of concept. Traction within the community has to be
> developed.
>
> Next the issues with the plugin have to be identified. Then the work on
> the plugin code for each platform has to be defined and 3 more bounties
> could be offered.
>
> The goal should be to include the plugin in VM's by default.
>
> With a proper solution to the plugin issues, SSL server functionality
> should be no problem.
>
>
> I offer to take on part one and help manage the coordination of this
> project.
>
> So is there anyone capable and willing to help out on the plugin on on one
> or more platforms for part of the bounty ?
>
> Are there others who wish to raise the bounty ?
>
> We will need a couple of good beta testers who understand this field as
> well.
>
>
> Sven
>
>
>

> It is a complete implementation with the bare minimum of supported
> algorithms, based on the spec.   It is 100% in Smalltalk.  It worked last
> time I tested in, several years ago.  I will be able to tell you more this
> weekend.   Others have been adding fixes/small features since, but not a
> lot - the base is pretty solid.  I painstakingly followed the RFC for
> SSL - I think I read it over 20 times!  The only issue would be algorithm
> support. I never benchmarked it, but seeing the difference in performance
> between Smlltalk crypto code like 3DES and plugin code, there is a
> SIGNIFICANT improvement (10x?).   I do not know the state of Profiling
> under Pharo.  If someone could help set me up, then I will profile it this
> weekend.  I imagine both performance and certification were the reasons
> for SqueakSSL.
>
> There is also an SSH implementation I wrote that does dynamic window
> resizing on net latency.

> Rob,
>
> On 12 May 2011, at 16:38, Rob Withers wrote:
>
> > It is a complete implementation with the bare minimum of supported algorithms, based on the spec.   It is 100% in Smalltalk.  It worked last time I tested in, several years ago.  I will be able to tell you more this weekend.   Others have been adding fixes/small features since, but not a lot - the base is pretty solid.  I painstakingly followed the RFC for SSL - I think I read it over 20 times!  The only issue would be algorithm support. I never benchmarked it, but seeing the difference in performance between Smlltalk crypto code like 3DES and plugin code, there is a SIGNIFICANT improvement (10x?).   I do not know the state of Profiling under Pharo.  If someone could help set me up, then I will profile it this weekend.  I imagine both performance and certification were the reasons for SqueakSSL.
> >
> > There is also an SSH implementation I wrote that does dynamic window resizing on net latency.
>
> It would be *very cool* if that code base could be ported to Pharo.
> It is really quite impressive that you did all this from the spec.
>
> I think that it would be possible to adjust my plan a bit, so that eventually, my code would use either the native plugin or your Smalltalk implementation. Then, we can compare features, functionality, bugs, etc..
>
> Does that sound like a good idea ?

> How can I use the pharo tag from inside of Pharo, with the Monticello Browser?
>
> Rob
>
> -----Original Message----- From: Stéphane Ducasse
> Sent: Thursday, May 12, 2011 10:10 AM
> To: [hidden email]
> Subject: Re: [Pharo-project] SSL/HTTPS - SecureSocketStream/SSLSessionforPharo/Squeak and other Smalltalk implementations
>
> Or you keep
> Cryptology and publish a pharo tag.
>
> Stef
>
> On May 12, 2011, at 3:52 PM, Rob Withers wrote:
>
>> Try this, after loading the Crypto packages.  I haven't tried it myself, yet, as I only got on this list and loaded Pharo a few days ago, but I will probably dig into it over the weekend.  I did load Crypto, though.
>>
>> http://www.squeaksource.com/Cryptography/SSL-mtf.16.mcz
>>
>> Is there a Pharo repository I could drop ported code into?
>>
>> Regards,
>> Rob
>>
>>
>>
>>
>>
>> -----Original Message----- From: Sven Van Caekenberghe
>> Sent: Tuesday, May 10, 2011 1:42 PM
>> To: Pharo Project
>> Subject: [Pharo-project] SSL/HTTPS - SecureSocketStream/SSLSession forPharo/Squeak and other Smalltalk implementations
>>
>> SSL/HTTPS - SecureSocketStream/SSLSession for Pharo/Squeak and other Smalltalk implementations
>>
>>
>> HTTP client and server functionality has become an essential part of any Smalltalk implementation. There exist various open source solutions delivering this functionality.
>>
>> Users quickly find out that HTTPS is not as readily available as HTTP. Although a solution exists in the form of SqueakSSL, it is not easily portable to Pharo and certainly not to other Smalltalk implementations. Furthermore, SqueakSSL is incomplete and more a working proof of concept than a high quality, high performance, maintainable piece of code, partially because of its dependence on the internals of SocketStream.
>>
>> Hence, some people wanting HTTPS on Pharo as well as one organisation have come foreword with a bounty to force a solution. Here are the requirements, a plan of action and a number of steps to build such a solution.
>>
>>
>> First of all, it would be stupid not to start from the work already done for SqueakSSL. Any new solution should start from the existing plugin, from the existing set of primitives and from the current implementation of these primitives for the Windows, Mac OS X and Linux platforms.
>>
>> The Smalltalk part of the solution should be as platform independent as possible. We therefor need a new, clean implementation of SocketStream, with proper internal buffering and optimised versions of the bulk input and output primitives. The code should be time and space efficient. At first, only binary I/O is needed. Encoding/decoding is best done by wrapping the stream. Maybe a bivalent stream could be an option. The current ASCII support in quite limited.
>>
>> Next a SecureSocketStream can be implemented using this clean base combined with the SSLSession primitives. Here too, time and space efficiency are important although SSL will by definition be less efficient.
>>
>> Eventually, HTTP frameworks can then use this SecureSocketStream to actually implement HTTPS. Although the primary requirement is client functionality, server functionality should be provided as well.
>>
>> Next the existing issues with certificates need to be resolved, on all 3 platforms.
>>
>> Finally, the C code of the plugin for each of the 3 platforms needs to be cleaned up and improved where necessary. This requires contributions from developers who know and understand a platform as well as how SSL is supposed to work. Furthermore they have to understand and be motivated to work on Smalltalk plugins. Up to 3 different developers might be needed.
>>
>> The current implementation on Mac OS X does not seem to (directly) use the standard OpenSSL library from Linux although this library exists on Mac OS X. This could be an opportunity to reduce the number of implementations of the plugin to 2 API's.
>>
>> Needless to say the whole implementation should be open source licensed (MIT), be documented and have a number of units tests to cover it's main functionality as well edge cases encountered in the field.
>>
>> Friendly cooperation with other open source projects in the Smalltalk world is a plus. This project should benefit the whole community.
>>
>>
>> The bounty should be split in different parts.
>>
>> The first part is setting up the new project, delivering the new SocketStream, SSLSession and SecureSocketStream, including the HTTPS client proof of concept. Traction within the community has to be developed.
>>
>> Next the issues with the plugin have to be identified. Then the work on the plugin code for each platform has to be defined and 3 more bounties could be offered.
>>
>> The goal should be to include the plugin in VM's by default.
>>
>> With a proper solution to the plugin issues, SSL server functionality should be no problem.
>>
>>
>> I offer to take on part one and help manage the coordination of this project.
>>
>> So is there anyone capable and willing to help out on the plugin on on one or more platforms for part of the bounty ?
>>
>> Are there others who wish to raise the bounty ?
>>
>> We will need a couple of good beta testers who understand this field as well.
>>
>>
>> Sven
>>
>>
>>
>
>
>

> But the performance is an issue. And I think that there was a discussion
> several years ago that lead to choice a plugin instead of the
> all-smalltalk code (independently of the queality of the smalltalk
> code). Also a point was made about the maintainability of the smalltalk
> code with respect to a library of the underlying OS, with respect to
> CERT issues and 0-day exploits. given the few resources in the community
> for simpler (relative to crypto) projects like *completion, RB and so, I
> think is a wise decision to use the proved, tested and maintained OS
> libraries through a plugin part of the standard VM. Of course, the
> smalltalk implementation can be used as a fallback for platforms where a
> plugin isn't available.

> "Miguel Cobá"<[hidden email]> wrote:
>> But the performance is an issue. And I think that there was a discussion
>> several years ago that lead to choice a plugin instead of the
>> all-smalltalk code (independently of the queality of the smalltalk
>> code). Also a point was made about the maintainability of the smalltalk
>> code with respect to a library of the underlying OS, with respect to
>> CERT issues and 0-day exploits. given the few resources in the community
>> for simpler (relative to crypto) projects like *completion, RB and so, I
>> think is a wise decision to use the proved, tested and maintained OS
>> libraries through a plugin part of the standard VM. Of course, the
>> smalltalk implementation can be used as a fallback for platforms where a
>> plugin isn't available.
>
> These are certainly important things to consider, but I think the arguments need to be more specific and thought through. There are probably dozens of tried and proven libraries for pretty much anything out there, so does that mean there's no point implementing anything? Moreover an external interface is not maintenance free either, they can be surprisingly brittle in many ways and libraries tend to have their own quirks that you may need to accommodate. Anyway, I'd like to focus on the SSL performance argument here.
>
> The primary bottleneck of an all-smalltalk SSL implementation are the hashes and symmetric ciphers, which have to chew on every byte of data going through a connection and are too slow in Smalltalk (orders of magnitude slower). The public key algorithms would be the second bottleneck but those are not nearly as bad (more like 5 - 10 times slower depending on the size of the key, and that's primarily because of lack of efficient Integer>>raisedTo:modulo: primitive that peforms well for large integers). Moreover the public key operations happen when a connection is being established, they are not involved in actual data transfer, so long lasting heavily used connections will offset their cost quite easily. To sum up the pub-key ops are no concern on the client, they will likely be a concern on a server (unless it doesn't need to handle many concurrent connections in rapid sequence). So with all that said, I'm confident that a Smalltalk implementation of SSL/TLS can perform as well as any other as long as the hashes and ciphers are farmed out (http://www.cincomsmalltalk.com/userblogs/cst/blogView?printTitle=Xtreams-SSH2).
>

> Of course that doesn't automatically mean that is should be implemented in Smalltalk. Given that you currently have both available in Squeak (native, Cryptography, and external, SqueakSSL) see how they stack up to each other. How easy are they to configure and use, where do they work and where they don't, do they provide the same features. I'd be curious how the external implementation deals with the issue of verification of the server certificate (I don't mean checking that it's valid, the library can presumably do that for you, but how does the application check that the certificate identifies the entity that it wants to reach?). What about session management (caching, resumption control, load-balancing etc)?
>
> Anyway, lots of things to consider indeed. One thing that wrapping an external library usually doesn't give you is thorough understanding of the mechanics and capabilities of the protocol, which is double-plus important with security. You won't get a secure connection if you don't configure/use the protocol correctly. And you have to configure it to some degree, there isn't an appropriate one-size fits all default. Arguably you shouldn't need to actually implement one to get that understanding, but it sure does help a lot :-).
>
> My 2c,
>
> Martin
>
>
>