Sockstream vs ZnSocketStream

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Sockstream vs ZnSocketStream

Luc Fabresse
Hi all,

 I am *experimenting* with the replacement of the current network layer with OCEAN.
 My current plan is to make Zinc tests green while working on the top of OCEAN.
 And investigate the removal of all the current Network classes (does it play?).
 After looking at Zinc code, there are few dependencies.
 Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils.
 It seems to play so far (green tests).

 But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream).
 Now remember the mails of Sven.

 So my questions are:

1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?

2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream?
why the code isn't something like that:
ZnNetworkingUtils>>initialize
super initialize.
self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) .
self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])

Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)). 
So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.

Thanks for giving me some light here,

Luc
Reply | Threaded
Open this post in threaded view
|

Re: Sockstream vs ZnSocketStream

Ben Coman
Luc Fabresse wrote:

> Hi all,
>
>  I am *experimenting* with the replacement of the current network layer
> with OCEAN.
>  My current plan is to make Zinc tests green while working on the top of
> OCEAN.
>  And investigate the removal of all the current Network classes (does it
> play?).
>  After looking at Zinc code, there are few dependencies.
>  Basically, I implemented an OCNSOcketStream (inheriting from SocketStream)
> and redefined some methods of ZnNetworkingUtils.
>  It seems to play so far (green tests).
>
>  But then, I discovered that Zodiac introduces a new SocketStream hierarchy
> (ZdcAbstractSocketStream).
>  Now remember the mails of Sven.
>
>  So my questions are:
>
> 1) Is the intent of ZdcSocketStream to be a future replacement for
> SocketStream?
>
> 2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of
> ZdcSocketStream?
> why the code isn't something like that:
>  ZnNetworkingUtils>>initialize
> super initialize.
> self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent:
> [ SocketStream ]) .
>  self secureSocketStreamClass: (Smalltalk globals at:
> #ZdcSecureSocketStream ifAbsent: [ nil ])
>
> Moreover, I guess that ZdcSocketStream should speed up things (it says
> optimized version in the name of the superclass ;-)).
>  So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.
>
> Thanks for giving me some light here,
>
> Luc
>
>  
I'm not even close to knowing the answers you'd like, but it looks
really interesting.
Slide 9 at [1] shows Alien being used to interact with the OS Network
API.  What are you thoughts on NativeBoost being used instead ?

[1] http://www.slideshare.net/esug/ocean-5305932

cheers -ben



Reply | Threaded
Open this post in threaded view
|

Re: Sockstream vs ZnSocketStream

Luc Fabresse

2013/5/10 Ben Coman <[hidden email]>
Luc Fabresse wrote:
Hi all,

 I am *experimenting* with the replacement of the current network layer
with OCEAN.
 My current plan is to make Zinc tests green while working on the top of
OCEAN.
 And investigate the removal of all the current Network classes (does it
play?).
 After looking at Zinc code, there are few dependencies.
 Basically, I implemented an OCNSOcketStream (inheriting from SocketStream)
and redefined some methods of ZnNetworkingUtils.
 It seems to play so far (green tests).

 But then, I discovered that Zodiac introduces a new SocketStream hierarchy
(ZdcAbstractSocketStream).
 Now remember the mails of Sven.

 So my questions are:

1) Is the intent of ZdcSocketStream to be a future replacement for
SocketStream?

2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of
ZdcSocketStream?
why the code isn't something like that:
 ZnNetworkingUtils>>initialize
super initialize.
self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent:
[ SocketStream ]) .
 self secureSocketStreamClass: (Smalltalk globals at:
#ZdcSecureSocketStream ifAbsent: [ nil ])

Moreover, I guess that ZdcSocketStream should speed up things (it says
optimized version in the name of the superclass ;-)).
 So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.

Thanks for giving me some light here,

Luc

 
I'm not even close to knowing the answers you'd like, but it looks really interesting. Slide 9 at [1] shows Alien being used to interact with the OS Network API.  What are you thoughts on NativeBoost being used instead ?

It is old slides ;-)
History:
At the beginning we would like to replace the Network VM plugin by wraping the POSIX socket layer through Alien.
At that time, we chose Alien because it supports callbacks.
And it was a nightmare to use Nativeboost because it required a special VM (with NB plugin) but that VM had not some Network primitives about  ipv6 and UDP.
Now things are much much better ;-)
one VM and everything inside.
 
But now we focus on the image part (Network API)  that still uses the plugin as backend (no need for Alien at all).
When it will work we will implement another backend with NB probably.
I know that Igor started one.
But I would to investigate wrapping ZeroMQ (I saw some mails about that).

Cheers,

Luc
 

[1] http://www.slideshare.net/esug/ocean-5305932


cheers -ben




Reply | Threaded
Open this post in threaded view
|

Re: Sockstream vs ZnSocketStream

Sven Van Caekenberghe-2
In reply to this post by Luc Fabresse
Hi Luc,

On 09 May 2013, at 22:05, Luc Fabresse <[hidden email]> wrote:

> Hi all,
>
>  I am *experimenting* with the replacement of the current network layer with OCEAN.
>  My current plan is to make Zinc tests green while working on the top of OCEAN.
>  And investigate the removal of all the current Network classes (does it play?).
>  After looking at Zinc code, there are few dependencies.
>  Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils.
>  It seems to play so far (green tests).

It is great that you are picking this up again.

If you mean that your OCNSocketStream with OCEAN Sockets can now run the full Zn test suite, that would be very good news indeed !

>  But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream).
>  Now remember the mails of Sven.
>
>  So my questions are:
>
> 1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?
>
> 2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream?
> why the code isn't something like that:
>
> ZnNetworkingUtils>>initialize
> super initialize.
> self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) .
> self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])
>
> Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)).
>
> So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.
>
> Thanks for giving me some light here,
>
> Luc

When I started writing Zinc, now already more than 2 years ago, I used the regular SocketStreams. But I limited my use to the following semantic options and just the regular ReadStream API:

        stream
                binary;
                shouldSignal: true;
                autoFlush: false

Some time later I did Zodiac. Before doing the TLS/SSL streams I wanted to make sure I correctly understood the SocketStream API and semantics that I was using. That is why I first reimplemented regular socket streams. Hence the hierarchy:

        ZdcAbstractSocketStream
                ZdcSimpleSocketStream
                        ZdcOptimizedSocketStream
                                ZdcSocketStream

My goal too was to be able to run the full Zn test suite. That is still possible, you can switch the plain socket stream being used by doing

        ZnNetworkingUtils default socketStreamClass: ZdcSocketStream

After that came the actual ZdcSecureSocketStream. Nowadays, Zodiac is a full part of Pharo, and thus used in the wild, which validates the implementation.

Yes, I think it would be way better (and easier for you) to plug into the hierarchy described above. If you need any help, just ask. I am very interested in that. Maybe the 'pluggability' can be improved.

Now the question 'should ZdcSocketStream replace SocketStream ?' can only be answered by users like yourself. As fas as I use socket streams I think it can, but I am sure some people will protest when they can no longer abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues.

The SocketStream that we have today works, but it is overly complex in API and implementation. It is also inefficient: try following what happens when you write a 1Mb ByteArray.

Regards,

Sven

--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org


Reply | Threaded
Open this post in threaded view
|

Re: Sockstream vs ZnSocketStream

stephane ducasse
cool signature :)


> Hi Luc,
>
> On 09 May 2013, at 22:05, Luc Fabresse <[hidden email]> wrote:
>
>> Hi all,
>>
>> I am *experimenting* with the replacement of the current network layer with OCEAN.
>> My current plan is to make Zinc tests green while working on the top of OCEAN.
>> And investigate the removal of all the current Network classes (does it play?).
>> After looking at Zinc code, there are few dependencies.
>> Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils.
>> It seems to play so far (green tests).
>
> It is great that you are picking this up again.
>
> If you mean that your OCNSocketStream with OCEAN Sockets can now run the full Zn test suite, that would be very good news indeed !
>
>> But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream).
>> Now remember the mails of Sven.
>>
>> So my questions are:
>>
>> 1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?
>>
>> 2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream?
>> why the code isn't something like that:
>>
>> ZnNetworkingUtils>>initialize
>> super initialize.
>> self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) .
>> self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])
>>
>> Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)).
>>
>> So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.
>>
>> Thanks for giving me some light here,
>>
>> Luc
>
> When I started writing Zinc, now already more than 2 years ago, I used the regular SocketStreams. But I limited my use to the following semantic options and just the regular ReadStream API:
>
> stream
> binary;
> shouldSignal: true;
> autoFlush: false
>
> Some time later I did Zodiac. Before doing the TLS/SSL streams I wanted to make sure I correctly understood the SocketStream API and semantics that I was using. That is why I first reimplemented regular socket streams. Hence the hierarchy:
>
> ZdcAbstractSocketStream
> ZdcSimpleSocketStream
> ZdcOptimizedSocketStream
> ZdcSocketStream
>
> My goal too was to be able to run the full Zn test suite. That is still possible, you can switch the plain socket stream being used by doing
>
> ZnNetworkingUtils default socketStreamClass: ZdcSocketStream
>
> After that came the actual ZdcSecureSocketStream. Nowadays, Zodiac is a full part of Pharo, and thus used in the wild, which validates the implementation.
>
> Yes, I think it would be way better (and easier for you) to plug into the hierarchy described above. If you need any help, just ask. I am very interested in that. Maybe the 'pluggability' can be improved.
>
> Now the question 'should ZdcSocketStream replace SocketStream ?' can only be answered by users like yourself. As fas as I use socket streams I think it can, but I am sure some people will protest when they can no longer abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues.
>
> The SocketStream that we have today works, but it is overly complex in API and implementation. It is also inefficient: try following what happens when you write a 1Mb ByteArray.
>
> Regards,
>
> Sven
>
> --
> Sven Van Caekenberghe
> Proudly supporting Pharo
> http://pharo.org
> http://association.pharo.org
> http://consortium.pharo.org
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Sockstream vs ZnSocketStream

Luc Fabresse
In reply to this post by Sven Van Caekenberghe-2
Hi Sven,

 Thanks for the explanation.
 You confirm what I was thinking, I will inegrate the ZdcAbstractSocketStream.

 I never used deeply SocketStream.
 So, I will also have a closer look at SocketStream to understand what you said about  "abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues". 
 My first impression is that we could have an "Encoder" hierarchy on top of a binary streams.
 Perhaps it is already there.

Thanks,

#Luc


#Luc


2013/5/10 Sven Van Caekenberghe <[hidden email]>
Hi Luc,

On 09 May 2013, at 22:05, Luc Fabresse <[hidden email]> wrote:

> Hi all,
>
>  I am *experimenting* with the replacement of the current network layer with OCEAN.
>  My current plan is to make Zinc tests green while working on the top of OCEAN.
>  And investigate the removal of all the current Network classes (does it play?).
>  After looking at Zinc code, there are few dependencies.
>  Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils.
>  It seems to play so far (green tests).

It is great that you are picking this up again.

If you mean that your OCNSocketStream with OCEAN Sockets can now run the full Zn test suite, that would be very good news indeed !

>  But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream).
>  Now remember the mails of Sven.
>
>  So my questions are:
>
> 1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?
>
> 2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream?
> why the code isn't something like that:
>
> ZnNetworkingUtils>>initialize
>       super initialize.
>       self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) .
>       self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])
>
> Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)).
>
> So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.
>
> Thanks for giving me some light here,
>
> Luc

When I started writing Zinc, now already more than 2 years ago, I used the regular SocketStreams. But I limited my use to the following semantic options and just the regular ReadStream API:

        stream
                binary;
                shouldSignal: true;
                autoFlush: false

Some time later I did Zodiac. Before doing the TLS/SSL streams I wanted to make sure I correctly understood the SocketStream API and semantics that I was using. That is why I first reimplemented regular socket streams. Hence the hierarchy:

        ZdcAbstractSocketStream
                ZdcSimpleSocketStream
                        ZdcOptimizedSocketStream
                                ZdcSocketStream

My goal too was to be able to run the full Zn test suite. That is still possible, you can switch the plain socket stream being used by doing

        ZnNetworkingUtils default socketStreamClass: ZdcSocketStream

After that came the actual ZdcSecureSocketStream. Nowadays, Zodiac is a full part of Pharo, and thus used in the wild, which validates the implementation.

Yes, I think it would be way better (and easier for you) to plug into the hierarchy described above. If you need any help, just ask. I am very interested in that. Maybe the 'pluggability' can be improved.

Now the question 'should ZdcSocketStream replace SocketStream ?' can only be answered by users like yourself. As fas as I use socket streams I think it can, but I am sure some people will protest when they can no longer abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues.

The SocketStream that we have today works, but it is overly complex in API and implementation. It is also inefficient: try following what happens when you write a 1Mb ByteArray.

Regards,

Sven

--
Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org



Reply | Threaded
Open this post in threaded view
|

Re: Sockstream vs ZnSocketStream

Sven Van Caekenberghe-2

On 11 May 2013, at 06:34, Luc Fabresse <[hidden email]> wrote:

> Hi Sven,
>
>  Thanks for the explanation.
>  You confirm what I was thinking, I will inegrate the ZdcAbstractSocketStream.

Cool.
Is there a place where I can already see what you did up to now ?

>  I never used deeply SocketStream.
>  So, I will also have a closer look at SocketStream to understand what you said about  "abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues".
>  My first impression is that we could have an "Encoder" hierarchy on top of a binary streams.
>  Perhaps it is already there.

Yes, there are ZnCharacterReadStream and ZnCharacterWriteStream that convert a binary stream into a character read and write stream respectively, using a decoder/encoder. BTW, there are also ZnBufferedReadStream and ZnBufferedWriteStream.

Sven

> Thanks,
>
> #Luc
>
>
> #Luc
>
>
> 2013/5/10 Sven Van Caekenberghe <[hidden email]>
> Hi Luc,
>
> On 09 May 2013, at 22:05, Luc Fabresse <[hidden email]> wrote:
>
> > Hi all,
> >
> >  I am *experimenting* with the replacement of the current network layer with OCEAN.
> >  My current plan is to make Zinc tests green while working on the top of OCEAN.
> >  And investigate the removal of all the current Network classes (does it play?).
> >  After looking at Zinc code, there are few dependencies.
> >  Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils.
> >  It seems to play so far (green tests).
>
> It is great that you are picking this up again.
>
> If you mean that your OCNSocketStream with OCEAN Sockets can now run the full Zn test suite, that would be very good news indeed !
>
> >  But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream).
> >  Now remember the mails of Sven.
> >
> >  So my questions are:
> >
> > 1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?
> >
> > 2) why  ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream?
> > why the code isn't something like that:
> >
> > ZnNetworkingUtils>>initialize
> >       super initialize.
> >       self socketStreamClass:  (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) .
> >       self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])
> >
> > Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)).
> >
> > So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.
> >
> > Thanks for giving me some light here,
> >
> > Luc
>
> When I started writing Zinc, now already more than 2 years ago, I used the regular SocketStreams. But I limited my use to the following semantic options and just the regular ReadStream API:
>
>         stream
>                 binary;
>                 shouldSignal: true;
>                 autoFlush: false
>
> Some time later I did Zodiac. Before doing the TLS/SSL streams I wanted to make sure I correctly understood the SocketStream API and semantics that I was using. That is why I first reimplemented regular socket streams. Hence the hierarchy:
>
>         ZdcAbstractSocketStream
>                 ZdcSimpleSocketStream
>                         ZdcOptimizedSocketStream
>                                 ZdcSocketStream
>
> My goal too was to be able to run the full Zn test suite. That is still possible, you can switch the plain socket stream being used by doing
>
>         ZnNetworkingUtils default socketStreamClass: ZdcSocketStream
>
> After that came the actual ZdcSecureSocketStream. Nowadays, Zodiac is a full part of Pharo, and thus used in the wild, which validates the implementation.
>
> Yes, I think it would be way better (and easier for you) to plug into the hierarchy described above. If you need any help, just ask. I am very interested in that. Maybe the 'pluggability' can be improved.
>
> Now the question 'should ZdcSocketStream replace SocketStream ?' can only be answered by users like yourself. As fas as I use socket streams I think it can, but I am sure some people will protest when they can no longer abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues.
>
> The SocketStream that we have today works, but it is overly complex in API and implementation. It is also inefficient: try following what happens when you write a 1Mb ByteArray.
>
> Regards,
>
> Sven
>
> --
> Sven Van Caekenberghe
> Proudly supporting Pharo
> http://pharo.org
> http://association.pharo.org
> http://consortium.pharo.org
>
>
>