In-memory FileSystem write streams not being polymorphic

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

In-memory FileSystem write streams not being polymorphic

Nicolás Papagna Maldonado
Hi all!

I've been working in a fix of PharoFilesOpener and decided to test the current behavior first before changing it, so I extracted the dependency to FileSystem and tried to use an instance created through FileSystem memory for testing.

The code gets the write stream for a file and then sends the #isReadOnly message to do some stuff.

The thing is that the streams returned form the in-memory file system are instances of WriteStream (see MemoryHandle >> writeStream) instead of StandardFileStream (see FileHandle >> writeStream) and so they don't respond the #isReadOnly message.

Is this the expected way the in-memory file system should work? Shouldn't it return polymorphic file streams?

If this is a bug, what change would you recommend to do?
I was thinking about returning StandardFileStreams instances from MemoryHandle >> writeStream.

Cheers!

Nico PM
Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Camillo Bruni-3

On 2013-11-11, at 14:19, Nicolás Papagna Maldonado <[hidden email]> wrote:

> Hi all!
>
> I've been working in a fix of PharoFilesOpener and decided to test the current behavior first before changing it, so I extracted the dependency to FileSystem and tried to use an instance created through FileSystem memory for testing.
>
> The code gets the write stream for a file and then sends the #isReadOnly message to do some stuff.
>
> The thing is that the streams returned form the in-memory file system are instances of WriteStream (see MemoryHandle >> writeStream) instead of StandardFileStream (see FileHandle >> writeStream) and so they don't respond the #isReadOnly message.
>
> Is this the expected way the in-memory file system should work? Shouldn't it return polymorphic file streams?

You are absolutely right, it definitely should, but this requires a bit of work (see below)

> If this is a bug, what change would you recommend to do?
> I was thinking about returning StandardFileStreams instances from MemoryHandle >> writeStream.

The memory filesystem works by definition in the image, whereas the StandardFileStream is bound
to external file operations through the primitives (see for instance #basicNext).
I think you have two choices here:
1. rewrite your code that it does not rely on the #isReadOnly but rather check up front
   if you can read the file
2. implement your own proper FileStream that works on the internal ByteArray of the
   memory FileSystem (maybe there is already a decent solution somewhere in the Zinc code)

or of course, you start looking at porting XStreams to pharo ;), which on the long run will
solve many more problems. The current situation is not that satisfactory :/

signature.asc (457 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stéphane Ducasse


or of course, you start looking at porting XStreams to pharo ;), which on the long run will
solve many more problems. The current situation is not that satisfactory

having experience with it and thinking about a plan for the beginning of 40 would be great.
I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
Stef

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
It's just a matter of selecting a strategy. I've proposed two:
A) create a wrapper class for legacy Stream compatibility selectors
B) create extensions for Legacy Stream compatibility selectors
My preference goes to A)

The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.

An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.

Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.


2013/11/12 Stéphane Ducasse <[hidden email]>


or of course, you start looking at porting XStreams to pharo ;), which on the long run will
solve many more problems. The current situation is not that satisfactory

having experience with it and thinking about a plan for the beginning of 40 would be great.
I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
Stef


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
And of course, I forgot all the usages of ReadWriteStream.
I started rewriting some usage in Squeak, and good news, 95% of usage is unecessary, a WriteStream is sufficient.


2013/11/12 Nicolas Cellier <[hidden email]>
It's just a matter of selecting a strategy. I've proposed two:
A) create a wrapper class for legacy Stream compatibility selectors
B) create extensions for Legacy Stream compatibility selectors
My preference goes to A)

The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.

An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.

Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.


2013/11/12 Stéphane Ducasse <[hidden email]>


or of course, you start looking at porting XStreams to pharo ;), which on the long run will
solve many more problems. The current situation is not that satisfactory

having experience with it and thinking about a plan for the beginning of 40 would be great.
I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
Stef



Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stephan Eggermont-3
In reply to this post by Nicolás Papagna Maldonado
How could we recognize the using of a deprecated stream?
With a tool like DeprecationFinder it should be easy to walk the repositories
and find out where they are used.

Stephan

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stéphane Ducasse
In reply to this post by Nicolas Cellier

On Nov 12, 2013, at 2:33 PM, Nicolas Cellier <[hidden email]> wrote:

And of course, I forgot all the usages of ReadWriteStream.
I started rewriting some usage in Squeak, and good news, 95% of usage is unecessary, a WriteStream is sufficient.

:)
if you want to push some of these changes please do.
In the past we renamed all the hardcoded names with nsStream to be able to plug another but we never went further.

Stef


2013/11/12 Nicolas Cellier <[hidden email]>
It's just a matter of selecting a strategy. I've proposed two:
A) create a wrapper class for legacy Stream compatibility selectors
B) create extensions for Legacy Stream compatibility selectors
My preference goes to A)

The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.

An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.

Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.


2013/11/12 Stéphane Ducasse <[hidden email]>


or of course, you start looking at porting XStreams to pharo ;), which on the long run will
solve many more problems. The current situation is not that satisfactory

having experience with it and thinking about a plan for the beginning of 40 would be great.
I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
Stef




Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stéphane Ducasse
In reply to this post by Nicolas Cellier

On Nov 12, 2013, at 2:31 PM, Nicolas Cellier <[hidden email]> wrote:

It's just a matter of selecting a strategy. I've proposed two:
A) create a wrapper class for legacy Stream compatibility selectors
B) create extensions for Legacy Stream compatibility selectors
My preference goes to A)

if I understand correctly I prefer A too.
Because like that we can decide to even throw away later the wrapper and a wrapper is like a clear API.

The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.

An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.

Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.


2013/11/12 Stéphane Ducasse <[hidden email]>


or of course, you start looking at porting XStreams to pharo ;), which on the long run will
solve many more problems. The current situation is not that satisfactory

having experience with it and thinking about a plan for the beginning of 40 would be great.
I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
Stef



Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Sven Van Caekenberghe-2
In reply to this post by Nicolas Cellier
Nicolas,

Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?

If yes, from which repository using which Configuration ?

I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.

It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.

The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.

The biggest gain is of course if clients switch to the newer API ;-)

Sven  

On 12 Nov 2013, at 14:31, Nicolas Cellier <[hidden email]> wrote:

> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)
>
> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
> Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
>
>
> 2013/11/12 Stéphane Ducasse <[hidden email]>
>
>>
>> or of course, you start looking at porting XStreams to pharo ;), which on the long run will
>> solve many more problems. The current situation is not that satisfactory
>
> having experience with it and thinking about a plan for the beginning of 40 would be great.
> I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
> Stef
>
>


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
Hi Sven,
The last Squeak/Pharo locations I am aware of are:
- http://www.squeaksource.org/Xtreams for the Xtreams package
- http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo)
A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/

Those versions above do not load in Pharo3.0 due to several problems
- FileDirectory deprecation (there is an extension method that should be removed in Pharo branch)
- Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found...

I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?




2013/11/12 Sven Van Caekenberghe <[hidden email]>
Nicolas,

Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?

If yes, from which repository using which Configuration ?

I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.

It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.

The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.

The biggest gain is of course if clients switch to the newer API ;-)

Sven

On 12 Nov 2013, at 14:31, Nicolas Cellier <[hidden email]> wrote:

> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)
>
> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
> Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
>
>
> 2013/11/12 Stéphane Ducasse <[hidden email]>
>
>>
>> or of course, you start looking at porting XStreams to pharo ;), which on the long run will
>> solve many more problems. The current situation is not that satisfactory
>
> having experience with it and thinking about a plan for the beginning of 40 would be great.
> I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
> Stef
>
>



Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
And the tests pass, except the file tests which are based on creating a stream thru (FileDirectory default / ...)


2013/11/12 Nicolas Cellier <[hidden email]>
Hi Sven,
The last Squeak/Pharo locations I am aware of are:
- http://www.squeaksource.org/Xtreams for the Xtreams package
- http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo)
A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/

Those versions above do not load in Pharo3.0 due to several problems
- FileDirectory deprecation (there is an extension method that should be removed in Pharo branch)
- Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found...

I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?




2013/11/12 Sven Van Caekenberghe <[hidden email]>
Nicolas,

Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?

If yes, from which repository using which Configuration ?

I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.

It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.

The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.

The biggest gain is of course if clients switch to the newer API ;-)

Sven

On 12 Nov 2013, at 14:31, Nicolas Cellier <[hidden email]> wrote:

> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)
>
> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point...
> Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
>
>
> 2013/11/12 Stéphane Ducasse <[hidden email]>
>
>>
>> or of course, you start looking at porting XStreams to pharo ;), which on the long run will
>> solve many more problems. The current situation is not that satisfactory
>
> having experience with it and thinking about a plan for the beginning of 40 would be great.
> I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice.
> Stef
>
>




Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Frank Shearar-3
Yep, they do. build.squeak.org uses this repo. See also
https://code.google.com/p/xtreams/issues/detail?id=2

About time for us to pick up this work?

frank

On 12 November 2013 21:29, Nicolas Cellier
<[hidden email]> wrote:

> And the tests pass, except the file tests which are based on creating a
> stream thru (FileDirectory default / ...)
>
>
> 2013/11/12 Nicolas Cellier <[hidden email]>
>>
>> Hi Sven,
>> The last Squeak/Pharo locations I am aware of are:
>> - http://www.squeaksource.org/Xtreams for the Xtreams package
>> - http://www.squeaksource.org/MetacelloRepository for
>> ConfigurationOfXtreams (but I just copied the last Config to above repo)
>> A very important repository for resources of all kind from the original
>> authors is https://code.google.com/p/xtreams/
>>
>> Those versions above do not load in Pharo3.0 due to several problems
>> - FileDirectory deprecation (there is an extension method that should be
>> removed in Pharo branch)
>> - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly
>> in Pharo (at least the one found from the ConfigurationOfXtreams) because
>> ShortRunArray is not found...
>>
>> I will try and publish a Pharo branch, but before I do so, is anyone aware
>> of more recent work, or repository?
>>
>>
>>
>>
>> 2013/11/12 Sven Van Caekenberghe <[hidden email]>
>>>
>>> Nicolas,
>>>
>>> Is it currently possible to load some latest version of Xtream into Pharo
>>> 3.0 ?
>>>
>>> If yes, from which repository using which Configuration ?
>>>
>>> I know that at ESUG 2012, Sean and Martin worked a bit on getting all
>>> different versions better in sync.
>>>
>>> It would be cool if we could at least load it, separate from the
>>> transition strategy (I agree with your proposal BTW), because many people do
>>> not know or have not seen what we are actually talking about.
>>>
>>> The clean, start from scratch approach of Xtreams also includes a much
>>> tighter and semantically better defined API. IMHO, a consequence is that
>>> #get / #next and #put: / #nextPut: are not just plain aliases (a modern use
>>> of exception handling is one big difference). The compatibility layer might
>>> be more of a challenge.
>>>
>>> The biggest gain is of course if clients switch to the newer API ;-)
>>>
>>> Sven
>>>
>>> On 12 Nov 2013, at 14:31, Nicolas Cellier
>>> <[hidden email]> wrote:
>>>
>>> > It's just a matter of selecting a strategy. I've proposed two:
>>> > A) create a wrapper class for legacy Stream compatibility selectors
>>> > B) create extensions for Legacy Stream compatibility selectors
>>> > My preference goes to A)
>>> >
>>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek
>>> > upTo: ...), otherwise we will import all the cruft in Xtream and would go
>>> > back to our starting point...
>>> > Once the minimal support written (a few hours should be enough), we
>>> > should gradually switch each every legacy Stream usage -> Xtream.
>>> >
>>> > An area which require more work is those Streams that have mixed
>>> > conventions (one portion is interpreted as text, another as binary).
>>> > In theory that's easy, we just have two streams and they both wrap on a
>>> > low level binary stream, but that means we have to be very cautious with
>>> > buffers and caches.
>>> >
>>> > Another area of work is usage of ugly selectors like name (we try to
>>> > access the file name from the Stream API, arghh). Those usages are bad and
>>> > require a rewrite.
>>> >
>>> >
>>> > 2013/11/12 Stéphane Ducasse <[hidden email]>
>>> >
>>> >>
>>> >> or of course, you start looking at porting XStreams to pharo ;), which
>>> >> on the long run will
>>> >> solve many more problems. The current situation is not that
>>> >> satisfactory
>>> >
>>> > having experience with it and thinking about a plan for the beginning
>>> > of 40 would be great.
>>> > I know that nicolas ported XTream to pharo/squeak. Now understanding
>>> > how integrate it would be nice.
>>> > Stef
>>> >
>>> >
>>>
>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
In original VW works, there is no FileDirectory nor FileReference nor FileSystem nor...
So Xtreams-Terminals was thought as a dialect specific package...
Of course, these terminals are still 95% compatible between Squeak4.x and Pharo3.x, so we can make a better packaging.

Before deciding what is a good package delimitation, and what's not, I wanted to experiment a bit without compatibility constraints.
So I created a .Pharo3 branch in Terminals and TerminalsTests.
There's a little more than FileDirectory references: I also used the FileHandle class provided by FileSystem package, rather than Xtreams-Support XTIOFileHandle.

Anyway, the Squeak/Pharo version is way behind the visualworks version, so yes, there is definitely some work needed.
Up to now, I diffed the versions in VW and manually ported to Squeak.
The good thing is that I can try to learn and understand the changes.
It is also mandatory to cop with the small differences (some VW method might be absent in Squeak/Pharo and require a Xtreams-Support counterpart, the classes need a XT prefix, ...).
The bad things are that
- there are many versions to integrate, and the history is not linear;
- I have only ten fingers, two eyes, and a single brain, the whole comes with very limited multitasking capabilities.
There must be a better way, like trying to automate the port of shared implementations, and review only the dialect-specific parts.
Or just put more brains, finger and eyes in the loop...


2013/11/12 Frank Shearar <[hidden email]>
Yep, they do. build.squeak.org uses this repo. See also
https://code.google.com/p/xtreams/issues/detail?id=2

About time for us to pick up this work?

frank

On 12 November 2013 21:29, Nicolas Cellier
<[hidden email]> wrote:
> And the tests pass, except the file tests which are based on creating a
> stream thru (FileDirectory default / ...)
>
>
> 2013/11/12 Nicolas Cellier <[hidden email]>
>>
>> Hi Sven,
>> The last Squeak/Pharo locations I am aware of are:
>> - http://www.squeaksource.org/Xtreams for the Xtreams package
>> - http://www.squeaksource.org/MetacelloRepository for
>> ConfigurationOfXtreams (but I just copied the last Config to above repo)
>> A very important repository for resources of all kind from the original
>> authors is https://code.google.com/p/xtreams/
>>
>> Those versions above do not load in Pharo3.0 due to several problems
>> - FileDirectory deprecation (there is an extension method that should be
>> removed in Pharo branch)
>> - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly
>> in Pharo (at least the one found from the ConfigurationOfXtreams) because
>> ShortRunArray is not found...
>>
>> I will try and publish a Pharo branch, but before I do so, is anyone aware
>> of more recent work, or repository?
>>
>>
>>
>>
>> 2013/11/12 Sven Van Caekenberghe <[hidden email]>
>>>
>>> Nicolas,
>>>
>>> Is it currently possible to load some latest version of Xtream into Pharo
>>> 3.0 ?
>>>
>>> If yes, from which repository using which Configuration ?
>>>
>>> I know that at ESUG 2012, Sean and Martin worked a bit on getting all
>>> different versions better in sync.
>>>
>>> It would be cool if we could at least load it, separate from the
>>> transition strategy (I agree with your proposal BTW), because many people do
>>> not know or have not seen what we are actually talking about.
>>>
>>> The clean, start from scratch approach of Xtreams also includes a much
>>> tighter and semantically better defined API. IMHO, a consequence is that
>>> #get / #next and #put: / #nextPut: are not just plain aliases (a modern use
>>> of exception handling is one big difference). The compatibility layer might
>>> be more of a challenge.
>>>
>>> The biggest gain is of course if clients switch to the newer API ;-)
>>>
>>> Sven
>>>
>>> On 12 Nov 2013, at 14:31, Nicolas Cellier
>>> <[hidden email]> wrote:
>>>
>>> > It's just a matter of selecting a strategy. I've proposed two:
>>> > A) create a wrapper class for legacy Stream compatibility selectors
>>> > B) create extensions for Legacy Stream compatibility selectors
>>> > My preference goes to A)
>>> >
>>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek
>>> > upTo: ...), otherwise we will import all the cruft in Xtream and would go
>>> > back to our starting point...
>>> > Once the minimal support written (a few hours should be enough), we
>>> > should gradually switch each every legacy Stream usage -> Xtream.
>>> >
>>> > An area which require more work is those Streams that have mixed
>>> > conventions (one portion is interpreted as text, another as binary).
>>> > In theory that's easy, we just have two streams and they both wrap on a
>>> > low level binary stream, but that means we have to be very cautious with
>>> > buffers and caches.
>>> >
>>> > Another area of work is usage of ugly selectors like name (we try to
>>> > access the file name from the Stream API, arghh). Those usages are bad and
>>> > require a rewrite.
>>> >
>>> >
>>> > 2013/11/12 Stéphane Ducasse <[hidden email]>
>>> >
>>> >>
>>> >> or of course, you start looking at porting XStreams to pharo ;), which
>>> >> on the long run will
>>> >> solve many more problems. The current situation is not that
>>> >> satisfactory
>>> >
>>> > having experience with it and thinking about a plan for the beginning
>>> > of 40 would be great.
>>> > I know that nicolas ported XTream to pharo/squeak. Now understanding
>>> > how integrate it would be nice.
>>> > Stef
>>> >
>>> >
>>>
>>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
So the latest ConfigurationOfXtreams found either in http://www.squeaksource.org/Xtreams or http://www.squeaksource.org/MetacelloRepository more or less load in Pharo3.0...
More or less means that it still tries to load FFI which fails to compile...
And I don't know why it loads, and i don't know why it fails...
FFI is required only by Xtreams-Xtra which is not (or should not be) in the 'default' group for #'pharo3.x'
It's not very important, because most of the Xtreams packages are already loaded when the failure occurs.

When i check with this, it does not tell me that it will load FFI (maybe because I have a version loaded?):
ConfigurationOfXtreams project latestVersion fetch loadDirective.

But when I load with this, it tries to compile FFI again:
ConfigurationOfXtreams project latestVersion load.


2013/11/13 Nicolas Cellier <[hidden email]>
In original VW works, there is no FileDirectory nor FileReference nor FileSystem nor...
So Xtreams-Terminals was thought as a dialect specific package...
Of course, these terminals are still 95% compatible between Squeak4.x and Pharo3.x, so we can make a better packaging.

Before deciding what is a good package delimitation, and what's not, I wanted to experiment a bit without compatibility constraints.
So I created a .Pharo3 branch in Terminals and TerminalsTests.
There's a little more than FileDirectory references: I also used the FileHandle class provided by FileSystem package, rather than Xtreams-Support XTIOFileHandle.

Anyway, the Squeak/Pharo version is way behind the visualworks version, so yes, there is definitely some work needed.
Up to now, I diffed the versions in VW and manually ported to Squeak.
The good thing is that I can try to learn and understand the changes.
It is also mandatory to cop with the small differences (some VW method might be absent in Squeak/Pharo and require a Xtreams-Support counterpart, the classes need a XT prefix, ...).
The bad things are that
- there are many versions to integrate, and the history is not linear;
- I have only ten fingers, two eyes, and a single brain, the whole comes with very limited multitasking capabilities.
There must be a better way, like trying to automate the port of shared implementations, and review only the dialect-specific parts.
Or just put more brains, finger and eyes in the loop...


2013/11/12 Frank Shearar <[hidden email]>
Yep, they do. build.squeak.org uses this repo. See also
https://code.google.com/p/xtreams/issues/detail?id=2

About time for us to pick up this work?

frank

On 12 November 2013 21:29, Nicolas Cellier
<[hidden email]> wrote:
> And the tests pass, except the file tests which are based on creating a
> stream thru (FileDirectory default / ...)
>
>
> 2013/11/12 Nicolas Cellier <[hidden email]>
>>
>> Hi Sven,
>> The last Squeak/Pharo locations I am aware of are:
>> - http://www.squeaksource.org/Xtreams for the Xtreams package
>> - http://www.squeaksource.org/MetacelloRepository for
>> ConfigurationOfXtreams (but I just copied the last Config to above repo)
>> A very important repository for resources of all kind from the original
>> authors is https://code.google.com/p/xtreams/
>>
>> Those versions above do not load in Pharo3.0 due to several problems
>> - FileDirectory deprecation (there is an extension method that should be
>> removed in Pharo branch)
>> - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly
>> in Pharo (at least the one found from the ConfigurationOfXtreams) because
>> ShortRunArray is not found...
>>
>> I will try and publish a Pharo branch, but before I do so, is anyone aware
>> of more recent work, or repository?
>>
>>
>>
>>
>> 2013/11/12 Sven Van Caekenberghe <[hidden email]>
>>>
>>> Nicolas,
>>>
>>> Is it currently possible to load some latest version of Xtream into Pharo
>>> 3.0 ?
>>>
>>> If yes, from which repository using which Configuration ?
>>>
>>> I know that at ESUG 2012, Sean and Martin worked a bit on getting all
>>> different versions better in sync.
>>>
>>> It would be cool if we could at least load it, separate from the
>>> transition strategy (I agree with your proposal BTW), because many people do
>>> not know or have not seen what we are actually talking about.
>>>
>>> The clean, start from scratch approach of Xtreams also includes a much
>>> tighter and semantically better defined API. IMHO, a consequence is that
>>> #get / #next and #put: / #nextPut: are not just plain aliases (a modern use
>>> of exception handling is one big difference). The compatibility layer might
>>> be more of a challenge.
>>>
>>> The biggest gain is of course if clients switch to the newer API ;-)
>>>
>>> Sven
>>>
>>> On 12 Nov 2013, at 14:31, Nicolas Cellier
>>> <[hidden email]> wrote:
>>>
>>> > It's just a matter of selecting a strategy. I've proposed two:
>>> > A) create a wrapper class for legacy Stream compatibility selectors
>>> > B) create extensions for Legacy Stream compatibility selectors
>>> > My preference goes to A)
>>> >
>>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek
>>> > upTo: ...), otherwise we will import all the cruft in Xtream and would go
>>> > back to our starting point...
>>> > Once the minimal support written (a few hours should be enough), we
>>> > should gradually switch each every legacy Stream usage -> Xtream.
>>> >
>>> > An area which require more work is those Streams that have mixed
>>> > conventions (one portion is interpreted as text, another as binary).
>>> > In theory that's easy, we just have two streams and they both wrap on a
>>> > low level binary stream, but that means we have to be very cautious with
>>> > buffers and caches.
>>> >
>>> > Another area of work is usage of ugly selectors like name (we try to
>>> > access the file name from the Stream API, arghh). Those usages are bad and
>>> > require a rewrite.
>>> >
>>> >
>>> > 2013/11/12 Stéphane Ducasse <[hidden email]>
>>> >
>>> >>
>>> >> or of course, you start looking at porting XStreams to pharo ;), which
>>> >> on the long run will
>>> >> solve many more problems. The current situation is not that
>>> >> satisfactory
>>> >
>>> > having experience with it and thinking about a plan for the beginning
>>> > of 40 would be great.
>>> > I know that nicolas ported XTream to pharo/squeak. Now understanding
>>> > how integrate it would be nice.
>>> > Stef
>>> >
>>> >
>>>
>>>
>>
>



Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stéphane Ducasse
Thanks nicolas for checking that.

Stef


So the latest ConfigurationOfXtreams found either in http://www.squeaksource.org/Xtreams or http://www.squeaksource.org/MetacelloRepository more or less load in Pharo3.0...
More or less means that it still tries to load FFI which fails to compile...
And I don't know why it loads, and i don't know why it fails...
FFI is required only by Xtreams-Xtra which is not (or should not be) in the 'default' group for #'pharo3.x'
It's not very important, because most of the Xtreams packages are already loaded when the failure occurs.

When i check with this, it does not tell me that it will load FFI (maybe because I have a version loaded?):
ConfigurationOfXtreams project latestVersion fetch loadDirective.

But when I load with this, it tries to compile FFI again:
ConfigurationOfXtreams project latestVersion load.


2013/11/13 Nicolas Cellier <[hidden email]>
In original VW works, there is no FileDirectory nor FileReference nor FileSystem nor...
So Xtreams-Terminals was thought as a dialect specific package...
Of course, these terminals are still 95% compatible between Squeak4.x and Pharo3.x, so we can make a better packaging.

Before deciding what is a good package delimitation, and what's not, I wanted to experiment a bit without compatibility constraints.
So I created a .Pharo3 branch in Terminals and TerminalsTests.
There's a little more than FileDirectory references: I also used the FileHandle class provided by FileSystem package, rather than Xtreams-Support XTIOFileHandle.

Anyway, the Squeak/Pharo version is way behind the visualworks version, so yes, there is definitely some work needed.
Up to now, I diffed the versions in VW and manually ported to Squeak.
The good thing is that I can try to learn and understand the changes.
It is also mandatory to cop with the small differences (some VW method might be absent in Squeak/Pharo and require a Xtreams-Support counterpart, the classes need a XT prefix, ...).
The bad things are that
- there are many versions to integrate, and the history is not linear;
- I have only ten fingers, two eyes, and a single brain, the whole comes with very limited multitasking capabilities.
There must be a better way, like trying to automate the port of shared implementations, and review only the dialect-specific parts.
Or just put more brains, finger and eyes in the loop...


2013/11/12 Frank Shearar <[hidden email]>
Yep, they do. build.squeak.org uses this repo. See also
https://code.google.com/p/xtreams/issues/detail?id=2

About time for us to pick up this work?

frank

On 12 November 2013 21:29, Nicolas Cellier
<[hidden email]> wrote:
> And the tests pass, except the file tests which are based on creating a
> stream thru (FileDirectory default / ...)
>
>
> 2013/11/12 Nicolas Cellier <[hidden email]>
>>
>> Hi Sven,
>> The last Squeak/Pharo locations I am aware of are:
>> - http://www.squeaksource.org/Xtreams for the Xtreams package
>> - http://www.squeaksource.org/MetacelloRepository for
>> ConfigurationOfXtreams (but I just copied the last Config to above repo)
>> A very important repository for resources of all kind from the original
>> authors is https://code.google.com/p/xtreams/
>>
>> Those versions above do not load in Pharo3.0 due to several problems
>> - FileDirectory deprecation (there is an extension method that should be
>> removed in Pharo branch)
>> - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly
>> in Pharo (at least the one found from the ConfigurationOfXtreams) because
>> ShortRunArray is not found...
>>
>> I will try and publish a Pharo branch, but before I do so, is anyone aware
>> of more recent work, or repository?
>>
>>
>>
>>
>> 2013/11/12 Sven Van Caekenberghe <[hidden email]>
>>>
>>> Nicolas,
>>>
>>> Is it currently possible to load some latest version of Xtream into Pharo
>>> 3.0 ?
>>>
>>> If yes, from which repository using which Configuration ?
>>>
>>> I know that at ESUG 2012, Sean and Martin worked a bit on getting all
>>> different versions better in sync.
>>>
>>> It would be cool if we could at least load it, separate from the
>>> transition strategy (I agree with your proposal BTW), because many people do
>>> not know or have not seen what we are actually talking about.
>>>
>>> The clean, start from scratch approach of Xtreams also includes a much
>>> tighter and semantically better defined API. IMHO, a consequence is that
>>> #get / #next and #put: / #nextPut: are not just plain aliases (a modern use
>>> of exception handling is one big difference). The compatibility layer might
>>> be more of a challenge.
>>>
>>> The biggest gain is of course if clients switch to the newer API ;-)
>>>
>>> Sven
>>>
>>> On 12 Nov 2013, at 14:31, Nicolas Cellier
>>> <[hidden email]> wrote:
>>>
>>> > It's just a matter of selecting a strategy. I've proposed two:
>>> > A) create a wrapper class for legacy Stream compatibility selectors
>>> > B) create extensions for Legacy Stream compatibility selectors
>>> > My preference goes to A)
>>> >
>>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek
>>> > upTo: ...), otherwise we will import all the cruft in Xtream and would go
>>> > back to our starting point...
>>> > Once the minimal support written (a few hours should be enough), we
>>> > should gradually switch each every legacy Stream usage -> Xtream.
>>> >
>>> > An area which require more work is those Streams that have mixed
>>> > conventions (one portion is interpreted as text, another as binary).
>>> > In theory that's easy, we just have two streams and they both wrap on a
>>> > low level binary stream, but that means we have to be very cautious with
>>> > buffers and caches.
>>> >
>>> > Another area of work is usage of ugly selectors like name (we try to
>>> > access the file name from the Stream API, arghh). Those usages are bad and
>>> > require a rewrite.
>>> >
>>> >
>>> > 2013/11/12 Stéphane Ducasse <[hidden email]>
>>> >
>>> >>
>>> >> or of course, you start looking at porting XStreams to pharo ;), which
>>> >> on the long run will
>>> >> solve many more problems. The current situation is not that
>>> >> satisfactory
>>> >
>>> > having experience with it and thinking about a plan for the beginning
>>> > of 40 would be great.
>>> > I know that nicolas ported XTream to pharo/squeak. Now understanding
>>> > how integrate it would be nice.
>>> > Stef
>>> >
>>> >
>>>
>>>
>>
>




Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Chris Muller-3
In reply to this post by Nicolas Cellier
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
<[hidden email]> wrote:
> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)

By wrappers you mean the Xtreams are the innards doing the work and
the wrappers providing the legacy API?

This would be a great way to test Xtreams.

> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
> ...), otherwise we will import all the cruft in Xtream and would go back to
> our starting point...
> Once the minimal support written (a few hours should be enough), we should
> gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions
> (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low
> level binary stream, but that means we have to be very cautious with buffers
> and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access
> the file name from the Stream API, arghh). Those usages are bad and require
> a rewrite.

Are you saying a FileStream knowing its #name or #filename is bad?

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
Yes, a Wrapper would provide the legacy API.

And yes, the name of a stream should better not be part of the API.
Most stream don't have a name, and adding such API adds all sort of un-needed complexity.
It's an internal detail that can eventually help for reporting error if accessible, but nothing more.

If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).


2013/11/13 Chris Muller <[hidden email]>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
<[hidden email]> wrote:
> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)

By wrappers you mean the Xtreams are the innards doing the work and
the wrappers providing the legacy API?

This would be a great way to test Xtreams.

> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
> ...), otherwise we will import all the cruft in Xtream and would go back to
> our starting point...
> Once the minimal support written (a few hours should be enough), we should
> gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions
> (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low
> level binary stream, but that means we have to be very cautious with buffers
> and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access
> the file name from the Stream API, arghh). Those usages are bad and require
> a rewrite.

Are you saying a FileStream knowing its #name or #filename is bad?


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Chris Muller-3
I know nothing about Xtreams but, IMO, this obsession with sterility
borders on mental illness.

"All sort of un-needed complexity?"  That's overstating it a bit,
don't you think?

So you must really feel stressed that ALL Object's, in fact, have a
#name, huh?  I admit this seems to push the limits but...

what's the point of having any kind of different streams at all then?
It's the _differences_ between them that makes composing them useful.
Compression, encryption, filtering, sockets, files, circular, etc.
You think you'll be able to do all that and get away with all of them
having exactly the same API?

IMHO working around that, passing extra objects around, sounds more
stressful than letting a stream on a _file_ know its filename..

> If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).

How does that solution allow uniformity in stream-using code?

On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
<[hidden email]> wrote:

> Yes, a Wrapper would provide the legacy API.
>
> And yes, the name of a stream should better not be part of the API.
> Most stream don't have a name, and adding such API adds all sort of
> un-needed complexity.
> It's an internal detail that can eventually help for reporting error if
> accessible, but nothing more.
>
> If it really need it, the application certainly can retrieve the name from a
> higher level object (a FIleReference, FileDirectory or whatever).
>
>
> 2013/11/13 Chris Muller <[hidden email]>
>>
>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>> <[hidden email]> wrote:
>> > It's just a matter of selecting a strategy. I've proposed two:
>> > A) create a wrapper class for legacy Stream compatibility selectors
>> > B) create extensions for Legacy Stream compatibility selectors
>> > My preference goes to A)
>>
>> By wrappers you mean the Xtreams are the innards doing the work and
>> the wrappers providing the legacy API?
>>
>> This would be a great way to test Xtreams.
>>
>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
>> > ...), otherwise we will import all the cruft in Xtream and would go back
>> > to
>> > our starting point...
>> > Once the minimal support written (a few hours should be enough), we
>> > should
>> > gradually switch each every legacy Stream usage -> Xtream.
>> >
>> > An area which require more work is those Streams that have mixed
>> > conventions
>> > (one portion is interpreted as text, another as binary).
>> > In theory that's easy, we just have two streams and they both wrap on a
>> > low
>> > level binary stream, but that means we have to be very cautious with
>> > buffers
>> > and caches.
>> >
>> > Another area of work is usage of ugly selectors like name (we try to
>> > access
>> > the file name from the Stream API, arghh). Those usages are bad and
>> > require
>> > a rewrite.
>>
>> Are you saying a FileStream knowing its #name or #filename is bad?
>>
>

Reply | Threaded
Open this post in threaded view
|

Fwd: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
In reply to this post by Nicolas Cellier

---------- Forwarded message ----------
From: Nicolas Cellier <[hidden email]>
Date: 2013/11/13
Subject: Re: [Pharo-dev] In-memory FileSystem write streams not being polymorphic

The long answer is simple: the more responsibility you put in Stream, the more complexity you get.
The first complexity that I'm speaking of is non-uniformity and random implementation (or un-implementation) of a set of features.
I mean some streams in the huge hierarchy implement only half of the contract, but hey, what was the contract exactly?
Ah, yes, there were no contract, just hacks left here and there randomly and concurrently in a big hierarchy.
I add a new subclass, but don't implement all the features, there are too many of them, and I don't know them all..
I add a new feature, but don't implement it in all the classes, there are too many of them, and I don't know them all.
This procedure invariably ends up with a blob of un-maintanable code, were two stream would not even agree on upToEnd behavior - I think you remember it :)

If the goal is to replicate all the accumulated responsibilities of Squeak Streams, but with a clarified contract, I think this is a dead end: too much cruft to support, too many features, too many undefined or not well defined behaviors to be clarified.

Xtreams takes the opposite path: concentrate on constructing a common, simple and uniform API, concerning streams, just basic stream methods.
Some specialized streams then implement some specialized behavior, and you can compose them (by wrapping) when you need the specific API.
This way, you ain't got to implement/maintain feature A into a few dozen of classes.
And your brand new SpecialFeatureBStream only has to implement a few well known behaviours and your Special Feature B.

The short answer is even more simple : a stream does not have a name, like a collection does not have a name, because we ain't gonna need it.
If we really need a name, then we'll create a XTNamedReadStream and a XTNamedWriteStream responding to #name, but I doubt we'll do.


2013/11/13 Chris Muller <[hidden email]>
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <[hidden email]> wrote:
Yes, a Wrapper would provide the legacy API.

And yes, the name of a stream should better not be part of the API.
Most stream don't have a name, and adding such API adds all sort of un-needed complexity.
It's an internal detail that can eventually help for reporting error if accessible, but nothing more.

 
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.

"All sort of un-needed complexity?"  That's overstating it a bit, don't you think?

So you must really feel stressed that ALL Object's, in fact, have a #name, huh?  I admit this seems to push the limits but... 

what's the point of having any kind of different streams at all then?  It's the _differences_ between them that makes composing them useful.  Compression, encryption, filtering, sockets, files, circular, etc.  You think you'll be able to do all that and get away with all of them having exactly the same API?

IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..

If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).

How does that solution allow uniformity in stream-using code?




 


2013/11/13 Chris Muller <[hidden email]>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
<[hidden email]> wrote:
> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)

By wrappers you mean the Xtreams are the innards doing the work and
the wrappers providing the legacy API?

This would be a great way to test Xtreams.

> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
> ...), otherwise we will import all the cruft in Xtream and would go back to
> our starting point...
> Once the minimal support written (a few hours should be enough), we should
> gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions
> (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low
> level binary stream, but that means we have to be very cautious with buffers
> and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access
> the file name from the Stream API, arghh). Those usages are bad and require
> a rewrite.

Are you saying a FileStream knowing its #name or #filename is bad?





Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
Just forgot to add this reductio ad absurdum:
I suggest Object ^'John Doe', this is a relatively current name, so it shouldn't hurt.


2013/11/13 Nicolas Cellier <[hidden email]>

---------- Forwarded message ----------
From: Nicolas Cellier <[hidden email]>
Date: 2013/11/13
Subject: Re: [Pharo-dev] In-memory FileSystem write streams not being polymorphic

The long answer is simple: the more responsibility you put in Stream, the more complexity you get.
The first complexity that I'm speaking of is non-uniformity and random implementation (or un-implementation) of a set of features.
I mean some streams in the huge hierarchy implement only half of the contract, but hey, what was the contract exactly?
Ah, yes, there were no contract, just hacks left here and there randomly and concurrently in a big hierarchy.
I add a new subclass, but don't implement all the features, there are too many of them, and I don't know them all..
I add a new feature, but don't implement it in all the classes, there are too many of them, and I don't know them all.
This procedure invariably ends up with a blob of un-maintanable code, were two stream would not even agree on upToEnd behavior - I think you remember it :)

If the goal is to replicate all the accumulated responsibilities of Squeak Streams, but with a clarified contract, I think this is a dead end: too much cruft to support, too many features, too many undefined or not well defined behaviors to be clarified.

Xtreams takes the opposite path: concentrate on constructing a common, simple and uniform API, concerning streams, just basic stream methods.
Some specialized streams then implement some specialized behavior, and you can compose them (by wrapping) when you need the specific API.
This way, you ain't got to implement/maintain feature A into a few dozen of classes.
And your brand new SpecialFeatureBStream only has to implement a few well known behaviours and your Special Feature B.

The short answer is even more simple : a stream does not have a name, like a collection does not have a name, because we ain't gonna need it.
If we really need a name, then we'll create a XTNamedReadStream and a XTNamedWriteStream responding to #name, but I doubt we'll do.


2013/11/13 Chris Muller <[hidden email]>
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <[hidden email]> wrote:
Yes, a Wrapper would provide the legacy API.

And yes, the name of a stream should better not be part of the API.
Most stream don't have a name, and adding such API adds all sort of un-needed complexity.
It's an internal detail that can eventually help for reporting error if accessible, but nothing more.

 
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.

"All sort of un-needed complexity?"  That's overstating it a bit, don't you think?

So you must really feel stressed that ALL Object's, in fact, have a #name, huh?  I admit this seems to push the limits but... 

what's the point of having any kind of different streams at all then?  It's the _differences_ between them that makes composing them useful.  Compression, encryption, filtering, sockets, files, circular, etc.  You think you'll be able to do all that and get away with all of them having exactly the same API?

IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..

If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).

How does that solution allow uniformity in stream-using code?




 


2013/11/13 Chris Muller <[hidden email]>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
<[hidden email]> wrote:
> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)

By wrappers you mean the Xtreams are the innards doing the work and
the wrappers providing the legacy API?

This would be a great way to test Xtreams.

> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
> ...), otherwise we will import all the cruft in Xtream and would go back to
> our starting point...
> Once the minimal support written (a few hours should be enough), we should
> gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions
> (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low
> level binary stream, but that means we have to be very cautious with buffers
> and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access
> the file name from the Stream API, arghh). Those usages are bad and require
> a rewrite.

Are you saying a FileStream knowing its #name or #filename is bad?






12