In-memory FileSystem write streams not being polymorphic

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Frank Shearar-3
On 13 November 2013 16:48, Chris Muller <[hidden email]> wrote:

> I know nothing about Xtreams but, IMO, this obsession with sterility
> borders on mental illness.
>
> "All sort of un-needed complexity?"  That's overstating it a bit,
> don't you think?
>
> So you must really feel stressed that ALL Object's, in fact, have a
> #name, huh?  I admit this seems to push the limits but...
>
> what's the point of having any kind of different streams at all then?
> It's the _differences_ between them that makes composing them useful.
> Compression, encryption, filtering, sockets, files, circular, etc.
> You think you'll be able to do all that and get away with all of them
> having exactly the same API?

They don't have the same API. And they shouldn't. And that's Nicolas'
whole point. A Stream does not have a name (nor should it - what's the
name of the compressed output of an audio stream from your
microphone?). A FileStream can extend this API, and that's fine. But
keep that out of the Stream API.

So Nicolas is arguing that _difference_ should be reflected in the
API. File streams have names, but generic streams do not.

As it happens, Xtreams takes a very disciplined approach to this. Some
streams have positions, and so you can go back. Some can't. This is
reflected in the API. This is good.

> IMHO working around that, passing extra objects around, sounds more
> stressful than letting a stream on a _file_ know its filename..

Mu. Streams have no names. File streams have names.

frank

>> If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
>
> How does that solution allow uniformity in stream-using code?
>
> On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
> <[hidden email]> wrote:
>> Yes, a Wrapper would provide the legacy API.
>>
>> And yes, the name of a stream should better not be part of the API.
>> Most stream don't have a name, and adding such API adds all sort of
>> un-needed complexity.
>> It's an internal detail that can eventually help for reporting error if
>> accessible, but nothing more.
>>
>> If it really need it, the application certainly can retrieve the name from a
>> higher level object (a FIleReference, FileDirectory or whatever).
>>
>>
>> 2013/11/13 Chris Muller <[hidden email]>
>>>
>>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>>> <[hidden email]> wrote:
>>> > It's just a matter of selecting a strategy. I've proposed two:
>>> > A) create a wrapper class for legacy Stream compatibility selectors
>>> > B) create extensions for Legacy Stream compatibility selectors
>>> > My preference goes to A)
>>>
>>> By wrappers you mean the Xtreams are the innards doing the work and
>>> the wrappers providing the legacy API?
>>>
>>> This would be a great way to test Xtreams.
>>>
>>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
>>> > ...), otherwise we will import all the cruft in Xtream and would go back
>>> > to
>>> > our starting point...
>>> > Once the minimal support written (a few hours should be enough), we
>>> > should
>>> > gradually switch each every legacy Stream usage -> Xtream.
>>> >
>>> > An area which require more work is those Streams that have mixed
>>> > conventions
>>> > (one portion is interpreted as text, another as binary).
>>> > In theory that's easy, we just have two streams and they both wrap on a
>>> > low
>>> > level binary stream, but that means we have to be very cautious with
>>> > buffers
>>> > and caches.
>>> >
>>> > Another area of work is usage of ugly selectors like name (we try to
>>> > access
>>> > the file name from the Stream API, arghh). Those usages are bad and
>>> > require
>>> > a rewrite.
>>>
>>> Are you saying a FileStream knowing its #name or #filename is bad?
>>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
Exactly, every specialized stream has its specialized API and/or specialized implementation.
File streams don't even have a name, because they need not to.
You can browse XTFileReadStream and XTFileWriteStream, you won't find such thing.
The file has a name, the stream does not need any.
Before adding anything to a stream, ask first, why am i going to need this?


2013/11/13 Frank Shearar <[hidden email]>
On 13 November 2013 16:48, Chris Muller <[hidden email]> wrote:
> I know nothing about Xtreams but, IMO, this obsession with sterility
> borders on mental illness.
>
> "All sort of un-needed complexity?"  That's overstating it a bit,
> don't you think?
>
> So you must really feel stressed that ALL Object's, in fact, have a
> #name, huh?  I admit this seems to push the limits but...
>
> what's the point of having any kind of different streams at all then?
> It's the _differences_ between them that makes composing them useful.
> Compression, encryption, filtering, sockets, files, circular, etc.
> You think you'll be able to do all that and get away with all of them
> having exactly the same API?

They don't have the same API. And they shouldn't. And that's Nicolas'
whole point. A Stream does not have a name (nor should it - what's the
name of the compressed output of an audio stream from your
microphone?). A FileStream can extend this API, and that's fine. But
keep that out of the Stream API.

So Nicolas is arguing that _difference_ should be reflected in the
API. File streams have names, but generic streams do not.

As it happens, Xtreams takes a very disciplined approach to this. Some
streams have positions, and so you can go back. Some can't. This is
reflected in the API. This is good.

> IMHO working around that, passing extra objects around, sounds more
> stressful than letting a stream on a _file_ know its filename..

Mu. Streams have no names. File streams have names.

frank

>> If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
>
> How does that solution allow uniformity in stream-using code?
>
> On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
> <[hidden email]> wrote:
>> Yes, a Wrapper would provide the legacy API.
>>
>> And yes, the name of a stream should better not be part of the API.
>> Most stream don't have a name, and adding such API adds all sort of
>> un-needed complexity.
>> It's an internal detail that can eventually help for reporting error if
>> accessible, but nothing more.
>>
>> If it really need it, the application certainly can retrieve the name from a
>> higher level object (a FIleReference, FileDirectory or whatever).
>>
>>
>> 2013/11/13 Chris Muller <[hidden email]>
>>>
>>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>>> <[hidden email]> wrote:
>>> > It's just a matter of selecting a strategy. I've proposed two:
>>> > A) create a wrapper class for legacy Stream compatibility selectors
>>> > B) create extensions for Legacy Stream compatibility selectors
>>> > My preference goes to A)
>>>
>>> By wrappers you mean the Xtreams are the innards doing the work and
>>> the wrappers providing the legacy API?
>>>
>>> This would be a great way to test Xtreams.
>>>
>>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
>>> > ...), otherwise we will import all the cruft in Xtream and would go back
>>> > to
>>> > our starting point...
>>> > Once the minimal support written (a few hours should be enough), we
>>> > should
>>> > gradually switch each every legacy Stream usage -> Xtream.
>>> >
>>> > An area which require more work is those Streams that have mixed
>>> > conventions
>>> > (one portion is interpreted as text, another as binary).
>>> > In theory that's easy, we just have two streams and they both wrap on a
>>> > low
>>> > level binary stream, but that means we have to be very cautious with
>>> > buffers
>>> > and caches.
>>> >
>>> > Another area of work is usage of ugly selectors like name (we try to
>>> > access
>>> > the file name from the Stream API, arghh). Those usages are bad and
>>> > require
>>> > a rewrite.
>>>
>>> Are you saying a FileStream knowing its #name or #filename is bad?
>>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Chris Muller-3
On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier
<[hidden email]> wrote:
> Exactly, every specialized stream has its specialized API and/or specialized
> implementation.
> File streams don't even have a name, because they need not to.
> You can browse XTFileReadStream and XTFileWriteStream, you won't find such
> thing.
> The file has a name, the stream does not need any.
> Before adding anything to a stream, ask first, why am i going to need this?

I agree with what Frank said, but not your suggestion that:

> File streams don't even have a name, because they need not to.
> If it really need it, the application certainly can retrieve the name from a higher level object (a
> FIleReference, FileDirectory or whatever).

because doing that means the FileStream is no longer
fully-encapsulated.  And now for the same method to support a
non-FileStream stream, what will be passed as the new FileReference
argument?  nil?  So the code went from stream-delegation to
case-logic.

This is why organic growth happened in the original Stream hierarchy
-- because it was "needed", not random.  Not strict, yes, but not
"random" either.

One final thing to consider about selector "uniformity" is how it can
negate the ability to effectively trace code (via senders and
implementors).  Wow, how many senders of #next...?

Stream-composition is cool, though, and I understand that to be a
benefit of the stricter API.



> 2013/11/13 Frank Shearar <[hidden email]>
>>
>> On 13 November 2013 16:48, Chris Muller <[hidden email]> wrote:
>> > I know nothing about Xtreams but, IMO, this obsession with sterility
>> > borders on mental illness.
>> >
>> > "All sort of un-needed complexity?"  That's overstating it a bit,
>> > don't you think?
>> >
>> > So you must really feel stressed that ALL Object's, in fact, have a
>> > #name, huh?  I admit this seems to push the limits but...
>> >
>> > what's the point of having any kind of different streams at all then?
>> > It's the _differences_ between them that makes composing them useful.
>> > Compression, encryption, filtering, sockets, files, circular, etc.
>> > You think you'll be able to do all that and get away with all of them
>> > having exactly the same API?
>>
>> They don't have the same API. And they shouldn't. And that's Nicolas'
>> whole point. A Stream does not have a name (nor should it - what's the
>> name of the compressed output of an audio stream from your
>> microphone?). A FileStream can extend this API, and that's fine. But
>> keep that out of the Stream API.
>>
>> So Nicolas is arguing that _difference_ should be reflected in the
>> API. File streams have names, but generic streams do not.
>>
>> As it happens, Xtreams takes a very disciplined approach to this. Some
>> streams have positions, and so you can go back. Some can't. This is
>> reflected in the API. This is good.
>>
>> > IMHO working around that, passing extra objects around, sounds more
>> > stressful than letting a stream on a _file_ know its filename..
>>
>> Mu. Streams have no names. File streams have names.
>>
>> frank
>>
>> >> If it really need it, the application certainly can retrieve the name
>> >> from a higher level object (a FIleReference, FileDirectory or whatever).
>> >
>> > How does that solution allow uniformity in stream-using code?
>> >
>> > On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
>> > <[hidden email]> wrote:
>> >> Yes, a Wrapper would provide the legacy API.
>> >>
>> >> And yes, the name of a stream should better not be part of the API.
>> >> Most stream don't have a name, and adding such API adds all sort of
>> >> un-needed complexity.
>> >> It's an internal detail that can eventually help for reporting error if
>> >> accessible, but nothing more.
>> >>
>> >> If it really need it, the application certainly can retrieve the name
>> >> from a
>> >> higher level object (a FIleReference, FileDirectory or whatever).
>> >>
>> >>
>> >> 2013/11/13 Chris Muller <[hidden email]>
>> >>>
>> >>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>> >>> <[hidden email]> wrote:
>> >>> > It's just a matter of selecting a strategy. I've proposed two:
>> >>> > A) create a wrapper class for legacy Stream compatibility selectors
>> >>> > B) create extensions for Legacy Stream compatibility selectors
>> >>> > My preference goes to A)
>> >>>
>> >>> By wrappers you mean the Xtreams are the innards doing the work and
>> >>> the wrappers providing the legacy API?
>> >>>
>> >>> This would be a great way to test Xtreams.
>> >>>
>> >>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek
>> >>> > upTo:
>> >>> > ...), otherwise we will import all the cruft in Xtream and would go
>> >>> > back
>> >>> > to
>> >>> > our starting point...
>> >>> > Once the minimal support written (a few hours should be enough), we
>> >>> > should
>> >>> > gradually switch each every legacy Stream usage -> Xtream.
>> >>> >
>> >>> > An area which require more work is those Streams that have mixed
>> >>> > conventions
>> >>> > (one portion is interpreted as text, another as binary).
>> >>> > In theory that's easy, we just have two streams and they both wrap
>> >>> > on a
>> >>> > low
>> >>> > level binary stream, but that means we have to be very cautious with
>> >>> > buffers
>> >>> > and caches.
>> >>> >
>> >>> > Another area of work is usage of ugly selectors like name (we try to
>> >>> > access
>> >>> > the file name from the Stream API, arghh). Those usages are bad and
>> >>> > require
>> >>> > a rewrite.
>> >>>
>> >>> Are you saying a FileStream knowing its #name or #filename is bad?
>> >>>
>> >>
>> >
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stéphane Ducasse
In reply to this post by Nicolas Cellier
thanks for this discussion it is interesting. 
I like small api fully implemented vs large and bogus ones

Stef

On Nov 13, 2013, at 6:47 PM, Nicolas Cellier <[hidden email]> wrote:


---------- Forwarded message ----------
From: Nicolas Cellier <[hidden email]>
Date: 2013/11/13
Subject: Re: [Pharo-dev] In-memory FileSystem write streams not being polymorphic

The long answer is simple: the more responsibility you put in Stream, the more complexity you get.
The first complexity that I'm speaking of is non-uniformity and random implementation (or un-implementation) of a set of features.
I mean some streams in the huge hierarchy implement only half of the contract, but hey, what was the contract exactly?
Ah, yes, there were no contract, just hacks left here and there randomly and concurrently in a big hierarchy.
I add a new subclass, but don't implement all the features, there are too many of them, and I don't know them all..
I add a new feature, but don't implement it in all the classes, there are too many of them, and I don't know them all.
This procedure invariably ends up with a blob of un-maintanable code, were two stream would not even agree on upToEnd behavior - I think you remember it :)

If the goal is to replicate all the accumulated responsibilities of Squeak Streams, but with a clarified contract, I think this is a dead end: too much cruft to support, too many features, too many undefined or not well defined behaviors to be clarified.

Xtreams takes the opposite path: concentrate on constructing a common, simple and uniform API, concerning streams, just basic stream methods.
Some specialized streams then implement some specialized behavior, and you can compose them (by wrapping) when you need the specific API.
This way, you ain't got to implement/maintain feature A into a few dozen of classes.
And your brand new SpecialFeatureBStream only has to implement a few well known behaviours and your Special Feature B.

The short answer is even more simple : a stream does not have a name, like a collection does not have a name, because we ain't gonna need it.
If we really need a name, then we'll create a XTNamedReadStream and a XTNamedWriteStream responding to #name, but I doubt we'll do.


2013/11/13 Chris Muller <[hidden email]>
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <[hidden email]> wrote:
Yes, a Wrapper would provide the legacy API.

And yes, the name of a stream should better not be part of the API.
Most stream don't have a name, and adding such API adds all sort of un-needed complexity.
It's an internal detail that can eventually help for reporting error if accessible, but nothing more.

 
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.

"All sort of un-needed complexity?"  That's overstating it a bit, don't you think?

So you must really feel stressed that ALL Object's, in fact, have a #name, huh?  I admit this seems to push the limits but... 

what's the point of having any kind of different streams at all then?  It's the _differences_ between them that makes composing them useful.  Compression, encryption, filtering, sockets, files, circular, etc.  You think you'll be able to do all that and get away with all of them having exactly the same API?

IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..

If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).

How does that solution allow uniformity in stream-using code?




 


2013/11/13 Chris Muller <[hidden email]>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
<[hidden email]> wrote:
> It's just a matter of selecting a strategy. I've proposed two:
> A) create a wrapper class for legacy Stream compatibility selectors
> B) create extensions for Legacy Stream compatibility selectors
> My preference goes to A)

By wrappers you mean the Xtreams are the innards doing the work and
the wrappers providing the legacy API?

This would be a great way to test Xtreams.

> The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo:
> ...), otherwise we will import all the cruft in Xtream and would go back to
> our starting point...
> Once the minimal support written (a few hours should be enough), we should
> gradually switch each every legacy Stream usage -> Xtream.
>
> An area which require more work is those Streams that have mixed conventions
> (one portion is interpreted as text, another as binary).
> In theory that's easy, we just have two streams and they both wrap on a low
> level binary stream, but that means we have to be very cautious with buffers
> and caches.
>
> Another area of work is usage of ugly selectors like name (we try to access
> the file name from the Stream API, arghh). Those usages are bad and require
> a rewrite.

Are you saying a FileStream knowing its #name or #filename is bad?






Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
In reply to this post by Chris Muller-3
Chris,
I have to completely disagree.
The idea of breaking a complex object into simpler parts is not just theories and extremism.
Assigning several roles to an object is sometimes convenient for a start, but generally does scale very badly.
It's a collective experience driven by pragmatic considerations in developping in all sort of languages.
And it's my own experience too.

A file is like the container, the stream is about reading/writing the contents.
I think I remember the confusion was made at the beginning in st-80, because it was simple enough at that time.
So it's rather an historical slag, not something that was added later for this special case.
And it is something that should have been revised, because when we added pipes, sockets or other external streams, that ruined the idea of confusing the two things into a not well delimited blob without clear responsibilities.
Though we kept the original implementation and started to distort the package by adding faked polymorphism like name ^'john doe'.

Clearly, I need no stream on its contents to rename a file, query its permissions, or delete it, so why bother with internal states of a stream if I'm interested in handling the file?
If I am interested about the contents, I don't need to ever know about the path of the file nor its name, or shall I change a $A read from it into a $B if the file is named 'turnAllAinB'?

Where exactly are we going to need this encapsulated blob which pretend to be both a file and a stream? I would be happy to show that a rewrite is both simple and good looking at end application sites.
Of course, inside the lava of multi year hacked stream hierarchy, it might be a bit less obvious to disentangle the spaghetti.
Hence more radical solutions: table rase and complete rewrite.





2013/11/13 Chris Muller <[hidden email]>
On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier
<[hidden email]> wrote:
> Exactly, every specialized stream has its specialized API and/or specialized
> implementation.
> File streams don't even have a name, because they need not to.
> You can browse XTFileReadStream and XTFileWriteStream, you won't find such
> thing.
> The file has a name, the stream does not need any.
> Before adding anything to a stream, ask first, why am i going to need this?

I agree with what Frank said, but not your suggestion that:

> File streams don't even have a name, because they need not to.
> If it really need it, the application certainly can retrieve the name from a higher level object (a
> FIleReference, FileDirectory or whatever).

because doing that means the FileStream is no longer
fully-encapsulated.  And now for the same method to support a
non-FileStream stream, what will be passed as the new FileReference
argument?  nil?  So the code went from stream-delegation to
case-logic.

This is why organic growth happened in the original Stream hierarchy
-- because it was "needed", not random.  Not strict, yes, but not
"random" either.

One final thing to consider about selector "uniformity" is how it can
negate the ability to effectively trace code (via senders and
implementors).  Wow, how many senders of #next...?

Stream-composition is cool, though, and I understand that to be a
benefit of the stricter API.



> 2013/11/13 Frank Shearar <[hidden email]>
>>
>> On 13 November 2013 16:48, Chris Muller <[hidden email]> wrote:
>> > I know nothing about Xtreams but, IMO, this obsession with sterility
>> > borders on mental illness.
>> >
>> > "All sort of un-needed complexity?"  That's overstating it a bit,
>> > don't you think?
>> >
>> > So you must really feel stressed that ALL Object's, in fact, have a
>> > #name, huh?  I admit this seems to push the limits but...
>> >
>> > what's the point of having any kind of different streams at all then?
>> > It's the _differences_ between them that makes composing them useful.
>> > Compression, encryption, filtering, sockets, files, circular, etc.
>> > You think you'll be able to do all that and get away with all of them
>> > having exactly the same API?
>>
>> They don't have the same API. And they shouldn't. And that's Nicolas'
>> whole point. A Stream does not have a name (nor should it - what's the
>> name of the compressed output of an audio stream from your
>> microphone?). A FileStream can extend this API, and that's fine. But
>> keep that out of the Stream API.
>>
>> So Nicolas is arguing that _difference_ should be reflected in the
>> API. File streams have names, but generic streams do not.
>>
>> As it happens, Xtreams takes a very disciplined approach to this. Some
>> streams have positions, and so you can go back. Some can't. This is
>> reflected in the API. This is good.
>>
>> > IMHO working around that, passing extra objects around, sounds more
>> > stressful than letting a stream on a _file_ know its filename..
>>
>> Mu. Streams have no names. File streams have names.
>>
>> frank
>>
>> >> If it really need it, the application certainly can retrieve the name
>> >> from a higher level object (a FIleReference, FileDirectory or whatever).
>> >
>> > How does that solution allow uniformity in stream-using code?
>> >
>> > On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
>> > <[hidden email]> wrote:
>> >> Yes, a Wrapper would provide the legacy API.
>> >>
>> >> And yes, the name of a stream should better not be part of the API.
>> >> Most stream don't have a name, and adding such API adds all sort of
>> >> un-needed complexity.
>> >> It's an internal detail that can eventually help for reporting error if
>> >> accessible, but nothing more.
>> >>
>> >> If it really need it, the application certainly can retrieve the name
>> >> from a
>> >> higher level object (a FIleReference, FileDirectory or whatever).
>> >>
>> >>
>> >> 2013/11/13 Chris Muller <[hidden email]>
>> >>>
>> >>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>> >>> <[hidden email]> wrote:
>> >>> > It's just a matter of selecting a strategy. I've proposed two:
>> >>> > A) create a wrapper class for legacy Stream compatibility selectors
>> >>> > B) create extensions for Legacy Stream compatibility selectors
>> >>> > My preference goes to A)
>> >>>
>> >>> By wrappers you mean the Xtreams are the innards doing the work and
>> >>> the wrappers providing the legacy API?
>> >>>
>> >>> This would be a great way to test Xtreams.
>> >>>
>> >>> > The legacy support MUST be minimal (next nextPut: nextPutAll: peek
>> >>> > upTo:
>> >>> > ...), otherwise we will import all the cruft in Xtream and would go
>> >>> > back
>> >>> > to
>> >>> > our starting point...
>> >>> > Once the minimal support written (a few hours should be enough), we
>> >>> > should
>> >>> > gradually switch each every legacy Stream usage -> Xtream.
>> >>> >
>> >>> > An area which require more work is those Streams that have mixed
>> >>> > conventions
>> >>> > (one portion is interpreted as text, another as binary).
>> >>> > In theory that's easy, we just have two streams and they both wrap
>> >>> > on a
>> >>> > low
>> >>> > level binary stream, but that means we have to be very cautious with
>> >>> > buffers
>> >>> > and caches.
>> >>> >
>> >>> > Another area of work is usage of ugly selectors like name (we try to
>> >>> > access
>> >>> > the file name from the Stream API, arghh). Those usages are bad and
>> >>> > require
>> >>> > a rewrite.
>> >>>
>> >>> Are you saying a FileStream knowing its #name or #filename is bad?
>> >>>
>> >>
>> >
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Chris Muller-3
These are really excellent arguments, Nicolas.  Good discussion.

On Wed, Nov 13, 2013 at 3:32 PM, Nicolas Cellier
<[hidden email]> wrote:

> Chris,
> I have to completely disagree.
> The idea of breaking a complex object into simpler parts is not just
> theories and extremism.
> Assigning several roles to an object is sometimes convenient for a start,
> but generally does scale very badly.
> It's a collective experience driven by pragmatic considerations in
> developping in all sort of languages.
> And it's my own experience too.
>
> A file is like the container, the stream is about reading/writing the
> contents.
> I think I remember the confusion was made at the beginning in st-80, because
> it was simple enough at that time.
> So it's rather an historical slag, not something that was added later for
> this special case.
> And it is something that should have been revised, because when we added
> pipes, sockets or other external streams, that ruined the idea of confusing
> the two things into a not well delimited blob without clear
> responsibilities.
> Though we kept the original implementation and started to distort the
> package by adding faked polymorphism like name ^'john doe'.
>
> Clearly, I need no stream on its contents to rename a file, query its
> permissions, or delete it, so why bother with internal states of a stream if
> I'm interested in handling the file?
> If I am interested about the contents, I don't need to ever know about the
> path of the file nor its name, or shall I change a $A read from it into a $B
> if the file is named 'turnAllAinB'?
>
> Where exactly are we going to need this encapsulated blob which pretend to
> be both a file and a stream? I would be happy to show that a rewrite is both
> simple and good looking at end application sites.
> Of course, inside the lava of multi year hacked stream hierarchy, it might
> be a bit less obvious to disentangle the spaghetti.
> Hence more radical solutions: table rase and complete rewrite.
>
>
>
>
>
> 2013/11/13 Chris Muller <[hidden email]>
>>
>> On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier
>> <[hidden email]> wrote:
>> > Exactly, every specialized stream has its specialized API and/or
>> > specialized
>> > implementation.
>> > File streams don't even have a name, because they need not to.
>> > You can browse XTFileReadStream and XTFileWriteStream, you won't find
>> > such
>> > thing.
>> > The file has a name, the stream does not need any.
>> > Before adding anything to a stream, ask first, why am i going to need
>> > this?
>>
>> I agree with what Frank said, but not your suggestion that:
>>
>> > File streams don't even have a name, because they need not to.
>> > If it really need it, the application certainly can retrieve the name
>> > from a higher level object (a
>> > FIleReference, FileDirectory or whatever).
>>
>> because doing that means the FileStream is no longer
>> fully-encapsulated.  And now for the same method to support a
>> non-FileStream stream, what will be passed as the new FileReference
>> argument?  nil?  So the code went from stream-delegation to
>> case-logic.
>>
>> This is why organic growth happened in the original Stream hierarchy
>> -- because it was "needed", not random.  Not strict, yes, but not
>> "random" either.
>>
>> One final thing to consider about selector "uniformity" is how it can
>> negate the ability to effectively trace code (via senders and
>> implementors).  Wow, how many senders of #next...?
>>
>> Stream-composition is cool, though, and I understand that to be a
>> benefit of the stricter API.
>>
>>
>>
>> > 2013/11/13 Frank Shearar <[hidden email]>
>> >>
>> >> On 13 November 2013 16:48, Chris Muller <[hidden email]> wrote:
>> >> > I know nothing about Xtreams but, IMO, this obsession with sterility
>> >> > borders on mental illness.
>> >> >
>> >> > "All sort of un-needed complexity?"  That's overstating it a bit,
>> >> > don't you think?
>> >> >
>> >> > So you must really feel stressed that ALL Object's, in fact, have a
>> >> > #name, huh?  I admit this seems to push the limits but...
>> >> >
>> >> > what's the point of having any kind of different streams at all then?
>> >> > It's the _differences_ between them that makes composing them useful.
>> >> > Compression, encryption, filtering, sockets, files, circular, etc.
>> >> > You think you'll be able to do all that and get away with all of them
>> >> > having exactly the same API?
>> >>
>> >> They don't have the same API. And they shouldn't. And that's Nicolas'
>> >> whole point. A Stream does not have a name (nor should it - what's the
>> >> name of the compressed output of an audio stream from your
>> >> microphone?). A FileStream can extend this API, and that's fine. But
>> >> keep that out of the Stream API.
>> >>
>> >> So Nicolas is arguing that _difference_ should be reflected in the
>> >> API. File streams have names, but generic streams do not.
>> >>
>> >> As it happens, Xtreams takes a very disciplined approach to this. Some
>> >> streams have positions, and so you can go back. Some can't. This is
>> >> reflected in the API. This is good.
>> >>
>> >> > IMHO working around that, passing extra objects around, sounds more
>> >> > stressful than letting a stream on a _file_ know its filename..
>> >>
>> >> Mu. Streams have no names. File streams have names.
>> >>
>> >> frank
>> >>
>> >> >> If it really need it, the application certainly can retrieve the
>> >> >> name
>> >> >> from a higher level object (a FIleReference, FileDirectory or
>> >> >> whatever).
>> >> >
>> >> > How does that solution allow uniformity in stream-using code?
>> >> >
>> >> > On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
>> >> > <[hidden email]> wrote:
>> >> >> Yes, a Wrapper would provide the legacy API.
>> >> >>
>> >> >> And yes, the name of a stream should better not be part of the API.
>> >> >> Most stream don't have a name, and adding such API adds all sort of
>> >> >> un-needed complexity.
>> >> >> It's an internal detail that can eventually help for reporting error
>> >> >> if
>> >> >> accessible, but nothing more.
>> >> >>
>> >> >> If it really need it, the application certainly can retrieve the
>> >> >> name
>> >> >> from a
>> >> >> higher level object (a FIleReference, FileDirectory or whatever).
>> >> >>
>> >> >>
>> >> >> 2013/11/13 Chris Muller <[hidden email]>
>> >> >>>
>> >> >>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>> >> >>> <[hidden email]> wrote:
>> >> >>> > It's just a matter of selecting a strategy. I've proposed two:
>> >> >>> > A) create a wrapper class for legacy Stream compatibility
>> >> >>> > selectors
>> >> >>> > B) create extensions for Legacy Stream compatibility selectors
>> >> >>> > My preference goes to A)
>> >> >>>
>> >> >>> By wrappers you mean the Xtreams are the innards doing the work and
>> >> >>> the wrappers providing the legacy API?
>> >> >>>
>> >> >>> This would be a great way to test Xtreams.
>> >> >>>
>> >> >>> > The legacy support MUST be minimal (next nextPut: nextPutAll:
>> >> >>> > peek
>> >> >>> > upTo:
>> >> >>> > ...), otherwise we will import all the cruft in Xtream and would
>> >> >>> > go
>> >> >>> > back
>> >> >>> > to
>> >> >>> > our starting point...
>> >> >>> > Once the minimal support written (a few hours should be enough),
>> >> >>> > we
>> >> >>> > should
>> >> >>> > gradually switch each every legacy Stream usage -> Xtream.
>> >> >>> >
>> >> >>> > An area which require more work is those Streams that have mixed
>> >> >>> > conventions
>> >> >>> > (one portion is interpreted as text, another as binary).
>> >> >>> > In theory that's easy, we just have two streams and they both
>> >> >>> > wrap
>> >> >>> > on a
>> >> >>> > low
>> >> >>> > level binary stream, but that means we have to be very cautious
>> >> >>> > with
>> >> >>> > buffers
>> >> >>> > and caches.
>> >> >>> >
>> >> >>> > Another area of work is usage of ugly selectors like name (we try
>> >> >>> > to
>> >> >>> > access
>> >> >>> > the file name from the Stream API, arghh). Those usages are bad
>> >> >>> > and
>> >> >>> > require
>> >> >>> > a rewrite.
>> >> >>>
>> >> >>> Are you saying a FileStream knowing its #name or #filename is bad?
>> >> >>>
>> >> >>
>> >> >
>> >>
>> >
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
Of course, since this is a question of style, all was invented way before the computer era :)
https://en.wikipedia.org/wiki/Metonymy#Synecdoche


2013/11/13 Chris Muller <[hidden email]>
These are really excellent arguments, Nicolas.  Good discussion.

On Wed, Nov 13, 2013 at 3:32 PM, Nicolas Cellier
<[hidden email]> wrote:
> Chris,
> I have to completely disagree.
> The idea of breaking a complex object into simpler parts is not just
> theories and extremism.
> Assigning several roles to an object is sometimes convenient for a start,
> but generally does scale very badly.
> It's a collective experience driven by pragmatic considerations in
> developping in all sort of languages.
> And it's my own experience too.
>
> A file is like the container, the stream is about reading/writing the
> contents.
> I think I remember the confusion was made at the beginning in st-80, because
> it was simple enough at that time.
> So it's rather an historical slag, not something that was added later for
> this special case.
> And it is something that should have been revised, because when we added
> pipes, sockets or other external streams, that ruined the idea of confusing
> the two things into a not well delimited blob without clear
> responsibilities.
> Though we kept the original implementation and started to distort the
> package by adding faked polymorphism like name ^'john doe'.
>
> Clearly, I need no stream on its contents to rename a file, query its
> permissions, or delete it, so why bother with internal states of a stream if
> I'm interested in handling the file?
> If I am interested about the contents, I don't need to ever know about the
> path of the file nor its name, or shall I change a $A read from it into a $B
> if the file is named 'turnAllAinB'?
>
> Where exactly are we going to need this encapsulated blob which pretend to
> be both a file and a stream? I would be happy to show that a rewrite is both
> simple and good looking at end application sites.
> Of course, inside the lava of multi year hacked stream hierarchy, it might
> be a bit less obvious to disentangle the spaghetti.
> Hence more radical solutions: table rase and complete rewrite.
>
>
>
>
>
> 2013/11/13 Chris Muller <[hidden email]>
>>
>> On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier
>> <[hidden email]> wrote:
>> > Exactly, every specialized stream has its specialized API and/or
>> > specialized
>> > implementation.
>> > File streams don't even have a name, because they need not to.
>> > You can browse XTFileReadStream and XTFileWriteStream, you won't find
>> > such
>> > thing.
>> > The file has a name, the stream does not need any.
>> > Before adding anything to a stream, ask first, why am i going to need
>> > this?
>>
>> I agree with what Frank said, but not your suggestion that:
>>
>> > File streams don't even have a name, because they need not to.
>> > If it really need it, the application certainly can retrieve the name
>> > from a higher level object (a
>> > FIleReference, FileDirectory or whatever).
>>
>> because doing that means the FileStream is no longer
>> fully-encapsulated.  And now for the same method to support a
>> non-FileStream stream, what will be passed as the new FileReference
>> argument?  nil?  So the code went from stream-delegation to
>> case-logic.
>>
>> This is why organic growth happened in the original Stream hierarchy
>> -- because it was "needed", not random.  Not strict, yes, but not
>> "random" either.
>>
>> One final thing to consider about selector "uniformity" is how it can
>> negate the ability to effectively trace code (via senders and
>> implementors).  Wow, how many senders of #next...?
>>
>> Stream-composition is cool, though, and I understand that to be a
>> benefit of the stricter API.
>>
>>
>>
>> > 2013/11/13 Frank Shearar <[hidden email]>
>> >>
>> >> On 13 November 2013 16:48, Chris Muller <[hidden email]> wrote:
>> >> > I know nothing about Xtreams but, IMO, this obsession with sterility
>> >> > borders on mental illness.
>> >> >
>> >> > "All sort of un-needed complexity?"  That's overstating it a bit,
>> >> > don't you think?
>> >> >
>> >> > So you must really feel stressed that ALL Object's, in fact, have a
>> >> > #name, huh?  I admit this seems to push the limits but...
>> >> >
>> >> > what's the point of having any kind of different streams at all then?
>> >> > It's the _differences_ between them that makes composing them useful.
>> >> > Compression, encryption, filtering, sockets, files, circular, etc.
>> >> > You think you'll be able to do all that and get away with all of them
>> >> > having exactly the same API?
>> >>
>> >> They don't have the same API. And they shouldn't. And that's Nicolas'
>> >> whole point. A Stream does not have a name (nor should it - what's the
>> >> name of the compressed output of an audio stream from your
>> >> microphone?). A FileStream can extend this API, and that's fine. But
>> >> keep that out of the Stream API.
>> >>
>> >> So Nicolas is arguing that _difference_ should be reflected in the
>> >> API. File streams have names, but generic streams do not.
>> >>
>> >> As it happens, Xtreams takes a very disciplined approach to this. Some
>> >> streams have positions, and so you can go back. Some can't. This is
>> >> reflected in the API. This is good.
>> >>
>> >> > IMHO working around that, passing extra objects around, sounds more
>> >> > stressful than letting a stream on a _file_ know its filename..
>> >>
>> >> Mu. Streams have no names. File streams have names.
>> >>
>> >> frank
>> >>
>> >> >> If it really need it, the application certainly can retrieve the
>> >> >> name
>> >> >> from a higher level object (a FIleReference, FileDirectory or
>> >> >> whatever).
>> >> >
>> >> > How does that solution allow uniformity in stream-using code?
>> >> >
>> >> > On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier
>> >> > <[hidden email]> wrote:
>> >> >> Yes, a Wrapper would provide the legacy API.
>> >> >>
>> >> >> And yes, the name of a stream should better not be part of the API.
>> >> >> Most stream don't have a name, and adding such API adds all sort of
>> >> >> un-needed complexity.
>> >> >> It's an internal detail that can eventually help for reporting error
>> >> >> if
>> >> >> accessible, but nothing more.
>> >> >>
>> >> >> If it really need it, the application certainly can retrieve the
>> >> >> name
>> >> >> from a
>> >> >> higher level object (a FIleReference, FileDirectory or whatever).
>> >> >>
>> >> >>
>> >> >> 2013/11/13 Chris Muller <[hidden email]>
>> >> >>>
>> >> >>> On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier
>> >> >>> <[hidden email]> wrote:
>> >> >>> > It's just a matter of selecting a strategy. I've proposed two:
>> >> >>> > A) create a wrapper class for legacy Stream compatibility
>> >> >>> > selectors
>> >> >>> > B) create extensions for Legacy Stream compatibility selectors
>> >> >>> > My preference goes to A)
>> >> >>>
>> >> >>> By wrappers you mean the Xtreams are the innards doing the work and
>> >> >>> the wrappers providing the legacy API?
>> >> >>>
>> >> >>> This would be a great way to test Xtreams.
>> >> >>>
>> >> >>> > The legacy support MUST be minimal (next nextPut: nextPutAll:
>> >> >>> > peek
>> >> >>> > upTo:
>> >> >>> > ...), otherwise we will import all the cruft in Xtream and would
>> >> >>> > go
>> >> >>> > back
>> >> >>> > to
>> >> >>> > our starting point...
>> >> >>> > Once the minimal support written (a few hours should be enough),
>> >> >>> > we
>> >> >>> > should
>> >> >>> > gradually switch each every legacy Stream usage -> Xtream.
>> >> >>> >
>> >> >>> > An area which require more work is those Streams that have mixed
>> >> >>> > conventions
>> >> >>> > (one portion is interpreted as text, another as binary).
>> >> >>> > In theory that's easy, we just have two streams and they both
>> >> >>> > wrap
>> >> >>> > on a
>> >> >>> > low
>> >> >>> > level binary stream, but that means we have to be very cautious
>> >> >>> > with
>> >> >>> > buffers
>> >> >>> > and caches.
>> >> >>> >
>> >> >>> > Another area of work is usage of ugly selectors like name (we try
>> >> >>> > to
>> >> >>> > access
>> >> >>> > the file name from the Stream API, arghh). Those usages are bad
>> >> >>> > and
>> >> >>> > require
>> >> >>> > a rewrite.
>> >> >>>
>> >> >>> Are you saying a FileStream knowing its #name or #filename is bad?
>> >> >>>
>> >> >>
>> >> >
>> >>
>> >
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolás Papagna Maldonado
This post was updated on .
In reply to this post by Camillo Bruni-3
Thanks for the suggestions!

As you said, I finally managed to get over the isReadOnly message.

I've just commited the fix.

Cheers,

Nico PM
Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolas Cellier
I have prototyped some of legacy Squeak/Pharo Stream API wrapped over Xtreams at http://www.squeaksource.com/Xtreams - see Xtreams-LegacyStreamAPI package.


2013/11/16 Nicolás Papagna Maldonado <[hidden email]>
Thanks for the suggestions!

As you said, I finally manages to get over the isReadOnly message.

I've just commited the fix.

Cheers,

Nico PM



--
View this message in context: http://forum.world.st/In-memory-FileSystem-write-streams-not-being-polymorphic-tp4721006p4722612.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.


Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Nicolás Papagna Maldonado
In reply to this post by Stéphane Ducasse
Hi Stef!

You're thinking of Xstream as an alternative FileSytemStore or to replace an specific part of the Files package?

Besides the specific issue I mentioned, how do the dev team feel about the current FileSystem-Core implementation?

Cheers,

Nico PM
Reply | Threaded
Open this post in threaded view
|

Re: In-memory FileSystem write streams not being polymorphic

Stéphane Ducasse

On Nov 17, 2013, at 3:47 AM, Nicolás Papagna Maldonado <[hidden email]> wrote:

> Hi Stef!
>
> You're thinking of Xstream as an alternative FileSytemStore or to replace an
> specific part of the Files package?

I did not look at the code of both but from what the people are saying I think that replacing everything would be better :)

> Besides the specific issue I mentioned, how do the dev team feel about the
> current FileSystem-Core implementation?

if you have specific feeling about it just let us know.
I know that camille spent time to make it using faster primitive.

Sometimes I do not like certin messages because they are not conveying what they really do.
For example, I'm always confused by resolve: but for some people it is obvious (I'm too much into interpreter to not
think how to resolve a variable with a context).

Stef


>
> Cheers,
>
> Nico PM
>
>
>
> --
> View this message in context: http://forum.world.st/In-memory-FileSystem-write-streams-not-being-polymorphic-tp4721006p4722771.html
> Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
>


12