[ANN] Nile: a trait-based stream library
Locked
[ANN] Nile: a trait-based stream library
 
|
Hi,
I have just released a first version of the Nile Stream hierarchy. This is a complete reimplementation of the squeak stream hierarchy. It provides: - traits for code reuse. I'm currently implementing new clients to test the granularity ; there are currently two clients: Random and History. - lots of tests: same tests are used for the squeak stream hierarchy and Nile (they are implemented in traits). 292 run, 2 expected failures for the squeak stream hierarchy. If you don't have this results, please tell me. - a diagram of the implementation: http://damien.cassou.free.fr/documents/internship_2007/nile/complete_hierarchy.pdf You can load Nile using: MCHttpRepository location: 'http://www.squeaksource.com/Nile' user: '' password: '' Load Nile-Tests to get Nile and all associated tests. You can also load Nile-All to have only Nile without the tests. If loading Nile-Tests asks you about loading an older dependency of Tests, you *must* agree. This is because it seems there is a problem with this package in last Squeak 3.10 alpha versions. Do you have some clients which will be worth implementing with the traits Nile provides? Please tell me. -- Damien Cassou |
Re: [ANN] Nile: a trait-based stream library
|
|
Hi Damien,
> Do you have some clients which will be worth implementing with the > traits Nile provides? Please tell me. I had a look at your implementation and it really looks nice. I have some feature requests ;-) - Right now the only available stream classes work on in-memory collections. I would like to be able to use them on Files and Sockets as well. - The current Squeak Stream implementation lacks the possibility (and it is difficult to do so in a clean way) to wrap streams with other streams. What I would like to be able to do wrappers that encode, decode (URL, HTML, BASE64, Cryptography, ...), filter, select, collect, parse streams in different ways. See [1] for some examples. The wrapping could also be used to provide synchronization on shared streams. - #upTo: is cool. For most purposes this is however too basic. I need many different efficient and reliable variations of that. It should be possible to read up to ... - the first sub-collection of elements - the first place a block predicate is satisfied - the first place a regular expression matches - the first place a of a set of the above conditions matches - etc. All that looks like a lot of work, however I think your framework could handle that in a very clean way. Just some ideas ;-) Cheers, Lukas [1] http://wiki.cs.uiuc.edu/PatternStories/FunWithStreams -- Lukas Renggli http://www.lukas-renggli.ch |
Re: [ANN] Nile: a trait-based stream library
|
|
Hi Lukas,
thank you very much for this really helpful comment. Here are some answers/questions: 2007/5/2, Lukas Renggli <[hidden email]>: > - Right now the only available stream classes work on in-memory > collections. I would like to be able to use them on Files and Sockets > as well. Implementing streams on Files and Sockets is a bit difficult for me because I haven't any knowledge about them. > - The current Squeak Stream implementation lacks the possibility (and > it is difficult to do so in a clean way) to wrap streams with other > streams. What I would like to be able to do wrappers that encode, > decode (URL, HTML, BASE64, Cryptography, ...), filter, select, > collect, parse streams in different ways. See [1] for some examples. > The wrapping could also be used to provide synchronization on shared > streams. You would like the framework to let you chain the streams, aren't you? So that you have a stream which interact with another stream. > - #upTo: is cool. For most purposes this is however too basic. I need > many different efficient and reliable variations of that. It should be > possible to read up to ... > - the first sub-collection of elements > - the first place a block predicate is satisfied > - the first place a regular expression matches > - the first place a of a set of the above conditions matches It would not make sense to implement all of this. But what about a #upToSatisfying: which evaluates the block for each element and stops when it returns true? -- Damien Cassou |
Re: [ANN] Nile: a trait-based stream library
|
|
> > - Right now the only available stream classes work on in-memory
> > collections. I would like to be able to use them on Files and Sockets > > as well. > > Implementing streams on Files and Sockets is a bit difficult for me > because I haven't any knowledge about them. I know that these are difficult. That's also why those streams are currently duplicating all functionality. However with your traits that should be simple to connect the few primitive methods ;-) > > - The current Squeak Stream implementation lacks the possibility (and > > it is difficult to do so in a clean way) to wrap streams with other > > streams. What I would like to be able to do wrappers that encode, > > decode (URL, HTML, BASE64, Cryptography, ...), filter, select, > > collect, parse streams in different ways. See [1] for some examples. > > The wrapping could also be used to provide synchronization on shared > > streams. > > You would like the framework to let you chain the streams, aren't you? > So that you have a stream which interact with another stream. Yes, pipes of streams. s := GzipStream on: (Base64Stream on: (WriteStream on: String new)) s nextPutAll: 'foobar' s contents would return a string of 'foobar' zipped and converted to BASE64. > > - #upTo: is cool. For most purposes this is however too basic. I need > > many different efficient and reliable variations of that. It should be > > possible to read up to ... > > - the first sub-collection of elements > > - the first place a block predicate is satisfied > > - the first place a regular expression matches > > - the first place a of a set of the above conditions matches > > It would not make sense to implement all of this. But what about a > #upToSatisfying: which evaluates the block for each element and stops > when it returns true? No that is not enough, because the context within the stream is missing. What we probably need is another object that you can configure for the search and that does something sophisticated to do an efficient search on the underlying collection. Cheers, Lukas -- Lukas Renggli http://www.lukas-renggli.ch |
Re: [ANN] Nile: a trait-based stream library
|
|
In reply to this post by Lukas Renggli
Lukas Renggli wrote:
> - The current Squeak Stream implementation lacks the possibility (and > it is difficult to do so in a clean way) to wrap streams with other > streams. Actually, some streams in Squeak (notably the zip streams) do have that ability. Try this for example: file := FileStream newFileNamed: 'compressed.gz'. encoder := GZipWriteStream on: file. encoder nextPutAll: 'Hello World'. encoder close. "also closes file" file := FileStream readOnlyFileNamed: 'compressed.gz'. decoder := GZipReadStream on: file. Transcript cr; show: decoder contents. decoder close. "also closes file" As for whether it wouldn't be possible to generalize this cleanly in an EncoderStream and a DecoderStream (or a generalized TranscoderStream if you like) I'll leave that to the reader. Just keep in mind that if all you can think of is using a hammer everything starts looking a *lot* like a nail... Cheers, - Andreas |
|
|
In reply to this post by Lukas Renggli
>From: "Lukas Renggli" <[hidden email]>
>Reply-To: The general-purpose Squeak developers >list<[hidden email]> >To: "The general-purpose Squeak developers >list"<[hidden email]> >Subject: Re: [ANN] Nile: a trait-based stream library >Date: Wed, 2 May 2007 15:42:43 +0200 > >I had a look at your implementation and it really looks nice. I have >some feature requests ;-) > >- The current Squeak Stream implementation lacks the possibility (and >it is difficult to do so in a clean way) to wrap streams with other >streams. What I would like to be able to do wrappers that encode, >decode (URL, HTML, BASE64, Cryptography, ...), filter, select, >collect, parse streams in different ways. See [1] for some examples. >The wrapping could also be used to provide synchronization on shared >streams. > >[1] http://wiki.cs.uiuc.edu/PatternStories/FunWithStreams The things mentioned in "FunWithStreams" look a lot like my LazyList implementation. For example, it talks about collect and select having issues because the stream may be infinite. And the suggested solution is to return a new kind of stream that "decorates" the previous stream. This is just how the LazyList implementation works. It is just that instead of "SelectCollection", it always returns a new LazyList option that decorates the previous one. So you might have a look at the project to get some ideas. :) _________________________________________________________________ MSN is giving away a trip to Vegas to see Elton John. Enter to win today. http://msnconcertcontest.com?icid-nceltontagline |
|
|
In reply to this post by Lukas Renggli
>From: "Lukas Renggli" <[hidden email]>
>Reply-To: The general-purpose Squeak developers >list<[hidden email]> >To: "The general-purpose Squeak developers >list"<[hidden email]> >Subject: Re: [ANN] Nile: a trait-based stream library >Date: Wed, 2 May 2007 17:51:56 +0200 > >No that is not enough, because the context within the stream is >missing. What we probably need is another object that you can >configure for the search and that does something sophisticated to do >an efficient search on the underlying collection. You might get some help here: http://www.cse.unsw.edu.au/~dons/papers/fusion.pdf http://www.cse.unsw.edu.au/~dons/papers/CLS07.html It is an extremely fast string implementation in Haskell (beats non-hand-optimized C in at least some cases). Maybe something in there would give you some ideas. :) _________________________________________________________________ Get a FREE Web site, company branded e-mail and more from Microsoft Office Live! http://clk.atdmt.com/MRT/go/mcrssaub0050001411mrt/direct/01/ |
|
|
In reply to this post by Lukas Renggli
> Yes, pipes of streams.
> > s := GzipStream on: (Base64Stream on: (WriteStream on: String new)) > s nextPutAll: 'foobar' > > s contents would return a string of 'foobar' zipped and converted to > BASE64. This would be great! Rio uses this form for gzip support and it makes sense to add further adaptors like this. In Rio I made a stream to stream copyTo: function, so you can compress, by copying from a readStream to a writeStream configured as above. Do you have a stream to stream copy function? If not I added one to Stream, you could steal it from Rio. I do not know if you are aware that Flow is now available in 3.9, and since this is a streams based implementation it may make more sense to use Flows primitives for file/socket support. best regards Keith |
Re: [ANN] Nile: a trait-based stream library
|
|
In reply to this post by Lukas Renggli
> > You would like the framework to let you chain the streams, aren't you?
> > So that you have a stream which interact with another stream. > > Yes, pipes of streams. I've just created a decoder in a trait and it is highly pluggable :-) Let's see an example: testCombineWithNumberReader | stream | stream := NSSelectStream selectBlock: [:each | each even] inputStream: (NSNumberReader inputStream: (NSReadableCollectionStream on: '1420 245 211 12')). self deny: stream atEnd. self assert: stream peek = 1420. self assert: stream next = 1420. self deny: stream atEnd. self assert: stream peek = 12. self assert: stream next = 12. self assert: stream atEnd This code uses three stream: - NSSelectStream reads its input and only returns elements matching the selectBlock (http://wiki.cs.uiuc.edu/PatternStories/SelectStream) - NSNumberReader is waiting for characters on its input and converts them to numbers. Spaces separate numbers. - NSReadableCollectionStream is equivalent to ReadStream. The implementation part is really cool. NSNumberReader is made of: - a constructor - 3 getters and 3 setters - a #realNext method: NSNumberReader>>realNext |number| [self inputStream peekFor: Character space] whileTrue. self inputStream atEnd ifTrue: [NSStreamAtEndError signal]. number := (self inputStream upTo: Character space) asNumber. ^ number That's all. Nothing more is needed to create NSNumberReader. The other methods come from a trait. NSSelectBlock is implemented in exactly the same way. It's #realNext is: NSSelectBlock>>realNext self inputStream do: [:each | (selectBlock value: each) ifTrue: [^ each]]. NSStreamAtEndError signal. What do you think? -- Damien Cassou |
Re: [ANN] Nile: a trait-based stream library
|
|
In reply to this post by keith1y
On 2 mai 07, at 21:50, Keith Hodges wrote: >> Yes, pipes of streams. >> >> s := GzipStream on: (Base64Stream on: (WriteStream on: String new)) >> s nextPutAll: 'foobar' >> >> s contents would return a string of 'foobar' zipped and converted >> to BASE64. > This would be great! > > Rio uses this form for gzip support and it makes sense to add > further adaptors like this. In Rio I made a stream to stream > copyTo: function, so you can compress, by copying from a readStream > to a writeStream configured as above. > > Do you have a stream to stream copy function? If not I added one to > Stream, you could steal it from Rio. > > I do not know if you are aware that Flow is now available in 3.9, > and since this is a streams based implementation it may make more > sense to use Flows primitives for file/socket support. We tried Flow in 3.2 but it was not clear what we could reuse from it. So you are suggesting the primitives... good. Do you know if we can load it without destroying the current streams? Who ported it to 3.9? Is craig working on it again? Stef |
re: Nile: a trait-based stream library
|
|
Hi Stef-- > Who ported it to 3.9? Is craig working on it again? I did, to make the Naiad client and Spoon VM simulator support work in 3.9. I announced it here a few weeks ago. http://netjam.org/spoon/releases/current thanks, -C -- Craig Latta improvisational musical informaticist www.netjam.org Smalltalkers do: [:it | All with: Class, (And love: it)] |
Re: [ANN] Nile: a trait-based stream library
|
|
In reply to this post by Damien Cassou-3
I always wonder the cost in terms of speed of the wrapping of streams.
Stef On 3 mai 07, at 14:18, Damien Cassou wrote: >> > You would like the framework to let you chain the streams, >> aren't you? >> > So that you have a stream which interact with another stream. >> >> Yes, pipes of streams. > > I've just created a decoder in a trait and it is highly pluggable :-) > Let's see an example: > > testCombineWithNumberReader > | stream | > stream := NSSelectStream > selectBlock: [:each | each even] > inputStream: (NSNumberReader > inputStream: > (NSReadableCollectionStream on: '1420 245 211 12')). > self deny: stream atEnd. > self assert: stream peek = 1420. > self assert: stream next = 1420. > self deny: stream atEnd. > self assert: stream peek = 12. > self assert: stream next = 12. > self assert: stream atEnd > > > This code uses three stream: > - NSSelectStream reads its input and only returns elements matching > the selectBlock (http://wiki.cs.uiuc.edu/PatternStories/SelectStream) > - NSNumberReader is waiting for characters on its input and converts > them to numbers. Spaces separate numbers. > - NSReadableCollectionStream is equivalent to ReadStream. > > The implementation part is really cool. NSNumberReader is made of: > > - a constructor > - 3 getters and 3 setters > - a #realNext method: > NSNumberReader>>realNext > |number| > [self inputStream peekFor: Character space] whileTrue. > self inputStream atEnd ifTrue: [NSStreamAtEndError signal]. > number := (self inputStream upTo: Character space) asNumber. > ^ number > > That's all. Nothing more is needed to create NSNumberReader. The other > methods come from a trait. > > NSSelectBlock is implemented in exactly the same way. It's > #realNext is: > > NSSelectBlock>>realNext > self inputStream do: [:each | (selectBlock value: each) ifTrue: [^ > each]]. > NSStreamAtEndError signal. > > What do you think? > > -- > Damien Cassou > > |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |