I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code? Bernhard |
Hi Bernhard. The deprecation is still in progress. Now to setup encoding for MultiByteFileStream you should set #converter: (FileLocator imageDirectory / 'test.txt') asFileReference readStreamDo: [ :s | s converter: UTF8TextConverter new. s contents ] 2017-12-30 13:27 GMT+01:00 Bernhard Pieber <[hidden email]>: I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. |
In reply to this post by bpi
Hi Bernhard
Have a look at how guille started to implement File package. (Guille is actively converting all the image users of the old hierarchy but last time he losts a couple of hours because changing epicea and other users of positionable streams and sourceArray... at the same time was leading to complex situations :). But Guille told me that he learned and will come back to this.) He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream. writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag | extension fileName outputStream | extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. fileName := baseName, extension. fileName := FileSystem disk checkName: fileName fixErrors: true. outputStream := (File named: fileName) writeStream. (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. outputStream close. writeSourceCodeFrom: aStream toFileReference: aFileReference aFileReference writeStreamDo: [ :outputStream | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. ]. self inform: 'Filed out to: ', String cr, aFileReference basename self inform: 'Filed out to: ', String cr, fileName. I hope it helps. Stef On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <[hidden email]> wrote: > I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. > > Can someone please point me to the right code? > > Bernhard |
In reply to this post by bpi
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/Zinc-Encoding-Meta/Zinc-Encoding-Meta.html
1.4. Converting Streams > On 30 Dec 2017, at 13:27, Bernhard Pieber <[hidden email]> wrote: > > I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. > > Can someone please point me to the right code? > > Bernhard |
Which reminds me that we should publish these HTML in another location....
Just too many things to do :( But at least we get pdf and HTML and more if needed for our books. On Sat, Dec 30, 2017 at 6:03 PM, Sven Van Caekenberghe <[hidden email]> wrote: > https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/Zinc-Encoding-Meta/Zinc-Encoding-Meta.html > > 1.4. Converting Streams > >> On 30 Dec 2017, at 13:27, Bernhard Pieber <[hidden email]> wrote: >> >> I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. >> >> Can someone please point me to the right code? >> >> Bernhard > > |
In reply to this post by Denis Kudriashov
Denis Kudriashov <[hidden email]>
wrote: > Hi Bernhard. > > The deprecation is still in progress. Considering the amount of non-image code using this, I would be interested to hear how this is going to be supported and documented. Stephan |
In reply to this post by Denis Kudriashov
Hi Denis,
Thank you for your answer. However, setting the UTF8TextConverter did not help because of a bug in upToAll:. I created a pull request to fix it: https://github.com/pharo-project/pharo/pull/632 I am afraid the CI failed for some reason I don't understand. :-/ The UTF8TextConverter was set by default, by the way. Bernhard > Am 30.12.2017 um 13:57 schrieb Denis Kudriashov <[hidden email]>: > > Hi Bernhard. > > The deprecation is still in progress. > Now to setup encoding for MultiByteFileStream you should set #converter: > > (FileLocator imageDirectory / 'test.txt') asFileReference readStreamDo: [ :s | > s converter: UTF8TextConverter new. > s contents > ] > > 2017-12-30 13:27 GMT+01:00 Bernhard Pieber <[hidden email]>: > I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. > > Can someone please point me to the right code? > > Bernhard |
In reply to this post by Stephane Ducasse-3
Hi Stef,
Thank you for your answer pointing me to the ZnCharacter...Stream. This seems to be meant as the replacement I was looking for. I will have a closer look. Bernhard > Am 30.12.2017 um 17:35 schrieb Stephane Ducasse <[hidden email]>: > > Hi Bernhard > > Have a look at how guille started to implement File package. > (Guille is actively converting all the image users of the old > hierarchy but last time he losts a couple of hours > because changing epicea and other users of positionable streams and > sourceArray... at the same time > was leading to complex situations :). But Guille told me that he > learned and will come back to this.) > > He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream. > > writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag > > | extension fileName outputStream | > extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. > fileName := baseName, extension. > fileName := FileSystem disk checkName: fileName fixErrors: true. > outputStream := (File named: fileName) writeStream. > (ZnCrPortableWriteStream on: (ZnCharacterWriteStream > on: outputStream > encoding: 'utf8')) nextPutAll: aStream contents. > > outputStream close. > > writeSourceCodeFrom: aStream toFileReference: aFileReference > > aFileReference writeStreamDo: [ :outputStream | > (ZnCrPortableWriteStream on: (ZnCharacterWriteStream > on: outputStream > encoding: 'utf8')) nextPutAll: aStream contents. > ]. > > self inform: 'Filed out to: ', String cr, aFileReference basename > > > self inform: 'Filed out to: ', String cr, fileName. > > I hope it helps. > > Stef > > On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <[hidden email]> wrote: >> I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. >> >> Can someone please point me to the right code? >> >> Bernhard > |
In reply to this post by Sven Van Caekenberghe-2
Hi Sven,
Thanks for the answer. This is exactly what I was looking for. And it is great that there is also documentation. However, ZnCharacterReadStream does not implement #upToAll:. The implementation from PositionableStream cannot easily be borrowed because there seems to be no way to reset the position. Is there a plan to support the most commonly used messages from the normal streams? Bernhard > Am 30.12.2017 um 18:03 schrieb Sven Van Caekenberghe <[hidden email]>: > > https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/Zinc-Encoding-Meta/Zinc-Encoding-Meta.html > > 1.4. Converting Streams > >> On 30 Dec 2017, at 13:27, Bernhard Pieber <[hidden email]> wrote: >> >> I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. >> >> Can someone please point me to the right code? >> >> Bernhard |
> On 31 Dec 2017, at 16:21, Bernhard Pieber <[hidden email]> wrote: > > Hi Sven, > > Thanks for the answer. This is exactly what I was looking for. And it is great that there is also documentation. > > However, ZnCharacterReadStream does not implement #upToAll:. The implementation from PositionableStream cannot easily be borrowed because there seems to be no way to reset the position. Is there a plan to support the most commonly used messages from the normal streams? The Stream API is much too wide, it has to be reduced and split in a number of elementary streams that can be composed to add desired behaviour (like character encoding/decoding, buffering, line end translation, ...). Positioning in a stream (moving backwards without restrictions) is not generally possible (think of a network stream), that is why it is not implemented. > Bernhard > >> Am 30.12.2017 um 18:03 schrieb Sven Van Caekenberghe <[hidden email]>: >> >> https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/Zinc-Encoding-Meta/Zinc-Encoding-Meta.html >> >> 1.4. Converting Streams >> >>> On 30 Dec 2017, at 13:27, Bernhard Pieber <[hidden email]> wrote: >>> >>> I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. >>> >>> Can someone please point me to the right code? >>> >>> Bernhard > > |
IIUC that means that a stream implementation compatible with the standard streams cannot be implemented on top of the new streams, right?
What is the plan about #upToAll:, #match: and similar messages which are very useful for parsing? Where should they go? Bernhard > Am 31.12.2017 um 17:07 schrieb Sven Van Caekenberghe <[hidden email]>: > >> On 31 Dec 2017, at 16:21, Bernhard Pieber <[hidden email]> wrote: >> >> Hi Sven, >> >> Thanks for the answer. This is exactly what I was looking for. And it is great that there is also documentation. >> >> However, ZnCharacterReadStream does not implement #upToAll:. The implementation from PositionableStream cannot easily be borrowed because there seems to be no way to reset the position. Is there a plan to support the most commonly used messages from the normal streams? > > The Stream API is much too wide, it has to be reduced and split in a number of elementary streams that can be composed to add desired behaviour (like character encoding/decoding, buffering, line end translation, ...). Positioning in a stream (moving backwards without restrictions) is not generally possible (think of a network stream), that is why it is not implemented. > >> Bernhard >> >>> Am 30.12.2017 um 18:03 schrieb Sven Van Caekenberghe <[hidden email]>: >>> >>> https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/Zinc-Encoding-Meta/Zinc-Encoding-Meta.html >>> >>> 1.4. Converting Streams >>> >>>> On 30 Dec 2017, at 13:27, Bernhard Pieber <[hidden email]> wrote: >>>> >>>> I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. >>>> >>>> Can someone please point me to the right code? >>>> >>>> Bernhard |
In reply to this post by Stephan Eggermont-3
On Sun, Dec 31, 2017 at 12:55 PM, Stephan Eggermont <[hidden email]> wrote:
> Denis Kudriashov <[hidden email]> > wrote: >> Hi Bernhard. >> >> The deprecation is still in progress. > > Considering the amount of non-image code using this, I would be interested > to hear how this is going to be supported and documented. Step by step. :) We cannot continue to have so bad streams without taking action. So we start to make sure that Pharo itself does not use the old streams. Then it means that we will be able to have the old stream library as a separate package and people will slowly migrate. Now it would be nice to have a rewrite rule to help but it requires time. Stef |
In reply to this post by bpi
> On 31 Dec 2017, at 16:01, Bernhard Pieber <[hidden email]> wrote: > > I created a pull request to fix it: > https://github.com/pharo-project/pharo/pull/632 This implementation (which is not using #position) is acceptable. Nice tests. I am still a bit puzzled by the usefulness of this method though. |
In reply to this post by Stephane Ducasse-3
Stephane Ducasse-3 wrote
> Hi Bernhard > > Have a look at how guille started to implement File package. > (Guille is actively converting all the image users of the old > hierarchy but last time he losts a couple of hours > because changing epicea and other users of positionable streams and > sourceArray... at the same time > was leading to complex situations :). But Guille told me that he > learned and will come back to this.) > > He is using ZnCharacterWriteStream and wrapper like > ZnCrPortableWriteStream. > > writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag > > | extension fileName outputStream | > extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. > fileName := baseName, extension. > fileName := FileSystem disk checkName: fileName fixErrors: true. > outputStream := (File named: fileName) writeStream. > (ZnCrPortableWriteStream on: (ZnCharacterWriteStream > on: outputStream > encoding: 'utf8')) nextPutAll: aStream contents. > > outputStream close. > > writeSourceCodeFrom: aStream toFileReference: aFileReference > > aFileReference writeStreamDo: [ :outputStream | > (ZnCrPortableWriteStream on: (ZnCharacterWriteStream > on: outputStream > encoding: 'utf8')) nextPutAll: aStream contents. > ]. > > self inform: 'Filed out to: ', String cr, aFileReference basename > > > self inform: 'Filed out to: ', String cr, fileName. > > I hope it helps. > > Stef > > On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber < > bernhard@ > > wrote: >> I just saw that the FileStream hierarchy including MultiByteFileStream is >> deprecated. However, I can't find what file stream class should be used >> to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also >> uses MultiByteFileStream. >> >> Can someone please point me to the right code? >> >> Bernhard Maybe it's just me, but I'd a big fan of convenience methods over explicit stacking when it comes to multi-layer streams. - It documents the intended/valid ways to stack (such as which streams it makes sense to wrap in a ZnCrPortableWriteStream, or which streams one normally would want to #buffered:do:) - It makes the transition (in terms of conciseness) from "traditional" filestreams less jarring (provided convenience methods provide sensible defaults). Say, for instance, a #withEncoding:do:* on read/writeStream, so instead of the above, one can do: fileName := FileSystem disk checkName: fileName fixErrors: true. (outputStream :=(File named: fileName) writeStream) withEncoding: 'utf8' do: [:encodedWriteStream | encodedWriteStream nextPutAll: aStream contents]. outputStream close. and aFileReference writeStreamDo: [ :outputStream | outputStream withEncoding: 'utf8' do: [:encodedStream | encodedStream nextPutAll: aStream contents. ]]. or input := aFileReference readStreamDo: [ :inputStream | inputStream withEncoding: 'utf8' do: [:decodingStream | decodingStream contents]] It's more involved than the current multibytestreams, but takes away the burden of determining the correct stack of ZnStreams for the common case. For instance, you'd normally want file write streams to also be wrapped by a buffered stream, and have a single flush at end of withEncoding:do:, while the same isn't necessary for withEncoding:do: on the base WriteStream class. Cheers, Henry *Bad name, I know, but you get the point. -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html |
Hi henry
I had problems to compare the two versions because they are probably on multiple mails. Could you email it? Because convenient methods are important but I do not see the point. Stef (in my constant quest for the better) On Tue, Jan 2, 2018 at 1:13 PM, Henrik Sperre Johansen <[hidden email]> wrote: > Stephane Ducasse-3 wrote >> Hi Bernhard >> >> Have a look at how guille started to implement File package. >> (Guille is actively converting all the image users of the old >> hierarchy but last time he losts a couple of hours >> because changing epicea and other users of positionable streams and >> sourceArray... at the same time >> was leading to complex situations :). But Guille told me that he >> learned and will come back to this.) >> >> He is using ZnCharacterWriteStream and wrapper like >> ZnCrPortableWriteStream. >> >> writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag >> >> | extension fileName outputStream | >> extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. >> fileName := baseName, extension. >> fileName := FileSystem disk checkName: fileName fixErrors: true. >> outputStream := (File named: fileName) writeStream. >> (ZnCrPortableWriteStream on: (ZnCharacterWriteStream >> on: outputStream >> encoding: 'utf8')) nextPutAll: aStream contents. >> >> outputStream close. >> >> writeSourceCodeFrom: aStream toFileReference: aFileReference >> >> aFileReference writeStreamDo: [ :outputStream | >> (ZnCrPortableWriteStream on: (ZnCharacterWriteStream >> on: outputStream >> encoding: 'utf8')) nextPutAll: aStream contents. >> ]. >> >> self inform: 'Filed out to: ', String cr, aFileReference basename >> >> >> self inform: 'Filed out to: ', String cr, fileName. >> >> I hope it helps. >> >> Stef >> >> On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber < > >> bernhard@ > >> > wrote: >>> I just saw that the FileStream hierarchy including MultiByteFileStream is >>> deprecated. However, I can't find what file stream class should be used >>> to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also >>> uses MultiByteFileStream. >>> >>> Can someone please point me to the right code? >>> >>> Bernhard > > Maybe it's just me, but I'd a big fan of convenience methods over explicit > stacking when it comes to multi-layer streams. > - It documents the intended/valid ways to stack (such as which streams it > makes sense to wrap in a ZnCrPortableWriteStream, or which streams one > normally would want to #buffered:do:) > - It makes the transition (in terms of conciseness) from "traditional" > filestreams less jarring (provided convenience methods provide sensible > defaults). > > Say, for instance, a #withEncoding:do:* on read/writeStream, so instead of > the above, one can do: > > fileName := FileSystem disk checkName: fileName fixErrors: true. > (outputStream :=(File named: fileName) writeStream) > withEncoding: 'utf8' > do: [:encodedWriteStream | > encodedWriteStream nextPutAll: aStream contents]. > outputStream close. > > and > aFileReference writeStreamDo: [ :outputStream | > outputStream withEncoding: 'utf8' do: [:encodedStream | > encodedStream nextPutAll: aStream contents. ]]. > or > input := aFileReference readStreamDo: [ :inputStream | > inputStream withEncoding: 'utf8' do: [:decodingStream | > decodingStream contents]] > > It's more involved than the current multibytestreams, but takes away the > burden of determining the correct stack of ZnStreams for the common case. > For instance, you'd normally want file write streams to also be wrapped by a > buffered stream, and have a single flush at end of withEncoding:do:, while > the same isn't necessary for withEncoding:do: on the base WriteStream class. > > Cheers, > Henry > > *Bad name, I know, but you get the point. > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html > |
In reply to this post by Henrik Sperre Johansen
On Tue, Jan 2, 2018 at 1:13 PM, Henrik Sperre Johansen <[hidden email]> wrote: Stephane Ducasse-3 wrote It's not just you :). I was during last sprint (while seeing such idioms appear) adding convenience methods like writeStreamEncoded: anEncoding writeStreamDo: aBlock encoded: anEncoding readStreamEncoded: anEncoding readStreamDo: aBlock encoded: anEncoding But my image fantastically crashed, the save image failed and I lost everything...
|
Free forum by Nabble | Edit this page |