Smalltalk › Pharo › Pharo Smalltalk Developers

lineConversion for WriteStream

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

17 messages Options

Peter Uhnak

lineConversion for WriteStream

Is there a reason why only MultiByteFileStream provides lineEndConvention: and not WriteStream?

The problem is that if I use MemoryStore in tests it breaks, because DiskStore and MemoryStore return different write streams.

'/tmp/test.txt' asFileReference writeStream ==> "MultiByteFileStream: '/tmp/test.txt'".

(FileSystem memory / 'test.txt') writeStream ==> "a WriteStream"

Can we either:

a) have `lineEndConvention:` directly in the WriteStream, or

b) have MemoryStore return MultiByteFileStream, or

c) both?

Thanks,

Peter

Sven Van Caekenberghe-2

Re: lineConversion for WriteStream

MultiByteFileStream is terrible (too complex) because it does too much (at once). Let's keep WriteStream simple (and even simplify it). You do not want characters that change magically.

That being said, line end conventions can be a pain. In general I would say that you should write either something platform specific (you can fetch the EOL terminator somewhere), or you write something specific (say CRLF for web stuff), while during reading you should accept anything (that is easy to do). You know: be strict in what you produce, liberal in what you accept.

> On 26 May 2016, at 11:39, Peter Uhnák <[hidden email]> wrote:
>
> Is there a reason why only MultiByteFileStream provides lineEndConvention: and not WriteStream?
>
> The problem is that if I use MemoryStore in tests it breaks, because DiskStore and MemoryStore return different write streams.
>
> '/tmp/test.txt' asFileReference writeStream ==> "MultiByteFileStream: '/tmp/test.txt'".
> (FileSystem memory / 'test.txt') writeStream ==> "a WriteStream"
>
> Can we either:
>
> a) have `lineEndConvention:` directly in the WriteStream, or
> b) have MemoryStore return MultiByteFileStream, or
> c) both?
>
> Thanks,
> Peter

Peter Uhnak

Re: lineConversion for WriteStream

> In general I would say that you should write either something platform specific or you write something specific

Except that I cannot do that because the system doesn't support neither.

And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.

On Thu, May 26, 2016 at 12:37 PM, Sven Van Caekenberghe <[hidden email]> wrote:

MultiByteFileStream is terrible (too complex) because it does too much (at once). Let's keep WriteStream simple (and even simplify it). You do not want characters that change magically.

That being said, line end conventions can be a pain. In general I would say that you should write either something platform specific (you can fetch the EOL terminator somewhere), or you write something specific (say CRLF for web stuff), while during reading you should accept anything (that is easy to do). You know: be strict in what you produce, liberal in what you accept.

> On 26 May 2016, at 11:39, Peter Uhnák <[hidden email]> wrote:
>
> Is there a reason why only MultiByteFileStream provides lineEndConvention: and not WriteStream?
>
> The problem is that if I use MemoryStore in tests it breaks, because DiskStore and MemoryStore return different write streams.
>
> '/tmp/test.txt' asFileReference writeStream ==> "MultiByteFileStream: '/tmp/test.txt'".
> (FileSystem memory / 'test.txt') writeStream ==> "a WriteStream"
>
> Can we either:
>
> a) have `lineEndConvention:` directly in the WriteStream, or
> b) have MemoryStore return MultiByteFileStream, or
> c) both?
>
> Thanks,
> Peter

Sven Van Caekenberghe-2

Re: lineConversion for WriteStream

> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
>
> > In general I would say that you should write either something platform specific or you write something specific
>
> Except that I cannot do that because the system doesn't support neither.
> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.

Yes, that CR is from days long gone ;-)

But you can certainly do whatever you want, you have to use the right streams. Make sure you start with a binary stream, wrap it to do character encoding/decoding, use #readLine, or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).

> On Thu, May 26, 2016 at 12:37 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> MultiByteFileStream is terrible (too complex) because it does too much (at once). Let's keep WriteStream simple (and even simplify it). You do not want characters that change magically.
>
> That being said, line end conventions can be a pain. In general I would say that you should write either something platform specific (you can fetch the EOL terminator somewhere), or you write something specific (say CRLF for web stuff), while during reading you should accept anything (that is easy to do). You know: be strict in what you produce, liberal in what you accept.
>
> > On 26 May 2016, at 11:39, Peter Uhnák <[hidden email]> wrote:
> >
> > Is there a reason why only MultiByteFileStream provides lineEndConvention: and not WriteStream?
> >
> > The problem is that if I use MemoryStore in tests it breaks, because DiskStore and MemoryStore return different write streams.
> >
> > '/tmp/test.txt' asFileReference writeStream ==> "MultiByteFileStream: '/tmp/test.txt'".
> > (FileSystem memory / 'test.txt') writeStream ==> "a WriteStream"
> >
> > Can we either:
> >
> > a) have `lineEndConvention:` directly in the WriteStream, or
> > b) have MemoryStore return MultiByteFileStream, or
> > c) both?
> >
> > Thanks,
> > Peter
>
>
>

Peter Uhnak

Re: lineConversion for WriteStream

On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
>
> > In general I would say that you should write either something platform specific or you write something specific
>
> Except that I cannot do that because the system doesn't support neither.
> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.

Yes, that CR is from days long gone ;-)

> or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).

I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.

I know I can use #lf or whatnot, but I am not creating the content, I am saving it.

Peter

Sven Van Caekenberghe-2

Re: lineConversion for WriteStream

> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
>
>
>
> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
> >
> > > In general I would say that you should write either something platform specific or you write something specific
> >
> > Except that I cannot do that because the system doesn't support neither.
> > And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>
> Yes, that CR is from days long gone ;-)
>
> > or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>
> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>
> Peter

Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?

Peter Uhnak

Re: lineConversion for WriteStream

Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.

Anyway; this is not system-breaking problem, just annoying.

Peter

On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
>
>
>
> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
> >
> > > In general I would say that you should write either something platform specific or you write something specific
> >
> > Except that I cannot do that because the system doesn't support neither.
> > And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>
> Yes, that CR is from days long gone ;-)
>
> > or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>
> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>
> Peter

Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?

Sven Van Caekenberghe-2

Re: lineConversion for WriteStream

> On 26 May 2016, at 20:20, Peter Uhnák <[hidden email]> wrote:
>
> Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.

Well, STONWriter, NeoJSONWriter and NeoCSVWriter allow you to set the line end convention, you are not alone in wanting the standard unix line end.

> Anyway; this is not system-breaking problem, just annoying.
>
> Peter
>
> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
> >
> >
> >
> > On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> >
> > > On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
> > >
> > > > In general I would say that you should write either something platform specific or you write something specific
> > >
> > > Except that I cannot do that because the system doesn't support neither.
> > > And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
> >
> > Yes, that CR is from days long gone ;-)
> >
> > > or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
> >
> > I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
> > I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
> >
> > Peter
>
> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>
>
>

Peter Uhnak

Re: lineConversion for WriteStream

Ah, thanks, I'll take a look at that.

I was using STON toStringPretty:

On Thu, May 26, 2016 at 8:30 PM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 26 May 2016, at 20:20, Peter Uhnák <[hidden email]> wrote:
>
> Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.

Well, STONWriter, NeoJSONWriter and NeoCSVWriter allow you to set the line end convention, you are not alone in wanting the standard unix line end.

> Anyway; this is not system-breaking problem, just annoying.
>
> Peter
>
> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
> >
> >
> >
> > On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> >
> > > On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
> > >
> > > > In general I would say that you should write either something platform specific or you write something specific
> > >
> > > Except that I cannot do that because the system doesn't support neither.
> > > And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
> >
> > Yes, that CR is from days long gone ;-)
> >
> > > or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
> >
> > I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
> > I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
> >
> > Peter
>
> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>
>
>

Ben Coman

Re: lineConversion for WriteStream

In reply to this post by Sven Van Caekenberghe-2

On Fri, May 27, 2016 at 2:30 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>
>> On 26 May 2016, at 20:20, Peter Uhnák <[hidden email]> wrote:
>>
>> Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.
>
> Well, STONWriter, NeoJSONWriter and NeoCSVWriter allow you to set the line end convention, you are not alone in wanting the standard unix line end.

So why don't we do it?

CR does seem a major legacy [1] ... Early Mac OS up to version 9;
Commodore 8-bit machines, Acorn BBC, ZX Spectrum, TRS-80, Apple II
family, Oberon, MIT Lisp Machine and OS-9

I read [2] its a "non-issue with windows 7. Just use unix-newlines.
The only application (of the few tested) I have found which does not
understand unix-newlines as newlines is the useless notepad. For
instance it seems the following applications understand unix-newlines
just fine in windows 7: cmd scripts; powershell scripts; word 2013 (I
can open a txt file with unix-newlines, though I never use that, I can
also paste text with unix-newlines and get correct/desired line
breaking) ; OneNote 2013 (pasting text) ; wordpad (not that I use it)
; Sublime Text 3 (naturally, just on the list because it the best;
Eclipse."

And now Microsoft is taking small steps towards a Linux desktop ;) [3] [4]

So it looks to me like LF wins!
It may take a while and some effort, but can we agree on a three point strategy?
1. Default CR line ending must change!
2. Change it to LF.
3. Default auto-conversion of "text" input to LF?

[1] https://en.wikipedia.org/wiki/Newline#History
[2] https://blog.codinghorror.com/the-great-newline-schism/
[3] http://www.zdnet.com/article/microsoft-and-canonical-partner-to-bring-ubuntu-to-windows-10/
[4] https://insights.ubuntu.com/2016/03/30/ubuntu-on-windows-the-ubuntu-userspace-for-windows-developers/

cheers -ben

>
>> Anyway; this is not system-breaking problem, just annoying.
>>
>> Peter
>>
>> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>
>> > On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
>> >
>> >
>> >
>> > On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>> >
>> > > On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
>> > >
>> > > > In general I would say that you should write either something platform specific or you write something specific
>> > >
>> > > Except that I cannot do that because the system doesn't support neither.
>> > > And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>> >
>> > Yes, that CR is from days long gone ;-)
>> >
>> > > or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>> >
>> > I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
>> > I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>> >
>> > Peter
>>
>> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>>
>>
>>
>
>

Sven Van Caekenberghe-2

Re: lineConversion for WriteStream

> On 27 May 2016, at 05:07, Ben Coman <[hidden email]> wrote:
>
> On Fri, May 27, 2016 at 2:30 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>>
>>> On 26 May 2016, at 20:20, Peter Uhnák <[hidden email]> wrote:
>>>
>>> Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.
>>
>> Well, STONWriter, NeoJSONWriter and NeoCSVWriter allow you to set the line end convention, you are not alone in wanting the standard unix line end.
>
> So why don't we do it?
>
> CR does seem a major legacy [1] ... Early Mac OS up to version 9;
> Commodore 8-bit machines, Acorn BBC, ZX Spectrum, TRS-80, Apple II
> family, Oberon, MIT Lisp Machine and OS-9
>
> I read [2] its a "non-issue with windows 7. Just use unix-newlines.
> The only application (of the few tested) I have found which does not
> understand unix-newlines as newlines is the useless notepad. For
> instance it seems the following applications understand unix-newlines
> just fine in windows 7: cmd scripts; powershell scripts; word 2013 (I
> can open a txt file with unix-newlines, though I never use that, I can
> also paste text with unix-newlines and get correct/desired line
> breaking) ; OneNote 2013 (pasting text) ; wordpad (not that I use it)
> ; Sublime Text 3 (naturally, just on the list because it the best;
> Eclipse."
>
> And now Microsoft is taking small steps towards a Linux desktop ;) [3] [4]
>
> So it looks to me like LF wins!
> It may take a while and some effort, but can we agree on a three point strategy?
> 1. Default CR line ending must change!
> 2. Change it to LF.
> 3. Default auto-conversion of "text" input to LF?

Yes, why not. I believe they did the same in Cuis years ago, switched the whole image, all defaults and source code from #cr to #lf and never looked back.

You made a good argument, #cr on itself is obsolete, and if Windows users can live with it, there is no longer any reason not to choose for #lf.

> [1] https://en.wikipedia.org/wiki/Newline#History
> [2] https://blog.codinghorror.com/the-great-newline-schism/
> [3] http://www.zdnet.com/article/microsoft-and-canonical-partner-to-bring-ubuntu-to-windows-10/
> [4] https://insights.ubuntu.com/2016/03/30/ubuntu-on-windows-the-ubuntu-userspace-for-windows-developers/
>
> cheers -ben
>
>>
>>> Anyway; this is not system-breaking problem, just annoying.
>>>
>>> Peter
>>>
>>> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>
>>>> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
>>>>
>>>>
>>>>
>>>> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>
>>>>> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
>>>>>
>>>>>> In general I would say that you should write either something platform specific or you write something specific
>>>>>
>>>>> Except that I cannot do that because the system doesn't support neither.
>>>>> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>>>>
>>>> Yes, that CR is from days long gone ;-)
>>>>
>>>>> or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>>>>
>>>> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
>>>> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>>>>
>>>> Peter
>>>
>>> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>>>
>>>
>>>
>>
>>
>

stepharo

Re: lineConversion for WriteStream

In reply to this post by Ben Coman

Yes we should probably migrate to LF and I would also
go a for newer stream library.

Stef

>> Well, STONWriter, NeoJSONWriter and NeoCSVWriter allow you to set the line end convention, you are not alone in wanting the standard unix line end.
> So why don't we do it?
>
> CR does seem a major legacy [1] ... Early Mac OS up to version 9;
> Commodore 8-bit machines, Acorn BBC, ZX Spectrum, TRS-80, Apple II
> family, Oberon, MIT Lisp Machine and OS-9
>
> I read [2] its a "non-issue with windows 7. Just use unix-newlines.
> The only application (of the few tested) I have found which does not
> understand unix-newlines as newlines is the useless notepad. For
> instance it seems the following applications understand unix-newlines
> just fine in windows 7: cmd scripts; powershell scripts; word 2013 (I
> can open a txt file with unix-newlines, though I never use that, I can
> also paste text with unix-newlines and get correct/desired line
> breaking) ; OneNote 2013 (pasting text) ; wordpad (not that I use it)
> ; Sublime Text 3 (naturally, just on the list because it the best;
> Eclipse."
>
> And now Microsoft is taking small steps towards a Linux desktop ;) [3] [4]
>
> So it looks to me like LF wins!
> It may take a while and some effort, but can we agree on a three point strategy?
> 1. Default CR line ending must change!
> 2. Change it to LF.
> 3. Default auto-conversion of "text" input to LF?
>
> [1] https://en.wikipedia.org/wiki/Newline#History
> [2] https://blog.codinghorror.com/the-great-newline-schism/
> [3] http://www.zdnet.com/article/microsoft-and-canonical-partner-to-bring-ubuntu-to-windows-10/
> [4] https://insights.ubuntu.com/2016/03/30/ubuntu-on-windows-the-ubuntu-userspace-for-windows-developers/
>
> cheers -ben
>
>>> Anyway; this is not system-breaking problem, just annoying.
>>>
>>> Peter
>>>
>>> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>
>>>> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
>>>>
>>>>
>>>>
>>>> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>
>>>>> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
>>>>>
>>>>>> In general I would say that you should write either something platform specific or you write something specific
>>>>> Except that I cannot do that because the system doesn't support neither.
>>>>> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>>>> Yes, that CR is from days long gone ;-)
>>>>
>>>>> or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>>>> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
>>>> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>>>>
>>>> Peter
>>> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>>>
>>>
>>>
>>
>

monty-3

Re: lineConversion for WriteStream

In reply to this post by Peter Uhnak

If you upgrade XMLParser, the DOM file printing methods like printToFileNamed: now automatically use your OS's linebreak, and XMLWriter, which always allowed setting the linebreak, now has an option to detect your OS's linebreak and use that.

Sent: Thursday, May 26, 2016 at 2:20 PM
From: "Peter Uhnák" <[hidden email]>
To: "Pharo Development List" <[hidden email]>
Subject: Re: [Pharo-dev] lineConversion for WriteStream

Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.
Anyway; this is not system-breaking problem, just annoying.

Peter

On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email][[hidden email]]> wrote:
>
>
>
> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email][[hidden email]]> wrote:
>
> > On 26 May 2016, at 13:29, Peter Uhnák <[hidden email][[hidden email]]> wrote:
> >
> > > In general I would say that you should write either something platform specific or you write something specific
> >
> > Except that I cannot do that because the system doesn't support neither.
> > And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>
> Yes, that CR is from days long gone ;-)
>
> > or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>
> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>
> Peter

Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?

stepharo

Re: lineConversion for WriteStream

In reply to this post by Peter Uhnak

Le 26/5/16 à 13:29, Peter Uhnák a écrit :

> In general I would say that you should write either something platform specific or you write something specific

Except that I cannot do that because the system doesn't support neither.

And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.

+ 1
now the strategy to switch smootly is not clear to me.

In addition I would love to kill all the stream hierarchy.

On Thu, May 26, 2016 at 12:37 PM, Sven Van Caekenberghe <[hidden email]> wrote:

MultiByteFileStream is terrible (too complex) because it does too much (at once). Let's keep WriteStream simple (and even simplify it). You do not want characters that change magically.

That being said, line end conventions can be a pain. In general I would say that you should write either something platform specific (you can fetch the EOL terminator somewhere), or you write something specific (say CRLF for web stuff), while during reading you should accept anything (that is easy to do). You know: be strict in what you produce, liberal in what you accept.

> On 26 May 2016, at 11:39, Peter Uhnák <[hidden email]> wrote:
>
> Is there a reason why only MultiByteFileStream provides lineEndConvention: and not WriteStream?
>
> The problem is that if I use MemoryStore in tests it breaks, because DiskStore and MemoryStore return different write streams.
>
> '/tmp/test.txt' asFileReference writeStream ==> "MultiByteFileStream: '/tmp/test.txt'".
> (FileSystem memory / 'test.txt') writeStream ==> "a WriteStream"
>
> Can we either:
>
> a) have `lineEndConvention:` directly in the WriteStream, or
> b) have MemoryStore return MultiByteFileStream, or
> c) both?
>
> Thanks,
> Peter

stepharo

Re: lineConversion for WriteStream

In reply to this post by Ben Coman

> So it looks to me like LF wins!
> It may take a while and some effort, but can we agree on a three point strategy?
> 1. Default CR line ending must change!
> 2. Change it to LF.
> 3. Default auto-conversion of "text" input to LF?
Would be so nice :)

>
> [1] https://en.wikipedia.org/wiki/Newline#History
> [2] https://blog.codinghorror.com/the-great-newline-schism/
> [3] http://www.zdnet.com/article/microsoft-and-canonical-partner-to-bring-ubuntu-to-windows-10/
> [4] https://insights.ubuntu.com/2016/03/30/ubuntu-on-windows-the-ubuntu-userspace-for-windows-developers/
>
> cheers -ben
>
>>> Anyway; this is not system-breaking problem, just annoying.
>>>
>>> Peter
>>>
>>> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>
>>>> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
>>>>
>>>>
>>>>
>>>> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>
>>>>> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
>>>>>
>>>>>> In general I would say that you should write either something platform specific or you write something specific
>>>>> Except that I cannot do that because the system doesn't support neither.
>>>>> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>>>> Yes, that CR is from days long gone ;-)
>>>>
>>>>> or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>>>> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
>>>> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>>>>
>>>> Peter
>>> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>>>
>>>
>>>
>>
>

stepharo

Re: lineConversion for WriteStream

In reply to this post by monty-3

Tx monty this is cool!

Le 8/7/16 à 09:44, monty a écrit :

> If you upgrade XMLParser, the DOM file printing methods like printToFileNamed: now automatically use your OS's linebreak, and XMLWriter, which always allowed setting the linebreak, now has an option to detect your OS's linebreak and use that.
>
> Sent: Thursday, May 26, 2016 at 2:20 PM
> From: "Peter Uhnák" <[hidden email]>
> To: "Pharo Development List" <[hidden email]>
> Subject: Re: [Pharo-dev] lineConversion for WriteStream
>
> Well I was saving e.g. STON or XML file… but some apps outside didn't particularly like it… even `cat` doesn't like CR.
> Anyway; this is not system-breaking problem, just annoying.
>
> Peter
>
> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
>
>> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email][[hidden email]]> wrote:
>>
>>
>>
>> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email][[hidden email]]> wrote:
>>
>>> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email][[hidden email]]> wrote:
>>>
>>>> In general I would say that you should write either something platform specific or you write something specific
>>> Except that I cannot do that because the system doesn't support neither.
>>> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
>> Yes, that CR is from days long gone ;-)
>>
>>> or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
>> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
>> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
>>
>> Peter
> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
>
>
>
>

monty-3

Re: lineConversion for WriteStream

In reply to this post by stepharo

You could silently make all #cr methods write/return LFs instead, but that would break existing code that actually expects messages like Character class>>#cr or Character class>>#lf to use the character they're named after (Smalltalk/X actually does something like this and it's caused me grief).

A safer way would be adding a new message like #lineEnding/#newLine defined everywhere #cr and #lf are and change uses of #cr as a line ending (for example, splitting on it, or sending #cr to a write stream) in the image to the new message.

> Sent: Friday, July 08, 2016 at 5:43 AM
> From: stepharo <[hidden email]>
> To: [hidden email]
> Subject: Re: [Pharo-dev] lineConversion for WriteStream
>
>
> > So it looks to me like LF wins!
> > It may take a while and some effort, but can we agree on a three point strategy?
> > 1. Default CR line ending must change!
> > 2. Change it to LF.
> > 3. Default auto-conversion of "text" input to LF?
> Would be so nice :)
>
>
> >
> > [1] https://en.wikipedia.org/wiki/Newline#History
> > [2] https://blog.codinghorror.com/the-great-newline-schism/
> > [3] http://www.zdnet.com/article/microsoft-and-canonical-partner-to-bring-ubuntu-to-windows-10/
> > [4] https://insights.ubuntu.com/2016/03/30/ubuntu-on-windows-the-ubuntu-userspace-for-windows-developers/
> >
> > cheers -ben
> >
> >>> Anyway; this is not system-breaking problem, just annoying.
> >>>
> >>> Peter
> >>>
> >>> On Thu, May 26, 2016 at 2:50 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> >>>
> >>>> On 26 May 2016, at 14:06, Peter Uhnák <[hidden email]> wrote:
> >>>>
> >>>>
> >>>>
> >>>> On Thu, May 26, 2016 at 1:40 PM, Sven Van Caekenberghe <[hidden email]> wrote:
> >>>>
> >>>>> On 26 May 2016, at 13:29, Peter Uhnák <[hidden email]> wrote:
> >>>>>
> >>>>>> In general I would say that you should write either something platform specific or you write something specific
> >>>>> Except that I cannot do that because the system doesn't support neither.
> >>>>> And the fact that the default line ending is CR is just bullshit… it's 2016, not 1986.
> >>>> Yes, that CR is from days long gone ;-)
> >>>>
> >>>>> or #cr #lf or #crlf as needed, and/or make that last one a parameter (OSPlatform current lineEnding).
> >>>> I am piping unknown content into the file, thus the need for `lineEndConvention:` and the reason of this entire thread. So as I said, the system doesn't support it.
> >>>> I know I can use #lf or whatnot, but I am not creating the content, I am saving it.
> >>>>
> >>>> Peter
> >>> Well, maybe I don't understand your use case, but if you do not know what is inside, why not save it as is, binary even, not doing any conversions ?
> >>>
> >>>
> >>>
> >>
> >
>
>
>