About cr and lf

Hi Stef,

On 5 August 2017 at 18:14, Stephane Ducasse <[hidden email]> wrote:
> So one step at a time:
>
> - #cr and #lf just put this character in the stream.
> - #newLine put the underlying OS line ending. ( and sorry for the
> nostalgic but nl sucks and to me this is netherlands and not newline
> :).

Beyond the nostalgic and humorous, there is actually a justification
for the shorter #nl.

Adding a new line is a common operation, and more compact code is more
readable.  So I find:

stream
    << 'Line 1 goes here'; nl;
    << 'Line 2 goes here'; nl.

better than:

stream
    nextPutAll: 'Line 1 goes here'; newLine;
    nextPutAll: 'Line 2 goes here'; newLine.


Although this is still subjective.

Cheers,
Alistair

> Then we should revisit all the cr inside the system and use newline.
> Then we should think about the internal usage of cr by default in
> Pharo (We should change it).
>
> Does it make sense?
> Stef
>
>
>
> On Sat, Aug 5, 2017 at 4:08 PM, K K Subbu <[hidden email]> wrote:
>> On Friday 04 August 2017 07:11 PM, Damien Pollet wrote:
>>>
>>> Stream >> newLineFor: platform
>>>      self nextPutAll: platform lineEnding
>>>
>>
>> EOL encoding is a property of a text file and not a platform. While files
>> created on a certain proprietary platforms ;-) may use an encoding that does
>> not make it a platform property. Pharo should be liberal in accepting and
>> preserving differently encoded files. E.g. Text editor vim has a setting
>> (fileformats) for eol encoding to view or edit files with cr or crlf EOL
>> encoding. If this is not set, vim falls back to nl.
>>
>> I agree that existing messages like cr or nl should not be changed. Instead
>> new methods (*eol) could be added to text files for decoding (read side) and
>> encoding (write side). This should be liberal in accepting cr, nl, crnl
>> decodes while reading a text file. While writing a new stream, eol should
>> follow this order:
>>       1. if file eol property is set (cr/nl/crnl) it should use it
>>       2. else fallback to check platform eol property.
>>       3. Else, use cr (for backward compatibility with old files)
>> E.g.
>>
>>  (aStream atEOL) ifFalse: [ c := aStream next ]
>>  aStream nextPutAll: 'hello world' ; writeEOL
>>  record := aStream readLine. "strip eol"
>>  aStream nextPutLine: 'hello world'
>>
>> This is not going to be an easy change but it would make new code cleaner.
>>
>> Regards .. Subbu
>>
>

Hi,

just to (hopefully) clarify my intention, maybe pseudocode would describe my thoughts better.

Stream>>beForWindows
        "Use consistenly Windows line endings (CRLF) in the stream"
        self convertLineEndings: true
        lineEnding := String crlf

Stream>>convertLineEndings: aBoolean
        "automatically convert line endings to the predefined one"
        convertLineEndings := aBoolean

Stream>>nl
        self nextPutAll: lineEnding

Stream>>cr
        convertLineEndings ifTrue: [
                self deprected: 'Using #cr/#lf for generic newlines is deprected, use #nl instead".
                self nl.
        ] ifFalse: [
                self nextPutAll: String cr.
        ]


So when "convertLineEndings = true", then using anything else than #nl would warn the user that they should use #nl instead (as they explicitly requested to have the line endings consistend).

And when "convertLineEndings = false", it would behave pretty much the same way as now, #cr would write #cr, etc.


With such approach imho

* output of existing code wouldn't be broken
* when switching to new scheme (#nl) the programmer would be warned where the missed a change
* it will be easier to keep the newlines consistent

Peter

On Sat, Aug 05, 2017 at 11:30:58AM +0200, Esteban Lorenzano wrote:
>

> > On 5 Aug 2017, at 11:17, Peter Uhnak <[hidden email]> wrote:
> >
> >> I think there is a consensus we need to keep #cr and #lf as intended
> >
> > Is there?
> >
> > My argument was that there's no (obvious) reason to combine different line endings in the same document. Therefore if you were to use #cr, #lf, #crlf, you would actually mean that you just want to enter newline.
>
> no, sometimes you want to enforce a specific line ending. You will not mix, but you will not use the one from platform. Also, sometimes you actually can use #cr and #lf different as their immediate, common use (I’ve seen some weird exporting formats).
>
> >
> > Similar problem arises when you would write a multiline string:
> >
> > stream nextPutAll: 'first line with enter
> > second line'.
> >
> > Stored in method this will most likely contain Pharo's internal representation (#cr), even though you just want a new line, and not specifically #cr. (I've lost count how many times tests on CI failed because of this.)
> >
> > Considering the above, my opinion is:
> >
> > 1) by default #cr, #lf, #crLf, #nl (#newLine) will write whatever is globally configured for the stream (#beFor*)
>
> No, I strongly disagree.
> #cr and #lf are ascii characters and should not be redefined.
>
> > 2) if one wanted to combine different line endings in the same stream, there should be an option to disable autoconversion. (Stream>>noNewLineAutoconversion)
> >
> > If (1) is much more common than (2), then imho autoconversion should cause no issues.
> > If (1) is NOT that much more common than (2), then autoconversion wouldn't be beneficial.
> >
> > Autoconversion could also make transition easier because existing code will suddenly work as intended here without breaking anything (hopefully).
>
> Sorry, I do not see what this approach solves than the other one does not (and as I see, this one is a lot more complicated and prone to confusion).
>
> cheers,
> Esteban
>
> >
> > Peter
> >
> >
> > On Sat, Aug 05, 2017 at 10:49:02AM +0200, Esteban Lorenzano wrote:
> >> I think there is a consensus we need to keep #cr and #lf as intended, yet to add some kind of #newLine (which btw is different to EOL :P) vocabulary, isn’t?
> >>
> >> In this, I favour Peter approach for define line ending convention (the way #newLine will work)… and of course by default it should use the one from the current platform.
> >>
> >> anything agains this approach?
> >>
> >> Esteban
> >>
> >>
> >>> On 4 Aug 2017, at 23:48, Tudor Girba <[hidden email]> wrote:
> >>>
> >>> +1.
> >>>
> >>> We need a basic representation of those characters. Logical ones should be derived from the simple ones.
> >>>
> >>> Doru
> >>>
> >>>
> >>>> On Aug 4, 2017, at 3:44 PM, Esteban Lorenzano <[hidden email]> wrote:
> >>>>
> >>>>
> >>>>> On 4 Aug 2017, at 15:41, Damien Pollet <[hidden email]> wrote:
> >>>>>
> >>>>> I agree with Pablo, #cr and #lf should not be clever and just be names for the carriage return and linefeed characters/codepoints.
> >>>>
> >>>> +1
> >>>>
> >>>>>
> >>>>> Making #newLine's behavior dependent on the current platform disturbs me, though. I'd rather have:
> >>>>>
> >>>>> Stream >> newLineFor: platform
> >>>>>   self nextPutAll: platform lineEnding
> >>>>>
> >>>>> Stream >> newLineForCurrentPlatform
> >>>>>   self newLineFor: OSPlatform current
> >>>>>
> >>>>> Stream >> newLineForWindows "convenience for the most common platforms
> >>>>> Stream >> newLineForUnix
> >>>>> Stream >> newLineForHistoricReasons
> >>>>>
> >>>>> Stream >> newLine
> >>>>>   "delegates to one of the above, I'd argue for unix for convenience, but windows is the technically correct combination of cr + lf, and cr only is the historic one"
> >>>>>
> >>>>>
> >>>>> On 4 August 2017 at 14:25, [hidden email] <[hidden email]> wrote:
> >>>>> To me it is clear that cr and lf should be in streams. But they should put the 'cr' or 'lf' character only. And of course the platform independent newline should be also.
> >>>>>
> >>>>> The first (cr, lf) should be used by the code wanting to have absolute control of what is in the stream. The later (newline) when you just want a new line.
> >>>>>
> >>>>> The two have completely different behaviour, ones are really low level, the other is higher level.
> >>>>>
> >>>>> On 4 Aug 2017 14:20, "Esteban Lorenzano" <[hidden email]> wrote:
> >>>>>
> >>>>>> On 4 Aug 2017, at 14:06, Stephane Ducasse <[hidden email]> wrote:
> >>>>>>
> >>>>>> Well. This is not implemented like that in Pharo.
> >>>>>>
> >>>>>> cr is bad because it does not mean that it is independent of the platform.
> >>>>>> So cr can be redefined as newLine and keep but not used inside the system.
> >>>>>
> >>>>> sometimes you actually want to write a cr (or a lf). So it needs to remain in the system, of course.
> >>>>> now, including #newLine can be cool (most of the times you want the “platform compatible” new line). Also I would consider including #nl, abbreviated… just for convenience :P
> >>>>>
> >>>>> Esteban
> >>>>>
> >>>>>>
> >>>>>> Stef
> >>>>>>
> >>>>>> On Fri, Aug 4, 2017 at 12:50 PM, Jan Vrany <[hidden email]> wrote:
> >>>>>>> On Fri, 2017-08-04 at 12:03 +0200, Stephane Ducasse wrote:
> >>>>>>>> Hi guys
> >>>>>>>>
> >>>>>>>> While writing pillar code, I ended up using "stream cr" and it
> >>>>>>>> worries
> >>>>>>>> me to still expand usage
> >>>>>>>> of a pattern I would like to remove.
> >>>>>>>>
> >>>>>>>> Let us imagine that we would like to prepare the migration from cr.
> >>>>>>>> I was thinking that we could replace cr invocation by newLine so that
> >>>>>>>> after newLine
> >>>>>>>> could be redefined as
> >>>>>>>>
> >>>>>>>> Stream >> newLine
> >>>>>>>>     self nextPutAll: OSPlatform current lineEnding
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> what do you think about this approach?
> >>>>>>>
> >>>>>>> Why not? But please keep #cr.
> >>>>>>>
> >>>>>>> Section 5.9.4.1 of ANSI reads:
> >>>>>>>
> >>>>>>> Message: cr
> >>>>>>>
> >>>>>>> Synopsis
> >>>>>>> Writes an end-of-line sequence to the receiver.
> >>>>>>>
> >>>>>>> Definition: <puttableStream>
> >>>>>>> A sequence of character objects that constitute the implementation-
> >>>>>>> defined end-of-line sequence is added to the receiver in the same
> >>>>>>> manner as if the message  #nextPutAll: was sent to the receiver with
> >>>>>>> an argument string whose elements are the sequence of characters.
> >>>>>>>
> >>>>>>> Return Value
> >>>>>>> UNSPECIFIED
> >>>>>>> Errors
> >>>>>>> It is erroneous if any element of the end-of-line sequence is an
> >>>>>>> object that does not conform to the receiver's sequence value type .
> >>>>>>>
> >>>>>>> my 2c,
> >>>>>>>
> >>>>>>> Jan
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Stef
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Damien Pollet
> >>>>> type less, do more [ | ] http://people.untyped.org/damien.pollet
> >>>>
> >>>
> >>> --
> >>> www.tudorgirba.com
> >>> www.feenk.com
> >>>
> >>> "Presenting is storytelling."
> >>>
> >>>
> >>
> >>
> >
>
>

On Sun, Aug 6, 2017 at 10:54 AM, Stephane Ducasse <[hidden email]> wrote:

Agreed :)

But so far what do we do?


- #cr and #lf just put this character in the stream.

yes. I would even be a bit more agressive. Streams work on any collection, not only on strings (that's why I think this is buggy!). We should move cr and lf to the character write streams. People that want to write characters should use a character write stream not a general purpose stream (on e.g., a Set).

 

- #newLine put the underlying OS line ending.

this could just exist in a decorator, used in "text writing streams". This implies files, transcript, stdio. But not sockets, collections

Then we should revisit all the cr inside the system and use newline.

yes

 

Then we should think about the internal usage of cr by default in
Pharo (We should change it).

Yes, and we should teach people (this means documenting :P) the correct usage of streams :).

Does it make sense?
Stef

Stef

On Sun, Aug 6, 2017 at 10:36 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>
>> On 6 Aug 2017, at 08:59, Guillermo Polito <[hidden email]> wrote:
>>
>> Can somebody propose an implementation besides on top of this discussion? I propose such an implementation should take the form of a stream decorator instead of changing the base implementation.
>
> YES, a decorator !
>
> We want simpler streams, not the old complex ones. Less API, less functionality.
>
>> Guille
>>
>> On Sun, Aug 6, 2017 at 12:01 AM, Peter Uhnak <[hidden email]> wrote:
>> Hi,
>>
>> just to (hopefully) clarify my intention, maybe pseudocode would describe my thoughts better.
>>
>> Stream>>beForWindows
>>         "Use consistenly Windows line endings (CRLF) in the stream"
>>         self convertLineEndings: true
>>         lineEnding := String crlf
>>
>> Stream>>convertLineEndings: aBoolean
>>         "automatically convert line endings to the predefined one"
>>         convertLineEndings := aBoolean
>>
>> Stream>>nl
>>         self nextPutAll: lineEnding
>>
>> Stream>>cr
>>         convertLineEndings ifTrue: [
>>                 self deprected: 'Using #cr/#lf for generic newlines is deprected, use #nl instead".
>>                 self nl.
>>         ] ifFalse: [
>>                 self nextPutAll: String cr.
>>         ]
>>
>>
>> So when "convertLineEndings = true", then using anything else than #nl would warn the user that they should use #nl instead (as they explicitly requested to have the line endings consistend).
>>
>> And when "convertLineEndings = false", it would behave pretty much the same way as now, #cr would write #cr, etc.
>>
>>
>> With such approach imho
>>
>> * output of existing code wouldn't be broken
>> * when switching to new scheme (#nl) the programmer would be warned where the missed a change
>> * it will be easier to keep the newlines consistent
>>
>> Peter
>>
>> On Sat, Aug 05, 2017 at 11:30:58AM +0200, Esteban Lorenzano wrote:
>> >
>> > > On 5 Aug 2017, at 11:17, Peter Uhnak <[hidden email]> wrote:
>> > >
>> > >> I think there is a consensus we need to keep #cr and #lf as intended
>> > >
>> > > Is there?
>> > >
>> > > My argument was that there's no (obvious) reason to combine different line endings in the same document. Therefore if you were to use #cr, #lf, #crlf, you would actually mean that you just want to enter newline.
>> >
>> > no, sometimes you want to enforce a specific line ending. You will not mix, but you will not use the one from platform. Also, sometimes you actually can use #cr and #lf different as their immediate, common use (I’ve seen some weird exporting formats).
>> >
>> > >
>> > > Similar problem arises when you would write a multiline string:
>> > >
>> > > stream nextPutAll: 'first line with enter
>> > > second line'.
>> > >
>> > > Stored in method this will most likely contain Pharo's internal representation (#cr), even though you just want a new line, and not specifically #cr. (I've lost count how many times tests on CI failed because of this.)
>> > >
>> > > Considering the above, my opinion is:
>> > >
>> > > 1) by default #cr, #lf, #crLf, #nl (#newLine) will write whatever is globally configured for the stream (#beFor*)
>> >
>> > No, I strongly disagree.
>> > #cr and #lf are ascii characters and should not be redefined.
>> >
>> > > 2) if one wanted to combine different line endings in the same stream, there should be an option to disable autoconversion. (Stream>>noNewLineAutoconversion)
>> > >
>> > > If (1) is much more common than (2), then imho autoconversion should cause no issues.
>> > > If (1) is NOT that much more common than (2), then autoconversion wouldn't be beneficial.
>> > >
>> > > Autoconversion could also make transition easier because existing code will suddenly work as intended here without breaking anything (hopefully).
>> >
>> > Sorry, I do not see what this approach solves than the other one does not (and as I see, this one is a lot more complicated and prone to confusion).
>> >
>> > cheers,
>> > Esteban
>> >
>> > >
>> > > Peter
>> > >
>> > >
>> > > On Sat, Aug 05, 2017 at 10:49:02AM +0200, Esteban Lorenzano wrote:
>> > >> I think there is a consensus we need to keep #cr and #lf as intended, yet to add some kind of #newLine (which btw is different to EOL :P) vocabulary, isn’t?
>> > >>
>> > >> In this, I favour Peter approach for define line ending convention (the way #newLine will work)… and of course by default it should use the one from the current platform.
>> > >>
>> > >> anything agains this approach?
>> > >>
>> > >> Esteban
>> > >>
>> > >>
>> > >>> On 4 Aug 2017, at 23:48, Tudor Girba <[hidden email]> wrote:
>> > >>>
>> > >>> +1.
>> > >>>
>> > >>> We need a basic representation of those characters. Logical ones should be derived from the simple ones.
>> > >>>
>> > >>> Doru
>> > >>>
>> > >>>
>> > >>>> On Aug 4, 2017, at 3:44 PM, Esteban Lorenzano <[hidden email]> wrote:
>> > >>>>
>> > >>>>
>> > >>>>> On 4 Aug 2017, at 15:41, Damien Pollet <[hidden email]> wrote:
>> > >>>>>
>> > >>>>> I agree with Pablo, #cr and #lf should not be clever and just be names for the carriage return and linefeed characters/codepoints.
>> > >>>>
>> > >>>> +1
>> > >>>>
>> > >>>>>
>> > >>>>> Making #newLine's behavior dependent on the current platform disturbs me, though. I'd rather have:
>> > >>>>>
>> > >>>>> Stream >> newLineFor: platform
>> > >>>>>   self nextPutAll: platform lineEnding
>> > >>>>>
>> > >>>>> Stream >> newLineForCurrentPlatform
>> > >>>>>   self newLineFor: OSPlatform current
>> > >>>>>
>> > >>>>> Stream >> newLineForWindows "convenience for the most common platforms
>> > >>>>> Stream >> newLineForUnix
>> > >>>>> Stream >> newLineForHistoricReasons
>> > >>>>>
>> > >>>>> Stream >> newLine
>> > >>>>>   "delegates to one of the above, I'd argue for unix for convenience, but windows is the technically correct combination of cr + lf, and cr only is the historic one"
>> > >>>>>
>> > >>>>>
>> > >>>>> On 4 August 2017 at 14:25, [hidden email] <[hidden email]> wrote:
>> > >>>>> To me it is clear that cr and lf should be in streams. But they should put the 'cr' or 'lf' character only. And of course the platform independent newline should be also.
>> > >>>>>
>> > >>>>> The first (cr, lf) should be used by the code wanting to have absolute control of what is in the stream. The later (newline) when you just want a new line.
>> > >>>>>
>> > >>>>> The two have completely different behaviour, ones are really low level, the other is higher level.
>> > >>>>>
>> > >>>>> On 4 Aug 2017 14:20, "Esteban Lorenzano" <[hidden email]> wrote:
>> > >>>>>
>> > >>>>>> On 4 Aug 2017, at 14:06, Stephane Ducasse <[hidden email]> wrote:
>> > >>>>>>
>> > >>>>>> Well. This is not implemented like that in Pharo.
>> > >>>>>>
>> > >>>>>> cr is bad because it does not mean that it is independent of the platform.
>> > >>>>>> So cr can be redefined as newLine and keep but not used inside the system.
>> > >>>>>
>> > >>>>> sometimes you actually want to write a cr (or a lf). So it needs to remain in the system, of course.
>> > >>>>> now, including #newLine can be cool (most of the times you want the “platform compatible” new line). Also I would consider including #nl, abbreviated… just for convenience :P
>> > >>>>>
>> > >>>>> Esteban
>> > >>>>>
>> > >>>>>>
>> > >>>>>> Stef
>> > >>>>>>
>> > >>>>>> On Fri, Aug 4, 2017 at 12:50 PM, Jan Vrany <[hidden email]> wrote:
>> > >>>>>>> On Fri, 2017-08-04 at 12:03 +0200, Stephane Ducasse wrote:
>> > >>>>>>>> Hi guys
>> > >>>>>>>>
>> > >>>>>>>> While writing pillar code, I ended up using "stream cr" and it
>> > >>>>>>>> worries
>> > >>>>>>>> me to still expand usage
>> > >>>>>>>> of a pattern I would like to remove.
>> > >>>>>>>>
>> > >>>>>>>> Let us imagine that we would like to prepare the migration from cr.
>> > >>>>>>>> I was thinking that we could replace cr invocation by newLine so that
>> > >>>>>>>> after newLine
>> > >>>>>>>> could be redefined as
>> > >>>>>>>>
>> > >>>>>>>> Stream >> newLine
>> > >>>>>>>>     self nextPutAll: OSPlatform current lineEnding
>> > >>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> what do you think about this approach?
>> > >>>>>>>
>> > >>>>>>> Why not? But please keep #cr.
>> > >>>>>>>
>> > >>>>>>> Section 5.9.4.1 of ANSI reads:
>> > >>>>>>>
>> > >>>>>>> Message: cr
>> > >>>>>>>
>> > >>>>>>> Synopsis
>> > >>>>>>> Writes an end-of-line sequence to the receiver.
>> > >>>>>>>
>> > >>>>>>> Definition: <puttableStream>
>> > >>>>>>> A sequence of character objects that constitute the implementation-
>> > >>>>>>> defined end-of-line sequence is added to the receiver in the same
>> > >>>>>>> manner as if the message  #nextPutAll: was sent to the receiver with
>> > >>>>>>> an argument string whose elements are the sequence of characters.
>> > >>>>>>>
>> > >>>>>>> Return Value
>> > >>>>>>> UNSPECIFIED
>> > >>>>>>> Errors
>> > >>>>>>> It is erroneous if any element of the end-of-line sequence is an
>> > >>>>>>> object that does not conform to the receiver's sequence value type .
>> > >>>>>>>
>> > >>>>>>> my 2c,
>> > >>>>>>>
>> > >>>>>>> Jan
>> > >>>>>>>
>> > >>>>>>>>
>> > >>>>>>>> Stef
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> Damien Pollet
>> > >>>>> type less, do more [ | ] http://people.untyped.org/damien.pollet
>> > >>>>
>> > >>>
>> > >>> --
>> > >>> www.tudorgirba.com
>> > >>> www.feenk.com
>> > >>>
>> > >>> "Presenting is storytelling."
>> > >>>
>> > >>>
>> > >>
>> > >>
>> > >
>> >
>> >
>>
>>
>>
>>
>> --
>>
>> Guille Polito
>>
>> Research Engineer
>> French National Center for Scientific Research - http://www.cnrs.fr
>>
>>
>> Web: http://guillep.github.io
>> Phone: <a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13
>
>

--

Guille Polito Research Engineer French National Center for Scientific Research - http://www.cnrs.fr Web: http://guillep.github.io Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13