support of various line ends in trunk
Locked
12
12
support of various line ends in trunk
 
|
1] ABOUT RECENT CHANGES IN TRUNK:
st-80 line end policy was really simple (always CR). It was the best choice possible as long as staying in Smalltalk world. However, I was kind of fed-up with all the hacks for converting to/from various line end flavours, half of which not working... - Since communicating with external world is vital for my own view of Squeak - Since It is far more simple to handle the zoo of line delimiters in Kernel (CompositionScanner / DispalyScanner / String / Stream) I just added this support in trunk. Now, we should be able to import any line termination transparently in the image. For exporting, nothing changed, we still have to care, no magic here, this is driven by external applications requirements. To profit by the new possibilities, just use: - (String>>linesDo:) rather than searching indexOf: Character cr - (Stream>>nextLine) rather than upTo: Character cr There might be some LF/CR-LF support lacking here and there (there are so many #cr senders...), but that shouldn't be hard to fix. 2] IMPORTANT NOTE AND QUESTION: SocketStream>>nextLine does insist on finding a CR-LF pair. This is used in some major protocols. But I find this abusive, and would like to change the default behavior to that of Stream. This would be a nice property that a SocketStream behaves like a FileStream or an ExternalStream. Should I proceed ? Nicolas |
Re: support of various line ends in trunk
|
|
Nicolas Cellier wrote:
> 1] ABOUT RECENT CHANGES IN TRUNK: [...] > - Since It is far more simple to handle the zoo of line delimiters in Kernel > (CompositionScanner / DispalyScanner / String / Stream) Hey, you beat me to it ;-) I had looked at this a little while ago but missed the subtlety in CompositionScanner and friends. Great job! > 2] IMPORTANT NOTE AND QUESTION: > > SocketStream>>nextLine does insist on finding a CR-LF pair. > This is used in some major protocols. > But I find this abusive, and would like to change the default behavior > to that of Stream. > This would be a nice property that a SocketStream behaves like a > FileStream or an ExternalStream. > Should I proceed ? I don't really care as long as the default behavior is strict (i.e., no magic detection of line ends). OTOH, I'm not sure how useful it would be; there is something to be said for strict handling in network protocols. Cheers, - Andreas |
Re: Re: support of various line ends in trunk
|
|
2009/11/16 Andreas Raab <[hidden email]>:
> Nicolas Cellier wrote: >> >> 1] ABOUT RECENT CHANGES IN TRUNK: > > [...] >> >> - Since It is far more simple to handle the zoo of line delimiters in >> Kernel >> (CompositionScanner / DispalyScanner / String / Stream) > > Hey, you beat me to it ;-) I had looked at this a little while ago but > missed the subtlety in CompositionScanner and friends. Great job! > >> 2] IMPORTANT NOTE AND QUESTION: >> >> SocketStream>>nextLine does insist on finding a CR-LF pair. >> This is used in some major protocols. >> But I find this abusive, and would like to change the default behavior >> to that of Stream. >> This would be a nice property that a SocketStream behaves like a >> FileStream or an ExternalStream. >> Should I proceed ? > > I don't really care as long as the default behavior is strict (i.e., no > magic detection of line ends). OTOH, I'm not sure how useful it would be; > there is something to be said for strict handling in network protocols. > There is a #nextLineCrLf for strict requirements, but of course that would put the pressure on package maintainers... > Cheers, > - Andreas > > |
Re: support of various line ends in trunk
|
|
In reply to this post by Nicolas Cellier
Hi Nicolas,
Nicolas Cellier wrote: > 1] ABOUT RECENT CHANGES IN TRUNK: > > st-80 line end policy was really simple (always CR). > It was the best choice possible as long as staying in Smalltalk world. > However, I was kind of fed-up with all the hacks for converting > to/from various line end flavours, half of which not working... > > - Since communicating with external world is vital for my own view of Squeak > - Since It is far more simple to handle the zoo of line delimiters in Kernel > (CompositionScanner / DispalyScanner / String / Stream) > I just added this support in trunk. > > Now, we should be able to import any line termination transparently in > the image. > For exporting, nothing changed, we still have to care, no magic here, > this is driven by external applications requirements. > I think you got this one wrong. In Cuis, in a workspace you can tell the line ending of each line (cr, lf or crlf) and you can actually type all three. Please try it! Use <Enter>, <Shift-Enter> and <Cmd/Alt-Enter>. This way you can edit a text file, and keep it consistent. Otherwise, if you edit an existing file that was edited with a Unix or Windows editor and add CRs to it it will use more than one convention, without you realizing. Showing all in the same way is misleading. Different Strings should look different in the editor! > To profit by the new possibilities, just use: > - (String>>linesDo:) rather than searching indexOf: Character cr > - (Stream>>nextLine) rather than upTo: Character cr > > There might be some LF/CR-LF support lacking here and there (there are > so many #cr senders...), but that shouldn't be hard to fix. > > 2] IMPORTANT NOTE AND QUESTION: > > SocketStream>>nextLine does insist on finding a CR-LF pair. > This is used in some major protocols. > But I find this abusive, and would like to change the default behavior > to that of Stream. > This would be a nice property that a SocketStream behaves like a > FileStream or an ExternalStream. > Should I proceed ? > > Nicolas Cheers, Juan Vuletich |
Re: support of various line ends in trunk
|
|
2009/11/16 Juan Vuletich <[hidden email]>:
> Hi Nicolas, > > Nicolas Cellier wrote: >> >> 1] ABOUT RECENT CHANGES IN TRUNK: >> >> st-80 line end policy was really simple (always CR). >> It was the best choice possible as long as staying in Smalltalk world. >> However, I was kind of fed-up with all the hacks for converting >> to/from various line end flavours, half of which not working... >> >> - Since communicating with external world is vital for my own view of >> Squeak >> - Since It is far more simple to handle the zoo of line delimiters in >> Kernel >> (CompositionScanner / DispalyScanner / String / Stream) >> I just added this support in trunk. >> >> Now, we should be able to import any line termination transparently in >> the image. >> For exporting, nothing changed, we still have to care, no magic here, >> this is driven by external applications requirements. >> > > I think you got this one wrong. In Cuis, in a workspace you can tell the > line ending of each line (cr, lf or crlf) and you can actually type all > three. Please try it! Use <Enter>, <Shift-Enter> and <Cmd/Alt-Enter>. This > way you can edit a text file, and keep it consistent. Otherwise, if you edit > an existing file that was edited with a Unix or Windows editor and add CRs > to it it will use more than one convention, without you realizing. Showing > all in the same way is misleading. Different Strings should look different > in the editor! > Hi Juan, 1) Having the possibility to handle mixed conventions does not mean we are forced to use it ! We can continue to guessLineEndConvention, I did not change this. For writing the file back, either a guess or an explicit requirement might do as well, I did not break anything... 2) I have plenty of mixed conventions files in windows world, not created by Squeak. I cannot guessLineEndConvention on such files. Generally Squeak do a real mess and introduce spurious empty lines in this case. The lineEndTransparent is my best option for reading these files. 3) For displaying, I think we generally don't care whether a CR, LF or CR-LF is used internally What is the semantic of these characters ? To me they all mean the same, no difference, I'm not a typewritter. BUT SEEING CR-LF DISPLAYED SOMETIMES AS TWO LINES AND LF AS NO LINE BREAK IS AWFULL 4) In case we have to compare strings with different lineEndConventions, I suggest the comparing tools make the appropriate conversion first. I don't think this is going to be a real problem. 5) In case we do care of differences because of external requirements, and we want to visualize them, then a specialized DisplayScanner could display a boxed glyph for CR and LF, and an arrow for TAB. (better than this empty box that doesn't tell which character is under) That should be an option of ParagraphEditor 6) I'll have a look in Cuis (need some time...) Nicolas >> To profit by the new possibilities, just use: >> - (String>>linesDo:) rather than searching indexOf: Character cr >> - (Stream>>nextLine) rather than upTo: Character cr >> >> There might be some LF/CR-LF support lacking here and there (there are >> so many #cr senders...), but that shouldn't be hard to fix. >> >> 2] IMPORTANT NOTE AND QUESTION: >> >> SocketStream>>nextLine does insist on finding a CR-LF pair. >> This is used in some major protocols. >> But I find this abusive, and would like to change the default behavior >> to that of Stream. >> This would be a nice property that a SocketStream behaves like a >> FileStream or an ExternalStream. >> Should I proceed ? >> >> Nicolas > > Cheers, > Juan Vuletich > > |
Re: support of various line ends in trunk
|
|
Hi Nicolas,
Nicolas Cellier wrote: > 2009/11/16 Juan Vuletich <[hidden email]>: > >> Hi Nicolas, >> >> Nicolas Cellier wrote: >> >>> 1] ABOUT RECENT CHANGES IN TRUNK: >>> >>> st-80 line end policy was really simple (always CR). >>> It was the best choice possible as long as staying in Smalltalk world. >>> However, I was kind of fed-up with all the hacks for converting >>> to/from various line end flavours, half of which not working... >>> >>> - Since communicating with external world is vital for my own view of >>> Squeak >>> - Since It is far more simple to handle the zoo of line delimiters in >>> Kernel >>> (CompositionScanner / DispalyScanner / String / Stream) >>> I just added this support in trunk. >>> >>> Now, we should be able to import any line termination transparently in >>> the image. >>> For exporting, nothing changed, we still have to care, no magic here, >>> this is driven by external applications requirements. >>> >>> >> I think you got this one wrong. In Cuis, in a workspace you can tell the >> line ending of each line (cr, lf or crlf) and you can actually type all >> three. Please try it! Use <Enter>, <Shift-Enter> and <Cmd/Alt-Enter>. This >> way you can edit a text file, and keep it consistent. Otherwise, if you edit >> an existing file that was edited with a Unix or Windows editor and add CRs >> to it it will use more than one convention, without you realizing. Showing >> all in the same way is misleading. Different Strings should look different >> in the editor! >> >> > > Hi Juan, > > 1) Having the possibility to handle mixed conventions does not mean we > are forced to use it ! > Sure. I didn't say otherwise. > We can continue to guessLineEndConvention, I did not change this. > For writing the file back, either a guess or an explicit requirement > might do as well, I did not break anything... > I didn't say you broke something. I say that you're not doing the best. I say that the way Cuis does it is better: Show the user the real contents in the string, and let him do whatever he wants. > 2) I have plenty of mixed conventions files in windows world, not > created by Squeak. > I cannot guessLineEndConvention on such files. > Generally Squeak do a real mess and introduce spurious empty lines in this case. > The lineEndTransparent is my best option for reading these files. > I think the best is to see all those different line endings, and be able to convert them to whatever you want with one keystroke. In Cuis, ctrl-u converts all line ends to cr. It is trivial to add options for the other conventions. > 3) For displaying, I think we generally don't care whether a CR, LF or > CR-LF is used internally > What is the semantic of these characters ? > I don't know. For different people they could have different semantics. The system should not make a decision for the user. Computers look really silly when they try to be smart... > To me they all mean the same, no difference, I'm not a typewritter. > BUT SEEING CR-LF DISPLAYED SOMETIMES AS TWO LINES AND LF AS NO LINE > BREAK IS AWFULL > Indeed. Check Cuis. > 4) In case we have to compare strings with different lineEndConventions, > I suggest the comparing tools make the appropriate conversion first. I > don't think this is going to be a real problem. > Sure. > 5) In case we do care of differences because of external requirements, > and we want to visualize them, > then a specialized DisplayScanner could display a boxed glyph for CR > and LF, and an arrow for TAB. > (better than this empty box that doesn't tell which character is under) > That should be an option of ParagraphEditor > I disagree. That should be the default behavior, as in Cuis. The user should be in command. > 6) I'll have a look in Cuis (need some time...) > Great! > Nicolas > Cheers, Juan Vuletich >>> To profit by the new possibilities, just use: >>> - (String>>linesDo:) rather than searching indexOf: Character cr >>> - (Stream>>nextLine) rather than upTo: Character cr >>> >>> There might be some LF/CR-LF support lacking here and there (there are >>> so many #cr senders...), but that shouldn't be hard to fix. >>> >>> 2] IMPORTANT NOTE AND QUESTION: >>> >>> SocketStream>>nextLine does insist on finding a CR-LF pair. >>> This is used in some major protocols. >>> But I find this abusive, and would like to change the default behavior >>> to that of Stream. >>> This would be a nice property that a SocketStream behaves like a >>> FileStream or an ExternalStream. >>> Should I proceed ? >>> >>> Nicolas >>> >> Cheers, >> Juan Vuletich >> >> >> > > > |
Re: support of various line ends in trunk
|
|
My 2 cents.
Import: a text stream should detect a line-ending sequence automatically, or use explicitly specified line-ending sequence, if it instructed so. Text editors should not bother with line end formats: their main purpose is to correctly display & edit the text, so its much more important to display it correctly, but quite unimportant in what format the edited text is stored. If current text editor uses cr as line ending - i'm fine with that. Lets then make sure, that it doesn't needs to deal with lf char, since any lf occurences should be wiped out (converted) before going into text editor - at text import stage. Export: at text export, one could always tell , what line endings he wants to see (by default it could use Smalltalk>>platformLineEndings ). I don't think that Squeak meant to be used as a generic-purpose text editor, thus, i don't see real importance in making a smart import-edit-export cycle which should care about preserving line ending sequence unchanged. There are a lot of text editors who can do this and much much more. And squeak text-editing is really suck comparing to existing word processors. Lets just face it: Squeak is not text-editing tool and never been. If one wants a full-featured text editor(s) in squeak - it can build one, with own import-edit-export cycle. But i don't see why basic system functionality, which meant to be used just for editing 1-2 lines of source code, should care so much about that. 2009/11/17 Juan Vuletich <[hidden email]>: > Hi Nicolas, > > Nicolas Cellier wrote: >> >> 2009/11/16 Juan Vuletich <[hidden email]>: >> >>> >>> Hi Nicolas, >>> >>> Nicolas Cellier wrote: >>> >>>> >>>> 1] ABOUT RECENT CHANGES IN TRUNK: >>>> >>>> st-80 line end policy was really simple (always CR). >>>> It was the best choice possible as long as staying in Smalltalk world. >>>> However, I was kind of fed-up with all the hacks for converting >>>> to/from various line end flavours, half of which not working... >>>> >>>> - Since communicating with external world is vital for my own view of >>>> Squeak >>>> - Since It is far more simple to handle the zoo of line delimiters in >>>> Kernel >>>> (CompositionScanner / DispalyScanner / String / Stream) >>>> I just added this support in trunk. >>>> >>>> Now, we should be able to import any line termination transparently in >>>> the image. >>>> For exporting, nothing changed, we still have to care, no magic here, >>>> this is driven by external applications requirements. >>>> >>>> >>> >>> I think you got this one wrong. In Cuis, in a workspace you can tell the >>> line ending of each line (cr, lf or crlf) and you can actually type all >>> three. Please try it! Use <Enter>, <Shift-Enter> and <Cmd/Alt-Enter>. >>> This >>> way you can edit a text file, and keep it consistent. Otherwise, if you >>> edit >>> an existing file that was edited with a Unix or Windows editor and add >>> CRs >>> to it it will use more than one convention, without you realizing. >>> Showing >>> all in the same way is misleading. Different Strings should look >>> different >>> in the editor! >>> >>> >> >> Hi Juan, >> >> 1) Having the possibility to handle mixed conventions does not mean we >> are forced to use it ! >> > > Sure. I didn't say otherwise. > >> We can continue to guessLineEndConvention, I did not change this. >> For writing the file back, either a guess or an explicit requirement >> might do as well, I did not break anything... >> > > I didn't say you broke something. I say that you're not doing the best. I > say that the way Cuis does it is better: Show the user the real contents in > the string, and let him do whatever he wants. > >> 2) I have plenty of mixed conventions files in windows world, not >> created by Squeak. >> I cannot guessLineEndConvention on such files. >> Generally Squeak do a real mess and introduce spurious empty lines in this >> case. >> The lineEndTransparent is my best option for reading these files. >> > > I think the best is to see all those different line endings, and be able to > convert them to whatever you want with one keystroke. In Cuis, ctrl-u > converts all line ends to cr. It is trivial to add options for the other > conventions. > >> 3) For displaying, I think we generally don't care whether a CR, LF or >> CR-LF is used internally >> What is the semantic of these characters ? >> > > I don't know. For different people they could have different semantics. The > system should not make a decision for the user. Computers look really silly > when they try to be smart... > >> To me they all mean the same, no difference, I'm not a typewritter. >> BUT SEEING CR-LF DISPLAYED SOMETIMES AS TWO LINES AND LF AS NO LINE >> BREAK IS AWFULL >> > > Indeed. Check Cuis. > >> 4) In case we have to compare strings with different lineEndConventions, >> I suggest the comparing tools make the appropriate conversion first. I >> don't think this is going to be a real problem. >> > > Sure. > >> 5) In case we do care of differences because of external requirements, >> and we want to visualize them, >> then a specialized DisplayScanner could display a boxed glyph for CR >> and LF, and an arrow for TAB. >> (better than this empty box that doesn't tell which character is under) >> That should be an option of ParagraphEditor >> > > I disagree. That should be the default behavior, as in Cuis. The user should > be in command. > >> 6) I'll have a look in Cuis (need some time...) >> > > Great! > >> Nicolas >> > > Cheers, > Juan Vuletich > >>>> To profit by the new possibilities, just use: >>>> - (String>>linesDo:) rather than searching indexOf: Character cr >>>> - (Stream>>nextLine) rather than upTo: Character cr >>>> >>>> There might be some LF/CR-LF support lacking here and there (there are >>>> so many #cr senders...), but that shouldn't be hard to fix. >>>> >>>> 2] IMPORTANT NOTE AND QUESTION: >>>> >>>> SocketStream>>nextLine does insist on finding a CR-LF pair. >>>> This is used in some major protocols. >>>> But I find this abusive, and would like to change the default behavior >>>> to that of Stream. >>>> This would be a nice property that a SocketStream behaves like a >>>> FileStream or an ExternalStream. >>>> Should I proceed ? >>>> >>>> Nicolas >>>> >>> >>> Cheers, >>> Juan Vuletich >>> >>> >>> >> >> >> > > > -- Best regards, Igor Stasenko AKA sig. |
Re: support of various line ends in trunk
|
|
Igor Stasenko wrote:
> My 2 cents. > > Import: a text stream should detect a line-ending sequence > automatically, or use explicitly specified line-ending sequence, if it > instructed so. > Text editors should not bother with line end formats: their main > purpose is to correctly display & edit the text, so its much more > important to display it correctly, but quite unimportant in what > format the edited text is stored. > If current text editor uses cr as line ending - i'm fine with that. > Lets then make sure, that it doesn't needs to deal with lf char, since > any lf occurences should be wiped out (converted) before going into > text editor - at text import stage. > > Export: at text export, one could always tell , what line endings he > wants to see (by default it could use Smalltalk>>platformLineEndings > ). > > I don't think that Squeak meant to be used as a generic-purpose text > editor, thus, i don't see real importance in making a smart > import-edit-export cycle which should care about preserving line > ending sequence unchanged. There simply should not be any import-edit-export cycle at all. The purpose of an editor is to show the user the characters that comprise the String being edited, and allow him to add / remove characters at will. This is so obvious that I feel silly saying it. Why would some characters be silently messed by the system? I see no reason. It is like the horrible "ASCII mode" in FTP, that breaks every file it downloads. > There are a lot of text editors who can do > this and much much more. And squeak text-editing is really suck > comparing to existing word processors. > Lets just face it: Squeak is not text-editing tool and never been. > If one wants a full-featured text editor(s) in squeak - it can build > one, with own import-edit-export cycle. But i don't see why basic > system functionality, which meant to be used just for editing 1-2 > lines of source code, should care so much about that. > Ok. So let's not fix it. BTW, are you sure you're the same Igor who once said "IMO, in 21st century unicode support is A MUST for any development environment. Of course, if you consider Cuis as dev environment, not as a toy." ? Cheers, Juan Vuletich |
Re: support of various line ends in trunk
|
|
2009/11/17 Juan Vuletich <[hidden email]>:
> Igor Stasenko wrote: >> >> My 2 cents. >> >> Import: a text stream should detect a line-ending sequence >> automatically, or use explicitly specified line-ending sequence, if it >> instructed so. >> Text editors should not bother with line end formats: their main >> purpose is to correctly display & edit the text, so its much more >> important to display it correctly, but quite unimportant in what >> format the edited text is stored. >> If current text editor uses cr as line ending - i'm fine with that. >> Lets then make sure, that it doesn't needs to deal with lf char, since >> any lf occurences should be wiped out (converted) before going into >> text editor - at text import stage. >> >> Export: at text export, one could always tell , what line endings he >> wants to see (by default it could use Smalltalk>>platformLineEndings >> ). >> >> I don't think that Squeak meant to be used as a generic-purpose text >> editor, thus, i don't see real importance in making a smart >> import-edit-export cycle which should care about preserving line >> ending sequence unchanged. > > There simply should not be any import-edit-export cycle at all. The purpose > of an editor is to show the user the characters that comprise the String > being edited, and allow him to add / remove characters at will. This is so > obvious that I feel silly saying it. Why would some characters be silently > messed by the system? I see no reason. It is like the horrible "ASCII mode" > in FTP, that breaks every file it downloads. > Look. Take a blank paper and write some text. Now tell me, where there any control characters lf's, cr's? Print some text using printer. Do you see any control characters there? Also, do you still think that any character which is sent to printer having a visual representation on a paper? Text is meant to be a presentation media, and how it is stored internally is completely irrelevant and up to implementation. User should not care about line endings nor about any other archaic crap which comes from 70-80's ancient terminals. Once we're talking about converting a binary stream(file) into a text , it is a conversion from meaningless sequence of bytes into a representation media , which is text. And once we having this media, we're not talking about control characters anymore. User should not be able to input any terminal control characters, simply because there is no such in text editor as well as there are no terminal (hey we living in 21st centiry!). I hope it is cleared my point. >> There are a lot of text editors who can do >> this and much much more. And squeak text-editing is really suck >> comparing to existing word processors. >> Lets just face it: Squeak is not text-editing tool and never been. >> If one wants a full-featured text editor(s) in squeak - it can build >> one, with own import-edit-export cycle. But i don't see why basic >> system functionality, which meant to be used just for editing 1-2 >> lines of source code, should care so much about that. >> > > Ok. So let's not fix it. BTW, are you sure you're the same Igor who once > said "IMO, in 21st century unicode support is A MUST for any development > environment. Of course, if you consider Cuis as dev environment, not as a > toy." ? > I don't see how it contradicts with the above. In contrast, i see much in common: We live in 21st century, free from black and white text terminals with control characters mess. > Cheers, > Juan Vuletich > > -- Best regards, Igor Stasenko AKA sig. |
Re: support of various line ends in trunk
|
|
Igor Stasenko wrote:
> 2009/11/17 Juan Vuletich <[hidden email]>: > >> Igor Stasenko wrote: >> >>> My 2 cents. >>> >>> Import: a text stream should detect a line-ending sequence >>> automatically, or use explicitly specified line-ending sequence, if it >>> instructed so. >>> Text editors should not bother with line end formats: their main >>> purpose is to correctly display & edit the text, so its much more >>> important to display it correctly, but quite unimportant in what >>> format the edited text is stored. >>> If current text editor uses cr as line ending - i'm fine with that. >>> Lets then make sure, that it doesn't needs to deal with lf char, since >>> any lf occurences should be wiped out (converted) before going into >>> text editor - at text import stage. >>> >>> Export: at text export, one could always tell , what line endings he >>> wants to see (by default it could use Smalltalk>>platformLineEndings >>> ). >>> >>> I don't think that Squeak meant to be used as a generic-purpose text >>> editor, thus, i don't see real importance in making a smart >>> import-edit-export cycle which should care about preserving line >>> ending sequence unchanged. >>> >> There simply should not be any import-edit-export cycle at all. The purpose >> of an editor is to show the user the characters that comprise the String >> being edited, and allow him to add / remove characters at will. This is so >> obvious that I feel silly saying it. Why would some characters be silently >> messed by the system? I see no reason. It is like the horrible "ASCII mode" >> in FTP, that breaks every file it downloads. >> >> > > Look. Take a blank paper and write some text. Now tell me, where there > any control characters lf's, cr's? > Print some text using printer. Do you see any control characters > there? Also, do you still think that any character which is sent to > printer having a visual representation on a paper? > Text is meant to be a presentation media, and how it is stored > internally is completely irrelevant and up to implementation. > User should not care about line endings nor about any other archaic > crap which comes from 70-80's ancient terminals. > You seem to be confusing a tool for displaying text with a tool for editing text. A text editor is for editing text. In Squeak, we are talking about editing Strings. For example, a system for "displaying" music is an audio player. A system for "displaying" HTML is a web browser. None of them allows editing the content, and none of them shows anything but the rendered media. But music software and HTML editors show all the "internal" structure, so the user (media creator) can have control. > Once we're talking about converting a binary stream(file) into a text > , it is a conversion from meaningless sequence of > bytes into a representation media , which is text. And once we having > this media, we're not talking about control characters anymore. > User should not be able to input any terminal control characters, > simply because there is no such in text editor as well as there are no > terminal (hey we living in 21st centiry!). > In Squeak, the editor is the way a developer has to interact (i.e. edit) String objects. It is not an end-user editor adapted to whatever end-user needs you have in mind. > I hope it is cleared my point. > > >>> There are a lot of text editors who can do >>> this and much much more. And squeak text-editing is really suck >>> comparing to existing word processors. >>> Lets just face it: Squeak is not text-editing tool and never been. >>> If one wants a full-featured text editor(s) in squeak - it can build >>> one, with own import-edit-export cycle. But i don't see why basic >>> system functionality, which meant to be used just for editing 1-2 >>> lines of source code, should care so much about that. >>> >>> >> Ok. So let's not fix it. BTW, are you sure you're the same Igor who once >> said "IMO, in 21st century unicode support is A MUST for any development >> environment. Of course, if you consider Cuis as dev environment, not as a >> toy." ? >> >> > > I don't see how it contradicts with the above. In contrast, i see much > in common: We live in 21st century, free > from black and white text terminals with control characters mess. > In one paragraph you say that until it is perfect, it is just a toy. In the other you say that as it was always defective, we should accept it as it is. A clear contradiction to me. >> Cheers, >> Juan Vuletich Cheers, Juan Vuletich |
Re: support of various line ends in trunk
|
|
2009/11/17 Juan Vuletich <[hidden email]>:
> Igor Stasenko wrote: >> >> 2009/11/17 Juan Vuletich <[hidden email]>: >> >>> >>> Igor Stasenko wrote: >>> >>>> >>>> My 2 cents. >>>> >>>> Import: a text stream should detect a line-ending sequence >>>> automatically, or use explicitly specified line-ending sequence, if it >>>> instructed so. >>>> Text editors should not bother with line end formats: their main >>>> purpose is to correctly display & edit the text, so its much more >>>> important to display it correctly, but quite unimportant in what >>>> format the edited text is stored. >>>> If current text editor uses cr as line ending - i'm fine with that. >>>> Lets then make sure, that it doesn't needs to deal with lf char, since >>>> any lf occurences should be wiped out (converted) before going into >>>> text editor - at text import stage. >>>> >>>> Export: at text export, one could always tell , what line endings he >>>> wants to see (by default it could use Smalltalk>>platformLineEndings >>>> ). >>>> >>>> I don't think that Squeak meant to be used as a generic-purpose text >>>> editor, thus, i don't see real importance in making a smart >>>> import-edit-export cycle which should care about preserving line >>>> ending sequence unchanged. >>>> >>> >>> There simply should not be any import-edit-export cycle at all. The >>> purpose >>> of an editor is to show the user the characters that comprise the String >>> being edited, and allow him to add / remove characters at will. This is >>> so >>> obvious that I feel silly saying it. Why would some characters be >>> silently >>> messed by the system? I see no reason. It is like the horrible "ASCII >>> mode" >>> in FTP, that breaks every file it downloads. >>> >>> >> >> Look. Take a blank paper and write some text. Now tell me, where there >> any control characters lf's, cr's? >> Print some text using printer. Do you see any control characters >> there? Also, do you still think that any character which is sent to >> printer having a visual representation on a paper? >> Text is meant to be a presentation media, and how it is stored >> internally is completely irrelevant and up to implementation. >> User should not care about line endings nor about any other archaic >> crap which comes from 70-80's ancient terminals. >> > > You seem to be confusing a tool for displaying text with a tool for editing > text. A text editor is for editing text. In Squeak, we are talking about > editing Strings. For example, a system for "displaying" music is an audio > player. A system for "displaying" HTML is a web browser. None of them allows > editing the content, and none of them shows anything but the rendered media. > But music software and HTML editors show all the "internal" structure, so > the user (media creator) can have control. > There are a binary editors which allowing you to change byte by byte using hex values. But since we are talking about text editor, it is meant for editing a text, but not literally Strings or control characters. So, if you want a String editor, which allows you to insert/remove/replace any character at any position - you can implement it. But then i think you can just use an inspector. String is not Text. If we are talking about editing text, please stop talking about control characters as well as Strings, because obviously text is not only about strings and control chars, its also about formatting (font, color, alignment etc) and many other things.. You won't be able to represent the text as a media (including for editing) if your abstraction stops at the point of having control characters only. What control character should represent an image? and what is for link? It is easy to see that control chars doesn't help you with that, they should live where they belong to - in a files, where text is converted from live objects into a file using specific binary format (XML/HTML/PDF/plain text/choose yours). That's where we could choose one or another line endings etc.. but not in text editor, which is a nonsense because it deals with live objects. >> Once we're talking about converting a binary stream(file) into a text >> , it is a conversion from meaningless sequence of >> bytes into a representation media , which is text. And once we having >> this media, we're not talking about control characters anymore. >> User should not be able to input any terminal control characters, >> simply because there is no such in text editor as well as there are no >> terminal (hey we living in 21st centiry!). >> > > In Squeak, the editor is the way a developer has to interact (i.e. edit) > String objects. It is not an end-user editor adapted to whatever end-user > needs you have in mind. > Hmm.. i was always thinking that text editor is for editing the text, not Strings. :) Look, we write tools for users, but not for machine. Same as we should invent languages to serve for developers, not for compilers. >> I hope it is cleared my point. >> >> >>>> >>>> There are a lot of text editors who can do >>>> this and much much more. And squeak text-editing is really suck >>>> comparing to existing word processors. >>>> Lets just face it: Squeak is not text-editing tool and never been. >>>> If one wants a full-featured text editor(s) in squeak - it can build >>>> one, with own import-edit-export cycle. But i don't see why basic >>>> system functionality, which meant to be used just for editing 1-2 >>>> lines of source code, should care so much about that. >>>> >>>> >>> >>> Ok. So let's not fix it. BTW, are you sure you're the same Igor who once >>> said "IMO, in 21st century unicode support is A MUST for any development >>> environment. Of course, if you consider Cuis as dev environment, not as a >>> toy." ? >>> >>> >> >> I don't see how it contradicts with the above. In contrast, i see much >> in common: We live in 21st century, free >> from black and white text terminals with control characters mess. >> > > In one paragraph you say that until it is perfect, it is just a toy. In the > other you say that as it was always defective, we should accept it as it is. > A clear contradiction to me. > Sorry, you seem misunderstood. Text internal representation is completely irrelevant. I just said, that cr char is ok to me as an internal end-of-line marker and i don't see any reason why we should change that. It is an implementation detail. Internally,You could use any other character to mark the line ends, as well as you can keep a collection of lines for editing, instead using of a single buffer , then you don't need a line separators at all. >>> Cheers, >>> Juan Vuletich > > Cheers, > Juan Vuletich > -- Best regards, Igor Stasenko AKA sig. |
Re: support of various line ends in trunk
|
|
Igor Stasenko wrote:
> > ... > That's where we could choose one or another line endings etc.. but not > in text editor, which is a nonsense because it deals with live > objects. > The editors in Squeak (in browsers, for example) deal with live objects: instances of class String. They might include any sequence of characters. Including those you don't like. >> In Squeak, the editor is the way a developer has to interact (i.e. edit) >> String objects. It is not an end-user editor adapted to whatever end-user >> needs you have in mind. >> >> > > Hmm.. i was always thinking that text editor is for editing the text, > not Strings. :) > Then may be you're wrong. > Look, we write tools for users, but not for machine. > Same as we should invent languages to serve for developers, not for compilers. > Dev tools are for developers. That's what the base system is about. Not about the machine, not about end-users. Just developers. Cheers, Juan Vuletich |
Re: support of various line ends in trunk
|
|
2009/11/17 Juan Vuletich <[hidden email]>:
> Igor Stasenko wrote: >> >> ... >> That's where we could choose one or another line endings etc.. but not >> in text editor, which is a nonsense because it deals with live >> objects. >> > > The editors in Squeak (in browsers, for example) deal with live objects: > instances of class String. They might include any sequence of characters. > Including those you don't like. > Squeak's Text editor operating with instances of Text, to be more precise. And Text instance is already far more than a String, but i think you know it yourself , so i wonder why you persist pointing me to Strings. The issue we're arguing is about String <-> Text conversion, and internal representation of text in editor. I stating, that we could completely get rid of any notion of controls characters in Text , as well as in text editors, and make implementation cleaner & leaner and free of any bizarre & cryptic character mangling. And then we could have only few places, where we should care about control chars: String <=> Text conversion and/or Stream <=> Text conversion. >>> In Squeak, the editor is the way a developer has to interact (i.e. edit) >>> String objects. It is not an end-user editor adapted to whatever end-user >>> needs you have in mind. >>> >>> >> >> Hmm.. i was always thinking that text editor is for editing the text, >> not Strings. :) >> > > Then may be you're wrong. > >> Look, we write tools for users, but not for machine. >> Same as we should invent languages to serve for developers, not for >> compilers. >> > > Dev tools are for developers. That's what the base system is about. Not > about the machine, not about end-users. Just developers. > And who are the developers from text-editor point of view? Aren't they just users who need to edit a text? I barely see, where i could make use of editor which allows me to edit control chars. During my lifetime experience as developer, i never had a need to deal with control characters in editor. I'd rather prefer to convert it before/after editing than inside an editor, or use binary editor, where i can see & change every byte. I didn't said that we won't need to deal with line endings, but please tell me, why it should be a text editor? I know, that TextEditor in Squeak tries to do everything at once.. but don't you think that maybe it is time to repurpose it for only text editing, while leave rest to other tools? Lets create & use a separate tools for that. Text editor - for editing a text. String/text converter - for conversion & dealing with line endings and other idiosyncrazy. And then both, developers and users will be pretty happy. > Cheers, > Juan Vuletich > > -- Best regards, Igor Stasenko AKA sig. |
Re: support of various line ends in trunk
|
|
Igor Stasenko wrote:
> 2009/11/17 Juan Vuletich <[hidden email]>: > >> Igor Stasenko wrote: >> >>> ... >>> That's where we could choose one or another line endings etc.. but not >>> in text editor, which is a nonsense because it deals with live >>> objects. >>> >>> >> The editors in Squeak (in browsers, for example) deal with live objects: >> instances of class String. They might include any sequence of characters. >> Including those you don't like. >> >> > > Squeak's Text editor operating with instances of Text, to be more > precise. And Text instance is already far more than a String, but i > think you know it yourself , so i wonder why you persist pointing me > to Strings. > Because it is irrelevant to what we were discussing. > The issue we're arguing is about String <-> Text conversion, and > internal representation of text in editor. > That's the issue you're arguing about now. The original discussion was whether the editor should handle or not the line ending conversions our model objects already support. > I stating, that we could completely get rid of any notion of controls > characters in Text , as well as in text editors, > and make implementation cleaner & leaner and free of any bizarre & > cryptic character mangling. > And then we could have only few places, where we should care about > control chars: String <=> Text conversion and/or > Stream <=> Text conversion. > Yes, that would be possible. Of course, that doesn't preclude the editor to actually support our model objects. >>>> In Squeak, the editor is the way a developer has to interact (i.e. edit) >>>> String objects. It is not an end-user editor adapted to whatever end-user >>>> needs you have in mind. >>>> >>>> >>>> >>> Hmm.. i was always thinking that text editor is for editing the text, >>> not Strings. :) >>> >>> >> Then may be you're wrong. >> >> >>> Look, we write tools for users, but not for machine. >>> Same as we should invent languages to serve for developers, not for >>> compilers. >>> >>> >> Dev tools are for developers. That's what the base system is about. Not >> about the machine, not about end-users. Just developers. >> >> > > And who are the developers from text-editor point of view? Aren't they > just users who need to edit a text? > I barely see, where i could make use of editor which allows me to edit > control chars. > I never talked about control chars. We were talking about line ending conversions. A typical scenario is needing to edit ini files for real users that are stored in a server. I need to support users with Windows and Mac. And I want to leave the files as they need them. The best way to do that is with the Cuis editor. Then, I'm in control. > During my lifetime experience as developer, i never had a need to deal > with control characters in editor. > I'd rather prefer to convert it before/after editing than inside an > editor, or use binary editor, where i can see & change every byte. > > I didn't said that we won't need to deal with line endings, but please > tell me, why it should be a text editor? > I said it countless times now: Because it is the most comfortable tool to deal with String objects. > I know, that TextEditor in Squeak tries to do everything at once.. but > don't you think that maybe it is time to repurpose it for only text > editing, while leave rest to other tools? > Lets create & use a separate tools for that. Text editor - for editing > a text. String/text converter - for conversion & dealing with line > endings and other idiosyncrazy. > No, I'd repurpose it only for string editing. > And then both, developers and users will be pretty happy. > Indeed. Cheers, Juan Vuletich |
Re: support of various line ends in trunk
|
|
On 17.11.2009, at 15:47, Juan Vuletich wrote:
> > I never talked about control chars. We were talking about line ending conversions. A typical scenario is needing to edit ini files for real users that are stored in a server. I need to support users with Windows and Mac. And I want to leave the files as they need them. The best way to do that is with the Cuis editor. Then, I'm in control. Squeak's editor used to be binary-safe. You could visit any file in a file list, change a character, and save it back knowing that this would be the only byte altered. I wonder if there could be a convenient way to do that, while still interpreting e.g. utf-8 sequences as unicode characters. Maybe all that's needed is a stricter separation of model and rendering? - Bert - |
Re: support of various line ends in trunk
|
|
In reply to this post by Juan Vuletich-4
On 11/17/2009 5:15 AM, Juan Vuletich wrote:
> Igor Stasenko wrote: >> My 2 cents. >> >> Import: a text stream should detect a line-ending sequence >> automatically, or use explicitly specified line-ending sequence, if it >> instructed so. >> Text editors should not bother with line end formats: their main >> purpose is to correctly display & edit the text, so its much more >> important to display it correctly, but quite unimportant in what >> format the edited text is stored. >> If current text editor uses cr as line ending - i'm fine with that. >> Lets then make sure, that it doesn't needs to deal with lf char, since >> any lf occurences should be wiped out (converted) before going into >> text editor - at text import stage. >> >> Export: at text export, one could always tell , what line endings he >> wants to see (by default it could use Smalltalk>>platformLineEndings >> ). >> >> I don't think that Squeak meant to be used as a generic-purpose text >> editor, thus, i don't see real importance in making a smart >> import-edit-export cycle which should care about preserving line >> ending sequence unchanged. > > There simply should not be any import-edit-export cycle at all. The > purpose of an editor is to show the user the characters that comprise > the String being edited, and allow him to add / remove characters at > will. This is so obvious that I feel silly saying it. Why would some > characters be silently messed by the system? I see no reason. It is > like the horrible "ASCII mode" in FTP, that breaks every file it > downloads. Simple questions below. I do not necessarily know the answers. If I open up a Workspace and begin editing, whether Squeak or Cuis, by default what kind of line endings do I get? Will I ever in editing newly created Squeak text/strings by default ever have multiple different line-endings? If I ever have text in Squeak which has multiple line-endings, where did it come from? A file? Regardless of whether that file is on disk or in a stream via network. And if a file then there is somewhere in this process some form of reading the file from wherever and bringing into Squeak for reading, editing, displaying or whatever. And if I ever want to use my edits outside of Squeak, then there is the process of creating or updating the file with the edits. Therefore there must be an import/edit-display/export cycle. Otherwise it is all internal to Squeak and if so should only use Squeak's defaults. I personally fail to see why anybody would want to retain multiple line ending types in any file or text regardless as to whether they were there already or not. They in general only get there from multiple edits of the file, from multiple OSes. It is done for interoperability between platforms and editors but is not a natural thing for any editor to do otherwise. Does any editor anywhere when creating a new document insert more than one type of line ending? This is a display of what is natural. Personally as a simple end user. I would prefer that my files if being edited by some editor, somewhere, that it correct all of the crappy multiple line endings and simply use the platform native line endings when saving the file. I couldn't care less what the text editors internal representation was of those line endings. I see no real purpose for any file with multiple line endings. Fact is that we are operating on some specific platform. Whether Windows, Mac, Linux, BSD or whatever. By default and by nature all of those systems have expectations of what a line ending is. If I am on my Windows machine I would prefer all files I edit to be with Windows preferred line endings. And on my Mac with Macs. And on my Ubuntu Linux machine, Linuxes. If I send a file from one to another I am pleased if it gets converted to the native format for my file on that platform. It is an unpleasant experience to double-click a file and see Notepad open up and the entire file is on one line. But if I explicitly open the file in WordPad it displays just fine. Yuck! In text files multiple types of line endings serve absolutely no purpose. The are all equivalent and should all be converted to platform preference. And if they serve some purpose and would break something if otherwise replaced. We are then talking about an application specific file and that is a whole different ballgame. In general, I want an editor to edit any document, object, etc. as it would naturally as if it was also the creator. This is my opinion. But it is one from a long time of computer usage with many OSes and using text files between them. Windows almost always has a way of screwing up files with multiple or wrong line endings unless you explicitly select a client which does the right thing. |
Re: support of various line ends in trunk
|
|
In reply to this post by Juan Vuletich-4
On 11/17/2009 8:47 AM, Juan Vuletich wrote:
> > I never talked about control chars. We were talking about line ending > conversions. A typical scenario is needing to edit ini files for real > users that are stored in a server. I need to support users with > Windows and Mac. And I want to leave the files as they need them. The > best way to do that is with the Cuis editor. Then, I'm in control. You are doing this from Squeak? It would seem to me to be more natural for the editing in Squeak to be of a single consistent line ending and then the export to your server into the line ending type required there. Neither Mac nor Windows or anyone to my knowledge desire/require any line ending outside of their native default line ending. So I fail to see why this would be important in the editor and done better in the editor than in the export mechanism saving the edited text to a file whether local or network? And in a previous email you wrote: On 11/16/2009 4:10 PM, Juan Vuletich wrote: > > I think you got this one wrong. In Cuis, in a workspace you can tell the > line ending of each line (cr, lf or crlf) and you can actually type all > three. Please try it! Use <Enter>, <Shift-Enter> and <Cmd/Alt-Enter>. > This way you can edit a text file, and keep it consistent. Otherwise, if > you edit an existing file that was edited with a Unix or Windows editor > and add CRs to it it will use more than one convention, without you > realizing. Showing all in the same way is misleading. Different Strings > should look different in the editor! I still fail to see a purpose in having more than one line ending type in a single document or file. It isn't natural. And if only one type of line ending in a document/file/string. Editing (internal) line endings does not necessarily equal export or external line endings. If you have a purpose for multiple line endings in a single file/document/object/... I would like to know what that is. If not, then it would seem that simply having the ability to export to a platform explicit version (regardless of the platform of current use) is a valuable thing. But should be an explicit decision not a default one. Jimmie |
Re: support of various line ends in trunk
|
|
2009/11/17 Jimmie Houchin <[hidden email]>:
> On 11/17/2009 8:47 AM, Juan Vuletich wrote: >> >> I never talked about control chars. We were talking about line ending >> conversions. A typical scenario is needing to edit ini files for real users >> that are stored in a server. I need to support users with Windows and Mac. >> And I want to leave the files as they need them. The best way to do that is >> with the Cuis editor. Then, I'm in control. > > You are doing this from Squeak? > > It would seem to me to be more natural for the editing in Squeak to be of a > single consistent line ending and then the export to your server into the > line ending type required there. > > Neither Mac nor Windows or anyone to my knowledge desire/require any line > ending outside of their native default line ending. So I fail to see why > this would be important in the editor and done better in the editor than in > the export mechanism saving the edited text to a file whether local or > network? > You simply never worked with cygwin under windows, nor transfered files edited in windows to QNX et vice et versa. > And in a previous email you wrote: > On 11/16/2009 4:10 PM, Juan Vuletich wrote: >> >> I think you got this one wrong. In Cuis, in a workspace you can tell the >> line ending of each line (cr, lf or crlf) and you can actually type all >> three. Please try it! Use <Enter>, <Shift-Enter> and <Cmd/Alt-Enter>. >> This way you can edit a text file, and keep it consistent. Otherwise, if >> you edit an existing file that was edited with a Unix or Windows editor >> and add CRs to it it will use more than one convention, without you >> realizing. Showing all in the same way is misleading. Different Strings >> should look different in the editor! > > I still fail to see a purpose in having more than one line ending type in a > single document or file. It isn't natural. > > And if only one type of line ending in a document/file/string. > Editing (internal) line endings does not necessarily equal export or > external line endings. > > If you have a purpose for multiple line endings in a single > file/document/object/... I would like to know what that is. > > If not, then it would seem that simply having the ability to export to a > platform explicit version (regardless of the platform of current use) is a > valuable thing. But should be an explicit decision not a default one. > > Jimmie > It's not the desire of any one, it is just reallity. Simple modern code editors (string editors) can deal with mixed line endings and most tools won't care, this is just transparent, and we can have that in Squeak to, it's easy and cheap. For example, plenty of squeak code did contain line feeds, and that did not hurt... (except when the ugly boxes were displayed). If we have applications with strict requirements, then no problem, we will continue to do the convert from/edit/convert to cycle. Qui peut le plus peut le moins. Nicoals |
Re: support of various line ends in trunk
|
|
In reply to this post by Bert Freudenberg
2009/11/17 Bert Freudenberg <[hidden email]>:
> On 17.11.2009, at 15:47, Juan Vuletich wrote: >> >> I never talked about control chars. We were talking about line ending conversions. A typical scenario is needing to edit ini files for real users that are stored in a server. I need to support users with Windows and Mac. And I want to leave the files as they need them. The best way to do that is with the Cuis editor. Then, I'm in control. > > Squeak's editor used to be binary-safe. You could visit any file in a file list, change a character, and save it back knowing that this would be the only byte altered. I wonder if there could be a convenient way to do that, while still interpreting e.g. utf-8 sequences as unicode characters. Maybe all that's needed is a stricter separation of model and rendering? > I fail see how binary-safety is important for text editing. Open any text file, press cr in the middle of any line. Now what chars will be inserted? What do you expect to be expected? Depending on OS, depending on original contents of file? What if file consists of a single line w/o any line separators? And lastly, what if i insert a unicode char and save it, while used by tools which understand only ASCII chars? Sticking with binary-safety, which i think just a side effect of current implementation, not a useful feature, is wrong direction in designing text editors. > - Bert - > > > -- Best regards, Igor Stasenko AKA sig. |
Re: support of various line ends in trunk
|
|
In reply to this post by Jimmie Houchin-3
Jimmie Houchin wrote:
> On 11/17/2009 5:15 AM, Juan Vuletich wrote: >> >> There simply should not be any import-edit-export cycle at all. The >> purpose of an editor is to show the user the characters that comprise >> the String being edited, and allow him to add / remove characters at >> will. This is so obvious that I feel silly saying it. Why would some >> characters be silently messed by the system? I see no reason. It is >> like the horrible "ASCII mode" in FTP, that breaks every file it >> downloads. > > Simple questions below. I do not necessarily know the answers. > > If I open up a Workspace and begin editing, whether Squeak or Cuis, by > default what kind of line endings do I get? Cr. > Will I ever in editing newly created Squeak text/strings by default > ever have multiple different line-endings? No, if the only way you modify them is an editor. > If I ever have text in Squeak which has multiple line-endings, where > did it come from? > A file? Regardless of whether that file is on disk or in a stream via > network. Yes. Or some code that built it. Perhaps clipboard? Not sure. Cheers, Juan Vuletich |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |