Comments in STON

>
> Basically STONCStyleCommentsSkipStream>>#nextNonCommentChar should know when it is inside a string and then ignore comments. But that means, in the case of STON/JSON, handling both ' (single) and " (double) quoted strings as well as interpret \' and \". All this should be doable, but it is not that easy to do elegantly. I'll see what I can do.
>
> Sven
>
>> On 16 Jul 2015, at 17:35, Guillermo Polito <[hidden email]> wrote:
>>
>> Hi Sven!
>>
>> I'm finally using the new STON with comments and I found one little problem: my ston file has urls :/
>>
>> {
>> // comment ok, skipped
>> /* comment ok, skipped */
>>    "aURL": "http://ouch.it.will.be.skipped/and/break?my=ston"
>> }
>>
>> That is, doing:
>>
>> STON fromStreamWithComments: '{
>> // comment ok, skipped
>> /* comment ok, skipped */
>>    "aURL": "http://ouch.it.will.be.skipped/and/break?my=ston"
>> }' readStream
>>
>> Will fail with an invalid input error because after skipping comments my STON stream remains like
>>
>> STON fromStreamWithComments: '{
>>    "aURL": "http:
>> }
>>
>> Any thoughts?
>> Guille
>>
>> El dom., 14 de jun. de 2015 a la(s) 3:06 a. m., stepharo <[hidden email]> escribió:
>>
>>
>> Le 10/6/15 20:39, Sven Van Caekenberghe a écrit :
>>>> On 10 Jun 2015, at 15:13, stepharo <[hidden email]> wrote:
>>>>
>>>> This is in the entreprise book. I added a subsection on comments.
>>>> May be I committed but forgot to push.
>>>> Now I did it.
>>> And now I see it.
>>> I made a small change.
>> super!
>>
>>>> Stef
>>>>
>>>>
>>>> Le 9/6/15 21:50, Sven Van Caekenberghe a écrit :
>>>>> Where exactly can I find what you changed ?
>>>>> I don't see any commit of you on github ...
>>>>>
>>>>>> On 09 Jun 2015, at 20:10, stepharo <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Le 9/6/15 20:06, stepharo a écrit :
>>>>>>> I will update the STON chapter :)
>>>>>> Done :)
>>>>>>
>>>>>>> Le 8/6/15 23:39, Sven Van Caekenberghe a écrit :
>>>>>>>> I would do it like this: use a wrapping stream to skip comments.
>>>>>>>>
>>>>>>>> ===
>>>>>>>> Name: STON-Core-SvenVanCaekenberghe.59
>>>>>>>> Author: SvenVanCaekenberghe
>>>>>>>> Time: 8 June 2015, 11:34:32.886167 pm
>>>>>>>> UUID: 7ab942b5-9d1a-44c3-9d58-da29feb72380
>>>>>>>> Ancestors: STON-Core-SvenVanCaekenberghe.58
>>>>>>>>
>>>>>>>> Add experimental STONCStyleCommentsSkipStream and tests
>>>>>>>> ===
>>>>>>>> Name: STON-Tests-SvenVanCaekenberghe.54
>>>>>>>> Author: SvenVanCaekenberghe
>>>>>>>> Time: 8 June 2015, 11:34:52.918771 pm
>>>>>>>> UUID: 3dcb2c9c-6efe-4d62-bf36-3bbe91f09d08
>>>>>>>> Ancestors: STON-Tests-SvenVanCaekenberghe.53
>>>>>>>>
>>>>>>>> Add experimental STONCStyleCommentsSkipStream and tests
>>>>>>>> ===
>>>>>>>>
>>>>>>>> Then you can do as follows:
>>>>>>>>
>>>>>>>> STON fromStream: (STONCStyleCommentsSkipStream on: 'Point[/* this is X*/ 1, /* this is Y*/ 2] // Nice huh ?' readStream).
>>>>>>>>
>>>>>>>>      => 1@2
>>>>>>>>
>>>>>>>> STON fromStream: (STONCStyleCommentsSkipStream on: '// Here is how you create a point:
>>>>>>>> Point[
>>>>>>>>      // this is X
>>>>>>>>      1,
>>>>>>>>      // this is Y
>>>>>>>>      2 ]
>>>>>>>> // Nice huh ?' readStream).
>>>>>>>>
>>>>>>>>      => 1@2
>>>>>>>>
>>>>>>>> This helper class would be useable in other contexts too (like NeoJSON).
>>>>>>>>
>>>>>>>> The advantage is that it does not change the STON (or JSON) syntax itself, it just add some functionality for those who want it.
>>>>>>>>
>>>>>>>> What do you think ?
>>>>>>>>
>>>>>>>> Sven
>>>>>>>>
>>>>>>>> PS: this probably needs some more testing
>>>>>>>>
>>>>>>>>> On 08 Jun 2015, at 17:16, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>>> On 08 Jun 2015, at 17:03, Damien Cassou <[hidden email]> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sven Van Caekenberghe <[hidden email]> writes:
>>>>>>>>>>
>>>>>>>>>>> I am not so sure we should add that.
>>>>>>>>>>>
>>>>>>>>>>> The JSON spec explicitly does not allow comments because of fear of abuse (that the comments would be used to add semantic meaning outside the spec).
>>>>>>>>>> really? That's surprising. Comments in configuration files are very
>>>>>>>>>> important. Even more when the configuration files are templates for new
>>>>>>>>>> users.
>>>>>>>>> I know some people are using STON in ways that I did not originally anticipate, that is of course cool, but writing configuration files was not the original goal. STON is a version of JSON that is closer to our object model.
>>>>>>>>>
>>>>>>>>> Like I said, I will think about it.
>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Damien Cassou
>>>>>>>>>> http://damiencassou.seasidehosting.st
>>>>>>>>>>
>>>>>>>>>> "Success is the ability to go from one failure to another without
>>>>>>>>>> losing enthusiasm." --Winston Churchill
>>>
>>
>
>

> On 17 Jul 2015, at 09:00, stepharo <[hidden email]> wrote:
>
> Hi sven
>
>
> Le 16/7/15 21:28, Sven Van Caekenberghe a écrit :
>> Guille,
>>
>> I understand the problem.
>>
>> The reason is that STONCStyleCommentsSkipStream was/is a quick hack and does not know/understand anything about strings.
>>
>> It seems that the C preprocessor ignores comments in C strings, so we could consider adding that.
> What is the C preprocessor.

>>
>> Basically STONCStyleCommentsSkipStream>>#nextNonCommentChar should know when it is inside a string and then ignore comments. But that means, in the case of STON/JSON, handling both ' (single) and " (double) quoted strings as well as interpret \' and \". All this should be doable, but it is not that easy to do elegantly. I'll see what I can do.
>>
>> Sven
>>
>>> On 16 Jul 2015, at 17:35, Guillermo Polito <[hidden email]> wrote:
>>>
>>> Hi Sven!
>>>
>>> I'm finally using the new STON with comments and I found one little problem: my ston file has urls :/
>>>
>>> {
>>> // comment ok, skipped
>>> /* comment ok, skipped */
>>>   "aURL": "http://ouch.it.will.be.skipped/and/break?my=ston"
>>> }
>>>
>>> That is, doing:
>>>
>>> STON fromStreamWithComments: '{
>>> // comment ok, skipped
>>> /* comment ok, skipped */
>>>   "aURL": "http://ouch.it.will.be.skipped/and/break?my=ston"
>>> }' readStream
>>>
>>> Will fail with an invalid input error because after skipping comments my STON stream remains like
>>>
>>> STON fromStreamWithComments: '{
>>>   "aURL": "http:
>>> }
>>>
>>> Any thoughts?
>>> Guille
>>>
>>> El dom., 14 de jun. de 2015 a la(s) 3:06 a. m., stepharo <[hidden email]> escribió:
>>>
>>>
>>> Le 10/6/15 20:39, Sven Van Caekenberghe a écrit :
>>>>> On 10 Jun 2015, at 15:13, stepharo <[hidden email]> wrote:
>>>>>
>>>>> This is in the entreprise book. I added a subsection on comments.
>>>>> May be I committed but forgot to push.
>>>>> Now I did it.
>>>> And now I see it.
>>>> I made a small change.
>>> super!
>>>
>>>>> Stef
>>>>>
>>>>>
>>>>> Le 9/6/15 21:50, Sven Van Caekenberghe a écrit :
>>>>>> Where exactly can I find what you changed ?
>>>>>> I don't see any commit of you on github ...
>>>>>>
>>>>>>> On 09 Jun 2015, at 20:10, stepharo <[hidden email]> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Le 9/6/15 20:06, stepharo a écrit :
>>>>>>>> I will update the STON chapter :)
>>>>>>> Done :)
>>>>>>>
>>>>>>>> Le 8/6/15 23:39, Sven Van Caekenberghe a écrit :
>>>>>>>>> I would do it like this: use a wrapping stream to skip comments.
>>>>>>>>>
>>>>>>>>> ===
>>>>>>>>> Name: STON-Core-SvenVanCaekenberghe.59
>>>>>>>>> Author: SvenVanCaekenberghe
>>>>>>>>> Time: 8 June 2015, 11:34:32.886167 pm
>>>>>>>>> UUID: 7ab942b5-9d1a-44c3-9d58-da29feb72380
>>>>>>>>> Ancestors: STON-Core-SvenVanCaekenberghe.58
>>>>>>>>>
>>>>>>>>> Add experimental STONCStyleCommentsSkipStream and tests
>>>>>>>>> ===
>>>>>>>>> Name: STON-Tests-SvenVanCaekenberghe.54
>>>>>>>>> Author: SvenVanCaekenberghe
>>>>>>>>> Time: 8 June 2015, 11:34:52.918771 pm
>>>>>>>>> UUID: 3dcb2c9c-6efe-4d62-bf36-3bbe91f09d08
>>>>>>>>> Ancestors: STON-Tests-SvenVanCaekenberghe.53
>>>>>>>>>
>>>>>>>>> Add experimental STONCStyleCommentsSkipStream and tests
>>>>>>>>> ===
>>>>>>>>>
>>>>>>>>> Then you can do as follows:
>>>>>>>>>
>>>>>>>>> STON fromStream: (STONCStyleCommentsSkipStream on: 'Point[/* this is X*/ 1, /* this is Y*/ 2] // Nice huh ?' readStream).
>>>>>>>>>
>>>>>>>>>     => 1@2
>>>>>>>>>
>>>>>>>>> STON fromStream: (STONCStyleCommentsSkipStream on: '// Here is how you create a point:
>>>>>>>>> Point[
>>>>>>>>>     // this is X
>>>>>>>>>     1,
>>>>>>>>>     // this is Y
>>>>>>>>>     2 ]
>>>>>>>>> // Nice huh ?' readStream).
>>>>>>>>>
>>>>>>>>>     => 1@2
>>>>>>>>>
>>>>>>>>> This helper class would be useable in other contexts too (like NeoJSON).
>>>>>>>>>
>>>>>>>>> The advantage is that it does not change the STON (or JSON) syntax itself, it just add some functionality for those who want it.
>>>>>>>>>
>>>>>>>>> What do you think ?
>>>>>>>>>
>>>>>>>>> Sven
>>>>>>>>>
>>>>>>>>> PS: this probably needs some more testing
>>>>>>>>>
>>>>>>>>>> On 08 Jun 2015, at 17:16, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>>>>>>>
>>>>>>>>>>> On 08 Jun 2015, at 17:03, Damien Cassou <[hidden email]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Sven Van Caekenberghe <[hidden email]> writes:
>>>>>>>>>>>
>>>>>>>>>>>> I am not so sure we should add that.
>>>>>>>>>>>>
>>>>>>>>>>>> The JSON spec explicitly does not allow comments because of fear of abuse (that the comments would be used to add semantic meaning outside the spec).
>>>>>>>>>>> really? That's surprising. Comments in configuration files are very
>>>>>>>>>>> important. Even more when the configuration files are templates for new
>>>>>>>>>>> users.
>>>>>>>>>> I know some people are using STON in ways that I did not originally anticipate, that is of course cool, but writing configuration files was not the original goal. STON is a version of JSON that is closer to our object model.
>>>>>>>>>>
>>>>>>>>>> Like I said, I will think about it.
>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Damien Cassou
>>>>>>>>>>> http://damiencassou.seasidehosting.st
>>>>>>>>>>>
>>>>>>>>>>> "Success is the ability to go from one failure to another without
>>>>>>>>>>> losing enthusiasm." --Winston Churchill
>>>>
>>>
>>
>>
>
>

> On 16 Jul 2015, at 21:28, Sven Van Caekenberghe <[hidden email]> wrote:
>
> Guille,
>
> I understand the problem.
>
> The reason is that STONCStyleCommentsSkipStream was/is a quick hack and does not know/understand anything about strings.
>
> It seems that the C preprocessor ignores comments in C strings, so we could consider adding that.
>
> Basically STONCStyleCommentsSkipStream>>#nextNonCommentChar should know when it is inside a string and then ignore comments. But that means, in the case of STON/JSON, handling both ' (single) and " (double) quoted strings as well as interpret \' and \". All this should be doable, but it is not that easy to do elegantly. I'll see what I can do.

> Sven
>
>> On 16 Jul 2015, at 17:35, Guillermo Polito <[hidden email]> wrote:
>>
>> Hi Sven!
>>
>> I'm finally using the new STON with comments and I found one little problem: my ston file has urls :/
>>
>> {
>> // comment ok, skipped
>> /* comment ok, skipped */
>>  "aURL": "http://ouch.it.will.be.skipped/and/break?my=ston"
>> }
>>
>> That is, doing:
>>
>> STON fromStreamWithComments: '{
>> // comment ok, skipped
>> /* comment ok, skipped */
>>  "aURL": "http://ouch.it.will.be.skipped/and/break?my=ston"
>> }' readStream
>>
>> Will fail with an invalid input error because after skipping comments my STON stream remains like
>>
>> STON fromStreamWithComments: '{
>>  "aURL": "http:
>> }
>>
>> Any thoughts?
>> Guille
>>
>> El dom., 14 de jun. de 2015 a la(s) 3:06 a. m., stepharo <[hidden email]> escribió:
>>
>>
>> Le 10/6/15 20:39, Sven Van Caekenberghe a écrit :
>>>> On 10 Jun 2015, at 15:13, stepharo <[hidden email]> wrote:
>>>>
>>>> This is in the entreprise book. I added a subsection on comments.
>>>> May be I committed but forgot to push.
>>>> Now I did it.
>>> And now I see it.
>>> I made a small change.
>> super!
>>
>>>
>>>> Stef
>>>>
>>>>
>>>> Le 9/6/15 21:50, Sven Van Caekenberghe a écrit :
>>>>> Where exactly can I find what you changed ?
>>>>> I don't see any commit of you on github ...
>>>>>
>>>>>> On 09 Jun 2015, at 20:10, stepharo <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Le 9/6/15 20:06, stepharo a écrit :
>>>>>>> I will update the STON chapter :)
>>>>>> Done :)
>>>>>>
>>>>>>> Le 8/6/15 23:39, Sven Van Caekenberghe a écrit :
>>>>>>>> I would do it like this: use a wrapping stream to skip comments.
>>>>>>>>
>>>>>>>> ===
>>>>>>>> Name: STON-Core-SvenVanCaekenberghe.59
>>>>>>>> Author: SvenVanCaekenberghe
>>>>>>>> Time: 8 June 2015, 11:34:32.886167 pm
>>>>>>>> UUID: 7ab942b5-9d1a-44c3-9d58-da29feb72380
>>>>>>>> Ancestors: STON-Core-SvenVanCaekenberghe.58
>>>>>>>>
>>>>>>>> Add experimental STONCStyleCommentsSkipStream and tests
>>>>>>>> ===
>>>>>>>> Name: STON-Tests-SvenVanCaekenberghe.54
>>>>>>>> Author: SvenVanCaekenberghe
>>>>>>>> Time: 8 June 2015, 11:34:52.918771 pm
>>>>>>>> UUID: 3dcb2c9c-6efe-4d62-bf36-3bbe91f09d08
>>>>>>>> Ancestors: STON-Tests-SvenVanCaekenberghe.53
>>>>>>>>
>>>>>>>> Add experimental STONCStyleCommentsSkipStream and tests
>>>>>>>> ===
>>>>>>>>
>>>>>>>> Then you can do as follows:
>>>>>>>>
>>>>>>>> STON fromStream: (STONCStyleCommentsSkipStream on: 'Point[/* this is X*/ 1, /* this is Y*/ 2] // Nice huh ?' readStream).
>>>>>>>>
>>>>>>>>    => 1@2
>>>>>>>>
>>>>>>>> STON fromStream: (STONCStyleCommentsSkipStream on: '// Here is how you create a point:
>>>>>>>> Point[
>>>>>>>>    // this is X
>>>>>>>>    1,
>>>>>>>>    // this is Y
>>>>>>>>    2 ]
>>>>>>>> // Nice huh ?' readStream).
>>>>>>>>
>>>>>>>>    => 1@2
>>>>>>>>
>>>>>>>> This helper class would be useable in other contexts too (like NeoJSON).
>>>>>>>>
>>>>>>>> The advantage is that it does not change the STON (or JSON) syntax itself, it just add some functionality for those who want it.
>>>>>>>>
>>>>>>>> What do you think ?
>>>>>>>>
>>>>>>>> Sven
>>>>>>>>
>>>>>>>> PS: this probably needs some more testing
>>>>>>>>
>>>>>>>>> On 08 Jun 2015, at 17:16, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>>> On 08 Jun 2015, at 17:03, Damien Cassou <[hidden email]> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sven Van Caekenberghe <[hidden email]> writes:
>>>>>>>>>>
>>>>>>>>>>> I am not so sure we should add that.
>>>>>>>>>>>
>>>>>>>>>>> The JSON spec explicitly does not allow comments because of fear of abuse (that the comments would be used to add semantic meaning outside the spec).
>>>>>>>>>> really? That's surprising. Comments in configuration files are very
>>>>>>>>>> important. Even more when the configuration files are templates for new
>>>>>>>>>> users.
>>>>>>>>> I know some people are using STON in ways that I did not originally anticipate, that is of course cool, but writing configuration files was not the original goal. STON is a version of JSON that is closer to our object model.
>>>>>>>>>
>>>>>>>>> Like I said, I will think about it.
>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Damien Cassou
>>>>>>>>>> http://damiencassou.seasidehosting.st
>>>>>>>>>>
>>>>>>>>>> "Success is the ability to go from one failure to another without
>>>>>>>>>> losing enthusiasm." --Winston Churchill
>>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>