Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Hannes Hirzel
Hello Juan and Sven

About the Cuis port of NeoCSV.

I think for this failing test

(1 out of 50 here
https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Tests.pck.st#L635
)

we have to involve Sven van Caekenberghe, the author of the package in
the discussion.

The NeoCSV package has its own number parser called  NeoNumberParser
to make it independent. And as you write that parser relies on $b
digitValue = 11 by the underlying Smalltalk core.

So the answer is yes, if Cuis has $b digitValue = 11, the tests Sven
requires will work unchanged.

However maybe he is willing not to require that parsing lower case
letters a..f in hexadecimal is required, as you, Juan, write that this
leads to ambiguity.

So we have three options

1) Sven removes

self
assert:
(NeoNumberParser
parse: '7b'
base: 16)
equals: 123.

from the test to facilitate future ports (there will be soon another
one as he has updated the package last week end.


2) Cuis adopts
        $b digitValue = 11


3) When porting I remove the requirement self
assert:
(NeoNumberParser
parse: '7b'
base: 16)
equals: 123.

from the test.


I tend to prefer option 1 or 2.

What do you think?

Kind regards

Hannes


cc: to Sven

On 5/13/15, Juan Vuletich <[hidden email]> wrote:

> Cool Hannes!
>
> I see that #testHexadecimalIntegers is failing. Does changing
> #digitValue so that $b digitValue = 11 fix this? If so, do you think
> that changing this behavior is risky for the regular Cuis parser? Or
> maybe an alternative is to change senders, and add a new method
> #caseInsensitiveDigitValue or similar?
>
> Thanks,
> Juan Vuletich
>
> On 5/9/2015 5:26 PM, H. Hirzel wrote:
>> Hello
>>
>> I have ported the NeoCSV package from Pharo 4.0
>>
>> Neo-CSV-Core (SvenVanCaekenberghe.21)
>>
>> Neo-CSV-Tests (SvenVanCaekenberghe.18)
>>
>>
>> It is available here
>>
>>      https://github.com/hhzl/Cuis-NeoCSV
>>
>> For it to work it needs the attached changeset for Cuis core.
>> Discussion of the addition in another thread.
>>
>>
>> Regards
>>
>> Hannes Hirzel
>
>

_______________________________________________
Cuis mailing list
[hidden email]
http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
Reply | Threaded
Open this post in threaded view
|

Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Hannes Hirzel
P.S. The description of the NeoNumberParser

https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Core.pck.st#L69

On 5/13/15, H. Hirzel <[hidden email]> wrote:

> Hello Juan and Sven
>
> About the Cuis port of NeoCSV.
>
> I think for this failing test
>
> (1 out of 50 here
> https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Tests.pck.st#L635
> )
>
> we have to involve Sven van Caekenberghe, the author of the package in
> the discussion.
>
> The NeoCSV package has its own number parser called  NeoNumberParser
> to make it independent. And as you write that parser relies on $b
> digitValue = 11 by the underlying Smalltalk core.
>
> So the answer is yes, if Cuis has $b digitValue = 11, the tests Sven
> requires will work unchanged.
>
> However maybe he is willing not to require that parsing lower case
> letters a..f in hexadecimal is required, as you, Juan, write that this
> leads to ambiguity.
>
> So we have three options
>
> 1) Sven removes
>
> self
> assert:
> (NeoNumberParser
> parse: '7b'
> base: 16)
> equals: 123.
>
> from the test to facilitate future ports (there will be soon another
> one as he has updated the package last week end.
>
>
> 2) Cuis adopts
>         $b digitValue = 11
>
>
> 3) When porting I remove the requirement self
> assert:
> (NeoNumberParser
> parse: '7b'
> base: 16)
> equals: 123.
>
> from the test.
>
>
> I tend to prefer option 1 or 2.
>
> What do you think?
>
> Kind regards
>
> Hannes
>
>
> cc: to Sven
>
> On 5/13/15, Juan Vuletich <[hidden email]> wrote:
>> Cool Hannes!
>>
>> I see that #testHexadecimalIntegers is failing. Does changing
>> #digitValue so that $b digitValue = 11 fix this? If so, do you think
>> that changing this behavior is risky for the regular Cuis parser? Or
>> maybe an alternative is to change senders, and add a new method
>> #caseInsensitiveDigitValue or similar?
>>
>> Thanks,
>> Juan Vuletich
>>
>> On 5/9/2015 5:26 PM, H. Hirzel wrote:
>>> Hello
>>>
>>> I have ported the NeoCSV package from Pharo 4.0
>>>
>>> Neo-CSV-Core (SvenVanCaekenberghe.21)
>>>
>>> Neo-CSV-Tests (SvenVanCaekenberghe.18)
>>>
>>>
>>> It is available here
>>>
>>>      https://github.com/hhzl/Cuis-NeoCSV
>>>
>>> For it to work it needs the attached changeset for Cuis core.
>>> Discussion of the addition in another thread.
>>>
>>>
>>> Regards
>>>
>>> Hannes Hirzel
>>
>>
>

_______________________________________________
Cuis mailing list
[hidden email]
http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
Reply | Threaded
Open this post in threaded view
|

Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Hannes Hirzel
In reply to this post by Hannes Hirzel
Hello Sven

Yes, that would make it work. But do we want that? A test suite that
senses the environment where it is run in and lowers the barriers
accordingly?

Then I rather prefer a failing test and I change the test on the Cuis
side and document that in the porting log.

Make it explicit that people see what was changed.

In this case the impact is minimal, I think. People for whom the lower
case hexadecimal numbers are an issue can do the fix themselves.

Or we change the number parser in NeoCSV to do an #asUpperCase before
asking for the digitValue?

The goal is that repeated porting exercises are smooth. Ideally only
minimal changes or none at all.

Hannes


BTW later on I'd like to do as well a port to Amber.


On 5/13/15, Sven Van Caekenberghe <[hidden email]> wrote:

> Hi Hannes,
>
> It seems indeed that Cuis consistently only handles uppercase hex, not just
> in Character, but also in ByteArray>>#readHexFrom: and #hex, while Pharo
> allows both.
>
> Maybe we could relax the test by skipping the lowercase version when $a
> digitValue ~= 10 ?
>
> Sven
>
>> On 13 May 2015, at 21:52, H. Hirzel <[hidden email]> wrote:
>>
>> Hello Juan and Sven
>>
>> About the Cuis port of NeoCSV.
>>
>> I think for this failing test
>>
>> (1 out of 50 here
>> https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Tests.pck.st#L635
>> )
>>
>> we have to involve Sven van Caekenberghe, the author of the package in
>> the discussion.
>>
>> The NeoCSV package has its own number parser called  NeoNumberParser
>> to make it independent. And as you write that parser relies on $b
>> digitValue = 11 by the underlying Smalltalk core.
>>
>> So the answer is yes, if Cuis has $b digitValue = 11, the tests Sven
>> requires will work unchanged.
>>
>> However maybe he is willing not to require that parsing lower case
>> letters a..f in hexadecimal is required, as you, Juan, write that this
>> leads to ambiguity.
>>
>> So we have three options
>>
>> 1) Sven removes
>>
>> self
>> assert:
>> (NeoNumberParser
>> parse: '7b'
>> base: 16)
>> equals: 123.
>>
>> from the test to facilitate future ports (there will be soon another
>> one as he has updated the package last week end.
>>
>>
>> 2) Cuis adopts
>>        $b digitValue = 11
>>
>>
>> 3) When porting I remove the requirement self
>> assert:
>> (NeoNumberParser
>> parse: '7b'
>> base: 16)
>> equals: 123.
>>
>> from the test.
>>
>>
>> I tend to prefer option 1 or 2.
>>
>> What do you think?
>>
>> Kind regards
>>
>> Hannes
>>
>>
>> cc: to Sven
>>
>> On 5/13/15, Juan Vuletich <[hidden email]> wrote:
>>> Cool Hannes!
>>>
>>> I see that #testHexadecimalIntegers is failing. Does changing
>>> #digitValue so that $b digitValue = 11 fix this? If so, do you think
>>> that changing this behavior is risky for the regular Cuis parser? Or
>>> maybe an alternative is to change senders, and add a new method
>>> #caseInsensitiveDigitValue or similar?
>>>
>>> Thanks,
>>> Juan Vuletich
>>>
>>> On 5/9/2015 5:26 PM, H. Hirzel wrote:
>>>> Hello
>>>>
>>>> I have ported the NeoCSV package from Pharo 4.0
>>>>
>>>> Neo-CSV-Core (SvenVanCaekenberghe.21)
>>>>
>>>> Neo-CSV-Tests (SvenVanCaekenberghe.18)
>>>>
>>>>
>>>> It is available here
>>>>
>>>>     https://github.com/hhzl/Cuis-NeoCSV
>>>>
>>>> For it to work it needs the attached changeset for Cuis core.
>>>> Discussion of the addition in another thread.
>>>>
>>>>
>>>> Regards
>>>>
>>>> Hannes Hirzel
>>>
>>>
>
>

_______________________________________________
Cuis mailing list
[hidden email]
http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
Reply | Threaded
Open this post in threaded view
|

Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Juan Vuletich-4
In reply to this post by Hannes Hirzel
Hi Folks,

On 5/13/2015 5:08 PM, H. Hirzel wrote:
> P.S. The description of the NeoNumberParser
>
> https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Core.pck.st#L69
>

According to that syntax definition, 'e-', 'e+', 'E-', 'E+' are ok, but
'e' and 'E' could mean both exponent and digit.

For instance '1e3' could mean 1000 or it could mean 483 if in base 16.
Same goes for '1E3'. Cuis (like Smalltalk-80) supports uppercase for
digits and lowercase for exp.

I tried to find a spec  for numbers in CSV, but googling a bit I
couldn't find any.

If the ambiguity can be solved someway in NeoCSV, I'd like the
NeoNumberParser to handle it, without affecting the Cuis defaults.
Unless the solution to the ambiguity is so compelling that I can't
resist using it in Cuis too.

Thanks,
Juan Vuletich

> On 5/13/15, H. Hirzel<[hidden email]>  wrote:
>> Hello Juan and Sven
>>
>> About the Cuis port of NeoCSV.
>>
>> I think for this failing test
>>
>> (1 out of 50 here
>> https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Tests.pck.st#L635
>> )
>>
>> we have to involve Sven van Caekenberghe, the author of the package in
>> the discussion.
>>
>> The NeoCSV package has its own number parser called  NeoNumberParser
>> to make it independent. And as you write that parser relies on $b
>> digitValue = 11 by the underlying Smalltalk core.
>>
>> So the answer is yes, if Cuis has $b digitValue = 11, the tests Sven
>> requires will work unchanged.
>>
>> However maybe he is willing not to require that parsing lower case
>> letters a..f in hexadecimal is required, as you, Juan, write that this
>> leads to ambiguity.
>>
>> So we have three options
>>
>> 1) Sven removes
>>
>> self
>> assert:
>> (NeoNumberParser
>> parse: '7b'
>> base: 16)
>> equals: 123.
>>
>> from the test to facilitate future ports (there will be soon another
>> one as he has updated the package last week end.
>>
>>
>> 2) Cuis adopts
>>          $b digitValue = 11
>>
>>
>> 3) When porting I remove the requirement self
>> assert:
>> (NeoNumberParser
>> parse: '7b'
>> base: 16)
>> equals: 123.
>>
>> from the test.
>>
>>
>> I tend to prefer option 1 or 2.
>>
>> What do you think?
>>
>> Kind regards
>>
>> Hannes
>>
>>
>> cc: to Sven
>>
>> On 5/13/15, Juan Vuletich<[hidden email]>  wrote:
>>> Cool Hannes!
>>>
>>> I see that #testHexadecimalIntegers is failing. Does changing
>>> #digitValue so that $b digitValue = 11 fix this? If so, do you think
>>> that changing this behavior is risky for the regular Cuis parser? Or
>>> maybe an alternative is to change senders, and add a new method
>>> #caseInsensitiveDigitValue or similar?
>>>
>>> Thanks,
>>> Juan Vuletich
>>>
>>> On 5/9/2015 5:26 PM, H. Hirzel wrote:
>>>> Hello
>>>>
>>>> I have ported the NeoCSV package from Pharo 4.0
>>>>
>>>> Neo-CSV-Core (SvenVanCaekenberghe.21)
>>>>
>>>> Neo-CSV-Tests (SvenVanCaekenberghe.18)
>>>>
>>>>
>>>> It is available here
>>>>
>>>>       https://github.com/hhzl/Cuis-NeoCSV
>>>>
>>>> For it to work it needs the attached changeset for Cuis core.
>>>> Discussion of the addition in another thread.
>>>>
>>>>
>>>> Regards
>>>>
>>>> Hannes Hirzel
>>>


_______________________________________________
Cuis mailing list
[hidden email]
http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
Reply | Threaded
Open this post in threaded view
|

Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Phil B
On Wed, 2015-05-13 at 21:49 -0300, Juan Vuletich wrote:

> Hi Folks,
>
> On 5/13/2015 5:08 PM, H. Hirzel wrote:
> > P.S. The description of the NeoNumberParser
> >
> > https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Core.pck.st#L69
> >
>
> According to that syntax definition, 'e-', 'e+', 'E-', 'E+' are ok, but
> 'e' and 'E' could mean both exponent and digit.
>
> For instance '1e3' could mean 1000 or it could mean 483 if in base 16.
> Same goes for '1E3'. Cuis (like Smalltalk-80) supports uppercase for
> digits and lowercase for exp.
>
> I tried to find a spec  for numbers in CSV, but googling a bit I
> couldn't find any.
>

Heh... good luck with that.  CSV is a horribly ambiguous 'standard' that
pretty much means 'text delimited by something' in the real world.
Granted, there's a spec from 2005 but that doesn't do much good when
most of the critical software that emits it originated before 1990 and
isn't likely to change to comply with 'The Standard'. It's a losing
battle making any accommodations in Cuis for it.

> If the ambiguity can be solved someway in NeoCSV, I'd like the
> NeoNumberParser to handle it, without affecting the Cuis defaults.
> Unless the solution to the ambiguity is so compelling that I can't
> resist using it in Cuis too.
>

The package really should allow for the flexibility to handle the
permutations that will be thrown at it or it is likely the wrong CSV
parser to be working with.

> Thanks,
> Juan Vuletich
>
> > On 5/13/15, H. Hirzel<[hidden email]>  wrote:
> >> Hello Juan and Sven
> >>
> >> About the Cuis port of NeoCSV.
> >>
> >> I think for this failing test
> >>
> >> (1 out of 50 here
> >> https://github.com/hhzl/Cuis-NeoCSV/blob/master/Neo-CSV-Tests.pck.st#L635
> >> )
> >>
> >> we have to involve Sven van Caekenberghe, the author of the package in
> >> the discussion.
> >>
> >> The NeoCSV package has its own number parser called  NeoNumberParser
> >> to make it independent. And as you write that parser relies on $b
> >> digitValue = 11 by the underlying Smalltalk core.
> >>
> >> So the answer is yes, if Cuis has $b digitValue = 11, the tests Sven
> >> requires will work unchanged.
> >>
> >> However maybe he is willing not to require that parsing lower case
> >> letters a..f in hexadecimal is required, as you, Juan, write that this
> >> leads to ambiguity.
> >>
> >> So we have three options
> >>
> >> 1) Sven removes
> >>
> >> self
> >> assert:
> >> (NeoNumberParser
> >> parse: '7b'
> >> base: 16)
> >> equals: 123.
> >>
> >> from the test to facilitate future ports (there will be soon another
> >> one as he has updated the package last week end.
> >>
> >>
> >> 2) Cuis adopts
> >>          $b digitValue = 11
> >>
> >>
> >> 3) When porting I remove the requirement self
> >> assert:
> >> (NeoNumberParser
> >> parse: '7b'
> >> base: 16)
> >> equals: 123.
> >>
> >> from the test.
> >>
> >>
> >> I tend to prefer option 1 or 2.
> >>
> >> What do you think?
> >>
> >> Kind regards
> >>
> >> Hannes
> >>
> >>
> >> cc: to Sven
> >>
> >> On 5/13/15, Juan Vuletich<[hidden email]>  wrote:
> >>> Cool Hannes!
> >>>
> >>> I see that #testHexadecimalIntegers is failing. Does changing
> >>> #digitValue so that $b digitValue = 11 fix this? If so, do you think
> >>> that changing this behavior is risky for the regular Cuis parser? Or
> >>> maybe an alternative is to change senders, and add a new method
> >>> #caseInsensitiveDigitValue or similar?
> >>>
> >>> Thanks,
> >>> Juan Vuletich
> >>>
> >>> On 5/9/2015 5:26 PM, H. Hirzel wrote:
> >>>> Hello
> >>>>
> >>>> I have ported the NeoCSV package from Pharo 4.0
> >>>>
> >>>> Neo-CSV-Core (SvenVanCaekenberghe.21)
> >>>>
> >>>> Neo-CSV-Tests (SvenVanCaekenberghe.18)
> >>>>
> >>>>
> >>>> It is available here
> >>>>
> >>>>       https://github.com/hhzl/Cuis-NeoCSV
> >>>>
> >>>> For it to work it needs the attached changeset for Cuis core.
> >>>> Discussion of the addition in another thread.
> >>>>
> >>>>
> >>>> Regards
> >>>>
> >>>> Hannes Hirzel
> >>>
>
>
> _______________________________________________
> Cuis mailing list
> [hidden email]
> http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org



_______________________________________________
Cuis mailing list
[hidden email]
http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org
Reply | Threaded
Open this post in threaded view
|

Re: CSV Parser in Cuis available // issue about parsing hexadecimal number with lower case letters

Juan Vuletich-4
In reply to this post by Juan Vuletich-4
Hi Sven,

On 5/14/2015 8:58 AM, Sven Van Caekenberghe wrote:

>>
>> ...
>> According to that syntax definition, 'e-', 'e+', 'E-', 'E+' are ok, but 'e' and 'E' could mean both exponent and digit.
>>
>> For instance '1e3' could mean 1000 or it could mean 483 if in base 16. Same goes for '1E3'. Cuis (like Smalltalk-80) supports uppercase for digits and lowercase for exp.
> That is an interesting point indeed !
>
> The ambition of NeoNumberParser is not to answer all these questions, its purpose is to change the underlying stream requirements for day to day parsing of numbers.
>
> I make the following commit:
>
> ===
> Name: Neo-CSV-Tests-SvenVanCaekenberghe.19
> Author: SvenVanCaekenberghe
> Time: 14 May 2015, 1:30:58.136362 pm
> UUID: 6d1479ef-a779-421d-9b00-fbd2873b5450
> Ancestors: Neo-CSV-Tests-SvenVanCaekenberghe.18
>
> Change NeoNumberParserTests>>#testHexadecimalIntegers to not require lowercase hex digits to work on platforms where Character>>#digitValue only handles uppercase (like Cuis)
> ===
>

Thank you very much Sven!

Cheers,
Juan Vuletich

_______________________________________________
Cuis mailing list
[hidden email]
http://jvuletich.org/mailman/listinfo/cuis_jvuletich.org