Smalltalk › Instantiations

Importing CSV files in VAST v6.x ?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

8 messages Options

peter.ode

Importing CSV files in VAST v6.x ?

I've used NeoCSV in Pharo and see that NeoCSV is available for VA Smalltalk version 8.x — but that version of NeoCSV does not work in VAST 6.x.

I thought I'd ask here, before implementing my own simple CSV importer.

Does anyone have some existing code for CSV importing? Thx.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Mariano Martinez Peck-2

Re: Importing CSV files in VAST v6.x ?

Hi Peter,

I know it won't answer directly your question, but I just thought it was a good opportunity to say that in the same way we added Regex, LDAP, NeoJSON, etc, in 9.1, having CSV parsing as a new official feature is in the roadmap for VA 9.2. We will likely go with NeoCSV, probably starting from the Goodies version and update it to latest Pharo code. But of course, if the NeoCSV from the Goodies didn't work out in 6.x we cannot guarantee the one of 9.2 would.

BTW, I don't know what didn't work from NeoCSV exactly in 6.x, but maybe not too hard to fix and much easier than starting from scratch.

Best regards,

On Wed, Sep 19, 2018 at 8:46 PM Peter Ode <[hidden email]> wrote:

I've used NeoCSV in Pharo and see that NeoCSV is available for VA Smalltalk version 8.x — but that version of NeoCSV does not work in VAST 6.x.

I thought I'd ask here, before implementing my own simple CSV importer.

Does anyone have some existing code for CSV importing? Thx.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Mariano Martinez Peck
Software Engineer, Instantiations Inc.
[hidden email]

Noschvie

Re: Importing CSV files in VAST v6.x ?

Hi Peter and Marian

had a look into the code and found these issues:

.) NeoNumberParser>>#parseNumberInteger
stream peek digitValue -> stream peek asUppercase digitValue
stream next digitValue -> stream next asUppercase digitValue

.) NeoCSVWriter>>#fieldWriter:
"symbol capitalized" : #capitalized not available and can be replaced by wbCapitalized
but maybe #capitalized can be added as part of the framework by Instantiations

.) NeoCSVWriter>>#addObjectField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

.) NeoCSVWriter>>#addRawField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

Norbert

Mariano Martinez Peck-2

Re: Importing CSV files in VAST v6.x ?

On Mon, Sep 24, 2018 at 5:56 AM Norbert Schlemmer <[hidden email]> wrote:

Hi Peter and Marian
had a look into the code and found these issues:

Hi Norbert,

Thanks for sharing your fixes! Let me answer inline.

.) NeoNumberParser>>#parseNumberInteger
stream peek digitValue -> stream peek asUppercase digitValue
stream next digitValue -> stream next asUppercase digitValue

Good catch. I will discuss internally with the team why lower cases gives -1 in case of VA and whether we would be able to change it or not. At the very least, adding those #asUppercase does make sense.

VW answers 10.

Pharo answers 10.

GemStone answers 10.

Dolphin answers -1.

VA answers -1.

.) NeoCSVWriter>>#fieldWriter:
"symbol capitalized" : #capitalized not available and can be replaced by wbCapitalized
but maybe #capitalized can be added as part of the framework by Instantiations

Another good catch. Currently, Grease also implements such a method. We will see of including it on the base image in 9.2. If not, at the very least, it would be as extension method of NeoCSV.

.) NeoCSVWriter>>#addObjectField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

.) NeoCSVWriter>>#addRawField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

How did you catch these 2? You were just reading the code and noticed the comments of the method or you really hit the problem at runtime?

BTW, we already have some frameworks that provide this method as extension method:

Symbol >> value: anObject

^anObject perform: self

We discussed this during 9.1 and we thought it might be a bit too risky to add it (thinking about backward compatibility etc). But we can revise it again for 9.2.

Clearly, finding all those little broken places like those 2 isn't an easy task either.

Thanks for your detailed report Norbert.

br

Norbert

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Mariano Martinez Peck
Software Engineer, Instantiations Inc.
[hidden email]

Noschvie

Re: Importing CSV files in VAST v6.x ?

found both running the unit tests.

Don't want to use " Symbol >> value: anObject "

Norbert

Am Montag, 24. September 2018 14:21:29 UTC+2 schrieb marianopeck:

.) NeoCSVWriter>>#addObjectField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

.) NeoCSVWriter>>#addRawField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

How did you catch these 2? You were just reading the code and noticed the comments of the method or you really hit the problem at runtime?

BTW, we already have some frameworks that provide this method as extension method:

Symbol >> value: anObject

^anObject perform: self

We discussed this during 9.1 and we thought it might be a bit too risky to add it (thinking about backward compatibility etc). But we can revise it again for 9.2.
Clearly, finding all those little broken places like those 2 isn't an easy task either.

Mariano Martinez Peck-2

Re: Importing CSV files in VAST v6.x ?

On Mon, Sep 24, 2018 at 9:31 AM Norbert Schlemmer <[hidden email]> wrote:

found both running the unit tests.

Thanks Norbert. I did not notice them in my latest port because I also had Glorp loaded in that image and that brings Symbol >> #value:. Of course, we would have seen this later, but thanks for pointing to it sooner! And I agree with your changes. Thanks.

Don't want to use " Symbol >> value: anObject "

What do you mean? That you don't like the idea of having below method?

Symbol >> value: anObject

^anObject perform: self

BTW, how did you find the #parseNumberInteger ? Didn't get any test failure. Just wondering if I can improve the tests.

Norbert

Am Montag, 24. September 2018 14:21:29 UTC+2 schrieb marianopeck:

.) NeoCSVWriter>>#addObjectField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

.) NeoCSVWriter>>#addRawField:
(accessor isKindOf: Symbol) ifTrue: [ object perform: accessor. ] ifFalse: [ accessor value: object. ]

How did you catch these 2? You were just reading the code and noticed the comments of the method or you really hit the problem at runtime?

BTW, we already have some frameworks that provide this method as extension method:

Symbol >> value: anObject

^anObject perform: self

We discussed this during 9.1 and we thought it might be a bit too risky to add it (thinking about backward compatibility etc). But we can revise it again for 9.2.
Clearly, finding all those little broken places like those 2 isn't an easy task either.

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Mariano Martinez Peck
Software Engineer, Instantiations Inc.
[hidden email]

Noschvie

Re: Importing CSV files in VAST v6.x ?

Don't want to use " Symbol >> value: anObject "

What do you mean? That you don't like the idea of having below method?

Symbol >> value: anObject
^anObject perform: self

Yes
BTW, how did you find the #parseNumberInteger ? Didn't get any test failure. Just wondering if I can improve the tests.

Yes, don't like the idea of having this method.

#parseNumberInteger works for example 1B but not for 1b, I just used the unit tests

NeoNumberParserTests>>#testHexadecimalIntegers

.) NeoNumberParserTests>>#testFloats

#pairsDo: is missing

.) NeoCSVWriterTests>>#testRawFieldsDictionaries

test result is "red" too

Norbert

Mariano Martinez Peck-2

Re: Importing CSV files in VAST v6.x ?

On Tue, Sep 25, 2018 at 3:48 AM Norbert Schlemmer <[hidden email]> wrote:

Don't want to use " Symbol >> value: anObject "

What do you mean? That you don't like the idea of having below method?

Symbol >> value: anObject
^anObject perform: self

Yes
BTW, how did you find the #parseNumberInteger ? Didn't get any test failure. Just wondering if I can improve the tests.

Yes, don't like the idea of having this method.

OK, thanks for your opinion. We value customers thoughts.

#parseNumberInteger works for example 1B but not for 1b, I just used the unit tests
NeoNumberParserTests>>#testHexadecimalIntegers

OK, as I started the port from very latest NeoCSV it's original current unit test is like this (I did not change the bold part)

testHexadecimalIntegers

self assert: (NeoNumberParser parse: '7B' base: 16) equals: 123.

self assert: (NeoNumberParser parse: '-7B' base: 16) equals: -123.

self assert: (NeoNumberParser parse: '0' base: 16) equals: 0.

"On some platforms Character>>#digitValue only handles uppercase,

then NeoNumberParser cannot deal with lowercase hex characters"

$a digitValue = 10 ifFalse: [ ^ self ].

self assert: (NeoNumberParser parse: '7b' base: 16) equals: 123.

self assert: (NeoNumberParser parse: '-7b' base: 16) equals: -123

I now removed the bold part and applied your changes to #parseNumberInteger

.) NeoNumberParserTests>>#testFloats
#pairsDo: is missing

Yes, in my latest port this becomes an extension method from the NeoCSVVASTTestsApp

.) NeoCSVWriterTests>>#testRawFieldsDictionaries
test result is "red" too

This one works too on my latest port. #newFromPairs: has also become an extension method of NeoCSVVASTTestsApp

Thanks for your report!

Norbert

--
You received this message because you are subscribed to the Google Groups "VA Smalltalk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/va-smalltalk.
For more options, visit https://groups.google.com/d/optout.

Mariano Martinez Peck
Software Engineer, Instantiations Inc.
[hidden email]