Trying to import csv data, no love

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Trying to import csv data, no love

Casey Ransberger-2
Loaded the latest from 


Did...

rows := CSVParser parse: (FileStream readOnlyFileNamed: '/Users/casey/Desktop/SCDB_2010_01_justiceCentered_Vote.csv')

Got...

"Error: invalid utf8 input detected"

Relevant region of code might be...

MultiByteFileStream>>next

| char secondChar state |
char := converter nextFromStream: self.


converter is a UTF8TextConverter. Wondering if maybe I need a different text converter? Given that this is a CSV file, I'm thinking the system might be expecting UTF-8 where it's actually getting ASCII. The CSV package hasn't had any attention in a while, so that makes some sense (IIRC there was a shift to UTF-8 not very long ago, no?)

If anyone is wondering, I'm playing with an idea around dynamic data-centric web interfaces:) and need some raw data to represent, hence the CSV file.

Any ideas?
--
Casey Ransberger


Reply | Threaded
Open this post in threaded view
|

Re: Trying to import csv data, no love

Andreas.Raab
On 8/4/2010 11:25 PM, Casey Ransberger wrote:
> rows := CSVParser parse: (FileStream readOnlyFileNamed:
> '/Users/casey/Desktop/SCDB_2010_01_justiceCentered_Vote.csv')

Try changing this to:

file := FileStream readOnlyFileNamed: 'myfile.csv'.
file converter: Latin1TextConverter new.
rows := CSVParser parse: file.

Cheers,
   - Andreas

> Got...
>
> "Error: invalid utf8 input detected"
>
> Relevant region of code might be...
>
> MultiByteFileStream>>next
>
> | char secondChar state |
> char := converter nextFromStream: self.
>
>
> converter is a UTF8TextConverter. Wondering if maybe I need a different
> text converter? Given that this is a CSV file, I'm thinking the system
> might be expecting UTF-8 where it's actually getting ASCII. The CSV
> package hasn't had any attention in a while, so that makes some sense
> (IIRC there was a shift to UTF-8 not very long ago, no?)
>
> If anyone is wondering, I'm playing with an idea around dynamic
> data-centric web interfaces:) and need some raw data to represent, hence
> the CSV file.
>
> Any ideas?
> --
> Casey Ransberger
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Trying to import csv data, no love

Casey Ransberger-2
Haha! UI hangs and CPU spikes to the top; battery starts draining fast. 

"That tingling means it's working!" 

Only takes a couple of minutes, really. I suppose this is a question I'd have to ask eventually; is there a simple way to spin something like this off as a separate (green) thread, such that Morphic still responds while the job is running?

On Wed, Aug 4, 2010 at 11:42 PM, Andreas Raab <[hidden email]> wrote:
On 8/4/2010 11:25 PM, Casey Ransberger wrote:
rows := CSVParser parse: (FileStream readOnlyFileNamed:
'/Users/casey/Desktop/SCDB_2010_01_justiceCentered_Vote.csv')

Try changing this to:

file := FileStream readOnlyFileNamed: 'myfile.csv'.
file converter: Latin1TextConverter new.
rows := CSVParser parse: file.

Cheers,
 - Andreas


Got...

"Error: invalid utf8 input detected"

Relevant region of code might be...

MultiByteFileStream>>next

| char secondChar state |
char := converter nextFromStream: self.


converter is a UTF8TextConverter. Wondering if maybe I need a different
text converter? Given that this is a CSV file, I'm thinking the system
might be expecting UTF-8 where it's actually getting ASCII. The CSV
package hasn't had any attention in a while, so that makes some sense
(IIRC there was a shift to UTF-8 not very long ago, no?)

If anyone is wondering, I'm playing with an idea around dynamic
data-centric web interfaces:) and need some raw data to represent, hence
the CSV file.

Any ideas?
--
Casey Ransberger









--
Casey Ransberger


Reply | Threaded
Open this post in threaded view
|

Re: Trying to import csv data, no love

Andreas.Raab
On 8/5/2010 12:03 AM, Casey Ransberger wrote:
> Haha! UI hangs and CPU spikes to the top; battery starts draining fast.
>
> "That tingling means it's working!"
>
> Only takes a couple of minutes, really. I suppose this is a question I'd
> have to ask eventually; is there a simple way to spin something like
> this off as a separate (green) thread, such that Morphic still responds
> while the job is running?

Sure. Try this:

[rows := CSVParser parse: file.
SampledSound playSoundNamed: 'coyote'] forkAt: Processor
userBackgroundPriority.

When the coyote howls, you're done :-)

Cheers,
   - Andreas

>
> On Wed, Aug 4, 2010 at 11:42 PM, Andreas Raab <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On 8/4/2010 11:25 PM, Casey Ransberger wrote:
>
>         rows := CSVParser parse: (FileStream readOnlyFileNamed:
>         '/Users/casey/Desktop/SCDB_2010_01_justiceCentered_Vote.csv')
>
>
>     Try changing this to:
>
>     file := FileStream readOnlyFileNamed: 'myfile.csv'.
>     file converter: Latin1TextConverter new.
>     rows := CSVParser parse: file.
>
>     Cheers,
>       - Andreas
>
>
>         Got...
>
>         "Error: invalid utf8 input detected"
>
>         Relevant region of code might be...
>
>         MultiByteFileStream>>next
>
>         | char secondChar state |
>         char := converter nextFromStream: self.
>
>
>         converter is a UTF8TextConverter. Wondering if maybe I need a
>         different
>         text converter? Given that this is a CSV file, I'm thinking the
>         system
>         might be expecting UTF-8 where it's actually getting ASCII. The CSV
>         package hasn't had any attention in a while, so that makes some
>         sense
>         (IIRC there was a shift to UTF-8 not very long ago, no?)
>
>         If anyone is wondering, I'm playing with an idea around dynamic
>         data-centric web interfaces:) and need some raw data to
>         represent, hence
>         the CSV file.
>
>         Any ideas?
>         --
>         Casey Ransberger
>
>
>
>
>
>
>
>
>
> --
> Casey Ransberger
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Trying to import csv data, no love

Bert Freudenberg

On 05.08.2010, at 09:30, Andreas Raab wrote:

> On 8/5/2010 12:03 AM, Casey Ransberger wrote:
>> Haha! UI hangs and CPU spikes to the top; battery starts draining fast.
>>
>> "That tingling means it's working!"
>>
>> Only takes a couple of minutes, really. I suppose this is a question I'd
>> have to ask eventually; is there a simple way to spin something like
>> this off as a separate (green) thread, such that Morphic still responds
>> while the job is running?
>
> Sure. Try this:
>
> [rows := CSVParser parse: file.
> SampledSound playSoundNamed: 'coyote'] forkAt: Processor userBackgroundPriority.
>
> When the coyote howls, you're done :-)
>
> Cheers,
>  - Andreas

... and since you presumably want to use that list for something, you need to get it back into Morphic. To safely talk from a background thread to your Morphic UI, use this:

[rows := CSVParser parse: file.
WorldState addDeferredUIMessage: [myGUI updateList: rows].
] forkAt: Processor userBackgroundPriority.

- Bert -