Smalltalk › Pharo › Pharo Smalltalk Users

iterating through an ascii file by line

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

9 messages Options

Paul DeBruicker

iterating through an ascii file by line

Hi -

Is there an easy/standard way to open an ascii file and iterate through
it line by line?

I want to put several files contents into their own OrderedCollections
and then do awful things to them. The files are ~500MB each

Thanks

Paul

Camillo Bruni-3

Re: iterating through an ascii file by line

IMO there are only default methods for iterating over the lines of string

String >> #linesDo:

but most probably this is now what you want :P, so #nextDelimited: is your friend...

On 2012-07-31, at 18:25, Paul DeBruicker <[hidden email]> wrote:

> Hi -
>
> Is there an easy/standard way to open an ascii file and iterate through it line by line?
>
>
> I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each
>
>
> Thanks
>
> Paul
>

Paul DeBruicker

Re: iterating through an ascii file by line

I just found this:

http://www.squeaksource.com/SimpleTextParser.html

which does what I want. Thanks and sorry for the noise.

On 07/31/2012 09:30 AM, Camillo Bruni wrote:

> IMO there are only default methods for iterating over the lines of string
>
> String >> #linesDo:
>
> but most probably this is now what you want :P, so #nextDelimited: is your friend...
>
>
> On 2012-07-31, at 18:25, Paul DeBruicker <[hidden email]> wrote:
>
>> Hi -
>>
>> Is there an easy/standard way to open an ascii file and iterate through it line by line?
>>
>>
>> I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each
>>
>>
>> Thanks
>>
>> Paul
>>
>
>

Stéphane Ducasse

Re: iterating through an ascii file by line

On Jul 31, 2012, at 6:35 PM, Paul DeBruicker wrote:

> I just found this:
>
> http://www.squeaksource.com/SimpleTextParser.html
>
> which does what I want. Thanks and sorry for the noise.

this is not noise :)
I like reading such questions and their answers. This is global learning :)
Stef

>
>
>
>
>
> On 07/31/2012 09:30 AM, Camillo Bruni wrote:
>> IMO there are only default methods for iterating over the lines of string
>>
>> String >> #linesDo:
>>
>> but most probably this is now what you want :P, so #nextDelimited: is your friend...
>>
>>
>> On 2012-07-31, at 18:25, Paul DeBruicker <[hidden email]> wrote:
>>
>>> Hi -
>>>
>>> Is there an easy/standard way to open an ascii file and iterate through it line by line?
>>>
>>>
>>> I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each
>>>
>>>
>>> Thanks
>>>
>>> Paul
>>>
>>
>>
>
>

Stéphane Ducasse

Re: iterating through an ascii file by line

In reply to this post by Paul DeBruicker

It would be good to have configuration for this package.

On Jul 31, 2012, at 6:35 PM, Paul DeBruicker wrote:

> I just found this:
>
> http://www.squeaksource.com/SimpleTextParser.html
>
> which does what I want. Thanks and sorry for the noise.
>
>
>
>
>
> On 07/31/2012 09:30 AM, Camillo Bruni wrote:
>> IMO there are only default methods for iterating over the lines of string
>>
>> String >> #linesDo:
>>
>> but most probably this is now what you want :P, so #nextDelimited: is your friend...
>>
>>
>> On 2012-07-31, at 18:25, Paul DeBruicker <[hidden email]> wrote:
>>
>>> Hi -
>>>
>>> Is there an easy/standard way to open an ascii file and iterate through it line by line?
>>>
>>>
>>> I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each
>>>
>>>
>>> Thanks
>>>
>>> Paul
>>>
>>
>>
>
>

Sven Van Caekenberghe

Re: iterating through an ascii file by line

In reply to this post by Paul DeBruicker

Paul,

On 31 Jul 2012, at 18:25, Paul DeBruicker <[hidden email]> wrote:

> Is there an easy/standard way to open an ascii file and iterate through it line by line?
>
>
> I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each

This uses code that is already in Pharo:

'/tmp/foo.txt' asFileReference readStreamDo: [ :readStream |
| lineReader |
lineReader := ZnLineReader on: readStream.
Array streamContents: [ :output |
| line |
[ (line := lineReader nextLine) isEmpty ]
whileFalse: [ output nextPut: line ] ] ]

Works with any line end convention. Check the class comment of ZnLineReader.

Sven

hernanmd

Re: iterating through an ascii file by line

2012/8/1 Sven Van Caekenberghe <[hidden email]>

Paul,

On 31 Jul 2012, at 18:25, Paul DeBruicker <[hidden email]> wrote:

> Is there an easy/standard way to open an ascii file and iterate through it line by line?
>
>
> I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each

This uses code that is already in Pharo:

'/tmp/foo.txt' asFileReference readStreamDo: [ :readStream |
| lineReader |
lineReader := ZnLineReader on: readStream.
Array streamContents: [ :output |
| line |
[ (line := lineReader nextLine) isEmpty ]
whileFalse: [ output nextPut: line ] ] ]

Works with any line end convention. Check the class comment of ZnLineReader.

No, it doesn't work in latest Pharo 1.4 (14557)
MessageNotUnderstood: FileSystemReadStream>>isBinary

Cheers,

Hernán

Sven Van Caekenberghe

Re: iterating through an ascii file by line

Hernán,

On 01 Aug 2012, at 20:50, Hernán Morales Durand <[hidden email]> wrote:

>
> 2012/8/1 Sven Van Caekenberghe <[hidden email]>
> Paul,
>
> On 31 Jul 2012, at 18:25, Paul DeBruicker <[hidden email]> wrote:
>
> > Is there an easy/standard way to open an ascii file and iterate through it line by line?
> >
> >
> > I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each
>
> This uses code that is already in Pharo:
>
> '/tmp/foo.txt' asFileReference readStreamDo: [ :readStream |
> | lineReader |
> lineReader := ZnLineReader on: readStream.
> Array streamContents: [ :output |
> | line |
> [ (line := lineReader nextLine) isEmpty ]
> whileFalse: [ output nextPut: line ] ] ]
>
> Works with any line end convention. Check the class comment of ZnLineReader.
>
>
> No, it doesn't work in latest Pharo 1.4 (14557)
> MessageNotUnderstood: FileSystemReadStream>>isBinary
>
> Cheers,
>
> Hernán

I did try the code snippet on Pharo 2.0 before posting it, it worked.

I just tried the following on Pharo 1.4, and it works as well.

FileStream fileNamed: '/tmp/foo.txt' do: [ :readStream |
| lineReader |
lineReader := ZnLineReader on: readStream.
Array streamContents: [ :output |
| line |
[ (line := lineReader nextLine) isEmpty ]
whileFalse: [ output nextPut: line ] ] ]

Maybe you tried using some FileSystem port on 1.4 that is not as complete as the code that is by default in 2.0 ?
But ZnLineReader is independent of that, it has been part of Zn since years, in Pharo 1.3 and up.

Sven

hernanmd

Re: iterating through an ascii file by line

Hi Sven,
I didn't tried in Pharo 2.0 because is not stable. In Pharo 1.4 this works

FileStream fileNamed: 'd:\Test\filter.csv' do: [ :readStream |
        | lineReader |
        lineReader := ZnLineReader on: readStream.
        Array streamContents: [ :output |
                | line |
                [ (line := lineReader nextLine) isEmpty ]
                        whileFalse: [ output nextPut: line ] ] ].

but this script fails:

'd:\Test\filter.csv' asFileReference readStreamDo: [ :readStream |
         | lineReader |
         lineReader := ZnLineReader on: readStream.
         Array streamContents: [ :output |
                 | line |
                 [ (line := lineReader nextLine) isEmpty ]
                         whileFalse: [ output nextPut: line ] ] ].

probably a problem with #asFileReference
Cheers,

Hernán

2012/8/2 Sven Van Caekenberghe <[hidden email]>

Hernán,

On 01 Aug 2012, at 20:50, Hernán Morales Durand <[hidden email]> wrote:

>
> 2012/8/1 Sven Van Caekenberghe <[hidden email]>
> Paul,
>
> On 31 Jul 2012, at 18:25, Paul DeBruicker <[hidden email]> wrote:
>
> > Is there an easy/standard way to open an ascii file and iterate through it line by line?
> >
> >
> > I want to put several files contents into their own OrderedCollections and then do awful things to them. The files are ~500MB each
>
> This uses code that is already in Pharo:
>
> '/tmp/foo.txt' asFileReference readStreamDo: [ :readStream |
> | lineReader |
> lineReader := ZnLineReader on: readStream.
> Array streamContents: [ :output |
> | line |
> [ (line := lineReader nextLine) isEmpty ]
> whileFalse: [ output nextPut: line ] ] ]
>
> Works with any line end convention. Check the class comment of ZnLineReader.
>
>
> No, it doesn't work in latest Pharo 1.4 (14557)
> MessageNotUnderstood: FileSystemReadStream>>isBinary
>
> Cheers,
>
> Hernán

I did try the code snippet on Pharo 2.0 before posting it, it worked.

I just tried the following on Pharo 1.4, and it works as well.

FileStream fileNamed: '/tmp/foo.txt' do: [ :readStream |

| lineReader |
lineReader := ZnLineReader on: readStream.
Array streamContents: [ :output |
| line |
[ (line := lineReader nextLine) isEmpty ]
whileFalse: [ output nextPut: line ] ] ]

Maybe you tried using some FileSystem port on 1.4 that is not as complete as the code that is by default in 2.0 ?
But ZnLineReader is independent of that, it has been part of Zn since years, in Pharo 1.3 and up.

Sven