Efficiently writing to a file

> Is this a reasonable way of writing to a file?  It seems to run a
> little slower than I would expect.
>
> Thanks,
>
> Ian
>
> | myFile |
> myFile := StandardFileStream fileNamed: 'c:/test.txt'
>
> Transcript show: (Time millisecondsToRun: [
>    1 to: 10000 do: [
>    :x | myFile nextPutAll: ((x asString) , String crlf)
> ]]) asString , ' millseconds' ; cr.
>
> myFile close.

>
> On Apr 8, 2007, at 21:13 , Ian Oversby wrote:
>
> > Is this a reasonable way of writing to a file?  It seems to run a
> > little slower than I would expect.
> >
> > Thanks,
> >
> > Ian
> >
> > | myFile |
> > myFile := StandardFileStream fileNamed: 'c:/test.txt'
> >
> > Transcript show: (Time millisecondsToRun: [
> >    1 to: 10000 do: [
> >           :x | myFile nextPutAll: ((x asString) , String crlf)
> >       ]]) asString , ' millseconds' ; cr.
> >
> > myFile close.
>
> It is unbuffered. Write to an in-memory stream first and then put the
> whole thing into the file.
>
> - Bert -
>
>
> _______________________________________________
> Beginners mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/beginners
>

> Okay, I tried this and it still seems a bit slow:
>
> | myFile ios |
>
> ios := ReadWriteStream on: ''.
>
> Transcript show: 'Populate Buffer: ',
> (Time millisecondsToRun: [
> 1 to: 10000 do: [
> :x | ios nextPutAll: ((x asString) , String crlf)
> ]]) asString , ' millseconds' ; cr.
>

>
> On Apr 10, 2007, at 23:22 , Ian Oversby wrote:
>
> > Okay, I tried this and it still seems a bit slow:
> >
> > | myFile ios |
> >
> > ios := ReadWriteStream on: ''.
> >
> > Transcript show: 'Populate Buffer: ',
> >       (Time millisecondsToRun: [
> >               1 to: 10000 do: [
> >                       :x | ios nextPutAll: ((x asString) , String crlf)
> >                       ]]) asString , ' millseconds' ; cr.
> >
>
> >  Is this the correct way to use the stream?
>
> Why not write
>
>         1 to: 10000 do: [:x | ios print: x; cr]
>
> - Bert -

>
> Okay, I tried this and it still seems a bit slow:
>
> | myFile ios |
>
> ios := ReadWriteStream on: ''.
>
> Transcript show: 'Populate Buffer: ',
> (Time millisecondsToRun: [
> 1 to: 10000 do: [
> :x | ios nextPutAll: ((x asString) ,
> String crlf)
> ]]) asString , ' millseconds' ; cr.
>
> Transcript show: 'Position: ', (ios position) asString ; cr.
>
> myFile := StandardFileStream fileNamed: 'c:/test.txt'.
>
> Transcript show: 'Output Buffer: ',
>                     (Time millisecondsToRun: [
> myFile nextPutAll: (ios contents)]) asString,
>      ' milliseconds' ; cr.
>
> myFile close.
>
> # --
>
> Populate Buffer is showing ~350 milliseconds to run on my
> system whereas similar code in Perl is showing around 70
> milliseconds.  Is this the correct way to use the stream?
>
> Thanks,
>
> Ian
>
> On 08/04/07, Bert Freudenberg <[hidden email]> wrote:
> >

> On 10/04/07, Bert Freudenberg <[hidden email]> wrote:
>>
>> On Apr 10, 2007, at 23:22 , Ian Oversby wrote:
>>
>> > Okay, I tried this and it still seems a bit slow:
>> >
>> > | myFile ios |
>> >
>> > ios := ReadWriteStream on: ''.
>> >
>> > Transcript show: 'Populate Buffer: ',
>> >       (Time millisecondsToRun: [
>> >               1 to: 10000 do: [
>> >                       :x | ios nextPutAll: ((x asString) ,  
>> String crlf)
>> >                       ]]) asString , ' millseconds' ; cr.
>> >
>>
>> >  Is this the correct way to use the stream?
>>
>> Why not write
>>
>>         1 to: 10000 do: [:x | ios print: x; cr]
>>
>> - Bert -
>
> I don't know the difference between print: and nextPutAll:,

> >
> > Okay, I tried this and it still seems a bit slow:
> >
> > | myFile ios |
> >
> > ios := ReadWriteStream on: ''.
> >
> > Transcript show: 'Populate Buffer: ',
> >       (Time millisecondsToRun: [
> >               1 to: 10000 do: [
> >                       :x | ios nextPutAll: ((x asString) ,
> > String crlf)
> >                       ]]) asString , ' millseconds' ; cr.
> >
> > Transcript show: 'Position: ', (ios position) asString ; cr.
> >
> > myFile := StandardFileStream fileNamed: 'c:/test.txt'.
> >
> > Transcript show: 'Output Buffer: ',
> >                     (Time millisecondsToRun: [
> >       myFile nextPutAll: (ios contents)]) asString,
> >      ' milliseconds' ; cr.
> >
> > myFile close.
> >
> > # --
> >
> > Populate Buffer is showing ~350 milliseconds to run on my
> > system whereas similar code in Perl is showing around 70
> > milliseconds.  Is this the correct way to use the stream?
> >
> > Thanks,
> >
> > Ian
> >
> > On 08/04/07, Bert Freudenberg <[hidden email]> wrote:
> > >
>
> Try taking out the Transcript show's from inside the loop, that's very slow.
>
>
> Ramon Leon
> http://onsmalltalk.com
>
> _______________________________________________
> Beginners mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/beginners
>

>
> On Apr 10, 2007, at 23:50 , Ian Oversby wrote:
>
> > On 10/04/07, Bert Freudenberg <[hidden email]> wrote:
> >>
> >> On Apr 10, 2007, at 23:22 , Ian Oversby wrote:
> >>
> >> > Okay, I tried this and it still seems a bit slow:
> >> >
> >> > | myFile ios |
> >> >
> >> > ios := ReadWriteStream on: ''.
> >> >
> >> > Transcript show: 'Populate Buffer: ',
> >> >       (Time millisecondsToRun: [
> >> >               1 to: 10000 do: [
> >> >                       :x | ios nextPutAll: ((x asString) ,
> >> String crlf)
> >> >                       ]]) asString , ' millseconds' ; cr.
> >> >
> >>
> >> >  Is this the correct way to use the stream?
> >>
> >> Why not write
> >>
> >>         1 to: 10000 do: [:x | ios print: x; cr]
> >>
> >> - Bert -
> >
> > I don't know the difference between print: and nextPutAll:,
>
> Select "print:", press Cmd-m.
>
> > but cr
> > didn't put the correct line ending for Windows, and it still seems
> > equally slow.
>
> Well, that's about as fast as you get without specific optimizations.
> Most probably Perl uses a C function for formatting numbers, Squeak
> does not. If this is mission-critical for your app it can be
> optimized, but in general we choose flexibility over raw speed.

> > I didn't know I was using Transcript in a loop.  Still, I've
>
> My bad, you weren't, I misread the code.
>
> > appending to the stream.  Is there a profiler in Squeak I can
> > search for the problem myself with?
> >
> > Ian
>
> MessageTally spyOn: [ code you want to trace ]
>
> Will show you where the problem lies.
>
> Ramon Leon
> http://onsmalltalk.com

> Hi Ian,
>
> You can get significant improvements by, writing to the file in one go,
> avoiding any intermediate strings, and taking a good guess at the size
> of your buffer.
>
> | bs fs bt ft |
> bs := WriteStream on: (String new: 60000).
> bt := Time millisecondsToRun:
>         [1 to: 10000 do:
>                 [:i |
>                 i printOn: bs.
>                 bs
>                         nextPut: Character cr;
>                         nextPut: Character lf]].
> fs := StandardFileStream fileNamed: 'c:/test4.txt'.
> ft := Time millisecondsToRun:
>         [[fs nextPutAll: bs contents]
>                 ensure: [fs close]].
> Transcript
>         show: bt asString;
>         show: ' / ';
>         show: ft asString;
>         show: ' / ';
>         show: bt + ft asString;
>         cr.
>
> The above runs in about 162ms on my machine where as your initial code
> takes about 275ms.
>
> If, however, I load the VisualWorks implementations of these methods
> (change set attached) the above runs in about 65ms. I wouldn't use this
> change set if I were you - I've probably broken something!
>
> I think number printing could be improved considerably without any
> primitives.
>
> Thanks,
> Zulq.
>
>
> Ian Oversby wrote:
> > Is this a reasonable way of writing to a file?  It seems to run a
> > little slower than I would expect.
> >
> > Thanks,
> >
> > Ian
> >
> > | myFile |
> > myFile := StandardFileStream fileNamed: 'c:/test.txt'
> >
> > Transcript show: (Time millisecondsToRun: [
> >    1 to: 10000 do: [
> >         :x | myFile nextPutAll: ((x asString) , String crlf)
> >     ]]) asString , ' millseconds' ; cr.
> >
> > myFile close.
>
>
> 'From Squeak3.9 of 7 November 2006 [latest update: #7067] on 22 April 2007 at 11:36:08 pm'!
>
> !Integer methodsFor: 'vw' stamp: 'za 4/22/2007 23:06'!
> printDigitsOn: aStream base: b
>         "Print a representation of the receiver on the stream, aStream, in
>         base b where 2<=b<=256.  The receiver is known to be non-negative."
>
>         self >= b
>                 ifTrue:
>                         [self // b printDigitsOn: aStream base: b].
>         aStream nextPut: (Character digitValue: self \\ b)! !
>
> !Integer methodsFor: 'vw' stamp: 'za 4/22/2007 23:06'!
> printOn: aStream base: b
>         "Print a representation of the receiver on the stream, aStream, in
>         base b where 2<=b<=256."
>
>         b < 2
>                 ifTrue: [self error: (#errInvalidBase << #dialogs >> 'Invalid base: <1p>'
>                                         expandMacrosWith: b)].
>         self < 0
>                 ifTrue:
>                         [aStream nextPut: $-.
>                         self negated printOn: aStream base: b]
>                 ifFalse:
>                         [self printDigitsOn: aStream base: b]! !
>
>
> !Integer reorganize!
> ('arithmetic' * + - / // \\\ alignedTo: crossSumBase: quo:)
> ('benchmarks' benchFib benchmark tinyBenchmarks)
> ('bit manipulation' << >> allMask: anyBitOfMagnitudeFrom:to: anyMask: bitAnd: bitClear: bitInvert bitInvert32 bitOr: bitShift: bitShiftMagnitude: bitXor: highBit highBitOfMagnitude lowBit noMask:)
> ('comparing' < = > hash)
> ('converting' adaptToComplex:andSend: adaptToFraction:andSend: adaptToScaledDecimal:andSend: asCharacter asColorOfDepth: asComplex asFloat asFloatSimply asFraction asHexDigit asInteger asScaledDecimal: asYear)
> ('enumerating' timesRepeat:)
> ('explorer' explorerContents hasContentsInExplorer)
> ('mathematical functions' factorial gcd: lcm: raisedToInteger:modulo: raisedTo:modulo: take:)
> ('printing' asStringWithCommas asStringWithCommasSigned asTwoCharacterString asWords destinationBuffer: digitBuffer: isLiteral printOn:base:showRadix: printPaddedWith:to: printPaddedWith:to:base: printStringRadix:)
> ('printing-numerative' byteEncode:base: printOn:base:length:padded: printStringBase: printStringBase:length:padded: printStringHex printStringLength: printStringLength:padded: printStringPadded: printStringRoman radix: storeOn:base: storeOn:base:length:padded: storeStringBase:length:padded: storeStringHex)
> ('system primitives' lastDigit replaceFrom:to:with:startingAt:)
> ('testing' even isInteger isPowerOfTwo isPrime)
> ('tiles' asPrecedenceName)
> ('truncation and round off' asLargerPowerOfTwo asPowerOfTwo asSmallerPowerOfTwo atRandom atRandom: ceiling floor normalize rounded truncated)
> ('private' copyto: digitAdd: digitCompare: digitDiv:neg: digitLogic:op:length: digitLshift: digitMultiply:neg: digitRshift:bytes:lookfirst: digitSubtract: growby: growto: isProbablyPrimeWithK:andQ: print:on:prefix:length:padded: romanDigits:for:on:)
> ('vw' printDigitsOn:base: printOn:base:)
> !
>
>
> _______________________________________________
> Beginners mailing list
> [hidden email]
> http://lists.squeakfoundation.org/mailman/listinfo/beginners
>
>