Read buffering for FileStream

Hi,

I uploaded two packages to The Inbox Multilingual-ul.70 and Files-ul.47
(load them in this order) which add read buffering to FileStream's
subinstances. As usual, all tests are green (which were green before). I
made some benchmarks on my notebook (core2 duo 2.4GHz, windows vista,
3.11.4 vm) with the current trunk image:

(1 to: 3) collect: [ :each |
  [
  StandardFileStream
  readOnlyFileNamed: (SourceFiles at: 1) name
  do: [ :file |
  [ file next == nil ] whileFalse ] ] timeToRun ]

Old: #(80165 79901 79762)
New: #(2247 1770 1802)
Speedup: ~41.21


(1 to: 3) collect: [ :each |
  [
  MultiByteFileStream
  readOnlyFileNamed: (SourceFiles at: 1) name
  do: [ :file |
  [ file next == nil ] whileFalse ] ] timeToRun ]
Old: #(76133 76243 76287)
New: #(5751 5741 5752)
Speedup: ~13.26


(1 to: 3) collect: [ :each |
  [
  MultiByteFileStream
  readOnlyFileNamed: (SourceFiles at: 1) name
  do: [ :file |
  file
  ascii;
  wantsLineEndConversion: false;
  converter: UTF8TextConverter new.
  1 to: 10000 do: [ :i | file upTo: Character cr ] ] ] timeToRun ]
Old: #(1940 1907 1902)
New: #(297 293 292)
Speedup: ~6.52x


[
  CompiledMethod allInstancesDo: [ :each |
  each
  getSourceFromFile;
  timeStamp ] ] timeToRun
Old: 102988
New: 29290
Speedup: ~3.52x


Running all tests profiled with the Test Runner:
Old: 638341
New: 552340
Speedup: ~1.16x


Should we add this to the trunk until something better (than FileStream)
is available?

Cheers,
Levente

Levente Uzonyi wrote:
> Should we add this to the trunk until something better (than FileStream)
> is available?

Yeehaa! :-) Love it. (and sorry for the belated response; yesterday was
spent in a world-wide rolling server upgrade that kept me busy for most
of the day)

Let's benchmark it:

StandardFileStream allSubInstancesDo:[:sfs| sfs disableReadBuffering].
t1 := [Object compileAll] timeToRun.
StandardFileStream allSubInstancesDo:[:sfs| sfs enableReadBuffering].
t2 := [Object compileAll] timeToRun.
{t1. t2}

=> #(1004 535)

In other words we just sped up recompilation by oh, a mere factor of two
(YMMV, due to OS differences, caching behavior etc). This should be
noticable in a variety of conditions including reshaping classes, change
list operations etc.

Great job!

Cheers,
   - Andreas

Andreas Raab wrote:
> Let's benchmark it:
>
> StandardFileStream allSubInstancesDo:[:sfs| sfs disableReadBuffering].
> t1 := [Object compileAll] timeToRun.
> StandardFileStream allSubInstancesDo:[:sfs| sfs enableReadBuffering].
> t2 := [Object compileAll] timeToRun.
> {t1. t2}
>
> => #(1004 535)

And for comparison, here's the result when using Cog:

  #(701 253)

So Cog basically speeds up the compilation itself by 2x but can't do
anything about the speed of file operations; as a consequence the
speedup w/o read buffering is a mere 20-30%; with read buffering it's
2x; together it's a 4x improvement. Not too shabby...

Cheers,
   - Andreas

On Sun, Dec 06, 2009 at 01:10:56PM -0800, Andreas Raab wrote:
Wow, there's something you don't see every day. Outstanding!

Dave

On Sun, 6 Dec 2009, Andreas Raab wrote:

> Levente Uzonyi wrote:
>> Should we add this to the trunk until something better (than FileStream) is
>> available?
>
> Yeehaa! :-) Love it. (and sorry for the belated response; yesterday was spent

Great. :)

Thanks.


Levente

> Cheers,
>  - Andreas
>
>

Woo! Congrats and thank you, folks working on these improvements.