More on ExternalArray and write streams

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

More on ExternalArray and write streams

Schwab,Wilhelm K
Blair,

I have an evolving source of numeric data that is best stored as double
or (better yet) word arrays.  This data will arrive in small blobs at an
effective rate of one byte per millisecond or so.  It will go on for
some time, with unpredictable interruptions, so streaming into an
external array is the easiest way to handle it.  Interruptions???  The
idea is to use ordinary collections to (sparsely, in numerber crunching
speak) hold dense arrays of samples.

After some more work on it, I don't see any horrible problems, but
welcome any cautions you care to offer.  The following methods hopefully
allow streaming into at least DOUBLEArray.

Since you did not consider the behavior of ExternalArray>>copyFrom:to to
be incorrect, I have worked around problems with #contents using "the"
associated write stream's #position, either directly or by
#contentsViaIndexing below.  I kept one failing test to document the
failure mode, though you would probably want to delete it, or alter it
so that it passes.

Have a good one,

Bill


-----------------


!WriteStream methodsFor!

contentsViaIndexing
        "For streaming into ExternalArray subinstances.  Copies from 1 to the
receiver's
        #position."

                | padded samples |

        #wksDangerous.
        padded := self collection.
        samples := padded class new:self position.
        1 to:samples size do:[ :index |
                samples at:index put:( padded at:index ).
        ].

        ^samples.
! !
!WriteStream categoriesFor: #contentsViaIndexing!public! !



!ExternalArray methodsFor!

grow
        "Private - Increase the capacity of the receiver. Answer the receiver."

        self length:self length + self growSize.
! !
!ExternalArray categoriesFor: #grow!private! !

!ExternalArray methodsFor!

growSize
        "Private - Answer the number of elements by which the receiver should
grow, when growing!!
        (at least 2, in case the receiver is currently empty)"

        ^self size max: 2.
! !
!ExternalArray categoriesFor: #growSize!accessing!private! !


ExternalArray methodsFor!

appendToStream: puttableStream
        "Private - Append the receiver's elements to the argument, puttableStream.
        We can be able to do a fast block copy. Answer the receiver.
        Implementation note: Double dispatched from puttableStream>>nextPutAll:."

        puttableStream next: self size putAll: self startingAt: 1! !
!ExternalArray categoriesFor: #appendToStream:!double dispatch!private! !




!DolphinTestsWKS methodsFor!

testExternalArrayWriteStream
        "Use a write stream to add data to a DOUBLEArray, forcing it to grow; use
        #contents to obtain a copy of the DOUBLEArray - currently fails."

                | out anArray last |

        "Start short so the array will need to grow."
        anArray := DOUBLEArray new:2.
        out := WriteStream on:anArray.
        1 to:10 do:[ :each |
                out nextPut:( last := 50.0*each ).
        ].

        "#contents does not work as expected, but one can use the #position
        of the write stream."
        self should:[
                ( anArray at:out position ) equals:last.
        ].

        "Capture above as #contentsViaIndexing"
        self should:[
                out contentsViaIndexing last equals:last.
        ].


        "
        self should:[ out contents last class = Float ].
        self should:[ out contents last equals:last. ]."
! !
!DolphinTestsWKS categoriesFor: #testExternalArrayWriteStream!public! !

!DolphinTestsWKS methodsFor!

testExternalArrayAppendToStream

                | input out |

        input := ( 1 to:50 ) collect:[ :i | i asFloat. ].
        ( DOUBLEArray withAll:input ) last = ( input last ).
        input := DOUBLEArray withAll:input.

        out := WriteStream on:DOUBLEArray new.
        out nextPutAll:input.

        self should:[
                ( out collection at:out position ) = input last
        ].
        self should:[
                out contentsViaIndexing last = input last
        ].
! !
!DolphinTestsWKS categoriesFor: #testExternalArrayAppendToStream!public! !

!DolphinTestsWKS methodsFor!

testExternalArrayCopyFromTo
        "Fails because #copyFrom:to: answers bytes, not elements.
        Blair, this is included form completeness; you will probably want
        to delete it."

                | anArray |

        anArray := DOUBLEArray new:5.

        self should:[
                ( anArray copyFrom:1 to:4 ) first class = anArray elementClass.
        ].
! !
!DolphinTestsWKS categoriesFor: #testExternalArrayCopyFromTo!public! !




--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: More on ExternalArray and write streams

Schwab,Wilhelm K
Blair,

I should have better emphasized that I am particularly/most concerned
about ExternalArray>>appendToStream:.  That is a very new addition, and
has been through only simple tests a few image save/load cycles.

Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: More on ExternalArray and write streams

Schwab,Wilhelm K
In reply to this post by Schwab,Wilhelm K
Hello all,

One more test appears below.  This one looks like it might be more of a
problem.

Have a good one,

Bill


-----------------


!DolphinTestsWKS methodsFor!

testWORDArrayWriteStream

                | aWordArray out |

        aWordArray := WORDArray new.
        out := WriteStream on:aWordArray.

        50 timesRepeat:[
                out nextPutAll:#( 2001 2002 2003 2004 ).
        ].

        out contentsViaIndexing.
! !
!DolphinTestsWKS categoriesFor: #testWORDArrayWriteStream!public! !


--
Wilhelm K. Schwab, Ph.D.
[hidden email]