Write stream on an external array?

Blair,

This might be asking too much, but I have occaisionally wanted to use a
write stream to grow an external array.  In the spirit of "how hard can
it be?", I decided to take a shot at it, and it is almost working.

For example:

| out array |
out := WriteStream on:( array := DOUBLEArray new:10 ).
1 to:10 do:[ :each |
        out nextPut:50.0*each.
].
out contents.
array

#contents does not work as expected, I think because #copyFrom:to:
returns a byte array.  Try

   ( DOUBLEArray new:5 ) copyFrom:1 to:4

Should it instead answer a collection of doubles?

Forgetting for the moment about #contents (I'm not sure I want to copy
these arrays anyway),

| out array |
out := WriteStream on:( array := DOUBLEArray new:10 ).
1 to:20 do:[ :each |
        out nextPut:50.0*each.
].
array.

fails for want of #grow.  While arbitray

!ExternalArray methodsFor!
grow
        self length:self length + 10.
! !
!ExternalArray categoriesFor: #grow!public! !

appears to allow it to run.


I think this even works as expected:

| out array |
out := WriteStream with:( array := DOUBLEArray new:10 ).
1 to:5 do:[ :each |
        out nextPut:50.0*each.
].
array

What do you think?


Have a good one,

Bill

--
Wilhelm K. Schwab, Ph.D.
[hidden email]

Blair,

An updated #grow and a couple of (largely failing) tests follow.  Note
that the tests include using #nextPut: to grow a DOUBLEArray, and that
much works in my image.

If you disagree that #copyFrom:to: is broken, then #testExternalArray is
not needed.

Have a good one,

Bill

======================


!ExternalArray methodsFor!

grow
        "Private - Increase the capacity of the receiver. Answer the receiver."

        self length:self length + self growSize.
! !
!ExternalArray categoriesFor: #grow!private! !

!ExternalArray methodsFor!

growSize
        "Private - Answer the number of elements by which the receiver should
grow, when growing!!
        (at least 2, in case the receiver is currently empty)"

        ^self size max: 2.
! !
!ExternalArray categoriesFor: #growSize!accessing!private! !

!DolphinTestsWKS methodsFor!

testExternalArray
                | anArray |

        anArray := DOUBLEArray new:5.

        self should:[
                ( anArray copyFrom:1 to:4 ) first class = anArray elementClass.
        ].
! !
!DolphinTestsWKS categoriesFor: #testExternalArray!public! !
!DolphinTestsWKS methodsFor!

testExternalArrayWriteStream
        "Use a write stream to add data to a DOUBLEArray, forcing it to grow; use
        #contents to obtain a copy of the DOUBLEArray - currently fails."

                | out anArray last |

        "Start short so the array will need to grow."
        anArray := DOUBLEArray new:2.
        out := WriteStream on:anArray.
        1 to:10 do:[ :each |
                out nextPut:( last := 50.0*each ).
        ].

        self should:[
                anArray last = last.
        ].

        self should:[ last class = Float ].
        self should:[ out contents last class = Float ].
        self should:[ out contents last equals:last. ].
! !
!DolphinTestsWKS categoriesFor: #testExternalArrayWriteStream!public! !


--
Wilhelm K. Schwab, Ph.D.
[hidden email]

"Bill Schwab" <[hidden email]> wrote in message
news:cci8vg$1cv2$[hidden email]...
> Blair,
>
> An updated #grow and a couple of (largely failing) tests follow.  Note
> that the tests include using #nextPut: to grow a DOUBLEArray, and that
> much works in my image.
>
> If you disagree that #copyFrom:to: is broken, then #testExternalArray is
> not needed.

Bill, the ExternalArray hierarchy was never really intended to implement a
complete sequenceable collection protocol, just sufficient for external
interfacing purposes. Of course what is sufficient for one, may not be for
another, and I suppose it is reasonable to want to be able to treat a
DOUBLEArray as a normal array given its utility for representing large
collections of floats much more space efficiently than one can do with Float
objects in a normal Array.

I'm afraid it (i.e. making ExternalArrays "write streamable") is not the
sort of enhancement I would normally consider in a patch level, as it is
potentially wide ranging and may involve significant refactoring, and all of
that amounts to a lot of extra SUnits. So thats a lot of effort. To justify
a lot of effort in a PL there must either be a significant bug to address, a
commonly held need, or a private financial arrangement :-).

I don't really agree or disagree that #copyFrom:to: is broken, because that
would depend on whether it was considered as implementing that message of
sequenceable collections which is not really relevant to most of the
external structure hierarchy, or whether it was a way of copying a range of
bytes from a structure (which was its intended meaning in this case).
Ideally another name should have been chosen for copying the bytes, and when
the notion of representing arrays was added to the ExternalStructure
hierarchy #copyFrom:to: could have been added at that level only to behave
in the accepted <sequenceableCollection> manner. However implementating that
now may have wider effect, and so it would mean ensuring adequate test
coverage for direct and indirect uses of the existing meaning. Unfortunately
we will not be working on this in the near future, but you are free to do so
of course if you find it to be necessary for your work.

Regards

Blair