Manuscript (Case [Issue]20982) _Inbox - Random corrupted data when copying from very large byte array

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Manuscript (Case [Issue]20982) _Inbox - Random corrupted data when copying from very large byte array

Pharo Issue Tracker
Manuscript Notification
avatar
Bug in Project:  _Inbox: Not Spam  •  You are subscribed to this case
History:

This issue has been spotted after experimenting strange behavior with seaside upload.
After uploading a big file from a web browser, the model file generated within pharo image begins with 4 unexpected bytes.
This issue occurs randomly: sometimes the first 4 bytes are right. Sometimes the first 4 bytes are wrong.
This issue only occurs with Pharo 6.
This issue occurs for all platforms (Mac, Ubuntu, Windows)

Steps to reproduce:

I have been able to set up a small scenario that highlight the issue.

Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/web/download
Then, iterate over this process till spotting the issue:

=> start the pharo image
=> execute this piece of code in a playground

ZnServer startDefaultOn: 1701.
ZnServer default maximumEntitySize: 80* 1024 * 1024.
'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |
out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].
18202065 timesRepeat: [ out nextPut: 0 ]
].

=> Open a web browser page on: http://localhost:1701/form-test-3
=> Upload the file zip file previously generated ('myzip.zip').
=> If the web page displays: "contents=000000000a00..." (or anything unexpected), THIS IS THE ISSUE !
=> If the web page displays: "contents=504b03040a00..", the upload worked fine. You can close the image (without saving)

 

Debugging:

 

Bob Arning has been able to reproduce the issue with my scenario.
He dived into the code involved during this process, till reaching some "basic" methods where he saw the issue occuring.

Here are the conclusion till there:
=> A corruption occurs while reading an input stream with method ZnUtils class>>readUpToEnd:limit: 
The first 4 bytes may be altered randomely.
=> The first 4 bytes are initially correctly written to an outputStream.  
But, the first 4 bytes of this outputStream gets altered (corrupted), sometimes when the inner byte array grows OR when performing the final "outputStream contents"
=> Here is a piece of code that reproduce the issue (still randomely. stopping an restarting the image may change the behavior)

test4
"
self test4
"
    | species bufferSize buffer totalRead outputStream answer inputStream ba byte1 |
    

        ba := ByteArray new: 18202085.
        ba atAllPut: 99.
        1 to: 20 do: [  :i | ba at: i put: (#[80 75 3 4 10 7 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ].

    inputStream := ba readStream.
    bufferSize := 16384.
    species := ByteArray.
    buffer := species new: bufferSize.
    totalRead := 0.
    outputStream := nil. 
    [ inputStream atEnd ] whileFalse: [ | readCount |
        readCount := inputStream readInto: buffer startingAt: 1 count: bufferSize.
        totalRead = 0 ifTrue: [
            byte1 := buffer first.
        ].
        totalRead := totalRead + readCount.
        
        outputStream ifNil: [ 
            inputStream atEnd
                ifTrue: [ ^ buffer copyFrom: 1 to: readCount ] 
                ifFalse: [ outputStream := (species new: bufferSize) writeStream ] ].
        outputStream next: readCount putAll: buffer startingAt: 1.
        byte1 = outputStream contents first ifFalse: [ self halt ].
    ].
    answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream contents ].
    byte1 = answer first ifFalse: [ self halt ].
    ^answer

 

suspicions

This issue appeared with Pharo 6.

Some people suggested that it could be a vm issue, and to try my little scenario with the last available vm.

I am not sure where to find the last available vm.

I did the test using these elements:

https://files.pharo.org/image/60/latest.zip

https://files.pharo.org/get-files/70/pharo-mac-latest.zip/

 

The issue is still present
Title changed:
Corrupted data when copying from very large byte array  changed to  Random corrupted data when copying from very large byte array

Status changed:
Active  changed to  Work Needed

Priority Priority: 2 – Really Important Status Status: Work Needed
Assigned To Assigned to: Everyone Milestone Milestone: Pharo6.0

Go to Case
No longer need updates? Unsubscribe from this case.

Don't want Manuscript notifications anymore? Update your preferences.

Manuscript

_______________________________________________
Pharo-bugtracker mailing list
[hidden email]
https://lists.gforge.inria.fr/mailman/listinfo/pharo-bugtracker