|
History:
This issue has been spotted after experimenting strange behavior with seaside upload.
After uploading a big file from a web browser, the model file generated within pharo image begins with 4 unexpected bytes.
This issue occurs randomly: sometimes the first 4 bytes are right. Sometimes the first 4 bytes are wrong.
This issue only occurs with Pharo 6.
This issue occurs for all platforms (Mac, Ubuntu, Windows)
Steps to reproduce:
I have been able to set up a small scenario that highlight the issue.
Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/web/download
Then, iterate over this process till spotting the issue:
=> start the pharo image
=> execute this piece of code in a playground
ZnServer startDefaultOn: 1701.
ZnServer default maximumEntitySize: 80* 1024 * 1024.
'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |
out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].
18202065 timesRepeat: [ out nextPut: 0 ]
].
=> Open a web browser page on: http://localhost:1701/form-test-3
=> Upload the file zip file previously generated ('myzip.zip').
=> If the web page displays: "contents=000000000a00..." (or anything unexpected), THIS IS THE ISSUE !
=> If the web page displays: "contents=504b03040a00..", the upload worked fine. You can close the image (without saving)
Debugging:
Bob Arning has been able to reproduce the issue with my scenario.
He dived into the code involved during this process, till reaching some "basic" methods where he saw the issue occuring.
Here are the conclusion till there:
=> A corruption occurs while reading an input stream with method ZnUtils class>>readUpToEnd:limit:
The first 4 bytes may be altered randomely.
=> The first 4 bytes are initially correctly written to an outputStream.
But, the first 4 bytes of this outputStream gets altered (corrupted), sometimes when the inner byte array grows OR when performing the final "outputStream contents"
=> Here is a piece of code that reproduce the issue (still randomely. stopping an restarting the image may change the behavior)
test4
"
self test4
"
| species bufferSize buffer totalRead outputStream answer inputStream ba byte1 |
ba := ByteArray new: 18202085.
ba atAllPut: 99.
1 to: 20 do: [ :i | ba at: i put: (#[80 75 3 4 10 7 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ].
inputStream := ba readStream.
bufferSize := 16384.
species := ByteArray.
buffer := species new: bufferSize.
totalRead := 0.
outputStream := nil.
[ inputStream atEnd ] whileFalse: [ | readCount |
readCount := inputStream readInto: buffer startingAt: 1 count: bufferSize.
totalRead = 0 ifTrue: [
byte1 := buffer first.
].
totalRead := totalRead + readCount.
outputStream ifNil: [
inputStream atEnd
ifTrue: [ ^ buffer copyFrom: 1 to: readCount ]
ifFalse: [ outputStream := (species new: bufferSize) writeStream ] ].
outputStream next: readCount putAll: buffer startingAt: 1.
byte1 = outputStream contents first ifFalse: [ self halt ].
].
answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream contents ].
byte1 = answer first ifFalse: [ self halt ].
^answer
suspicions
This issue appeared with Pharo 6.
Some people suggested that it could be a vm issue, and to try my little scenario with the last available vm.
I am not sure where to find the last available vm.
I did the test using these elements:
https://files.pharo.org/image/60/latest.zip
https://files.pharo.org/get-files/70/pharo-mac-latest.zip/
The issue is still present
|
Title changed:
Corrupted data when copying from very large byte array
Random corrupted data when copying from very large byte array
Status changed:
Active
Work Needed
|
|
Priority: 2 – Really Important
|
|
Status: Work Needed
|
|
Assigned to: Everyone
|
|
Milestone: Pharo6.0
|
Go to Case
|
|