Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Mariano Martinez Peck
Guys,

I am getting "UTF8InvalidText: Invalid utf8 input detected" quite frequently on Pharo 6.0 which come from ombu background job. Has anything seen this too? Quite annoying...  

This is not the first time I hit it. This time I analyzed a bit why it was failing and it's because I removed a method manually...I mean, I did
MyClass removeSelector: #myMethod

And the omEntry is a EpMethodRemoval which represents the exact method I deleted...

Also I can see the MultiByteFileStream internal 'collection' is full of none ascii things. 

I can share a fuelout as well as the image with the files if you want. I can also share the ombu file in question. 
 
Contact me in private. 



----



[ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] in [ :readStream | 
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :error | ...
BlockClosure>>cull:
Context>>evaluateSignal:
Context>>handleSignal:
UTF8InvalidText(Exception)>>signal
UTF8InvalidText(Exception)>>signal:
UTF8TextConverter>>errorMalformedInput
UTF8TextConverter>>nextFromStream:
MultiByteFileStream>>next
MultiByteFileStream(PositionableStream)>>match:
MultiByteFileStream(PositionableStream)>>upToAll:
OmSTONEntryReader>>nextEntryPositionIfFound:ifNone:
OmBlockFileStore>>refreshNewBlocksFrom:
[ :readStream | self refreshNewBlocksFrom: readStream ] in OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since: in Block: [ :readStream | self refreshNewBlocksFrom: readStr...etc...
[ ^ aBlockClosure value: readStream ascii ] in [ :readStream | 
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ ^ aBlockClosure value: readStream ascii ]
BlockClosure>>on:do:
[ :readStream | 
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :readStream | ...
[ aBlock value: stream ] in FileReference(AbstractFileReference)>>readStreamDo: in Block: [ aBlock value: stream ]
BlockClosure>>ensure:
FileReference(AbstractFileReference)>>readStreamDo:
OmBlockFileStore(OmFileStore)>>readEntriesWith:
OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since:
[ | initialPosition initialLocalName |
self entryBuffer isEmpty
ifTrue: [ ^ self ].
fileReference
writeStreamDo: [ :fileStream | 
fileStream setToEnd.
initialPosition := fileStream position.
initialLocalName := self entryBuffer first value.
ZnBufferedWriteStream
on: fileStream
do: [ :aWriteStream | 
| anEntryWriter |
anEntryWriter := self newEntryWriter.
[ self entryBuffer isEmpty ]
whileFalse: [ | next entry |
next := self entryBuffer removeFirst.
entry := next key.

"Write entry to file"
anEntryWriter on: aWriteStream nextEntryPut: entry ] ].

"In Linux it was necessary to explicitely flush the file stream"
fileStream flush.
lastStreamPosition := fileStream size ].
self
refreshEntryPositionsByLocalNameStartingAt: initialPosition
since: initialLocalName ] in OmBlockFileStore(OmFileStore)>>flushEntryBuffer in Block: [ | initialPosition initialLocalName |...
[ caught := true.
self wait.
blockValue := mutuallyExcludedBlock value ] in Semaphore>>critical: in Block: [ caught := true....
BlockClosure>>ensure:
Semaphore>>critical:
OmBlockFileStore(OmFileStore)>>critical:
OmBlockFileStore(OmFileStore)>>flushEntryBuffer
OmDeferrer>>sendMessage
[ self sendMessage ] in OmDeferrer>>flush in Block: [ self sendMessage ]


Reply | Threaded
Open this post in threaded view
|

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Max Leske

 

On 25 July 2017 at 22:55:19, Mariano Martinez Peck ([hidden email]) wrote:

Guys,

I am getting "UTF8InvalidText: Invalid utf8 input detected" quite frequently on Pharo 6.0 which come from ombu background job. Has anything seen this too? Quite annoying...  

Yes, I've seen it a couple of times. No clue though why it happens, or when.



This is not the first time I hit it. This time I analyzed a bit why it was failing and it's because I removed a method manually...I mean, I did
MyClass removeSelector: #myMethod

And the omEntry is a EpMethodRemoval which represents the exact method I deleted...

Also I can see the MultiByteFileStream internal 'collection' is full of none ascii things. 

I can share a fuelout as well as the image with the files if you want. I can also share the ombu file in question. 
 
Contact me in private. 



----



[ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] in [ :readStream | 
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :error | ...
BlockClosure>>cull:
Context>>evaluateSignal:
Context>>handleSignal:
UTF8InvalidText(Exception)>>signal
UTF8InvalidText(Exception)>>signal:
UTF8TextConverter>>errorMalformedInput
UTF8TextConverter>>nextFromStream:
MultiByteFileStream>>next
MultiByteFileStream(PositionableStream)>>match:
MultiByteFileStream(PositionableStream)>>upToAll:
OmSTONEntryReader>>nextEntryPositionIfFound:ifNone:
OmBlockFileStore>>refreshNewBlocksFrom:
[ :readStream | self refreshNewBlocksFrom: readStream ] in OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since: in Block: [ :readStream | self refreshNewBlocksFrom: readStr...etc...
[ ^ aBlockClosure value: readStream ascii ] in [ :readStream | 
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ ^ aBlockClosure value: readStream ascii ]
BlockClosure>>on:do:
[ :readStream | 
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error | 
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :readStream | ...
[ aBlock value: stream ] in FileReference(AbstractFileReference)>>readStreamDo: in Block: [ aBlock value: stream ]
BlockClosure>>ensure:
FileReference(AbstractFileReference)>>readStreamDo:
OmBlockFileStore(OmFileStore)>>readEntriesWith:
OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since:
[ | initialPosition initialLocalName |
self entryBuffer isEmpty
ifTrue: [ ^ self ].
fileReference
writeStreamDo: [ :fileStream | 
fileStream setToEnd.
initialPosition := fileStream position.
initialLocalName := self entryBuffer first value.
ZnBufferedWriteStream
on: fileStream
do: [ :aWriteStream | 
| anEntryWriter |
anEntryWriter := self newEntryWriter.
[ self entryBuffer isEmpty ]
whileFalse: [ | next entry |
next := self entryBuffer removeFirst.
entry := next key.

"Write entry to file"
anEntryWriter on: aWriteStream nextEntryPut: entry ] ].

"In Linux it was necessary to explicitely flush the file stream"
fileStream flush.
lastStreamPosition := fileStream size ].
self
refreshEntryPositionsByLocalNameStartingAt: initialPosition
since: initialLocalName ] in OmBlockFileStore(OmFileStore)>>flushEntryBuffer in Block: [ | initialPosition initialLocalName |...
[ caught := true.
self wait.
blockValue := mutuallyExcludedBlock value ] in Semaphore>>critical: in Block: [ caught := true....
BlockClosure>>ensure:
Semaphore>>critical:
OmBlockFileStore(OmFileStore)>>critical:
OmBlockFileStore(OmFileStore)>>flushEntryBuffer
OmDeferrer>>sendMessage
[ self sendMessage ] in OmDeferrer>>flush in Block: [ self sendMessage ]


Reply | Threaded
Open this post in threaded view
|

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Henrik Sperre Johansen
There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:

Cheers,
Henry
Reply | Threaded
Open this post in threaded view
|

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Mariano Martinez Peck


On Wed, Jul 26, 2017 at 6:36 AM, Henrik Sperre Johansen <[hidden email]> wrote:
There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the
same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:


Hi Henry,

Thanks for the pointer. I patched my images with your workaround.  I will use it for a couple of days and I will later tell if you it seems to fix it or not.

Cheers,

 
Cheers,
Henry



--
View this message in context: http://forum.world.st/Lots-of-UTF8InvalidText-Invalid-utf8-input-detected-from-OMBU-tp4956734p4956805.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.




--
Reply | Threaded
Open this post in threaded view
|

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

tinchodias
Hello: 
So, is this patch working fine?

Cheers
Martin

On Thu, Jul 27, 2017 at 10:17 AM, Mariano Martinez Peck <[hidden email]> wrote:


On Wed, Jul 26, 2017 at 6:36 AM, Henrik Sperre Johansen <[hidden email]> wrote:
There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the
same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:


Hi Henry,

Thanks for the pointer. I patched my images with your workaround.  I will use it for a couple of days and I will later tell if you it seems to fix it or not.

Cheers,

 
Cheers,
Henry



--
View this message in context: http://forum.world.st/Lots-of-UTF8InvalidText-Invalid-utf8-input-detected-from-OMBU-tp4956734p4956805.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.




--

Reply | Threaded
Open this post in threaded view
|

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Mariano Martinez Peck

On Tue, Aug 1, 2017 at 11:34 AM, Martin Dias <[hidden email]> wrote:
Hello: 
So, is this patch working fine?



Well...it's has been a couple of days already and I haven't seen the issue again, so I guess yes, the workaround works...

 
Cheers
Martin

On Thu, Jul 27, 2017 at 10:17 AM, Mariano Martinez Peck <[hidden email]> wrote:


On Wed, Jul 26, 2017 at 6:36 AM, Henrik Sperre Johansen <[hidden email]> wrote:
There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the
same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:


Hi Henry,

Thanks for the pointer. I patched my images with your workaround.  I will use it for a couple of days and I will later tell if you it seems to fix it or not.

Cheers,

 
Cheers,
Henry



--
View this message in context: http://forum.world.st/Lots-of-UTF8InvalidText-Invalid-utf8-input-detected-from-OMBU-tp4956734p4956805.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.




--




--