Smalltalk › Pharo › Pharo Smalltalk Developers

Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

6 messages Options

Mariano Martinez Peck

Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Guys,

I am getting "UTF8InvalidText: Invalid utf8 input detected" quite frequently on Pharo 6.0 which come from ombu background job. Has anything seen this too? Quite annoying...

This is not the first time I hit it. This time I analyzed a bit why it was failing and it's because I removed a method manually...I mean, I did

MyClass removeSelector: #myMethod

And the omEntry is a EpMethodRemoval which represents the exact method I deleted...

Also I can see the MultiByteFileStream internal 'collection' is full of none ascii things.

I can share a fuelout as well as the image with the files if you want. I can also share the ombu file in question.

Contact me in private.

----

[ :error |

(OmFileStoreReadingError

readingError: error

on: self fileReference

position: readStream position) signal ] in [ :readStream |

[ ^ aBlockClosure value: readStream ascii ]

on: Error

do: [ :error |

(OmFileStoreReadingError

readingError: error

on: self fileReference

position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :error | ...

BlockClosure>>cull:

Context>>evaluateSignal:

Context>>handleSignal:

UTF8InvalidText(Exception)>>signal

UTF8InvalidText(Exception)>>signal:

UTF8TextConverter>>errorMalformedInput

UTF8TextConverter>>nextFromStream:

MultiByteFileStream>>next

MultiByteFileStream(PositionableStream)>>match:

MultiByteFileStream(PositionableStream)>>upToAll:

OmSTONEntryReader>>nextEntryPositionIfFound:ifNone:

OmBlockFileStore>>refreshNewBlocksFrom:

[ :readStream | self refreshNewBlocksFrom: readStream ] in OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since: in Block: [ :readStream | self refreshNewBlocksFrom: readStr...etc...

[ ^ aBlockClosure value: readStream ascii ] in [ :readStream |

[ ^ aBlockClosure value: readStream ascii ]

on: Error

do: [ :error |

(OmFileStoreReadingError

readingError: error

on: self fileReference

position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ ^ aBlockClosure value: readStream ascii ]

BlockClosure>>on:do:

[ :readStream |

[ ^ aBlockClosure value: readStream ascii ]

on: Error

do: [ :error |

(OmFileStoreReadingError

readingError: error

on: self fileReference

position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :readStream | ...

[ aBlock value: stream ] in FileReference(AbstractFileReference)>>readStreamDo: in Block: [ aBlock value: stream ]

BlockClosure>>ensure:

FileReference(AbstractFileReference)>>readStreamDo:

OmBlockFileStore(OmFileStore)>>readEntriesWith:

OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since:

[ | initialPosition initialLocalName |

self entryBuffer isEmpty

ifTrue: [ ^ self ].

fileReference

writeStreamDo: [ :fileStream |

fileStream setToEnd.

initialPosition := fileStream position.

initialLocalName := self entryBuffer first value.

ZnBufferedWriteStream

on: fileStream

do: [ :aWriteStream |

| anEntryWriter |

anEntryWriter := self newEntryWriter.

[ self entryBuffer isEmpty ]

whileFalse: [ | next entry |

next := self entryBuffer removeFirst.

entry := next key.

"Write entry to file"

anEntryWriter on: aWriteStream nextEntryPut: entry ] ].

"In Linux it was necessary to explicitely flush the file stream"

fileStream flush.

lastStreamPosition := fileStream size ].

self

refreshEntryPositionsByLocalNameStartingAt: initialPosition

since: initialLocalName ] in OmBlockFileStore(OmFileStore)>>flushEntryBuffer in Block: [ | initialPosition initialLocalName |...

[ caught := true.

self wait.

blockValue := mutuallyExcludedBlock value ] in Semaphore>>critical: in Block: [ caught := true....

BlockClosure>>ensure:

Semaphore>>critical:

OmBlockFileStore(OmFileStore)>>critical:

OmBlockFileStore(OmFileStore)>>flushEntryBuffer

OmDeferrer>>sendMessage

[ self sendMessage ] in OmDeferrer>>flush in Block: [ self sendMessage ]

Mariano
http://marianopeck.wordpress.com

Max Leske

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

On 25 July 2017 at 22:55:19, Mariano Martinez Peck ([hidden email]) wrote:

Guys,

I am getting "UTF8InvalidText: Invalid utf8 input detected" quite frequently on Pharo 6.0 which come from ombu background job. Has anything seen this too? Quite annoying...

Yes, I've seen it a couple of times. No clue though why it happens, or when.

This is not the first time I hit it. This time I analyzed a bit why it was failing and it's because I removed a method manually...I mean, I did
MyClass removeSelector: #myMethod

And the omEntry is a EpMethodRemoval which represents the exact method I deleted...

Also I can see the MultiByteFileStream internal 'collection' is full of none ascii things.

I can share a fuelout as well as the image with the files if you want. I can also share the ombu file in question.

Contact me in private.

----

[ :error |
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] in [ :readStream |
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error |
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :error | ...
BlockClosure>>cull:
Context>>evaluateSignal:
Context>>handleSignal:
UTF8InvalidText(Exception)>>signal
UTF8InvalidText(Exception)>>signal:
UTF8TextConverter>>errorMalformedInput
UTF8TextConverter>>nextFromStream:
MultiByteFileStream>>next
MultiByteFileStream(PositionableStream)>>match:
MultiByteFileStream(PositionableStream)>>upToAll:
OmSTONEntryReader>>nextEntryPositionIfFound:ifNone:
OmBlockFileStore>>refreshNewBlocksFrom:
[ :readStream | self refreshNewBlocksFrom: readStream ] in OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since: in Block: [ :readStream | self refreshNewBlocksFrom: readStr...etc...
[ ^ aBlockClosure value: readStream ascii ] in [ :readStream |
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error |
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ ^ aBlockClosure value: readStream ascii ]
BlockClosure>>on:do:
[ :readStream |
[ ^ aBlockClosure value: readStream ascii ]
on: Error
do: [ :error |
(OmFileStoreReadingError
readingError: error
on: self fileReference
position: readStream position) signal ] ] in OmBlockFileStore(OmFileStore)>>readEntriesWith: in Block: [ :readStream | ...
[ aBlock value: stream ] in FileReference(AbstractFileReference)>>readStreamDo: in Block: [ aBlock value: stream ]
BlockClosure>>ensure:
FileReference(AbstractFileReference)>>readStreamDo:
OmBlockFileStore(OmFileStore)>>readEntriesWith:
OmBlockFileStore>>refreshEntryPositionsByLocalNameStartingAt:since:
[ | initialPosition initialLocalName |
self entryBuffer isEmpty
ifTrue: [ ^ self ].
fileReference
writeStreamDo: [ :fileStream |
fileStream setToEnd.
initialPosition := fileStream position.
initialLocalName := self entryBuffer first value.
ZnBufferedWriteStream
on: fileStream
do: [ :aWriteStream |
| anEntryWriter |
anEntryWriter := self newEntryWriter.
[ self entryBuffer isEmpty ]
whileFalse: [ | next entry |
next := self entryBuffer removeFirst.
entry := next key.

"Write entry to file"
anEntryWriter on: aWriteStream nextEntryPut: entry ] ].

"In Linux it was necessary to explicitely flush the file stream"
fileStream flush.
lastStreamPosition := fileStream size ].
self
refreshEntryPositionsByLocalNameStartingAt: initialPosition
since: initialLocalName ] in OmBlockFileStore(OmFileStore)>>flushEntryBuffer in Block: [ | initialPosition initialLocalName |...
[ caught := true.
self wait.
blockValue := mutuallyExcludedBlock value ] in Semaphore>>critical: in Block: [ caught := true....
BlockClosure>>ensure:
Semaphore>>critical:
OmBlockFileStore(OmFileStore)>>critical:
OmBlockFileStore(OmFileStore)>>flushEntryBuffer
OmDeferrer>>sendMessage
[ self sendMessage ] in OmDeferrer>>flush in Block: [ self sendMessage ]

--
Mariano
http://marianopeck.wordpress.com

Henrik Sperre Johansen

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:

Cheers,
Henry

Mariano Martinez Peck

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

On Wed, Jul 26, 2017 at 6:36 AM, Henrik Sperre Johansen <[hidden email]> wrote:

There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the
same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:

Hi Henry,

Thanks for the pointer. I patched my images with your workaround. I will use it for a couple of days and I will later tell if you it seems to fix it or not.

Cheers,

Cheers,
Henry

--
View this message in context: http://forum.world.st/Lots-of-UTF8InvalidText-Invalid-utf8-input-detected-from-OMBU-tp4956734p4956805.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Mariano
http://marianopeck.wordpress.com

tinchodias

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

Hello:

So, is this patch working fine?

Cheers

Martin

On Thu, Jul 27, 2017 at 10:17 AM, Mariano Martinez Peck <[hidden email]> wrote:

On Wed, Jul 26, 2017 at 6:36 AM, Henrik Sperre Johansen <[hidden email]> wrote:
There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the
same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:

Hi Henry,

Thanks for the pointer. I patched my images with your workaround. I will use it for a couple of days and I will later tell if you it seems to fix it or not.

Cheers,

Cheers,
Henry

--
View this message in context: http://forum.world.st/Lots-of-UTF8InvalidText-Invalid-utf8-input-detected-from-OMBU-tp4956734p4956805.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

--
Mariano
http://marianopeck.wordpress.com

Mariano Martinez Peck

Re: Lots of "UTF8InvalidText: Invalid utf8 input detected" from OMBU

On Tue, Aug 1, 2017 at 11:34 AM, Martin Dias <[hidden email]> wrote:

Hello:
So, is this patch working fine?

Well...it's has been a couple of days already and I haven't seen the issue again, so I guess yes, the workaround works...

Cheers
Martin

On Thu, Jul 27, 2017 at 10:17 AM, Mariano Martinez Peck <[hidden email]> wrote:

On Wed, Jul 26, 2017 at 6:36 AM, Henrik Sperre Johansen <[hidden email]> wrote:
There's https://pharo.fogbugz.com/f/cases/20112/, your error could be the
same.
(a multibyte character crossing the initial-guess block boundary)

As outlined there, the (temp) fix is to put an error handler around the
nextEntryPositionIfFound:ifNone: call in refreshNewBlocksFrom:

Hi Henry,

Thanks for the pointer. I patched my images with your workaround. I will use it for a couple of days and I will later tell if you it seems to fix it or not.

Cheers,

Cheers,
Henry

--
View this message in context: http://forum.world.st/Lots-of-UTF8InvalidText-Invalid-utf8-input-detected-from-OMBU-tp4956734p4956805.html
Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

--
Mariano
http://marianopeck.wordpress.com

Mariano
http://marianopeck.wordpress.com