[5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

David T. Lewis
Indeed, a text with runs that refer to positions beyond the end of
the string does not seem right. Possibly a bug in text editing that
permits this condition to occur?

Meanwhile, I was looking at your tip about the runStopIndex being 71
for the funky styled text and 70 otherwise. Attached is an update to
CharacterBlockScanner>>characterBlockAtPoint:index:in: that prevents
the out of range condition and fixes the FooClient class comment display.

This might be just a band-aid over a problem elsewhere, but it does
seem to work.

Dave


On Wed, Jul 04, 2018 at 06:55:20PM -0400, Bob Arning wrote:

> A little more...
>
> This problem may ultimately be due to something broken in the text being
> displayed. If I look at the funky class comment, it has these runs
>
> a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font:
> 1} . #() . {a TextEmphasis code: 1} . #()}
>
> The total length of the run sizes is 77, yet the string is only 70 bytes
> long.
>
> If I copy that text out and paste into a workspace, the runs now look like:
>
> a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1}
> . #() . {a TextEmphasis code: 1}}
>
> and the size of the runs matches the size of the string at 70. I wonder
> if other occurrences of this problem might have a similar cause.
>
>
> On 7/4/18 4:20 PM, David T. Lewis wrote:
> >On Wed, Jul 04, 2018 at 03:46:16PM -0400, Bob Arning wrote:
> >>Here is a side-by-side comparison with the styled version on the right
> >>and the same text unstyled on the left. Note the difference in
> >>runStopIndex
> >>
> >I see it now. Thanks!
> >
> >Dave
> >
> >
> >>On 7/4/18 2:49 PM, David T. Lewis wrote:
> >>>I tried reverting #scanCharactersFrom:to:in:rightX:stopConditions:kern:
> >>>to the earlier version (cmm 6/12/2010 11:52), but I still get failures
> >>>with cursor and mouse in the class comment text when browsing the
> >>>FooClient
> >>>class. There must be something else going on here, and it is certainly
> >>>not obvious to me why it is happening only with the FooClient test
> >>>example.
> >>>
> >>>Dave
> >>>
> >>>On Wed, Jul 04, 2018 at 02:10:00PM -0400, Bob Arning wrote:
> >>>>perhaps this will help:
> >>>>
> >>>>This version does not seem to exhibit the error and it limits stopIndex
> >>>>to the size of the string
> >>>>
> >>>>!CharacterScanner methodsFor: 'scanning' stamp: 'cmm 6/12/2010 11:52'!
> >>>>scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
> >>>>rightX stopConditions: stops kern: kernDelta
> >>>>
> >>>>?????? | startEncoding selector |
> >>>>?????? (sourceString isByteString) ifTrue: [^ self
> >>>>basicScanCharactersFrom: startIndex *to: (stopIndex min: sourceString
> >>>>size)* in: sourceString rightX: rightX stopConditions: stops kern:
> >>>>kernDelta.].
> >>>>
> >>>>?????? (sourceString isWideString) ifTrue: [
> >>>>?????? ?????? startIndex > stopIndex ifTrue: [lastIndex := stopIndex. ^
> >>>>stops endOfRun].
> >>>>?????? ?????? startEncoding :=?? (sourceString at: startIndex)
> >>>>leadingChar.
> >>>>?????? ?????? selector := EncodedCharSet scanSelectorAt: startEncoding.
> >>>>?????? ?????? ^ self perform: selector withArguments: (Array with:
> >>>>startIndex with: stopIndex with: sourceString with: rightX with: stops
> >>>>with: kernDelta).
> >>>>?????? ].
> >>>>
> >>>>?????? ^ stops endOfRun
> >>>>! !
> >>>>
> >>>>This version fails and does not limit the stopIndex.
> >>>>
> >>>>!CharacterScanner methodsFor: 'scanning' stamp: 'nice 10/22/2013 20:49'!
> >>>>scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
> >>>>rightX
> >>>>?????? ^sourceString scanCharactersFrom: startIndex *to: stopIndex*
> >>>>with:
> >>>>self rightX: rightX font: font! !
> >>>>
> >>>>
> >>>>On 7/4/18 11:53 AM, David T. Lewis wrote:
> >>>>>On Wed, Jul 04, 2018 at 02:35:59PM +0200, karl ramberg wrote:
> >>>>>>On Wed, Jul 4, 2018 at 11:10 AM K K Subbu <[hidden email]>
> >>>>>>wrote:
> >>>>>>
> >>>>>>>On Tuesday 03 July 2018 04:40 AM, Tim Johnson wrote:
> >>>>>>>>1. File-in the .cs as attached*.
> >>>>>>>>2. In a system browser, click on the class, then click on the [?]
> >>>>>>>>button
> >>>>>>>>to view class comment.
> >>>>>>>>3. Notice the styled text in the comment.
> >>>>>>>>4. Click around the comment pane a bit.
> >>>>>>>>5. Eventually, see "Error: subscript is out of bounds: 71" appear
> >>>>>>>>over
> >>>>>>>>and over until you interrupt.
> >>>>>>>In step 4, it is sufficient to click once near the bottom of the pane
> >>>>>>>when the cursor changes from I-bar to arrow and then wait. This will
> >>>>>>>trigger the error, but the error window triggers itself recursively
> >>>>>>>around 50 times before stopping and displaying the windows.
> >>>>>>>
> >>>>>>>I added an assert to catch it before recursion:
> >>>>>>>
> >>>>>>>     self assert: lastIndex < sourceString size.
> >>>>>>>
> >>>>>>>The assert window itself throws up another error, but that is a topic
> >>>>>>>for another thread ;-).
> >>>>>>>
> >>>>>>>Regards .. Subbu
> >>>>>>Hi,
> >>>>>>See class comment i NewParagraph for a hint of what happens.
> >>>>>>A empty lineSpan 71 to: 70 is added to the end of the paragraph, when
> >>>>>>you
> >>>>>>double click below the text.
> >>>>>>When the character scanner looks for characters outside of the string
> >>>>>>at
> >>>>>>index 71 it cause a recursion.
> >>>>>>
> >>>>>>Best,
> >>>>>>Karl
> >>>>>>
> >>>>>Good tiip to look at the NewParagraph class comment, there is
> >>>>>apparently
> >>>>>some left over messiness in here somewhere.
> >>>>>
> >>>>>Another way to trigger the bug using the test case with FooClient is
> >>>>>to position the cursor in the comment pane and use keyboard right
> >>>>>cursor
> >>>>>to move to the end of the comment. This causes the same bounds error
> >>>>>but
> >>>>>takes a different path through the code leading to the error.
> >>>>>
> >>>>>Looking through the stack in a debugger after triggering the error with
> >>>>>the mouse, here are a few notes (but no conclusions):
> >>>>>
> >>>>>On mouseUp: we enter Editor>>selectWord then call
> >>>>>Editor>>selectWordLeftDelimiters:rightDelimiters:
> >>>>>which has this:
> >>>>>
> >>>>> "Select the whole text when clicking before first or after last
> >>>>> character"
> >>>>> (here > string size or: [here < 2]) ifTrue: [^self selectFrom: 1 to:
> >>>>> string size].
> >>>>>
> >>>>>where string is the 70 character String derived from the Text.
> >>>>>
> >>>>>At this point we are trying to select from 1 to 70 (the whole string
> >>>>>for
> >>>>>the entire text) in TextEditor>>selectFrom:to:
> >>>>>
> >>>>>This calls Editor>>selectInvisiblyFrom:to: and here the stop index is
> >>>>>set past the size of the string:
> >>>>>
> >>>>>   selectInvisiblyFrom: start to: stop
> >>>>>      "Select the designated characters, inclusive.  Make no visual
> >>>>>      changes."
> >>>>>
> >>>>>      self markIndex: start pointIndex: stop + 1
> >>>>>
> >>>>> From this point we are trying to go from index 1 to index 71 over a
> >>>>>string
> >>>>>of size 70, and eventually end up with the out of bounds exception.
> >>>>>
> >>>>>A clue that this might be accumulation of kludges: Looking at senders
> >>>>>of
> >>>>>selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a
> >>>>>number
> >>>>>of methods that subtract 1 from the stop index when before calling the
> >>>>>method
> >>>>>that adds 1 back to the stop index. This certainly does not have a good
> >>>>>smell.
> >>>>>
> >>>>>It is not clear to me why selectInvisiblyFrom:to: would be adding one
> >>>>>to
> >>>>>the
> >>>>>stop index, but I see that the method stamp is jmv 5/27/2011, which
> >>>>>suggests
> >>>>>to me that we should take a look at Cuis to figure out how this is
> >>>>>intended
> >>>>>to function. Cuis does not show any cases of senders of
> >>>>>selectInvisiblyFrom:to:
> >>>>>that subtract 1 from the stop index, and the Cuis implementation of
> >>>>>selectInvisiblyFrom:to: is that same as Squeak.
> >>>>>
> >>>>>Dave
> >>>>>
> >>>>>
> >
>

>




CharacterBlockScanner-characterBlockAtPointindexin.st (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Chris Muller-4
In reply to this post by David T. Lewis
> The 'browse origin' search does not seem to find it (maybe I'm using it wrong?).

Most likely.  It searches in the repository that's first in teh list
(excepting package-cache, of course) for that package.

My guess is, you still have the "squeak50" repository first.

If you put "trunk" first, then browse origin gives you the below:

How can we make this easier to use so that 'browse origin' can
possibly be useful in cases like this?

________________
Name: Morphic-nice.559
Author: nice
Time: 3 August 2011, 11:47:38.397 pm
UUID: 2d464aec-2c88-4ad0-9f51-6c208b5cdf82
Ancestors: Morphic-nice.558

Let TextEditor be event driven and let us ban sensor usage from it.

Details:

I tried to apply Cuis 3.3 methods where possible and kept Squeakism
where necessary.

There are unused methods that crept into the Editor classes, but we'll
see later about further convergence with Cuis or not...

Like Cuis, I renamed ivar beginTypeInBlock -> beginTypeInIndex because
this is an Integer index, not a CharacterBlock.
Code is already long enough to avoid such traps.
I also removed sensor and the structure of the TextEditor did change a
lot. So it's hard to say if code is reloadable... The inbox will be a
test place.

Like Cuis I removed the typeAheadStream which was useless already
since we recompose after each key stroke.
Instead of such stream, the keyboard event is passed as argument to
editing methods.

Some small divergences:

I did not define TextEditor>>#processKeyboardEvent: but TextEditor>>#keyStroke:
My rationale was that mouseUp: mouseDown: and mouseMove: were not
renamed, so why renaming keyStroke: ?
On the other hand, it would be easier to follow code with different
selectors, and maybe also good for a VM to reduce unecessary
polymorphism.

I did not define TextEditor>>#dispathOn: but
TextEditor>>#dispatchOnKeyboardEvent:
I felt this was more clear... less unecessary polymorphism. My mood
was changing ;)

These above two behaviours have not been unified (Cuis only deal with
256 characters and can offer a dispatch table for all characters, we
can't).

Unlike Cuis I did not implement multi selection (?).
I also kept the oldInterval and otherInterval ivars because I don't
know if I can touch them.
I also kept selectionShowing because I'm totally unaware of these details.

I did not yet imported the SimpleEditor.
Thanks to Juan for leading the process, and cross the fingers to see
if update is possible or require intermediate stages.

>
> But the method originally entered Squeak here:
>
>   Name: Morphic-ar.149
>   Author: ar
>   Time: 6 August 2009, 4:23:12 am
>   UUID: fc14d160-97da-6142-8002-827d24037a57
>   Ancestors: Morphic-jmv.148
>
>   Integrate Cuis text editors. The new classes are Editor and TextEditor
>   (in Morphic-Text Support). TextMorph now honors the preference
>   #userNewEditors to determine whether to use the (old) TextMorphEditor
>   or the (new) TextEditor (shameless plug: pragma preferences rock; check
>   out TextMorph class>>useNewEditors).  There is still a ways to go before
>   this will work but it gives us the starting point to knock down the
>   remaining issues with the editors (mostly keyboard shortcut issues).
>
>
> If you look at senders of selectInvisiblyFrom:to: in Squeak, there are
> several cases where the sender subtracts 1 from the stop index before
> sending. There are no such cases in Cuis.
>
> Dave
>

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Tim Johnson-2
In reply to this post by Bob Arning-2
Hi Bob & David,

Thanks for all your help with this.

The runs are short because in my original fileOut, the class was named MoodleClient, but I hand-edited the .cs to rename it FooClient and to remove all my proprietary methods I'm not yet ready to share.

I did try to explain this in my original post which is now in the distant realm of the thread.  :)

Best,
Tim

On Jul 4, 2018, at 3:55 PM, Bob Arning wrote:

A little more...

This problem may ultimately be due to something broken in the text being displayed. If I look at the funky class comment, it has these runs

a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1} . #()}

The total length of the run sizes is 77, yet the string is only 70 bytes long.

If I copy that text out and paste into a workspace, the runs now look like:

a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1}}

and the size of the runs matches the size of the string at 70. I wonder if other occurrences of this problem might have a similar cause.


On 7/4/18 4:20 PM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 03:46:16PM -0400, Bob Arning wrote:
Here is a side-by-side comparison with the styled version on the right 
and the same text unstyled on the left. Note the difference in runStopIndex

I see it now. Thanks!

Dave


On 7/4/18 2:49 PM, David T. Lewis wrote:
I tried reverting #scanCharactersFrom:to:in:rightX:stopConditions:kern:
to the earlier version (cmm 6/12/2010 11:52), but I still get failures
with cursor and mouse in the class comment text when browsing the FooClient
class. There must be something else going on here, and it is certainly
not obvious to me why it is happening only with the FooClient test
example.

Dave

On Wed, Jul 04, 2018 at 02:10:00PM -0400, Bob Arning wrote:
perhaps this will help:

This version does not seem to exhibit the error and it limits stopIndex
to the size of the string

!CharacterScanner methodsFor: 'scanning' stamp: 'cmm 6/12/2010 11:52'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX stopConditions: stops kern: kernDelta

?????? | startEncoding selector |
?????? (sourceString isByteString) ifTrue: [^ self
basicScanCharactersFrom: startIndex *to: (stopIndex min: sourceString
size)* in: sourceString rightX: rightX stopConditions: stops kern:
kernDelta.].

?????? (sourceString isWideString) ifTrue: [
?????? ?????? startIndex > stopIndex ifTrue: [lastIndex := stopIndex. ^
stops endOfRun].
?????? ?????? startEncoding :=?? (sourceString at: startIndex) 
leadingChar.
?????? ?????? selector := EncodedCharSet scanSelectorAt: startEncoding.
?????? ?????? ^ self perform: selector withArguments: (Array with:
startIndex with: stopIndex with: sourceString with: rightX with: stops
with: kernDelta).
?????? ].

?????? ^ stops endOfRun
! !

This version fails and does not limit the stopIndex.

!CharacterScanner methodsFor: 'scanning' stamp: 'nice 10/22/2013 20:49'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX
?????? ^sourceString scanCharactersFrom: startIndex *to: stopIndex* with:
self rightX: rightX font: font! !


On 7/4/18 11:53 AM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 02:35:59PM +0200, karl ramberg wrote:
On Wed, Jul 4, 2018 at 11:10 AM K K Subbu [hidden email] wrote:

On Tuesday 03 July 2018 04:40 AM, Tim Johnson wrote:
1. File-in the .cs as attached*.
2. In a system browser, click on the class, then click on the [?] 
button
to view class comment.
3. Notice the styled text in the comment.
4. Click around the comment pane a bit.
5. Eventually, see "Error: subscript is out of bounds: 71" appear over
and over until you interrupt.
In step 4, it is sufficient to click once near the bottom of the pane
when the cursor changes from I-bar to arrow and then wait. This will
trigger the error, but the error window triggers itself recursively
around 50 times before stopping and displaying the windows.

I added an assert to catch it before recursion:

    self assert: lastIndex < sourceString size.

The assert window itself throws up another error, but that is a topic
for another thread ;-).

Regards .. Subbu
Hi,
See class comment i NewParagraph for a hint of what happens.
A empty lineSpan 71 to: 70 is added to the end of the paragraph, when 
you
double click below the text.
When the character scanner looks for characters outside of the string at
index 71 it cause a recursion.

Best,
Karl

Good tiip to look at the NewParagraph class comment, there is apparently
some left over messiness in here somewhere.

Another way to trigger the bug using the test case with FooClient is
to position the cursor in the comment pane and use keyboard right cursor
to move to the end of the comment. This causes the same bounds error but
takes a different path through the code leading to the error.

Looking through the stack in a debugger after triggering the error with
the mouse, here are a few notes (but no conclusions):

On mouseUp: we enter Editor>>selectWord then call
Editor>>selectWordLeftDelimiters:rightDelimiters:
which has this:

	"Select the whole text when clicking before first or after last
	character"
	(here > string size or: [here < 2]) ifTrue: [^self selectFrom: 1 to:
	string size].

where string is the 70 character String derived from the Text.

At this point we are trying to select from 1 to 70 (the whole string for
the entire text) in TextEditor>>selectFrom:to:

This calls Editor>>selectInvisiblyFrom:to: and here the stop index is
set past the size of the string:

  selectInvisiblyFrom: start to: stop
     "Select the designated characters, inclusive.  Make no visual
     changes."

     self markIndex: start pointIndex: stop + 1

From this point we are trying to go from index 1 to index 71 over a 
string
of size 70, and eventually end up with the out of bounds exception.

A clue that this might be accumulation of kludges: Looking at senders of
selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a 
number
of methods that subtract 1 from the stop index when before calling the
method
that adds 1 back to the stop index. This certainly does not have a good
smell.

It is not clear to me why selectInvisiblyFrom:to: would be adding one to
the
stop index, but I see that the method stamp is jmv 5/27/2011, which
suggests
to me that we should take a look at Cuis to figure out how this is 
intended
to function. Cuis does not show any cases of senders of
selectInvisiblyFrom:to:
that subtract 1 from the stop index, and the Cuis implementation of
selectInvisiblyFrom:to: is that same as Squeak.

Dave



        

      

      

      

    





Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Tim Johnson-2
Sorry:  I should have said my hand-renaming the class in the .cs *may* be the cause of what you found, but not that it necessarily *is* the cause of what you found.

Thanks,
Tim

On Jul 4, 2018, at 10:59 PM, Tim Johnson wrote:

Hi Bob & David,

Thanks for all your help with this.

The runs are short because in my original fileOut, the class was named MoodleClient, but I hand-edited the .cs to rename it FooClient and to remove all my proprietary methods I'm not yet ready to share.

I did try to explain this in my original post which is now in the distant realm of the thread.  :)

Best,
Tim

On Jul 4, 2018, at 3:55 PM, Bob Arning wrote:

A little more...

This problem may ultimately be due to something broken in the text being displayed. If I look at the funky class comment, it has these runs

a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1} . #()}

The total length of the run sizes is 77, yet the string is only 70 bytes long.

If I copy that text out and paste into a workspace, the runs now look like:

a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1}}

and the size of the runs matches the size of the string at 70. I wonder if other occurrences of this problem might have a similar cause.


On 7/4/18 4:20 PM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 03:46:16PM -0400, Bob Arning wrote:
Here is a side-by-side comparison with the styled version on the right 
and the same text unstyled on the left. Note the difference in runStopIndex

I see it now. Thanks!

Dave


On 7/4/18 2:49 PM, David T. Lewis wrote:
I tried reverting #scanCharactersFrom:to:in:rightX:stopConditions:kern:
to the earlier version (cmm 6/12/2010 11:52), but I still get failures
with cursor and mouse in the class comment text when browsing the FooClient
class. There must be something else going on here, and it is certainly
not obvious to me why it is happening only with the FooClient test
example.

Dave

On Wed, Jul 04, 2018 at 02:10:00PM -0400, Bob Arning wrote:
perhaps this will help:

This version does not seem to exhibit the error and it limits stopIndex
to the size of the string

!CharacterScanner methodsFor: 'scanning' stamp: 'cmm 6/12/2010 11:52'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX stopConditions: stops kern: kernDelta

?????? | startEncoding selector |
?????? (sourceString isByteString) ifTrue: [^ self
basicScanCharactersFrom: startIndex *to: (stopIndex min: sourceString
size)* in: sourceString rightX: rightX stopConditions: stops kern:
kernDelta.].

?????? (sourceString isWideString) ifTrue: [
?????? ?????? startIndex > stopIndex ifTrue: [lastIndex := stopIndex. ^
stops endOfRun].
?????? ?????? startEncoding :=?? (sourceString at: startIndex) 
leadingChar.
?????? ?????? selector := EncodedCharSet scanSelectorAt: startEncoding.
?????? ?????? ^ self perform: selector withArguments: (Array with:
startIndex with: stopIndex with: sourceString with: rightX with: stops
with: kernDelta).
?????? ].

?????? ^ stops endOfRun
! !

This version fails and does not limit the stopIndex.

!CharacterScanner methodsFor: 'scanning' stamp: 'nice 10/22/2013 20:49'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX
?????? ^sourceString scanCharactersFrom: startIndex *to: stopIndex* with:
self rightX: rightX font: font! !


On 7/4/18 11:53 AM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 02:35:59PM +0200, karl ramberg wrote:
On Wed, Jul 4, 2018 at 11:10 AM K K Subbu [hidden email] wrote:

On Tuesday 03 July 2018 04:40 AM, Tim Johnson wrote:
1. File-in the .cs as attached*.
2. In a system browser, click on the class, then click on the [?] 
button
to view class comment.
3. Notice the styled text in the comment.
4. Click around the comment pane a bit.
5. Eventually, see "Error: subscript is out of bounds: 71" appear over
and over until you interrupt.
In step 4, it is sufficient to click once near the bottom of the pane
when the cursor changes from I-bar to arrow and then wait. This will
trigger the error, but the error window triggers itself recursively
around 50 times before stopping and displaying the windows.

I added an assert to catch it before recursion:

    self assert: lastIndex < sourceString size.

The assert window itself throws up another error, but that is a topic
for another thread ;-).

Regards .. Subbu
Hi,
See class comment i NewParagraph for a hint of what happens.
A empty lineSpan 71 to: 70 is added to the end of the paragraph, when 
you
double click below the text.
When the character scanner looks for characters outside of the string at
index 71 it cause a recursion.

Best,
Karl

Good tiip to look at the NewParagraph class comment, there is apparently
some left over messiness in here somewhere.

Another way to trigger the bug using the test case with FooClient is
to position the cursor in the comment pane and use keyboard right cursor
to move to the end of the comment. This causes the same bounds error but
takes a different path through the code leading to the error.

Looking through the stack in a debugger after triggering the error with
the mouse, here are a few notes (but no conclusions):

On mouseUp: we enter Editor>>selectWord then call
Editor>>selectWordLeftDelimiters:rightDelimiters:
which has this:

	"Select the whole text when clicking before first or after last
	character"
	(here > string size or: [here < 2]) ifTrue: [^self selectFrom: 1 to:
	string size].

where string is the 70 character String derived from the Text.

At this point we are trying to select from 1 to 70 (the whole string for
the entire text) in TextEditor>>selectFrom:to:

This calls Editor>>selectInvisiblyFrom:to: and here the stop index is
set past the size of the string:

  selectInvisiblyFrom: start to: stop
     "Select the designated characters, inclusive.  Make no visual
     changes."

     self markIndex: start pointIndex: stop + 1

From this point we are trying to go from index 1 to index 71 over a 
string
of size 70, and eventually end up with the out of bounds exception.

A clue that this might be accumulation of kludges: Looking at senders of
selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a 
number
of methods that subtract 1 from the stop index when before calling the
method
that adds 1 back to the stop index. This certainly does not have a good
smell.

It is not clear to me why selectInvisiblyFrom:to: would be adding one to
the
stop index, but I see that the method stamp is jmv 5/27/2011, which
suggests
to me that we should take a look at Cuis to figure out how this is 
intended
to function. Cuis does not show any cases of senders of
selectInvisiblyFrom:to:
that subtract 1 from the stop index, and the Cuis implementation of
selectInvisiblyFrom:to: is that same as Squeak.

Dave



        

      

      

      

    







Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
In reply to this post by Bob Arning-2
On Thursday 05 July 2018 04:25 AM, Bob Arning wrote:

> This problem may ultimately be due to something broken in the text being
> displayed. If I look at the funky class comment, it has these runs
>
> a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font:
> 1} . #() . {a TextEmphasis code: 1} . #()}
>
> The total length of the run sizes is 77, yet the string is only 70 bytes
> long.
> If I copy that text out and paste into a workspace, the runs now look like:
>
> a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1}
> . #() . {a TextEmphasis code: 1}}
>
Could it be that the original string had trailing whitespaces, four with
emphasis and three without any style? perhaps a CRLF->CR reduction?

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
In reply to this post by David T. Lewis
On Thursday 05 July 2018 05:30 AM, David T. Lewis wrote:
> Indeed, a text with runs that refer to positions beyond the end of
> the string does not seem right. Possibly a bug in text editing that
> permits this condition to occur?
I think Bob's right. In fact, the cs need not even be filed in. Just
browsing it in ChangeList is sufficient to trigger the error.
The text ivar produced by the scanner has the wrong runs.

> Meanwhile, I was looking at your tip about the runStopIndex being 71
> for the funky styled text and 70 otherwise. Attached is an update to
> CharacterBlockScanner>>characterBlockAtPoint:index:in: that prevents
> the out of range condition and fixes the FooClient class comment display.

It is the runs array that is incorrect in the text. Should we assert
runs size <= aString size in:

Text>>#setString: aString setRunsChecking: aRunArray
        "Check runs and do the best you can to make them fit..."

        string := aString.
        "check the runs"
        aRunArray ifNil: [^ aString asText].
        (aRunArray isKindOf: RunArray) ifFalse: [^ aString asText].
        aRunArray runs size = aRunArray values size ifFalse: [^ aString asText].
        aRunArray size = aString size ifFalse: [^ aString asText].
        runs := aRunArray

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
In reply to this post by Tim Johnson-2
On Thursday 05 July 2018 11:29 AM, Tim Johnson wrote:
>
> The runs are short because in my original fileOut, the class was named
> MoodleClient, but I hand-edited the .cs to rename it FooClient and to
> remove all my proprietary methods I'm not yet ready to share.
>
> I did try to explain this in my original post which is now in the
> distant realm of the thread.  :)

Me bad :-(. got too carried away by the puzzle. Sorry!

I edited two bytes in place "20 3" to "14 2", so the runs encoding match
the actual string. The error dialogs don't appear anymore. Do you have
another cs that has the error?

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
In reply to this post by David T. Lewis
On Wednesday 04 July 2018 09:23 PM, David T. Lewis wrote:
> A clue that this might be accumulation of kludges: Looking at senders of
> selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a number
> of methods that subtract 1 from the stop index when before calling the method
> that adds 1 back to the stop index. This certainly does not have a good smell.
Looks like different intervals for dealing with selections and
substrings. I see that the names used are start/stop instead of
startIndex/stopIndex. I found this consistent pattern in all methods
that deal with selections.

   stop = (stopIndex - 1)
   stopIndex = (stop + 1)

Regular methods that deal with substrings like
AbstractFont>>#widthOString... and BitBlt>>#primDisplayString... and the
scanner methods iterate over aString with:

    startIndex to: stopIndex

while Editor>>selectionInterval extracts selections with startIndex to:
(stopIndex-1).

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Selected post

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

David T. Lewis
On Thu, Jul 05, 2018 at 02:51:58PM +0530, K K Subbu wrote:

> On Wednesday 04 July 2018 09:23 PM, David T. Lewis wrote:
> >A clue that this might be accumulation of kludges: Looking at senders of
> >selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a number
> >of methods that subtract 1 from the stop index when before calling the
> >method
> >that adds 1 back to the stop index. This certainly does not have a good
> >smell.
> Looks like different intervals for dealing with selections and
> substrings. I see that the names used are start/stop instead of
> startIndex/stopIndex. I found this consistent pattern in all methods
> that deal with selections.
>
>   stop = (stopIndex - 1)
>   stopIndex = (stop + 1)
>
> Regular methods that deal with substrings like
> AbstractFont>>#widthOString... and BitBlt>>#primDisplayString... and the
> scanner methods iterate over aString with:
>
>    startIndex to: stopIndex
>
> while Editor>>selectionInterval extracts selections with startIndex to:
> (stopIndex-1).
>
> Regards .. Subbu
>

Ah yes, that makes sense now. Thank you for the explanation.

Dave
 

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Bob Arning-2
In reply to this post by Tim Johnson-2
Yes, it is the cause and the editor could be a bit more robust in the face of such puzzles. :-)

On 7/5/18 2:01 AM, Tim Johnson wrote:
Sorry:  I should have said my hand-renaming the class in the .cs *may* be the cause of what you found, but not that it necessarily *is* the cause of what you found.

Thanks,
Tim

On Jul 4, 2018, at 10:59 PM, Tim Johnson wrote:

Hi Bob & David,

Thanks for all your help with this.

The runs are short because in my original fileOut, the class was named MoodleClient, but I hand-edited the .cs to rename it FooClient and to remove all my proprietary methods I'm not yet ready to share.

I did try to explain this in my original post which is now in the distant realm of the thread.  :)

Best,
Tim

On Jul 4, 2018, at 3:55 PM, Bob Arning wrote:

A little more...

This problem may ultimately be due to something broken in the text being displayed. If I look at the funky class comment, it has these runs

a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1} . #()}

The total length of the run sizes is 77, yet the string is only 70 bytes long.

If I copy that text out and paste into a workspace, the runs now look like:

a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1}}

and the size of the runs matches the size of the string at 70. I wonder if other occurrences of this problem might have a similar cause.


On 7/4/18 4:20 PM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 03:46:16PM -0400, Bob Arning wrote:
Here is a side-by-side comparison with the styled version on the right 
and the same text unstyled on the left. Note the difference in runStopIndex

I see it now. Thanks!

Dave


On 7/4/18 2:49 PM, David T. Lewis wrote:
I tried reverting #scanCharactersFrom:to:in:rightX:stopConditions:kern:
to the earlier version (cmm 6/12/2010 11:52), but I still get failures
with cursor and mouse in the class comment text when browsing the FooClient
class. There must be something else going on here, and it is certainly
not obvious to me why it is happening only with the FooClient test
example.

Dave

On Wed, Jul 04, 2018 at 02:10:00PM -0400, Bob Arning wrote:
perhaps this will help:

This version does not seem to exhibit the error and it limits stopIndex
to the size of the string

!CharacterScanner methodsFor: 'scanning' stamp: 'cmm 6/12/2010 11:52'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX stopConditions: stops kern: kernDelta

?????? | startEncoding selector |
?????? (sourceString isByteString) ifTrue: [^ self
basicScanCharactersFrom: startIndex *to: (stopIndex min: sourceString
size)* in: sourceString rightX: rightX stopConditions: stops kern:
kernDelta.].

?????? (sourceString isWideString) ifTrue: [
?????? ?????? startIndex > stopIndex ifTrue: [lastIndex := stopIndex. ^
stops endOfRun].
?????? ?????? startEncoding :=?? (sourceString at: startIndex) 
leadingChar.
?????? ?????? selector := EncodedCharSet scanSelectorAt: startEncoding.
?????? ?????? ^ self perform: selector withArguments: (Array with:
startIndex with: stopIndex with: sourceString with: rightX with: stops
with: kernDelta).
?????? ].

?????? ^ stops endOfRun
! !

This version fails and does not limit the stopIndex.

!CharacterScanner methodsFor: 'scanning' stamp: 'nice 10/22/2013 20:49'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX
?????? ^sourceString scanCharactersFrom: startIndex *to: stopIndex* with:
self rightX: rightX font: font! !


On 7/4/18 11:53 AM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 02:35:59PM +0200, karl ramberg wrote:
On Wed, Jul 4, 2018 at 11:10 AM K K Subbu [hidden email] wrote:

On Tuesday 03 July 2018 04:40 AM, Tim Johnson wrote:
1. File-in the .cs as attached*.
2. In a system browser, click on the class, then click on the [?] 
button
to view class comment.
3. Notice the styled text in the comment.
4. Click around the comment pane a bit.
5. Eventually, see "Error: subscript is out of bounds: 71" appear over
and over until you interrupt.
In step 4, it is sufficient to click once near the bottom of the pane
when the cursor changes from I-bar to arrow and then wait. This will
trigger the error, but the error window triggers itself recursively
around 50 times before stopping and displaying the windows.

I added an assert to catch it before recursion:

    self assert: lastIndex < sourceString size.

The assert window itself throws up another error, but that is a topic
for another thread ;-).

Regards .. Subbu
Hi,
See class comment i NewParagraph for a hint of what happens.
A empty lineSpan 71 to: 70 is added to the end of the paragraph, when 
you
double click below the text.
When the character scanner looks for characters outside of the string at
index 71 it cause a recursion.

Best,
Karl

Good tiip to look at the NewParagraph class comment, there is apparently
some left over messiness in here somewhere.

Another way to trigger the bug using the test case with FooClient is
to position the cursor in the comment pane and use keyboard right cursor
to move to the end of the comment. This causes the same bounds error but
takes a different path through the code leading to the error.

Looking through the stack in a debugger after triggering the error with
the mouse, here are a few notes (but no conclusions):

On mouseUp: we enter Editor>>selectWord then call
Editor>>selectWordLeftDelimiters:rightDelimiters:
which has this:

	"Select the whole text when clicking before first or after last
	character"
	(here > string size or: [here < 2]) ifTrue: [^self selectFrom: 1 to:
	string size].

where string is the 70 character String derived from the Text.

At this point we are trying to select from 1 to 70 (the whole string for
the entire text) in TextEditor>>selectFrom:to:

This calls Editor>>selectInvisiblyFrom:to: and here the stop index is
set past the size of the string:

  selectInvisiblyFrom: start to: stop
     "Select the designated characters, inclusive.  Make no visual
     changes."

     self markIndex: start pointIndex: stop + 1

From this point we are trying to go from index 1 to index 71 over a 
string
of size 70, and eventually end up with the out of bounds exception.

A clue that this might be accumulation of kludges: Looking at senders of
selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a 
number
of methods that subtract 1 from the stop index when before calling the
method
that adds 1 back to the stop index. This certainly does not have a good
smell.

It is not clear to me why selectInvisiblyFrom:to: would be adding one to
the
stop index, but I see that the method stamp is jmv 5/27/2011, which
suggests
to me that we should take a look at Cuis to figure out how this is 
intended
to function. Cuis does not show any cases of senders of
selectInvisiblyFrom:to:
that subtract 1 from the stop index, and the Cuis implementation of
selectInvisiblyFrom:to: is that same as Squeak.

Dave










    



Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Bob Arning-2
In reply to this post by David T. Lewis

It's good to remember what startIndex/startBlock/selectionStart and stopIndex/stopBlock/selectionStop are used for. If you have a paragraph with two characters 'ab'

...and the 'a' is selected, then this is what the paragraph is thinking

selectionStart:     a CharacterBlock with index 1 and character $a and rectangle 3@0 corner: [hidden email]
 in a TextLine 1 to: 2
selectionStop:     a CharacterBlock with index 2 and character $b and rectangle 11@0 corner: [hidden email]
 in a TextLine 1 to: 2

...if the cursor is between 'a' and 'b' then this happens

selectionStart:     a CharacterBlock with index 2 and character $b and rectangle 11@0 corner: [hidden email]
 in a TextLine 1 to: 2
selectionStop:     a CharacterBlock with index 2 and character $b and rectangle 11@0 corner: [hidden email]
 in a TextLine 1 to: 2

So, in the first case, it can highlight from the topLeft of selectionStart to the bottomLeft of selectionStop. In the second case, since the two are equal, it just displays the cursor along the left edge of either.

In either case, the stopIndex is one more than last selected character and

selection
    "Answer the text in the paragraph that is currently selected."

    ^paragraph text copyFrom: self startIndex to: self stopIndex - 1

returns the selected characters.

On 7/5/18 7:17 AM, David T. Lewis wrote:
On Thu, Jul 05, 2018 at 02:51:58PM +0530, K K Subbu wrote:
On Wednesday 04 July 2018 09:23 PM, David T. Lewis wrote:
A clue that this might be accumulation of kludges: Looking at senders of
selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a number
of methods that subtract 1 from the stop index when before calling the 
method
that adds 1 back to the stop index. This certainly does not have a good 
smell.
Looks like different intervals for dealing with selections and 
substrings. I see that the names used are start/stop instead of 
startIndex/stopIndex. I found this consistent pattern in all methods 
that deal with selections.

  stop = (stopIndex - 1)
  stopIndex = (stop + 1)

Regular methods that deal with substrings like 
AbstractFont>>#widthOString... and BitBlt>>#primDisplayString... and the 
scanner methods iterate over aString with:

   startIndex to: stopIndex

while Editor>>selectionInterval extracts selections with startIndex to: 
(stopIndex-1).

Regards .. Subbu

Ah yes, that makes sense now. Thank you for the explanation.

Dave
 




Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

David T. Lewis
Thank you for this explanation, it is very helpful.

Dave

On Thu, Jul 05, 2018 at 08:21:14AM -0400, Bob Arning wrote:

> It's good to remember what startIndex/startBlock/selectionStart and
> stopIndex/stopBlock/selectionStop are used for. If you have a paragraph
> with two characters 'ab'
>
> ...and the 'a' is selected, then this is what the paragraph is thinking
>
> selectionStart: ?????? a CharacterBlock with index 1 and character $a and
> rectangle 3@0 corner: 11@18.0
> ??in a TextLine 1 to: 2
> selectionStop: ?????? a CharacterBlock with index 2 and character $b and
> rectangle 11@0 corner: 20@18.0
> ??in a TextLine 1 to: 2
>
> ...if the cursor is between 'a' and 'b' then this happens
>
> selectionStart: ?????? a CharacterBlock with index 2 and character $b and
> rectangle 11@0 corner: 20@18.0
> ??in a TextLine 1 to: 2
> selectionStop: ?????? a CharacterBlock with index 2 and character $b and
> rectangle 11@0 corner: 20@18.0
> ??in a TextLine 1 to: 2
>
> So, in the first case, it can highlight from the topLeft of
> selectionStart to the bottomLeft of selectionStop. In the second case,
> since the two are equal, it just displays the cursor along the left edge
> of either.
>
> In either case, the stopIndex is one more than last selected character and
>
> selection
> ?????? "Answer the text in the paragraph that is currently selected."
>
> ?????? ^paragraph text copyFrom: self startIndex to: self stopIndex - 1
>
> returns the selected characters.
>
> On 7/5/18 7:17 AM, David T. Lewis wrote:
> >On Thu, Jul 05, 2018 at 02:51:58PM +0530, K K Subbu wrote:
> >>On Wednesday 04 July 2018 09:23 PM, David T. Lewis wrote:
> >>>A clue that this might be accumulation of kludges: Looking at senders of
> >>>selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a
> >>>number
> >>>of methods that subtract 1 from the stop index when before calling the
> >>>method
> >>>that adds 1 back to the stop index. This certainly does not have a good
> >>>smell.
> >>Looks like different intervals for dealing with selections and
> >>substrings. I see that the names used are start/stop instead of
> >>startIndex/stopIndex. I found this consistent pattern in all methods
> >>that deal with selections.
> >>
> >>   stop = (stopIndex - 1)
> >>   stopIndex = (stop + 1)
> >>
> >>Regular methods that deal with substrings like
> >>AbstractFont>>#widthOString... and BitBlt>>#primDisplayString... and the
> >>scanner methods iterate over aString with:
> >>
> >>    startIndex to: stopIndex
> >>
> >>while Editor>>selectionInterval extracts selections with startIndex to:
> >>(stopIndex-1).
> >>
> >>Regards .. Subbu
> >>
> >Ah yes, that makes sense now. Thank you for the explanation.
> >
> >Dave
> >  
> >
>

>


Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Tim Johnson-2
In reply to this post by Bob Arning-2
Hi all,

I'm sorry if it seems like I may have wasted a dozen hours of your time on a simple problem of a hand-edited changeset.

However, I honorably swear that (a) this problem was happening to me before FooClient.cs even existed, and (b) this has happened to me in various images maybe 3-6 times over the past few years.  And I'm not too exotic with my images:  I just use Monticello and fileIn/fileOut.

I notice (off-hand) that Monticello actually strips formatting from class comments either during export or import, and changesets do not.

I would like to be able to provide instructions for getting a class into the problem condition, but I haven't found them yet.  I can provide my original class as a changeset without modification, but not for a few days, as I am away from that computer.  

I am trying to reproduce the situation without the need to share a changeset.  This may involve finding the exact source of the styled text which I pasted into the class comment.  (I am trying this in stock 5.1 image, in case improvements to the boilerplate text could have something to do with this.)

My efforts to construct a test case are being hampered, however.  I'll post separately.

Thanks again,
Tim




On Jul 5, 2018, at 4:17 AM, Bob Arning wrote:

Yes, it is the cause and the editor could be a bit more robust in the face of such puzzles. :-)

On 7/5/18 2:01 AM, Tim Johnson wrote:
Sorry:  I should have said my hand-renaming the class in the .cs *may* be the cause of what you found, but not that it necessarily *is* the cause of what you found.

Thanks,
Tim

On Jul 4, 2018, at 10:59 PM, Tim Johnson wrote:

Hi Bob & David,

Thanks for all your help with this.

The runs are short because in my original fileOut, the class was named MoodleClient, but I hand-edited the .cs to rename it FooClient and to remove all my proprietary methods I'm not yet ready to share.

I did try to explain this in my original post which is now in the distant realm of the thread.  :)

Best,
Tim

On Jul 4, 2018, at 3:55 PM, Bob Arning wrote:

A little more...

This problem may ultimately be due to something broken in the text being displayed. If I look at the funky class comment, it has these runs

a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1} . #()}

The total length of the run sizes is 77, yet the string is only 70 bytes long.

If I copy that text out and paste into a workspace, the runs now look like:

a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1}}

and the size of the runs matches the size of the string at 70. I wonder if other occurrences of this problem might have a similar cause.


On 7/4/18 4:20 PM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 03:46:16PM -0400, Bob Arning wrote:
Here is a side-by-side comparison with the styled version on the right 
and the same text unstyled on the left. Note the difference in runStopIndex

I see it now. Thanks!

Dave


On 7/4/18 2:49 PM, David T. Lewis wrote:
I tried reverting #scanCharactersFrom:to:in:rightX:stopConditions:kern:
to the earlier version (cmm 6/12/2010 11:52), but I still get failures
with cursor and mouse in the class comment text when browsing the FooClient
class. There must be something else going on here, and it is certainly
not obvious to me why it is happening only with the FooClient test
example.

Dave

On Wed, Jul 04, 2018 at 02:10:00PM -0400, Bob Arning wrote:
perhaps this will help:

This version does not seem to exhibit the error and it limits stopIndex
to the size of the string

!CharacterScanner methodsFor: 'scanning' stamp: 'cmm 6/12/2010 11:52'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX stopConditions: stops kern: kernDelta

?????? | startEncoding selector |
?????? (sourceString isByteString) ifTrue: [^ self
basicScanCharactersFrom: startIndex *to: (stopIndex min: sourceString
size)* in: sourceString rightX: rightX stopConditions: stops kern:
kernDelta.].

?????? (sourceString isWideString) ifTrue: [
?????? ?????? startIndex > stopIndex ifTrue: [lastIndex := stopIndex. ^
stops endOfRun].
?????? ?????? startEncoding :=?? (sourceString at: startIndex) 
leadingChar.
?????? ?????? selector := EncodedCharSet scanSelectorAt: startEncoding.
?????? ?????? ^ self perform: selector withArguments: (Array with:
startIndex with: stopIndex with: sourceString with: rightX with: stops
with: kernDelta).
?????? ].

?????? ^ stops endOfRun
! !

This version fails and does not limit the stopIndex.

!CharacterScanner methodsFor: 'scanning' stamp: 'nice 10/22/2013 20:49'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX
?????? ^sourceString scanCharactersFrom: startIndex *to: stopIndex* with:
self rightX: rightX font: font! !


On 7/4/18 11:53 AM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 02:35:59PM +0200, karl ramberg wrote:
On Wed, Jul 4, 2018 at 11:10 AM K K Subbu [hidden email] wrote:

On Tuesday 03 July 2018 04:40 AM, Tim Johnson wrote:
1. File-in the .cs as attached*.
2. In a system browser, click on the class, then click on the [?] 
button
to view class comment.
3. Notice the styled text in the comment.
4. Click around the comment pane a bit.
5. Eventually, see "Error: subscript is out of bounds: 71" appear over
and over until you interrupt.
In step 4, it is sufficient to click once near the bottom of the pane
when the cursor changes from I-bar to arrow and then wait. This will
trigger the error, but the error window triggers itself recursively
around 50 times before stopping and displaying the windows.

I added an assert to catch it before recursion:

    self assert: lastIndex < sourceString size.

The assert window itself throws up another error, but that is a topic
for another thread ;-).

Regards .. Subbu
Hi,
See class comment i NewParagraph for a hint of what happens.
A empty lineSpan 71 to: 70 is added to the end of the paragraph, when 
you
double click below the text.
When the character scanner looks for characters outside of the string at
index 71 it cause a recursion.

Best,
Karl

Good tiip to look at the NewParagraph class comment, there is apparently
some left over messiness in here somewhere.

Another way to trigger the bug using the test case with FooClient is
to position the cursor in the comment pane and use keyboard right cursor
to move to the end of the comment. This causes the same bounds error but
takes a different path through the code leading to the error.

Looking through the stack in a debugger after triggering the error with
the mouse, here are a few notes (but no conclusions):

On mouseUp: we enter Editor>>selectWord then call
Editor>>selectWordLeftDelimiters:rightDelimiters:
which has this:

	"Select the whole text when clicking before first or after last
	character"
	(here > string size or: [here < 2]) ifTrue: [^self selectFrom: 1 to:
	string size].

where string is the 70 character String derived from the Text.

At this point we are trying to select from 1 to 70 (the whole string for
the entire text) in TextEditor>>selectFrom:to:

This calls Editor>>selectInvisiblyFrom:to: and here the stop index is
set past the size of the string:

  selectInvisiblyFrom: start to: stop
     "Select the designated characters, inclusive.  Make no visual
     changes."

     self markIndex: start pointIndex: stop + 1

From this point we are trying to go from index 1 to index 71 over a 
string
of size 70, and eventually end up with the out of bounds exception.

A clue that this might be accumulation of kludges: Looking at senders of
selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a 
number
of methods that subtract 1 from the stop index when before calling the
method
that adds 1 back to the stop index. This certainly does not have a good
smell.

It is not clear to me why selectInvisiblyFrom:to: would be adding one to
the
stop index, but I see that the method stamp is jmv 5/27/2011, which
suggests
to me that we should take a look at Cuis to figure out how this is 
intended
to function. Cuis does not show any cases of senders of
selectInvisiblyFrom:to:
that subtract 1 from the stop index, and the Cuis implementation of
selectInvisiblyFrom:to: is that same as Squeak.

Dave










    





Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Bob Arning-2

A useful step would be to check this:

(Text allInstances select: [ :e | e size ~= e runs size]) explore

and see if it finds anything. If people did it periodically (like before and after loading new code from somewhere) we might know if/how these anomalies might be created.


On 7/5/18 12:14 PM, Tim Johnson wrote:
Hi all,

I'm sorry if it seems like I may have wasted a dozen hours of your time on a simple problem of a hand-edited changeset.

However, I honorably swear that (a) this problem was happening to me before FooClient.cs even existed, and (b) this has happened to me in various images maybe 3-6 times over the past few years.  And I'm not too exotic with my images:  I just use Monticello and fileIn/fileOut.

I notice (off-hand) that Monticello actually strips formatting from class comments either during export or import, and changesets do not.

I would like to be able to provide instructions for getting a class into the problem condition, but I haven't found them yet.  I can provide my original class as a changeset without modification, but not for a few days, as I am away from that computer.  

I am trying to reproduce the situation without the need to share a changeset.  This may involve finding the exact source of the styled text which I pasted into the class comment.  (I am trying this in stock 5.1 image, in case improvements to the boilerplate text could have something to do with this.)

My efforts to construct a test case are being hampered, however.  I'll post separately.

Thanks again,
Tim




On Jul 5, 2018, at 4:17 AM, Bob Arning wrote:

Yes, it is the cause and the editor could be a bit more robust in the face of such puzzles. :-)

On 7/5/18 2:01 AM, Tim Johnson wrote:
Sorry:  I should have said my hand-renaming the class in the .cs *may* be the cause of what you found, but not that it necessarily *is* the cause of what you found.

Thanks,
Tim

On Jul 4, 2018, at 10:59 PM, Tim Johnson wrote:

Hi Bob & David,

Thanks for all your help with this.

The runs are short because in my original fileOut, the class was named MoodleClient, but I hand-edited the .cs to rename it FooClient and to remove all my proprietary methods I'm not yet ready to share.

I did try to explain this in my original post which is now in the distant realm of the thread.  :)

Best,
Tim

On Jul 4, 2018, at 3:55 PM, Bob Arning wrote:

A little more...

This problem may ultimately be due to something broken in the text being displayed. If I look at the funky class comment, it has these runs

a RunArray runs: #(1 35 18 20 3) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1} . #()}

The total length of the run sizes is 77, yet the string is only 70 bytes long.

If I copy that text out and paste into a workspace, the runs now look like:

a RunArray runs: #(1 35 18 16) values: {#() . {a TextFontChange font: 1} . #() . {a TextEmphasis code: 1}}

and the size of the runs matches the size of the string at 70. I wonder if other occurrences of this problem might have a similar cause.


On 7/4/18 4:20 PM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 03:46:16PM -0400, Bob Arning wrote:
Here is a side-by-side comparison with the styled version on the right 
and the same text unstyled on the left. Note the difference in runStopIndex

I see it now. Thanks!

Dave


On 7/4/18 2:49 PM, David T. Lewis wrote:
I tried reverting #scanCharactersFrom:to:in:rightX:stopConditions:kern:
to the earlier version (cmm 6/12/2010 11:52), but I still get failures
with cursor and mouse in the class comment text when browsing the FooClient
class. There must be something else going on here, and it is certainly
not obvious to me why it is happening only with the FooClient test
example.

Dave

On Wed, Jul 04, 2018 at 02:10:00PM -0400, Bob Arning wrote:
perhaps this will help:

This version does not seem to exhibit the error and it limits stopIndex
to the size of the string

!CharacterScanner methodsFor: 'scanning' stamp: 'cmm 6/12/2010 11:52'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX stopConditions: stops kern: kernDelta

?????? | startEncoding selector |
?????? (sourceString isByteString) ifTrue: [^ self
basicScanCharactersFrom: startIndex *to: (stopIndex min: sourceString
size)* in: sourceString rightX: rightX stopConditions: stops kern:
kernDelta.].

?????? (sourceString isWideString) ifTrue: [
?????? ?????? startIndex > stopIndex ifTrue: [lastIndex := stopIndex. ^
stops endOfRun].
?????? ?????? startEncoding :=?? (sourceString at: startIndex) 
leadingChar.
?????? ?????? selector := EncodedCharSet scanSelectorAt: startEncoding.
?????? ?????? ^ self perform: selector withArguments: (Array with:
startIndex with: stopIndex with: sourceString with: rightX with: stops
with: kernDelta).
?????? ].

?????? ^ stops endOfRun
! !

This version fails and does not limit the stopIndex.

!CharacterScanner methodsFor: 'scanning' stamp: 'nice 10/22/2013 20:49'!
scanCharactersFrom: startIndex to: stopIndex in: sourceString rightX:
rightX
?????? ^sourceString scanCharactersFrom: startIndex *to: stopIndex* with:
self rightX: rightX font: font! !


On 7/4/18 11:53 AM, David T. Lewis wrote:
On Wed, Jul 04, 2018 at 02:35:59PM +0200, karl ramberg wrote:
On Wed, Jul 4, 2018 at 11:10 AM K K Subbu [hidden email] wrote:

On Tuesday 03 July 2018 04:40 AM, Tim Johnson wrote:
1. File-in the .cs as attached*.
2. In a system browser, click on the class, then click on the [?] 
button
to view class comment.
3. Notice the styled text in the comment.
4. Click around the comment pane a bit.
5. Eventually, see "Error: subscript is out of bounds: 71" appear over
and over until you interrupt.
In step 4, it is sufficient to click once near the bottom of the pane
when the cursor changes from I-bar to arrow and then wait. This will
trigger the error, but the error window triggers itself recursively
around 50 times before stopping and displaying the windows.

I added an assert to catch it before recursion:

    self assert: lastIndex < sourceString size.

The assert window itself throws up another error, but that is a topic
for another thread ;-).

Regards .. Subbu
Hi,
See class comment i NewParagraph for a hint of what happens.
A empty lineSpan 71 to: 70 is added to the end of the paragraph, when 
you
double click below the text.
When the character scanner looks for characters outside of the string at
index 71 it cause a recursion.

Best,
Karl

Good tiip to look at the NewParagraph class comment, there is apparently
some left over messiness in here somewhere.

Another way to trigger the bug using the test case with FooClient is
to position the cursor in the comment pane and use keyboard right cursor
to move to the end of the comment. This causes the same bounds error but
takes a different path through the code leading to the error.

Looking through the stack in a debugger after triggering the error with
the mouse, here are a few notes (but no conclusions):

On mouseUp: we enter Editor>>selectWord then call
Editor>>selectWordLeftDelimiters:rightDelimiters:
which has this:

	"Select the whole text when clicking before first or after last
	character"
	(here > string size or: [here < 2]) ifTrue: [^self selectFrom: 1 to:
	string size].

where string is the 70 character String derived from the Text.

At this point we are trying to select from 1 to 70 (the whole string for
the entire text) in TextEditor>>selectFrom:to:

This calls Editor>>selectInvisiblyFrom:to: and here the stop index is
set past the size of the string:

  selectInvisiblyFrom: start to: stop
     "Select the designated characters, inclusive.  Make no visual
     changes."

     self markIndex: start pointIndex: stop + 1

From this point we are trying to go from index 1 to index 71 over a 
string
of size 70, and eventually end up with the out of bounds exception.

A clue that this might be accumulation of kludges: Looking at senders of
selectInvisiblyFrom:to: (such as TextEditor>>addText:event:) shows a 
number
of methods that subtract 1 from the stop index when before calling the
method
that adds 1 back to the stop index. This certainly does not have a good
smell.

It is not clear to me why selectInvisiblyFrom:to: would be adding one to
the
stop index, but I see that the method stamp is jmv 5/27/2011, which
suggests
to me that we should take a look at Cuis to figure out how this is 
intended
to function. Cuis does not show any cases of senders of
selectInvisiblyFrom:to:
that subtract 1 from the stop index, and the Cuis implementation of
selectInvisiblyFrom:to: is that same as Squeak.

Dave















    



Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
In reply to this post by Bob Arning-2
Thank you very much for this lucid explanation. I have added it to

  http://wiki.squeak.org/squeak/2229 (ParagraphEditor)

Regards .. Subbu

On Thursday 05 July 2018 05:51 PM, Bob Arning wrote:

> It's good to remember what startIndex/startBlock/selectionStart and
> stopIndex/stopBlock/selectionStop are used for. If you have a paragraph
> with two characters 'ab'
>
> ...and the 'a' is selected, then this is what the paragraph is thinking
>
> selectionStart:     a CharacterBlock with index 1 and character $a and
> rectangle 3@0 corner: 11@18.0
>   in a TextLine 1 to: 2
> selectionStop:     a CharacterBlock with index 2 and character $b and
> rectangle 11@0 corner: 20@18.0
>   in a TextLine 1 to: 2
>
> ...if the cursor is between 'a' and 'b' then this happens
>
> selectionStart:     a CharacterBlock with index 2 and character $b and
> rectangle 11@0 corner: 20@18.0
>   in a TextLine 1 to: 2
> selectionStop:     a CharacterBlock with index 2 and character $b and
> rectangle 11@0 corner: 20@18.0
>   in a TextLine 1 to: 2
>
> So, in the first case, it can highlight from the topLeft of
> selectionStart to the bottomLeft of selectionStop. In the second case,
> since the two are equal, it just displays the cursor along the left edge
> of either.
>
> In either case, the stopIndex is one more than last selected character and
>
> selection
>      "Answer the text in the paragraph that is currently selected."
>
>      ^paragraph text copyFrom: self startIndex to: self stopIndex - 1
>
> returns the selected characters.

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
In reply to this post by Bob Arning-2
On Thursday 05 July 2018 10:09 PM, Bob Arning wrote:
> A useful step would be to check this:
>
> (Text allInstances select: [ :e | e size ~= e runs size]) explore
>
> and see if it finds anything. If people did it periodically (like before
> and after loading new code from somewhere) we might know if/how these
> anomalies might be created.
>
Text drops bad runs silently in:

setString: aString setRunsChecking: aRunArray
        ...
        aRunArray size = aString size ifFalse: [^ aString asText].
        runs := aRunArray

This method is called from PositionableStream>>#nextChunkText but not
from #nextChunkTextFromStream: methods of converter used by
MultiByteFileStream.

Should these methods be changed to check runs? or should we catch the
error in MultiByteStream>>#nextChunkText post conversion?

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

Bob Arning-2

Well, that drops styling altogether if it is not exactly right. You could try to patch up the runs with:

aRunArray size > aString size ifTrue: [aRunArray := aRunArray copy from: 1 to aString size].

aRunArray size < aString size ifTrue: [aRunArray  addLast: {} times: aString size - aRunArray size].


On 7/5/18 2:32 PM, K K Subbu wrote:
On Thursday 05 July 2018 10:09 PM, Bob Arning wrote:
A useful step would be to check this:

(Text allInstances select: [ :e | e size ~= e runs size]) explore

and see if it finds anything. If people did it periodically (like before and after loading new code from somewhere) we might know if/how these anomalies might be created.

Text drops bad runs silently in:

setString: aString setRunsChecking: aRunArray
    ...
    aRunArray size = aString size ifFalse: [^ aString asText].
    runs := aRunArray

This method is called from PositionableStream>>#nextChunkText but not from #nextChunkTextFromStream: methods of converter used by MultiByteFileStream.

Should these methods be changed to check runs? or should we catch the error in MultiByteStream>>#nextChunkText post conversion?

Regards .. Subbu




Reply | Threaded
Open this post in threaded view
|

Re: [5.2a] styled text in a class comment spawns endless subscript out-of-bounds

K K Subbu
On Friday 06 July 2018 12:19 AM, Bob Arning wrote:
> Well, that drops styling altogether if it is not exactly right. You
> could try to patch up the runs with:
>
> aRunArray size > aString size ifTrue: [aRunArray := aRunArray copy from:
> 1 to aString size].
>
> aRunArray size < aString size ifTrue: [aRunArray  addLast: {} times:
> aString size - aRunArray size].
>

Good idea. I will upload a changeset into Inbox.

Also, #setString:setRunsChecking: and #setString:setRuns: are
categorized as private but are being sent directly from other classes
like converters and streams.

Should I change senders to send #string:runs: instead and merge these
two methods? Are there any implications for efficiency if the check is
done at every Text initialization?

Thanks and Regards .. Subbu

12