Smalltalk › Squeak › Squeak - Dev

[Please Review] Refactoring for #literalsDo: etc.

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

11 messages Options

marcel.taeumel

[Please Review] Refactoring for #literalsDo: etc.

Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?

- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?

- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,

Marcel

literals-do.1.cs (25K) Download Attachment

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi, there.

Take this method for example:

MyFoo >> bar

| u w x y z |

u := [ true ifTrue: [self halt] ifFalse: [ self flag: #boo ] ].
w := [:thing | Morph new].

x := 'Foo'.
y := 42.
z := Array new.

^ #(tree flower cat)

Would this be a useful output for #literals?

{#flag: . #boo . #halt . #ifTrue:ifFalse: . #Morph . Morph . #Morph=>Morph
. 'Foo' . 42 . #Array . Array . #Array=>Array . #tree . #flower . #cat .
#(#tree #flower #cat)}

Or rather this one:

{#flag: . #boo . #halt . #Morph=>Morph . 'Foo' . 42 . #Array=>Array .
#(#tree #flower #cat) . #ifTrue:ifFalse: . #bar . #MyFoo=>MyFoo}

Best,
Marcel

marcel.taeumel wrote

> Hi, there.
>
> Please find attached a change set that (tries to) clean up everything
> related to enumerating or testing literals in CompiledCode,
> CompiledMethod, and CompiledBlock.
>
> I have three important questions:
>
> - The purpose of #hasLiteralThorough is not needed anymore because we
> enumerate and test always in a "thorough" way?
> - Are the enumeration boundaries in CompiledCode, CompiledMethod,
> CompiledBlock in #literalsDo: correct?
> - What is a literal as expected in #hasLiteral: etc. to answer true? Just
> symbols or also bindings (symbol-to-class) and classes themselves?
>
> Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.
>
>
> Best,
> Marcel
>
>
>
> image.png (375K)
> <http://forum.world.st/attachment/5099756/0/image.png>
> literals-do.1.cs (25K)
> <http://forum.world.st/attachment/5099756/1/literals-do.1.cs>

--
Sent from: http://forum.world.st/Squeak-Dev-f45488.html

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi, there.

I will move it to Trunk then and we figure specifics out later. Nothing
should break. Yet, browser icons for #flag: and #halt would then also work
for inner blocks. I noticed some issue there, which got me onto this track
here. :-)

Best,
Marcel

--
Sent from: http://forum.world.st/Squeak-Dev-f45488.html

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

In reply to this post by marcel.taeumel

Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:

EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:

EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,

Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

literals-do.6.cs (41K) Download Attachment

Nicolas Cellier

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi Marcel,

beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :

Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi Nicolas,

thanks. :-) That's correct. It's an extension to high-level test and enumeration. I just discovered that I have to integrate (or update) CompiledCode >> #refersTo:bytecodeScanner:thorough: to respect those special selectors.

Best,

Marcel

Am 24.06.2019 10:56:21 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi all,

next round. Please find attached a changeset:

- senders browsing is only slightly slower but *thorough* all the time

- #hasLiteral: now also works for #>= and other special literals, even though not enumerable

- #literalsDo: enumerates the *raw* literals (for copy etc.) in that byte array (i.e. after header, before first byte code)

- #allLiteralsDo: enumerates the *real* literals, that is, it is thorough but skips special objects such as outer scope, method selector, and class binding -- (for #hasLiteral:, tools etc.)

I just thought about writing some tests for it but those would have to account for V3 vs. SistaV1. Hmmm... maybe just tests for #hasLiteral: and #allLiteralsDo:. Not that low-level #literalsDo:.

Best,

Marcel

Am 24.06.2019 11:27:31 schrieb Marcel Taeumel <[hidden email]>:
Hi Nicolas,

thanks. :-) That's correct. It's an extension to high-level test and enumeration. I just discovered that I have to integrate (or update) CompiledCode >> #refersTo:bytecodeScanner:thorough: to respect those special selectors.

Best,
Marcel

Am 24.06.2019 10:56:21 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

literals-do.7.cs (62K) Download Attachment

Eliot Miranda-2

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi Marcel,

On Mon, Jun 24, 2019 at 5:32 AM Marcel Taeumel <[hidden email]> wrote:

Hi all,

next round. Please find attached a changeset:

thanks. Sorry to be late to this party.

- senders browsing is only slightly slower but *thorough* all the time

I think that's safe. Thorough might be retained as a way of not descending into Array literals and Pragmas. But if you're happy that we always traverse these (and avoid being caught by recursion in circular strucutres0 THEN FINE.

- #hasLiteral: now also works for #>= and other special literals, even though not enumerable
- #literalsDo: enumerates the *raw* literals (for copy etc.) in that byte array (i.e. after header, before first byte code)
- #allLiteralsDo: enumerates the *real* literals, that is, it is thorough but skips special objects such as outer scope, method selector, and class binding -- (for #hasLiteral:, tools etc.)

I just thought about writing some tests for it but those would have to account for V3 vs. SistaV1. Hmmm... maybe just tests for #hasLiteral: and #allLiteralsDo:. Not that low-level #literalsDo:.

Alas yes. ut there are some prototype tests you can use. See LiteralRefLocatorTest. These tests compile two versions of each tested method, in the primary and secondary bytecode sets. So if you start there you'll find that testing both V3PlusClosures and SistaV1 is taken care of.

Best,
Marcel

Am 24.06.2019 11:27:31 schrieb Marcel Taeumel <[hidden email]>:
Hi Nicolas,

thanks. :-) That's correct. It's an extension to high-level test and enumeration. I just discovered that I have to integrate (or update) CompiledCode >> #refersTo:bytecodeScanner:thorough: to respect those special selectors.

Best,
Marcel

Am 24.06.2019 10:56:21 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

_,,,^..^,,,_

best, Eliot

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi Eliot,

thanks for the reply. :-)

Do you think we need a way to not decend into Array literals and Pragmas *but* CompiledCode (i.e., SistaV1)? At the time of writing, my changeset offers #allLiteralsDo: to go all the way down and #literalsDo: to just enumerate the stuff between header and first bytecode.

Best,

Marcel

Am 27.06.2019 00:42:03 schrieb Eliot Miranda <[hidden email]>:

Hi Marcel,

On Mon, Jun 24, 2019 at 5:32 AM Marcel Taeumel <[hidden email]> wrote:
Hi all,

next round. Please find attached a changeset:

thanks. Sorry to be late to this party.

- senders browsing is only slightly slower but *thorough* all the time

I think that's safe. Thorough might be retained as a way of not descending into Array literals and Pragmas. But if you're happy that we always traverse these (and avoid being caught by recursion in circular strucutres0 THEN FINE.

- #hasLiteral: now also works for #>= and other special literals, even though not enumerable
- #literalsDo: enumerates the *raw* literals (for copy etc.) in that byte array (i.e. after header, before first byte code)
- #allLiteralsDo: enumerates the *real* literals, that is, it is thorough but skips special objects such as outer scope, method selector, and class binding -- (for #hasLiteral:, tools etc.)

I just thought about writing some tests for it but those would have to account for V3 vs. SistaV1. Hmmm... maybe just tests for #hasLiteral: and #allLiteralsDo:. Not that low-level #literalsDo:.

Alas yes. ut there are some prototype tests you can use. See LiteralRefLocatorTest. These tests compile two versions of each tested method, in the primary and secondary bytecode sets. So if you start there you'll find that testing both V3PlusClosures and SistaV1 is taken care of.

Best,
Marcel

Am 24.06.2019 11:27:31 schrieb Marcel Taeumel <[hidden email]>:
Hi Nicolas,

thanks. :-) That's correct. It's an extension to high-level test and enumeration. I just discovered that I have to integrate (or update) CompiledCode >> #refersTo:bytecodeScanner:thorough: to respect those special selectors.

Best,
Marcel

Am 24.06.2019 10:56:21 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

--
_,,,^..^,,,_
best, Eliot

marcel.taeumel

Re: [Please Review] Refactoring for #literalsDo: etc.

Hi all! :-)

I think I managed to (somewhat) complete the refactoring of #literalsDo:, #allLiterals, and #hasLiteral:. My goals were as follows:

1. Reduce duplicate enumeration code in the form of "1 to: self numLiterals -1 do:".

2. Fix regresssions that occurred in SistaV1 because of blocks not being inlined at the byte-code level anymore (compared to V3).

3. Understand and clarify the meaning of "literal" in the programming interface.

4. Always enumerate literals thoroughly for #allLiteralsDo: (new) and #hasLiteral:.

Please find attached a changeset to try out. The interface to please review is:

- all implementors of #literalsDo: and #literals and #literalAt: (-> does not decend)

- all implementors of #codeLiteralsDo: and #codeLiterals (-> decends into compiled code objects)

- all implementors of #allLiteralsDo: and #allLiterals (-> decends into compiled code objects and arrays and bindings and pragmas ... to make enumeration "thorough")

- all implementors of #hasLiteral: and #hasLiteralSuchThat: and #literalEqual:

- all implementors of #messagesDo: and #messages and #sendsMessage:

Please also take a look at the (rewritten) tests:

LiteralRefLocatorTest >> #testFindLiteralsInBytecode

LiteralRefLocatorTest >> #testThoroughFindLiteralsInBytecode

The following benchmarks look promising:

[SystemNavigation default allCallsOn: #drawOn:] bench

BEFORE: '41.3 per second. 24.2 milliseconds per run.'

AFTER: '37.7 per second. 26.5 milliseconds per run.'

[SystemNavigation default allSelectorsAndMethodsDo: [:b :s :m | m allLiterals]] bench

BEFORE: '33.1 per second. 30.2 milliseconds per run.'

AFTER: '32 per second. 31.2 milliseconds per run.'

[SystemNavigation default allCallsOnClass: OrderedCollection] bench

BEFORE: '34.9 per second. 28.7 milliseconds per run.'

AFTER: '4.04 per second. 248 milliseconds per run.'

[(PasteUpMorph >> #tryInvokeHalo:) allLiterals] bench

BEFORE: '1,320,000 per second. 759 nanoseconds per run.'

AFTER: '951,000 per second. 1.05 microseconds per run.'

[(PasteUpMorph >> #tryInvokeHalo:) hasLiteral: #second] bench

BEFORE: '3,710,000 per second. 270 nanoseconds per run.'

AFTER: '1,170,000 per second. 855 nanoseconds per run.'

[(Object >> #yourself) allLiterals] bench

BEFORE: '3,570,000 per second. 280 nanoseconds per run.'

AFTER: '3,660,000 per second. 273 nanoseconds per run.'

Here are the things I could not manage to fix or implement yet:

1. Enumerating special selectors via #allLiteralsDo:. Yet, works for #hasLiteral: and #messagesDo: because of using InstructionStream scanning.

2. Enumerating special literals via #allLiteralsDo:. Yet, works for #hasLiteral: because of using InstructionStream scanning.

It would be nice to have a version of BytecodeEncoder class >> #scanBlockOrNilForLiteral: that would be able to just check for any special selector or literal.

Best,

Marcel

Am 27.06.2019 09:31:56 schrieb Marcel Taeumel <[hidden email]>:
Hi Eliot,

thanks for the reply. :-)

Do you think we need a way to not decend into Array literals and Pragmas *but* CompiledCode (i.e., SistaV1)? At the time of writing, my changeset offers #allLiteralsDo: to go all the way down and #literalsDo: to just enumerate the stuff between header and first bytecode.

Best,
Marcel

Am 27.06.2019 00:42:03 schrieb Eliot Miranda <[hidden email]>:

Hi Marcel,

On Mon, Jun 24, 2019 at 5:32 AM Marcel Taeumel <[hidden email]> wrote:
Hi all,

next round. Please find attached a changeset:

thanks. Sorry to be late to this party.

- senders browsing is only slightly slower but *thorough* all the time

I think that's safe. Thorough might be retained as a way of not descending into Array literals and Pragmas. But if you're happy that we always traverse these (and avoid being caught by recursion in circular strucutres0 THEN FINE.

- #hasLiteral: now also works for #>= and other special literals, even though not enumerable
- #literalsDo: enumerates the *raw* literals (for copy etc.) in that byte array (i.e. after header, before first byte code)
- #allLiteralsDo: enumerates the *real* literals, that is, it is thorough but skips special objects such as outer scope, method selector, and class binding -- (for #hasLiteral:, tools etc.)

I just thought about writing some tests for it but those would have to account for V3 vs. SistaV1. Hmmm... maybe just tests for #hasLiteral: and #allLiteralsDo:. Not that low-level #literalsDo:.

Alas yes. ut there are some prototype tests you can use. See LiteralRefLocatorTest. These tests compile two versions of each tested method, in the primary and secondary bytecode sets. So if you start there you'll find that testing both V3PlusClosures and SistaV1 is taken care of.

Best,
Marcel

Am 24.06.2019 11:27:31 schrieb Marcel Taeumel <[hidden email]>:
Hi Nicolas,

thanks. :-) That's correct. It's an extension to high-level test and enumeration. I just discovered that I have to integrate (or update) CompiledCode >> #refersTo:bytecodeScanner:thorough: to respect those special selectors.

Best,
Marcel

Am 24.06.2019 10:56:21 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

--
_,,,^..^,,,_
best, Eliot

literals-do.15.cs (111K) Download Attachment

Eliot Miranda-2

Re: [Please Review] Refactoring for #literalsDo: etc.

In reply to this post by marcel.taeumel

Hi Marcel,

On Jun 27, 2019, at 12:31 AM, Marcel Taeumel <[hidden email]> wrote:

Hi Eliot,

thanks for the reply. :-)

Do you think we need a way to not decend into Array literals and Pragmas *but* CompiledCode (i.e., SistaV1)? At the time of writing, my changeset offers #allLiteralsDo: to go all the way down and #literalsDo: to just enumerate the stuff between header and first bytecode.

I don’t think we need a way not to descend. The only circumstance I can think of where this would be essential is if someone constructed a recursive array literal. Given that that’s not something that arises in everyday code, that large deeply nested array Kieran’s are few and far between and that contemporary performance is good I’d say that keeping it simple outweighs carrying the weight of a thorough: parameter, especially given all the extra machinery needed to support the two bytecode sets.

Best,
Marcel

Am 27.06.2019 00:42:03 schrieb Eliot Miranda <[hidden email]>:

Hi Marcel,

On Mon, Jun 24, 2019 at 5:32 AM Marcel Taeumel <[hidden email]> wrote:
Hi all,

next round. Please find attached a changeset:

thanks. Sorry to be late to this party.

- senders browsing is only slightly slower but *thorough* all the time

I think that's safe. Thorough might be retained as a way of not descending into Array literals and Pragmas. But if you're happy that we always traverse these (and avoid being caught by recursion in circular strucutres0 THEN FINE.

- #hasLiteral: now also works for #>= and other special literals, even though not enumerable
- #literalsDo: enumerates the *raw* literals (for copy etc.) in that byte array (i.e. after header, before first byte code)
- #allLiteralsDo: enumerates the *real* literals, that is, it is thorough but skips special objects such as outer scope, method selector, and class binding -- (for #hasLiteral:, tools etc.)

I just thought about writing some tests for it but those would have to account for V3 vs. SistaV1. Hmmm... maybe just tests for #hasLiteral: and #allLiteralsDo:. Not that low-level #literalsDo:.

Alas yes. ut there are some prototype tests you can use. See LiteralRefLocatorTest. These tests compile two versions of each tested method, in the primary and secondary bytecode sets. So if you start there you'll find that testing both V3PlusClosures and SistaV1 is taken care of.

Best,
Marcel

Am 24.06.2019 11:27:31 schrieb Marcel Taeumel <[hidden email]>:
Hi Nicolas,

thanks. :-) That's correct. It's an extension to high-level test and enumeration. I just discovered that I have to integrate (or update) CompiledCode >> #refersTo:bytecodeScanner:thorough: to respect those special selectors.

Best,
Marcel

Am 24.06.2019 10:56:21 schrieb Nicolas Cellier <[hidden email]>:

Hi Marcel,
beware, there are specialSelectors and inlined selectors too (do: ifTrue: etc...), so it might happen that low level literals enumeration does not fit higher level...

Le lun. 24 juin 2019 à 10:48, Marcel Taeumel <[hidden email]> a écrit :
Hi, there.

Please find attached a new version of this refactoring.

I discovered more recent code for scanning literals (that seems to be at the bytecode level, not object level):

BytecodeEncoder class >> #scanBlockOrNilForLiteral:
EncoderForSistaV1 class >> #scanBlockOrNilForLiteral:
EncoderForV3 class >> #scanBlockOrNilForLiteral:

However, I could not see a way to enumerate literals this way. Did I miss something? Makes me wonder about the entire use of CompiledCode >> #literalsDo:. We could replace #hasLiteral: with an implementation similar to Behavior >> #whichSelectorsreferTo:thorough: using #scanBlockOrNilForLiteral:.

Thoughts? Eliot? :-) How is the conceptional relationship between "has literal" and "literals do"?

Best,
Marcel

Am 28.05.2019 14:49:59 schrieb Marcel Taeumel <[hidden email]>:
Hi, there.

Please find attached a change set that (tries to) clean up everything related to enumerating or testing literals in CompiledCode, CompiledMethod, and CompiledBlock.

I have three important questions:

- The purpose of #hasLiteralThorough is not needed anymore because we enumerate and test always in a "thorough" way?
- Are the enumeration boundaries in CompiledCode, CompiledMethod, CompiledBlock in #literalsDo: correct?
- What is a literal as expected in #hasLiteral: etc. to answer true? Just symbols or also bindings (symbol-to-class) and classes themselves?

Try exploring this result: "(Morph >> #fullDrawOn:) literals" or similar.

Best,
Marcel

--
_,,,^..^,,,_
best, Eliot