translated considered harmful (in its current implementation)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

translated considered harmful (in its current implementation)

Eliot Miranda-2
Hi All,

    I have occasion to simulate the simulator (chancing down a context manipulation bug).  In doing so I notice that there seems to be some awful ) N^2 behaviour in doing things as straight-forward as setting up the world menu.  Here's a back trace from the simulator of a current trunk image:

  16r3061C8 M Dictionary>associationsDo: 16r16BC028: a(n) Dictionary
  16r306200 M Dictionary>keysDo: 16r16BC028: a(n) Dictionary
  16r306238 M [] in Dictionary>keys 16r16BC028: a(n) Dictionary
  16r306280 M Array class(SequenceableCollection class)>new:streamContents: 16r16FA068: a(n) Array class
  16r3062C0 M Dictionary>keys 16r16BC028: a(n) Dictionary
  16r3062F0 M PackageOrganizer>packageNames 16r1540BE8: a(n) PackageOrganizer
  16r306320 M TextDomainManager class>allKnownDomains 16r188A848: a(n) TextDomainManager class
  16r306358 M ByteString(String)>translatedInAllDomains 16r3479FF8: a(n) ByteString
  16r306390 M ByteString(String)>translated 16r3479FF8: a(n) ByteString
  16r3063D8 M [] in TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303F38 M Array(SequenceableCollection)>do: 16rB681C8: a(n) Array
  16r303F70 M TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303FC0 I TheWorldMenu>addSaveAndQuit: 16r3C93C0: a(n) TheWorldMenu

I.e. for every item in the world menu we are enumerating over all packages.  Shurely shome mistake, hic!, ed.
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

marcel.taeumel
Hi Eliot.

Hmm... in 2010 there was a change in String >> #translated to also include some kind of domain knowledge based on "thisContext sender method". (I suppose it got merged with all Etoys changes in Squeak 5.1).

['world' translated] bench.
---NO DOMAIN LOOKUP --- '847,000 per second. 1.18 microseconds per run. 3.69926 % GC time.'
---DO DOMAIN LOOKUP  --- '9,190 per second. 109 microseconds per run. 3.57928 % GC time.'

Given that there are 1558 senders in a Trunk image, this might be worth improving. :-)

I propose to change #translatedInAllDomains, which is kind of a "detect" looking desperately for some translation that has not the given form.... enumerating all package names along the way to be used as domain names.

Domain names are for GetText support. Each .mo file can represent a different domain. Such a modular separation helps external tools (and humans) to better browse and understand the kind of translation that is needed. It's rather common practice, I suppose.

Maybe we should just support a wildcard '*' as valid domain name and then let the GetTextTranslator figure out how to enumerate all the accessible .mo files?

String >> translatedInAllDomains
^self translatedTo: LocaleID current inDomain: '*'

['world' translated] bench.
---WILDCARD---  '408,000 per second. 2.45 microseconds per run. 1.55938 % GC time.'

Now somebody has to change GetTextTranslator. I hope it does not break any generic GetText support to enumerate all .mo files in a locale directory.

Best,
Marcel

Am 28.10.2020 06:59:07 schrieb Eliot Miranda <[hidden email]>:

Hi All,

    I have occasion to simulate the simulator (chancing down a context manipulation bug).  In doing so I notice that there seems to be some awful ) N^2 behaviour in doing things as straight-forward as setting up the world menu.  Here's a back trace from the simulator of a current trunk image:

  16r3061C8 M Dictionary>associationsDo: 16r16BC028: a(n) Dictionary
  16r306200 M Dictionary>keysDo: 16r16BC028: a(n) Dictionary
  16r306238 M [] in Dictionary>keys 16r16BC028: a(n) Dictionary
  16r306280 M Array class(SequenceableCollection class)>new:streamContents: 16r16FA068: a(n) Array class
  16r3062C0 M Dictionary>keys 16r16BC028: a(n) Dictionary
  16r3062F0 M PackageOrganizer>packageNames 16r1540BE8: a(n) PackageOrganizer
  16r306320 M TextDomainManager class>allKnownDomains 16r188A848: a(n) TextDomainManager class
  16r306358 M ByteString(String)>translatedInAllDomains 16r3479FF8: a(n) ByteString
  16r306390 M ByteString(String)>translated 16r3479FF8: a(n) ByteString
  16r3063D8 M [] in TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303F38 M Array(SequenceableCollection)>do: 16rB681C8: a(n) Array
  16r303F70 M TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303FC0 I TheWorldMenu>addSaveAndQuit: 16r3C93C0: a(n) TheWorldMenu

I.e. for every item in the world menu we are enumerating over all packages.  Shurely shome mistake, hic!, ed.
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

codefrau
The mechanism assumes that basically all translation files are loaded so a match is found quickly. There probably should be an early return if there are no translations (as is the case with trunk images, I think?).

- Vanessa -

On Wed, Oct 28, 2020 at 1:53 AM Marcel Taeumel <[hidden email]> wrote:
Hi Eliot.

Hmm... in 2010 there was a change in String >> #translated to also include some kind of domain knowledge based on "thisContext sender method". (I suppose it got merged with all Etoys changes in Squeak 5.1).

['world' translated] bench.
---NO DOMAIN LOOKUP --- '847,000 per second. 1.18 microseconds per run. 3.69926 % GC time.'
---DO DOMAIN LOOKUP  --- '9,190 per second. 109 microseconds per run. 3.57928 % GC time.'

Given that there are 1558 senders in a Trunk image, this might be worth improving. :-)

I propose to change #translatedInAllDomains, which is kind of a "detect" looking desperately for some translation that has not the given form.... enumerating all package names along the way to be used as domain names.

Domain names are for GetText support. Each .mo file can represent a different domain. Such a modular separation helps external tools (and humans) to better browse and understand the kind of translation that is needed. It's rather common practice, I suppose.

Maybe we should just support a wildcard '*' as valid domain name and then let the GetTextTranslator figure out how to enumerate all the accessible .mo files?

String >> translatedInAllDomains
^self translatedTo: LocaleID current inDomain: '*'

['world' translated] bench.
---WILDCARD---  '408,000 per second. 2.45 microseconds per run. 1.55938 % GC time.'

Now somebody has to change GetTextTranslator. I hope it does not break any generic GetText support to enumerate all .mo files in a locale directory.

Best,
Marcel

Am 28.10.2020 06:59:07 schrieb Eliot Miranda <[hidden email]>:

Hi All,

    I have occasion to simulate the simulator (chancing down a context manipulation bug).  In doing so I notice that there seems to be some awful ) N^2 behaviour in doing things as straight-forward as setting up the world menu.  Here's a back trace from the simulator of a current trunk image:

  16r3061C8 M Dictionary>associationsDo: 16r16BC028: a(n) Dictionary
  16r306200 M Dictionary>keysDo: 16r16BC028: a(n) Dictionary
  16r306238 M [] in Dictionary>keys 16r16BC028: a(n) Dictionary
  16r306280 M Array class(SequenceableCollection class)>new:streamContents: 16r16FA068: a(n) Array class
  16r3062C0 M Dictionary>keys 16r16BC028: a(n) Dictionary
  16r3062F0 M PackageOrganizer>packageNames 16r1540BE8: a(n) PackageOrganizer
  16r306320 M TextDomainManager class>allKnownDomains 16r188A848: a(n) TextDomainManager class
  16r306358 M ByteString(String)>translatedInAllDomains 16r3479FF8: a(n) ByteString
  16r306390 M ByteString(String)>translated 16r3479FF8: a(n) ByteString
  16r3063D8 M [] in TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303F38 M Array(SequenceableCollection)>do: 16rB681C8: a(n) Array
  16r303F70 M TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303FC0 I TheWorldMenu>addSaveAndQuit: 16r3C93C0: a(n) TheWorldMenu

I.e. for every item in the world menu we are enumerating over all packages.  Shurely shome mistake, hic!, ed.
_,,,^..^,,,_
best, Eliot



Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

marcel.taeumel
Hi Vanessa.

There probably should be an early return if there are no translations (as is the case with trunk images, I think?).

Last time the translations where updated (or merged from Etoys) was with the 5.1 release. Tim (tfel) also used some automation to do that. Since then, we have added more #translated calls and corrected existing ones as well. So, even in a 5.2 oder 5.3 release, there are likely to be a lot of misses and thus a lot of calls to #translatedInAllDomains.

Well, instead of supporting the '*' domain, we could also just remove that behavior. Not optimal but a quick workaround. Not sure how often that helps anyway. For example, having a "Yes -> Ja" translation for one domain might as well apply to another domain. But that itself looks like a workaround for poor translation jobs. :-D

Best,
Marcel

Am 28.10.2020 19:49:31 schrieb Vanessa Freudenberg <[hidden email]>:

The mechanism assumes that basically all translation files are loaded so a match is found quickly. There probably should be an early return if there are no translations (as is the case with trunk images, I think?).

- Vanessa -

On Wed, Oct 28, 2020 at 1:53 AM Marcel Taeumel <[hidden email]> wrote:
Hi Eliot.

Hmm... in 2010 there was a change in String >> #translated to also include some kind of domain knowledge based on "thisContext sender method". (I suppose it got merged with all Etoys changes in Squeak 5.1).

['world' translated] bench.
---NO DOMAIN LOOKUP --- '847,000 per second. 1.18 microseconds per run. 3.69926 % GC time.'
---DO DOMAIN LOOKUP  --- '9,190 per second. 109 microseconds per run. 3.57928 % GC time.'

Given that there are 1558 senders in a Trunk image, this might be worth improving. :-)

I propose to change #translatedInAllDomains, which is kind of a "detect" looking desperately for some translation that has not the given form.... enumerating all package names along the way to be used as domain names.

Domain names are for GetText support. Each .mo file can represent a different domain. Such a modular separation helps external tools (and humans) to better browse and understand the kind of translation that is needed. It's rather common practice, I suppose.

Maybe we should just support a wildcard '*' as valid domain name and then let the GetTextTranslator figure out how to enumerate all the accessible .mo files?

String >> translatedInAllDomains
^self translatedTo: LocaleID current inDomain: '*'

['world' translated] bench.
---WILDCARD---  '408,000 per second. 2.45 microseconds per run. 1.55938 % GC time.'

Now somebody has to change GetTextTranslator. I hope it does not break any generic GetText support to enumerate all .mo files in a locale directory.

Best,
Marcel

Am 28.10.2020 06:59:07 schrieb Eliot Miranda <[hidden email]>:

Hi All,

    I have occasion to simulate the simulator (chancing down a context manipulation bug).  In doing so I notice that there seems to be some awful ) N^2 behaviour in doing things as straight-forward as setting up the world menu.  Here's a back trace from the simulator of a current trunk image:

  16r3061C8 M Dictionary>associationsDo: 16r16BC028: a(n) Dictionary
  16r306200 M Dictionary>keysDo: 16r16BC028: a(n) Dictionary
  16r306238 M [] in Dictionary>keys 16r16BC028: a(n) Dictionary
  16r306280 M Array class(SequenceableCollection class)>new:streamContents: 16r16FA068: a(n) Array class
  16r3062C0 M Dictionary>keys 16r16BC028: a(n) Dictionary
  16r3062F0 M PackageOrganizer>packageNames 16r1540BE8: a(n) PackageOrganizer
  16r306320 M TextDomainManager class>allKnownDomains 16r188A848: a(n) TextDomainManager class
  16r306358 M ByteString(String)>translatedInAllDomains 16r3479FF8: a(n) ByteString
  16r306390 M ByteString(String)>translated 16r3479FF8: a(n) ByteString
  16r3063D8 M [] in TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303F38 M Array(SequenceableCollection)>do: 16rB681C8: a(n) Array
  16r303F70 M TheWorldMenu>fillIn:from: 16r3C93C0: a(n) TheWorldMenu
  16r303FC0 I TheWorldMenu>addSaveAndQuit: 16r3C93C0: a(n) TheWorldMenu

I.e. for every item in the world menu we are enumerating over all packages.  Shurely shome mistake, hic!, ed.
_,,,^..^,,,_
best, Eliot



Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

timrowledge
Wow. That's some expensive code.
Dare I ask why it tries to set a property on a method?

Is this a place where A Dreaded Preference might actually be sensible?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: D: Detonate



Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

marcel.taeumel
Hi Tim.

Dare I ask why it tries to set a property on a method?

A cache because package-name lookup for methods is expensive:

m := Morph >> #drawOn:.
[PackageOrganizer default packageOfMethod: m methodReference ifNone: []] bench.
--- '3,580 per second. 279 microseconds per run. 0.41983 % GC time.'

Almost as expensive as that not optimized #translatedInAllDomains ^__^

['Hello World' translatedInAllDomains] bench.
--- '9,250 per second. 108 microseconds per run. 3.19872 % GC time.'

Best,
Marcel

Am 29.10.2020 20:41:02 schrieb tim Rowledge <[hidden email]>:

Wow. That's some expensive code.
Dare I ask why it tries to set a property on a method?

Is this a place where A Dreaded Preference might actually be sensible?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: D: Detonate





Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

timrowledge


> On 2020-11-02, at 8:17 AM, Marcel Taeumel <[hidden email]> wrote:
>
> Hi Tim.
>
> > Dare I ask why it tries to set a property on a method?
>
> A cache because package-name lookup for methods is expensive:

OK; I'm not brave enough to want to know why that is required. I think I'll just put a preference in for my systems and avoid the entire problem!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or



Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

marcel.taeumel
Or just recompile #translated to do nothing in your systems. :-)

Best,
Marcel

Am 02.11.2020 20:00:18 schrieb tim Rowledge <[hidden email]>:



> On 2020-11-02, at 8:17 AM, Marcel Taeumel wrote:
>
> Hi Tim.
>
> > Dare I ask why it tries to set a property on a method?
>
> A cache because package-name lookup for methods is expensive:

OK; I'm not brave enough to want to know why that is required. I think I'll just put a preference in for my systems and avoid the entire problem!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or





Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

timrowledge


> On 2020-11-03, at 12:29 AM, Marcel Taeumel <[hidden email]> wrote:
>
> Or just recompile #translated to do nothing in your systems. :-)

Nowhere near complicated enough...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Klingon Code Warrior:- 5) "Specs are for the weak and timid!"



Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

Eliot Miranda-2
In reply to this post by marcel.taeumel


On Tue, Nov 3, 2020 at 12:29 AM Marcel Taeumel <[hidden email]> wrote:
Or just recompile #translated to do nothing in your systems. :-)

This suggestion fills me with dread.  Localization seems to me to be an increasingly important facility for a development environment.  And if we support it so badly that the only way to make it acceptable is to turn it off we have failed.  Further, it can't be that hard, can it?  Is our real problem that we don't have a specification against which we can optimize?


Best,
Marcel

Am 02.11.2020 20:00:18 schrieb tim Rowledge <[hidden email]>:



> On 2020-11-02, at 8:17 AM, Marcel Taeumel wrote:
>
> Hi Tim.
>
> > Dare I ask why it tries to set a property on a method?
>
> A cache because package-name lookup for methods is expensive:

OK; I'm not brave enough to want to know why that is required. I think I'll just put a preference in for my systems and avoid the entire problem!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or






--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

marcel.taeumel
Hi Eliot.

Further, it can't be that hard, can it? Is our real problem that we don't have a specification against which we can optimize?

I gave two simple solutions for #translateInAllDomains further up in this discussion. It's doable.

Best,
Marcel

Am 03.11.2020 20:34:53 schrieb Eliot Miranda <[hidden email]>:



On Tue, Nov 3, 2020 at 12:29 AM Marcel Taeumel <[hidden email]> wrote:
Or just recompile #translated to do nothing in your systems. :-)

This suggestion fills me with dread.  Localization seems to me to be an increasingly important facility for a development environment.  And if we support it so badly that the only way to make it acceptable is to turn it off we have failed.  Further, it can't be that hard, can it?  Is our real problem that we don't have a specification against which we can optimize?


Best,
Marcel

Am 02.11.2020 20:00:18 schrieb tim Rowledge <[hidden email]>:



> On 2020-11-02, at 8:17 AM, Marcel Taeumel wrote:
>
> Hi Tim.
>
> > Dare I ask why it tries to set a property on a method?
>
> A cache because package-name lookup for methods is expensive:

OK; I'm not brave enough to want to know why that is required. I think I'll just put a preference in for my systems and avoid the entire problem!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or






--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: translated considered harmful (in its current implementation)

marcel.taeumel
Hi all!

Should be better now. Update to 20069. 

Best,
Marcel

Am 04.11.2020 09:08:51 schrieb Marcel Taeumel <[hidden email]>:

Hi Eliot.

Further, it can't be that hard, can it? Is our real problem that we don't have a specification against which we can optimize?

I gave two simple solutions for #translateInAllDomains further up in this discussion. It's doable.

Best,
Marcel

Am 03.11.2020 20:34:53 schrieb Eliot Miranda <[hidden email]>:



On Tue, Nov 3, 2020 at 12:29 AM Marcel Taeumel <[hidden email]> wrote:
Or just recompile #translated to do nothing in your systems. :-)

This suggestion fills me with dread.  Localization seems to me to be an increasingly important facility for a development environment.  And if we support it so badly that the only way to make it acceptable is to turn it off we have failed.  Further, it can't be that hard, can it?  Is our real problem that we don't have a specification against which we can optimize?


Best,
Marcel

Am 02.11.2020 20:00:18 schrieb tim Rowledge <[hidden email]>:



> On 2020-11-02, at 8:17 AM, Marcel Taeumel wrote:
>
> Hi Tim.
>
> > Dare I ask why it tries to set a property on a method?
>
> A cache because package-name lookup for methods is expensive:

OK; I'm not brave enough to want to know why that is required. I think I'll just put a preference in for my systems and avoid the entire problem!


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: IO: Illogical Or






--
_,,,^..^,,,_
best, Eliot