Can i has underscores? :)

2010/4/15 Stéphane Rollandin <[hidden email]>:
>> what benefits for keeping old syntax?
>
> backward compatibility; I have around 100000 lines of code to care for...
>

This is a technical issue, which i asked to leave aside.
What else?

> Stef
>

--
Best regards,
Igor Stasenko AKA sig.

> This is a technical issue, which i asked to leave aside.

I won't leave it aside, because this discussion is not theoretical to
me: if you break backward compatibility, you put me in trouble. It's as
simple as that.

Otherwise I have no objection, and never had any objection, to any
change in Smalltalk, Squeak, the French government, the recipe for
pancakes or the US constitution. "Live and let live" is my motto;
please let me live, don't break my code.

If you provide a safe way to convert my packages to your new syntax,
without reformatting them in the process, I have no objection whatsoever
to your ideas.

I would even be happy if that makes you happy :)


Stef

On 15 April 2010 00:38, Juan Vuletich <[hidden email]> wrote: it was not ignored. Not by me :) I downloaded your code once i seen
the message and want to try it out later.
But it is probably will not be integrated before release, so that's
why i think it was not attracted much attention.

>>  - better visual appearance of source code (seeing a := b is a way
>> better than seeing a _ b )
>>
>
> You always had that.
>
Not always. There are still a lot of old code in sources, which using
underscore assignments.
I'd prefer to never stumble over that again :)

> So, what is missing? The import filter to convert existing code to suit your
> taste; the one you're not programming because it is not that important for
> you? Something else?
>

It is not about my taste, it is about a decision i have to make,
what code to generate with my OpenGL stuff. Either use underscores freely or
collapse them and let users to deal with consequences and with
inability to have 1:1
correspondence with OpenGL identifiers.
Try read this one:
COMPRESSED_SIGNED_RED_RGTC1
COMPRESSED_SRGB_ALPHA_S3TC_DXT5_EXT
and this:
COMPRESSEDSIGNEDREDRGTC1
COMPRESSEDSRGBALPHAS3TCDXT5EXT

Yes, these names are ugly as C is. But that's not the point.
The point is, that OpenGL is a widely adopted standard, which i like
to bring to Squeak.
And it is obvious to me, that retaining an original formatting is
better for any potential user.

> Cheers,
> Juan Vuletich
>

--
Best regards,
Igor Stasenko AKA sig.

2010/4/15 Stéphane Rollandin <[hidden email]>: I never said i want to break anyone's code. Please, read what i said again.

> If you provide a safe way to convert my packages to your new syntax, without
> reformatting them in the process, I have no objection whatsoever to your
> ideas.
>
Without reformatting, you mean without altering the spaces/tabs etc,
but just change _ to := in right places?
If that so, then i will start making it right away.

> I would even be happy if that makes you happy :)
>
>
> Stef
>


--
Best regards,
Igor Stasenko AKA sig.

Here the code, Stéphane.

Please, let me know, if it meets your expectations.

On 15 April 2010 01:15, Igor Stasenko <[hidden email]> wrote:

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko wrote:
Good to know!

Shout can do that for you. And if you save such method, it will use your
preference. I use it to convert any method I save in Cuis to use left
arrow :)

Why are you still speaking as if you couldn't use underscores in
identifiers when in fact you can? Or if someone had told you that you
shouldn't? I guess most people (me included) believe it is better to use
them in this case!

Cheers,
Juan Vuletich

On 15 April 2010 03:26, Juan Vuletich <[hidden email]> wrote: Err.. let me tell you what will happen if i, say, add an instance
variable to some basic class -
it will recompile all subclasses. And when some of those methods could
contain underscores as assignment,
while some of them (a newer ones) could use underscores in
selectors/identifiers.
No matter what you choose in a system-wide preference, you won't be
able to recompile the all classes.
Will Shout be able to help me with that? :)
It is obvious, that to prevent this from happening, all sources, which
is going to the compiler, should
use same syntax, or each class should have own #useUnderscores setting.
But that wouldn't help, if i add an instance variable with underscore
in base class,
while subclass will still insist on using underscores for assignment -
then it will be unable to access that ivar
in own methods :)
Well, my first question was 'i can?'. It appeared that i can't (but i
will try your fixes).
Also, how you suppose to deal with the above (recompiling a base class)?

> Cheers,
> Juan Vuletich
>
>



--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko wrote:
>
> Err.. let me tell you what will happen if i, say, add an instance
> variable to some basic class -
> it will recompile all subclasses. And when some of those methods could
> contain underscores as assignment,
> while some of them (a newer ones) could use underscores in
> selectors/identifiers.
>  

Shout has nothing to do with this. Shout is used by PluggableTextMorph
when rendering code. So recompiling existing classes by adding ivars
does nothing to the methods. They are left exactly as they were.
Activating the #syntaxHighlightingAsYouTypeAnsiAssignment preference
only affects how code is shown. If you save a method that was rendered
converting all assignments to ':=', it will be saved that way.

> No matter what you choose in a system-wide preference, you won't be
> able to recompile the all classes.
> Will Shout be able to help me with that? :)
> It is obvious, that to prevent this from happening, all sources, which
> is going to the compiler, should
> use same syntax, or each class should have own #useUnderscores setting.
>  

Not at all. Just tried this:
Preference #allowUnderscoreAssignments set to true.
Preference #allowUnderscoreSelectors set to true.
Preference #syntaxHighlightingAsYouTypeAnsiAssignment set to true.

Then the following method uses underscores both in identifiers and
assignments:
        test_underscore
            | a_a b_b c_c d_d_ |
            a_a _ 1.
            b_b := 2.
            "c_c _3." "Not possible. _3 is taken as an unary selector"
            "d_d_ 4" "Not possible. d_d_ is considered the varname"

And Shout renders it as:
        test_underscore
            | a_a b_b c_c d_d_ |
            a_a := 1.
            b_b := 2.
            "c_c _3." "Not possible. _3 is taken as an unary selector"
            "d_d_ 4" "Not possible. d_d_ is considered the varname"

Shout and the regular Parser can tell whether an underscore is an
assignment or part of an identifier. The only problem is the ambiguity
in lines with c_c and d_d_ variables, that I had to comment, as the
second underscore in each of the is considered part of an identifier and
not an assignment. That could cause problems with very few old methods
that fail to include some whitespace before and after the assignment.
But as the idea is to use Shout to render the methods, when you save one
of these, you'll have to add that whitespace yourself.

> But that wouldn't help, if i add an instance variable with underscore
> in base class,
> while subclass will still insist on using underscores for assignment -
> then it will be unable to access that ivar
> in own methods :)
>
>  

Not true, as the example above shows.

Cheers,
Juan Vuletich

On 4/14/2010 5:22 PM, Igor Stasenko wrote:
> Here the code, Stéphane.
>
> Please, let me know, if it meets your expectations.

Ah! See *now* we're talking. This is useful because we can immediately
put this to a real test along the lines of:

Smalltalk allClasses do:[:cls|
   cls selectorsAndMethodsDo:[:sel :method|
     | source filtered node compiled |
     source := (cls sourceCodeAt: sel) asString.
     (source indexOf: $_) > 0 ifTrue:[
       filtered := (UnderscoreFilter on: source readStream)
         replacement: ':='; upToEnd.
       source = filtered ifFalse:[
         Transcript cr; show: cls name, '>>', sel.
         node := cls compilerClass new compile: filtered
           in: cls classified: nil notifying: nil ifFail: nil..
         compiled := node generate: method trailer.
         self assert: (method equivalentTo: compiled).
       ].
     ].
   ].
] displayingProgress:[:cls| 'Verifying ', cls name].

What this does is verifying that your underscore filter indeed replaces
all the source code correctly. It also spits out where underscores are
used and I was positively surprised that it seems as if there are
precisely 11 methods with underscore assignments left in Squeak 4.1.

The other thing we can do with it is to wrap it into a more useful form
where we 'auto-detect' whether to translate underscores and do so only
if needed. Check out the attached code - it provides the fileIn
facilities in a separate class so that we can properly modify and
install variations by subclassing. Also note that the idea is that
generally we detect the underscore usage by looking at the fileOut
version (which is fast) but if not present we might look at the file
contents, too (this is left as an exercise for the interested reader :-)

BTW, I've also added a test for a fileIn translation that shows a case
where your filter goes wrong - when having something like #(1 2 3 _ 4 5)
you're replacing the underscore in the literal array with := even though
it's equivalent to #'_'.

Between UnderscoreFilter and FileInManager we're slowly getting there.
The issues left are:
* Dealing with is Monticello. If we can figure out how to extract a
SystemVersion string from it we should be able to use the same basic
approach.
* Using the Unicode glyph. I'm not certain what the best way to deal
with this is, but it'd be good if we could also support the 'real'
left-arrow glyph and translate underscore to left-arrow instead of :=.
* Typing the Unicode glyph. This could be done by a simple preference
that maps underscore to left-arrow.

Cheers,
   - Andreas

Le 15/04/2010 02:22, Igor Stasenko a écrit :
> Here the code, Stéphane.
>
> Please, let me know, if it meets your expectations.

Thanks, I'll try it. Does it preserve the dates, also ?

Stef

On 15 April 2010 07:39, Andreas Raab <[hidden email]> wrote: reading files.
My proposal was more explicit - i.e. put something like: Me usingUnderscores. !
But yours also ok , except that some fileouts don't using the 'From
Squeak blabla' headers (see an attachment for instance).
If file contains a fileout version header, then we can make some
assumptions what to expect from its contents.

> BTW, I've also added a test for a fileIn translation that shows a case where
> your filter goes wrong - when having something like #(1 2 3 _ 4 5) you're
> replacing the underscore in the literal array with := even though it's
> equivalent to #'_'.
>

Interesting edge case :)
Btw its inconsistent in current image:
#( _ )
printed as
 #(#':=')

Preference were:  allow underscore assignments = true, allow
underscore selectors = false.

With allow underscore assignments = false, allow underscore selectors = false
it throws an error while parsing such literal:
 Illegal character (char code 95 16r5F) ->

With allow underscore assignments = false, allow underscore selectors = true
it prints
 #(#'_')

in older image (took 3.10.2), it printed as
#(#'_')

I think it should print same results irrespectively of what preference set.
And my pick is:  Illegal character (char code 95 16r5F)
IMO, we should disallow an ambiguous things in syntax.  And #( _ ) is ambiguous.

Another interesting thing, which i found is, that syntax allows an
arbitrary number of spaces between
# and (
so, even:

#  fooo
prints as #fooo

is this correct smalltalk syntax?

Here is updated version, which deals with array literals ( i hope ).

> Between UnderscoreFilter and FileInManager we're slowly getting there. The
> issues left are:
> * Dealing with is Monticello. If we can figure out how to extract a
> SystemVersion string from it we should be able to use the same basic
> approach.

> * Using the Unicode glyph. I'm not certain what the best way to deal with
> this is, but it'd be good if we could also support the 'real' left-arrow
> glyph and translate underscore to left-arrow instead of :=.

Well, first, we certaintly need a better default font support of
unicode characters.
Making it work only for left-arrow will be just another hack, which i
would prefer to not do at all.
Either we doing it good (unicode support), or not doing at all.

> * Typing the Unicode glyph. This could be done by a simple preference that
> maps underscore to left-arrow.
>
Yeah, this is trivial thing :)

> Cheers,
>  - Andreas


--
Best regards,
Igor Stasenko AKA sig.

2010/4/15 Stéphane Rollandin <[hidden email]>:
> Le 15/04/2010 02:22, Igor Stasenko a écrit :
>>
>> Here the code, Stéphane.
>>
>> Please, let me know, if it meets your expectations.
>
> Thanks, I'll try it. Does it preserve the dates, also ?
>
It not deals with dates (i suppose you meant method stamps). So, they
are remain unchanged.

> Stef
>
>
>
>



--
Best regards,
Igor Stasenko AKA sig.

Oops, that was a bit buggy :)

This is an updated one, merged with Andreas.

It works fine with this:

Smalltalk allClasses do:[:cls|
 cls selectorsAndMethodsDo:[:sel :method|
   | source filtered node compiled |
   source := (cls sourceCodeAt: sel) asString.
   (source indexOf: $_) > 0 ifTrue:[
     filtered := (UnderscoreFilter on: source readStream)
       replacement: ':='; upToEnd.
     source = filtered ifFalse:[
       Transcript cr; show: cls name, '>>', sel.
       node := cls compilerClass new compile: filtered
         in: cls classified: nil notifying: nil ifFail: nil..
       compiled := node generate: method trailer.
       self assert: (method equivalentTo: compiled).
     ].
   ].
 ].
] displayingProgress:[:cls| 'Verifying ', cls name].

--
Best regards,
Igor Stasenko AKA sig.

btw, i think that UnderscoreFilter
should be demoted from being a subclass of Stream,
since i found that it won't work with parser anyways.
Parser using an evil skip: -1, skip: -2
when parsing numbers, and it won't go well with filtered input.

Another alternative is to alter the parser to not use a position-able protocol,
only #next and #peek (and maybe skip:, but only with a positive offsets).

I think it would be good to assume for parser, that source stream may
not support positioning.

--
Best regards,
Igor Stasenko AKA sig.

Igor Stasenko wrote:
> ...
> Interesting edge case :)
> Btw its inconsistent in current image:
> #( _ )
> printed as
>  #(#':=')
>  

To fix that, in #xUnderscore, just change the line that reads:
                ^token := #':='
to make it read:
                ^token := #'_'

I already did it for Cuis.

Cheers,
Juan Vuletich