String Interpolation

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Sven Van Caekenberghe-2


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>


Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Guillermo Polito
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.
 - and we should escape it

On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Sven Van Caekenberghe-2
In reply to this post by Nicolas Cellier
Nicolas,

> On 28 Sep 2017, at 23:56, Nicolas Cellier <[hidden email]> wrote:
>
> Hi Sven,
> for now it works because the context into which the block is executed still has full access to the variables and receiver of outer context from which the block is declared.
>
> But it's not necessarily the case in all dialects.
> In VW for example, I get nil instead of foobar
> #('loop on value 1 in nil' 'loop on value 2 in nil' 'loop on value 3 in nil')
>
> This is because the optimized block has its own method (a CompiledBlock) and a restricted context (a BlockContext).
> The receiver is the BlockClosure and this closure has no copiedValues from the outerContext and an outerContext set to nil because some analyzer in the compilation phase thought the closure would never access the outerContext (no return to outerContext) nor any of its variable.
>
> Once we'll have clean blocks - depending on the implementation - things might change in Pharo too.
> But we must ask Clement on this subject (I add not taken time to check the implementation he proposes).

I think I understand: you basically say that the compiler could remove/optimise away local/temp variables if it decides they are not used, hence accessing them dynamically will fail. This could be compiler/dialect specific.

That being said, such behaviour would also make the debugger look strange (assigned local/temp is in the source code but not in the context).

In Pharo, even the following (currently) works fine:

  [ :x :y | '{x}+{y}={x+y}' interpolate ] value: 7 value: 8.

Sven

> 2017-09-28 19:03 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
>
>
> > On 28 Sep 2017, at 18:50, Nicolas Cellier <[hidden email]> wrote:
> >
> >
> >
> > 2017-09-28 16:20 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
> > Hi,
> >
> > I got into a little office discussion about string interpolation as it is done in different programming languages.
> >
> > In Pharo we have String>>#format: which is pretty nice. It works as follows:
> >
> > | x y |
> > x := 123.
> > y := #foo.
> > 'x={1} and y={2}' format: { x. y }.
> >
> > It is also possible to use a dictionary with keys, like this:
> >
> > | x y |
> > x := 123.
> > y := #foo.
> > 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
> >
> > But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
> >
> > Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
> >
> > | x y |
> > x := 123.
> > y := #foo.
> > 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
> >
> >  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
> >
> > Here is the implementation I came up with:
> >
> > String>>#interpolate
> >   "Format the receiver by interpolating the evaluation of expressions
> >   in between curly brackets in the context of the sender as in the following 3 oneline examples.
> >   'Today is {Date today}' interpolate.
> >   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
> >   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
> >
> >   | senderContext |
> >   senderContext := thisContext sender.
> >   ^ self class new: self size streamContents: [ :out | | stream |
> >       stream := self readStream.
> >       [ stream atEnd ] whileFalse: [ | currentChar |
> >         (currentChar := stream next) == ${
> >           ifTrue: [ | expression result |
> >             expression := stream upTo: $}.
> >             result := Compiler new
> >               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
> >             out nextPutAll: result asString ]
> >           ifFalse: [
> >             currentChar == $\
> >               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
> >               ifFalse: [ out nextPut: currentChar ] ] ] ]
> >
> > It is a hack that could certainly be improved. And there is of course an obvious security problem.
> >
> > Thoughts ?
> >
> > Sven
> >
> >
> > Nice!
> > The only objection I see is that it may fail in blocks if they don't know that they have to refer to outer context, especially once we have clean blocks
>
> Yes, there are probably some edge cases. Error handling is tricky too.
>
> > | outer |
> > outer := 'foobar'.
> > ^#( 1 2 3 ) collect: [:x | 'loop on value {x} in {outer}' interpolate]
>
> That example works for me in a Playground. How does it fail for you ?
>


Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Damien Pollet
In reply to this post by Guillermo Polito
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Denis Kudriashov
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet

Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Denis Kudriashov
2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

Language designers do not care about it of course :)
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet


Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Denis Kudriashov
In reply to this post by Denis Kudriashov

2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

And I forgot debugger. It should be possible to step over "interpolated expressions"
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet


Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Henrik Sperre Johansen
In reply to this post by Sven Van Caekenberghe-2
Sven Van Caekenberghe-2 wrote
> Nicolas,
>
>> On 28 Sep 2017, at 23:56, Nicolas Cellier &lt;

> nicolas.cellier.aka.nice@

> &gt; wrote:
>>
>> Hi Sven,
>> for now it works because the context into which the block is executed
>> still has full access to the variables and receiver of outer context from
>> which the block is declared.
>>
>> But it's not necessarily the case in all dialects.
>> In VW for example, I get nil instead of foobar
>> #('loop on value 1 in nil' 'loop on value 2 in nil' 'loop on value 3 in
>> nil')
>>
>> This is because the optimized block has its own method (a CompiledBlock)
>> and a restricted context (a BlockContext).
>> The receiver is the BlockClosure and this closure has no copiedValues
>> from the outerContext and an outerContext set to nil because some
>> analyzer in the compilation phase thought the closure would never access
>> the outerContext (no return to outerContext) nor any of its variable.
>>
>> Once we'll have clean blocks - depending on the implementation - things
>> might change in Pharo too.
>> But we must ask Clement on this subject (I add not taken time to check
>> the implementation he proposes).
>
> I think I understand: you basically say that the compiler could
> remove/optimise away local/temp variables if it decides they are not used,
> hence accessing them dynamically will fail. This could be compiler/dialect
> specific.
>
> That being said, such behaviour would also make the debugger look strange
> (assigned local/temp is in the source code but not in the context).
>
> In Pharo, even the following (currently) works fine:
>
>   [ :x :y | '{x}+{y}={x+y}' interpolate ] value: 7 value: 8.
>
> Sven

Not quite; a clean blocks defining feature is that there's no references to
any variables defined in an outer scope. In other words, a block where all
variables used are either block params, or defined locally in the block. So
the above example is a clean block, and would still work.
Removing temps identified as unused, is an orthagonal optimization.
It's possible to define even more types of blocks based on the defining
scope used (for instance, if a block only uses instance scoped vars/self
calls, it's essentially an anonymous method, and could be cached as such),
but clean/unclean is the most useful distinction.

Like Denis said though, there's really quite a few tools you'd need to
update if you want them to function "correctly" with string interpolation.
Opinions differ, but at least to me, those tools being (mostly*) reliable is
worth more than a less verbose version of format:. If one could have one's
cake and eat it too, however...
(For reference, I just spent some time writing a translator from a custom
application-defined executable string format -> standard blocks for exactly
this reason, once you lose reliable senders/implementors/refactorings, it
can become hard to evolve an application in a confident manner)

Cheers,
Henry

* We already have things like perform: aPartial, selector of course



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Guillermo Polito
In reply to this post by Denis Kudriashov
if the compiler plugin correctly models such special syntax with special AST nodes, that could be even possible without much effort

On Tue, Oct 3, 2017 at 5:42 PM, Denis Kudriashov <[hidden email]> wrote:

2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

And I forgot debugger. It should be possible to step over "interpolated expressions"
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Thierry Goubier


2017-10-04 9:50 GMT+02:00 Guillermo Polito <[hidden email]>:
if the compiler plugin correctly models such special syntax with special AST nodes, that could be even possible without much effort

Interesting. Would that imply that by having those special ast nodes, we would get the decompilation working for the debugger?

Thierry


 

On Tue, Oct 3, 2017 at 5:42 PM, Denis Kudriashov <[hidden email]> wrote:

2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

And I forgot debugger. It should be possible to step over "interpolated expressions"
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13


Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Igor Stasenko
IMO, best would be to make it via compiler plugin.

Same, as i proposed for object literals,
a compiler could detect a pattern  <string literal> interpolate
and replace it  at compile time to produce/expand it into proper context-aware semantic, something like this:

{'a' . x . 'b' . y. 'c' .z } gather: #asString

That will allow to eliminate run-time security risk(due to #evaluate:), as well as run-time execution cost, 
and allows a nice and convenient and clean syntax.

Without compiler modification, it's hard to reach best security & performance & convenience ideals.



On 4 October 2017 at 11:06, Thierry Goubier <[hidden email]> wrote:


2017-10-04 9:50 GMT+02:00 Guillermo Polito <[hidden email]>:
if the compiler plugin correctly models such special syntax with special AST nodes, that could be even possible without much effort

Interesting. Would that imply that by having those special ast nodes, we would get the decompilation working for the debugger?

Thierry


 

On Tue, Oct 3, 2017 at 5:42 PM, Denis Kudriashov <[hidden email]> wrote:

2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

And I forgot debugger. It should be possible to step over "interpolated expressions"
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13





--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Sven Van Caekenberghe-2


> On 4 Oct 2017, at 11:36, Igor Stasenko <[hidden email]> wrote:
>
> IMO, best would be to make it via compiler plugin.
>
> Same, as i proposed for object literals,
> a compiler could detect a pattern  <string literal> interpolate

Which means the #interpolate message send is compiled away, right ?
So no guessing by the compiler, no new syntax ?

Sounds like a good idea.

> and replace it  at compile time to produce/expand it into proper context-aware semantic, something like this:
>
> {'a' . x . 'b' . y. 'c' .z } gather: #asString
>
> That will allow to eliminate run-time security risk(due to #evaluate:), as well as run-time execution cost,
> and allows a nice and convenient and clean syntax.
>
> Without compiler modification, it's hard to reach best security & performance & convenience ideals.
>
>
>
> On 4 October 2017 at 11:06, Thierry Goubier <[hidden email]> wrote:
>
>
> 2017-10-04 9:50 GMT+02:00 Guillermo Polito <[hidden email]>:
> if the compiler plugin correctly models such special syntax with special AST nodes, that could be even possible without much effort
>
> Interesting. Would that imply that by having those special ast nodes, we would get the decompilation working for the debugger?
>
> Thierry
>
>
>  
>
> On Tue, Oct 3, 2017 at 5:42 PM, Denis Kudriashov <[hidden email]> wrote:
>
> 2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
> Hi.
>
> While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it
>
> And I forgot debugger. It should be possible to step over "interpolated expressions"
>  
>
> 2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
> On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
> Why not having an opal plugin?
>
> The opal plugin may read strings in the form:
>
> "lalala {some expression} lololo"
>
> and replace at compile time that by:
>
> "lalala {1} lololo" format { some expression }
>
> If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:
>
> String streamContents: [:str |
>     str
>         nextPutAll: 'lalala ';
>         nextPutAll: (some expression) printString;
>         nextPutAll: ' lololo' ]
>
> The thing to think about is what is the delimiter for {some expression}.
>  - a too used one may break lots of existing code.
>
> …or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.
>
>  - and we should escape it
>
> indeed
>  
> On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>
>
> > On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
> >
> > This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.
>
> That is true.
>
> I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.
>
> > Anyway, nice idea.
> >
> > -- Pavel
> >
> > Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> > Hi,
> >
> > I got into a little office discussion about string interpolation as it is done in different programming languages.
> >
> > In Pharo we have String>>#format: which is pretty nice. It works as follows:
> >
> > | x y |
> > x := 123.
> > y := #foo.
> > 'x={1} and y={2}' format: { x. y }.
> >
> > It is also possible to use a dictionary with keys, like this:
> >
> > | x y |
> > x := 123.
> > y := #foo.
> > 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
> >
> > But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
> >
> > Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
> >
> > | x y |
> > x := 123.
> > y := #foo.
> > 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
> >
> >  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
> >
> > Here is the implementation I came up with:
> >
> > String>>#interpolate
> >   "Format the receiver by interpolating the evaluation of expressions
> >   in between curly brackets in the context of the sender as in the following 3 oneline examples.
> >   'Today is {Date today}' interpolate.
> >   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
> >   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
> >
> >   | senderContext |
> >   senderContext := thisContext sender.
> >   ^ self class new: self size streamContents: [ :out | | stream |
> >       stream := self readStream.
> >       [ stream atEnd ] whileFalse: [ | currentChar |
> >         (currentChar := stream next) == ${
> >           ifTrue: [ | expression result |
> >             expression := stream upTo: $}.
> >             result := Compiler new
> >               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
> >             out nextPutAll: result asString ]
> >           ifFalse: [
> >             currentChar == $\
> >               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
> >               ifFalse: [ out nextPut: currentChar ] ] ] ]
> >
> > It is a hack that could certainly be improved. And there is of course an obvious security problem.
> >
> > Thoughts ?
> >
> > Sven
> >
> >
>
>
>
>
>
> --
>    
> Guille Polito
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
> CRIStAL - UMR 9189
> French National Center for Scientific Research - http://www.cnrs.fr
>
> Web: http://guillep.github.io
> Phone: +33 06 52 70 66 13
>
>
>
> --
> Damien Pollet
> type less, do more [ | ] http://people.untyped.org/damien.pollet
>
>
>
>
>
> --
>    
> Guille Polito
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et Automatique de Lille
> CRIStAL - UMR 9189
> French National Center for Scientific Research - http://www.cnrs.fr
>
> Web: http://guillep.github.io
> Phone: +33 06 52 70 66 13
>
>
>
>
> --
> Best regards,
> Igor Stasenko.


Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Denis Kudriashov
In reply to this post by Igor Stasenko
Hi Igor.

Did you see that we have now #asMethodConstant?

DateAndTime now asMethodConst

It is not compiled time but behaviour is very close to it. 
The trick is that first time execution will replace full expression with the result as literal (the receiver of #asMethodConst). So at second time it will be just literal reading. 

2017-10-04 11:36 GMT+02:00 Igor Stasenko <[hidden email]>:
IMO, best would be to make it via compiler plugin.

Same, as i proposed for object literals,
a compiler could detect a pattern  <string literal> interpolate
and replace it  at compile time to produce/expand it into proper context-aware semantic, something like this:

{'a' . x . 'b' . y. 'c' .z } gather: #asString

That will allow to eliminate run-time security risk(due to #evaluate:), as well as run-time execution cost, 
and allows a nice and convenient and clean syntax.

Without compiler modification, it's hard to reach best security & performance & convenience ideals.



On 4 October 2017 at 11:06, Thierry Goubier <[hidden email]> wrote:


2017-10-04 9:50 GMT+02:00 Guillermo Polito <[hidden email]>:
if the compiler plugin correctly models such special syntax with special AST nodes, that could be even possible without much effort

Interesting. Would that imply that by having those special ast nodes, we would get the decompilation working for the debugger?

Thierry


 

On Tue, Oct 3, 2017 at 5:42 PM, Denis Kudriashov <[hidden email]> wrote:

2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

And I forgot debugger. It should be possible to step over "interpolated expressions"
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13





--
Best regards,
Igor Stasenko.

Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Sean P. DeNigris
Administrator
In reply to this post by Igor Stasenko
Igor Stasenko wrote
> IMO, best would be to make it via compiler plugin.

It seems there is always a tension between newbie-friendliness/purity and
max-performance/deployment. There are so many other things about a Smalltalk
image that are insecure and any compiler "tricks" are additional places to
hang up the sizable community that doesn't need that security or efficiency.
Would it be possible to implement it as Sven suggested and then provide a
compiler plugin as part of a deployment hardening process like we used to
have where e.g. tools are disabled, etc?



-----
Cheers,
Sean
--
Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Cheers,
Sean
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Igor Stasenko


On 4 October 2017 at 17:27, Sean P. DeNigris <[hidden email]> wrote:
Igor Stasenko wrote
> IMO, best would be to make it via compiler plugin.

It seems there is always a tension between newbie-friendliness/purity and
max-performance/deployment. There are so many other things about a Smalltalk
image that are insecure and any compiler "tricks" are additional places to
hang up the sizable community that doesn't need that security or efficiency.
Would it be possible to implement it as Sven suggested and then provide a
compiler plugin as part of a deployment hardening process like we used to
have where e.g. tools are disabled, etc?

Of course, you can have both:
- the actual implementation of message as fallback when compiler plugin is not present
- the compiler plugin that does the magic, following same semantics 


-----
Cheers,
Sean



--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Igor Stasenko
In reply to this post by Denis Kudriashov


On 4 October 2017 at 12:46, Denis Kudriashov <[hidden email]> wrote:
Hi Igor.

Did you see that we have now #asMethodConstant?

DateAndTime now asMethodConst

It is not compiled time but behaviour is very close to it. 
The trick is that first time execution will replace full expression with the result as literal (the receiver of #asMethodConst). So at second time it will be just literal reading. 

Aha. Nice. But that useful only for constant expressions, i.e. when you need to reformat static data.
Not quite fits for use case coined by Sven in this topic.

 
2017-10-04 11:36 GMT+02:00 Igor Stasenko <[hidden email]>:
IMO, best would be to make it via compiler plugin.

Same, as i proposed for object literals,
a compiler could detect a pattern  <string literal> interpolate
and replace it  at compile time to produce/expand it into proper context-aware semantic, something like this:

{'a' . x . 'b' . y. 'c' .z } gather: #asString

That will allow to eliminate run-time security risk(due to #evaluate:), as well as run-time execution cost, 
and allows a nice and convenient and clean syntax.

Without compiler modification, it's hard to reach best security & performance & convenience ideals.



On 4 October 2017 at 11:06, Thierry Goubier <[hidden email]> wrote:


2017-10-04 9:50 GMT+02:00 Guillermo Polito <[hidden email]>:
if the compiler plugin correctly models such special syntax with special AST nodes, that could be even possible without much effort

Interesting. Would that imply that by having those special ast nodes, we would get the decompilation working for the debugger?

Thierry


 

On Tue, Oct 3, 2017 at 5:42 PM, Denis Kudriashov <[hidden email]> wrote:

2017-10-03 17:39 GMT+02:00 Denis Kudriashov <[hidden email]>:
Hi.

While idea looks cool it will require a lot of tool support. Senders, var/class references, rename refactorings should be aware of it

And I forgot debugger. It should be possible to step over "interpolated expressions"
 

2017-10-03 17:29 GMT+02:00 Damien Pollet <[hidden email]>:
On 3 October 2017 at 14:07, Guillermo Polito <[hidden email]> wrote:
Why not having an opal plugin?

The opal plugin may read strings in the form:

"lalala {some expression} lololo"

and replace at compile time that by:

"lalala {1} lololo" format { some expression }

If we're going to extend the compiler, we might as avoid parsing at runtime by desugaring more like:

String streamContents: [:str |
    str
        nextPutAll: 'lalala ';
        nextPutAll: (some expression) printString;
        nextPutAll: ' lololo' ]

The thing to think about is what is the delimiter for {some expression}.
 - a too used one may break lots of existing code.

…or we could change the string quotes to mean "dynamic string in which interpolations can be used" and keep single quotes for literal strings only.

 - and we should escape it

indeed
 
On Fri, Sep 29, 2017 at 5:40 AM, Sven Van Caekenberghe <[hidden email]> wrote:


> On 29 Sep 2017, at 08:54, Pavel Krivanek <[hidden email]> wrote:
>
> This solution will not work for environments without sources too where names like t1, t2 are used for temporary variables.

That is true.

I often wonder why we can't keep at least the variables names, it would not be that expensive. There was this problem with FFI that needed source code access as well. It would also help the debugger and make the decompiler more powerful.

> Anyway, nice idea.
>
> -- Pavel
>
> Dne čtvrtek 28. září 2017 Sven Van Caekenberghe <[hidden email]> napsal(a):
> Hi,
>
> I got into a little office discussion about string interpolation as it is done in different programming languages.
>
> In Pharo we have String>>#format: which is pretty nice. It works as follows:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={1} and y={2}' format: { x. y }.
>
> It is also possible to use a dictionary with keys, like this:
>
> | x y |
> x := 123.
> y := #foo.
> 'x={x} and y={y}' format: { #x->x. #y->y } asDictionary.
>
> But this is not true string interpolation as described in [ https://en.wikipedia.org/wiki/String_interpolation ]. The idea is to write the value generating expressions directly inside the strings.
>
> Since in Pharo we add features not by extending the syntax but by adding messages I wondered if it could be done for string interpolation. The goal is to make the following work:
>
> | x y |
> x := 123.
> y := #foo.
> 'It seems x equals {x} and y equals {y} while Pi is still {Float pi}' interpolate.
>
>  => 'It seems x equals 123 and y equals foo while Pi is still 3.141592653589793'
>
> Here is the implementation I came up with:
>
> String>>#interpolate
>   "Format the receiver by interpolating the evaluation of expressions
>   in between curly brackets in the context of the sender as in the following 3 oneline examples.
>   'Today is {Date today}' interpolate.
>   | x | x := 123. 'x equals {x} and pi equals {Float pi}' interpolate.
>   'In {#strings} you can escape \{ by prefixing it with \\' interpolate."
>
>   | senderContext |
>   senderContext := thisContext sender.
>   ^ self class new: self size streamContents: [ :out | | stream |
>       stream := self readStream.
>       [ stream atEnd ] whileFalse: [ | currentChar |
>         (currentChar := stream next) == ${
>           ifTrue: [ | expression result |
>             expression := stream upTo: $}.
>             result := Compiler new
>               evaluate: expression in: senderContext to: nil notifying: nil ifFail: [ ^ nil ] logged: false.
>             out nextPutAll: result asString ]
>           ifFalse: [
>             currentChar == $\
>               ifTrue: [ stream atEnd ifFalse: [ out nextPut: stream next ] ]
>               ifFalse: [ out nextPut: currentChar ] ] ] ]
>
> It is a hack that could certainly be improved. And there is of course an obvious security problem.
>
> Thoughts ?
>
> Sven
>
>





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13




--
Damien Pollet
type less, do more [ | ] http://people.untyped.org/damien.pollet





--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: <a href="tel:+33%206%2052%2070%2066%2013" value="+33652706613" target="_blank">+33 06 52 70 66 13





--
Best regards,
Igor Stasenko.




--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Igor Stasenko
In reply to this post by Igor Stasenko


On 4 October 2017 at 22:30, Igor Stasenko <[hidden email]> wrote:


On 4 October 2017 at 17:27, Sean P. DeNigris <[hidden email]> wrote:
Igor Stasenko wrote
> IMO, best would be to make it via compiler plugin.

It seems there is always a tension between newbie-friendliness/purity and
max-performance/deployment. There are so many other things about a Smalltalk
image that are insecure and any compiler "tricks" are additional places to
hang up the sizable community that doesn't need that security or efficiency.
Would it be possible to implement it as Sven suggested and then provide a
compiler plugin as part of a deployment hardening process like we used to
have where e.g. tools are disabled, etc?

Of course, you can have both:
- the actual implementation of message as fallback when compiler plugin is not present
- the compiler plugin that does the magic, following same semantics 

sorry, i meant 'you MUST have both', not 'you CAN have both' :)
 



--
Best regards,
Igor Stasenko.



--
Best regards,
Igor Stasenko.
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Esteban A. Maringolo
In reply to this post by Denis Kudriashov
2017-10-04 6:46 GMT-03:00 Denis Kudriashov <[hidden email]>:
Hi Igor.

Did you see that we have now #asMethodConstant?

DateAndTime now asMethodConst

It is not compiled time but behaviour is very close to it. 
The trick is that first time execution will replace full expression with the result as literal (the receiver of #asMethodConst). So at second time it will be just literal reading. 


It is weird that a regular message send makes the returned value a method constant.
How is that cleaned up? Do I have to search for all the methods that contain such message send?

Dolphin had the ##() syntax, which basically creates a method constant, but it is evaluated right away.

So you could write the seconds in a day as ##(60*60*24), or, of course, any more complex expression. But the difference is that it is resolved at compile time.

However I don't see how this relates with the String interpolation :) 

Esteban A. Maringolo


 
Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Denis Kudriashov

2017-10-04 21:38 GMT+02:00 Esteban A. Maringolo <[hidden email]>:
2017-10-04 6:46 GMT-03:00 Denis Kudriashov <[hidden email]>:
Hi Igor.

Did you see that we have now #asMethodConstant?

DateAndTime now asMethodConst

It is not compiled time but behaviour is very close to it. 
The trick is that first time execution will replace full expression with the result as literal (the receiver of #asMethodConst). So at second time it will be just literal reading. 


It is weird that a regular message send makes the returned value a method constant.
How is that cleaned up? Do I have to search for all the methods that contain such message send?

What the problem? This message is a tool. You use it when you need it. When you modify the method the value is cleaned.
Nice thing is that it not requires special syntax. And you can debug expression as normal part of method.
 

Dolphin had the ##() syntax, which basically creates a method constant, but it is evaluated right away.

So you could write the seconds in a day as ##(60*60*24), or, of course, any more complex expression. But the difference is that it is resolved at compile time.

However I don't see how this relates with the String interpolation :) 

Esteban A. Maringolo


 

Reply | Threaded
Open this post in threaded view
|

Re: String Interpolation

Esteban A. Maringolo
2017-10-04 16:49 GMT-03:00 Denis Kudriashov <[hidden email]>:
> 2017-10-04 21:38 GMT+02:00 Esteban A. Maringolo <[hidden email]>:

>>> It is not compiled time but behaviour is very close to it.
>>> The trick is that first time execution will replace full expression with
>>> the result as literal (the receiver of #asMethodConst). So at second time it
>>> will be just literal reading.

>> It is weird that a regular message send makes the returned value a method
>> constant.
>> How is that cleaned up? Do I have to search for all the methods that
>> contain such message send?

> What the problem? This message is a tool. You use it when you need it. When
> you modify the method the value is cleaned.
> Nice thing is that it not requires special syntax. And you can debug
> expression as normal part of method.

I don't see any problem, I just said "weird".

It's a "special return" message send. A trick of sorts, no different
than #doesNotUnderstand: or other special selectors.

Regards,

Esteban A. Maringolo

12