Call for design for a literal programming doc similar to PythonDocTest

Hi all

I want something similar in the spirit to PythonDocTest
https://docs.python.org/2/library/doctest.html

I'm talking about

basename
     "Returns the base of the basename,
         i.e.
         /foo/gloops.taz basename is 'gloops.taz'
         / basename is '/'"

Pragmas do not work well i.e.,
basename
     "Returns the base of the basename"
      <expr: '''/foo/gloops.taz'' asFileReference basename' result:
'gloops.taz'>


We should invent a syntax to be put inside comments and that we can
easily parse because we need to improve
the use and discovery of the library.

I was thinking about

basename
     "Returns the base of the basename"
     "
     '/foo/gloops.taz' asFileReference basename
     >>> 'gloops.taz'
     "

Do you have any idea?

I cannot not do anything and just complain that our methods are not that
well documented.
We as a community should take this and build an super cool system.

I tried and defined >>> on Object to see if it works!

Object >>> aResultingObject
     "If the method comment contains >>> then it is a pharo documentated
test. We can check that it is true."

     "
     '/foo/gloops.taz' asFileReference basename
    >>> 'gloops.taz'
     "

     ^ self = aResultingObject


Stef

Hi,

I think this is an interesting idea.

However, one thing to keep in mind is that one reason why these solutions exist is because they assume no UI environment. Thus, the only thing they have is text and the solution is built around it.

In Pharo, we can think of concatenating information in the environment. If we have a casual connection between examples and the code they exemplify, the interface can provide all this information at the same time. The advantage here is that the syntax remains simple.

At the same time, we also want to have Pillar more thoroughly supported in our tools. Could it be that we could extend Pillar to support literate programming?

Cheers,
Doru


--
www.tudorgirba.com
www.feenk.com

"It's not how it is, it is how we see it."

2016-09-15 17:09 GMT-03:00 Tudor Girba <[hidden email]>:
> However, one thing to keep in mind is that one reason why these solutions exist is because they assume no UI environment. Thus, the only thing they have is text and the solution is built around it.
>
> In Pharo, we can think of concatenating information in the environment. If we have a casual connection between examples and the code they exemplify, the interface can provide all this information at the same time. The advantage here is that the syntax remains simple.

+1

I wouldn't like that syntax, nor PharoDoc (a la JavaDoc), nor anything
"polluting" the code that way.

The comment or examples can be attributes of the method itself, just
like the class comment are today.

Regards!

Esteban A. Maringolo

On Fri, Sep 16, 2016 at 2:45 AM, stepharo <[hidden email]> wrote:
basename
     "Returns the base of the basename"
     """
     '/foo/gloops.taz' asFileReference basename
        ==> 'gloops.taz'
     """
    self size == 0
    "the root node"
        ifTrue: [ ^ '/'].
     ^ self at: self size

I like this idea.  But a fine semantic....
We should think of these less as "tests"
and more as "howto" snippets that are tested for correctness.

They should show just the *simplest* was to get an object to which the
method can be sent.
Maybe they could be also displayed via the gtExample interface ??

cheers -ben

May be. Right now I want to make sure that all the little examples that
are written for example
inside methods are correct.

Stef
>
> cheers -ben
>
>

Le 15/9/16 à 22:16, Esteban A. Maringolo a écrit :
> 2016-09-15 17:09 GMT-03:00 Tudor Girba <[hidden email]>:
>> However, one thing to keep in mind is that one reason why these solutions exist is because they assume no UI environment. Thus, the only thing they have is text and the solution is built around it.
>>
>> In Pharo, we can think of concatenating information in the environment. If we have a casual connection between examples and the code they exemplify, the interface can provide all this information at the same time. The advantage here is that the syntax remains simple.
> +1
>
> I wouldn't like that syntax, nor PharoDoc (a la JavaDoc), nor anything
> "polluting" the code that way.

It is not polution, it is documentation.
Apparently you are not the guy that got the HUGE pain to write the
method comment and the chapter about FileSystem.
I'm probably too stupid. To understand what each method was doing I had
to TRY and ERROR.
I recently try to understand certain behavior of the Date package and
same syndrome. Without an example it is impossible to
get **exactly** what the method is doing.

So that you do not use it in your code is your style and life. Now Pharo
must improve from that perspective.
I do not want to read five methods with different context to understand
that basename includes the extension

  '/foo/gloops.taz' asFileReference basename is 'gloops.taz

and that name does not.

Sorry but as one of the leader of Pharo if I do not have such exigence
and vision for our system then I should go do something else.

Stef

>
> The comment or examples can be attributes of the method itself, just
> like the class comment are today.
>
> Regards!
>
> Esteban A. Maringolo
>
>

I'd be more interested with a package level doc than a class doc or test.

It is at another level and it will come.
The manisfest is here and we should update Nautilus to show other information.

Now I'm curious to see how many people will use that when I see the level of description in configuration of.

Package level doc is quite useful to outline how some things are working together, something which is quite hard to figure out except by reading external doc or inferring things by walking through the code or a running test.

I'm the guy that propose SmallUML to embed description of UML diagram as package level 5 years ago and before that I tried
to add png of diagram so I'm convinced about that.

Having the ability to read about a package would be very useful. Basically, this is what Java provides.

http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/javadoc.html#packagecomment

Phil

On Thu, Sep 15, 2016 at 8:45 PM, stepharo <[hidden email]> wrote:

Hi all

I want something similar in the spirit to PythonDocTest https://docs.python.org/2/library/doctest.html

I'm talking about

basename
    "Returns the base of the basename,
        i.e.
        /foo/gloops.taz basename is 'gloops.taz'
        / basename is '/'"

Pragmas do not work well i.e.,
basename
    "Returns the base of the basename"
     <expr: '''/foo/gloops.taz'' asFileReference basename' result: 'gloops.taz'>


We should invent a syntax to be put inside comments and that we can easily parse because we need to improve
the use and discovery of the library.

I was thinking about

basename
    "Returns the base of the basename"
    "
    '/foo/gloops.taz' asFileReference basename
    >>> 'gloops.taz'
    "

Do you have any idea?

I cannot not do anything and just complain that our methods are not that well documented.
We as a community should take this and build an super cool system.

I tried and defined >>> on Object to see if it works!

Object >>> aResultingObject
    "If the method comment contains >>> then it is a pharo documentated test. We can check that it is true."

    "
    '/foo/gloops.taz' asFileReference basename
   >>> 'gloops.taz'
    "

    ^ self = aResultingObject


Stef

I will play with it :)

I was thinking to have >>> instead of association to make sure that we can disambiguate from ->

Thanks for the protoype.

This kind of little action can change completely the face of Pharo library. I always ***LOVED*** little examples inside method

when I discovered Smalltalk.

This plus examples at the class side and link with nautilus execution -> gt inspector will really improve the situation.

Stef

2016-09-15 20:45 GMT+02:00 stepharo <[hidden email]>:

Hi all

I want something similar in the spirit to PythonDocTest https://docs.python.org/2/library/doctest.html

I'm talking about

basename
    "Returns the base of the basename,
        i.e.
        /foo/gloops.taz basename is 'gloops.taz'
        / basename is '/'"

Pragmas do not work well i.e.,
basename
    "Returns the base of the basename"
     <expr: '''/foo/gloops.taz'' asFileReference basename' result: 'gloops.taz'>


We should invent a syntax to be put inside comments and that we can easily parse because we need to improve
the use and discovery of the library.

I was thinking about

basename
    "Returns the base of the basename"
    "
    '/foo/gloops.taz' asFileReference basename
    >>> 'gloops.taz'
    "

Do you have any idea?

I cannot not do anything and just complain that our methods are not that well documented.
We as a community should take this and build an super cool system.

I tried and defined >>> on Object to see if it works!

Object >>> aResultingObject
    "If the method comment contains >>> then it is a pharo documentated test. We can check that it is true."

    "
    '/foo/gloops.taz' asFileReference basename
   >>> 'gloops.taz'
    "

    ^ self = aResultingObject

Ok, this is a quick hack ( do not look at the code :), yes using regex here is a bit fragil)

You can add code in comments between backticks (`)

The formatter will highlight the text like smalltalk code (or not if it is not valid code).

+ an icon styler with an icon showing a warning icon for faulty code or an information icon otherwise

you can click on the icon,

if the code is an association

expression -> result

it executes the expression and compares it with the result, (with assert:equals: ) opens debugger if it fails and does

nothing otherwise

if the code is just an expression, it opens an inspector.

This is just one way to do some tests and experiments with this idea, don 't yet know if this is a good idea or if

we can / should find a better way to connect code with examples.

first result, I find expressions in comments, highlighted as code, confusing :)

(file in attached cs in a recent pharo 6.0 and look at the method AbstractFileReference>>#baseName . Or

add an expression with backticks in a method comment

` your code here `

 

Stef

Hi nicolai

I was thinking that I would prefer to have

    "` '/foo/gloops.taz' asFileReference basename -> 'gloops.taz'.`"

instead of
    ` '/foo/gloops.taz' asFileReference basename -> 'gloops.taz'.` <- suceeds, no output

So that we can use ` in the future in plain code.
Also it has the advantage that it is backward compatible.

Else we could have

    ``
But I do not know.

Tell me what you think.

I would love to browse the complete system to convert existing one liner with such

Stef

Ok, this is a quick hack ( do not look at the code :), yes using regex here is a bit fragil)

You can add code in comments between backticks (`)

The formatter will highlight the text like smalltalk code (or not if it is not valid code).

+ an icon styler with an icon showing a warning icon for faulty code or an information icon otherwise

you can click on the icon,

if the code is an association

expression -> result

it executes the expression and compares it with the result, (with assert:equals: ) opens debugger if it fails and does

nothing otherwise

if the code is just an expression, it opens an inspector.

This is just one way to do some tests and experiments with this idea, don 't yet know if this is a good idea or if

we can / should find a better way to connect code with examples.

first result, I find expressions in comments, highlighted as code, confusing :)

(file in attached cs in a recent pharo 6.0 and look at the method AbstractFileReference>>#baseName . Or

add an expression with backticks in a method comment

` your code here `

 

Stef

Hi,

I was thinking on the metro way to work, and I also saw that this discussion is actually split in multiple threads, so it was not easy to follow :).

Some of my feelings about this:

- Pragmas are nice because they are easy to "interpret". Parsing them is already provided. However, putting expressions or long examples into them starts to be awkward. It seems it's pushin the syntax too much, isn't it?

- Comments are nice because they are ignored! But right now comments are simple plain text. This thread is to convert *some* comments into executable examples. But this should be just some comments and not all of them, am I right? So what happens when we want to have comments written in Pillar for example? What I mean is that having examples is just an example of something we would like to do with comments.

But we can also imagine having comment interpreters. Something like this:

">>>PillarDoc
!This is a Pillar title
And I'm just a paragraph
And I can link a class like @Object.
"


">>>ExampleDoc
self assert: '/foo/gloops.taz' asFileReference basename equals: 'gloops.taz'
"

What I like from this is that:
  - comments that do not specify an interpreter are just plain old text. Thus backward compatible.
  - comments that specify an interpreter that is not present, are just plain old text. A decoupling as it happens with the settings.
  - an ExampleDoc interpreter can be just a test case instance, thus we could use assertions instead of special >>> messages.

Things that would require more thought:
  - there is an overlap between these interpreted comments and pragmas... There are for sure cases that solutions can be imagined with both.
  - there is an overlap between examples and test cases. I saw many people that argued that already. I am not against examples, but I think we should (whichever implementation is chosen) draw a line and set some guidelines. Stef, probably you have already in mind when you want a test case and when an example and what's the distance. It would be good if we can transform that implicit thought in something explicit (maybe even a lint rule in the future?)

Guille

Hi nicolai

I was thinking that I would prefer to have

    "` '/foo/gloops.taz' asFileReference basename -> 'gloops.taz'.`"

instead of
    ` '/foo/gloops.taz' asFileReference basename -> 'gloops.taz'.` <- suceeds, no output

So that we can use ` in the future in plain code.
Also it has the advantage that it is backward compatible.

Else we could have

    ``
But I do not know.

Tell me what you think.

I would love to browse the complete system to convert existing one liner with such

Stef

Ok, this is a quick hack ( do not look at the code :), yes using regex here is a bit fragil)

You can add code in comments between backticks (`)

The formatter will highlight the text like smalltalk code (or not if it is not valid code).

+ an icon styler with an icon showing a warning icon for faulty code or an information icon otherwise

you can click on the icon,

if the code is an association

expression -> result

it executes the expression and compares it with the result, (with assert:equals: ) opens debugger if it fails and does

nothing otherwise

if the code is just an expression, it opens an inspector.

This is just one way to do some tests and experiments with this idea, don 't yet know if this is a good idea or if

we can / should find a better way to connect code with examples.

first result, I find expressions in comments, highlighted as code, confusing :)

(file in attached cs in a recent pharo 6.0 and look at the method AbstractFileReference>>#baseName . Or

add an expression with backticks in a method comment

` your code here `

 

Stef

On Thu, Sep 15, 2016 at 10:09 PM, Tudor Girba <[hidden email]> wrote:
> Hi,
>
> I think this is an interesting idea.
>
> However, one thing to keep in mind is that one reason why these solutions exist is because they assume no UI environment. Thus, the only thing they have is text and the solution is built around it.
>
> In Pharo, we can think of concatenating information in the environment. If we have a casual connection between examples and the code they exemplify, the interface can provide all this information at the same time. The advantage here is that the syntax remains simple.
>
> At the same time, we also want to have Pillar more thoroughly supported in our tools. Could it be that we could extend Pillar to support literate programming?

Just I backup this idea. You should have a look to http://orgmode.org/
with babel mode that allows to mix different programming languages in
the same document: http://orgmode.org/worg/org-contrib/babel/

Lot of ideas should be borrowed for doing a better Pillar I guess.
The only problem of org-mode is that you don't have the link to the
live environment that we are used to.
Having a org-mode clone in Pharo (i.e Pillar+++) would be great.

At the same time, this is a long term solution.
We also need an immediate solution like the one proposed by Stéphane.

Regards,
--
Serge Stinckwich
UCBN & UMI UMMISCO 209 (IRD/UPMC)
Every DSL ends up being Smalltalk
http://www.doesnotunderstand.org/

Hi,

I was thinking on the metro way to work, and I also saw that this discussion is actually split in multiple threads, so it was not easy to follow :).

Some of my feelings about this:

- Pragmas are nice because they are easy to "interpret". Parsing them is already provided. However, putting expressions or long examples into them starts to be awkward. It seems it's pushin the syntax too much, isn't it?

- Comments are nice because they are ignored! But right now comments are simple plain text. This thread is to convert *some* comments into executable examples. But this should be just some comments and not all of them, am I right? So what happens when we want to have comments written in Pillar for example? What I mean is that having examples is just an example of something we would like to do with comments.

But we can also imagine having comment interpreters. Something like this:

">>>PillarDoc
!This is a Pillar title
And I'm just a paragraph
And I can link a class like @Object.
"


">>>ExampleDoc
self assert: '/foo/gloops.taz' asFileReference basename equals: 'gloops.taz'
"

We could have for class comment

">>>UMLDescription
UMLClass named: 'Visitor'

Arrow from:

But this can start to be complex
"

What I like from this is that:
  - comments that do not specify an interpreter are just plain old text. Thus backward compatible.
  - comments that specify an interpreter that is not present, are just plain old text. A decoupling as it happens with the settings.
  - an ExampleDoc interpreter can be just a test case instance, thus we could use assertions instead of special >>> messages.

I do not know because all the comments should be pillar compatible.
If you do not use pillar commands you just get plain text.
You see we can easily detect comment too: if a text contains >>> then it is a doc

Then
ExampleDoc
is an extra syntax.

I'm not found with the
    self assert:
because it feels like something coming out of nowhere. We will have to explain: yes self is bound to an instance of the testCase...
and to me this is implementation details:
    if I give expr1 and expr2 the implementation can build self assert: expr1 equals: expr2

I liked the simplicity of nicolai' solution
I would just have a comment and a message >>> so that we
    => nearly no syntax change
    => something really lightweight and optional


What I care is that

    - we get simple examples right in the method
    - one line no more.
    - that these examples are correct and always validated (I do not see them as tests), their values is more in the correct documentation
    than the tests. They are basic output. I do not want to have full code in the method, in that case this should go one class side example
    or plain tests.

Things that would require more thought:
  - there is an overlap between these interpreted comments and pragmas... There are for sure cases that solutions can be imagined with both.

I do not think that expressions can fit inside pragmas

  - there is an overlap between examples and test cases. I saw many people that argued that already. I am not against examples, but I think we should (whichever implementation is chosen) draw a line and set some guidelines.

To me I do not care that they are tests.
Their values is active documentation that can be automatically validated. I do not expect to run them but the system
can garantee that they are correct.

Stef, probably you have already in mind when you want a test case and when an example and what's the distance. It would be good if we can transform that implicit thought in something explicit (maybe even a lint rule in the future?)

I do not get what you mean here.

For me a test is a scenario.
    If I do that and that then this happens
    a test can use a class example
   
Now for me a test should not rely one a online part of method comment.    

These onliners are just make the method comments better.

Guille

-------- Original Message --------

Hi nicolai

I was thinking that I would prefer to have

    "` '/foo/gloops.taz' asFileReference basename -> 'gloops.taz'.`"

instead of
    ` '/foo/gloops.taz' asFileReference basename -> 'gloops.taz'.` <- suceeds, no output

So that we can use ` in the future in plain code.
Also it has the advantage that it is backward compatible.

Else we could have

    ``
But I do not know.

Tell me what you think.

I would love to browse the complete system to convert existing one liner with such

Stef

Ok, this is a quick hack ( do not look at the code :), yes using regex here is a bit fragil)

You can add code in comments between backticks (`)

The formatter will highlight the text like smalltalk code (or not if it is not valid code).

+ an icon styler with an icon showing a warning icon for faulty code or an information icon otherwise

you can click on the icon,

if the code is an association

expression -> result

it executes the expression and compares it with the result, (with assert:equals: ) opens debugger if it fails and does

nothing otherwise

if the code is just an expression, it opens an inspector.

This is just one way to do some tests and experiments with this idea, don 't yet know if this is a good idea or if

we can / should find a better way to connect code with examples.

first result, I find expressions in comments, highlighted as code, confusing :)

(file in attached cs in a recent pharo 6.0 and look at the method AbstractFileReference>>#baseName . Or

add an expression with backticks in a method comment

` your code here `

 

Stef