swiki source suited to recent Squeaks

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
40 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
Well, here's a weird thing that I don't recall ever coming up against before.

I've got a basic swiki page component for Seaside, as previously mentioned. Now we have Levente's (original plus small fixes) parser/generator to convert swiki markup to html. I join them together to see what happens ... and weird stuff happens. Testing in a workspace is fine BUT not when getting text back from a browser.

It turns out that the browser (safari and chrome) sends a string with CRLF line ends to us. Now, I'm a long way from keeping up to date with web stuff but really?  I thought we got LFs because unixy-things.

The practical issue is that the grammar provided in PEGParser class>>#grammarWiki has a load of places reliant on \n and so, for example, the Preformatted & Code tags simply get ignored.

Two obvious questions come to mind here
a) what on earth? CRLF? Is that normal or is it an artefact of some Seaside setup I can change?
b) if we need to change the grammar to cope with crlf, what is the best way? I don't find the grammar terribly intuitable and can't spot any rule explanation. I've tried changing the Preformatted rule for example to
Preformatted <- "---\r\n" .{"---\r\n"}
and the parser doesn't even recognise the swiki tags.

I'd hate to have to do a crlf -> lf conversion every time, it seems so inelegant.

Oh - http://code.google.com/p/support/wiki/WikiSyntax (as referenced in #grammarWiki) seems to be a dead page now, which makes it a not very good bit of documentation! Bizarrely there doesn't appear to be much related info found by google.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: CM: Circulate Memory



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Jakob Reschke
Am Di., 13. Aug. 2019 um 22:46 Uhr schrieb tim Rowledge <[hidden email]>:

It turns out that the browser (safari and chrome) sends a string with CRLF line ends to us. Now, I'm a long way from keeping up to date with web stuff but really?  I thought we got LFs because unixy-things.
[...]

Two obvious questions come to mind here
a) what on earth? CRLF? Is that normal or is it an artefact of some Seaside setup I can change?

CRLF is common among Internet text protocols, such as HTTP and SMTP.

For textarea form elements in HTML, see https://stackoverflow.com/questions/14217101/what-character-represents-a-new-line-in-a-text-area Seems like CRLF is up to standards there as well, at least for the representation on the wire.

In general, while HTTP does not mandate the end of line marker for the body of the message (notwithstanding HTML form specs), it says in RFC2616: "When in canonical form, media subtypes of the "text" type use CRLF as the text line break. HTTP relaxes this requirement and allows the transport of text media with plain CR or LF alone representing a line break when it is done consistently for an entire entity-body. HTTP applications MUST accept CRLF, bare CR, and bare LF as being representative of a line break in text media received via HTTP."


Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Levente Uzonyi
In reply to this post by timrowledge
Hi Tim,

On Tue, 13 Aug 2019, tim Rowledge wrote:

> Well, here's a weird thing that I don't recall ever coming up against before.
>
> I've got a basic swiki page component for Seaside, as previously mentioned. Now we have Levente's (original plus small fixes) parser/generator to convert swiki markup to html. I join them together to see what happens ... and weird stuff happens. Testing in a workspace is fine BUT not when getting text back from a browser.

If you use Seaside, its #renderOn: implementations may be clashing with
StreamingHtmlCanvas's. I just checked three such methods in my image
(Seaside2 + StreamingHtmlCanvas loaded at the same time), and
the implementations are the same, so no issue there.

But, if you use Seaside, you might want to consider using TinyWiki.

>
> It turns out that the browser (safari and chrome) sends a string with CRLF line ends to us. Now, I'm a long way from keeping up to date with web stuff but really?  I thought we got LFs because unixy-things.
>
> The practical issue is that the grammar provided in PEGParser class>>#grammarWiki has a load of places reliant on \n and so, for example, the Preformatted & Code tags simply get ignored.
>
> Two obvious questions come to mind here
> a) what on earth? CRLF? Is that normal or is it an artefact of some Seaside setup I can change?
> b) if we need to change the grammar to cope with crlf, what is the best way? I don't find the grammar terribly intuitable and can't spot any rule explanation. I've tried changing the Preformatted rule for example to
> Preformatted <- "---\r\n" .{"---\r\n"}
> and the parser doesn't even recognise the swiki tags.
>
> I'd hate to have to do a crlf -> lf conversion every time, it seems so inelegant.
>
> Oh - http://code.google.com/p/support/wiki/WikiSyntax (as referenced in #grammarWiki) seems to be a dead page now, which makes it a not very good bit of documentation! Bizarrely there doesn't appear to be much related info found by google.
crlf is pretty much an internet thing. Even Squeak's converter method is
called #withInternetLineEndings.

I have attached updated GoogleWikiCompiler and #grammarWiki
implementations. The new grammar should accept crlf, cr and lf line
endings. Weird thing is that \n refers to cr, while \r refers to lf in
PEGParser's grammar, so crlf is \n\r...

The google page is gone, but its content is not:
https://web.archive.org/web/20150418033327/http://code.google.com/p/support/wiki/WikiSyntax

Levente

>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Strange OpCodes: CM: Circulate Memory


PEGParser class-grammarWiki.st (2K) Download Attachment
GoogleWikiCompiler.2.st (15K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Squeak - Dev mailing list

If I may jump in on this conversation.

I have an alpha version of what I call SeasideDoc (I will be uploading to SqueakSource once I get some account stuff resolved)



The landing page gives the roadmap I have in mind.

In the left hand menu, if you click on Swiki, it expands to all the current Swiki pages which appear to display just fine.

The methodology was that I scraped the wiki content and stored it in classes in the image.  The classes are subclasses of the standard Seaside renderable component that I call 'doclets'

My apologies if this is irrelevant to what you are trying to accomplish.





Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
In reply to this post by Levente Uzonyi


> <PEGParser class-grammarWiki.st><GoogleWikiCompiler.2.st>

Thank you for the 'fixed' grammar. It even makes sort-of-sense to me, though I couldn't replicate it myself.
In my current setup the code/pre/list etc stuff is now working nicely. With a tiny bit of furniture it will make a very good start to a usable wiki.

The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- Has a one-way ticket on the Disoriented Express.



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Chris Cunnington-4


On Aug 14, 2019, at 6:15 PM, tim Rowledge <[hidden email]> wrote:

The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?

No. SqueakSource since forever has had a wiki for every project. It’s called TinyWiki. It’s the kernel of Lukas Reggli’s Pier wiki CMS. 
It requires SmaCC, which Levente says can be loaded in 5.2. I guess if you change SmalltalkImage>>#associationAt:ifAbsent: to Smalltalk globals or something. 
TinyWiki uses the Visitor pattern and is gem. You can download it here [1]. You’ll need to add this initialize method.[2] Then you can register it as a regular Seaside component. [3] 

Chris 

Nazareth - This Flight Tonight


[1]

Installer ss 
project: 'SmaccDevelopment';
install: 'SmaCC-lr.14.mcz';
install: 'SmaCCDev-lr.18.mcz'.

Installer ss 
project: 'ss2';
install: 'TinyWiki'.


[2]

TWWiki>>initialize
super initialize.
self setModel: TWFolder new initializeHelp


[3] 

TWWiki registerAsApplication: 'TinyWiki'


Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Levente Uzonyi
In reply to this post by timrowledge
On Wed, 14 Aug 2019, tim Rowledge wrote:

>
>
>> <PEGParser class-grammarWiki.st><GoogleWikiCompiler.2.st>
>
> Thank you for the 'fixed' grammar. It even makes sort-of-sense to me, though I couldn't replicate it myself.
> In my current setup the code/pre/list etc stuff is now working nicely. With a tiny bit of furniture it will make a very good start to a usable wiki.
>
> The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?

TinyWiki is the wiki used by SqueakSource to provide wikis for the
projects. So, I suggest searching for TinyWiki on squeaksource.com.

Levente

>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Useful random insult:- Has a one-way ticket on the Disoriented Express.

Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
In reply to this post by Chris Cunnington-4


> On 2019-08-14, at 4:12 PM, Chris Cunnington <[hidden email]> wrote:
>
>
>
>> On Aug 14, 2019, at 6:15 PM, tim Rowledge <[hidden email]> wrote:
>>
>> The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?
>
> No. SqueakSource since forever has had a wiki for every project. It’s called TinyWiki. It’s the kernel of Lukas Reggli’s Pier wiki CMS.

OK. An obvious question here is why that just didn't appear in my searching. This has been a bit of a theme recently; looks like we are failing at being noticed...

I'll give it a try.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Oxymorons: Legally drunk



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Chris Cunnington-4
I would like to say the the first rule of fight club is that you don’t talk about fight club, but I imagine it’s that Seaside creates random urls and so cannot be scraped by Google.

Chris

> On Aug 15, 2019, at 2:57 PM, tim Rowledge <[hidden email]> wrote:
>
>
>
>> On 2019-08-14, at 4:12 PM, Chris Cunnington <[hidden email]> wrote:
>>
>>
>>
>>> On Aug 14, 2019, at 6:15 PM, tim Rowledge <[hidden email]> wrote:
>>>
>>> The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?
>>
>> No. SqueakSource since forever has had a wiki for every project. It’s called TinyWiki. It’s the kernel of Lukas Reggli’s Pier wiki CMS.
>
> OK. An obvious question here is why that just didn't appear in my searching. This has been a bit of a theme recently; looks like we are failing at being noticed...
>
> I'll give it a try.
>
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Oxymorons: Legally drunk
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge


> On 2019-08-15, at 12:01 PM, Chris Cunnington <[hidden email]> wrote:
>
> I would like to say the the first rule of fight club is that you don’t talk about fight club, but I imagine it’s that Seaside creates random urls and so cannot be scraped by Google.

It's not just with google that we fail though, sadly. The search within squeaksource is not so useful either. For example almost the first thing I did when you mentioned it was to look for the 'ss2' project - which didn't get found. Sure 'tinywiki' was found but we must be able to do better at this kind of thing somehow. More thoughtful tagging would help a lot.

One of the real annoyances I have with the 'cool names clique' that has long existed in software project naming-land is that it horribly obscures things. In fact that annoyance was why I labelled the host windows code as 'Ariethfa Ffenestri' - as a tweak to the nose of 'cool names', since it is just a literal translation of the 'proper' project name.

Obviously one thing that would help a lot is us collectively making better use of SqueakMap. Fat chance of that...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Any nitwit can understand computers.  Many do.



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
In reply to this post by Chris Cunnington-4
Loading TinyWiki is actually a smidge simpler than than suggested but there are two methods that need tweaking and it sure could use some htlml layout help.

Oh and it doesn't save anything anywhere except in-image. That may be a good thing for some purposes or it may be a bad thing. I tihnk it would be better saved to files in general, though I may try dumping to a postgresv3 connected database just for grins.
--------------
Installer ss
    project: 'SmaccDevelopment';
    install: 'SmaCC-lr.14.mcz'.
   " install: ‘SmaCCDev-lr.18.mcz’. <— this doesn’t appear to be needed and avoids the problem with SmalltalkImage>>#associationAt:ifAbsent:”

Installer ss
    project: 'ss2';
    install: 'TinyWiki'.

“then edit in these two methods and run the class init”
TWWiki>>initialize
    super initialize.
    self setModel: TWFolder new initializeHelp

TWWiki class>>#initialize    "self initialize"
    WAAdmin register: self asApplicationAt:  #tinywiki
-----------------

Start the WAWebServerAdaptor on port 8080
Browse to localhost:8080/tinywiki

The 'action buttons' are in a vertical oriented list, which looks awful; any hints on changing that would be welcome; I did compare the produced html with that for the tinywikis on squeaksource and it looked very similar. In the simple file wiki based on Ramon's blog I used  a pair of buttons within a form to get side-by-side buttons. I've no idea which might be the 'nice' way to do it. I have a horrible feeling it might involve CSS.

So that's an outline of one of the two approaches that appear practical...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Oxymorons: Clearly misunderstood



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
In reply to this post by Chris Cunnington-4
... and the other approach outline -

Loading up the Xtreams, the variant of Ramon's blog-post wikipage, Levente's parser and some other fiddling by me goes like this

"install SimpleFileWiki.st"

Installer ss
      project: 'MetacelloRepository';
      install: 'ConfigurationOfXtreams'.
(Smalltalk at: #ConfigurationOfXtreams) project bleedingEdge load.

"install /home/pi/DizietFS/Desktop/GoogleWikiCompiler.st"

and then the same Seaside startup/init as before.

The downside of this approach is the much, much, bigger lump of code installed by Xtreams. If, however, you want the (apparently, no personal opinion on this yet) better streaming, and if you have use fo the FFI stuff, then this may be a price worth paying. Clearly the PEGParser stuff seems a bit more expansive and perhaps more up to date than the SmaCC package.

It wouldn't take much to enable either swikipage class/package to use the other parser, of course. Both really only interface with a parser in one place.

What would seem like a really good idea to me would be getting a decent variant (by whatever definition on chooses) included in the SeaSide package(s) since  swiki is a very good demo that has real uses beyond mere demo-ness.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RDLI: Rotate Disk Left Immediate



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Chris Cunnington-4
In reply to this post by timrowledge


On Aug 15, 2019, at 7:33 PM, tim Rowledge <[hidden email]> wrote:

The 'action buttons' are in a vertical oriented list, which looks awful; any hints on changing that would be welcome; I did compare the produced html with that for the tinywikis on squeaksource and it looked very similar.

The method that renders those links is #renderWikiActionsOn:. It collects an array from #actions and iterates over them. That’s why it's a vertical list.

Chris 



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge


> On 2019-08-16, at 6:57 AM, Chris Cunnington <[hidden email]> wrote:
>
>
>
>> On Aug 15, 2019, at 7:33 PM, tim Rowledge <[hidden email]> wrote:
>>
>> The 'action buttons' are in a vertical oriented list, which looks awful; any hints on changing that would be welcome; I did compare the produced html with that for the tinywikis on squeaksource and it looked very similar.
>
> The method that renders those links is #renderWikiActionsOn:. It collects an array from #actions and iterates over them. That’s why it's a vertical list.

That's what I thought until I looked at the html for the squeaksource wiki page and saw the same construct resulting in a horizontal row. At which point I decided that I was correct in my long held opinion that html has become a gateway to insanity.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Don't diddle code to make it faster; find a better algorithm.



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Chris Cunnington-4
In reply to this post by timrowledge


On Aug 15, 2019, at 7:33 PM, tim Rowledge <[hidden email]> wrote:

Oh and it doesn't save anything anywhere except in-image. That may be a good thing for some purposes or it may be a bad thing. I tihnk it would be better saved to files in general, though I may try dumping to a postgresv3 connected database just for grins.

I’ve got a demo image of TinyWiki displaying Swiki page content from a new server. [1] About 10 pages. This image has a tool prototype called SwikiToTinyWiki which opens Swiki XML files in the pages directory and makes objects that the TinyWiki Visitor pattern can use. 

Chris

[1]

Squeak 4.4
can be opened with Cog 4.0.3427 or similar (non-Spur)
localhost:8080/tinywiki
  



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Levente Uzonyi
In reply to this post by timrowledge
On Thu, 15 Aug 2019, tim Rowledge wrote:

> ... and the other approach outline -
>
> Loading up the Xtreams, the variant of Ramon's blog-post wikipage, Levente's parser and some other fiddling by me goes like this
>
> "install SimpleFileWiki.st"
>
> Installer ss
>      project: 'MetacelloRepository';
>      install: 'ConfigurationOfXtreams'.
> (Smalltalk at: #ConfigurationOfXtreams) project bleedingEdge load.
>
> "install /home/pi/DizietFS/Desktop/GoogleWikiCompiler.st"
>
> and then the same Seaside startup/init as before.
>
> The downside of this approach is the much, much, bigger lump of code installed by Xtreams. If, however, you want the (apparently, no personal opinion on this yet) better streaming, and if you have use fo the FFI stuff, then this may be a price worth paying. Clearly the PEGParser stuff seems a bit more expansive and perhaps more up to date than the SmaCC package.

Well, you can install only what you need from Xtreams (without Metacello,
stuff that depends on FFI, like Xtream-Xtras, or dependencies like
XML-Parser):

Installer ss
  project: 'Xtreams';
  addPackage: 'Xtreams-Support';
  addPackage: 'Xtreams-Core';
  addPackage: 'Xtreams-Terminals';
  addPackage: 'Xtreams-TerminalsFileDirectory';
  addPackage: 'Xtreams-Transforms';
  addPackage: 'Xtreams-Substreams';
  addPackage: 'Xtreams-Parsing';
  install.

Levente

>
> It wouldn't take much to enable either swikipage class/package to use the other parser, of course. Both really only interface with a parser in one place.
>
> What would seem like a really good idea to me would be getting a decent variant (by whatever definition on chooses) included in the SeaSide package(s) since  swiki is a very good demo that has real uses beyond mere demo-ness.
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Strange OpCodes: RDLI: Rotate Disk Left Immediate

Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
In reply to this post by Chris Cunnington-4


> On 2019-08-16, at 11:57 AM, Chris Cunnington <[hidden email]> wrote:
>
>
>
>> On Aug 15, 2019, at 7:33 PM, tim Rowledge <[hidden email]> wrote:
>>
>> Oh and it doesn't save anything anywhere except in-image. That may be a good thing for some purposes or it may be a bad thing. I tihnk it would be better saved to files in general, though I may try dumping to a postgresv3 connected database just for grins.
>
> I’ve got a demo image of TinyWiki displaying Swiki page content from a new server.

Interesting. A lot of stuff in that image!

(Mind you, I did manage to completely lock it up by trying to visit examplebrowser; as best I can tell it was choking on trying to find the sources file. Whilst it's perfectly ok to complain about not finding a total lockup is a bit bad. I hope we've improved that bit since 4.notmuch )


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
May the bugs of many programs nest on your hard drive.



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

timrowledge
With much helpful explanation from Levente I was able to make just enough sense of the PEG parser grammar and operation to add a horizontal rule rule, and then to factor the heading rule to count the number of $= to work out the heading level rather than having level specific tags.
Trying to do the same for lists has, however completely eluded me. And the table cells simply can't have anything other than plain-ish text in them without making stuff go very odd.

The other problem is that the code is currently really set up to use google's swiki syntax, which appears to be as dead as a very dead thing. Which means it would be nice to rework it to use some other syntax that has at least a bit of current usage. Would using Markdown with (some) github extensions suit us? Is anyone feeling conversant enough with a PEG grammar to tackle that ?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- One clown short of a circus.



Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

Edgar De Cleene
Folks:
I happy with my only Squeak solution for a personal/sharing swiki.
But people on external to Smalltalk world uses another tools.
 Some exploration show React is popular this days so searching some not
complicate wiki made with it.
I was able to made with https://docusaurus.io/en/
It uses .md files for serving to web.
How you convert http://wiki.squeak.org/squeak to lots of .md ?
Not so easy
I found https://pandoc.org/  which seems a wonderful tool to have
But unlucky us only works with utf-8  and not ISO-8859-1


curl --silent http://wiki.squeak.org/squeak/1 | pandoc --from html --to
markdown_strict -o 1.md
UTF-8 decoding error in - at byte offset 7466 (92).
The input must be a UTF-8 encoded text.


Some more work and pandoc -f html -t markdown 1.html -o 1.md made a readable
document
Still have some to rip, but is usable
I dedicate the week end to have the complete .md documents.
Feedback ?


Edgar
@morplenauta










Reply | Threaded
Open this post in threaded view
|

Re: swiki source suited to recent Squeaks

keithy
In reply to this post by timrowledge

My idea was just to export everything as sources in a comprehensive hierarchy. This could be loaded from sources with minimal tooling (i.e. cuis) via Sake/Packages, where Sake handles the dependencies. This worked reasonably well and could handle the whole Seaside dependency tree (for which they wrote a custom website)

Keith



12