swiki source suited to recent Squeaks

Well, here's a weird thing that I don't recall ever coming up against before.

I've got a basic swiki page component for Seaside, as previously mentioned. Now we have Levente's (original plus small fixes) parser/generator to convert swiki markup to html. I join them together to see what happens ... and weird stuff happens. Testing in a workspace is fine BUT not when getting text back from a browser.

It turns out that the browser (safari and chrome) sends a string with CRLF line ends to us. Now, I'm a long way from keeping up to date with web stuff but really?  I thought we got LFs because unixy-things.

The practical issue is that the grammar provided in PEGParser class>>#grammarWiki has a load of places reliant on \n and so, for example, the Preformatted & Code tags simply get ignored.

Two obvious questions come to mind here
a) what on earth? CRLF? Is that normal or is it an artefact of some Seaside setup I can change?
b) if we need to change the grammar to cope with crlf, what is the best way? I don't find the grammar terribly intuitable and can't spot any rule explanation. I've tried changing the Preformatted rule for example to
Preformatted <- "---\r\n" .{"---\r\n"}
and the parser doesn't even recognise the swiki tags.

I'd hate to have to do a crlf -> lf conversion every time, it seems so inelegant.

Oh - http://code.google.com/p/support/wiki/WikiSyntax (as referenced in #grammarWiki) seems to be a dead page now, which makes it a not very good bit of documentation! Bizarrely there doesn't appear to be much related info found by google.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: CM: Circulate Memory

Hi Tim,

On Tue, 13 Aug 2019, tim Rowledge wrote:

> Well, here's a weird thing that I don't recall ever coming up against before.
>
> I've got a basic swiki page component for Seaside, as previously mentioned. Now we have Levente's (original plus small fixes) parser/generator to convert swiki markup to html. I join them together to see what happens ... and weird stuff happens. Testing in a workspace is fine BUT not when getting text back from a browser.

If you use Seaside, its #renderOn: implementations may be clashing with
StreamingHtmlCanvas's. I just checked three such methods in my image
(Seaside2 + StreamingHtmlCanvas loaded at the same time), and
the implementations are the same, so no issue there.

But, if you use Seaside, you might want to consider using TinyWiki.
crlf is pretty much an internet thing. Even Squeak's converter method is
called #withInternetLineEndings.

I have attached updated GoogleWikiCompiler and #grammarWiki
implementations. The new grammar should accept crlf, cr and lf line
endings. Weird thing is that \n refers to cr, while \r refers to lf in
PEGParser's grammar, so crlf is \n\r...

The google page is gone, but its content is not:
https://web.archive.org/web/20150418033327/http://code.google.com/p/support/wiki/WikiSyntax

Levente

>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Strange OpCodes: CM: Circulate Memory

> <PEGParser class-grammarWiki.st><GoogleWikiCompiler.2.st>

Thank you for the 'fixed' grammar. It even makes sort-of-sense to me, though I couldn't replicate it myself.
In my current setup the code/pre/list etc stuff is now working nicely. With a tiny bit of furniture it will make a very good start to a usable wiki.

The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- Has a one-way ticket on the Disoriented Express.

On Wed, 14 Aug 2019, tim Rowledge wrote:

>
>
>> <PEGParser class-grammarWiki.st><GoogleWikiCompiler.2.st>
>
> Thank you for the 'fixed' grammar. It even makes sort-of-sense to me, though I couldn't replicate it myself.
> In my current setup the code/pre/list etc stuff is now working nicely. With a tiny bit of furniture it will make a very good start to a usable wiki.
>
> The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?

TinyWiki is the wiki used by SqueakSource to provide wikis for the
projects. So, I suggest searching for TinyWiki on squeaksource.com.

Levente

>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Useful random insult:- Has a one-way ticket on the Disoriented Express.

> On 2019-08-14, at 4:12 PM, Chris Cunnington <[hidden email]> wrote:
>
>
>
>> On Aug 14, 2019, at 6:15 PM, tim Rowledge <[hidden email]> wrote:
>>
>> The only 'TinyWiki' stuff google threw up was an html-only approach - was that the one you were thinking of?
>
> No. SqueakSource since forever has had a wiki for every project. It’s called TinyWiki. It’s the kernel of Lukas Reggli’s Pier wiki CMS.

OK. An obvious question here is why that just didn't appear in my searching. This has been a bit of a theme recently; looks like we are failing at being noticed...

I'll give it a try.


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Oxymorons: Legally drunk

I would like to say the the first rule of fight club is that you don’t talk about fight club, but I imagine it’s that Seaside creates random urls and so cannot be scraped by Google.

Chris

> On 2019-08-15, at 12:01 PM, Chris Cunnington <[hidden email]> wrote:
>
> I would like to say the the first rule of fight club is that you don’t talk about fight club, but I imagine it’s that Seaside creates random urls and so cannot be scraped by Google.

It's not just with google that we fail though, sadly. The search within squeaksource is not so useful either. For example almost the first thing I did when you mentioned it was to look for the 'ss2' project - which didn't get found. Sure 'tinywiki' was found but we must be able to do better at this kind of thing somehow. More thoughtful tagging would help a lot.

One of the real annoyances I have with the 'cool names clique' that has long existed in software project naming-land is that it horribly obscures things. In fact that annoyance was why I labelled the host windows code as 'Ariethfa Ffenestri' - as a tweak to the nose of 'cool names', since it is just a literal translation of the 'proper' project name.

Obviously one thing that would help a lot is us collectively making better use of SqueakMap. Fat chance of that...


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Any nitwit can understand computers.  Many do.

Loading TinyWiki is actually a smidge simpler than than suggested but there are two methods that need tweaking and it sure could use some htlml layout help.

Oh and it doesn't save anything anywhere except in-image. That may be a good thing for some purposes or it may be a bad thing. I tihnk it would be better saved to files in general, though I may try dumping to a postgresv3 connected database just for grins.
--------------
Installer ss
    project: 'SmaccDevelopment';
    install: 'SmaCC-lr.14.mcz'.
   " install: ‘SmaCCDev-lr.18.mcz’. <— this doesn’t appear to be needed and avoids the problem with SmalltalkImage>>#associationAt:ifAbsent:”

Installer ss
    project: 'ss2';
    install: 'TinyWiki'.

“then edit in these two methods and run the class init”
TWWiki>>initialize
    super initialize.
    self setModel: TWFolder new initializeHelp

TWWiki class>>#initialize    "self initialize"
    WAAdmin register: self asApplicationAt:  #tinywiki
-----------------

Start the WAWebServerAdaptor on port 8080
Browse to localhost:8080/tinywiki

The 'action buttons' are in a vertical oriented list, which looks awful; any hints on changing that would be welcome; I did compare the produced html with that for the tinywikis on squeaksource and it looked very similar. In the simple file wiki based on Ramon's blog I used  a pair of buttons within a form to get side-by-side buttons. I've no idea which might be the 'nice' way to do it. I have a horrible feeling it might involve CSS.

So that's an outline of one of the two approaches that appear practical...

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Oxymorons: Clearly misunderstood

... and the other approach outline -

Loading up the Xtreams, the variant of Ramon's blog-post wikipage, Levente's parser and some other fiddling by me goes like this

"install SimpleFileWiki.st"

Installer ss
      project: 'MetacelloRepository';
      install: 'ConfigurationOfXtreams'.
(Smalltalk at: #ConfigurationOfXtreams) project bleedingEdge load.

"install /home/pi/DizietFS/Desktop/GoogleWikiCompiler.st"

and then the same Seaside startup/init as before.

The downside of this approach is the much, much, bigger lump of code installed by Xtreams. If, however, you want the (apparently, no personal opinion on this yet) better streaming, and if you have use fo the FFI stuff, then this may be a price worth paying. Clearly the PEGParser stuff seems a bit more expansive and perhaps more up to date than the SmaCC package.

It wouldn't take much to enable either swikipage class/package to use the other parser, of course. Both really only interface with a parser in one place.

What would seem like a really good idea to me would be getting a decent variant (by whatever definition on chooses) included in the SeaSide package(s) since  swiki is a very good demo that has real uses beyond mere demo-ness.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Strange OpCodes: RDLI: Rotate Disk Left Immediate

> On 2019-08-16, at 6:57 AM, Chris Cunnington <[hidden email]> wrote:
>
>
>
>> On Aug 15, 2019, at 7:33 PM, tim Rowledge <[hidden email]> wrote:
>>
>> The 'action buttons' are in a vertical oriented list, which looks awful; any hints on changing that would be welcome; I did compare the produced html with that for the tinywikis on squeaksource and it looked very similar.
>
> The method that renders those links is #renderWikiActionsOn:. It collects an array from #actions and iterates over them. That’s why it's a vertical list.

That's what I thought until I looked at the html for the squeaksource wiki page and saw the same construct resulting in a horizontal row. At which point I decided that I was correct in my long held opinion that html has become a gateway to insanity.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Don't diddle code to make it faster; find a better algorithm.

On Thu, 15 Aug 2019, tim Rowledge wrote:

Well, you can install only what you need from Xtreams (without Metacello,
stuff that depends on FFI, like Xtream-Xtras, or dependencies like
XML-Parser):

Installer ss
  project: 'Xtreams';
  addPackage: 'Xtreams-Support';
  addPackage: 'Xtreams-Core';
  addPackage: 'Xtreams-Terminals';
  addPackage: 'Xtreams-TerminalsFileDirectory';
  addPackage: 'Xtreams-Transforms';
  addPackage: 'Xtreams-Substreams';
  addPackage: 'Xtreams-Parsing';
  install.

Levente

>
> It wouldn't take much to enable either swikipage class/package to use the other parser, of course. Both really only interface with a parser in one place.
>
> What would seem like a really good idea to me would be getting a decent variant (by whatever definition on chooses) included in the SeaSide package(s) since  swiki is a very good demo that has real uses beyond mere demo-ness.
>
> tim
> --
> tim Rowledge; [hidden email]; http://www.rowledge.org/tim
> Strange OpCodes: RDLI: Rotate Disk Left Immediate

> On 2019-08-16, at 11:57 AM, Chris Cunnington <[hidden email]> wrote:
>
>
>
>> On Aug 15, 2019, at 7:33 PM, tim Rowledge <[hidden email]> wrote:
>>
>> Oh and it doesn't save anything anywhere except in-image. That may be a good thing for some purposes or it may be a bad thing. I tihnk it would be better saved to files in general, though I may try dumping to a postgresv3 connected database just for grins.
>
> I’ve got a demo image of TinyWiki displaying Swiki page content from a new server.

Interesting. A lot of stuff in that image!

(Mind you, I did manage to completely lock it up by trying to visit examplebrowser; as best I can tell it was choking on trying to find the sources file. Whilst it's perfectly ok to complain about not finding a total lockup is a bit bad. I hope we've improved that bit since 4.notmuch )


tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
May the bugs of many programs nest on your hard drive.

With much helpful explanation from Levente I was able to make just enough sense of the PEG parser grammar and operation to add a horizontal rule rule, and then to factor the heading rule to count the number of $= to work out the heading level rather than having level specific tags.
Trying to do the same for lists has, however completely eluded me. And the table cells simply can't have anything other than plain-ish text in them without making stuff go very odd.

The other problem is that the code is currently really set up to use google's swiki syntax, which appears to be as dead as a very dead thing. Which means it would be nice to rework it to use some other syntax that has at least a bit of current usage. Would using Markdown with (some) github extensions suit us? Is anyone feeling conversant enough with a PEG grammar to tackle that ?

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Useful random insult:- One clown short of a circus.

Folks:
I happy with my only Squeak solution for a personal/sharing swiki.
But people on external to Smalltalk world uses another tools.
 Some exploration show React is popular this days so searching some not
complicate wiki made with it.
I was able to made with https://docusaurus.io/en/
It uses .md files for serving to web.
How you convert http://wiki.squeak.org/squeak to lots of .md ?
Not so easy
I found https://pandoc.org/  which seems a wonderful tool to have
But unlucky us only works with utf-8  and not ISO-8859-1


curl --silent http://wiki.squeak.org/squeak/1 | pandoc --from html --to
markdown_strict -o 1.md
UTF-8 decoding error in - at byte offset 7466 (92).
The input must be a UTF-8 encoded text.


Some more work and pandoc -f html -t markdown 1.html -o 1.md made a readable
document
Still have some to rip, but is usable
I dedicate the week end to have the complete .md documents.
Feedback ?


Edgar
@morplenauta