[ANN] Phoedown - Markdown to HTML
Locked
[ANN] Phoedown - Markdown to HTML
 
|
Hi all,
I've published Phoedown, an FFI to hoedown, the standards compliant, fast, secure Markdown processing library written in C. - https://github.com/PierceNg/Phoedown - https://github.com/hoedown/hoedown A simple example: | md | md := (FileSystem memory / 'somefile.md') writeStreamDo: [ :ws | ws nextPutAll: ' ```smalltalk Transcript show: ''Happy New Year!''; cr ``` ' ]; contents. HdHtmlRenderer new setMarkdownExtension: HdMarkdownExtensions FencedCode; setMarkdownExtension: HdMarkdownExtensions NoIntraEmphasis; render: md This is the output: <pre><code class="language-smalltalk">Transcript show: 'Happy New Year!'; cr </code></pre> Pierce |
Re: [ANN] Phoedown - Markdown to HTML
|
|
I’m loving all the little utility libraries coming out (and appreciate the write ups on how things are done).
I’m curious why you chose to use ffi for this one in particular as it would seem to be quite straightforward to do it all in native Smalltalk (not to down play your integration in any way of course). Was performance a big thing? Or was it simply relying on others to keep it up to date? Tim Sent from my iPhone > On 1 Jan 2020, at 10:40, Pierce Ng <[hidden email]> wrote: > > Hi all, > > I've published Phoedown, an FFI to hoedown, the standards compliant, fast, secure Markdown > processing library written in C. > > - https://github.com/PierceNg/Phoedown > - https://github.com/hoedown/hoedown > > A simple example: > > | md | > md := (FileSystem memory / 'somefile.md') > writeStreamDo: [ :ws | > ws nextPutAll: > ' > ```smalltalk > Transcript show: ''Happy New Year!''; cr > ``` > ' ]; > contents. > HdHtmlRenderer new > setMarkdownExtension: HdMarkdownExtensions FencedCode; > setMarkdownExtension: HdMarkdownExtensions NoIntraEmphasis; > render: md > > This is the output: > > <pre><code class="language-smalltalk">Transcript show: 'Happy New Year!'; cr > </code></pre> > > Pierce > > |
Re: [ANN] Phoedown - Markdown to HTML
|
|
On Wed, Jan 01, 2020 at 02:35:03PM +0100, Tim Mackinnon wrote:
> I’m curious why you chose to use ffi for this one in particular as it > would seem to be quite straightforward to do it all in native > Smalltalk (not to down play your integration in any way of course). > Was performance a big thing? Or was it simply relying on others to > keep it up to date? I wanted something that works out of the box. I did not want to implement MD parsing myself. Also I saw hoedown's wiki page on bindings and thought I'd put Smalltalk on the map there. https://github.com/hoedown/hoedown/wiki/Bindings Pierce |
Re: [ANN] Phoedown - Markdown to HTML
|
Administrator
|
Pierce Ng-3 wrote
> Also I saw hoedown's wiki page on bindings and thought I'd put Smalltalk > on the map there. Great! ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Cheers,
Sean |
Re: [ANN] Phoedown - Markdown to HTML
|
|
In reply to this post by Pierce Ng-3
That makes sense - I’m getting the impression that ffi is getting very easy these days and maybe we should use it more to focus on “other” things.
It does complicate deployment a bit - but if you’re server/cloud based then that’s less of an issue anyway . This said, many of our nastiest bugs seem to centre around ffi a c based libraries - but maybe using them more will tease this out. Tim Sent from my iPhone > On 2 Jan 2020, at 01:27, Pierce Ng <[hidden email]> wrote: > >> On Wed, Jan 01, 2020 at 02:35:03PM +0100, Tim Mackinnon wrote: >> I’m curious why you chose to use ffi for this one in particular as it >> would seem to be quite straightforward to do it all in native >> Smalltalk (not to down play your integration in any way of course). >> Was performance a big thing? Or was it simply relying on others to >> keep it up to date? > > I wanted something that works out of the box. I did not want to > implement MD parsing myself. > > Also I saw hoedown's wiki page on bindings and thought I'd put Smalltalk > on the map there. > > https://github.com/hoedown/hoedown/wiki/Bindings > > Pierce > > |
Re: [ANN] Phoedown - Markdown to HTML
|
Administrator
|
Tim Mackinnon wrote
> I’m getting the impression that ffi is getting very easy these days and > maybe we should use it more to focus on “other” things... This said, many > of our nastiest bugs... While I dream of a world where everything is in-image as pure Smalltalk, given the reality of limited manpower, I think of outside library use as a way to "cheat" and get a lot more from that limited engineering resource. I look forward to the mythical day where "everything's done" and we can come back around and replace some of these with Smalltalk, but then of course we'll own the maintenance. That said, as you pointed out, bugs seem to be more severe and difficult to diagnose, so we'll see how it goes as FFI use becomes more and more common... ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Cheers,
Sean |
Re: [ANN] Phoedown - Markdown to HTML
|
|
On Thu, Jan 2, 2020 at 3:47 PM Sean P. DeNigris <[hidden email]> wrote:
> > Tim Mackinnon wrote > > I’m getting the impression that ffi is getting very easy these days and > > maybe we should use it more to focus on “other” things... This said, many > > of our nastiest bugs... > > That said, as you pointed out, bugs seem to be > more severe and difficult to diagnose, so we'll see how it goes as FFI use > becomes more and more common... It seems to have played very well for the Python folks. It should work even better for Pharo. :) Esteban A. Maringolo |
Re: [ANN] Phoedown - Markdown to HTML
|
|
In reply to this post by Sean P. DeNigris
On Thu, Jan 02, 2020 at 12:56:55PM -0600, Sean P. DeNigris wrote:
> bugs seem to be > more severe and difficult to diagnose, so we'll see how it goes as FFI use > becomes more and more common... There's a saying about (aspects of) building software which source I cannot recall: If it hurts, do it more frequently. I believe FFI should be pushed as much as possible to work out bugs and develop new capabilities. Looking forward to playing with threaded FFI. Pierce |
Re: [ANN] Phoedown - Markdown to HTML
|
|
+1
> On Jan 2, 2020, at 9:24 PM, Pierce Ng <[hidden email]> wrote: > > On Thu, Jan 02, 2020 at 12:56:55PM -0600, Sean P. DeNigris wrote: >> bugs seem to be >> more severe and difficult to diagnose, so we'll see how it goes as FFI use >> becomes more and more common... > > There's a saying about (aspects of) building software which source I > cannot recall: > > If it hurts, do it more frequently. > > I believe FFI should be pushed as much as possible to work out bugs and > develop new capabilities. Looking forward to playing with threaded FFI. > > Pierce > |
|
|
In reply to this post by Pierce Ng-3
Hi,
Good work :) { #category : #finalization } HdBufferOpaque class >> finalizeResourceData: aHandle [ HdFFILibrary uniqueInstance ffiCall: #(void hoedown_buffer_free (ExternalAddress aHandle)) ] wow… does this works at all? I admit it is not how it is intended to work :P > On 3 Jan 2020, at 06:24, Pierce Ng <[hidden email]> wrote: > > On Thu, Jan 02, 2020 at 12:56:55PM -0600, Sean P. DeNigris wrote: >> bugs seem to be >> more severe and difficult to diagnose, so we'll see how it goes as FFI use >> becomes more and more common... > > There's a saying about (aspects of) building software which source I > cannot recall: > > If it hurts, do it more frequently. > > I believe FFI should be pushed as much as possible to work out bugs and > develop new capabilities. Looking forward to playing with threaded FFI. +1 FFI introduces a whole new universe of possibilities we cannot reasonable cover in-image nowadays. Now, while *there are* problems still to solve, I think people have the wrong impression the FFI layer is causing most of the bugs around (TTF for example) while this is not true at all :) For example, recent bug around memory corruption was claimed several times it was because of FFI (and even I thought it was around it at the beginning) while in fact it was a problem on how compaction was working. Other example, TTF itself: problems were around when we had a plugin and we still have some with the FFI implementation. But the problem is not around the plugin or the FFI implementation but around the bad usage (not thread safe) we do of it in-image. The reason why people think some problems are FFI related is because we are pushing Pharo in directions not-fully-tested before. The usage of FFI just unveils a lot of problems on how we handle things there were present before, but they were not as visible as they are now. Esteban > > Pierce > |
Re: [ANN] Phoedown - Markdown to HTML
|
Just while we are at the notion of markdowns, I have at https://github.com/kasperosterbye/PillarRichTextRender a first build of a pharo based github markdown to pillar. From pillar you can go to html.
But just to prove the point of the strength of importing working software from outside using FFI, the markdown parser I am working on currently has two serious limitations: a) it do not do table (because my project really was about rendering pillar inside the image, and could not figure out how to render tables inlined in text). b) image address has to be hard-coded - cannot be relative to some url. — Kasper
|
Re: [ANN] Phoedown - Markdown to HTML
|
|
In reply to this post by Sean P. DeNigris
On 2020-01-02 10:56 a.m., Sean P. DeNigris wrote:
> While I dream of a world where everything is in-image as pure Smalltalk, > given the reality of limited manpower, I think of outside library use as a > way to "cheat" and get a lot more from that limited engineering resource. Agree, I've used the original Markdown.pl implementation for years same as I would any other shell script, via OSProcess markdown: someContent ^UnixProcess pipeString: someContent throughCommand: (FileDirectory default fullPathFor: 'Markdown.pl') Never saw a need to rewrite what already works in its original form. -- Ramón León VP of Technology Alliance Reservations Network |
Re: [ANN] Phoedown - Markdown to HTML
|
|
In reply to this post by Sean P. DeNigris
On 2020-01-02 10:56 a.m., Sean P. DeNigris wrote:
> While I dream of a world where everything is in-image as pure Smalltalk, > given the reality of limited manpower, I think of outside library use as a > way to "cheat" and get a lot more from that limited engineering resource. Agree, I've used the original Markdown.pl implementation for years same as I would any other shell script, via OSProcess markdown: someContent ^UnixProcess pipeString: someContent throughCommand: (FileDirectory default fullPathFor: 'Markdown.pl') Never saw a need to rewrite what already works in its original form. -- Ramón León |
|
|
In reply to this post by Ramon Leon-5
Hello, first of all, have a wonderful 2020.. sorry for jumping in, I'm not a contributor to Pharo, and actually never a user of it ( when I work in Smalltalk, I use VW mostly, and still need to find the time to try Pharo seriously), but I do work in Smalltalk (and Pharo is a dialect of) from early 80'... and , as for many of you, Smalltalk is not the only language I use (I wish it was the most used, but not the only one, because one tool is not enough). Well, the idea that all should be done in Smalltalk seems to me something that us as a community have had as a goal from day one, but, are us sure it's not something that reduce the power of Smalltalk instead of allowing broader adoption?
Smalltalk is at is best for modelling difficult problems. Look at Python, it's very popular between Data Scientist, but it just expose in a nice way an interface to Macchine Learning libraries, For example TensorFlow has a lot of optimised c++ code inside. Python made easy to interface with C like languages, and it has good modelling capabilities, so it was chosen. But Smalltalk is better on the latter, this is a campground we should dominate, instead we are absent. And Pharo, being open source, could have been a good player in this field. If we spend our time to reinvent the wheel we can't get to far .. Look at Node.js, you can find libraries for connecting everything, people doesn't rewrote everything in JS/ So if you work in Node, you are fast at building stuff not because of the power of the language, but because to the libraries you can pick from the shelf and use. So I think a wonderful and easy integration framework is time better spent that redoing something already well done on other languages. Object are for reuse, but we try to rebuild... And on top of the framework, lot of smalltalk classes for an easy usage of the outside work already done. Obviously, all I said is not valid if Smalltalk is considered only an experimental language or a playground, but this was, and probably still is, the place for Squeak. If I remember well, Pharo was born for industrial grade application... Sorry for this rant, but I liked Smalltalk from day one, the day I read the famous Byte's article, and still try to understand why it's not the number one language, but I think we, as a community, did a lot on the wrong direction. BTW: I still like Smalltalk a lot and really appreciate the work done by all of the Smalltalk communities. again, have a wonderful year giorgio On Fri, Jan 3, 2020 at 4:55 PM Ramon Leon <[hidden email]> wrote: On 2020-01-02 10:56 a.m., Sean P. DeNigris wrote: |
|
|
Ciao Giorgio, come stai? Mi fa piacere risentirti dopo tanto tempo. Forse (visto il successo della tua mail), sarebbe opportuno cominciare collaborazioni in Italia! Sentiamoci. Lorenzo Da: Pharo-users [mailto:[hidden email]] Per conto di giorgio ferraris Hello, first of all, have a wonderful 2020.. sorry for jumping in, I'm not a contributor to Pharo, and actually never a user of it ( when I work in Smalltalk, I use VW mostly, and still need to find the time to try Pharo seriously), but I do work in Smalltalk (and Pharo is a dialect of) from early 80'... and , as for many of you, Smalltalk is not the only language I use (I wish it was the most used, but not the only one, because one tool is not enough). Well, the idea that all should be done in Smalltalk seems to me something that us as a community have had as a goal from day one, but, are us sure it's not something that reduce the power of Smalltalk instead of allowing broader adoption?
Smalltalk is at is best for modelling difficult problems. Look at Python, it's very popular between Data Scientist, but it just expose in a nice way an interface to Macchine Learning libraries, For example TensorFlow has a lot of optimised c++ code inside. Python made easy to interface with C like languages, and it has good modelling capabilities, so it was chosen. But Smalltalk is better on the latter, this is a campground we should dominate, instead we are absent. And Pharo, being open source, could have been a good player in this field. If we spend our time to reinvent the wheel we can't get to far .. Look at Node.js, you can find libraries for connecting everything, people doesn't rewrote everything in JS/ So if you work in Node, you are fast at building stuff not because of the power of the language, but because to the libraries you can pick from the shelf and use. So I think a wonderful and easy integration framework is time better spent that redoing something already well done on other languages. Object are for reuse, but we try to rebuild... And on top of the framework, lot of smalltalk classes for an easy usage of the outside work already done. Obviously, all I said is not valid if Smalltalk is considered only an experimental language or a playground, but this was, and probably still is, the place for Squeak. If I remember well, Pharo was born for industrial grade application... Sorry for this rant, but I liked Smalltalk from day one, the day I read the famous Byte's article, and still try to understand why it's not the number one language, but I think we, as a community, did a lot on the wrong direction. BTW: I still like Smalltalk a lot and really appreciate the work done by all of the Smalltalk communities. again, have a wonderful year giorgio On Fri, Jan 3, 2020 at 4:55 PM Ramon Leon <[hidden email]> wrote:
|
Re: [ANN] Phoedown - Markdown to HTML
|
|
In reply to this post by Pierce Ng-3
Hi,
Really interesting work. I wonder if something similar could be done to bridge simdjson[1] with Pharo/NeoJSON (and maybe a fast binary objects serializer) to read huge JSON streams and files in seconds. [1] https://github.com/lemire/simdjson Thanks for this work :-), Offray On 1/01/20 4:40 a. m., Pierce Ng wrote: > Hi all, > > I've published Phoedown, an FFI to hoedown, the standards compliant, fast, secure Markdown > processing library written in C. > > - https://github.com/PierceNg/Phoedown > - https://github.com/hoedown/hoedown > > A simple example: > > | md | > md := (FileSystem memory / 'somefile.md') > writeStreamDo: [ :ws | > ws nextPutAll: > ' > ```smalltalk > Transcript show: ''Happy New Year!''; cr > ``` > ' ]; > contents. > HdHtmlRenderer new > setMarkdownExtension: HdMarkdownExtensions FencedCode; > setMarkdownExtension: HdMarkdownExtensions NoIntraEmphasis; > render: md > > This is the output: > > <pre><code class="language-smalltalk">Transcript show: 'Happy New Year!'; cr > </code></pre> > > Pierce > > |
«
Return to Pharo Smalltalk Users
|
1 view|%1 views
| Free forum by Nabble | Edit this page |