Support Regular Expressions in Trunk

> Hi, there!
>
> Do we want to support and maintain regular expressions in trunk?
>
> http://www.squeaksource.com/Regex.html
>
> Best,
> Marcel
>
>
>
> --
> View this message in context: http://forum.world.st/Support-Regular-Expressions-in-Trunk-tp4825937.html
> Sent from the Squeak - Dev mailing list archive at Nabble.com.
>
>

> On Tue, May 12, 2015 at 10:27:20AM -0700, tim Rowledge wrote:
>> I love the idea of well-integrated expressions etc being available for string handling and in tools. The tricky bit is that ?well-integrated?.
>>
>> If at all possible it should be done without any extra primitives. Unless we have a convincing use case for such string handling needing to be frequently done and very fast I don?t see much value in adding more C code. With Cog and SISTA making the system so much faster we have to consider the possibility that doing all the marshalling, converting, stack fapping, c-code calling, returning, re-converting may take more time than leaving it to Smalltalk.
>> I suppose the counter-argument would be that using a standard library guarantees correct results. Except of course we have a lot of experience with supposedly standard libraries being utterly screwed. Jpeg libraries, anyone? ALSA?
>>
>> As for the actual expression grammar, I imagine most people would be assuming the unix regexp form which has the advantage that a lot of people are familiar with it. And the disadvantage that a very large number of people are not.
>>
>
> There are two distict packages.
>
> The package at www.squeaksource.com/Regex is the Vassili Bykov implementation.

> It requires no plugin support or special primitives.
>
> -- Regular Expression Matcher v 1.1 (C) 1996, 1999 Vassili Bykov
> -- See `documentation' protocol of RxParser class for user's guide.
>
> The PCRE package is Andrew Greenberg's Perl compatible regular expressions.
> This uses the RePlugin plugin that can be found in the VMMaker package.
> RePlugin is an internal plugin in the Unix VMs (interpreter/cog/spur), I'm
> not sure about other platforms or SqueakJS.
>
> http://wiki.squeak.org/squeak/558
>
> Dave

> On Tue, May 12, 2015 at 10:27:20AM -0700, tim Rowledge wrote:
> > I love the idea of well-integrated expressions etc being available for string handling and in tools. The tricky bit is that ?well-integrated?.
> >
> > If at all possible it should be done without any extra primitives. Unless we have a convincing use case for such string handling needing to be frequently done and very fast I don?t see much value in adding more C code. With Cog and SISTA making the system so much faster we have to consider the possibility that doing all the marshalling, converting, stack fapping, c-code calling, returning, re-converting may take more time than leaving it to Smalltalk.
> > I suppose the counter-argument would be that using a standard library guarantees correct results. Except of course we have a lot of experience with supposedly standard libraries being utterly screwed. Jpeg libraries, anyone? ALSA?
> >
> > As for the actual expression grammar, I imagine most people would be assuming the unix regexp form which has the advantage that a lot of people are familiar with it. And the disadvantage that a very large number of people are not.
> >
>
> There are two distict packages.
>
> The package at www.squeaksource.com/Regex is the Vassili Bykov implementation.
> It requires no plugin support or special primitives.
>
> -- Regular Expression Matcher v 1.1 (C) 1996, 1999 Vassili Bykov
> -- See `documentation' protocol of RxParser class for user's guide.
>
> The PCRE package is Andrew Greenberg's Perl compatible regular expressions.
> This uses the RePlugin plugin that can be found in the VMMaker package.
> RePlugin is an internal plugin in the Unix VMs (interpreter/cog/spur), I'm
> not sure about other platforms or SqueakJS.
>
>   http://wiki.squeak.org/squeak/558
>

> On Tue, May 12, 2015 at 06:17:09PM -0400, David T. Lewis wrote:
>> On Tue, May 12, 2015 at 10:27:20AM -0700, tim Rowledge wrote:
>>> I love the idea of well-integrated expressions etc being available for string handling and in tools. The tricky bit is that ?well-integrated?.
>>>
>>> If at all possible it should be done without any extra primitives. Unless we have a convincing use case for such string handling needing to be frequently done and very fast I don?t see much value in adding more C code. With Cog and SISTA making the system so much faster we have to consider the possibility that doing all the marshalling, converting, stack fapping, c-code calling, returning, re-converting may take more time than leaving it to Smalltalk.
>>> I suppose the counter-argument would be that using a standard library guarantees correct results. Except of course we have a lot of experience with supposedly standard libraries being utterly screwed. Jpeg libraries, anyone? ALSA?
>>>
>>> As for the actual expression grammar, I imagine most people would be assuming the unix regexp form which has the advantage that a lot of people are familiar with it. And the disadvantage that a very large number of people are not.
>>>
>>
>> There are two distict packages.
>>
>> The package at www.squeaksource.com/Regex is the Vassili Bykov implementation.
>> It requires no plugin support or special primitives.
>>
>> -- Regular Expression Matcher v 1.1 (C) 1996, 1999 Vassili Bykov
>> -- See `documentation' protocol of RxParser class for user's guide.
>>
>> The PCRE package is Andrew Greenberg's Perl compatible regular expressions.
>> This uses the RePlugin plugin that can be found in the VMMaker package.
>> RePlugin is an internal plugin in the Unix VMs (interpreter/cog/spur), I'm
>> not sure about other platforms or SqueakJS.
>>
>>   http://wiki.squeak.org/squeak/558
>>
>
> Actually there are probably more than just two. SqueakMap also has BRegexp:
>
>  Description:
>    BRegexp is Perl5 compatible regular expression library for Squeak.
>    -Supports multilingualized Squeak.
>    -Supports only Windows environment.
>
>  Home page:
>    http://kminami.fc2web.com/Squeak/Goodies/BRegexp/index.html
>
> SqueakMap has the Andrew Greenberg PCRE as "Regular Expression Plugin"
> which seems to load despite an error that pops up in the SAR postscript
> processing.
>
> I do not see an entry in SqueakMap for the Vassili Bykov Regex package.
>
> I am fairly sure that regex enthusiasts will have excellent reasons for
> favoring the Perl compatible PCRE in some cases (fast, powerful, well
> documented), and equally excellent reasons to favor the Vassily Bykov
> implementation in other cases (portable, all Smalltalk, works in SqueakJS).
>
> So it seems to me that both of these excellent packages deserve good
> entries in SqueakMap to make them easy to find, and easy to load in
> the latest Squeak images. If that is done, then I cannot think of any
> reason that either one of them would need to be maintained within trunk.

>
> Or to say it another way: If these packages are not important enough to
> make them easily loadable from SqueakMap, then they are not very important.
> If they are not very important, then we should not put them into trunk.
> So please make them loadable :-)
>
> Dave
>
>
>

> Am 12.05.2015 17:07, schrieb Levente Uzonyi:
>> Definitely. I've been playing with the idea - while I was working on
>> my still unreleased DNS package - but I came to the conclusion that
>> the VM side has to be cleaned first.
>> The plugin should use the platform's support library, instead of some
>> ~10 years old version.
> Please, no! It's much easier to work with some old library that has
> consistent behaviour everywhere than it is to find and work around the
> differences between platform libraries e.g. on Windows, Linux, and some
> regular UNIX variant. At work I need to do that with VA Smalltalk code page
> conversion (UTF-8 to ISO8859L1 and back) - code that works on the client
> doesn't work on the server, so there's a special case and workaround needed
> to get everything right.
> The only acceptable option if you really want to use an external library
> would be to include a standard regex C library with well-defined
> functionality in the VM.

>>> The plugin should use the platform's support library, instead of some
>>> ~10 years old version.
>> Please, no! It's much easier to work with some old library that has
>> consistent behaviour everywhere than it is to find and work around the
>> differences between platform libraries e.g. on Windows, Linux, and
>> some regular UNIX variant. At work I need to do that with VA Smalltalk
>> code page conversion (UTF-8 to ISO8859L1 and back) - code that works
>> on the client doesn't work on the server, so there's a special case
>> and workaround needed to get everything right.
>> The only acceptable option if you really want to use an external
>> library would be to include a standard regex C library with
>> well-defined functionality in the VM.
>
> Are you saying that the PCRE library behaves differently on different
> platforms?
>

> Am 13.05.2015 03:02, schrieb Levente Uzonyi:
>> On Tue, 12 May 2015, Hans-Martin Mosner wrote:
>>
>>> Am 12.05.2015 17:07, schrieb Levente Uzonyi:
> ...
>>>> The plugin should use the platform's support library, instead of some
>>>> ~10 years old version.
>>> Please, no! It's much easier to work with some old library that has
>>> consistent behaviour everywhere than it is to find and work around the
>>> differences between platform libraries e.g. on Windows, Linux, and some
>>> regular UNIX variant. At work I need to do that with VA Smalltalk code
>>> page conversion (UTF-8 to ISO8859L1 and back) - code that works on the
>>> client doesn't work on the server, so there's a special case and
>>> workaround needed to get everything right.
>>> The only acceptable option if you really want to use an external library
>>> would be to include a standard regex C library with well-defined
>>> functionality in the VM.
>>
>> Are you saying that the PCRE library behaves differently on different
>> platforms?
>>
>
> Nope, PCRE should behave consistently across platforms if the same version is
> installed on those platforms (although I'd be somewhat unsure about its
> handling of line ending conventions and unicode/different character sets on
> different platforms.)
> But I was responding to "platform's support library" which might be anything
> from PCRE through some POSIX compatible regex (from the GNU C library or some
> other implementation) to anything else that might be there (see
> http://unixpapa.com/incnote/regex.html for a list of possibilities).

> On Wed, 13 May 2015, Hans-Martin Mosner wrote:
>
>> Am 13.05.2015 03:02, schrieb Levente Uzonyi:
>>> On Tue, 12 May 2015, Hans-Martin Mosner wrote:
>>>
>>>> Am 12.05.2015 17:07, schrieb Levente Uzonyi:
>> ...
>>>>> The plugin should use the platform's support library, instead of some
>>>>> ~10 years old version.
>>>> Please, no! It's much easier to work with some old library that has
>>>> consistent behaviour everywhere than it is to find and work around the
>>>> differences between platform libraries e.g. on Windows, Linux, and
>>>> some
>>>> regular UNIX variant. At work I need to do that with VA Smalltalk code
>>>> page conversion (UTF-8 to ISO8859L1 and back) - code that works on the
>>>> client doesn't work on the server, so there's a special case and
>>>> workaround needed to get everything right.
>>>> The only acceptable option if you really want to use an external
>>>> library
>>>> would be to include a standard regex C library with well-defined
>>>> functionality in the VM.
>>>
>>> Are you saying that the PCRE library behaves differently on different
>>> platforms?
>>>
>>
>> Nope, PCRE should behave consistently across platforms if the same
>> version is
>> installed on those platforms (although I'd be somewhat unsure about its
>> handling of line ending conventions and unicode/different character sets
>> on
>> different platforms.)
>> But I was responding to "platform's support library" which might be
>> anything
>> from PCRE through some POSIX compatible regex (from the GNU C library or
>> some
>> other implementation) to anything else that might be there (see
>> http://unixpapa.com/incnote/regex.html for a list of possibilities).
>
> The RePlugin is a wrapper for the PCRE library, so what I'm saying is that
> we should use the system's installed PCRE library (or statically link an
> up-to-date version), instead of a really old fork of it.
> The PCRE API won't change anymore, since PCRE2 is here.
>