petit parser - exceeded maximum number of instance variables

I learned something today. While talking about the maximum number of instance variabes I always thought it is a limit per class. But now I know it is a limit on an object. As I reached 230 instance variables I started to split the parser into two. But now I reached the limit and cannot add instance variables.

I need some ideas about how to structure the parser. I think it will be too hard to find parts of the grammar that are distinct so it would be possible to separate them into different classes. Another way would be to access some of the parser via method calls instead of accessing the instVar parsers.

So does anybody has some experience in how to structure parsers?

thanks,

Norbert
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Hi,

Alberto encountered this when he worked on the Java parser. The solution is to split the parser into multiple classes that inherit from each other.

For example, in the mentioned example we have:
PPJavaLexicon
  PPJavaSyntax
    PPJavaParser


Cheers,
Doru


On 7 Jun 2011, at 12:02, Norbert Hartl wrote:

--
www.tudorgirba.com

"In a world where everything is moving ever faster,
one might have better chances to win by moving slower."




_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

No, that is exactly what I did and it does not work. The number of instance variables takes the super classes into account. I splitted it into two classes: One having 89 and the second having 163 instance variables. The sum is 252. If you take one instVar from PPParser and one from PPDelegatingParser that makes a total of 254. And that is the number I get if I do

MyParser instSize

while

MyParser instVarNames size = 163

The java parser has only 195 instance variables so no problem. So either I did something completely wrong or I need a new strategy.

Norbert

Am 07.06.2011 um 12:35 schrieb Tudor Girba:


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

PPCompositeParser has its limits. I hate the instance variable mess,
but it kind of makes it convenient to refer to other productions.

In Helvetia there is a different implementation of PPCompositeParser
that works with the same grammar source but compiles variable accesses
to use a dictionary instead. This then scales to an almost infinite
number of productions. I agree that introducing the Helvetia
dependendency is not feasible though.

I wonder if we could use some other trick that wouldn't require a
custom compiler and that would work across platforms?

> Another way would be to access some of the parser via method calls instead of accessing the instVar parsers.

That is a possiblity, if they don't depend on each other circularly
and if you don't mind the object duplication.

Also you can build your productions more imperatively (like in many
examples and the tests you see floating around) instead of purely
declaratively as it is mostly done with PPCompositeParser.

> So does anybody has some experience in how to structure parsers?

Typically grammars are quite clustered and have complex leaf nodes
(the tokens in general, string and number literals, comments, etc.)
that can be factored out.

Cheers,
Lukas

--
Lukas Renggli
www.lukas-renggli.ch
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

Am 07.06.2011 um 15:46 schrieb Lukas Renggli:

> PPCompositeParser has its limits. I hate the instance variable mess,
> but it kind of makes it convenient to refer to other productions.
>
> In Helvetia there is a different implementation of PPCompositeParser
> that works with the same grammar source but compiles variable accesses
> to use a dictionary instead. This then scales to an almost infinite
> number of productions. I agree that introducing the Helvetia
> dependendency is not feasible though.
>
Well, it might be feasible for me :) I downloaded the Helvetia one-click and couldn't find the different impementation of PPCompositeParser.

> I wonder if we could use some other trick that wouldn't require a
> custom compiler and that would work across platforms?
>
That would be wonderful. The only cross-platform trick that I know is adding "self" in front of a ruleName :)

>> Another way would be to access some of the parser via method calls instead of accessing the instVar parsers.
>
> That is a possiblity, if they don't depend on each other circularly
> and if you don't mind the object duplication.
>
I think that is my only chance at the moment.

> Also you can build your productions more imperatively (like in many
> examples and the tests you see floating around) instead of purely
> declaratively as it is mostly done with PPCompositeParser.
>
I have to look into this. As I have more than 254 rules it means the grammer is rather complex. One reason I would like to do it in a declarative fashion because I think it increases maintainability.

>> So does anybody has some experience in how to structure parsers?
>
> Typically grammars are quite clustered and have complex leaf nodes
> (the tokens in general, string and number literals, comments, etc.)
> that can be factored out.
>
That is what I did. I factored out the leaf rules into a base class before I encountered the problem. I'm using an ugly approach now. I include a delegate in parser that contains the basic type rules. In my parser I have rules that need to be changed from

^ bString / hString / definedValue

to

^ basicType bString / basicType hString / definedValue

I don't really like it and it makes the usage of the petit parser UI more cumbersome. But it is the best thing at the moment that lets me proceed.

Norbert


_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev

>> In Helvetia there is a different implementation of PPCompositeParser
>> that works with the same grammar source but compiles variable accesses
>> to use a dictionary instead. This then scales to an almost infinite
>> number of productions. I agree that introducing the Helvetia
>> dependendency is not feasible though.
>>
> Well, it might be feasible for me :) I downloaded the Helvetia one-click and couldn't find the different impementation of PPCompositeParser.

CUCompositeParser. It performs a series of transformations on all
production methods, so

  1. it automatically adds the return statement
  2. it automatically converts literals to their respective parsers
  3. it automatically converts variable accesses to dictionary accesses

>> I wonder if we could use some other trick that wouldn't require a
>> custom compiler and that would work across platforms?
>>
> That would be wonderful. The only cross-platform trick that I know is adding "self" in front of a ruleName :)

That has different semantics, so I don't think that this is a good solution.

If you don't mind writing something like

   self @ #otherProd

or with some heavy meta-programming

    #otherProd ref

to refer to other productions, one could use the dictionary approach
from the Helvetia image. With a rewrite rule you could quickly fix the
existing code.

Lukas

--
Lukas Renggli
www.lukas-renggli.ch
_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev