CodeImporter. 256 literals limit

I have written a tool (Pharo5) where user gives an input file to it, where the content is a smalltalk code, a DSL. I used a subclass of CodeImporter class to evaluate this input file.

Recently my user used an input file where it hit the 256 literal limit (total of unique string, number, method name, etc), down in OpalEncoderForV3PlusClosures >> genPushLiteral:. The number seems to be hard coded and related to byte code generator, not something I can simply increase. I wasn't aware of this limitation.

Before I overhaul my tool, I thought I should ask. Is there another alternative to evaluate a smalltalk file/script? The file is small, 27k, but the number of unique literals in it is > 256. Is it possible at all, seeing that the limit is related to byte code generator.

Thank you in advance

--

Andreas Sunardi

Hi,

If your tool works in Pharo 6, you can use the other bytecode set which supports up to 32k literals. To do so, go to:

World Menu > Settings > Compiler > Encoder

and pick SistaV1 instead of V3PlusClosures

Try to load your code. The default Pharo 6 VM supports both bytecode sets.

Alternatively you need to split your methods with many literals in multiple methods with less literals, which is usually quite tricky to do right.

On Tue, Aug 29, 2017 at 4:52 AM, Andreas Sunardi <[hidden email]> wrote:

I have written a tool (Pharo5) where user gives an input file to it, where the content is a smalltalk code, a DSL. I used a subclass of CodeImporter class to evaluate this input file.

Recently my user used an input file where it hit the 256 literal limit (total of unique string, number, method name, etc), down in OpalEncoderForV3PlusClosures >> genPushLiteral:. The number seems to be hard coded and related to byte code generator, not something I can simply increase. I wasn't aware of this limitation.

Before I overhaul my tool, I thought I should ask. Is there another alternative to evaluate a smalltalk file/script? The file is small, 27k, but the number of unique literals in it is > 256. Is it possible at all, seeing that the limit is related to byte code generator.

Thank you in advance

--

Andreas Sunardi

That sounds good. Unfortunately for me, my work environment is using RedHat6/CentOS6 with glibc 2.12. Is there Pharo6 with glibc < 2.15 support. Or is there a way for me to build that myself?

It's quite a departure to change my DSL into defining multiple methods. But that's my own problem.

I'm happy to hear Pharo 6 can support a lot more literals. I tested my code on Pharo 6 (on my Windows box) and it works. I see SistaV1 compiler in Pharo 5 setting, but that causes Pharo to crash.

Thank you guys for your answers. I think I have the information I need to make decision. But if there is a way to get Pharo 6 that works with glibc < 2.15, please let me know.

Thank you

--

Andreas

On Tue, Aug 29, 2017 at 12:34 AM, Clément Bera <[hidden email]> wrote:

Hi,

If your tool works in Pharo 6, you can use the other bytecode set which supports up to 32k literals. To do so, go to:

World Menu > Settings > Compiler > Encoder

and pick SistaV1 instead of V3PlusClosures

Try to load your code. The default Pharo 6 VM supports both bytecode sets.

Alternatively you need to split your methods with many literals in multiple methods with less literals, which is usually quite tricky to do right.

On Tue, Aug 29, 2017 at 4:52 AM, Andreas Sunardi <[hidden email]> wrote:

I have written a tool (Pharo5) where user gives an input file to it, where the content is a smalltalk code, a DSL. I used a subclass of CodeImporter class to evaluate this input file.

Recently my user used an input file where it hit the 256 literal limit (total of unique string, number, method name, etc), down in OpalEncoderForV3PlusClosures >> genPushLiteral:. The number seems to be hard coded and related to byte code generator, not something I can simply increase. I wasn't aware of this limitation.

Before I overhaul my tool, I thought I should ask. Is there another alternative to evaluate a smalltalk file/script? The file is small, 27k, but the number of unique literals in it is > 256. Is it possible at all, seeing that the limit is related to byte code generator.

Thank you in advance

--

Andreas Sunardi

my work environment is using RedHat6/CentOS6 with glibc 2.12

That's seven years of unpatched security vulnerabilities! Are you sure you really want to stay at such great risk?

On Tue, Aug 29, 2017 at 8:30 AM, Andreas Sunardi <[hidden email]> wrote:

That sounds good. Unfortunately for me, my work environment is using RedHat6/CentOS6 with glibc 2.12. Is there Pharo6 with glibc < 2.15 support. Or is there a way for me to build that myself?

It's quite a departure to change my DSL into defining multiple methods. But that's my own problem.

I'm happy to hear Pharo 6 can support a lot more literals. I tested my code on Pharo 6 (on my Windows box) and it works. I see SistaV1 compiler in Pharo 5 setting, but that causes Pharo to crash.

Thank you guys for your answers. I think I have the information I need to make decision. But if there is a way to get Pharo 6 that works with glibc < 2.15, please let me know.

Thank you

--

Andreas

On Tue, Aug 29, 2017 at 12:34 AM, Clément Bera <[hidden email]> wrote:

Hi,

If your tool works in Pharo 6, you can use the other bytecode set which supports up to 32k literals. To do so, go to:

World Menu > Settings > Compiler > Encoder

and pick SistaV1 instead of V3PlusClosures

Try to load your code. The default Pharo 6 VM supports both bytecode sets.

Alternatively you need to split your methods with many literals in multiple methods with less literals, which is usually quite tricky to do right.

On Tue, Aug 29, 2017 at 4:52 AM, Andreas Sunardi <[hidden email]> wrote:

I have written a tool (Pharo5) where user gives an input file to it, where the content is a smalltalk code, a DSL. I used a subclass of CodeImporter class to evaluate this input file.

Recently my user used an input file where it hit the 256 literal limit (total of unique string, number, method name, etc), down in OpalEncoderForV3PlusClosures >> genPushLiteral:. The number seems to be hard coded and related to byte code generator, not something I can simply increase. I wasn't aware of this limitation.

Before I overhaul my tool, I thought I should ask. Is there another alternative to evaluate a smalltalk file/script? The file is small, 27k, but the number of unique literals in it is > 256. Is it possible at all, seeing that the limit is related to byte code generator.

Thank you in advance

--

Andreas Sunardi

you can try the new (still experimental) packaging for Pharo: 

CentOS 6.8:

# Add the repo

$ yum-config-manager --add-repo http://download.opensuse.org/repositories/devel:/languages:/pharo:/latest/CentOS_6/devel:languages:pharo:latest.repo

# Install 32bit packages (with X11 dependency for *-ui or not)

$ yum install pharo6-32-ui.i686 or pharo6-32.i386

# Install 64bit packages

$ yum install pharo6-64-ui.x86_64 pharo6-64.x86_64

that will install correct vm for your distribution.

cheers,

Esteban

On 29 Aug 2017, at 18:58, Andreas Sunardi <[hidden email]> wrote:

Point well taken. Unfortunately, I'm not in charge of the infrastructure. I don't know if my experience is typical or not, but it's difficult for IT to find a period to disrupt engineering projects, potentially breaking tools, without drawing fire from engineering :)

On Tue, Aug 29, 2017 at 8:45 AM, Richard Sargent <[hidden email]> wrote:

my work environment is using RedHat6/CentOS6 with glibc 2.12

That's seven years of unpatched security vulnerabilities! Are you sure you really want to stay at such great risk?

On Tue, Aug 29, 2017 at 8:30 AM, Andreas Sunardi <[hidden email]> wrote:

That sounds good. Unfortunately for me, my work environment is using RedHat6/CentOS6 with glibc 2.12. Is there Pharo6 with glibc < 2.15 support. Or is there a way for me to build that myself?

It's quite a departure to change my DSL into defining multiple methods. But that's my own problem.

I'm happy to hear Pharo 6 can support a lot more literals. I tested my code on Pharo 6 (on my Windows box) and it works. I see SistaV1 compiler in Pharo 5 setting, but that causes Pharo to crash.

Thank you guys for your answers. I think I have the information I need to make decision. But if there is a way to get Pharo 6 that works with glibc < 2.15, please let me know.

Thank you

--

Andreas

On Tue, Aug 29, 2017 at 12:34 AM, Clément Bera <[hidden email]> wrote:

Hi,

If your tool works in Pharo 6, you can use the other bytecode set which supports up to 32k literals. To do so, go to:

World Menu > Settings > Compiler > Encoder

and pick SistaV1 instead of V3PlusClosures

Try to load your code. The default Pharo 6 VM supports both bytecode sets.

Alternatively you need to split your methods with many literals in multiple methods with less literals, which is usually quite tricky to do right.

On Tue, Aug 29, 2017 at 4:52 AM, Andreas Sunardi <[hidden email]> wrote:

I have written a tool (Pharo5) where user gives an input file to it, where the content is a smalltalk code, a DSL. I used a subclass of CodeImporter class to evaluate this input file.

Recently my user used an input file where it hit the 256 literal limit (total of unique string, number, method name, etc), down in OpalEncoderForV3PlusClosures >> genPushLiteral:. The number seems to be hard coded and related to byte code generator, not something I can simply increase. I wasn't aware of this limitation.

Before I overhaul my tool, I thought I should ask. Is there another alternative to evaluate a smalltalk file/script? The file is small, 27k, but the number of unique literals in it is > 256. Is it possible at all, seeing that the limit is related to byte code generator.

Thank you in advance

--

Andreas Sunardi