Cannot login to topaz with native code enabled in OSX

Interesting ... I've passed this along to engineering ...

Could you try logging in using topaz (both linked and rpc) for completeness ...

Dale

On 10/09/2015 06:52 AM, Mariano Martinez Peck via Glass wrote:

Hi guys,

I am trying to migrate from 3.1.0.6 to GemStone 3.2.9. I am first in OSX (later Linux). I did follow the guide for the upgrade mechanism and it finished correctly. However, I am finding problems trying to log in via topaz when native code is enabled. 

Only if I set GEM_NATIVE_CODE_ENABLED=TRUE;  in gem.conf  I have this problem. If set to false, everything works. 

It's not a big deal as long as native code DOES work for seaside gems (couldn't test yet). But anyway, I wanted to report it.

The error happens when in topaz I try to execute the "login" command. 

Here is the full log.

% cat gemnetobject37766.log

 _____________________________________________________________________________

|                              NetLDI Child Task                              |

|                                                                             |

| CLIENTHOST: localhost                                                       |

|    VERSION: 3.2.9, Wed Sep  9 14:28:49 2015                                 |

|      BUILD: gss64_3_2_x_branch-37422                                        |

|  BUILT FOR: Darwin (Mac)                                                    |

|       MODE: 64 bit                                                          |

| RUNNING ON: 8-CPU MacBook-Pro.local x86_64 (Darwin 15.0.0 ) 2500MHz MacBookPro11,5

| 2048MB                                                                      |

| PROCESS ID: 37766     DATE: 10/09/2015 10:47:13 ART                         |

|   USER IDS: REAL=mariano (501) EFFECTIVE=mariano (501) LOGIN=mariano (501)  |

|    COMMAND: /Users/mariano/gsDevKitHome/gemstone/products/GemStone64Bit3.2.9-i386.Darwin/sys/gemnetobject

| TCP 5 30                                                                    |

|_____________________________________________________________________________|

[Info]:    the hostname is:

   GEMSTONE is:      "/opt/gemstone/product"

   gem's location is: /opt/gemstone/product/sys

   system config file is:     /debris/Sites/mariano/gemstone/etc/system.conf

   executable config file is: /debris/Sites/mariano/gemstone/etc/gem.conf

   gem's arguments are: TCP 5 30

 _____________________________________________________________________________

|             GemStone/S64 Object-Oriented Data Management System             |

|                   Copyright (C) GemTalk Systems 1986-2015                   |

|                            All rights reserved.                             |

|                           Covered by U.S Patents:                           |

|            6,256,637 Transactional virtual machine architecture             |

|              6,360,219 Object queues with concurrent updating               |

|                  6,567,905 Generational Garbage Collector.                  |

| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database

+-----------------------------------------------------------------------------+

|    PROGRAM: GEM, GemStone Session Process                                   |

|    VERSION: 3.2.9, Wed Sep  9 14:28:49 2015                                 |

|      BUILD: gss64_3_2_x_branch-37422                                        |

|  BUILT FOR: Darwin (Mac)                                                    |

|       MODE: 64 bit                                                          |

| RUNNING ON: 8-CPU MacBook-Pro.local x86_64 (Darwin 15.0.0 ) 2500MHz MacBookPro11,5

| 2048MB                                                                      |

| PROCESS ID: 37766     DATE: 10/09/2015 10:47:13 ART                         |

|   USER IDS: REAL=mariano (501) EFFECTIVE=mariano (501) LOGIN=mariano (501)  |

+-----------------------------------------------------------------------------+

|   GEMSTONE_NRS_ALL = #dir:/debris/Sites/mariano/gemstone/data#log:/debris/Sites/mariano/gemstone/log/%N%P.log

|_____________________________________________________________________________|

 _____________________________________________________________________________

|                             Configuration Files                             |

|                                                                             |

| System File: /debris/Sites/mariano/gemstone/etc/system.conf                 |

|                                                                             |

| Executable File: /debris/Sites/mariano/gemstone/etc/gem.conf                |

|_____________________________________________________________________________|

 _____________________________________________________________________________

|               Gem Configuration Options for process id 37766                |

|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;

GEM_GCI_LOG_ENABLED = FALSE;

GEM_ABORT_MAX_CRS = 0;

GEM_FREE_FRAME_CACHE_SIZE = -1;

GEM_FREE_FRAME_LIMIT = -1;

GEM_FREE_PAGEIDS_CACHE = 200;

GEM_HALT_ON_ERROR = -1;

GEM_KEEP_MIN_SOFTREFS = 0;

GEM_MAX_SMALLTALK_STACK_DEPTH = 1000;

GEM_NATIVE_CODE_ENABLED = 1;

GEM_PRIVATE_PAGE_CACHE_KB = 960KB;

GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;

GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;

GEM_PGSVR_FREE_FRAME_LIMIT = -1;

GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;

GEM_READ_AUTH_ERR_STUBS = FALSE;

GEM_REPOSITORY_IN_MEMORY = FALSE;

GEM_RPC_KEEPALIVE_INTERVAL = 0;

GEM_RPCGCI_TIMEOUT = 0;

GEM_RPC_USE_SSL = TRUE;

GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;

GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;

GEM_TEMPOBJ_CACHE_SIZE = 1800000KB;

GEM_TEMPOBJ_MESPACE_SIZE = 0KB;

GEM_TEMPOBJ_OOPMAP_SIZE = 0;

GEM_TEMPOBJ_SCOPES_SIZE = 2000;

GEM_TEMPOBJ_POMGEN_SIZE = 0KB;

GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 90;

GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;

GEM_TEMPOBJ_START_ADDR not used on this platform

LOG_WARNINGS = TRUE;

SHR_NUM_FREE_FRAME_SERVERS = -1;

SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_POLICY = 0;

SHR_PAGE_CACHE_LOCKED = FALSE;

SHR_PAGE_CACHE_NUM_PROCS = 10233;

SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;

SHR_PAGE_CACHE_PERMISSIONS = 660;

SHR_PAGE_CACHE_SIZE_KB = 2000000KB;

SHR_TARGET_FREE_FRAME_COUNT = -1;

SHR_WELL_KNOWN_PORT_NUMBER = 0;

(vmGc spaceSizes: eden init 2048K max 337536K , survivor init 384K max 56256K,

 vmGc    old max 1349952K, code max 360000K, perm max 180032K, pom 10 * 150016K = 1500160K,

 vmGc    remSet 33988K, meSpace max 1720460K oopMapSize 8388608  max footprint 5548M)

cmdLine= CONNECT AGENT

[Info]: RPC client/gem GCI levels = 864/864

[Info]: Client PID: 37764

native code DISABLED, offset from code memory to libgcilnk.so exceeds 32bits

native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:126322433, ipOffset 56, native code disabled for remainder of session

-----------------------------------------------------

GemStone: Error         Fatal

Fatal error in native code generation, emit_call_stub, jmp displacement exceeds 32bits

Error Category: 231169 [GemStone] Number: 4151  Arg Count: 1 Context : 20 exception : 20

Arg 1:   20

[Info]: Logging out at 10/09/2015 10:47:13 ART (1444398433)

*****************************************************

****** GemStone Abnormal Shutdown at 10/09/2015 10:47:13 ART

*****************************************************

-----------------------------------------------------

GemStone: Error         Fatal

Fatal error in native code generation, emit_call_stub, jmp displacement exceeds 32bits

Error Category: 231169 [GemStone] Number: 4151  Arg Count: 1 Context : 20 exception : 20

Arg 1:   20

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Okay ... it turns out that this is a limitation for using native code on the mac ... the mac's memory layout is not quite compatible with our current native code architecture ... it may be possible to reduce the size of  TOC and/or SPC to get around this limitation (i.e., it only shows up when the offset between the code-gen module and libgcilnkobj.so is greater than 32 bits ...).

Dale

On 10/09/2015 10:31 AM, Mariano Martinez Peck wrote:

On Fri, Oct 9, 2015 at 2:27 PM, Dale Henrichs via Glass <[hidden email][hidden email]> wrote:

Interesting ... I've passed this along to engineering ...

Could you try logging in using topaz (both linked and rpc) for completeness ...

I have just tried with both (passing -l or not to topaz), same result. 

 

Dale

On 10/09/2015 06:52 AM, Mariano Martinez Peck via Glass wrote:

Hi guys,

I am trying to migrate from 3.1.0.6 to GemStone 3.2.9. I am first in OSX (later Linux). I did follow the guide for the upgrade mechanism and it finished correctly. However, I am finding problems trying to log in via topaz when native code is enabled. 

Only if I set GEM_NATIVE_CODE_ENABLED=TRUE;  in gem.conf  I have this problem. If set to false, everything works. 

It's not a big deal as long as native code DOES work for seaside gems (couldn't test yet). But anyway, I wanted to report it.

The error happens when in topaz I try to execute the "login" command. 

Here is the full log.

% cat gemnetobject37766.log

 _____________________________________________________________________________

|                              NetLDI Child Task                              |

|                                                                             |

| CLIENTHOST: localhost                                                       |

|    VERSION: 3.2.9, Wed Sep  9 14:28:49 2015                                 |

|      BUILD: gss64_3_2_x_branch-37422                                        |

|  BUILT FOR: Darwin (Mac)                                                    |

|       MODE: 64 bit                                                          |

| RUNNING ON: 8-CPU MacBook-Pro.local x86_64 (Darwin 15.0.0 ) 2500MHz MacBookPro11,5

| 2048MB                                                                      |

| PROCESS ID: 37766     DATE: 10/09/2015 10:47:13 ART                         |

|   USER IDS: REAL=mariano (501) EFFECTIVE=mariano (501) LOGIN=mariano (501)  |

|    COMMAND: /Users/mariano/gsDevKitHome/gemstone/products/GemStone64Bit3.2.9-i386.Darwin/sys/gemnetobject

| TCP 5 30                                                                    |

|_____________________________________________________________________________|

[Info]:    the hostname is:

   GEMSTONE is:      "/opt/gemstone/product"

   gem's location is: /opt/gemstone/product/sys

   system config file is:     /debris/Sites/mariano/gemstone/etc/system.conf

   executable config file is: /debris/Sites/mariano/gemstone/etc/gem.conf

   gem's arguments are: TCP 5 30

 _____________________________________________________________________________

|             GemStone/S64 Object-Oriented Data Management System             |

|                   Copyright (C) GemTalk Systems 1986-2015                   |

|                            All rights reserved.                             |

|                           Covered by U.S Patents:                           |

|            6,256,637 Transactional virtual machine architecture             |

|              6,360,219 Object queues with concurrent updating               |

|                  6,567,905 Generational Garbage Collector.                  |

| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database

+-----------------------------------------------------------------------------+

|    PROGRAM: GEM, GemStone Session Process                                   |

|    VERSION: 3.2.9, Wed Sep  9 14:28:49 2015                                 |

|      BUILD: gss64_3_2_x_branch-37422                                        |

|  BUILT FOR: Darwin (Mac)                                                    |

|       MODE: 64 bit                                                          |

| RUNNING ON: 8-CPU MacBook-Pro.local x86_64 (Darwin 15.0.0 ) 2500MHz MacBookPro11,5

| 2048MB                                                                      |

| PROCESS ID: 37766     DATE: 10/09/2015 10:47:13 ART                         |

|   USER IDS: REAL=mariano (501) EFFECTIVE=mariano (501) LOGIN=mariano (501)  |

+-----------------------------------------------------------------------------+

|   GEMSTONE_NRS_ALL = #dir:/debris/Sites/mariano/gemstone/data#log:/debris/Sites/mariano/gemstone/log/%N%P.log

|_____________________________________________________________________________|

 _____________________________________________________________________________

|                             Configuration Files                             |

|                                                                             |

| System File: /debris/Sites/mariano/gemstone/etc/system.conf                 |

|                                                                             |

| Executable File: /debris/Sites/mariano/gemstone/etc/gem.conf                |

|_____________________________________________________________________________|

 _____________________________________________________________________________

|               Gem Configuration Options for process id 37766                |

|_____________________________________________________________________________|

DUMP_OPTIONS = TRUE;

GEM_GCI_LOG_ENABLED = FALSE;

GEM_ABORT_MAX_CRS = 0;

GEM_FREE_FRAME_CACHE_SIZE = -1;

GEM_FREE_FRAME_LIMIT = -1;

GEM_FREE_PAGEIDS_CACHE = 200;

GEM_HALT_ON_ERROR = -1;

GEM_KEEP_MIN_SOFTREFS = 0;

GEM_MAX_SMALLTALK_STACK_DEPTH = 1000;

GEM_NATIVE_CODE_ENABLED = 1;

GEM_PRIVATE_PAGE_CACHE_KB = 960KB;

GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE;

GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1;

GEM_PGSVR_FREE_FRAME_LIMIT = -1;

GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;

GEM_READ_AUTH_ERR_STUBS = FALSE;

GEM_REPOSITORY_IN_MEMORY = FALSE;

GEM_RPC_KEEPALIVE_INTERVAL = 0;

GEM_RPCGCI_TIMEOUT = 0;

GEM_RPC_USE_SSL = TRUE;

GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50;

GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE;

GEM_TEMPOBJ_CACHE_SIZE = 1800000KB;

GEM_TEMPOBJ_MESPACE_SIZE = 0KB;

GEM_TEMPOBJ_OOPMAP_SIZE = 0;

GEM_TEMPOBJ_SCOPES_SIZE = 2000;

GEM_TEMPOBJ_POMGEN_SIZE = 0KB;

GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 90;

GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;

GEM_TEMPOBJ_START_ADDR not used on this platform

LOG_WARNINGS = TRUE;

SHR_NUM_FREE_FRAME_SERVERS = -1;

SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_POLICY = 0;

SHR_PAGE_CACHE_LOCKED = FALSE;

SHR_PAGE_CACHE_NUM_PROCS = 10233;

SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900;

SHR_PAGE_CACHE_PERMISSIONS = 660;

SHR_PAGE_CACHE_SIZE_KB = 2000000KB;

SHR_TARGET_FREE_FRAME_COUNT = -1;

SHR_WELL_KNOWN_PORT_NUMBER = 0;

(vmGc spaceSizes: eden init 2048K max 337536K , survivor init 384K max 56256K,

 vmGc    old max 1349952K, code max 360000K, perm max 180032K, pom 10 * 150016K = 1500160K,

 vmGc    remSet 33988K, meSpace max 1720460K oopMapSize 8388608  max footprint 5548M)

cmdLine= CONNECT AGENT

[Info]: RPC client/gem GCI levels = 864/864

[Info]: Client PID: 37764

native code DISABLED, offset from code memory to libgcilnk.so exceeds 32bits

native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:126322433, ipOffset 56, native code disabled for remainder of session

-----------------------------------------------------

GemStone: Error         Fatal

Fatal error in native code generation, emit_call_stub, jmp displacement exceeds 32bits

Error Category: 231169 [GemStone] Number: 4151  Arg Count: 1 Context : 20 exception : 20

Arg 1:   20

[Info]: Logging out at 10/09/2015 10:47:13 ART (1444398433)

*****************************************************

****** GemStone Abnormal Shutdown at 10/09/2015 10:47:13 ART

*****************************************************

-----------------------------------------------------

GemStone: Error         Fatal

Fatal error in native code generation, emit_call_stub, jmp displacement exceeds 32bits

Error Category: 231169 [GemStone] Number: 4151  Arg Count: 1 Context : 20 exception : 20

Arg 1:   20

--

Mariano
http://marianopeck.wordpress.com

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

--

Mariano
http://marianopeck.wordpress.com

Well, the limitation _is_ mentioned in the SysAdminGuide for 3.2[1]:

Under some configurations on x86, in particular with Darwin, the 32-bit offset limit my be exceeded in some cases with a very large temporary object cache. If this occurs, native code is disabled.

I agree this isn't necessarily an obvious place to look for this information ... I've passed your comment along...good point.

Dale

[1] https://downloads.gemtalksystems.com/docs/GemStone64/3.2.x/GS64-SysAdmin-3.2/2-Clients.htm#pgfId-940825

On 10/09/2015 11:03 AM, Mariano Martinez Peck wrote:

OK, I can confirm I have a large TOC and large SPC :)
In any case, I can live with none JIT for my development machine. Just wanted to check with you in case it was not a known issue.

Maybe just adding a note in the user manual installation would be a good idea.

On 10/09/2015 11:20 AM, Mariano Martinez Peck wrote:

On Fri, Oct 9, 2015 at 3:16 PM, Dale Henrichs <[hidden email]> wrote:

Well, the limitation _is_ mentioned in the SysAdminGuide for 3.2[1]:

Under some configurations on x86, in particular with Darwin, the 32-bit offset limit my be exceeded in some cases with a very large temporary object cache. If this occurs, native code is disabled.

I agree this isn't necessarily an obvious place to look for this information ... I've passed your comment along...good point.

Ups...sorry I was reading the installation guide. Ok... 

mmmm the above comment looks like if GemStone would simply disable native code and continue with the interpreted way. If this is true, shouldn't the topaz login continue and finish successfully ?

Good point, but the abitility to disable the native code generation and cleanly proceed is a function of when that event happens ... if it happens early enough in the login procedure, we are not able to successfully disable native code generation ...