On Tue, Dec 17, 2013 at 12:28 PM, Mariano Martinez Peck <[hidden email]> wrote:
I continue my research with this topic. In my server (Linux box), everything works as described above. But....in my local OSX GemStone I have a real crash. I put a halt in a callback of Seaside. The adaptor is FastCGI. I clicked Remote Debug and the stack is written in the ObjectLog. Now...if I try to "Debug" and select that entry from the object log, my gem crash.
SAME test case in the Linux-based works perfectly. Of course, if I switch to GEM_NATIVE_CODE_ENABLED = FALSE; in my /opt/gemstone/product/seaside/etc/seaside30.conf then I can debug without problem. So in fact it is a native code problem.
In my case is not a big deal because locally I can put it in false, and the server works fine with true. But I thought about reporting it in case it is useful for you to find the cause. Here is the log: 14:04 mariano@MacBook-Pro-de-Mariano /opt/gemstone/log % cat gemnetobject25860.log _____________________________________________________________________________
| NetLDI Child Task | | | | CLIENTHOST: MacBook-Pro-de-Mariano.local |
| VERSION: 3.1.0.4, Mon Jun 17 13:14:05 2013 | | BUILD: gss64_3_1_0_x_branch-31024 | | BUILT FOR: Darwin (Mac) |
| MODE: 64 bit | | RUNNING ON: 2-CPU MacBook-Pro-de-Mariano.local x86_64 (Darwin 12.5.0 ) 2530MHz | MacBookPro5,1 2048MB |
| PROCESS ID: 25860 DATE: 12/27/2013 14:03:29 ART | | USER IDS: REAL=mariano (501) EFFECTIVE=mariano (501) | | COMMAND: /opt/gemstone/GemStone64Bit3.1.0.4-i386.Darwin/sys/gemnetobject |
| TCP 63159 30 | |_____________________________________________________________________________| exeConfig -d path [Info]: the hostname is: MacBook-Pro-de-Mariano.local
GEMSTONE is: "/opt/gemstone/product" gem's location is: /opt/gemstone/product/sys system config file is: /opt/gemstone/product/seaside/data/system.conf executable config file is: /opt/gemstone/product/seaside/data
gem's arguments are: TCP 63159 30 _____________________________________________________________________________ | GemStone/S64 Object-Oriented Data Management System |
| Copyright (C) VMware, Inc. 1986-2013 | | All rights reserved. | | Covered by U.S Patents: |
| 6,256,637 Transactional virtual machine architecture | | 6,360,219 Object queues with concurrent updating | | 6,567,905 Generational Garbage Collector. |
| 6,681,226 Selective Pessimistic Locking for a Concurrently Updateable Database +-----------------------------------------------------------------------------+ | PROGRAM: GEM, GemStone Session Process |
| VERSION: 3.1.0.4, Mon Jun 17 13:14:05 2013 | | BUILD: gss64_3_1_0_x_branch-31024 | | BUILT FOR: Darwin (Mac) |
| MODE: 64 bit | | RUNNING ON: 2-CPU MacBook-Pro-de-Mariano.local x86_64 (Darwin 12.5.0 ) 2530MHz | MacBookPro5,1 2048MB |
| PROCESS ID: 25860 DATE: 12/27/2013 14:03:29 ART | | USER IDS: REAL=mariano (501) EFFECTIVE=mariano (501) | +-----------------------------------------------------------------------------+
| GEMSTONE_NRS_ALL = #dir:$GEMSTONE_DATADIR#log:/$GEMSTONE_LOGDIR/%N%P.log#dir:/opt/gemstone/product/seaside/data |_____________________________________________________________________________|
_____________________________________________________________________________ | Configuration Files | | |
| System File: /opt/gemstone/GemStone64Bit3.1.0.4-i386.Darwin/seaside/data/system.conf | | | Executable File: /opt/gemstone/GemStone64Bit3.1.0.4-i386.Darwin/seaside/data/gem.conf
| Warning: File not found (errno=2,ENOENT, The file or directory specified cannot | be found) | | using defaults. |
|_____________________________________________________________________________| _____________________________________________________________________________ | Gem Configuration Options for process id 25860 |
|_____________________________________________________________________________| DUMP_OPTIONS = TRUE; GEM_GCI_LOG_ENABLED = FALSE; GEM_ABORT_MAX_CRS = 0; GEM_FREE_FRAME_CACHE_SIZE = -1;
GEM_FREE_FRAME_LIMIT = -1; GEM_FREE_PAGEIDS_CACHE = 200; GEM_HALT_ON_ERROR = 0; GEM_KEEP_MIN_SOFTREFS = 0; GEM_MAX_SMALLTALK_STACK_DEPTH = 1000; GEM_NATIVE_CODE_ENABLED = TRUE;
GEM_PRIVATE_PAGE_CACHE_KB = 960; GEM_PGSVR_COMPRESS_PAGE_TRANSFERS = FALSE; GEM_PGSVR_FREE_FRAME_CACHE_SIZE = -1; GEM_PGSVR_FREE_FRAME_LIMIT = -1; GEM_PGSVR_UPDATE_CACHE_ON_READ = FALSE;
GEM_RPC_KEEPALIVE_INTERVAL = 0; GEM_RPCGCI_TIMEOUT = 0; GEM_RPC_USE_SSL = TRUE; GEM_SOFTREF_CLEANUP_PERCENT_MEM = 50; GEM_TEMPOBJ_AGGRESSIVE_STUBBING = TRUE; GEM_TEMPOBJ_CACHE_SIZE = 1800000;
GEM_TEMPOBJ_MESPACE_SIZE = 0; GEM_TEMPOBJ_OOPMAP_SIZE = 0; GEM_TEMPOBJ_SCOPES_SIZE = 2000; GEM_TEMPOBJ_POMGEN_SIZE = 0; GEM_TEMPOBJ_POMGEN_PRUNE_ON_VOTE = 90; GEM_TEMPOBJ_POMGEN_SCAVENGE_INTERVAL = 1800;
GEM_TEMPOBJ_START_ADDR not used on this platform LOG_WARNINGS = TRUE; SHR_NUM_FREE_FRAME_SERVERS = -1; SHR_PAGE_CACHE_LOCKED = FALSE; SHR_PAGE_CACHE_NUM_PROCS = 10233;
SHR_PAGE_CACHE_NUM_SHARED_COUNTERS = 1900; SHR_PAGE_CACHE_PERMISSIONS = 660; SHR_PAGE_CACHE_SIZE_KB = 2000000; SHR_TARGET_FREE_FRAME_COUNT = -1; SHR_WELL_KNOWN_PORT_NUMBER = 0;
(vmGc spaceSizes: eden init 2048K max 337536K , survivor init 384K max 56256K, vmGc old max 1349952K, code max 360000K, perm max 180032K, pom 10 * 150016K = 1500160K, vmGc remSet 33988K, meSpace max 1720460K oopMapSize 8388608 max footprint 5548M)
[Info]: libssl-3.1.0.4-64.dylib: loaded cmdLine= [Info]: RPC client/gem/minimum GCI levels = 851/851/851 [Info]: Client PID: 6317 native code DISABLED, offset from code memory to libgcilnk.so exceeds 32bits
native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:3589377, ipOffset 56, native code disabled for remainder of session native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:3589377, ipOffset 96, native code disabled for remainder of session
native code error, emit_call displacement exceeds 32bits , methodId:3589377, ipOffset 140, native code disabled for remainder of session native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:3589377, ipOffset 144, native code disabled for remainder of session
native code error, emit_call_stub, jmp displacement exceeds 32bits , methodId:3589377, ipOffset 176, native code disabled for remainder of session [Info]: User ID: DataCurator [Info]: Repository: seaside
[Info]: Session ID: 5 [Info]: GCI Client Host: ::ffff:127.0.0.1 [Info]: Page server PID: -1 [Info]: Login Time: 12/27/2013 14:03:29.264 ART [Info]: libicudata-3.1.0.4-64.dylib: loaded
[Info]: libicuuc-3.1.0.4-64.dylib: loaded [Info]: libicui18n-3.1.0.4-64.dylib: loaded GemStone signal handler: signal 11 (SIGSEGV), received from process 0 userId 0 si_code: 0, unknown SI_
Gemstone Signal Handler: Signal 11, SIGSEGV Received HostFaultHandler: signal = 11 info->si_signo = 11 = 0xb info->si_code = 0 = 0x0
info->si_errno = 0 = 0x0 info->si_addr = 0x0 signal 11 , received from process 0 userId 0 si_code: 0, unknown SI_ Begin attempt to print C-level stack for current thread of process 25860 at: Fri Dec 27 14:03:46 ART 2013
0 libgcilnk-3.1.0.4-64.dylib 0x000000010135424b _Z15HostPrintCStackv + 158 1 libgcilnk-3.1.0.4-64.dylib 0x00000001013a34b6 HostFaultHandler + 838 2 libsystem_c.dylib 0x00007fff884d890a _sigtramp + 26
3 ??? 0x00007fff5fbf465c 0x0 + 140734799758940 4 libgcilnk-3.1.0.4-64.dylib 0x00000001011df7cd _Z19IntLpSupControlLoopP2omiPP10omObjSType + 605 5 libgcilnk-3.1.0.4-64.dylib 0x00000001011d6b38 _Z16IntExecuteMethodP13IntStateSTypePP10omObjSTypeS3_i + 376
6 libgcilnk-3.1.0.4-64.dylib 0x00000001010974c2 _ZL14executeFromCtxP2omyyyitiPKyi + 882 7 libgcilnk-3.1.0.4-64.dylib 0x0000000101097d15 _Z23GemDoExecuteFromContextyyyitPy + 149
8 libgcilnk-3.1.0.4-64.dylib 0x000000010104e7ba _ZL12dispatchLoopP2omP13LgcStateSTypeP13RpcStateSType + 7754 9 libgcilnk-3.1.0.4-64.dylib 0x000000010105087f _Z12GemDoRpcLoopP2omP13LgcStateSTypeP19IntGciActStateSType + 479
10 libgcilnk-3.1.0.4-64.dylib 0x000000010103a914 _ZL9doConnectv + 516 11 libgcilnk-3.1.0.4-64.dylib 0x000000010103b0e1 _ZL15dispatchCommandv + 817 12 libgcilnk-3.1.0.4-64.dylib 0x000000010103b6e1 _Z4GdbgPKci + 1281
13 libgcilnk-3.1.0.4-64.dylib 0x0000000101051afc gemMain + 2124 14 gem 0x0000000100001bd1 main + 376 15 gem 0x00000001000014b4 start + 52
End of C-level stack for current thread process 25860 _____________________________________________________________________________ | Possible Internal Error: HostCoreDump invoked at 12/27/2013 14:03:47.428 ART
in process 25860 __________________________________ HostCoredump: Waiting 60 seconds for C Debugger to attach, process 25860 Mariano http://marianopeck.wordpress.com
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Mariano, Yep that looks like a bug ... I'll report this one ... is there a chance that you can share your extent with us so that we can reproduce the bug? It might not be easy for us to isolate the problem otherwise. Thanks for the report. Dale From: "Mariano Martinez Peck" <[hidden email]> _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass |
Hi Dale, I have been bitten again by this issue. And now in the Linux server :( that means that in order to be able to Remote Debug errors ocurred in the server, I needed to disable the native code :( So I lost quite a bit of performance boost :(
The good news is that it is very easy to reproduce. Let me give you the specifications I have to see if you can reproduce it (you asked for a extent, but it is much easier to reproduce than that):
1) GemStone 3.0.1.4 2) CentOS 6 64 bits. 3) Glass 1.0-beta.9 4) Web adaptors: FastCGI run from WAGemStoneRunSeasideGems. 5) Very important STONE configuration option: GEM_NATIVE_CODE_ENABLED=TRUE;
6) Create a dummy WAComponent that implements: renderContentOn: html 'mariano' xxx. html table: [
html tableColumn: [ nil shouldNotArriveHere. ]
] When I run that, I do not even get a nice walkback but the message of the exception is printed in the screen. Look attached.
7) If I now go to GemTools -> Debug -> I see the message ""aMessageDoesNotUnderstood occurred (error 2010), aString does not understand 'xxx' ".". Actually, I see like 4 or 6 of them for a single click.
8) Click on either of them, and you will see that the debuggers opens on the "nil shouldNotArriveHere" while it should have halted in " 'mariano' xxx". 9) Put STONE configuration option to false: GEM_NATIVE_CODE_ENABLED=FALSE and stop and start stone again (and seaside adaptors)
10) Repeat all the steps and you will see that this time it really halts on " 'mariano' xxx " I hope this helps to reproduce this problem since it is very bad to loose the native code feature :(
Thanks in advance and let me know if I can be of any other help. On Sat, Jan 4, 2014 at 1:36 PM, Dale K. Henrichs <[hidden email]> wrote:
Mariano http://marianopeck.wordpress.com _______________________________________________ Glass mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/glass Screen Shot 2014-03-25 at 6.49.28 PM.png (53K) Download Attachment |
Free forum by Nabble | Edit this page |