Gemstone/s not starting on Solaris 10

Ran the "gemstone start" command on the solaris 10 server
The gemstone logs on server say the below--

--- Thu 01 May 2014 05:08:31 PDT ---
startstone[Error]Stone Process (id=27767) has died.
 The stone was unable to attach to a shared page cache on host '<stone's host>'.
Reason:  The cache monitor connect failed.
  Netldi 'netldi51' on host 'mstst231' reports the request 'shrpcmonitor gemserver51@123.166.88.141 50000 44 1 300' failed:
NetLdi interactive time limit (30 seconds) has been exceeded.

Any suggestions on how to proceed !!

I assume that this was from the stone log. What does it show in the shared page cache monitor log?

On May 1, 2014, at 5:31 AM, abhishekb1512 <[hidden email]> wrote:

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

#This is how we Start stoned and netldi inside a small script
  $GEMSTONE/bin/startnetldi -agemadmin
  $GEMSTONE/bin/startstone

# SPC LOGS ARE BELOW
--------------------------------------------------------------------------------------

|                              NetLDI Child Task                              |
|                                                                             |
| CLIENTHOST: 123.166.88.141                                                  |
|    VERSION: 5.1.5.1, Wed Aug  2 15:00:30 US/Pacific 2000                    |
|  BUILT FOR: SPARC (Solaris 2.6)                                             |
| RUNNING ON: 64-CPU mstst231 sun4u (Solaris 2.10 Generic_150400-04) 2660MHz  |
| sparcv9, 262144MB                                                           |
| PROCESS ID: 27768     DATE: Thu 01 May 2014 05:08:01 PDT                    |
|   USER IDS: REAL=gemadmin (2000) EFFECTIVE=gemadmin (2000)                  |
|    COMMAND: /gemstone/GemStone5.1.5-sparc.Solaris/sys/startshrpcmon gemserver51@123.166.88.141
| 50000 44 1 300                                                              |
|_____________________________________________________________________________|

SharedPageCache Monitor Summary:
   the hostname is:    mstst231
   GEMSTONE is:        "/gemstone/GemStone5.1.5-sparc.Solaris"
   shrpcmon arguments are:
      SharedPageCache Monitor name  gemserver51@123.166.88.141.
      Number of pages               50000.
      Number of processes           44.
      Cache locked in memory        1.
      SpinLockCount                 300.
 


             GemStone Object-Oriented Data Management System               |
|               Copyright (C) GemStone Systems, Inc. 1986-1999.               |
|                            All rights reserved.                             |
+-----------------------------------------------------------------------------+
|    PROGRAM: SHRPCMON,  GemStone SharedPageCache Monitor                     |
|    VERSION: 5.1.5.1, Wed Aug  2 15:00:30 US/Pacific 2000                    |
|  BUILT FOR: SPARC (Solaris 2.6)                                             |
| RUNNING ON: 64-CPU mstst231 sun4u (Solaris 2.10 Generic_150400-04) 2660MHz  |
| sparcv9, 262144MB                                                           |
| PROCESS ID: 27768     DATE: Thu 01 May 2014 05:08:01 PDT                    |
|   USER IDS: REAL=gemadmin (2000) EFFECTIVE=gemadmin (2000)                  |
|_____________________________________________________________________________|

<SHRPCMON, shell for GemStone SharedPageCache Monitor>
<Taking commands from command line.>
  NOTE:  Shared cache could not be locked in memory because this
         user does not have sufficient privileges to do so. (EPERM)
Shared memory region number 1 has id = 285212759.
The shared semaphore array has id = 234881078.

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

What we have here is the stone started the SPC monitor and the monitor appears to have started successfully but the stone is unable to communicate with the SPC monitor.

I’m not sure why the NetLdi is mentioned. That makes me wonder if the stone thinks that the SPC is on another host. How many network cards are on this machine? How many IP addresses are assigned to the machine? Is it possible that ‘mstst231’ resolves to something other than ‘123.166.88.141’ (or vice-versa)?

Did things ever work on this machine? If so, when did the problem start? Was there some change in the configuration about that time?

Is the problem repeatable? Can you run ‘gslist -x’ from another shell during the 30 seconds that the stone is waiting for the SPC monitor to start? Possibly the SPC monitor is actually hung and can’t respond. Is there anything produced in the SPC monitor log from ‘kill -SIGUSR1’? If not, do you have ‘pstack’ on the machine and do you know how to use it to get a peek what the SPC monitor is doing?

James

On May 1, 2014, at 9:37 PM, abhishekb1512 <[hidden email]> wrote:

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

Thanks Bill, for confirming that the shared cache was not an error. Gemstone has never run on this
server and its our first time. I simply ran gemstone script after installing gemstone. Also found this in the stone logs.

                             Configuration Files                             |                                                                             |
| System File: /gemstone/GemStone5.1.5-sparc.Solaris/data/system.conf         |
|                                                                             |
| Executable File: /gemstone/GemStone5.1.5-sparc.Solaris/bin/gemserver51.conf |
| Warning:  File not found (errno=2, ENOENT, The file or directory specified  |
| cannot be found)                                                            |

|           using defaults.

Friday, May 2, 2014, Bill.Erickson [via Smalltalk] <[hidden email]> wrote:
> The log file snippets you've supplied dont contain any errors (the message about the shared page cache not being locked in memory is just a warning -- the system should still be able to start up after that).  Could you search the logs for a real error, or give us some description of what's wrong?
>
> Also, GS/S 5.1.5.1 is getting pretty long in the tooth (nearly 14 years old!). You might want to consider upgrading.
>
> ------------------------------------------------------------------------
> Bill Erickson
> GemTalk Systems Engineering
> 15220 NW Greenbrier Parkway #240, Beaverton OR 97006
> ------------------------------------------------------------------------
>
>
> On Thu, May 1, 2014 at 9:37 PM, abhishekb1512 <[hidden email]> wrote:
>>
>> |                              NetLDI Child Task
>> |
>> |
>> |
>> | CLIENTHOST: <a href="tel:135.161.94.133" value="+13516194133">135.161.94.133
>> |
>> |    VERSION: 5.1.5.1, Wed Aug  2 15:00:30 US/Pacific 2000
>> |
>> |  BUILT FOR: SPARC (Solaris 2.6)
>> |
>> | RUNNING ON: 64-CPU mstst231 sun4u (Solaris 2.10 Generic_150400-04) 2660MHz
>> |
>> | sparcv9, 262144MB
>> |
>> | PROCESS ID: 27768     DATE: Thu 01 May 2014 05:08:01 PDT
>> |
>> |   USER IDS: REAL=gemadmin (2000) EFFECTIVE=gemadmin (2000)
>> |
>> |    COMMAND: /gemstone/GemStone5.1.5-sparc.Solaris/sys/startshrpcmon
>> [hidden email]
>> | 50000 44 1 300
>> |
>> |_____________________________________________________________________________|
>>
>> SharedPageCache Monitor Summary:
>>    the hostname is:    mstst231
>>    GEMSTONE is:        "/gemstone/GemStone5.1.5-sparc.Solaris"
>>    shrpcmon arguments are:
>>       SharedPageCache Monitor name  [hidden email].
>>       Number of pages               50000.
>>       Number of processes           44.
>>       Cache locked in memory        1.
>>       SpinLockCount                 300.
>>
>>
>>
>>              GemStone Object-Oriented Data Management System               |
>> |               Copyright (C) GemStone Systems, Inc. 1986-1999.
>> |
>> |                            All rights reserved.
>> |
>> +-----------------------------------------------------------------------------+
>> |    PROGRAM: SHRPCMON,  GemStone SharedPageCache Monitor
>> |
>> |    VERSION: 5.1.5.1, Wed Aug  2 15:00:30 US/Pacific 2000
>> |
>> |  BUILT FOR: SPARC (Solaris 2.6)
>> |
>> | RUNNING ON: 64-CPU mstst231 sun4u (Solaris 2.10 Generic_150400-04) 2660MHz
>> |
>> | sparcv9, 262144MB
>> |
>> | PROCESS ID: 27768     DATE: Thu 01 May 2014 05:08:01 PDT
>> |
>> |   USER IDS: REAL=gemadmin (2000) EFFECTIVE=gemadmin (2000)
>> |
>> |_____________________________________________________________________________|
>>
>> <SHRPCMON, shell for GemStone SharedPageCache Monitor>
>> <Taking commands from command line.>
>>   NOTE:  Shared cache could not be locked in memory because this
>>          user does not have sufficient privileges to do so. (EPERM)
>> Shared memory region number 1 has id = 285212759.
>> The shared semaphore array has id = 234881078.
>>
>>
>>
>> --
>> View this message in context: http://forum.world.st/Gemstone-s-not-starting-on-Solaris-10-tp4757431p4757539.html
>> Sent from the Gemstone/S mailing list archive at Nabble.com.
>> _______________________________________________
>> GemStone-Smalltalk mailing list
>> [hidden email]
>> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>
>
> _______________________________________________
> GemStone-Smalltalk mailing list
> [hidden email]
> http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk
>
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://forum.world.st/Gemstone-s-not-starting-on-Solaris-10-tp4757431p4757637.html
> To unsubscribe from Gemstone/s not starting on Solaris 10, click here.
> NAML

--

James,

We are trying to set up gemstone on a new hardware where its never run. Yes, the nslookup command maps the hostname to this IP I provided.
 I simply ran > $GEMSTONE/install/installgs" interactive script as root and proceeded with the default options. we did not setup the shared memory or anything . Checked and added couple stick bits to the binaries seen below

 -r-sr-xr-x   1 gemadmin gemadmin  439552 Sep  7  2000 pgsvrmain
-r-sr-xr-x   1 root     gemadmin  344000 Sep  7  2000 shrpcmonitor
-r-sr-xr-x   1 gemadmin gemadmin  510556 Sep  7  2000 pgsvr
-r-sr-xr-x   1 root     gemadmin  291152 Sep  7  2000 netldid
-r-sr-xr-x   1 gemadmin gemadmin 1731108 Sep  7  2000 gem
-r-sr-xr-x   1 gemadmin gemadmin  825280 Sep  7  2000 stoned

--Finally ran the below in a script to start netldi and gemstone
#Start stoned and netldi
  $GEMSTONE/bin/startnetldi -agemadmin
  $GEMSTONE/bin/startstone

It waits for 30 secs and then fails. Also found the below in stone            
                 Configuration Files                             |
|                                                                             |
| System File: /gemstone/GemStone5.1.5-sparc.Solaris/data/system.conf         |
|                                                                             |
| Executable File: /gemstone/GemStone5.1.5-sparc.Solaris/bin/gemserver51.conf |
| Warning:  File not found (errno=2, ENOENT, The file or directory specified  |
| cannot be found)                                                            |
|           using defaults. .


How  or rather from where do i start troubleshooting ?

It is possible that the stone (running as gemadmin) is unable to attach to memory owned by the shrpcmonitor (running as root). Is there a reason why you ran the install as root? Do you plan to run RPC gems as different OS users? I know that some places run that way but many others let all the GS processes run as one OS user (gemadmin) and use different database users. Do you have this configuration running on other machines?

At this point I’m mostly guessing. As Bill mentioned, version 5.1.5.1 was released in 2000. Do (did) you have a support contract? Can you upgrade the database?

On May 2, 2014, at 8:27 AM, abhishekb1512 <[hidden email]> wrote:

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk

I'm Assuming the support contract was not renewed ? Cost benefit Analysis of some sort. Well, the suites dont wanna spend cash on old tech, The geeks don't know how to tell them the opposite is true.

Anyway we also have an old solaris 5.8 system where Gemstone is working perfectly. Our aim was to mimic the files and other data in the .profile , basically copy all files from server to server. Then we tried to install gemstone via the installgs script.  This is where we are stuck now.

Can we simply compare the gemstone configuration files on both the servers to proceed ?

On May 5, 2014, at 1:03 AM, abhishekb1512 <[hidden email]> wrote:

It might be that easy but I suspect it will be a bit more complex. If you don’t get it figured out and want to discuss some professional services, feel free to contact me directly.

James

_______________________________________________
GemStone-Smalltalk mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk