Error with netldi, the connection to the repository is never made and the port becomes blocked
Locked
Error with netldi, the connection to the repository is never made and the port becomes blocked
 
|
Hello, First of nothing I want to describe my current environment:I'm having some troubles to connect to my repository. Due to that I have administrator rights over both machines and because I was interested on test Gemstone, I Have installed Gemstone on the linux machine (from now, I will call it "ML") and Jade in my Windows machine (from now, I will call it "MW"). Both, wire and VPN are the same network. All connections from VPN are succesfull, but if I try to connect using network wire the session is never And I noted something strange: _______________________________________________ GemStone-Smalltalk mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
I forgot to mention:
The Gemstone version is 3.2.8.1. |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
In reply to this post by Gemstone/S mailing list
Hi Ezequiel,
I would first try to test the connectivity without GemStone. From MW (Windows), try ‘ping HOST’ (where HOST is the hostname or IP address of ML). Do this in both the direct mode and in the VPN mode and let us know the result. If that succeeds in both cases, then try ‘telnet HOST PORT’ where PORT is the port on which the netldi process is listening (based on a configuration, services file, or ‘gslist -l’ on ML. This will tell you if MW is able to connect to ML on the appropriate port. Try this first from ML to confirm that you can make it work, then try from MW in each configuration. With this information we can determine more. James > On Mar 21, 2017, at 12:53 PM, ezequiel aciar via GemStone-Smalltalk <[hidden email]> wrote: > > Hello, > > I'm having some troubles to connect to my repository. > First of nothing I want to describe my current environment: > > I have two machines under my administration in the company for which I'm currently working. > Both of these are desinated for my diary work, one of these machine is running CentOS and the other one is my notebook with Windows 7. > > Due to that I have administrator rights over both machines and because I was interested on test Gemstone, I Have installed Gemstone on the linux machine (from now, I will call it "ML") and Jade in my Windows machine (from now, I will call it "MW"). > > The problem Is that now I cannot connect to my repository from MW while I'm connected to my company's LAN using network wire, but the connection is succesful If I connect MW to my company's network using a VPN client. > Both, wire and VPN are the same network. > > All connections from VPN are succesfull, but if I try to connect using network wire the session is never > > The port is not closed and there is not firewall in the network (I have asked yet) nor in ML. > I tried everything that came to my mind and I could not solve this problem. > > In one of my tests I've started netldi in debug mode (using option -d) in order to have some additional information in the log file. > > And I noted something strange: > > If I try to connect using network wire, netldi only log the line: > > "Attempting accept.." > > And never log anything again. > > After that the port seems to be blocked because I can't connect to my repository neither using VPN. > > If I want to connect using VPN y have to stop and restart netldi again. > > If you need additional info please ask me. > > Kind Regards > > Ezequiel > > _______________________________________________ > GemStone-Smalltalk mailing list > [hidden email] > http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk _______________________________________________ GemStone-Smalltalk mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
Hi James,
First of all, thanks for helping me. Ping and telnet (from MW) work with both connections direct and VPN But if try now to login using Jade, then the the port becomes blocked or something, because I cannot connect anymore with Telnet. I've made another test: I detected that is not only problem with my MW, because I tested connection from another pc with windows (connected using wire) and teh same error. So, I started a session using vpn and one using direct connection, then I check in the server for current connections to the port using "netstat -putona | grep 50387". 50387 is the port of netldi. I receive this result: WITHOUT CONNECTIONS: netstat -putona | grep 50387 (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 :::50387 :::* LISTEN 9362/netldid off (0.00/0/0) WITH ONE CONNECTION FROM VPN: netstat -putona | grep 50387 (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 :::50387 :::* LISTEN 9362/netldid off (0.00/0/0) tcp 0 0 ::ffff:133.94.1.30:50387 ::ffff:133.90.150.161:50546 ESTABLISHED 9397/gem keepalive (6975.87/0/0) /*Note that a gem process is assigned to the connection*/ WITH ONE CONNECTION FROM VPN AND ONE CONNECTION WITH NETWORK WIRE: netstat -putona | grep 50387 (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) tcp 0 0 :::50387 :::* LISTEN 9362/netldid off (0.00/0/0) tcp 0 0 ::ffff:133.94.1.30:50387 ::ffff:133.90.150.161:50546 ESTABLISHED 9397/gem keepalive (6975.87/0/0) tcp 112 0 ::ffff:133.94.1.30:50387 ::ffff:133.94.1.11:54096 ESTABLISHED 9362/netldid keepalive (7194.97/0/0) /* Now,instead of a gem process, the netldi process is assigned to the connection from MW using network wire */ This is really strange. I have asked to network administrators and they said that there is no firewall in the network and that the problem may be my ML. But I don't have any rule to reject connections in ML and the port is open too. Thanks James. Ezequiel |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
Hi Ezequiel,
This is indeed interesting (and not in a good way). The netldid daemon process should simply start the new gem process and continue listening for other requests. With netstat you have confirmed that netldid is holding the socket. The next thing would be to get a C stack to see where netldid is hung. With a clean startup before any connections, identify the PID for the netldid process (e.g., ‘gslist -l’) and send it a special signal: $ kill -SIGUSR1 <PID> The signal (which can be abbreviated as simply ‘usr1’) is interpreted by GemStone processes as an instruction to write the current C-level stack to the current log. Look for the netldi log (if you don’t know where it is try ‘gslist -x’) and go to the end and find the SIGUSR1 output. When idle, there should be something like ‘NetHibernateNoAction’ a few frames above main. Once you have the “normal” behavior, try your experiment that hangs and send it the signal. We are particularly interested in that stack. Wait ~30 seconds and repeat a couple times to see if it is in a tight loop or waiting on something. Let us know what that shows. Also, I’d be interested in what MW thinks is its name and IP address, and if that name and IP can be pinged from ML. Try this in both the wired network and VPN configuration. Finally, are there any other useful services on ML? For example, if it has a web server then it would be interesting to see how a web browser on MW reacted (though this is essentially what you confirmed with ping). James > On Mar 22, 2017, at 6:07 AM, brianstone via GemStone-Smalltalk <[hidden email]> wrote: > > Hi James, > > First of all, thanks for helping me. > > Ping and telnet (from MW) work with both connections direct and VPN > > But if try now to login using Jade, then the the port becomes blocked or > something, because I cannot connect anymore with Telnet. > > I've made another test: > > I detected that is not only problem with my MW, because I tested connection > from another pc with windows (connected using wire) and teh same error. > > So, I started a session using vpn and one using direct connection, then I > check in the server for current connections to the port using "netstat > -putona | grep 50387". 50387 is the port of netldi. > > I receive this result: > > WITHOUT CONNECTIONS: > > netstat -putona | grep 50387 > (Not all processes could be identified, non-owned process info > will not be shown, you would have to be root to see it all.) > tcp 0 0 :::50387 :::* > LISTEN 9362/netldid off (0.00/0/0) > > WITH ONE CONNECTION FROM VPN: > > netstat -putona | grep 50387 > (Not all processes could be identified, non-owned process info > will not be shown, you would have to be root to see it all.) > tcp 0 0 :::50387 :::* > LISTEN 9362/netldid off (0.00/0/0) > tcp 0 0 ::ffff:133.94.1.30:50387 ::ffff:133.90.150.161:50546 > ESTABLISHED 9397/gem keepalive (6975.87/0/0) > > /*Note that a gem process is assigned to the connection*/ > > WITH ONE CONNECTION FROM VPN AND ONE CONNECTION WITH NETWORK WIRE: > > netstat -putona | grep 50387 > (Not all processes could be identified, non-owned process info > will not be shown, you would have to be root to see it all.) > tcp 0 0 :::50387 :::* > LISTEN 9362/netldid off (0.00/0/0) > tcp 0 0 ::ffff:133.94.1.30:50387 ::ffff:133.90.150.161:50546 > ESTABLISHED 9397/gem keepalive (6975.87/0/0) > tcp 112 0 ::ffff:133.94.1.30:50387 ::ffff:133.94.1.11:54096 > ESTABLISHED 9362/netldid keepalive (7194.97/0/0) > > /* Now,instead of a gem process, the netldi process is assigned to the > connection from MW using network wire */ > > This is really strange. I have asked to network administrators and they said > that there is no firewall in the network and that the problem may be my ML. > > But I don't have any rule to reject connections in ML and the port is open > too. > > Thanks James. > > Ezequiel > > > > > -- > View this message in context: http://forum.world.st/Error-with-netldi-the-connection-to-the-repository-is-never-made-and-the-port-becomes-blocked-tp4939542p4939622.html > Sent from the Gemstone/S mailing list archive at Nabble.com. > _______________________________________________ > GemStone-Smalltalk mailing list > [hidden email] > http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk _______________________________________________ GemStone-Smalltalk mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
Hi James,
I made the tests you mentioned, I have obtained the following results: 1) Trying to ping MW from ML using name or IP, works. I can state that each machine 'sees' each other in the network. 2) I figured out that netldi is not finished normally when it was hang. It is killed instead of finish normally. I try to stop it using 'stopnetldi' command: stopnetldi gs64ldi-3281 stopnetldi[Info]: GemStone version '3.2.8.1' stopnetldi[Info]: GemStone server 'gs64ldi-3281' has been stopped. Then I check if the netldi was terminated and I se the following: gslist -l Status Version Owner Pid Port Started Type Name ------- --------- --------- ----- ----- ------------ ------ ---- killed 3.2.8.1 gemst643281 16056 50387 Mar 28 15:49 Netldi gs64ldi-3281 exists 3.2.8.1 gemst643281 15396 39139 Mar 28 15:10 cache newRepositorystone~7663a27bab8c7a96 exists 3.2.8.1 gemst643281 15394 41387 Mar 28 15:10 Stone newRepositorystone 3) I made tests sending SIGUSR1 signal to netldi process, I've noted some differences between a 'normal' log, and a log with the netldi hung, but I could not detect where the problem is. Maybe you can read it better, I have attached three files. a) The first, named "netldi-1.log" have the log resulting of make this actions: -Start netldi -Send SIGUSR1 signal -Establish a session using Jade from a VPN connection -Send SIGUSR1 signal -Stop netldi b)The second file, named "netldi-2.log", have the log resulting of make this actions : -Start netldi -Send SIGUSR1 signal -Try to establish a session using Jade from Wired connection -Send SIGUSR1 signal -Stop netldi c) And the third named as "netldi-3.log", have the same content as the 2nd but with some additionals SIGUSR1 signals sent to the netldi. netldi-1.log netldi-2.log netldi-3.log I hope this information will be useful. Regarding about an additional useful service on the server, I'm afraid that there is not something that can help us. But I will try to install apache, or NodeJS or something in order to start a little web server. I don't know when I will have the time to make this tests, but I'll try to make it as soon as possible. Tell me if you need some additional information, please. Ezequiel |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
Hi Ezequiel,
This is very helpful information. Let me summarize the information I got from the log files: netldi-1.log (successful connection) Before connection frame 8 is in waitForEvents when interrupted by signal handler. After connection frame 8 is again in waitForEvents, as it should be. netldi-3.log (failed connection) Before connection frame 8 is in waitForEvents (as above). After connection things are very different: #6 <signal handler called> #7 poll () from /lib64/libc.so.6 #8 __libc_res_nsend () from /lib64/libresolv.so.2 #9 __libc_res_nquery () from /lib64/libresolv.so.2 #10 _nss_dns_gethostbyaddr2_r () from /lib64/libnss_dns.so.2 #11 _nss_dns_gethostbyaddr_r () from /lib64/libnss_dns.so.2 #12 gethostbyaddr_r@@GLIBC_2.2.5 () from /lib64/libc.so.6 #13 getnameinfo () from /lib64/libc.so.6 #14 SocketGetPeerInfo () at build37291/src/socket.c:1714 #15 getServiceForClient () at build37291/src/nldicmn.c:1079 As requested, you repeated the signal after short pauses, and each of the three signals showed the process stuck in the same place (so either hung or a tight loop). So, the GemStone code (frame 14) is calling getnameinfo() in libc and it fails to return. In frames 10/11 we can see that it is attempting to do a DNS lookup of the host by the address. You were able to ping to MW from ML using both name and IP (as I suggested). The next thing to try is to see if name lookup works from the command line. After a ping gives you the IP address, try ‘host’ and ‘nslookup’ with the IP address as the argument. This should return the name (with other information). A brief Google search suggests that this may be an OS bug and might be fixed by upgrading your Linux kernel. See http://stackoverflow.com/questions/35880798/java-dns-resolution-hangs-forever. As a work-around, you might add MW to the /etc/hosts file on ML. This might avoid going to the DNS for the lookup. Alternatively, you could upgrade to GS/S 3.2.15 (or 3.3.3) and use the -N option on startnetldi to avoid the name lookup entirely. See https://downloads.gemtalksystems.com/docs/GemStone64/3.2.x/GS64-ReleaseNotes-3.2.15/1-ReleaseNotes.htm#pgfId-1613796. Let us know what you find! James
_______________________________________________ GemStone-Smalltalk mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
Hi James,
You found the problem :-) I have upgraded the linux kernel but still having problems. There is something wrong in my server or in the network, because 'nslookup' and 'host' fails to return the corresponding name of my MW (and of others machines too). But those commands works if I try to get the IP from the name. a)With Ip adress: - host 133.94.1.10 ;; connection timed out; no servers could be reached -nslookup 133.94.1.10 ;; Got SERVFAIL reply from 133.90.151.15, trying next server ;; Got SERVFAIL reply from 133.90.151.131, trying next server ;; connection timed out; trying next origin ;; Got SERVFAIL reply from 133.90.151.15, trying next server ;; Got SERVFAIL reply from 133.90.151.131, trying next server Got SERVFAIL reply;; connection timed out; no servers could be reached b)With name: -host COR8-DES-GMARAN COR8-DES-GMARAN.sibs.net has address 133.94.1.10 COR8-DES-GMARAN.sibs.net has IPv6 address 2002:855e:10a::855e:10a - nslookup COR8-DES-GMARAN Server: 133.90.151.131 Address: 133.90.151.131#53 Name: COR8-DES-GMARAN.sibs.net Address: 133.94.1.10 I have googled a lot, but I could not find something useful, or maybe yes but I didn't figured out that was important. Following your advice, I can connect by adding my ip and name to the hosts file in ML, but it will only work until my IP changes, because my IP is dynamic. So, summarizing: I have upgraded Linux kernel, but it does not fix the error. nslookup and host onyl resolves the ip from the name, but not the name from the IP. This seems to be some kind of problem with the DNS. I've connected via VPN to check DNS adresses and are the same that connected via wire. Do you have any idea about what the problem can be? I really appreciate your help James, you made a lot for me. Thanks! Ezequiel |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
Hi Ezequiel,
I’m glad that we have narrowed down the problem. From here, I’d say that getting reverse DNS working with Dynamic IP depends on the network tools used for each and is beyond the scope of the gemstone-smalltalk mailing list ;-). You will need to take that up with your network management team, but now that you can describe the problem in terms they can understand you may get more satisfaction. My suggestions are to (1) upgrade GemStone to avoid the reverse DNS lookup; (2) get a static IP address for MW from the network administrators; (3) manually add the current dynamic IP address to /etc/hosts (and update as needed); or (4) stick to the VPN connection. I assume that upgrading GemStone is the only thing that is entirely under your control (and 3.3.3 is better than 3.2.x). James > On Mar 30, 2017, at 8:25 AM, brianstone via GemStone-Smalltalk <[hidden email]> wrote: > > Hi James, > > You found the problem :-) > > I have upgraded the linux kernel but still having problems. > > There is something wrong in my server or in the network, because 'nslookup' > and 'host' fails to return the corresponding name of my MW (and of others > machines too). > But those commands works if I try to get the IP from the name. > > a)With Ip adress: > - host 133.94.1.10 > ;; connection timed out; no servers could be reached > > -nslookup 133.94.1.10 > ;; Got SERVFAIL reply from 133.90.151.15, trying next server > ;; Got SERVFAIL reply from 133.90.151.131, trying next server > ;; connection timed out; trying next origin > ;; Got SERVFAIL reply from 133.90.151.15, trying next server > ;; Got SERVFAIL reply from 133.90.151.131, trying next server > Got SERVFAIL reply;; connection timed out; no servers could be > reached > > b)With name: > -host COR8-DES-GMARAN > *COR8-DES-GMARAN.sibs.net has address 133.94.1.10* > COR8-DES-GMARAN.sibs.net has IPv6 address 2002:855e:10a::855e:10a > > - nslookup COR8-DES-GMARAN > Server: 133.90.151.131 > Address: 133.90.151.131#53 > > Name: COR8-DES-GMARAN.sibs.net > *Address: 133.94.1.10* > > I have googled a lot, but I could not find something useful, or maybe yes > but I didn't figured out that was important. > > Following your advice, I can connect by adding my ip and name to the hosts > file in ML, but it will only work until my IP changes, because my IP is > dynamic. > > So, summarizing: > > I have upgraded Linux kernel, but it does not fix the error. > /nslookup / and /host/ onyl resolves the ip from the name, but not the name > from the IP. > > This seems to be some kind of problem with the DNS. > I've connected via VPN to check DNS adresses and are the same that connected > via wire. > > Do you have any idea about what the problem can be? > > I really appreciate your help James, you made a lot for me. > Thanks! > > Ezequiel > > > > -- > View this message in context: http://forum.world.st/Error-with-netldi-the-connection-to-the-repository-is-never-made-and-the-port-becomes-blocked-tp4939542p4940569.html > Sent from the Gemstone/S mailing list archive at Nabble.com. > _______________________________________________ > GemStone-Smalltalk mailing list > [hidden email] > http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk _______________________________________________ GemStone-Smalltalk mailing list [hidden email] http://lists.gemtalksystems.com/mailman/listinfo/gemstone-smalltalk |
Re: Error with netldi, the connection to the repository is never made and the port becomes blocked
|
|
James,
You are right, now I can describe this problem better. I will talk to the network management team in order to resolve this issue. Thanks a lot for all your help. Kind Regards, Ezequiel |
«
Return to Gemstone/S
|
1 view|%1 views
| Free forum by Nabble | Edit this page |