Well somewhere someone needs to close the socket before it is
*forgotten* On 30-Apr-09, at 5:29 AM, Stéphane Ducasse wrote: > I would like to know why does the system rely on finalisation to > release socket > Apparently david mentioned that this is the source of huge problems. > So what would be the alternative? > > Stef -- = = = ======================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ======================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Cameron Sanders-3
So is that in the exception block before you get the walkback on the
socket allocation failure? On 30-Apr-09, at 8:45 AM, Cameron Sanders wrote: > Well, "Socket allInstances size inspect" is giving me numbers like 8, > 6, or 2 from the "ScriptLoader loadOBAlpha" do-it. When I look at the > process on the Mac (with the activity monitor), it says I have "Ports: > 91". A look at "open files and ports" shows a lengthy list of > filenames/devices that are open, but only about 35 show up. There are > a few surprises (to me) in the open files list: > -- = = = ======================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ======================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Very strange...
After a while on linux, I have an error in MCHttpRepository>>#allFileNames 'Could not resolve the server named: source.lukas-renggli.ch' while executing: instClass lukas install: 'OB-Regex'; the 6 opened fd turned into many ... all pointing to the change file Pharo0.1Core-10281cl.changes ls -l /proc/7932/fd | wc -l 577 ls -l /proc/7932/fd | sort -n -k 8 | head -20 | grep -v '^total' lrwx------ 1 nicolas parents 64 2009-04-30 23:03 0 -> /dev/pts/3 lrwx------ 1 nicolas parents 64 2009-04-30 23:03 1 -> /dev/pts/3 lrwx------ 1 nicolas parents 64 2009-04-30 23:03 2 -> /dev/pts/3 lrwx------ 1 nicolas parents 64 2009-04-30 23:03 3 -> socket:[19987] lr-x------ 1 nicolas parents 64 2009-04-30 23:03 4 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/package-cache/ lr-x------ 1 nicolas parents 64 2009-04-30 23:03 5 -> /windata/Smalltalk/Squeak/SqueakV39.sources* lrwx------ 1 nicolas parents 64 2009-04-30 23:03 6 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 7 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 8 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 9 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 10 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 11 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 12 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 13 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 14 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 15 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 16 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 17 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* lr-x------ 1 nicolas parents 64 2009-04-30 23:03 18 -> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* etc... Trying to restart, i have serverAdress 84.75.162.243(84-75-162-243.dclient.hispeed.ch),0(0) which I cannot ping outside squeak... So maybe squeak has correct diagnostic, but why all these opened change logs ? Nicolas 2009/4/30 John M McIntosh <[hidden email]>: > So is that in the exception block before you get the walkback on the > socket allocation failure? > > On 30-Apr-09, at 8:45 AM, Cameron Sanders wrote: > >> Well, "Socket allInstances size inspect" is giving me numbers like 8, >> 6, or 2 from the "ScriptLoader loadOBAlpha" do-it. When I look at the >> process on the Mac (with the activity monitor), it says I have "Ports: >> 91". A look at "open files and ports" shows a lengthy list of >> filenames/devices that are open, but only about 35 show up. There are >> a few surprises (to me) in the open files list: >> > > -- > = > = > = > ======================================================================== > John M. McIntosh <[hidden email]> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com > = > = > = > ======================================================================== > > > > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
With a simpler command, I reproduced the MC failure, but not the bunch
of opened change log: 1) ScriptLoader new installingInstaller 2) Installer lukas install: 'OB-Regex'. I get this interesting message on the XTerm console: getaddrinfo: Name or service not known 2009/4/30 Nicolas Cellier <[hidden email]>: > Very strange... > After a while on linux, I have an error in MCHttpRepository>>#allFileNames > > 'Could not resolve the server named: source.lukas-renggli.ch' > > while executing: > > instClass lukas > install: 'OB-Regex'; > > the 6 opened fd turned into many ... all pointing to the change file > Pharo0.1Core-10281cl.changes > > ls -l /proc/7932/fd | wc -l > 577 > > ls -l /proc/7932/fd | sort -n -k 8 | head -20 | grep -v '^total' > lrwx------ 1 nicolas parents 64 2009-04-30 23:03 0 -> /dev/pts/3 > lrwx------ 1 nicolas parents 64 2009-04-30 23:03 1 -> /dev/pts/3 > lrwx------ 1 nicolas parents 64 2009-04-30 23:03 2 -> /dev/pts/3 > lrwx------ 1 nicolas parents 64 2009-04-30 23:03 3 -> socket:[19987] > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 4 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/package-cache/ > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 5 -> > /windata/Smalltalk/Squeak/SqueakV39.sources* > lrwx------ 1 nicolas parents 64 2009-04-30 23:03 6 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 7 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 8 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 9 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 10 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 11 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 12 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 13 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 14 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 15 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 16 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 17 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > lr-x------ 1 nicolas parents 64 2009-04-30 23:03 18 -> > /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/Pharo0.1Core-10281cl.changes* > etc... > > Trying to restart, i have > serverAdress 84.75.162.243(84-75-162-243.dclient.hispeed.ch),0(0) > which I cannot ping outside squeak... > So maybe squeak has correct diagnostic, but why all these opened change logs ? > > Nicolas > > 2009/4/30 John M McIntosh <[hidden email]>: >> So is that in the exception block before you get the walkback on the >> socket allocation failure? >> >> On 30-Apr-09, at 8:45 AM, Cameron Sanders wrote: >> >>> Well, "Socket allInstances size inspect" is giving me numbers like 8, >>> 6, or 2 from the "ScriptLoader loadOBAlpha" do-it. When I look at the >>> process on the Mac (with the activity monitor), it says I have "Ports: >>> 91". A look at "open files and ports" shows a lengthy list of >>> filenames/devices that are open, but only about 35 show up. There are >>> a few surprises (to me) in the open files list: >>> >> >> -- >> = >> = >> = >> ======================================================================== >> John M. McIntosh <[hidden email]> Twitter: >> squeaker68882 >> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com >> = >> = >> = >> ======================================================================== >> >> >> >> >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Nicolas Cellier
On Thu, Apr 30, 2009 at 11:48:52PM +0200, Nicolas Cellier wrote:
> Very strange... > After a while on linux, I have an error in MCHttpRepository>>#allFileNames > > 'Could not resolve the server named: source.lukas-renggli.ch' > > while executing: > > instClass lukas > install: 'OB-Regex'; > > the 6 opened fd turned into many ... all pointing to the change file > Pharo0.1Core-10281cl.changes Well that's a bug for sure. Somebody is opening new file streams on the changes file, probably by accident. And not closing the one they opened last time. The bug in changes file handling could easily appear to be a socket problem, because the per-process limit on open descriptors would be shared between files and sockets (and pipes and ...). So find out why the changes file is being accidentally reopened, and the "socket problem" will probably go away. Dave _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Well yes, and the magic check would be
FileStream allSubInstances size inspect same place as the socket. Nothing of course finalization is involved in getting rid of zombie file handles too. And Dave is right you should only have one file handle open (read/ write) for *.changes On 30-Apr-09, at 4:30 PM, David T. Lewis wrote: > Well that's a bug for sure. Somebody is opening new file streams on > the changes file, probably by accident. And not closing the one they > opened last time. > > The bug in changes file handling could easily appear to be a socket > problem, because the per-process limit on open descriptors would be > shared between files and sockets (and pipes and ...). So find out > why the changes file is being accidentally reopened, and the "socket > problem" will probably go away. > > Dave -- = = = ======================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ======================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Nicolas Cellier
On a 293/OSX, I get a primitive failed.
When (HTTPSocket>>httpGetDocument: url args: args accept: mimeType request: requestString) calls (NetNameResolver addressForName: connectToHost timeout: 20.) connectToHost = '', which makes the primitive fail. Alexandre On 1 May 2009, at 00:11, Nicolas Cellier wrote: > With a simpler command, I reproduced the MC failure, but not the bunch > of opened change log: > > 1) ScriptLoader new installingInstaller > 2) Installer lukas install: 'OB-Regex'. > > I get this interesting message on the XTerm console: > > getaddrinfo: Name or service not known > > > 2009/4/30 Nicolas Cellier <[hidden email]>: >> Very strange... >> After a while on linux, I have an error in >> MCHttpRepository>>#allFileNames >> >> 'Could not resolve the server named: source.lukas-renggli.ch' >> >> while executing: >> >> instClass lukas >> install: 'OB-Regex'; >> >> the 6 opened fd turned into many ... all pointing to the change file >> Pharo0.1Core-10281cl.changes >> >> ls -l /proc/7932/fd | wc -l >> 577 >> >> ls -l /proc/7932/fd | sort -n -k 8 | head -20 | grep -v '^total' >> lrwx------ 1 nicolas parents 64 2009-04-30 23:03 0 -> /dev/pts/3 >> lrwx------ 1 nicolas parents 64 2009-04-30 23:03 1 -> /dev/pts/3 >> lrwx------ 1 nicolas parents 64 2009-04-30 23:03 2 -> /dev/pts/3 >> lrwx------ 1 nicolas parents 64 2009-04-30 23:03 3 -> socket:[19987] >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 4 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/package-cache/ >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 5 -> >> /windata/Smalltalk/Squeak/SqueakV39.sources* >> lrwx------ 1 nicolas parents 64 2009-04-30 23:03 6 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 7 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 8 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 9 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 10 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 11 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 12 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 13 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 14 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 15 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 16 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 17 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> lr-x------ 1 nicolas parents 64 2009-04-30 23:03 18 -> >> /windata/Smalltalk/Squeak/pharo0.1-10268dev09.04.1/ >> Pharo0.1Core-10281cl.changes* >> etc... >> >> Trying to restart, i have >> serverAdress 84.75.162.243(84-75-162-243.dclient.hispeed.ch),0(0) >> which I cannot ping outside squeak... >> So maybe squeak has correct diagnostic, but why all these opened >> change logs ? >> >> Nicolas >> >> 2009/4/30 John M McIntosh <[hidden email]>: >>> So is that in the exception block before you get the walkback on the >>> socket allocation failure? >>> >>> On 30-Apr-09, at 8:45 AM, Cameron Sanders wrote: >>> >>>> Well, "Socket allInstances size inspect" is giving me numbers >>>> like 8, >>>> 6, or 2 from the "ScriptLoader loadOBAlpha" do-it. When I look at >>>> the >>>> process on the Mac (with the activity monitor), it says I have >>>> "Ports: >>>> 91". A look at "open files and ports" shows a lengthy list of >>>> filenames/devices that are open, but only about 35 show up. There >>>> are >>>> a few surprises (to me) in the open files list: >>>> >>> >>> -- >>> = >>> = >>> = >>> = >>> = >>> = >>> = >>> ==================================================================== >>> John M. McIntosh <[hidden email]> Twitter: >>> squeaker68882 >>> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com >>> = >>> = >>> = >>> = >>> = >>> = >>> = >>> ==================================================================== >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> [hidden email] >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
The Unix ipv6 logic translates the lookup error number into a string
and prints to the console. If the name is "" an empty string, then yes it's EAI_NONAME see man 3 gai_strerror GAI_STRERROR(3) BSD Library Functions Manual GAI_STRERROR(3) NAME gai_strerror -- get error message string from EAI_xxx error code SYNOPSIS #include <sys/types.h> #include <sys/socket.h> #include <netdb.h> const char * gai_strerror(int ecode); DESCRIPTION The gai_strerror() function returns an error message string corresponding to the error code returned by getaddrinfo(3) or getnameinfo(3). The following error codes and their meaning are defined in <netdb.h>: EAI_AGAIN temporary failure in name resolution EAI_BADFLAGS invalid value for ai_flags EAI_BADHINTS invalid value for hints EAI_FAIL non-recoverable failure in name resolution EAI_FAMILY ai_family not supported EAI_MEMORY memory allocation failure EAI_NONAME hostname or servname not provided, or not known EAI_PROTOCOL resolved protocol is unknown EAI_SERVICE servname not supported for ai_socktype EAI_SOCKTYPE ai_socktype not supported EAI_SYSTEM system error returned in errno On 30-Apr-09, at 10:27 PM, Alexandre Bergel wrote: > On a 293/OSX, I get a primitive failed. > When (HTTPSocket>>httpGetDocument: url args: args accept: mimeType > request: requestString) > calls > (NetNameResolver addressForName: connectToHost timeout: 20.) > > connectToHost = '', which makes the primitive fail. > > Alexandre > > > On 1 May 2009, at 00:11, Nicolas Cellier wrote: > >> With a simpler command, I reproduced the MC failure, but not the >> bunch >> of opened change log: >> >> 1) ScriptLoader new installingInstaller >> 2) Installer lukas install: 'OB-Regex'. >> >> I get this interesting message on the XTerm console: >> >> getaddrinfo: Name or service not known -- = = = ======================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ======================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by johnmci
Hmm...
I suspect my linux /proc has been foolished for some reason and it was in fact some sockets... I tried again, during loading, the socket opened and close and were recognized as such in /proc, but after a certain point, listing /proc/$pid/fd was reporting change log open and close... though the loading did continue normally. I will try the whole procedure again, but all this is really slow (I had some other traffic going on my limited bandwith). Nicolas 2009/5/1 John M McIntosh <[hidden email]>: > Well yes, and the magic check would be > > FileStream allSubInstances size inspect > > same place as the socket. > > Nothing of course finalization is involved in getting rid of zombie > file handles too. > > And Dave is right you should only have one file handle open (read/ > write) for *.changes > > > On 30-Apr-09, at 4:30 PM, David T. Lewis wrote: > >> Well that's a bug for sure. Somebody is opening new file streams on >> the changes file, probably by accident. And not closing the one they >> opened last time. >> >> The bug in changes file handling could easily appear to be a socket >> problem, because the per-process limit on open descriptors would be >> shared between files and sockets (and pipes and ...). So find out >> why the changes file is being accidentally reopened, and the "socket >> problem" will probably go away. >> >> Dave > > -- > = > = > = > ======================================================================== > John M. McIntosh <[hidden email]> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com > = > = > = > ======================================================================== > > > > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by johnmci
I can repeat my problem on linux, don't know if at all the same as Max users.
It happens while doing: Installer wiresong project: 'ob'; install: 'OB-Refactory'. Once the load is finished: FileStream allSubInstances size -> 454 ls -l /proc/9091/fd | grep changes | wc -l -> 570 A little mismatch here... Note that it reached 1018 during load, for a total of 1024 opened fd, the upper limit... ls -l /proc/9091/fd | grep -v total | wc -l -> 1024 I can remove in image file stream: Smalltalk collectGarbage. FileStream allSubInstances size -> 4 But the file descriptors are not closed: ls -l /proc/9091/fd | grep changes | wc -l -> 570 Argh! After a garbage collect, I can proceed loading: Installer lukas project: 'omnibrowser'; install: 'OB-Regex'; install: 'OB-Tools' But dragging these 570 opened change log is not very sane... Gonna inquire if I can install a strace on my box. Nicolas 2009/5/1 John M McIntosh <[hidden email]>: > Well yes, and the magic check would be > > FileStream allSubInstances size inspect > > same place as the socket. > > Nothing of course finalization is involved in getting rid of zombie > file handles too. > > And Dave is right you should only have one file handle open (read/ > write) for *.changes > > > On 30-Apr-09, at 4:30 PM, David T. Lewis wrote: > >> Well that's a bug for sure. Somebody is opening new file streams on >> the changes file, probably by accident. And not closing the one they >> opened last time. >> >> The bug in changes file handling could easily appear to be a socket >> problem, because the per-process limit on open descriptors would be >> shared between files and sockets (and pipes and ...). So find out >> why the changes file is being accidentally reopened, and the "socket >> problem" will probably go away. >> >> Dave > > -- > = > = > = > ======================================================================== > John M. McIntosh <[hidden email]> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com > = > = > = > ======================================================================== > > > > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Nicolas Cellier
Forget it. I was the only one foolished :)
strace showed it was effectively an attempt to open more than 1018 change log... Then in absence of garbage collect every socket attempt just fails... (Too many open files) If we find the guilty change log opener, this will fix the issue at least on my linux box. Otherwise inserting a retryWithGC: here and there is a possible temporary workaround... But those files will have to be closed... 2009/5/1 Nicolas Cellier <[hidden email]>: > Hmm... > I suspect my linux /proc has been foolished for some reason and it was > in fact some sockets... > I tried again, during loading, the socket opened and close and were > recognized as such in /proc, > but after a certain point, listing /proc/$pid/fd was reporting change > log open and close... though the loading did continue normally. > I will try the whole procedure again, but all this is really slow (I > had some other traffic going on my limited bandwith). > Nicolas > > 2009/5/1 John M McIntosh <[hidden email]>: >> Well yes, and the magic check would be >> >> FileStream allSubInstances size inspect >> >> same place as the socket. >> >> Nothing of course finalization is involved in getting rid of zombie >> file handles too. >> >> And Dave is right you should only have one file handle open (read/ >> write) for *.changes >> >> >> On 30-Apr-09, at 4:30 PM, David T. Lewis wrote: >> >>> Well that's a bug for sure. Somebody is opening new file streams on >>> the changes file, probably by accident. And not closing the one they >>> opened last time. >>> >>> The bug in changes file handling could easily appear to be a socket >>> problem, because the per-process limit on open descriptors would be >>> shared between files and sockets (and pipes and ...). So find out >>> why the changes file is being accidentally reopened, and the "socket >>> problem" will probably go away. >>> >>> Dave >> >> -- >> = >> = >> = >> ======================================================================== >> John M. McIntosh <[hidden email]> Twitter: >> squeaker68882 >> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com >> = >> = >> = >> ======================================================================== >> >> >> >> >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by johnmci
Strange, I have figures different from Nicolas:
-- before ScriptLoader loadOBAlpha -- Socket allInstances size 0 HTTPSocket allInstances size 0 FileStream allInstances size 0 -- after ScriptLoader loadOBAlpha, the debugger is still up -- Socket allInstances size 0 HTTPSocket allInstances size 2 FileStream allInstances size 0 Alexandre On 1 May 2009, at 04:04, John M McIntosh wrote: > Well yes, and the magic check would be > > FileStream allSubInstances size inspect > > same place as the socket. > > Nothing of course finalization is involved in getting rid of zombie > file handles too. > > And Dave is right you should only have one file handle open (read/ > write) for *.changes > > > On 30-Apr-09, at 4:30 PM, David T. Lewis wrote: > >> Well that's a bug for sure. Somebody is opening new file streams on >> the changes file, probably by accident. And not closing the one they >> opened last time. >> >> The bug in changes file handling could easily appear to be a socket >> problem, because the per-process limit on open descriptors would be >> shared between files and sockets (and pipes and ...). So find out >> why the changes file is being accidentally reopened, and the "socket >> problem" will probably go away. >> >> Dave > > -- > = > = > = > = > = > ====================================================================== > John M. McIntosh <[hidden email]> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http:// > www.smalltalkconsulting.com > = > = > = > = > = > ====================================================================== > > > > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
That would be
> FileStream allSubInstances size inspect not FileStream allInstances size On 2-May-09, at 12:13 AM, Alexandre Bergel wrote: > Strange, I have figures different from Nicolas: > > -- before ScriptLoader loadOBAlpha -- > Socket allInstances size 0 > HTTPSocket allInstances size 0 > FileStream allInstances size 0 > > -- after ScriptLoader loadOBAlpha, the debugger is still up -- > Socket allInstances size 0 > HTTPSocket allInstances size 2 > FileStream allInstances size 0 > > Alexandre > > > On 1 May 2009, at 04:04, John M McIntosh wrote: > >> Well yes, and the magic check would be >> >> FileStream allSubInstances size inspect -- = = = ======================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ======================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
-- before ScriptLoader loadOBAlpha --
Socket allSubInstances size 0 HTTPSocket allSubInstances size 0 FileStream allSubInstances size 4 -- after ScriptLoader loadOBAlpha, the debugger is still up -- Socket allSubInstances size 2 HTTPSocket allSubInstances size 2 FileStream allSubInstances size 56 Alexandre On 2 May 2009, at 09:14, John M McIntosh wrote: > That would be > >> FileStream allSubInstances size inspect > > not > > FileStream allInstances size > > > On 2-May-09, at 12:13 AM, Alexandre Bergel wrote: > >> Strange, I have figures different from Nicolas: >> >> -- before ScriptLoader loadOBAlpha -- >> Socket allInstances size 0 >> HTTPSocket allInstances size 0 >> FileStream allInstances size 0 >> >> -- after ScriptLoader loadOBAlpha, the debugger is still up -- >> Socket allInstances size 0 >> HTTPSocket allInstances size 2 >> FileStream allInstances size 0 >> >> Alexandre >> >> >> On 1 May 2009, at 04:04, John M McIntosh wrote: >> >>> Well yes, and the magic check would be >>> >>> FileStream allSubInstances size inspect > > -- > = > = > = > = > = > ====================================================================== > John M. McIntosh <[hidden email]> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http:// > www.smalltalkconsulting.com > = > = > = > = > = > ====================================================================== > > > > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by johnmci
Yes but why don't we close them with an ensure or something like that.
I mean is the logic of the connection making it that we cannot use ensure or this is just a legacy? On Apr 30, 2009, at 7:20 PM, John M McIntosh wrote: > Well somewhere someone needs to close the socket before it is > *forgotten* > > On 30-Apr-09, at 5:29 AM, Stéphane Ducasse wrote: > >> I would like to know why does the system rely on finalisation to >> release socket >> Apparently david mentioned that this is the source of huge problems. >> So what would be the alternative? >> >> Stef > > -- > = > = > = > = > = > ====================================================================== > John M. McIntosh <[hidden email]> Twitter: > squeaker68882 > Corporate Smalltalk Consulting Ltd. http:// > www.smalltalkconsulting.com > = > = > = > = > = > ====================================================================== > > > > > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Alexandre Bergel
Apparnetly after 10296 I can load moose without problem.
So the fixes of nicolas closing remoteString solved the problem. Now this is only partially solved since I would really like to understand why it broke in the first place since we did not touch this code. Stef _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Oh, remoteString!!! I LOVE IT :)))
Stephane, remember this mail ? I didn't sent it to list. Sending now, just as a reminder. 9 December 2008 05:20 subject One big mess :) Here is an IRC log me chatting with Keith. I just hope that Pharo guys will address this issue somewhere in future. [03:36] <sig__> btw, i just found your #7239 [03:36] <sig__> i'm also took a look at this code time ago [03:36] <sig__> and i would recommend following: [03:37] <sig__> - completely remove using RemoteString from practice. [03:37] <sig__> - make sure that access to source streams is synchronized [03:37] <sig__> (no concurrency issues) [03:37] *** optikalmouse (n=[hidden email]) joined [03:38] <keithy> its a big job I started looking at it last wek [03:38] <sig__> RemoteString is very , very , very brittle thing [03:38] <keithy> do add you comment [03:39] <sig__> i wouldn't pass anything except actual source code outside source handling/management code [03:39] <keithy> definitely [03:39] <keithy> hmm [03:39] *** wgsilkie (n=aidan@203.109.162.247) joined [03:39] <keithy> or a proxy? [03:39] <sig__> remote string is a proxy! [03:40] <keithy> so... "I am a method" could I have my source please [03:40] <sig__> but you can easily damage source files with it [03:40] <sig__> or make it read complete mess instead of code [03:42] <sig__> because , from what i have seen, it assumes that there is nobody else accessing the source stream ATM, except single instance of RemoteString [03:46] <sig__> as well, as i don't like when different code in different classes able to manipulate with the source streams directly [03:46] <keithy> its ludicrous [03:46] <sig__> by passing array with two elements around [03:49] <sig__> check the references to SourceFiles :) [03:50] <sig__> 55 references in my image! [03:51] <sig__> a single, big, open wide security HOLE! :) [03:52] <keithy> and lots of horrible code [03:52] <sig__> yeah [03:52] *** wgsilkie quit (Remote closed the connection) [03:53] <sig__> (SourceFiles at: 2) [03:53] <sig__> very illustrative bit of code :) [03:54] *** bmp (n=[hidden email]) joined [03:55] <sig__> things like that makes me smile, when Edgar says "my image smaller , and therefore more modular" [03:55] <sig__> for GOD SAKE!! [03:59] <keithy> hmm, if you gzip the image does it get more modular? [04:00] <sig__> it is clear, that source management subsystem deserves a separate, well defined protocol. Without any exposure where it reads/takes source from, or where storing to [04:03] <sig__> and i can't call it a bug, it doesn't fits into any imaginable bugs category, simply because its not a bug, but design flaw [04:04] <sig__> and i think it would require few weeks to make it fixed [04:04] <keithy> an implementation flaw, no design obvious [04:04] *** lamneth2 (n=[hidden email]) joined [04:05] <sig__> well, an implementation should always follow a design [04:05] <sig__> actually a bug, to my sense, is an implementation flaw [04:06] <sig__> when it doesn't follows design [04:06] <sig__> but design flaw is when everything works (no bugs), but it works in an awfull way :) 2009/5/2 Stéphane Ducasse <[hidden email]>: > Apparnetly after 10296 I can load moose without problem. > So the fixes of nicolas closing remoteString solved the problem. > Now this is only partially solved since I would really like to > understand > why it broke in the first place since we did not touch this code. > > Stef > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Stéphane Ducasse
Beware: there are two issues here, not one.
(a) the faulty remoteString code is ill behaved. Fine that can be fixed or as Igor suggested refactored to oblivion. (b) The deeper issue is that we affected how finalization works so that this ill behavior now causes application failure. If in the past it worked, now it doesn't I don't see anyone really having a good answer other than perhaps my guess, and how do we get back to the point were ill behaviour by *my* code won't cause Socket Failures. No doubt people have ugly code that is *silently* busted like RemoteString, but they don't know it. And as you saw actually finding the culprit is difficult. Lastly some people DO use finalization to do resource cleanup on purpose, not as a safety fallback, so they will be impacted I think by the new Pharo behaviour. Wasn't finalization tagged as bloated and need fixing? Igor I'm sure suggested a few things before the flurry of code for optimizing semaphores and process switching? On 2-May-09, at 1:52 PM, Stéphane Ducasse wrote: > Yes but why don't we close them with an ensure or something like that. > I mean is the logic of the connection making it that we cannot use > ensure or this is just a legacy? > > On Apr 30, 2009, at 7:20 PM, John M McIntosh wrote: > >> Well somewhere someone needs to close the socket before it is >> *forgotten* >> >> On 30-Apr-09, at 5:29 AM, Stéphane Ducasse wrote: >> >>> I would like to know why does the system rely on finalisation to >>> release socket >>> Apparently david mentioned that this is the source of huge problems. >>> So what would be the alternative? >>> >>> Stef >> -- = = = ======================================================================== John M. McIntosh <[hidden email]> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com = = = ======================================================================== _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
2009/5/3 John M McIntosh <[hidden email]>:
> Beware: there are two issues here, not one. > > (a) the faulty remoteString code is ill behaved. Fine that can be > fixed or as Igor suggested refactored to oblivion. > IMO this should be done sooner or later. Its a pain to see it. Oh well, too bad, this is not the only thing which requires attention :) > (b) The deeper issue is that we affected how finalization works so > that this ill behavior now causes application failure. > If in the past it worked, now it doesn't I don't see anyone really > having a good answer other than perhaps my guess, > and how do we get back to the point were ill behaviour by *my* code > won't cause Socket Failures. > I used finalization in my code multiple times. It didn't caused many problems, if you know how to do it right (by taking an implementation into account). > No doubt people have ugly code that is *silently* busted like > RemoteString, but they don't know it. > And as you saw actually finding the culprit is difficult. > > Lastly some people DO use finalization to do resource cleanup on > purpose, not as a safety fallback, so they will > be impacted I think by the new Pharo behaviour. > It looks like i missed this part. Where i can read details about what is altered in finalization, which could break things? > Wasn't finalization tagged as bloated and need fixing? Igor I'm sure > suggested a few things before the flurry of > code for optimizing semaphores and process switching? > Yes, i proposed a little change to VM (just a couple lines of code in single method + registering additional special object) which would allow us more direct finalization. It not makes weak refs to be a full pledged ephemerons, but much easier to adopt in VM. And besides you wont find any finalization code in squeak which relies on ephemeron's special behavior -- reference 'value' slot weakly, only when reference in 'key' slot is dies. http://www.nabble.com/An-idea-about-better-finalization-support-td23186805.html > On 2-May-09, at 1:52 PM, Stéphane Ducasse wrote: > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
In reply to this post by Igor Stasenko
On May 3, 2009, at 1:09 AM, Igor Stasenko wrote: > Oh, remoteString!!! I LOVE IT :))) > > Stephane, remember this mail ? no :) but I agree we will have to address that in the future. But this will be a bit more complex > > I didn't sent it to list. Sending now, just as a reminder. > > 9 December 2008 05:20 > subject One big mess :) > > Here is an IRC log me chatting with Keith. > > I just hope that Pharo guys will address this issue somewhere in > future. > > [03:36] <sig__> btw, i just found your #7239 > [03:36] <sig__> i'm also took a look at this code time ago > [03:36] <sig__> and i would recommend following: > [03:37] <sig__> - completely remove using RemoteString from practice. > [03:37] <sig__> - make sure that access to source streams is > synchronized > [03:37] <sig__> (no concurrency issues) > [03:37] *** optikalmouse (n=user@bas1- > toronto10-1279398477.dsl.bell.ca) joined > [03:38] <keithy> its a big job I started looking at it last wek > [03:38] <sig__> RemoteString is very , very , very brittle thing yes a terrible one :) > > [03:38] <keithy> do add you comment > [03:39] <sig__> i wouldn't pass anything except actual source code > outside source handling/management code > [03:39] <keithy> definitely > [03:39] <keithy> hmm > [03:39] *** wgsilkie (n=aidan@203.109.162.247) joined > [03:39] <keithy> or a proxy? > [03:39] <sig__> remote string is a proxy! > [03:40] <keithy> so... "I am a method" could I have my source please > [03:40] <sig__> but you can easily damage source files with it > [03:40] <sig__> or make it read complete mess instead of code > [03:42] <sig__> because , from what i have seen, it assumes that there > is nobody else accessing the source stream ATM, except single instance > of RemoteString > [03:46] <sig__> as well, as i don't like when different code in > different classes able to manipulate with the source streams directly > [03:46] <keithy> its ludicrous > [03:46] <sig__> by passing array with two elements around > [03:49] <sig__> check the references to SourceFiles :) > [03:50] <sig__> 55 references in my image! > [03:51] <sig__> a single, big, open wide security HOLE! :) > [03:52] <keithy> and lots of horrible code > [03:52] <sig__> yeah I totally agree. > > [03:52] *** wgsilkie quit (Remote closed the connection) > [03:53] <sig__> (SourceFiles at: 2) > [03:53] <sig__> very illustrative bit of code :) > [03:54] *** bmp > (n=[hidden email] > ) > joined > [03:55] <sig__> things like that makes me smile, when Edgar says "my > image smaller , and therefore more modular" > [03:55] <sig__> for GOD SAKE!! > [03:59] <keithy> hmm, if you gzip the image does it get more modular? > [04:00] <sig__> it is clear, that source management subsystem deserves > a separate, well defined protocol. Without any exposure where it > reads/takes source from, or where storing to > [04:03] <sig__> and i can't call it a bug, it doesn't fits into any > imaginable bugs category, simply because its not a bug, but design > flaw > [04:04] <sig__> and i think it would require few weeks to make it > fixed > [04:04] <keithy> an implementation flaw, no design obvious > [04:04] *** lamneth2 (n=[hidden email] > ) joined > [04:05] <sig__> well, an implementation should always follow a design > [04:05] <sig__> actually a bug, to my sense, is an implementation flaw > [04:06] <sig__> when it doesn't follows design > [04:06] <sig__> but design flaw is when everything works (no bugs), > but it works in an awfull way :) > > > > 2009/5/2 Stéphane Ducasse <[hidden email]>: >> Apparnetly after 10296 I can load moose without problem. >> So the fixes of nicolas closing remoteString solved the problem. >> Now this is only partially solved since I would really like to >> understand >> why it broke in the first place since we did not touch this code. >> >> Stef >> >> _______________________________________________ >> Pharo-project mailing list >> [hidden email] >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > [hidden email] > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list [hidden email] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project |
Free forum by Nabble | Edit this page |