[OpenSmalltalk/opensmalltalk-vm] In forgetXDisplay() do not call aioDisable() for the socket connection (#550)
to the X11 server if Linux epoll event handling is being used, because
forgetXDisplay() is typically called after a fork(). With epoll event
handling the shared fd reference affects both parent and child badly.
You can view, comment on, or merge this pull request online at:
Re: [OpenSmalltalk/opensmalltalk-vm] In forgetXDisplay() do not call aioDisable() for the socket connection (#550)
IMO, not a good fix.
The fact the the epoll structure is shared between parent and childs (forked) process is a problem not only for the X11 socket connection, but for ALL the polled fd.
We shall close the epoll fd and re-create it in the forked vm.
Probably, we should use a close-on-exec flag for epoll epoll_create1(EPOLL_CLOEXEC ), so as to protect from accidental sharing, unless someone foresee a good scenario for sharing epoll fd (I fail to see one).
I made a couple of commits to the Cog branch that provide error messages on stderr rather than segfault the VM. This shows clearly that after a fork, the X11 event notification for the child process is being delivered to both the child VM and the parent VM, and also that the kernel is supplying empty event data to the parent when this happens.
I suspect that for epoll, the rule should be never unregister the fd if you are also going to close the fd. This would be different from non-epoll event event handling, so possibly there would be a need for e.g. aioDisableAndClose(fd) in addition to the existing aioDisable(fd). Or maybe is it sufficient to just reopen the aiofd after a fork as suggested above.