Unix VM file locking?

Hi -

Had a fun morning this morning with some customer trouble which was
partly caused by the Unix VM not using write locks. E.g., the Unix VM
will happily do things like:

f1 := FileStream newFileNamed: 'foo.bar'.
f1 nextPutAll: 'Hello World'.
f2 := FileStream oldFileNamed: 'foo.bar'.
f2 nextPutAll: 'Oh, noes'.

etc. Is there an easy way to patch the Unix file code to avoid this,
e.g., behave like Windows in such that it would disallow opening the
same file for multiple writes?

Cheers,
   - Andreas

On 25-Jan-08, at 10:38 AM, Andreas Raab wrote:

> etc. Is there an easy way to patch the Unix file code to avoid this,  
> e.g., behave like Windows in such that it would disallow opening the  
> same file for multiple writes?

I've been bitten by this as well, but in reverse. Code that works  
great on Unix is totally broken on Windows because you can't open the  
same file for writing twice.

Colin

Colin Putney wrote:
Easy to change. In sqWin32FilePrims.c function sqFileOpen() modify:

   h = CreateFileW(win32Path,
         writeFlag ? (GENERIC_READ | GENERIC_WRITE) : GENERIC_READ,
         writeFlag ? FILE_SHARE_READ : (FILE_SHARE_READ | FILE_SHARE_WRITE),
       "... etc ..."

to read:

   h = CreateFileW(win32Path,
         writeFlag ? (GENERIC_READ | GENERIC_WRITE) : GENERIC_READ,
         FILE_SHARE_READ | FILE_SHARE_WRITE,
       "... etc ..."

So what's the equivalent change for Unix?

Cheers,
   - Andreas

You can use flock/fcntl for placing an exclusive lock on file descriptor.

http://linux.die.net/man/2/flock

On 26/01/2008, Andreas Raab <[hidden email]> wrote:

--
Best regards,
Igor Stasenko AKA sig.

Hmmm, not sure if that works. flock is process based, and the process
is the VM here -- so I'm guessing that the snippet in the first email
in this thread will still have the broken behavior.

On Jan 26, 2008 2:28 PM, Igor Stasenko <[hidden email]> wrote:


--
Miron Brezuleanu

On Fri, Jan 25, 2008 at 05:12:39PM -0800, Andreas Raab wrote:
The Unix VM is opening the file with fopen(), which does not provide
such a control. However, you could use open() to open a file descriptor
with the O_EXCL and O_CREAT flags, then pass that file descriptor to
fdopen() to get the (FILE *) handle. I have not tested this, but I
think it would do what you are looking for here.

Dave

p.s. The Windows VM uses lower level HANDLE rather than the (FILE *)
used on other VMs. It would be interesting to know which approach
produces the best overall Squeak performance for applications that
do significant file I/O.

On Sat, Jan 26, 2008 at 02:28:11PM +0200, Igor Stasenko wrote:
> You can use flock/fcntl for placing an exclusive lock on file descriptor.
>
> http://linux.die.net/man/2/flock
>

This kind of file locking is implemented in OSProcess, see UnixOSProcessAccessor
category "file locking". For the problem that Andreas describes, you can do this:

OSProcessAccessor emulateWin32FileLocking: true. "dangerous for multiple OS processes"
f1 := FileStream newFileNamed: 'foo.bar'.
OSProcess accessor lockFile: f1 exclusive: true ifFail: [f1 close. self error: 'could not lock file'].
f1 nextPutAll: 'Hello World'.
f2 := FileStream oldFileNamed: 'foo.bar'.
OSProcess accessor lockFile: f2 exclusive: true ifFail: [f2 close. self error: 'could not lock file'].
f2 nextPutAll: 'Oh, noes'.

Look first at the unit tests UnixProcessFileLockTestCase, UnixProcessUnixFileLockingTestCase,
and UnixProcessWin32FileLockingTestCase. These attempt to provide some reasonable
test coverage between cooperating Squeak images by means of #forkSqueak.

For this issue (locking the external file from a single Squeak image), the
Windows file locking semantics give the right behavior. However (and this
is a *big* caveat), the Windows semantics (emulated on Unix) will not work
if you are trying to lock files between multiple OS processes, which is of
course the intent of file locking in the first place, so you should use
this only if you are sure that multiple OS process access is not every going
to be an issue for your application. The default setting is to not use Windows
file lock emulation (OSProcessAccessor emulateWin32FileLocking: false), which
is what you want for the more general case of locking files across multiple
OS processes.

Overall, you should use OSProcess file locking with caution. I have tried
to provide test coverage, but I do not know how much (if at all) anyone is
using this in a production setting, and I can't be certain how well it will
hold up under load. That caveat aside, it should work fine for the sort of
issue described here.

Dave  

 

Hi Andreas,

in 2005 at my work at netstyle.ch I wrote a plugin for the Unix and
Mac VM that directly accesses the Posix system level file functions.
It avoids all the bugs in the standard Squeak file-handling and allows
to reliably lock files on a per VM bases.

The package consists of the VM plugin, a file-accessor class and some
tests. It also consists of an OmniBase compatible stream class
(OmniBase has a different Stream hierarchy than Squeak) that tries to
emulate in-image file-locking, but as far as I remember this never
reliably worked.

Cheers,
Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch