[squeak-dev] Strange Unix file behavior?

Hi -

I just noticed the following strange behavior on Unix:

FileDirectory default fileExists: 'foo'.
   => false "as expected"

FileDirectory default fileExists: ''.
   => false "as expected"

FileDirectory default fileExists: '.'.
   => false "as expected"

FileDirectory default fileExists: '..'.
   => false "as expected"

FileDirectory default readOnlyFileNamed: 'foo'.
   => FileDoesNotExistException "as expected"

FileDirectory default readOnlyFileNamed: ''.
   => MultiByteFileStream: '/home/qwaq/server/bin/forums/' "Huh?"

FileDirectory default readOnlyFileNamed: '.'.
   => MultiByteFileStream: '/home/qwaq/server/bin/forums/.' "WTF?"

FileDirectory default readOnlyFileNamed: '..'.
   => MultiByteFileStream: '/home/qwaq/server/bin/forums/..' "????"

Is this expected behavior? If not, any ideas how to fix it?

Thanks,
   - Andreas

On Thu, Aug 07, 2008 at 12:17:50AM -0700, Andreas Raab wrote:
Strictly speaking, opening a directory as a file stream is not wrong.
Whether it's expected or desirable is a matter of opinion. For a unix
filesystem, a directory is a file, and you can for example open a directory
in a text editor, or view its contents with /bin/cat or /usr/bin/od.
Similarly, in Squeak it would make sense to be able to open a directory
file as a file stream, and then be able to read its contents.

Certainly it is inconsistent to report that the file directory
does not exist, but then allow you to open it as a file stream.
No, it's not expected behavior (at least not to me).

Also, this seems inconsistent:

(FileDirectory default readOnlyFileNamed: '.') size
 => 24576 "as expected"

(FileDirectory default readOnlyFileNamed: '.') next
  => nil "??? expected to read something"

(FileDirectory default readOnlyFileNamed: '.') contentsOfEntireFile
  => '' "??? expected a string"

My personal opinion is that allowing a directory file to be opened
as a file stream is good, and failing to allow the contents of the
stream to be read is bad.

Dave

David T. Lewis wrote:
> Strictly speaking, opening a directory as a file stream is not wrong.

Interesting. I was under the impression that that changed, quite some
time ago now. I recall using cat on directories in Minix 1.5, and
getting halfway-useful (if file-system specific) results, but modern
Linux and OS X machines don't seem to let you read anything out of
directories without using the special directory-traversal functions.

> Similarly, in Squeak it would make sense to be able to open a directory
> file as a file stream, and then be able to read its contents.

What would you expect read(2) on a directory to return? Is it standardised?

What would reading from a Smalltalk file stream on a directory return?
Bytes? Strings? Other file streams? Some other kind of object?

> My personal opinion is that allowing a directory file to be opened
> as a file stream is good, and failing to allow the contents of the
> stream to be read is bad.

I'd be happier treating directories as yet another kind of specialised
collection, rather than a stream. They can be encoded into a stream,
sure, but the details of the encoding will vary platform by platform,
and file-system-type by file-system-type.

Tony

Tony Garnock-Jones wrote:
> David T. Lewis wrote:
>> Strictly speaking, opening a directory as a file stream is not wrong.
>
> Interesting. I was under the impression that that changed, quite some
> time ago now. I recall using cat on directories in Minix 1.5, and
> getting halfway-useful (if file-system specific) results, but modern
> Linux and OS X machines don't seem to let you read anything out of
> directories without using the special directory-traversal functions.

They don't allow reading, but they do allow opening for reading.  The
resulting descriptor is meant to be used with openat() and other similar
system calls suffixed "at". openat() allows an application to avoid race
conditions that could occur when using open(2) to open files in
directories other than the current working directory. These race
conditions result from the fact that some component of the directory
prefix given to open() could be changed in parallel with the call to
open(). Such races can be avoided by opening a file descriptor for the
target directory, and then specifying that file descriptor as the dirfd
argument of openat().

Paolo

Paolo Bonzini wrote:

Wow. I mean... wow. Well, I suppose fds are Unix's equivalent to object
pointers, so it kind of makes sense. Still... wow.

(Looking at the openat manpage, I note it's linux-specific, and that a
similar operation exists on Solaris.)

Tony

>>>>> "Tony" == Tony Garnock-Jones <[hidden email]> writes:

Tony> What would you expect read(2) on a directory to return? Is it
Tony> standardised?

In the early days, read(2) on a fd opened on a directory returned
18-byte chunks... 2 bytes for inode, 16 bytes for NUL-padded filename.

When directories got more complicated, any differences were hidden between
readdir(3) routines, but ultimately, those still called read(2) on the
directory, just as you could directly.  Those routines are still documented in
my modern OpenBSD and OSX releases, and are the routines that Perl uses
to implement Perl's readdir() and glob() routines.

Reading further in the thread, it's not surprising that Linux reinvented the
wheel to a completely confusing result for those of us with an actual Unix
background.  Linux ain't Unix.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

Randal L. Schwartz wrote:
> In the early days, read(2) on a fd opened on a directory returned
> 18-byte chunks... 2 bytes for inode, 16 bytes for NUL-padded filename.

Right. That's the kind of thing I remember MINIX doing.

> Those routines are still documented in
> my modern OpenBSD and OSX releases, and are the routines that Perl uses
> to implement Perl's readdir() and glob() routines.

Looking at the manpage for read(2) on OS X 10.5, it reckons it'll
complain EISDIR if an attempt is made to read a directory. (Note: I
haven't actually tried it out! :-) perhaps I should)

There is, however, getdirentries(2), which I found via dir(5), and which
returns struct dirents.

This reinforces my prejudice that interrogating directories is something
that ought to be done via some other protocol than the byte- or
string-oriented stream protocols.

>>>>> "Tony" == Tony Garnock-Jones <[hidden email]> writes:

Tony> This reinforces my prejudice that interrogating directories is something
Tony> that ought to be done via some other protocol than the byte- or
Tony> string-oriented stream protocols.

Yes, as an abstraction, I think opening a readstream on a directory might be
worthy of refusing.  Even if it would work on Unix-like things (for some
meaning of "work"), it's probably gonna be really nasty in other environments,
and clearly not portable.

--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<[hidden email]> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion

> Reading further in the thread, it's not surprising that Linux reinvented the
> wheel to a completely confusing result for those of us with an actual Unix
> background.  Linux ain't Unix.

Interesting point of view, since those system calls were actually
introduced in Solaris and then ported to Linux.

In fact I think that things like this one, or like Linux's signalfd are
much more Unix-y than things like SYSV's STREAMS.

Paolo

On Thu, Aug 07, 2008 at 01:29:48PM +0100, Tony Garnock-Jones wrote:
> David T. Lewis wrote:
>
> >Similarly, in Squeak it would make sense to be able to open a directory
> >file as a file stream, and then be able to read its contents.
>
> What would you expect read(2) on a directory to return? Is it standardised?

Well, I had *expected* read() to read byes from the directory file, but as
Paolo explained, my expectation is not correct for newer unix-like systems.

So I retract my comment that it would make sense in Squeak to be able read
the directory file. Squeak is just doing what the underlying platform lets
it do, and there is nothing wrong with that.

Dave

David T. Lewis wrote:
> So I retract my comment that it would make sense in Squeak to be able read
> the directory file. Squeak is just doing what the underlying platform lets
> it do, and there is nothing wrong with that.

Yes, there is. Squeak supposedly provides common abstractions across
platforms and this difference in behavior between Windows and Unix/Mac
is clearly violating the common abstraction.

If I want to indulge in platform-specific quirks I use Python ;-)

Cheers,
   - Andreas