DirectoryPlugin updated on SqueakSource

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

DirectoryPlugin updated on SqueakSource

David T. Lewis
A few years ago (well, more than a few), I implemented DirectoryPlugin.
This claimed to (and I believe actually did) provide very large performance
benefits on images of that time. It also required some intrusive changes
on the image side, so it really was not appropriate for integration into
the image.

I recently updated the primitive and provided a separate image side access
class, This is now on SqueakSource.com as package DirectoryPlugin.

Since this was originally written circa 2003, the Squeak image has apparently
improved immensely. The directory stream primitives, which were the original
motivation for writing it, no longer provide any benefit at all. In fact,
they seem to be slightly slower than the standard (modern) image).

The file stat primitives do provide substantial performance benefit, so
these may still be of interest for modern images.

Here is the description from SqueakSource:

-------
Project Description

Home page http://wiki.squeak.org/squeak/2274

DirectoryPlugin provides access to a Posix directory stream for efficient
access to directory contents. It also provides methods for testing file
status and how to creating directory entries for a files and directories.

The directory stream primitives use the address of a directory stream as
a handle to identify a directory stream. This permits several directory
streams to be opened and manipulated independently by separate Smalltalk
processes. However, care must be taken to avoid passing the handle for an
invalid directory stream (such as one which has already been closed) to
one of these methods.

The plugin has been tested on Linux and on Windows. The directory stream
support produces a performance improvement only on Unix (Linux) systems.
The file access primitives provide substantial improvements for both Unix
and Windows systems. Other platforms have not been tested.

DirectoryPluginAccess provides access to primitives in DirectoryPlugin,
along with utility methods for various directory and file access functions.

Performance characteristics have changed significantly since Squeak circa
2003. The readdir() primitives no longer provide any benefit, but the file
testing primitives still yield a couple orders of magnitude improvement
for some functions.


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] DirectoryPlugin updated on SqueakSource

Eliot Miranda-2
Hi David,

On Sat, Dec 12, 2015 at 2:58 PM, David T. Lewis <[hidden email]> wrote:

A few years ago (well, more than a few), I implemented DirectoryPlugin.
This claimed to (and I believe actually did) provide very large performance
benefits on images of that time. It also required some intrusive changes
on the image side, so it really was not appropriate for integration into
the image.

I recently updated the primitive and provided a separate image side access
class, This is now on SqueakSource.com as package DirectoryPlugin.

Since this was originally written circa 2003, the Squeak image has apparently
improved immensely. The directory stream primitives, which were the original
motivation for writing it, no longer provide any benefit at all. In fact,
they seem to be slightly slower than the standard (modern) image).

The file stat primitives do provide substantial performance benefit, so
these may still be of interest for modern images.

Then what's involved in integrating only the stat primitives?  Both in the plugin and the image?
 

Here is the description from SqueakSource:

-------
Project Description

Home page http://wiki.squeak.org/squeak/2274

DirectoryPlugin provides access to a Posix directory stream for efficient
access to directory contents. It also provides methods for testing file
status and how to creating directory entries for a files and directories.

The directory stream primitives use the address of a directory stream as
a handle to identify a directory stream. This permits several directory
streams to be opened and manipulated independently by separate Smalltalk
processes. However, care must be taken to avoid passing the handle for an
invalid directory stream (such as one which has already been closed) to
one of these methods.

The plugin has been tested on Linux and on Windows. The directory stream
support produces a performance improvement only on Unix (Linux) systems.
The file access primitives provide substantial improvements for both Unix
and Windows systems. Other platforms have not been tested.

DirectoryPluginAccess provides access to primitives in DirectoryPlugin,
along with utility methods for various directory and file access functions.

Performance characteristics have changed significantly since Squeak circa
2003. The readdir() primitives no longer provide any benefit, but the file
testing primitives still yield a couple orders of magnitude improvement
for some functions.




--
_,,,^..^,,,_
best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: [Vm-dev] DirectoryPlugin updated on SqueakSource

David T. Lewis
On Sun, Dec 13, 2015 at 05:30:52PM -0800, Eliot Miranda wrote:

>  
> Hi David,
>
> On Sat, Dec 12, 2015 at 2:58 PM, David T. Lewis <[hidden email]> wrote:
>
> >
> > A few years ago (well, more than a few), I implemented DirectoryPlugin.
> > This claimed to (and I believe actually did) provide very large performance
> > benefits on images of that time. It also required some intrusive changes
> > on the image side, so it really was not appropriate for integration into
> > the image.
> >
> > I recently updated the primitive and provided a separate image side access
> > class, This is now on SqueakSource.com as package DirectoryPlugin.
> >
> > Since this was originally written circa 2003, the Squeak image has
> > apparently
> > improved immensely. The directory stream primitives, which were the
> > original
> > motivation for writing it, no longer provide any benefit at all. In fact,
> > they seem to be slightly slower than the standard (modern) image).
> >
> > The file stat primitives do provide substantial performance benefit, so
> > these may still be of interest for modern images.
> >
>
> Then what's involved in integrating only the stat primitives?  Both in the
> plugin and the image?
>

For the plugin, we could just remove the four directory stream primitives,
leave the remaining ten file stat primitives, and rename the module to
something like 'FileStatPlugin'.

For the image, the ten example methods in DirectoryPluginAccess class in
category 'file primitive access' illustrate how to access the primitives.

The tricky part is that these are unix/posix only, so they would have
to be optional primitives and integrated into the image in such a way
that things still work reasonably on other platforms. I have not really
looked into that aspect of the problem since 2003, but it might be worth
a second look now.

Dave