Smalltalk › Pharo › Pharo Smalltalk Users

FileReference#entries caching?

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

7 messages Options

Steven R. Baker

FileReference#entries caching?

Heya folks,

I'm loading some PNG icons from disk, and I'm noticing some weirdness
with FileReference#entries.

When I inspect: '/home/srbaker/Projects/FocusFrog/icons' asFileReference
entries it gives me an Array of size one, containing the refernce to the
only file there. Then I delete the file, the same thing gives an array
of size zero.

But then when I fill the directory with a bunch of icons (240, to be
exact), and inspect that line again I get zero. Eventually it gets the
right answer.

Is it cached or something? How do I guarantee that I'm *definitely*
getting the current state of the directory on disk? Should I not even be
using FileReference for this?

Thanks!

-Steven

Peter Uhnak

Re: FileReference#entries caching?

On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker <[hidden email]> wrote:

Is it cached or something? How do I guarantee that I'm *definitely*
getting the current state of the directory on disk?

This looks extremely strange, I've never seen that inspecting files/directories would be somehow "out of sync" with the disk, because it is always asking the disk.

Should I not even be using FileReference for this?

If you are loading png icons into Pharo, you can consider using https://github.com/peteruhnak/IconFactory/ instead (shameless self promotion :) )

Peter

Steven R. Baker

Re: FileReference#entries caching?

On 07/10/17 14:22, Peter Uhnák wrote:

On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker <[hidden email]> wrote:

Is it cached or something? How do I guarantee that I'm *definitely*
getting the current state of the directory on disk?

This looks extremely strange, I've never seen that inspecting files/directories would be somehow "out of sync" with the disk, because it is always asking the disk.

Should I not even be using FileReference for this?

If you are loading png icons into Pharo, you can consider using https://github.com/peteruhnak/IconFactory/ instead (shameless self promotion :) )

Will definitely switch to this; I wasn't aware of the pattern, but obviously I want to follow them where possible. :)

Peter

Guillermo Polito

Re: FileReference#entries caching?

Hi,

I don't think that FileSystem is caching entries... However, we found somehow related bug the other day with Mariano and Pablo while running OSSupprocess tests on travis/linux. The bug is as follows. The test was doing:

'/tmp' asFileReference entries do: [...]

And then it was calling 'lsof' over pharo's pid to know files that pharo has open.

funny thing is that lsof was telling us that '/tmp' was open! When we already retrieved the entries and the file reference was collected...

The same in osx or local linux mate running in a virtual box did not happen :/.

On Sat, Oct 7, 2017 at 5:44 PM, Steven R. Baker <[hidden email]> wrote:

On 07/10/17 14:22, Peter Uhnák wrote:

On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker <[hidden email]> wrote:

Is it cached or something? How do I guarantee that I'm *definitely*
getting the current state of the directory on disk?

This looks extremely strange, I've never seen that inspecting files/directories would be somehow "out of sync" with the disk, because it is always asking the disk.

Should I not even be using FileReference for this?

If you are loading png icons into Pharo, you can consider using https://github.com/peteruhnak/IconFactory/ instead (shameless self promotion :) )

Will definitely switch to this; I wasn't aware of the pattern, but obviously I want to follow them where possible. :)

Peter

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

alistairgrant

Re: FileReference#entries caching?

On Sun, Oct 08, 2017 at 06:33:08PM +0200, Guillermo Polito wrote:
> Hi,
>
> I don't think that FileSystem is caching entries...

I haven't seen any code in the Pharo image or FilePlugin that caches
data.

> However, we found somehow
> related bug the other day with Mariano and Pablo while running OSSupprocess
> tests on travis/linux. The bug is as follows. The test was doing:
>
> '/tmp' asFileReference entries do: [...]
>
> And then it was calling 'lsof' over pharo's pid to know files that pharo has
> open.
>
> funny thing is that lsof was telling us that '/tmp' was open! When we already
> retrieved the entries and the file reference was collected...
>
> The same in osx or local linux mate running in a virtual box did not happen :/.

I was able to reproduce the issue on Ubuntu 16.04 using
Pharo7.0-32bit-d13eed2.image.

I haven't tracked the cause down, but the problem doesn't exist with the
proposed fix I have for symbolic links (fogbugz 18279
https://pharo.fogbugz.com/f/cases/18279/isSymlink-seems-to-be-broken-on-Linux).
This patch also updates the code responsible for iterating over
directories, which is why it can resolve this issue as well.

Cheers,
Alistair

> On Sat, Oct 7, 2017 at 5:44 PM, Steven R. Baker <[hidden email]>
> wrote:
>
>
>
>
> On 07/10/17 14:22, Peter Uhn?k wrote:
>
> On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker <
> [hidden email]> wrote:
>
> Is it cached or something? How do I guarantee that I'm *definitely*
> getting the current state of the directory on disk?
>
>
> This looks extremely strange, I've never seen that inspecting files/
> directories would be somehow "out of sync" with the disk, because it is
> always asking the disk.
>
>
>
>
> Should I not even be using FileReference for this?
>
>
> If you are loading png icons into Pharo, you can consider using https:/
> /github.com/peteruhnak/IconFactory/ instead (shameless self promotion
> :) )
>
>
> Will definitely switch to this; I wasn't aware of the pattern, but
> obviously I want to follow them where possible. :)
>
>
> Peter
>
>
>
>
>
>
> --
> [presentation] Guille Polito
> [CNRS-filaire]
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et
> Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research -
> http://www.cnrs.fr
>
>
> Web: http://guillep.github.io
>
> Phone: +33 06 52 70 66 13
>

Guillermo Polito

Re: FileReference#entries caching?

On Sun, Oct 8, 2017 at 7:01 PM, Alistair Grant <[hidden email]> wrote:

On Sun, Oct 08, 2017 at 06:33:08PM +0200, Guillermo Polito wrote:
> Hi,
>
> I don't think that FileSystem is caching entries...

I haven't seen any code in the Pharo image or FilePlugin that caches
data.

> However, we found somehow
> related bug the other day with Mariano and Pablo while running OSSupprocess
> tests on travis/linux. The bug is as follows. The test was doing:
>
> '/tmp' asFileReference entries do: [...]
>
> And then it was calling 'lsof' over pharo's pid to know files that pharo has
> open.
>
> funny thing is that lsof was telling us that '/tmp' was open! When we already
> retrieved the entries and the file reference was collected...
>
> The same in osx or local linux mate running in a virtual box did not happen :/.

I was able to reproduce the issue on Ubuntu 16.04 using
Pharo7.0-32bit-d13eed2.image.

I haven't tracked the cause down, but the problem doesn't exist with the
proposed fix I have for symbolic links (fogbugz 18279
https://pharo.fogbugz.com/f/cases/18279/isSymlink-seems-to-be-broken-on-Linux).
This patch also updates the code responsible for iterating over
directories, which is why it can resolve this issue as well.

Nice! How much time until we can integrate such thing ? :D

Cheers,
Alistair

> On Sat, Oct 7, 2017 at 5:44 PM, Steven R. Baker <[hidden email]>
> wrote:
>
>
>
>
> On 07/10/17 14:22, Peter Uhn?k wrote:
>
> On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker <
> [hidden email]> wrote:
>
> Is it cached or something? How do I guarantee that I'm *definitely*
> getting the current state of the directory on disk?
>
>
> This looks extremely strange, I've never seen that inspecting files/
> directories would be somehow "out of sync" with the disk, because it is
> always asking the disk.
>
>
>
>
> Should I not even be using FileReference for this?
>
>
> If you are loading png icons into Pharo, you can consider using https:/
> /github.com/peteruhnak/IconFactory/ instead (shameless self promotion
> :) )
>
>
> Will definitely switch to this; I wasn't aware of the pattern, but
> obviously I want to follow them where possible. :)
>
>
> Peter
>
>
>
>
>
>
> --
> [presentation] Guille Polito
> [CNRS-filaire]

> Research Engineer
>
> Centre de Recherche en Informatique, Signal et
> Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research -
> http://www.cnrs.fr
>
>
> Web: http://guillep.github.io
>
> Phone: <a href="tel:%2B33%2006%2052%2070%2066%2013" value="+33652706613">+33 06 52 70 66 13
>

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

alistairgrant

Re: FileReference#entries caching?

On Sun, Oct 08, 2017 at 07:08:41PM +0200, Guillermo Polito wrote:

>
>
> On Sun, Oct 8, 2017 at 7:01 PM, Alistair Grant <[hidden email]> wrote:
>
> On Sun, Oct 08, 2017 at 06:33:08PM +0200, Guillermo Polito wrote:
> > Hi,
> >
> > I don't think that FileSystem is caching entries...
>
> I haven't seen any code in the Pharo image or FilePlugin that caches
> data.
>
>
> > However, we found somehow
> > related bug the other day with Mariano and Pablo while running
> OSSupprocess
> > tests on travis/linux. The bug is as follows. The test was doing:
> >
> > '/tmp' asFileReference entries do: [...]
> >
> > And then it was calling 'lsof' over pharo's pid to know files that pharo
> has
> > open.
> >
> > funny thing is that lsof was telling us that '/tmp' was open! When we
> already
> > retrieved the entries and the file reference was collected...
> >
> > The same in osx or local linux mate running in a virtual box did not
> happen :/.
>
> I was able to reproduce the issue on Ubuntu 16.04 using
> Pharo7.0-32bit-d13eed2.image.
>
> I haven't tracked the cause down, but the problem doesn't exist with the
> proposed fix I have for symbolic links (fogbugz 18279
> https://pharo.fogbugz.com/f/cases/18279/isSymlink-seems-
> to-be-broken-on-Linux).
> This patch also updates the code responsible for iterating over
> directories, which is why it can resolve this issue as well.
>
>
>
> Nice! How much time until we can integrate such thing ? :D

I submitted the code in August. Eliot kindly provided some great
feedback with lots of details. I submitted updated code on 6 September.
Since then I haven't seen anything from Eliot, so I hope everything is
OK.

Cheers,
Alistair

> Cheers,
> Alistair
>
>
> > On Sat, Oct 7, 2017 at 5:44 PM, Steven R. Baker <[hidden email]>
> > wrote:
> >
> >
> >
> >
> > On 07/10/17 14:22, Peter Uhn?k wrote:
> >
> > On Sat, Oct 7, 2017 at 1:02 PM, Steven R. Baker <
> > [hidden email]> wrote:
> >
> > Is it cached or something? How do I guarantee that I'm
> *definitely*
> > getting the current state of the directory on disk?
> >
> >
> > This looks extremely strange, I've never seen that inspecting
> files/
> > directories would be somehow "out of sync" with the disk, because
> it is
> > always asking the disk.
> >
> >
> >
> >
> > Should I not even be using FileReference for this?
> >
> >
> > If you are loading png icons into Pharo, you can consider using
> https:/
> > /github.com/peteruhnak/IconFactory/ instead (shameless self
> promotion
> > :) )
> >
> >
> > Will definitely switch to this; I wasn't aware of the pattern, but
> > obviously I want to follow them where possible. :)
> >
> >
> > Peter
> >
> >
> >
> >
> >
> >
> > --
> > [presentation] Guille Polito
> > [CNRS-filaire]
> > Research Engineer
> >
> > Centre de Recherche en Informatique, Signal et
> > Automatique de Lille
> >
> > CRIStAL - UMR 9189
> >
> > French National Center for Scientific Research
> -
> > http://www.cnrs.fr
> >
> >
> > Web: http://guillep.github.io
> >
> > Phone: +33 06 52 70 66 13
> >
>
>
>
>
>
> --
> [presentation] Guille Polito
> [CNRS-filaire]
> Research Engineer
>
> Centre de Recherche en Informatique, Signal et
> Automatique de Lille
>
> CRIStAL - UMR 9189
>
> French National Center for Scientific Research -
> http://www.cnrs.fr
>
>
> Web: http://guillep.github.io
>
> Phone: +33 06 52 70 66 13
>