Primitives 1600 and 1606 return out-of-date information

[VisualWorks 7.7.1 on Windows 7 x64 SP1]

I’m running into an issue where use of FindFirstFile() in Filename>>getDates and Filename>>fileSize appears to give me information that is significantly out-of-date (on Windows, anyway) and this is the likely  culprit,

http://msdn.microsoft.com/en-us/library/aa364418(v=vs.85).aspx

Note: In rare cases, file information on NTFS file systems may not be current at the time you call this function. To be assured of getting the current file information, call the GetFileInformationByHandle function.

Here’s a workspace to demonstrate the effect (only needs Regex in the clean image) using the file size example,

Transcript clear.

fn := 'test.txt' asFilename.

ws := fn writeStream.

external :=

                                [out := ExternalProcess fork: 'cmd'

                                                                                arguments: (Array

                                                                                                                with: '/C'

                                                                                                                with: 'dir'

                                                                                                                with: '/-C'

                                                                                                                with: fn asAbsoluteFilename asString).

                                rx := '.*File\(s\) *(\d*) bytes.*' asRegex.

                                rx matches: out.

                                rx subexpression: 2].

[1000 timesRepeat: [ws nextPut: $a].

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value;

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'Flushing...'.

ws flush.

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value;

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'Closing...']

                                ensure: [ws close].

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value

Output, with the key section highlighted,

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 0

External: 1000

In Smalltalk: 1000 (you can see here that Smalltalk implementation will pick up the correct size after another process had made use of a proper call, via either a ‘dir’ command or explorer refresh in the destination directory)

Closing...

In Smalltalk: 1000

External: 1000

Any chance this could be addressed at some point to make use of a more predictable Windows API?

Thanks,

-Boris

Here’s an implementation using existing facilities within VisualWorks that gives up-to-date information on the file size,

Filename>>rtFileSize

| tmp |

tmp := FileConnection openFileNamed: self mode: #readOnly creationRule: #noCreate.

^[tmp dataSize] ensure: [tmp close].

“Using #fileSize”

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 0

External: 1000

In Smalltalk: 1000

Closing...

In Smalltalk: 1000

External: 1000

“Using #rtFileSize”

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 1000

External: 1000

In Smalltalk: 1000

Closing...

In Smalltalk: 1000

External: 1000

Regards,

-Boris

[VisualWorks 7.7.1 on Windows 7 x64 SP1]

I’m running into an issue where use of FindFirstFile() in Filename>>getDates and Filename>>fileSize appears to give me information that is significantly out-of-date (on Windows, anyway) and this is the likely  culprit,

http://msdn.microsoft.com/en-us/library/aa364418(v=vs.85).aspx

Note: In rare cases, file information on NTFS file systems may not be current at the time you call this function. To be assured of getting the current file information, call the GetFileInformationByHandle function.

Here’s a workspace to demonstrate the effect (only needs Regex in the clean image) using the file size example,

Transcript clear.

fn := 'test.txt' asFilename.

ws := fn writeStream.

external :=

                                [out := ExternalProcess fork: 'cmd'

                                                                                arguments: (Array

                                                                                                                with: '/C'

                                                                                                                with: 'dir'

                                                                                                                with: '/-C'

                                                                                                                with: fn asAbsoluteFilename asString).

                                rx := '.*File\(s\) *(\d*) bytes.*' asRegex.

                                rx matches: out.

                                rx subexpression: 2].

[1000 timesRepeat: [ws nextPut: $a].

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value;

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'Flushing...'.

ws flush.

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value;

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'Closing...']

                                ensure: [ws close].

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value

Output, with the key section highlighted,

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 0

External: 1000

In Smalltalk: 1000 (you can see here that Smalltalk implementation will pick up the correct size after another process had made use of a proper call, via either a ‘dir’ command or explorer refresh in the destination directory)

Closing...

In Smalltalk: 1000

External: 1000

Any chance this could be addressed at some point to make use of a more predictable Windows API?

Thanks,

-Boris

Or,

Filename>>rtFileSize

                | tmp |

                tmp := FileConnection openFileNamed: self mode: #readOnly creationRule: #noCreate.

                ^[tmp input fileSize] ensure: [tmp close].

-Boris

Here’s an implementation using existing facilities within VisualWorks that gives up-to-date information on the file size,

Filename>>rtFileSize

| tmp |

tmp := FileConnection openFileNamed: self mode: #readOnly creationRule: #noCreate.

^[tmp dataSize] ensure: [tmp close].

“Using #fileSize”

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 0

External: 1000

In Smalltalk: 1000

Closing...

In Smalltalk: 1000

External: 1000

“Using #rtFileSize”

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 1000

External: 1000

In Smalltalk: 1000

Closing...

In Smalltalk: 1000

External: 1000

Regards,

-Boris

[VisualWorks 7.7.1 on Windows 7 x64 SP1]

I’m running into an issue where use of FindFirstFile() in Filename>>getDates and Filename>>fileSize appears to give me information that is significantly out-of-date (on Windows, anyway) and this is the likely  culprit,

http://msdn.microsoft.com/en-us/library/aa364418(v=vs.85).aspx

Note: In rare cases, file information on NTFS file systems may not be current at the time you call this function. To be assured of getting the current file information, call the GetFileInformationByHandle function.

Here’s a workspace to demonstrate the effect (only needs Regex in the clean image) using the file size example,

Transcript clear.

fn := 'test.txt' asFilename.

ws := fn writeStream.

external :=

                                [out := ExternalProcess fork: 'cmd'

                                                                                arguments: (Array

                                                                                                                with: '/C'

                                                                                                                with: 'dir'

                                                                                                                with: '/-C'

                                                                                                                with: fn asAbsoluteFilename asString).

                                rx := '.*File\(s\) *(\d*) bytes.*' asRegex.

                                rx matches: out.

                                rx subexpression: 2].

[1000 timesRepeat: [ws nextPut: $a].

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value;

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'Flushing...'.

ws flush.

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value;

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'Closing...']

                                ensure: [ws close].

Transcript

                cr;

                show: 'In Smalltalk: ' , fn fileSize printString;

                cr;

                show: 'External: ' , external value

Output, with the key section highlighted,

In Smalltalk: 0

External: 0

In Smalltalk: 0

Flushing...

In Smalltalk: 0

External: 1000

In Smalltalk: 1000 (you can see here that Smalltalk implementation will pick up the correct size after another process had made use of a proper call, via either a ‘dir’ command or explorer refresh in the destination directory)

Closing...

In Smalltalk: 1000

External: 1000

Any chance this could be addressed at some point to make use of a more predictable Windows API?

Thanks,

-Boris

Andres,

To boil this down a bit after a lengthy discussion, my main issue is that things like '(gi .\sample.iso).length' or Windows Explorer return up-to-date information about the file size and VisualWorks' implementation of #fileSize does not. The key point that I am making is that I would much rather get up-to-date information and be subject to normal file system behaviour as far as other processes’ impact on the file is concerned (i.e. exceptions on access) than get information back that could be _minutes_ old. Here's why I think it should be possible to fix it (whether it’s done in a VM or in Smalltalk, I don’t really care),

http://msdn.microsoft.com/en-us/library/aa363858(v=vs.85).aspx

The dwDesiredAccess parameter can be zero, allowing the application to query file attributes without accessing the file if the application is running with adequate security settings. This is useful to test for the existence of a file without opening it for read and/or write access, or to obtain other statistics about the file or directory. See Obtaining and Setting File Information and GetFileInformationByHandle.

http://msdn.microsoft.com/en-us/library/aa365257(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/aa364957(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/aa364952(v=vs.85).aspx

If you want to get up-to-date information, you'd better use GetFileInformationByHandle.

From FindFileFirst MSDN doc: "In rare cases, file attribute information on NTFS file systems may not be current at the time you call this function. To be assured of getting the current NTFS file system file attributes, call the GetFileInformationByHandle function."

I dare to say that on heavily loaded system running software heavily optimized for performance (i.e. which doesn't call FlushFileBuffers on the left and on the right) these cases are not rare at all, and it is relatively easy to reproduce.

The thing is that FindFirstFile() retrieves information from directory entry which is updated lazily. Maximum latency for directory entry update (if memory serves me well) -- in absense of FlushFileBuffers() calls on modified handle -- is 5 minutes; that's the best we can hope for.

-Boris

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud
Sent: 11 May 2011 16:16
To: [hidden email]
Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date information

From the POV of the VM, I'm concerned that adding anything other than reading what the directory says is bound to cause trouble.  For example, let's say you have the file foo.txt.  You want to know how large it is.

  But another process already opened foo.txt, called DeleteFile() on it, and has yet to close the file.  Then you cannot call CreateFile() because it fails with ERROR_ACCESS_DENIED.  This is bad because you cannot predict what another process is doing.

Or what if you're not supposed to open the file that was just deleted by the another app because the other app will recreate the file an instant later?  If the app assumes that the file is gone after DeleteFile() and CloseFile(), but the file is still there because the VM opened the file to determine its size before DeleteFile() was called by the app but the VM has yet to call CloseFile() so the file is still there, the other app might fail unexpectedly.

http://msdn.microsoft.com/en-us/library/aa363915%28VS.85%29.aspx

The same unpredictable problems should be expected when doing similar operations from the image.  Maybe there's another way to figure out what you need?

On 5/11/2011 12:55 PM, Boris Popov, DeepCove Labs wrote:

> If you hadn't requested a share mode that prevents others from doing

> certain things, the other application will still be able to do

> whatever the heck it is able to do normally, no? I'm not an expert on

> this, far from it, but as an example, doing #renameTo: in VisualWorks

> to a different path while Apache still has a handle on the file and is

> in the process of receiving data to write to it actually succeeds and

> Apache just keeps on writing to the new location. That's in the

> primitive 1603,

> 

> http://dl.dropbox.com/u/2737880/videos/rename%20a/index.html

> 

> Other fun things I can do include,

> 

> from delete. "in which case Apache continues writing to the file, but

> file disappears immediately after it's done with it"

> 

> or,

> 

> from renameTo: to.

> to delete. "in which case Apache continues writing to the new

> location, but file disappears immediately after it's done with it"

> 

> If anything, at least these methods should include a note that the

> information they return might not be up-to-date in certain

> circumstances, just like MSDN does when talking about FindFirstFile()

> and, ideally, include alternatives that do with their own sets of

> caveats.

> 

> -Boris

> 

> -----Original Message-----

> From: [hidden email] [mailto:[hidden email]] On

> Behalf Of Andres Valloud

> Sent: 11 May 2011 15:30

> To: [hidden email]

> Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date

> information

> 

> Ok, but note that it's more difficult to put that kind of stuff in the

> VM because the test procedure might fail in unexpected ways (e.g.:

> what happens if another process tries to delete the file while the VM

> has the file open?).

> 

> On 5/11/2011 9:14 AM, Boris Popov, DeepCove Labs wrote:

>> Andres,

>> 

>> I ended up using a Smalltalk implementation to get the result I was

>> looking for,

>> 

>> Filename>>rtFileSize

>> 

>>                   | tmp |

>>                   tmp := FileConnection openFileNamed: self mode:

>> #readOnly creationRule: #noCreate.

>>                   ^[tmp input fileSize] ensure: [tmp close].

>> 

>> Below you can see the difference in behaviour (sorry the videos are

>> quite wide), key is the difference in file size reporting in the

>> transcript as file is uploaded via cURL to a local Apache install,

>> 

>> http://dl.dropbox.com/u/2737880/videos/filesize%20a/index.html

>> http://dl.dropbox.com/u/2737880/videos/filesize%20b/index.html

>> 

>> It should be noted that Apache seems to be somewhat lax about the

>> share mode on the file that it opens for writing,

>> 

>> "12:06:20.6941918","httpd.exe","1320","CreateFile","C:\DeepCove\webda

>> v \f olders\ernest\payments\sample.iso","SUCCESS","Desired Access:

>> Generic Write, Read Attributes, Disposition: OverwriteIf, Options:

>> Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a,

>> ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult:

>> Created"

>> 

>> Full process monitor action on that file is available here,

>> 

>> http://dl.dropbox.com/u/2737880/videos/apache.zip

>> 

>> -Boris

>> 

>> -----Original Message-----

>> From: [hidden email] [mailto:[hidden email]] On

>> Behalf Of Andres Valloud

>> Sent: 11 May 2011 12:07

>> To: [hidden email]

>> Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date

>> information

>> 

>> Looking at the docs, it seems that using GetFileInformationByHandle

>> would require first to get a handle to the file.  However, in

>> CreateFile, we can see using share mode zero "[p]revents other

>> processes from opening a file or device if they request delete, read,

>> or write access".  Thus, it may not be possible to get a handle in

>> the

> 

>> first place.  In those cases, it appears we won't be able to get

> "current"

>> file information.  But how to distinguish between the two from a

>> Smalltalk application?...

>> 

>> On 5/11/2011 8:50 AM, Boris Popov, DeepCove Labs wrote:

>>> Andres,

>>> 

>>> The workspace is an approximation of the behaviour I see in

>>> production

>> 

>>> where our app monitors files that are modified by other applications.

>>> The effect is the same, however, using #fileSize returns values that

>>> are off compared to the results one gets from a simple dir, at which

>>> point #fileSize once again returns correct information, but only as

>>> of

>> 

>>> the last dir even as the file continues growing. I'll see if I can

>>> get

>> 

>>> a process monitor log of the events for the file as it keeps on

>>> growing to show you what I mean.

>>> 

>>> -Boris

>>> 

>>> -----Original Message-----

>>> From: [hidden email] [mailto:[hidden email]] On

>>> Behalf Of Andres Valloud

>>> Sent: 11 May 2011 11:44

>>> To: [hidden email]

>>> Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date

>>> information

>>> 

>>> Have you tried using #commit instead of #flush?

>>> 

>>> In Smalltalk: 0

>>> External: 0

>>> In Smalltalk: 0

>>> Committing...

>>> In Smalltalk: 1000

>>> External: 1000

>>> In Smalltalk: 1000

>>> Closing...

>>> In Smalltalk: 1000

>>> External: 1000

>>> 

>>> On 5/6/2011 11:02 AM, Boris Popov, DeepCove Labs wrote:

>>>> [VisualWorks 7.7.1 on Windows 7 x64 SP1]

>>>> 

>>>> I'm running into an issue where use of FindFirstFile() in

>>>> Filename>>getDates and Filename>>fileSize appears to give me

>>>> Filename>>information

>>>> that is significantly out-of-date (on Windows, anyway) and this is

>>>> the

>>> 

>>>> likely culprit,

>>>> 

>>>> http://msdn.microsoft.com/en-us/library/aa364418(v=vs.85).aspx

>>>> 

>>>> Note: In rare cases, file information on NTFS file systems may not

>>>> be

>> 

>>>> current at the time you call this function. To be assured of

>>>> getting

> 

>>>> the current file information, call the GetFileInformationByHandle

>>> function.

>>>> 

>>>> Here's a workspace to demonstrate the effect (only needs Regex in

>>>> the

>> 

>>>> clean image) using the file size example,

>>>> 

>>>> Transcript clear.

>>>> 

>>>> fn := 'test.txt' asFilename.

>>>> 

>>>> ws := fn writeStream.

>>>> 

>>>> external :=

>>>> 

>>>> [out := ExternalProcess fork: 'cmd'

>>>> 

>>>> arguments: (Array

>>>> 

>>>> with: '/C'

>>>> 

>>>> with: 'dir'

>>>> 

>>>> with: '/-C'

>>>> 

>>>> with: fn asAbsoluteFilename asString).

>>>> 

>>>> rx := '.*File\(s\) *(\d*) bytes.*' asRegex.

>>>> 

>>>> rx matches: out.

>>>> 

>>>> rx subexpression: 2].

>>>> 

>>>> [1000 timesRepeat: [ws nextPut: $a].

>>>> 

>>>> Transcript

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'External: ' , external value;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'Flushing...'.

>>>> 

>>>> ws flush.

>>>> 

>>>> Transcript

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'External: ' , external value;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'Closing...']

>>>> 

>>>> ensure: [ws close].

>>>> 

>>>> Transcript

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'External: ' , external value

>>>> 

>>>> Output, with the key section highlighted,

>>>> 

>>>> In Smalltalk: 0

>>>> 

>>>> External: 0

>>>> 

>>>> In Smalltalk: 0

>>>> 

>>>> Flushing...

>>>> 

>>>> In Smalltalk: 0

>>>> 

>>>> External: 1000

>>>> 

>>>> In Smalltalk: 1000 (you can see here that Smalltalk implementation

>>>> will pick up the correct size after another process had made use of

>>>> a

>> 

>>>> proper call, via either a 'dir' command or explorer refresh in the

>>>> destination

>>>> directory)

>>>> 

>>>> Closing...

>>>> 

>>>> In Smalltalk: 1000

>>>> 

>>>> External: 1000

>>>> 

>>>> Any chance this could be addressed at some point to make use of a

>>>> more

>>> 

>>>> predictable Windows API?

>>>> 

>>>> Thanks,

>>>> 

>>>> -Boris

>>>> 

>>>> 

>>>> 

>>>> _______________________________________________

>>>> vwnc mailing list

>>>> [hidden email]

>>>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

>>> _______________________________________________

>>> vwnc mailing list

>>> [hidden email]

>>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

>>> 

>> _______________________________________________

>> vwnc mailing list

>> [hidden email]

>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

>> 

> _______________________________________________

> vwnc mailing list

> [hidden email]

> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

> 

_______________________________________________

vwnc mailing list

[hidden email]

http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

In fact, the evil primitive 666, which is what I’m using now via the IOAccessor>>fileSize, does it the right way,

siz.LowPart = GetFileSize(fd->handle, &siz.HighPart);

Now, if there was a way to pass pass a zero for fdwAccess through to CreateFile in <primitive: 1610> (Filename)primOpenFileDirection:creation: I would be golden, but even with #readOnly = GENERIC_READ, I suspect I’ll be okay for a while.

private int accessModes[5] = {

                GENERIC_READ,                                               /* read-only */

                GENERIC_WRITE,                                             /* write-only */

                GENERIC_READ | GENERIC_WRITE,         /* read/write */

                GENERIC_WRITE,                                             /* append-only */

                GENERIC_READ | GENERIC_WRITE          /* read/append */

};

-Boris

Andres,

To boil this down a bit after a lengthy discussion, my main issue is that things like '(gi .\sample.iso).length' or Windows Explorer return up-to-date information about the file size and VisualWorks' implementation of #fileSize does not. The key point that I am making is that I would much rather get up-to-date information and be subject to normal file system behaviour as far as other processes’ impact on the file is concerned (i.e. exceptions on access) than get information back that could be _minutes_ old. Here's why I think it should be possible to fix it (whether it’s done in a VM or in Smalltalk, I don’t really care),

http://msdn.microsoft.com/en-us/library/aa363858(v=vs.85).aspx

The dwDesiredAccess parameter can be zero, allowing the application to query file attributes without accessing the file if the application is running with adequate security settings. This is useful to test for the existence of a file without opening it for read and/or write access, or to obtain other statistics about the file or directory. See Obtaining and Setting File Information and GetFileInformationByHandle.

http://msdn.microsoft.com/en-us/library/aa365257(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/aa364957(v=vs.85).aspx

http://msdn.microsoft.com/en-us/library/aa364952(v=vs.85).aspx

If you want to get up-to-date information, you'd better use GetFileInformationByHandle.

From FindFileFirst MSDN doc: "In rare cases, file attribute information on NTFS file systems may not be current at the time you call this function. To be assured of getting the current NTFS file system file attributes, call the GetFileInformationByHandle function."

I dare to say that on heavily loaded system running software heavily optimized for performance (i.e. which doesn't call FlushFileBuffers on the left and on the right) these cases are not rare at all, and it is relatively easy to reproduce.

The thing is that FindFirstFile() retrieves information from directory entry which is updated lazily. Maximum latency for directory entry update (if memory serves me well) -- in absense of FlushFileBuffers() calls on modified handle -- is 5 minutes; that's the best we can hope for.

-Boris

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On Behalf Of Andres Valloud
Sent: 11 May 2011 16:16
To: [hidden email]
Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date information

From the POV of the VM, I'm concerned that adding anything other than reading what the directory says is bound to cause trouble.  For example, let's say you have the file foo.txt.  You want to know how large it is.

  But another process already opened foo.txt, called DeleteFile() on it, and has yet to close the file.  Then you cannot call CreateFile() because it fails with ERROR_ACCESS_DENIED.  This is bad because you cannot predict what another process is doing.

Or what if you're not supposed to open the file that was just deleted by the another app because the other app will recreate the file an instant later?  If the app assumes that the file is gone after DeleteFile() and CloseFile(), but the file is still there because the VM opened the file to determine its size before DeleteFile() was called by the app but the VM has yet to call CloseFile() so the file is still there, the other app might fail unexpectedly.

http://msdn.microsoft.com/en-us/library/aa363915%28VS.85%29.aspx

The same unpredictable problems should be expected when doing similar operations from the image.  Maybe there's another way to figure out what you need?

On 5/11/2011 12:55 PM, Boris Popov, DeepCove Labs wrote:

> If you hadn't requested a share mode that prevents others from doing

> certain things, the other application will still be able to do

> whatever the heck it is able to do normally, no? I'm not an expert on

> this, far from it, but as an example, doing #renameTo: in VisualWorks

> to a different path while Apache still has a handle on the file and is

> in the process of receiving data to write to it actually succeeds and

> Apache just keeps on writing to the new location. That's in the

> primitive 1603,

> 

> http://dl.dropbox.com/u/2737880/videos/rename%20a/index.html

> 

> Other fun things I can do include,

> 

> from delete. "in which case Apache continues writing to the file, but

> file disappears immediately after it's done with it"

> 

> or,

> 

> from renameTo: to.

> to delete. "in which case Apache continues writing to the new

> location, but file disappears immediately after it's done with it"

> 

> If anything, at least these methods should include a note that the

> information they return might not be up-to-date in certain

> circumstances, just like MSDN does when talking about FindFirstFile()

> and, ideally, include alternatives that do with their own sets of

> caveats.

> 

> -Boris

> 

> -----Original Message-----

> From: [hidden email] [mailto:[hidden email]] On

> Behalf Of Andres Valloud

> Sent: 11 May 2011 15:30

> To: [hidden email]

> Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date

> information

> 

> Ok, but note that it's more difficult to put that kind of stuff in the

> VM because the test procedure might fail in unexpected ways (e.g.:

> what happens if another process tries to delete the file while the VM

> has the file open?).

> 

> On 5/11/2011 9:14 AM, Boris Popov, DeepCove Labs wrote:

>> Andres,

>> 

>> I ended up using a Smalltalk implementation to get the result I was

>> looking for,

>> 

>> Filename>>rtFileSize

>> 

>>                   | tmp |

>>                   tmp := FileConnection openFileNamed: self mode:

>> #readOnly creationRule: #noCreate.

>>                   ^[tmp input fileSize] ensure: [tmp close].

>> 

>> Below you can see the difference in behaviour (sorry the videos are

>> quite wide), key is the difference in file size reporting in the

>> transcript as file is uploaded via cURL to a local Apache install,

>> 

>> http://dl.dropbox.com/u/2737880/videos/filesize%20a/index.html

>> http://dl.dropbox.com/u/2737880/videos/filesize%20b/index.html

>> 

>> It should be noted that Apache seems to be somewhat lax about the

>> share mode on the file that it opens for writing,

>> 

>> "12:06:20.6941918","httpd.exe","1320","CreateFile","C:\DeepCove\webda

>> v \f olders\ernest\payments\sample.iso","SUCCESS","Desired Access:

>> Generic Write, Read Attributes, Disposition: OverwriteIf, Options:

>> Synchronous IO Non-Alert, Non-Directory File, Attributes: n/a,

>> ShareMode: Read, Write, Delete, AllocationSize: 0, OpenResult:

>> Created"

>> 

>> Full process monitor action on that file is available here,

>> 

>> http://dl.dropbox.com/u/2737880/videos/apache.zip

>> 

>> -Boris

>> 

>> -----Original Message-----

>> From: [hidden email] [mailto:[hidden email]] On

>> Behalf Of Andres Valloud

>> Sent: 11 May 2011 12:07

>> To: [hidden email]

>> Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date

>> information

>> 

>> Looking at the docs, it seems that using GetFileInformationByHandle

>> would require first to get a handle to the file.  However, in

>> CreateFile, we can see using share mode zero "[p]revents other

>> processes from opening a file or device if they request delete, read,

>> or write access".  Thus, it may not be possible to get a handle in

>> the

> 

>> first place.  In those cases, it appears we won't be able to get

> "current"

>> file information.  But how to distinguish between the two from a

>> Smalltalk application?...

>> 

>> On 5/11/2011 8:50 AM, Boris Popov, DeepCove Labs wrote:

>>> Andres,

>>> 

>>> The workspace is an approximation of the behaviour I see in

>>> production

>> 

>>> where our app monitors files that are modified by other applications.

>>> The effect is the same, however, using #fileSize returns values that

>>> are off compared to the results one gets from a simple dir, at which

>>> point #fileSize once again returns correct information, but only as

>>> of

>> 

>>> the last dir even as the file continues growing. I'll see if I can

>>> get

>> 

>>> a process monitor log of the events for the file as it keeps on

>>> growing to show you what I mean.

>>> 

>>> -Boris

>>> 

>>> -----Original Message-----

>>> From: [hidden email] [mailto:[hidden email]] On

>>> Behalf Of Andres Valloud

>>> Sent: 11 May 2011 11:44

>>> To: [hidden email]

>>> Subject: Re: [vwnc] Primitives 1600 and 1606 return out-of-date

>>> information

>>> 

>>> Have you tried using #commit instead of #flush?

>>> 

>>> In Smalltalk: 0

>>> External: 0

>>> In Smalltalk: 0

>>> Committing...

>>> In Smalltalk: 1000

>>> External: 1000

>>> In Smalltalk: 1000

>>> Closing...

>>> In Smalltalk: 1000

>>> External: 1000

>>> 

>>> On 5/6/2011 11:02 AM, Boris Popov, DeepCove Labs wrote:

>>>> [VisualWorks 7.7.1 on Windows 7 x64 SP1]

>>>> 

>>>> I'm running into an issue where use of FindFirstFile() in

>>>> Filename>>getDates and Filename>>fileSize appears to give me

>>>> Filename>>information

>>>> that is significantly out-of-date (on Windows, anyway) and this is

>>>> the

>>> 

>>>> likely culprit,

>>>> 

>>>> http://msdn.microsoft.com/en-us/library/aa364418(v=vs.85).aspx

>>>> 

>>>> Note: In rare cases, file information on NTFS file systems may not

>>>> be

>> 

>>>> current at the time you call this function. To be assured of

>>>> getting

> 

>>>> the current file information, call the GetFileInformationByHandle

>>> function.

>>>> 

>>>> Here's a workspace to demonstrate the effect (only needs Regex in

>>>> the

>> 

>>>> clean image) using the file size example,

>>>> 

>>>> Transcript clear.

>>>> 

>>>> fn := 'test.txt' asFilename.

>>>> 

>>>> ws := fn writeStream.

>>>> 

>>>> external :=

>>>> 

>>>> [out := ExternalProcess fork: 'cmd'

>>>> 

>>>> arguments: (Array

>>>> 

>>>> with: '/C'

>>>> 

>>>> with: 'dir'

>>>> 

>>>> with: '/-C'

>>>> 

>>>> with: fn asAbsoluteFilename asString).

>>>> 

>>>> rx := '.*File\(s\) *(\d*) bytes.*' asRegex.

>>>> 

>>>> rx matches: out.

>>>> 

>>>> rx subexpression: 2].

>>>> 

>>>> [1000 timesRepeat: [ws nextPut: $a].

>>>> 

>>>> Transcript

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'External: ' , external value;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'Flushing...'.

>>>> 

>>>> ws flush.

>>>> 

>>>> Transcript

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'External: ' , external value;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'Closing...']

>>>> 

>>>> ensure: [ws close].

>>>> 

>>>> Transcript

>>>> 

>>>> cr;

>>>> 

>>>> show: 'In Smalltalk: ' , fn fileSize printString;

>>>> 

>>>> cr;

>>>> 

>>>> show: 'External: ' , external value

>>>> 

>>>> Output, with the key section highlighted,

>>>> 

>>>> In Smalltalk: 0

>>>> 

>>>> External: 0

>>>> 

>>>> In Smalltalk: 0

>>>> 

>>>> Flushing...

>>>> 

>>>> In Smalltalk: 0

>>>> 

>>>> External: 1000

>>>> 

>>>> In Smalltalk: 1000 (you can see here that Smalltalk implementation

>>>> will pick up the correct size after another process had made use of

>>>> a

>> 

>>>> proper call, via either a 'dir' command or explorer refresh in the

>>>> destination

>>>> directory)

>>>> 

>>>> Closing...

>>>> 

>>>> In Smalltalk: 1000

>>>> 

>>>> External: 1000

>>>> 

>>>> Any chance this could be addressed at some point to make use of a

>>>> more

>>> 

>>>> predictable Windows API?

>>>> 

>>>> Thanks,

>>>> 

>>>> -Boris

>>>> 

>>>> 

>>>> 

>>>> _______________________________________________

>>>> vwnc mailing list

>>>> [hidden email]

>>>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

>>> _______________________________________________

>>> vwnc mailing list

>>> [hidden email]

>>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

>>> 

>> _______________________________________________

>> vwnc mailing list

>> [hidden email]

>> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

>> 

> _______________________________________________

> vwnc mailing list

> [hidden email]

> http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

> 

_______________________________________________

vwnc mailing list

[hidden email]

http://lists.cs.uiuc.edu/mailman/listinfo/vwnc