directory recursive delete problem on Windows

Hi all,

Grit just ran into an -interesting to say the least- problem with
recursive deletion of directories on Windows. We are not able to provide
a stripped down test case yet, but here some info:

- we extract a zip archive to provide an initial setup for Plopp data
- then the user can play around in demo mode (problem doesn't occur in a
full version as the deletion doesn't take place)
- upon quitting demo mode all data is erased via a recursive directory
delete


We have simple test up without running the UI to reproduce the problem.

The delete fails about 1 in 10 with a primitive failed in deleteDirectory.

There are no open files or anything obvious.

Now the interesting part...
If we add a delay of 1 sec after catching the error and then simply
retry the exception to delete the directory again, it works.

[Painter3DStorage storageDirectory recursiveDelete]
        on: Error
        do: [: exception |
                (Delay forMilliseconds: 1000) wait.
                exeption retry]

Any ideas?

Michael

Michael Rueger wrote:

> Now the interesting part...
> If we add a delay of 1 sec after catching the error and then simply
> retry the exception to delete the directory again, it works.

Adding a GC doesn't btw...

Michael

Might be worth a little effort to log the actual OS error? It may be  
as simple as the directory/FS being locked while the OS is busy.

tim
--
tim Rowledge; [hidden email]; http://www.rowledge.org/tim
Oxymorons: Advanced BASIC

On Aug 28, 2007, at 10:33 , tim Rowledge wrote:

> Might be worth a little effort to log the actual OS error? It may  
> be as simple as the directory/FS being locked while the OS is busy.

Yes - like a virus scanner or indexer etc. I hear these things cause  
all kinds of problems on Windows.

- Bert -

Well the mac source code does this below which does some sanity  
checks on the size of the path, then resolves the squeak supplied string
to the name understood by the flavor of os-x bsd unix.  (Mind there  
is a bug lurking here, but I'll let people guess).

Also note the side effect of changing lastPath to 0x00 if the path  
being deleted is the same as the lastPath. Where lastPath is a cached  
value
of the last directory used for performance reasons.

After all this we call rmdir() which
"The rmdir utility removes the directory entry specified by each  
directory argument, provided it is empty."


sqInt dir_Delete(char *pathString, sqInt pathStringLength)
{
   /* Delete the existing directory with the given path. */
   char name[DOCUMENT_NAME_SIZE+1];
   if (pathStringLength >= DOCUMENT_NAME_SIZE)
     return false;
   if (!ioFilenamefromStringofLengthresolveAliasesRetry
(name,pathString, pathStringLength, false, true))
     return false;
   if (strcmp(lastPath, name) == 0)
                lastPath[0] = 0x00;
               
   return rmdir(name) == 0;
}

For windows it does

int dir_Delete(char *pathString, int pathLength) {
   /* Delete the existing directory with the given path. */
   WCHAR win32Path[MAX_PATH];
   int sz;
   /* convert the file name into a null-terminated C string */
   sz = MultiByteToWideChar(CP_UTF8, 0, pathString, pathLength, NULL,  
0);
   if(sz > MAX_PATH) return 0;
   MultiByteToWideChar(CP_UTF8, 0, pathString, pathLength, win32Path,  
sz);
   win32Path[sz] = 0;
   if(hasCaseSensitiveDuplicate(win32Path)) return false;
   return RemoveDirectoryW(win32Path) == 0 ? false : true;
}


http://msdn2.microsoft.com/en-us/library/aa365488.aspx


Lastly wasn't there some google code of summer work, looking at  
passing back error messages from the VM in a more meaningful manner.


On Aug 28, 2007, at 10:20 AM, Michael Rueger wrote:

--
========================================================================
===
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
========================================================================
===

I have also seen this problem on Windows.
>From what I remember, disabling the windows indexing service + any virus checkers + Google desktop search, made it happen less often :)
 
Also, it was more likely to occur when an explorer, viewing the folders contents, was open.
And, from what I recall, it was made worse if the folder contained a .thumbs file and so it happened more often on XP than on Win2000.
 
In the end we solved the problem by using the Shell.dll function - SHFileOperation - with the FO_DELETE , FOF_NOERRORUI, FOF_SILENT, FOF_NOCONFIRMATION flags; to do the folder delete. We also removed any read only, archive, flags from the files in the folder before calling the function).
 
You could probably use FFI to call the function, or else make a plugin.
Cheers,
Andy

.

---

> Date: Tue, 28 Aug 2007 19:20:54 +0200

---

Email straight to your blog, upload jokes, photos and more. Windows Live Spaces, it's FREE!

I should have mentioned that SHFileOperation will remove a folder AND its contents in one operation. There is no need to delete the folder's contents first.
Also, when setting the folders files to not read-only; this is done RECURSIVELY.
i.e. the folder's files, + the folder's child folders' files, + the folder's folders' folders' files etc..
 
Depending on the flags used, this operation may NOT put the deleted folder into the recycle bin. So beware, it can instantly remove a large chunk of your disk if you get the folder path wrong !
 
Cheers,
Andy

---

From: [hidden email]
To: [hidden email]
Date: Tue, 28 Aug 2007 20:28:01 +0100
Subject: RE: directory recursive delete problem on Windows

I have also seen this problem on Windows.
>From what I remember, disabling the windows indexing service + any virus checkers + Google desktop search, made it happen less often :)
 
Also, it was more likely to occur when an explorer, viewing the folders contents, was open.
And, from what I recall, it was made worse if the folder contained a .thumbs file and so it happened more often on XP than on Win2000.
 
In the end we solved the problem by using the Shell.dll function - SHFileOperation - with the FO_DELETE , FOF_NOERRORUI, FOF_SILENT, FOF_NOCONFIRMATION flags; to do the folder delete. We also removed any read only, archive, flags from the files in the folder before calling the function).
 
You could probably use FFI to call the function, or else make a plugin.
Cheers,
Andy

.

---

> Date: Tue, 28 Aug 2007 19:20:54 +0200
> From: [hidden email]
> To: [hidden email]
> Subject: directory recursive delete problem on Windows
>
> Hi all,
>
> Grit just ran into an -interesting to say the least- problem with
> recursive deletion of directories on Windows. We are not able to provide
> a stripped down test case yet, but here some info:
>
> - we extract a zip archive to provide an initial setup for Plopp data
> - then the user can play around in demo mode (problem doesn't occur in a
> full version as the deletion doesn't take place)
> - upon quitting demo mode all data is erased via a recursive directory
> delete
>
>
> We have simple test up without running the UI to reproduce the problem.
>
> The delete fails about 1 in 10 with a primitive failed in deleteDirectory.
>
> There are no open files or anything obvious.
>
> Now the interesting part...
> If we add a delay of 1 sec after catching the error and then simply
> retry the exception to delete the directory again, it works.
>
> [Painter3DStorage storageDirectory recursiveDelete]
> on: Error
> do: [: exception |
> (Delay forMilliseconds: 1000) wait.
> exeption retry]
>
> Any ideas?
>
> Michael
>
>
>

---

Email straight to your blog, upload jokes, photos and more. Windows Live Spaces, it's FREE!

---

Are you the Quizmaster? Play BrainBattle with a friend now!

That's a fairly common problem with file deletion on Windows. On a
typical installation, there *are* often open files - the usual
suspects are a virus scanners and file system indexers/search engines;
the explorer may also open files if the directory is display in the
explorer.
They register for file change notifications, and when files are created,
they put them onto their todo list, and will eventually open them
(usually fairly quickly after they got created). They then read them
once, and deletion attempts will fail at that point.

If that's the cause, it is fairly difficult to work-around. The "proper"
way is first to try DeleteFile (talking plain Win32 API here); if that
fails with sharing violation, move the file to the trash folder, open
it for deletion, SetFileInformation with a delete disposition, and close
the file. In the latter case, the file will go away if the last handle
is closed.

If you wait some time and retry, whoever had the file open will have
closed it.

Regards,
Martin

While I can't dismiss the possibility of file indexers and virus
scanners, in my experience most problems are closer to home ;-) It
should also be easy to find out by running the code on a box without
much crapware installed.

Cheers,
   - Andreas

Martin v. Löwis wrote:

I've seen this with a VisualSmalltalk app, and a workaround that fixed the
problem was to use shell32.dll SHFileOperation.
(I have posted a detailed answer already on this thread, but it isn't
showing up in gmane. Just in case it has got lost, I've repeated the
original reply below)
--
I have also seen this problem on Windows.
>From what I remember, disabling the windows indexing service + any virus
checkers + Google desktop search, made it happen less often :)

Also, it was more likely to occur when an explorer, viewing the folders
contents, was open.
And, from what I recall, it was made worse if the folder contained a .thumbs
file and so it happened more often on XP than on Win2000.

In the end we solved the problem by using the Shell.dll function -
SHFileOperation - with the FO_DELETE , FOF_NOERRORUI, FOF_SILENT,
FOF_NOCONFIRMATION flags; to do the folder delete. We also removed any read
only, archive, flags from the files in the folder before calling the
function).

You could probably use FFI to call the function, or else make a plugin.
--
I should have mentioned that SHFileOperation will remove a folder AND its
contents in one operation. There is no need to delete the folder's contents
first.
Also, when setting the folders files to not read-only; this is done
RECURSIVELY.
i.e. the folder's files, + the folder's child folders' files, + the folder's
folders' folders' files etc..

Depending on the flags used, this operation may NOT put the deleted folder
into the recycle bin. So beware, it can instantly remove a large chunk of
your disk if you get the folder path wrong !