Smalltalk › Gnu

walking a directory tree

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

20 messages Options

SeanTAllen

walking a directory tree

i did see anything like this but, before i go ahead and code it,
i thought i would check to make sure i didn't miss anything.

is there a method to enumerate through all entries in a directory,
all the way down.. ( not just the top level in a directory )

so

/x/
x/y/yz
/x/y/z
/x/y/z/a

would return operate on entries from /x all the down to /x/y/z/a

if not, would there be interest in adding such to the base classes?

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

Paolo Bonzini-2

Re: walking a directory tree

Sean Allen wrote:
> i did see anything like this but, before i go ahead and code it,
> i thought i would check to make sure i didn't miss anything.
>
> is there a method to enumerate through all entries in a directory,
> all the way down.. ( not just the top level in a directory )

Yes,

(File name: 'foo') allFilesMatching: aPattern do: aBlock

the "aPattern" has to be in the same syntax as String>>#match:, so ? is
replaced by # and * works as in the shell.

Or you can use #all the same as I was suggesting for chown:

(File name: 'foo') all do: [ :file | ]
(File name: 'foo') all namesDo: [ :string | ]
(File name: 'foo') all filesMatching: aPattern do: aBlock
(File name: 'foo') all namesMatching: aPattern do: aBlock

FilePath>>#all is documented, albeit a bit cryptically:

Return a decorator of the receiver that will provide recursive
descent into directories for iteration methods.

Regarding your other message, Directory is present for backwards
compatibility. Everything it implemented has already been moved to
FilePath or File. Otherwise, navigating a filesystem like above would
have implied a "stat" system call for every file, to know whether to
create a File or Directory instance.

Directory still has a few class methods, most of which create instances
of File but were left there because those instances are always
directories (e.g. "Directory home").

I updated the class comment and will regenerate the manual in a couple
of hours. Thanks for your work again, every stumbling of yours will
make GST a better product for others to learn.

> Or are we supposed use in this fashion?
>
> file := File name: '/Users/Spooneybarger'
> file isDirectory ifTrue: [ file directories ]

Yes.

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

SeanTAllen

Re: walking a directory tree

On Jan 9, 2009, at 1:16 AM, Paolo Bonzini wrote:

> Sean Allen wrote:
>> i did see anything like this but, before i go ahead and code it,
>> i thought i would check to make sure i didn't miss anything.
>>
>> is there a method to enumerate through all entries in a directory,
>> all the way down.. ( not just the top level in a directory )
>
> Yes,
>
> (File name: 'foo') allFilesMatching: aPattern do: aBlock
>
> the "aPattern" has to be in the same syntax as String>>#match:, so ?
> is
> replaced by # and * works as in the shell.
>
> Or you can use #all the same as I was suggesting for chown:

i have to admit, i either missed that or it didnt stick in my head.

>
>
> (File name: 'foo') all do: [ :file | ]
> (File name: 'foo') all namesDo: [ :string | ]
> (File name: 'foo') all filesMatching: aPattern do: aBlock
> (File name: 'foo') all namesMatching: aPattern do: aBlock
>

I'm experiencing some weirdness with this...

this works:
file := File name: '/Users/Spooneybarger'
file all do: [ :e | e isDirectory ifTrue: [ e name printNl ] ]

if i do:
file := File name: '/Users/Spooneybarger'
file all do: [ :e | e isDirectory ifTrue: [ e directories printNl ] ]

i get no output other than: "Global garbage collection... done, heap
grown"
over and over and over

yet a boring old

file directories printNl

works fine.

i'm really just playing around right now. shouldnt i get output from the

ifTrue: [ e directories printNl ]

does the fact that I'm not mean that I have a messed up build?

> Thanks for your work again, every stumbling of yours will
> make GST a better product for others to learn.

I would call my stumblings work, but you are welcome nonetheless.
Thank you for the assistance in getting me up and runing.

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

Paolo Bonzini-2

Re: walking a directory tree

> if i do:
> file := File name: '/Users/Spooneybarger'
> file all do: [ :e | e isDirectory ifTrue: [ e directories printNl ] ]
>
> i get no output other than: "Global garbage collection... done, heap grown"
> over and over and over

The problem here is that every file that you get in "file all do:" is in
turn a RecursiveFileWrapper. In other words what you wanted was just

file all directories do: [ :each | each printNl ]

(try "file all directories").

I'm wondering whether this is totally counter-intuitive though. After
all, you're already doing a recursive visit, so it makes sense to get
the original files (it may make sense for other kinds of wrapper, but
probably not for RecursiveFileWrapper).

The attached patch fixes the above misfeature.

> i'm really just playing around right now. shouldnt i get output from the
>
> ifTrue: [ e directories printNl ]
>
> does the fact that I'm not mean that I have a messed up build?

No.

>> Thanks for your work again, every stumbling of yours will
>> make GST a better product for others to learn.
>
> I would call my stumblings work, but you are welcome nonetheless.

Your stumblings so far were mostly my fault, if it makes my intention
clearer. :-)

Paolo

diff --git a/kernel/VFS.st b/kernel/VFS.st
index 9a01b02..a472258 100644
--- a/kernel/VFS.st
+++ b/kernel/VFS.st
@@ -356,7 +356,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
[:name |
| f |
f := self at: name.
- aBlock value: f.
+ aBlock value: f file.
f isDirectory
ifTrue:
[((#('.' '..') includes: name) or: [f isSymbolicLink])
@@ -422,7 +422,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
self isDirectory ifFalse: [
^super lastAccessTime: accessDateTime lastModifyTime: modifyDateTime ].
self do: [ :each |
- each file lastAccessTime: accessDateTime lastModifyTime: modifyDateTime ]
+ each lastAccessTime: accessDateTime lastModifyTime: modifyDateTime ]
]

owner: ownerString group: groupString [
@@ -434,7 +434,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
"These special calls cache the uid and gid to avoid repeated lookups."
[
File setOwnerFor: nil owner: ownerString group: groupString.
- self do: [ :each | each file owner: ownerString group: groupString ]
+ self do: [ :each | each owner: ownerString group: groupString ]
] ensure: [ File setOwnerFor: nil owner: nil group: nil ]
]

@@ -445,8 +445,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
<category: 'accessing'>
self isDirectory ifFalse: [ ^super mode: anInteger ].

- self do: [ :each |
- each isDirectory ifFalse: [ each file mode: anInteger ] ]
+ self do: [ :each | each isDirectory ifFalse: [ each mode: anInteger ] ]
]

fileMode: fMode directoryMode: dMode [
@@ -459,8 +458,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
super mode: dMode.
self isDirectory ifTrue: [
self do: [ :each |
- each file
- mode: (each isDirectory
+ each mode: (each isDirectory
ifTrue: [ dMode ]
ifFalse: [ fMode ]) ] ]
]

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

Paolo Bonzini-2

Re: walking a directory tree

> The attached patch fixes the above misfeature.

Wrong patch attached, this is the correct one.

Paolo

diff --git a/kernel/FilePath.st b/kernel/FilePath.st
index 54a14dc..f35156e 100644
--- a/kernel/FilePath.st
+++ b/kernel/FilePath.st
@@ -736,7 +736,10 @@ size and timestamps.'>

all [
"Return a decorator of the receiver that will provide recursive
- descent into directories for iteration methods."
+ descent into directories for iteration methods. Furthermore,
+ iteration on the returned wrapper will not include '.' or '..'
+ directory entries, and will include the receiver (directly, not
+ via '.')."

<category: 'decoration'>
^Kernel.RecursiveFileWrapper on: self
@@ -813,7 +816,9 @@ size and timestamps.'>

do: aBlock [
"Evaluate aBlock once for each file in the directory represented by the
- receiver, passing its name."
+ receiver, passing a FilePath object (or a subclass) to it. It depends
+ on the subclass whether iteration will include the '.' and '..'
+ directory entries."

<category: 'enumerating'>
self namesDo: [ :name |
@@ -822,7 +827,8 @@ size and timestamps.'>

namesDo: aBlock [
"Evaluate aBlock once for each file in the directory represented by the
- receiver, passing its name."
+ receiver, passing its name. It depends on the subclass whether
+ iteration will include the '.' and '..' directory entries."

<category: 'enumerating'>
self subclassResponsibility
diff --git a/kernel/VFS.st b/kernel/VFS.st
index 9a01b02..d344a04 100644
--- a/kernel/VFS.st
+++ b/kernel/VFS.st
@@ -352,15 +352,16 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
"Same as the wrapped #do:, but reuses the file object for efficiency."

<category: 'enumerating'>
+ aBlock value: self file.
self file namesDo:
[:name |
- | f |
- f := self at: name.
- aBlock value: f.
- f isDirectory
- ifTrue:
- [((#('.' '..') includes: name) or: [f isSymbolicLink])
- ifFalse: [f do: aBlock]]]
+ | f period |
+ period := #('.' '..') includes: name.
+ period ifFalse: [
+ f := self at: name.
+ aBlock value: f file.
+ (f isDirectory and: [f isSymbolicLink not])
+ ifTrue: [f do: aBlock]]]
]

namesDo: aBlock prefixLength: anInteger [
@@ -371,15 +372,15 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
<category: 'private'>
self file namesDo:
[:name |
- | f |
- f := self at: name.
- aBlock value: (f asString copyFrom: anInteger).
- f isDirectory
- ifTrue:
- [((#('.' '..') includes: name) or: [f isSymbolicLink])
- ifFalse: [f
- namesDo: aBlock
- prefixLength: anInteger ]]]
+ | f period |
+ period := #('.' '..') includes: name.
+ period ifFalse: [
+ f := self at: name.
+ aBlock value: (f asString copyFrom: anInteger).
+ (f isDirectory and: [f isSymbolicLink not])
+ ifTrue: [f
+ namesDo: aBlock
+ prefixLength: anInteger ]]]
]

namesDo: aBlock [
@@ -387,6 +388,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
tree recursively."

<category: 'enumerating'>
+ aBlock value: '.'.
self namesDo: aBlock prefixLength: self asString size + 2
]

@@ -422,7 +424,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
self isDirectory ifFalse: [
^super lastAccessTime: accessDateTime lastModifyTime: modifyDateTime ].
self do: [ :each |
- each file lastAccessTime: accessDateTime lastModifyTime: modifyDateTime ]
+ each lastAccessTime: accessDateTime lastModifyTime: modifyDateTime ]
]

owner: ownerString group: groupString [
@@ -434,7 +436,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
"These special calls cache the uid and gid to avoid repeated lookups."
[
File setOwnerFor: nil owner: ownerString group: groupString.
- self do: [ :each | each file owner: ownerString group: groupString ]
+ self do: [ :each | each owner: ownerString group: groupString ]
] ensure: [ File setOwnerFor: nil owner: nil group: nil ]
]

@@ -445,8 +447,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
<category: 'accessing'>
self isDirectory ifFalse: [ ^super mode: anInteger ].

- self do: [ :each |
- each isDirectory ifFalse: [ each file mode: anInteger ] ]
+ self do: [ :each | each isDirectory ifFalse: [ each mode: anInteger ] ]
]

fileMode: fMode directoryMode: dMode [
@@ -459,8 +460,7 @@ VFS.FileWrapper subclass: RecursiveFileWrapper [
super mode: dMode.
self isDirectory ifTrue: [
self do: [ :each |
- each file
- mode: (each isDirectory
+ each mode: (each isDirectory
ifTrue: [ dMode ]
ifFalse: [ fMode ]) ] ]
]

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

SeanTAllen

Re: walking a directory tree

In reply to this post by Paolo Bonzini-2

On Jan 9, 2009, at 3:58 AM, Paolo Bonzini wrote:

>>>
>>> Thanks for your work again, every stumbling of yours will
>>> make GST a better product for others to learn.
>>
>> I would call my stumblings work, but you are welcome nonetheless.

That was supposed to say:

I wouldn't call my stumblings work...

sigh... ah... i should go to bed.

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

SeanTAllen

Re: walking a directory tree

In reply to this post by Paolo Bonzini-2

On Jan 9, 2009, at 3:58 AM, Paolo Bonzini wrote:

>> if i do:
>> file := File name: '/Users/Spooneybarger'
>> file all do: [ :e | e isDirectory ifTrue: [ e directories
>> printNl ] ]
>>
>> i get no output other than: "Global garbage collection... done,
>> heap grown"
>> over and over and over
>
> The problem here is that every file that you get in "file all do:"
> is in
> turn a RecursiveFileWrapper. In other words what you wanted was just
>
> file all directories do: [ :each | each printNl ]
>
> (try "file all directories").

I haven't applied you patch yet, as I want to understand how it would
be without the patch,
( I'm the curious sort like that... ) and I didn't get what I would
except from your example:

st> ( File name: '/Users/Spooneybarger' ) all directories do: [ :each
| each printNl ]
"Global garbage collection... done"
"Global garbage collection... done, heap grown"
"Global garbage collection... done, heap grown"
"Global garbage collection... done, heap grown"
"Global garbage collection... done, heap grown"
...snip...

What am I doing wrong?

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

Paolo Bonzini-2

Re: walking a directory tree

Sean Allen wrote:
>
> On Jan 9, 2009, at 3:58 AM, Paolo Bonzini wrote:
>
>>> if i do:
>>> file := File name: '/Users/Spooneybarger'
>>> file all do: [ :e | e isDirectory ifTrue: [ e directories printNl ] ]
>
> What am I doing wrong?

"file all do:" is doing an "ls -lR" which is already slow. :-)

Since without my patch "e" is in turn a RecursiveFileWrapper (the kind
of object returned by #all), each send of #directories in the block
would in turn invoke a recursive descent. So that's a loop of "ls -lR"s
inside an "ls -lR", and it's going to take a while. :-) Actually it's
going to be infinite, because sooner or later you'll do "e directories"
on "/Users/Spooneybarger/..".

With my patch, first of all "e" is a normal File so you have a loop of
"ls" inside an "ls -lR", and it's a bit faster. Second, "." and ".."
are not passed by "file all do:"; this matches the behavior of the
command-line utility "find", for example (they're still passed by "file
do:", which matches "ls -a").

Note that the output of

file all directories do: [ :each | each printNl ]

and

file all do: [ :e | e isDirectory ifTrue: [ e directories printNl ] ]

is different. Suppose you have

/a
/a/b
/a/c
/a/c/d

the first will print (more or less) exactly that. The latter would
print several Arrays (each with the list of subdirectories in each
directory) which will be:

(/a/. /a/.. /a/b /a/c)
(/a/b/. a/b/..)
(/a/c/. a/c/.. /a/c/d)
(/a/c/d/. a/c/d/..)

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

SeanTAllen

Re: walking a directory tree

In reply to this post by SeanTAllen

On Jan 9, 2009, at 4:31 AM, Sean Allen wrote:

>
> On Jan 9, 2009, at 3:58 AM, Paolo Bonzini wrote:
>
>>> if i do:
>>> file := File name: '/Users/Spooneybarger'
>>> file all do: [ :e | e isDirectory ifTrue: [ e directories
>>> printNl ] ]
>>>
>>> i get no output other than: "Global garbage collection... done,
>>> heap grown"
>>> over and over and over
>>
>> The problem here is that every file that you get in "file all do:"
>> is in
>> turn a RecursiveFileWrapper. In other words what you wanted was just
>>
>> file all directories do: [ :each | each printNl ]
>>
>> (try "file all directories").
>
> I haven't applied you patch yet, as I want to understand how it
> would be without the patch,
> ( I'm the curious sort like that... ) and I didn't get what I would
> except from your example:
>
> st> ( File name: '/Users/Spooneybarger' ) all directories do:
> [ :each | each printNl ]
> "Global garbage collection... done"
> "Global garbage collection... done, heap grown"
> "Global garbage collection... done, heap grown"
> "Global garbage collection... done, heap grown"
> "Global garbage collection... done, heap grown"
> ...snip...
>
> What am I doing wrong?
>

Ok ten minutes later, after ton of garbage collection message it
suddenly returned a mass of output.
So, suddenly it makes sense, its getting all the directories then
outputting the information
so because there are a ton of directories under that one... gc gets
called over and over and it
takes a long time to process...

Sorry, I shouldn't have assumed after just a minute that it wasnt
going to return anything and that I still had it wrong...

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk

SeanTAllen

Re: walking a directory tree

In reply to this post by Paolo Bonzini-2

On Jan 12, 2009, at 7:33 AM, Paolo Bonzini wrote:

>
>> Ok, I must be doing something wrong because I still dont have a
>> version
>> of kernel/VFS.st
>> that has the patch that ignores . and ..
>
> The patch is b19032be93bf6cdb714a1b6205f2926025cbd621. Currently it
> is
> the penultimate (there is another one over it).

ah i see... its different than the original patch you sent.

slowly learning git. if i knew it already that would have been easier.

thanks Paolo.

_______________________________________________
help-smalltalk mailing list
[hidden email]
http://lists.gnu.org/mailman/listinfo/help-smalltalk