[ANN] ExternalEnumeration

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] ExternalEnumeration

Randy Coulman

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Paul Baumann

Hi Randy,

 

Sorry to put you on the spot like this, but what advantage would Enumerator provide?

 

"Using an Enumerator"

#(one two three four five) elements groupedBy: #size

 

                "The standard way"

#(one two three four five) groupedBy: #size

 

The way I read the code it looks like a different way to enumerate that is usually slower, can only enumerate values, and is hard to for application code to configure in a backward-compatible way.

 

The framework would need to be greatly enhanced for it to be capable of providing a performance advantage. Specialized Enumerators would be needed for efficient implementation-specific access. The 'species' argument configures a return type that is inefficiently grown from default size with a writeStream that would then copy #contents to another collection. The writeStream is likely used because it allows any kind of collection to be used, but performance is more important and people generally expect a standard return type for the enumeration method. The framework would need  to be able to pass context as extra arguments through the block if you wanted to try to efficient block enumeration. Enumerator could be considered a step toward that kind of optimization, but it starts from something slower.

 

Consider how a SortedCollection responds from #select:. Enumerator uses the default sort block for #select: (instead of the same sort block preserved through #copyEmpty).  Note that SortedCollection is one kind of class that has a different kind of response for #select: (#copyEmpty) and #collect: (OrderedCollection). Enumerator can be configured for response type, but giving the species alone gives insufficient control over how the response is preconfigured with space or initialized with things like the sort block. OrderedCollection has different return types for #select: and #collect:. It is nice that users can figure the type of response, but existing code would likely get broken if application code is now expected to configure the enumerator in a backward-compatible way for response type.

 

Give examples of how Enumerator would enumerate with methods like #doWithIndex:, #keysAndValuesDo:, #associationsDo:, #keysDo:, and #valuesAndCountsDo:. There is only support for one-argument evaluations. The key of each object enumerated is often important for filling the response.

The Enumerator class should probably be an abstract superclass of an ValueEnumerator that has other siblings like KeyValueEnumerator and BasicEnumerator. Enumerator should probably allow a response to be configured rather than just the species of the response. The #elements method should provide an efficient Enumerator that is configured by default to answer the same kind of response as without sending #elements. Other subclasses of Enumerator would be to account for implementation-specific performance. Fast forward a few iterations and then it becomes evident that the most efficient and maintainable code involves some core extensions to the Collection classes rather than a separate parallel hierarchy of classes. The Enumerator pattern has some in common with the separation that IBM had originally built into ENVY/400 (pre-IBMST and VisualAge) collections. IBM replaced that design for better performance.

 

I'd want Enumerator to provide some form of real advantage right out of the box. That is possible, but the advantage of this implementation appears more theoretical than practical. The only advantage that Enumerator seems to currently provide is extra control over the kind of object that is returned from an enumeration. If that is the only advantage then Enumerator is a slower but slightly more readable alternative to an #inject:into:.

 

Paul Baumann

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Randy Coulman
Sent: Tuesday, April 29, 2014 09:49
To: VW NC
Subject: [vwnc] [ANN] ExternalEnumeration

 

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the <a href="http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Public&#43;Store&#43;Repository"> Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman



This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
Paul,

I didn't provide very much detail in the announcement to the list, but if you click through to the blog post, you'll see more details on the motivation for this library.  There are two main cases that I see:

1) You have a domain class that acts like a collection, but doesn't inherit from any of the collection classes.  Instead, it wraps an internal collection.  You generally don't want to expose the internal collection directly to client code, because then the client code can modify your internal collection without your class knowing about it.  In this case, your class can expose a #do: method that forwards on to the internal collection.  You can then wrap your object and its do selector in an Enumerator and have access to all of the enumeration protocol, but none of the modification protocol.

2) You have a complex data structure that can be enumerated using a #do:-style method.  Wrapping that object in an Enumerator allows you to use the full collection protocol on it.  For example, Behavior>>allSubclassesDo: or VisualComponent>>childrenDo:

Enumerator implements the "external iterator" pattern.

If I've already got a real collection, I don't need this library.  This is for cases where I have a collection-like object that can be enumerated, but doesn't expose the full enumeration protocol on its own.

Your other points are well-taken, and this library could evolve in the future to address them if the need arises.

Randy



On Tue, Apr 29, 2014 at 11:21 AM, Paul Baumann <[hidden email]> wrote:

Hi Randy,

 

Sorry to put you on the spot like this, but what advantage would Enumerator provide?

 

"Using an Enumerator"

#(one two three four five) elements groupedBy: #size

 

                "The standard way"

#(one two three four five) groupedBy: #size

 

The way I read the code it looks like a different way to enumerate that is usually slower, can only enumerate values, and is hard to for application code to configure in a backward-compatible way.

 

The framework would need to be greatly enhanced for it to be capable of providing a performance advantage. Specialized Enumerators would be needed for efficient implementation-specific access. The 'species' argument configures a return type that is inefficiently grown from default size with a writeStream that would then copy #contents to another collection. The writeStream is likely used because it allows any kind of collection to be used, but performance is more important and people generally expect a standard return type for the enumeration method. The framework would need  to be able to pass context as extra arguments through the block if you wanted to try to efficient block enumeration. Enumerator could be considered a step toward that kind of optimization, but it starts from something slower.

 

Consider how a SortedCollection responds from #select:. Enumerator uses the default sort block for #select: (instead of the same sort block preserved through #copyEmpty).  Note that SortedCollection is one kind of class that has a different kind of response for #select: (#copyEmpty) and #collect: (OrderedCollection). Enumerator can be configured for response type, but giving the species alone gives insufficient control over how the response is preconfigured with space or initialized with things like the sort block. OrderedCollection has different return types for #select: and #collect:. It is nice that users can figure the type of response, but existing code would likely get broken if application code is now expected to configure the enumerator in a backward-compatible way for response type.

 

Give examples of how Enumerator would enumerate with methods like #doWithIndex:, #keysAndValuesDo:, #associationsDo:, #keysDo:, and #valuesAndCountsDo:. There is only support for one-argument evaluations. The key of each object enumerated is often important for filling the response.

The Enumerator class should probably be an abstract superclass of an ValueEnumerator that has other siblings like KeyValueEnumerator and BasicEnumerator. Enumerator should probably allow a response to be configured rather than just the species of the response. The #elements method should provide an efficient Enumerator that is configured by default to answer the same kind of response as without sending #elements. Other subclasses of Enumerator would be to account for implementation-specific performance. Fast forward a few iterations and then it becomes evident that the most efficient and maintainable code involves some core extensions to the Collection classes rather than a separate parallel hierarchy of classes. The Enumerator pattern has some in common with the separation that IBM had originally built into ENVY/400 (pre-IBMST and VisualAge) collections. IBM replaced that design for better performance.

 

I'd want Enumerator to provide some form of real advantage right out of the box. That is possible, but the advantage of this implementation appears more theoretical than practical. The only advantage that Enumerator seems to currently provide is extra control over the kind of object that is returned from an enumeration. If that is the only advantage then Enumerator is a slower but slightly more readable alternative to an #inject:into:.

 

Paul Baumann

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Randy Coulman
Sent: Tuesday, April 29, 2014 09:49
To: VW NC
Subject: [vwnc] [ANN] ExternalEnumeration

 

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman



This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.



--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Paul Baumann

Yes. I see that Enumerator would have value for use by non-collection classes as an Iterator of a Composite Pattern or Chain Of Responsibility design pattern. Thanks for pointing that out.

 

Here are ideas you might want to consider for your design...

 

One technique for enumerators to handle enumeration of both keys and values with a one-argument block is to reuse a single instance of an association for keyed collections that don't store associations. Dictionary for example is implemented with associations so you could enumerate those. A KeyValueEnumerator iterator over an Array could create one association that is populated and reused for each execution. An association is not created for each item. The risk of that optimization technique is that the block may use the association argument itself instead of asking for key and value.

 

"This would be valid"

(Dictionary

                at: #one put: 1;

                at: #two put: 2;

                assocationEnum)

                                collect: [:assoc | assoc ].

 

"This would be answer a collection of two references to the same association"

#(one two) assocationEnum

                collect: [:assoc | assoc ].

 

"This is how a KeyValueEnumerator is kind of like a keysAndValuesDo:/doWithIndex: "

| dict |

dict := Dictionary new.

#(one two) assocationEnum

                do: [:assoc | dict at: assoc value put: assoc key  ].

^dict

 

 

Using an association to pass context into a block this way is just one step removed from having a different kind of association that knows more context that you'd like passed in. In other words, it is one way to do the efficient enumeration techniques by enumerating a single object that is configured and evaluated by the block. The code evaluated in the block would need to know to use #key, #value, and any other passed context.

 

Paul Baumann

 

 

 

From: Randy Coulman [mailto:[hidden email]]
Sent: Tuesday, April 29, 2014 15:05
To: Paul Baumann
Cc: VW NC
Subject: Re: [vwnc] [ANN] ExternalEnumeration

 

Paul,

 

I didn't provide very much detail in the announcement to the list, but if you click through to the blog post, you'll see more details on the motivation for this library.  There are two main cases that I see:

 

1) You have a domain class that acts like a collection, but doesn't inherit from any of the collection classes.  Instead, it wraps an internal collection.  You generally don't want to expose the internal collection directly to client code, because then the client code can modify your internal collection without your class knowing about it.  In this case, your class can expose a #do: method that forwards on to the internal collection.  You can then wrap your object and its do selector in an Enumerator and have access to all of the enumeration protocol, but none of the modification protocol.

 

2) You have a complex data structure that can be enumerated using a #do:-style method.  Wrapping that object in an Enumerator allows you to use the full collection protocol on it.  For example, Behavior>>allSubclassesDo: or VisualComponent>>childrenDo:

 

Enumerator implements the "external iterator" pattern.

 

If I've already got a real collection, I don't need this library.  This is for cases where I have a collection-like object that can be enumerated, but doesn't expose the full enumeration protocol on its own.

 

Your other points are well-taken, and this library could evolve in the future to address them if the need arises.

 

Randy

 

 

On Tue, Apr 29, 2014 at 11:21 AM, Paul Baumann <[hidden email]> wrote:

Hi Randy,

 

Sorry to put you on the spot like this, but what advantage would Enumerator provide?

 

"Using an Enumerator"

#(one two three four five) elements groupedBy: #size

 

                "The standard way"

#(one two three four five) groupedBy: #size

 

The way I read the code it looks like a different way to enumerate that is usually slower, can only enumerate values, and is hard to for application code to configure in a backward-compatible way.

 

The framework would need to be greatly enhanced for it to be capable of providing a performance advantage. Specialized Enumerators would be needed for efficient implementation-specific access. The 'species' argument configures a return type that is inefficiently grown from default size with a writeStream that would then copy #contents to another collection. The writeStream is likely used because it allows any kind of collection to be used, but performance is more important and people generally expect a standard return type for the enumeration method. The framework would need  to be able to pass context as extra arguments through the block if you wanted to try to efficient block enumeration. Enumerator could be considered a step toward that kind of optimization, but it starts from something slower.

 

Consider how a SortedCollection responds from #select:. Enumerator uses the default sort block for #select: (instead of the same sort block preserved through #copyEmpty).  Note that SortedCollection is one kind of class that has a different kind of response for #select: (#copyEmpty) and #collect: (OrderedCollection). Enumerator can be configured for response type, but giving the species alone gives insufficient control over how the response is preconfigured with space or initialized with things like the sort block. OrderedCollection has different return types for #select: and #collect:. It is nice that users can figure the type of response, but existing code would likely get broken if application code is now expected to configure the enumerator in a backward-compatible way for response type.

 

Give examples of how Enumerator would enumerate with methods like #doWithIndex:, #keysAndValuesDo:, #associationsDo:, #keysDo:, and #valuesAndCountsDo:. There is only support for one-argument evaluations. The key of each object enumerated is often important for filling the response.

The Enumerator class should probably be an abstract superclass of an ValueEnumerator that has other siblings like KeyValueEnumerator and BasicEnumerator. Enumerator should probably allow a response to be configured rather than just the species of the response. The #elements method should provide an efficient Enumerator that is configured by default to answer the same kind of response as without sending #elements. Other subclasses of Enumerator would be to account for implementation-specific performance. Fast forward a few iterations and then it becomes evident that the most efficient and maintainable code involves some core extensions to the Collection classes rather than a separate parallel hierarchy of classes. The Enumerator pattern has some in common with the separation that IBM had originally built into ENVY/400 (pre-IBMST and VisualAge) collections. IBM replaced that design for better performance.

 

I'd want Enumerator to provide some form of real advantage right out of the box. That is possible, but the advantage of this implementation appears more theoretical than practical. The only advantage that Enumerator seems to currently provide is extra control over the kind of object that is returned from an enumeration. If that is the only advantage then Enumerator is a slower but slightly more readable alternative to an #inject:into:.

 

Paul Baumann

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Randy Coulman
Sent: Tuesday, April 29, 2014 09:49
To: VW NC
Subject: [vwnc] [ANN] ExternalEnumeration

 

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the <a href="http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Public&#43;Store&#43;Repository" target="_blank"> Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman

 


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.



 

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman



This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
Paul,

I haven't had a chance to use Enumerator for much yet, and I haven't run across a case where I wanted to externally-iterate a collection with both keys and values.  If I find a need for it, I'll keep your suggestions in mind for how to implement it.

Randy


On Tue, Apr 29, 2014 at 1:56 PM, Paul Baumann <[hidden email]> wrote:

Yes. I see that Enumerator would have value for use by non-collection classes as an Iterator of a Composite Pattern or Chain Of Responsibility design pattern. Thanks for pointing that out.

 

Here are ideas you might want to consider for your design...

 

One technique for enumerators to handle enumeration of both keys and values with a one-argument block is to reuse a single instance of an association for keyed collections that don't store associations. Dictionary for example is implemented with associations so you could enumerate those. A KeyValueEnumerator iterator over an Array could create one association that is populated and reused for each execution. An association is not created for each item. The risk of that optimization technique is that the block may use the association argument itself instead of asking for key and value.

 

"This would be valid"

(Dictionary

                at: #one put: 1;

                at: #two put: 2;

                assocationEnum)

                                collect: [:assoc | assoc ].

 

"This would be answer a collection of two references to the same association"

#(one two) assocationEnum

                collect: [:assoc | assoc ].

 

"This is how a KeyValueEnumerator is kind of like a keysAndValuesDo:/doWithIndex: "

| dict |

dict := Dictionary new.

#(one two) assocationEnum

                do: [:assoc | dict at: assoc value put: assoc key  ].

^dict

 

 

Using an association to pass context into a block this way is just one step removed from having a different kind of association that knows more context that you'd like passed in. In other words, it is one way to do the efficient enumeration techniques by enumerating a single object that is configured and evaluated by the block. The code evaluated in the block would need to know to use #key, #value, and any other passed context.

 

Paul Baumann

 

 

 

From: Randy Coulman [mailto:[hidden email]]
Sent: Tuesday, April 29, 2014 15:05
To: Paul Baumann
Cc: VW NC
Subject: Re: [vwnc] [ANN] ExternalEnumeration

 

Paul,

 

I didn't provide very much detail in the announcement to the list, but if you click through to the blog post, you'll see more details on the motivation for this library.  There are two main cases that I see:

 

1) You have a domain class that acts like a collection, but doesn't inherit from any of the collection classes.  Instead, it wraps an internal collection.  You generally don't want to expose the internal collection directly to client code, because then the client code can modify your internal collection without your class knowing about it.  In this case, your class can expose a #do: method that forwards on to the internal collection.  You can then wrap your object and its do selector in an Enumerator and have access to all of the enumeration protocol, but none of the modification protocol.

 

2) You have a complex data structure that can be enumerated using a #do:-style method.  Wrapping that object in an Enumerator allows you to use the full collection protocol on it.  For example, Behavior>>allSubclassesDo: or VisualComponent>>childrenDo:

 

Enumerator implements the "external iterator" pattern.

 

If I've already got a real collection, I don't need this library.  This is for cases where I have a collection-like object that can be enumerated, but doesn't expose the full enumeration protocol on its own.

 

Your other points are well-taken, and this library could evolve in the future to address them if the need arises.

 

Randy

 

 

On Tue, Apr 29, 2014 at 11:21 AM, Paul Baumann <[hidden email]> wrote:

Hi Randy,

 

Sorry to put you on the spot like this, but what advantage would Enumerator provide?

 

"Using an Enumerator"

#(one two three four five) elements groupedBy: #size

 

                "The standard way"

#(one two three four five) groupedBy: #size

 

The way I read the code it looks like a different way to enumerate that is usually slower, can only enumerate values, and is hard to for application code to configure in a backward-compatible way.

 

The framework would need to be greatly enhanced for it to be capable of providing a performance advantage. Specialized Enumerators would be needed for efficient implementation-specific access. The 'species' argument configures a return type that is inefficiently grown from default size with a writeStream that would then copy #contents to another collection. The writeStream is likely used because it allows any kind of collection to be used, but performance is more important and people generally expect a standard return type for the enumeration method. The framework would need  to be able to pass context as extra arguments through the block if you wanted to try to efficient block enumeration. Enumerator could be considered a step toward that kind of optimization, but it starts from something slower.

 

Consider how a SortedCollection responds from #select:. Enumerator uses the default sort block for #select: (instead of the same sort block preserved through #copyEmpty).  Note that SortedCollection is one kind of class that has a different kind of response for #select: (#copyEmpty) and #collect: (OrderedCollection). Enumerator can be configured for response type, but giving the species alone gives insufficient control over how the response is preconfigured with space or initialized with things like the sort block. OrderedCollection has different return types for #select: and #collect:. It is nice that users can figure the type of response, but existing code would likely get broken if application code is now expected to configure the enumerator in a backward-compatible way for response type.

 

Give examples of how Enumerator would enumerate with methods like #doWithIndex:, #keysAndValuesDo:, #associationsDo:, #keysDo:, and #valuesAndCountsDo:. There is only support for one-argument evaluations. The key of each object enumerated is often important for filling the response.

The Enumerator class should probably be an abstract superclass of an ValueEnumerator that has other siblings like KeyValueEnumerator and BasicEnumerator. Enumerator should probably allow a response to be configured rather than just the species of the response. The #elements method should provide an efficient Enumerator that is configured by default to answer the same kind of response as without sending #elements. Other subclasses of Enumerator would be to account for implementation-specific performance. Fast forward a few iterations and then it becomes evident that the most efficient and maintainable code involves some core extensions to the Collection classes rather than a separate parallel hierarchy of classes. The Enumerator pattern has some in common with the separation that IBM had originally built into ENVY/400 (pre-IBMST and VisualAge) collections. IBM replaced that design for better performance.

 

I'd want Enumerator to provide some form of real advantage right out of the box. That is possible, but the advantage of this implementation appears more theoretical than practical. The only advantage that Enumerator seems to currently provide is extra control over the kind of object that is returned from an enumeration. If that is the only advantage then Enumerator is a slower but slightly more readable alternative to an #inject:into:.

 

Paul Baumann

 

 

From: [hidden email] [mailto:[hidden email]] On Behalf Of Randy Coulman
Sent: Tuesday, April 29, 2014 09:49
To: VW NC
Subject: [vwnc] [ANN] ExternalEnumeration

 

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman

 


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.



 

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman



This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.



--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Steffen Märcker
Hi Randy,

I am curios, did you have a look at the Reducers package? It offers  
similar advantages as the external Enumerator approach.

1. rich enumeration on custom types by implementing a single method,  
namely #reduce:init:
2. decoupling source object, enumeration functions and result aggregation

It tackles some of the points Paul pointed out, e.g. key-value-iteration.

An example: Creating a LUT of odd square numbers:

    (((1 to: 10**5) reducer map: #squared) filter: #odd)
       into: Dictionary

or compositional style:

    (Reducer filter: #odd) <> (Reducer map: #squared)
       from: (1 to: 10**5) into: Dictionary

where (...) <> (...) could be assigned to a variable and reused. Another  
advantage is that no intermediate collections are created at all.

Regards,
Steffen


Am 30.04.2014, 01:49 Uhr, schrieb Randy Coulman <[hidden email]>:

> Paul,
>
> I haven't had a chance to use Enumerator for much yet, and I haven't run
> across a case where I wanted to externally-iterate a collection with both
> keys and values.  If I find a need for it, I'll keep your suggestions in
> mind for how to implement it.
>
> Randy
>
>
> On Tue, Apr 29, 2014 at 1:56 PM, Paul Baumann  
> <[hidden email]>wrote:
>
>>  Yes. I see that Enumerator would have value for use by non-collection
>> classes as an Iterator of a Composite Pattern or Chain Of Responsibility
>> design pattern. Thanks for pointing that out.
>>
>>
>>
>> Here are ideas you might want to consider for your design...
>>
>>
>>
>> One technique for enumerators to handle enumeration of both keys and
>> values with a one-argument block is to reuse a single instance of an
>> association for keyed collections that don't store associations.  
>> Dictionary
>> for example is implemented with associations so you could enumerate  
>> those.
>> A KeyValueEnumerator iterator over an Array could create one association
>> that is populated and reused for each execution. An association is not
>> created for each item. The risk of that optimization technique is that  
>> the
>> block may use the association argument itself instead of asking for key  
>> and
>> value.
>>
>>
>>
>> "This would be valid"
>>
>> (Dictionary
>>
>>                 at: #one put: 1;
>>
>>                 at: #two put: 2;
>>
>>                 assocationEnum)
>>
>>                                 collect: [:assoc | assoc ].
>>
>>
>>
>> "This would be answer a collection of two references to the same
>> association"
>>
>> #(one two) assocationEnum
>>
>>                 collect: [:assoc | assoc ].
>>
>>
>>
>> "This is how a KeyValueEnumerator is kind of like a
>> keysAndValuesDo:/doWithIndex: "
>>
>> | dict |
>>
>> dict := Dictionary new.
>>
>> #(one two) assocationEnum
>>
>>                 do: [:assoc | dict at: assoc value put: assoc key  ].
>>
>> ^dict
>>
>>
>>
>>
>>
>> Using an association to pass context into a block this way is just one
>> step removed from having a different kind of association that knows more
>> context that you'd like passed in. In other words, it is one way to do  
>> the
>> efficient enumeration techniques by enumerating a single object that is
>> configured and evaluated by the block. The code evaluated in the block
>> would need to know to use #key, #value, and any other passed context.
>>
>>
>>
>> Paul Baumann
>>
>>
>>
>>
>>
>>
>>
>> *From:* Randy Coulman [mailto:[hidden email]]
>> *Sent:* Tuesday, April 29, 2014 15:05
>> *To:* Paul Baumann
>> *Cc:* VW NC
>> *Subject:* Re: [vwnc] [ANN] ExternalEnumeration
>>
>>
>>
>> Paul,
>>
>>
>>
>> I didn't provide very much detail in the announcement to the list, but  
>> if
>> you click through to the blog post, you'll see more details on the
>> motivation for this library.  There are two main cases that I see:
>>
>>
>>
>> 1) You have a domain class that acts like a collection, but doesn't
>> inherit from any of the collection classes.  Instead, it wraps an  
>> internal
>> collection.  You generally don't want to expose the internal collection
>> directly to client code, because then the client code can modify your
>> internal collection without your class knowing about it.  In this case,
>> your class can expose a #do: method that forwards on to the internal
>> collection.  You can then wrap your object and its do selector in an
>> Enumerator and have access to all of the enumeration protocol, but none  
>> of
>> the modification protocol.
>>
>>
>>
>> 2) You have a complex data structure that can be enumerated using a
>> #do:-style method.  Wrapping that object in an Enumerator allows you to  
>> use
>> the full collection protocol on it.  For example,
>> Behavior>>allSubclassesDo: or VisualComponent>>childrenDo:
>>
>>
>>
>> Enumerator implements the "external iterator" pattern.
>>
>>
>>
>> If I've already got a real collection, I don't need this library.  This  
>> is
>> for cases where I have a collection-like object that can be enumerated,  
>> but
>> doesn't expose the full enumeration protocol on its own.
>>
>>
>>
>> Your other points are well-taken, and this library could evolve in the
>> future to address them if the need arises.
>>
>>
>>
>> Randy
>>
>>
>>
>>
>>
>> On Tue, Apr 29, 2014 at 11:21 AM, Paul Baumann <[hidden email]>
>> wrote:
>>
>> Hi Randy,
>>
>>
>>
>> Sorry to put you on the spot like this, but what advantage would
>> Enumerator provide?
>>
>>
>>
>> "Using an Enumerator"
>>
>> #(one two three four five) elements groupedBy: #size
>>
>>
>>
>>                 "The standard way"
>>
>> #(one two three four five) groupedBy: #size
>>
>>
>>
>> The way I read the code it looks like a different way to enumerate that  
>> is
>> usually slower, can only enumerate values, and is hard to for  
>> application
>> code to configure in a backward-compatible way.
>>
>>
>>
>> The framework would need to be greatly enhanced for it to be capable of
>> providing a performance advantage. Specialized Enumerators would be  
>> needed
>> for efficient implementation-specific access. The 'species' argument
>> configures a return type that is inefficiently grown from default size  
>> with
>> a writeStream that would then copy #contents to another collection. The
>> writeStream is likely used because it allows any kind of collection to  
>> be
>> used, but performance is more important and people generally expect a
>> standard return type for the enumeration method. The framework would  
>> need
>> to be able to pass context as extra arguments through the block if you
>> wanted to try to efficient block enumeration. Enumerator could be
>> considered a step toward that kind of optimization, but it starts from
>> something slower.
>>
>>
>>
>> Consider how a SortedCollection responds from #select:. Enumerator uses
>> the default sort block for #select: (instead of the same sort block
>> preserved through #copyEmpty).  Note that SortedCollection is one kind  
>> of
>> class that has a different kind of response for #select: (#copyEmpty)  
>> and
>> #collect: (OrderedCollection). Enumerator can be configured for response
>> type, but giving the species alone gives insufficient control over how  
>> the
>> response is preconfigured with space or initialized with things like the
>> sort block. OrderedCollection has different return types for #select:  
>> and
>> #collect:. It is nice that users can figure the type of response, but
>> existing code would likely get broken if application code is now  
>> expected
>> to configure the enumerator in a backward-compatible way for response  
>> type.
>>
>>
>>
>> Give examples of how Enumerator would enumerate with methods like
>> #doWithIndex:, #keysAndValuesDo:, #associationsDo:, #keysDo:, and
>> #valuesAndCountsDo:. There is only support for one-argument evaluations.
>> The key of each object enumerated is often important for filling the
>> response.
>>
>> The Enumerator class should probably be an abstract superclass of an
>> ValueEnumerator that has other siblings like KeyValueEnumerator and
>> BasicEnumerator. Enumerator should probably allow a response to be
>> configured rather than just the species of the response. The #elements
>> method should provide an efficient Enumerator that is configured by  
>> default
>> to answer the same kind of response as without sending #elements. Other
>> subclasses of Enumerator would be to account for implementation-specific
>> performance. Fast forward a few iterations and then it becomes evident  
>> that
>> the most efficient and maintainable code involves some core extensions  
>> to
>> the Collection classes rather than a separate parallel hierarchy of
>> classes. The Enumerator pattern has some in common with the separation  
>> that
>> IBM had originally built into ENVY/400 (pre-IBMST and VisualAge)
>> collections. IBM replaced that design for better performance.
>>
>>
>>
>> I'd want Enumerator to provide some form of real advantage right out of
>> the box. That is possible, but the advantage of this implementation  
>> appears
>> more theoretical than practical. The only advantage that Enumerator  
>> seems
>> to currently provide is extra control over the kind of object that is
>> returned from an enumeration. If that is the only advantage then  
>> Enumerator
>> is a slower but slightly more readable alternative to an #inject:into:.
>>
>>
>>
>> Paul Baumann
>>
>>
>>
>>
>>
>> *From:* [hidden email] [mailto:[hidden email]] *On
>> Behalf Of *Randy Coulman
>> *Sent:* Tuesday, April 29, 2014 09:49
>> *To:* VW NC
>> *Subject:* [vwnc] [ANN] ExternalEnumeration
>>
>>
>>
>> Hello,
>>
>> I have just released  
>> ExternalEnumeration<https://github.com/randycoulman/ExternalEnumeration>
>> .
>>
>> It defines Enumerator, an object that takes a target object and a  
>> do:selector and provides the full, rich enumeration API that Smalltalk
>> programmers expect.
>>
>> Enumerator is an implementation of the External  
>> Iterator<http://c2.com/cgi/wiki?ExternalIterator>pattern.
>>
>> If you have a collection-like domain object, or a complex data  
>> structure,
>> you can implement a do: method for it, and then wrap it with an  
>> Enumeratorto give client code access to the full enumeration API.
>>
>> For example:
>>
>> subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
>>
>> (subclasses groupedBy: #superclass) inspect.
>>
>> ExternalEnumeration is available in the Cincom Public Store  
>> Repository<http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Public+Store+Repository>.
>> A snapshot of the current version is also on  
>> GitHub<https://github.com/randycoulman/ExternalEnumeration>
>> .
>>
>> The full release announcement is on my blog, Courageous  
>> Software<http://randycoulman.com/blog/2014/04/29/external-enumerators-in-smalltalk/>
>>
>> Randy
>>
>> --
>> Randy Coulman
>>
>> Email: [hidden email]
>>
>> Home: http://randycoulman.com
>>
>> Twitter: @randycoulman      GitHub: randycoulman
>>
>>
>>  ------------------------------
>>
>> This message may contain confidential information and is intended for
>> specific recipients unless explicitly noted otherwise. If you have  
>> reason
>> to believe you are not an intended recipient of this message, please  
>> delete
>> it and notify the sender. This message may not represent the opinion of
>> IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates,  
>> and
>> does not constitute a contract or guarantee. Unencrypted electronic  
>> mail is
>> not secure and the recipient of this message is expected to provide
>> safeguards from viruses and pursue alternate means of communication  
>> where
>> privacy or a binding message is desired.
>>
>>
>>
>>
>>
>> --
>> Randy Coulman
>>
>> Email: [hidden email]
>>
>> Home: http://randycoulman.com
>>
>> Twitter: @randycoulman      GitHub: randycoulman
>>
>> ------------------------------
>> This message may contain confidential information and is intended for
>> specific recipients unless explicitly noted otherwise. If you have  
>> reason
>> to believe you are not an intended recipient of this message, please  
>> delete
>> it and notify the sender. This message may not represent the opinion of
>> IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates,  
>> and
>> does not constitute a contract or guarantee. Unencrypted electronic  
>> mail is
>> not secure and the recipient of this message is expected to provide
>> safeguards from viruses and pursue alternate means of communication  
>> where
>> privacy or a binding message is desired.
>>
>
>
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Reinout Heeck-2
In reply to this post by Randy Coulman
On 4/29/2014 3:49 PM, Randy Coulman wrote:

Enumerator is an implementation of the External Iterator pattern.



I would expect an external iterator to implement ReadStream protocol (which overlaps for a large part with collection protocol).
The page you link to seems to say so in its very first sentence: "An ExternalIterator is an object that represents a location within a collection."


The Wikipedia article even makes it explicit:
"Note that an iterator performs traversal and also gives access to data elements in a container, but does not perform iteration (i.e., not without some significant liberty taken with that concept or with trivial use of the terminology)." http://en.wikipedia.org/wiki/Iterator

I write this not because there is anything wrong with your code but because I object to overloading pattern names.
One of the motivations to introduce patterns was to create a common vocabulary between programmers, gradually stretching 'External Iterator' to mean any/all objects that traverse and are external to the collection goes against that goal. (Particularly if they don't implement #next ;-)


Reinout
-------


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Reinout Heeck-2
In reply to this post by Randy Coulman
Hi Randy,

I see that that Enumerator does not subclass from Collection.
Is there a particular reason for this? I would expect collection-like classes to automatically inherit my collection extensions....

One could even make a case for multiple Enumerator classes, for example one that inherits from Collection and one that inherits from SequenceableCollection.
The latter would automagically support my extensions that use #first, #indexOf: etc.


Reinout
-------




On 4/29/2014 3:49 PM, Randy Coulman wrote:

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
In reply to this post by Steffen Märcker
HI, Steffen,

I wasn't aware of (or had forgotten about) the Reducers package.  I'll have to take a look at it.

Randy


On Thu, May 1, 2014 at 1:31 AM, Steffen Märcker <[hidden email]> wrote:
Hi Randy,

I am curios, did you have a look at the Reducers package? It offers similar advantages as the external Enumerator approach.

1. rich enumeration on custom types by implementing a single method, namely #reduce:init:
2. decoupling source object, enumeration functions and result aggregation

It tackles some of the points Paul pointed out, e.g. key-value-iteration.

An example: Creating a LUT of odd square numbers:

   (((1 to: 10**5) reducer map: #squared) filter: #odd)
      into: Dictionary

or compositional style:

   (Reducer filter: #odd) <> (Reducer map: #squared)
      from: (1 to: 10**5) into: Dictionary

where (...) <> (...) could be assigned to a variable and reused. Another advantage is that no intermediate collections are created at all.

Regards,
Steffen


Am 30.04.2014, 01:49 Uhr, schrieb Randy Coulman <[hidden email]>:

Paul,

I haven't had a chance to use Enumerator for much yet, and I haven't run
across a case where I wanted to externally-iterate a collection with both
keys and values.  If I find a need for it, I'll keep your suggestions in
mind for how to implement it.

Randy


On Tue, Apr 29, 2014 at 1:56 PM, Paul Baumann <[hidden email]>wrote:

 Yes. I see that Enumerator would have value for use by non-collection
classes as an Iterator of a Composite Pattern or Chain Of Responsibility
design pattern. Thanks for pointing that out.



Here are ideas you might want to consider for your design...



One technique for enumerators to handle enumeration of both keys and
values with a one-argument block is to reuse a single instance of an
association for keyed collections that don't store associations. Dictionary
for example is implemented with associations so you could enumerate those.
A KeyValueEnumerator iterator over an Array could create one association
that is populated and reused for each execution. An association is not
created for each item. The risk of that optimization technique is that the
block may use the association argument itself instead of asking for key and
value.



"This would be valid"

(Dictionary

                at: #one put: 1;

                at: #two put: 2;

                assocationEnum)

                                collect: [:assoc | assoc ].



"This would be answer a collection of two references to the same
association"

#(one two) assocationEnum

                collect: [:assoc | assoc ].



"This is how a KeyValueEnumerator is kind of like a
keysAndValuesDo:/doWithIndex: "

| dict |

dict := Dictionary new.

#(one two) assocationEnum

                do: [:assoc | dict at: assoc value put: assoc key  ].

^dict





Using an association to pass context into a block this way is just one
step removed from having a different kind of association that knows more
context that you'd like passed in. In other words, it is one way to do the
efficient enumeration techniques by enumerating a single object that is
configured and evaluated by the block. The code evaluated in the block
would need to know to use #key, #value, and any other passed context.



Paul Baumann







*From:* Randy Coulman [mailto:[hidden email]]
*Sent:* Tuesday, April 29, 2014 15:05
*To:* Paul Baumann
*Cc:* VW NC
*Subject:* Re: [vwnc] [ANN] ExternalEnumeration




Paul,



I didn't provide very much detail in the announcement to the list, but if
you click through to the blog post, you'll see more details on the
motivation for this library.  There are two main cases that I see:



1) You have a domain class that acts like a collection, but doesn't
inherit from any of the collection classes.  Instead, it wraps an internal
collection.  You generally don't want to expose the internal collection
directly to client code, because then the client code can modify your
internal collection without your class knowing about it.  In this case,
your class can expose a #do: method that forwards on to the internal
collection.  You can then wrap your object and its do selector in an
Enumerator and have access to all of the enumeration protocol, but none of
the modification protocol.



2) You have a complex data structure that can be enumerated using a
#do:-style method.  Wrapping that object in an Enumerator allows you to use
the full collection protocol on it.  For example,
Behavior>>allSubclassesDo: or VisualComponent>>childrenDo:



Enumerator implements the "external iterator" pattern.



If I've already got a real collection, I don't need this library.  This is
for cases where I have a collection-like object that can be enumerated, but
doesn't expose the full enumeration protocol on its own.



Your other points are well-taken, and this library could evolve in the
future to address them if the need arises.



Randy





On Tue, Apr 29, 2014 at 11:21 AM, Paul Baumann <[hidden email]>
wrote:

Hi Randy,



Sorry to put you on the spot like this, but what advantage would
Enumerator provide?



"Using an Enumerator"

#(one two three four five) elements groupedBy: #size



                "The standard way"

#(one two three four five) groupedBy: #size



The way I read the code it looks like a different way to enumerate that is
usually slower, can only enumerate values, and is hard to for application
code to configure in a backward-compatible way.



The framework would need to be greatly enhanced for it to be capable of
providing a performance advantage. Specialized Enumerators would be needed
for efficient implementation-specific access. The 'species' argument
configures a return type that is inefficiently grown from default size with
a writeStream that would then copy #contents to another collection. The
writeStream is likely used because it allows any kind of collection to be
used, but performance is more important and people generally expect a
standard return type for the enumeration method. The framework would need
to be able to pass context as extra arguments through the block if you
wanted to try to efficient block enumeration. Enumerator could be
considered a step toward that kind of optimization, but it starts from
something slower.



Consider how a SortedCollection responds from #select:. Enumerator uses
the default sort block for #select: (instead of the same sort block
preserved through #copyEmpty).  Note that SortedCollection is one kind of
class that has a different kind of response for #select: (#copyEmpty) and
#collect: (OrderedCollection). Enumerator can be configured for response
type, but giving the species alone gives insufficient control over how the
response is preconfigured with space or initialized with things like the
sort block. OrderedCollection has different return types for #select: and
#collect:. It is nice that users can figure the type of response, but
existing code would likely get broken if application code is now expected
to configure the enumerator in a backward-compatible way for response type.



Give examples of how Enumerator would enumerate with methods like
#doWithIndex:, #keysAndValuesDo:, #associationsDo:, #keysDo:, and
#valuesAndCountsDo:. There is only support for one-argument evaluations.
The key of each object enumerated is often important for filling the
response.

The Enumerator class should probably be an abstract superclass of an
ValueEnumerator that has other siblings like KeyValueEnumerator and
BasicEnumerator. Enumerator should probably allow a response to be
configured rather than just the species of the response. The #elements
method should provide an efficient Enumerator that is configured by default
to answer the same kind of response as without sending #elements. Other
subclasses of Enumerator would be to account for implementation-specific
performance. Fast forward a few iterations and then it becomes evident that
the most efficient and maintainable code involves some core extensions to
the Collection classes rather than a separate parallel hierarchy of
classes. The Enumerator pattern has some in common with the separation that
IBM had originally built into ENVY/400 (pre-IBMST and VisualAge)
collections. IBM replaced that design for better performance.



I'd want Enumerator to provide some form of real advantage right out of
the box. That is possible, but the advantage of this implementation appears
more theoretical than practical. The only advantage that Enumerator seems
to currently provide is extra control over the kind of object that is
returned from an enumeration. If that is the only advantage then Enumerator
is a slower but slightly more readable alternative to an #inject:into:.



Paul Baumann





*From:* [hidden email] [mailto:[hidden email]] *On
Behalf Of *Randy Coulman
*Sent:* Tuesday, April 29, 2014 09:49
*To:* VW NC
*Subject:* [vwnc] [ANN] ExternalEnumeration



Hello,

I have just released ExternalEnumeration<https://github.com/randycoulman/ExternalEnumeration>

.

It defines Enumerator, an object that takes a target object and a do:selector and provides the full, rich enumeration API that Smalltalk
programmers expect.

Enumerator is an implementation of the External Iterator<http://c2.com/cgi/wiki?ExternalIterator>pattern.


If you have a collection-like domain object, or a complex data structure,
you can implement a do: method for it, and then wrap it with an Enumeratorto give client code access to the full enumeration API.


For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.

(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository<http://www.cincomsmalltalk.com/CincomSmalltalkWiki/Public+Store+Repository>.
A snapshot of the current version is also on GitHub<https://github.com/randycoulman/ExternalEnumeration>
.

The full release announcement is on my blog, Courageous Software<http://randycoulman.com/blog/2014/04/29/external-enumerators-in-smalltalk/>


Randy

--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman


 ------------------------------


This message may contain confidential information and is intended for
specific recipients unless explicitly noted otherwise. If you have reason
to believe you are not an intended recipient of this message, please delete
it and notify the sender. This message may not represent the opinion of
IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and
does not constitute a contract or guarantee. Unencrypted electronic mail is
not secure and the recipient of this message is expected to provide
safeguards from viruses and pursue alternate means of communication where
privacy or a binding message is desired.





--
Randy Coulman

Email: [hidden email]

Home: http://randycoulman.com

Twitter: @randycoulman      GitHub: randycoulman

------------------------------

This message may contain confidential information and is intended for
specific recipients unless explicitly noted otherwise. If you have reason
to believe you are not an intended recipient of this message, please delete
it and notify the sender. This message may not represent the opinion of
IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and
does not constitute a contract or guarantee. Unencrypted electronic mail is
not secure and the recipient of this message is expected to provide
safeguards from viruses and pursue alternate means of communication where
privacy or a binding message is desired.






--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
In reply to this post by Reinout Heeck-2
Hi, Reinout,

Supporting ReadStream protocol would be an interesting extension here; I also wondered about integrating with XStreams.

I hadn't read the Wikipedia article; I find it odd that it says that "an Iterator ... does not perform iteration". 

Randy


On Thu, May 1, 2014 at 3:08 AM, Reinout Heeck <[hidden email]> wrote:
On 4/29/2014 3:49 PM, Randy Coulman wrote:

Enumerator is an implementation of the External Iterator pattern.



I would expect an external iterator to implement ReadStream protocol (which overlaps for a large part with collection protocol).
The page you link to seems to say so in its very first sentence: "An ExternalIterator is an object that represents a location within a collection."


The Wikipedia article even makes it explicit:
"Note that an iterator performs traversal and also gives access to data elements in a container, but does not perform iteration (i.e., not without some significant liberty taken with that concept or with trivial use of the terminology)." http://en.wikipedia.org/wiki/Iterator

I write this not because there is anything wrong with your code but because I object to overloading pattern names.
One of the motivations to introduce patterns was to create a common vocabulary between programmers, gradually stretching 'External Iterator' to mean any/all objects that traverse and are external to the collection goes against that goal. (Particularly if they don't implement #next ;-)


Reinout
-------


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc




--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
In reply to this post by Reinout Heeck-2
Reinout,

Enumerator doesn't inherit from Collection because it isn't a collection.  It implements a small subset of the behavior of a Collection.  It doesn't support adding and removing elements, etc.

I don't like inheriting from a class and then overriding half or more of the inherited API with #shouldNotImplement sends - I think that's a horrible misuse of inheritance.

In its current form Enumerator represents a pretty generic enumeration interface; I could see specializing it for different Collection types to provide a richer API when appropriate, or to do key/value iteration as Paul pointed out.

This is a first version and is really an experiment with the concept.  I wanted to play with it for myself and thought I'd share it with the community as well.  I'm happy to consider enhancements and improvements from others ("pull requests" as they say in GitHub-land), and as I use it and find holes, I'll improve it as well.

Randy


On Thu, May 1, 2014 at 4:10 AM, Reinout Heeck <[hidden email]> wrote:
Hi Randy,

I see that that Enumerator does not subclass from Collection.
Is there a particular reason for this? I would expect collection-like classes to automatically inherit my collection extensions....

One could even make a case for multiple Enumerator classes, for example one that inherits from Collection and one that inherits from SequenceableCollection.
The latter would automagically support my extensions that use #first, #indexOf: etc.


Reinout
-------





On 4/29/2014 3:49 PM, Randy Coulman wrote:

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc




--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Ralph Johnson



On Thu, May 1, 2014 at 9:34 AM, Randy Coulman <[hidden email]> wrote:
Enumerator doesn't inherit from Collection because it isn't a collection.  It implements a small subset of the behavior of a Collection.  It doesn't support adding and removing elements, etc.

Array doesn't support adding and removing elements, either.   Of the standard SequencialCollections, only OrderedCollection does that. 

I don't like inheriting from a class and then overriding half or more of the inherited API with #shouldNotImplement sends - I think that's a horrible misuse of inheritance.

But it happens in Collection.   This is one of the arguments for multiple inheritance; Collection would not have to have inherited methods that it can't use if it were broken into classes and then you could use multiple inheritance to combine them.   Of course, multiple inheritance introduces its own problems.

Collection doesn't really have that much that has to be overridden.   Typically people don't use #shouldNotImplement for the regular template methods but just for the abstract methods that they call.  So, Array would have it for #add: and #remove: but not for #addAll: and #removeAll:.   I haven't looked at VisualWorks for awhile so I don't know how it does it now, but it used to do it that way.

I agree with you that overriding inherited methods with #shouldNotImplement is a design smell.   But sometimes it is the best alternative.

-Ralph Johnson

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Reinout Heeck-2
In reply to this post by Randy Coulman
On 5/1/2014 4:34 PM, Randy Coulman wrote:
Reinout,

Enumerator doesn't inherit from Collection because it isn't a collection.  It implements a small subset of the behavior of a Collection.  It doesn't support adding and removing elements, etc.

Well strictly taken Collection isn't a collection either, it is an abstraction that seems to fit your Enumerator.


I don't like inheriting from a class and then overriding half or more of the inherited API with #shouldNotImplement sends - I think that's a horrible misuse of inheritance.

I understand the sentiment, but that is just what it is.

In practice you are undermining re-use of my code, you are guiding me to either stop using my extension methods (give up polymorphism) or to *copy* them to your hierarchy with all the problems that copying introduce.

As Ralph said it is just the way it is done in VW, and you don't need to override half of the API to make it work.



Cheers,

Reinout
-------

In its current form Enumerator represents a pretty generic enumeration interface; I could see specializing it for different Collection types to provide a richer API when appropriate, or to do key/value iteration as Paul pointed out.

This is a first version and is really an experiment with the concept.  I wanted to play with it for myself and thought I'd share it with the community as well.  I'm happy to consider enhancements and improvements from others ("pull requests" as they say in GitHub-land), and as I use it and find holes, I'll improve it as well.

Randy


On Thu, May 1, 2014 at 4:10 AM, Reinout Heeck <[hidden email]> wrote:
Hi Randy,

I see that that Enumerator does not subclass from Collection.
Is there a particular reason for this? I would expect collection-like classes to automatically inherit my collection extensions....

One could even make a case for multiple Enumerator classes, for example one that inherits from Collection and one that inherits from SequenceableCollection.
The latter would automagically support my extensions that use #first, #indexOf: etc.


Reinout
-------





On 4/29/2014 3:49 PM, Randy Coulman wrote:

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc




--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
In reply to this post by Ralph Johnson
You make a good point, Ralph.  This is where I really like Ruby's mixins.  In Ruby, you can implement the :each method (equivalent to Smalltalk's #do:), include the Enumerable mixin, and get all of the collection behavior for free, including anyone's extensions to Enumerable.

I think it would be possible with Traits in Smalltalk, but I haven't really played with them, so I'm not sure.

I'll re-evaluate my decision to not inherit from Collection based on the feedback I'm getting.

Randy


On Thu, May 1, 2014 at 8:05 AM, Ralph Johnson <[hidden email]> wrote:



On Thu, May 1, 2014 at 9:34 AM, Randy Coulman <[hidden email]> wrote:
Enumerator doesn't inherit from Collection because it isn't a collection.  It implements a small subset of the behavior of a Collection.  It doesn't support adding and removing elements, etc.

Array doesn't support adding and removing elements, either.   Of the standard SequencialCollections, only OrderedCollection does that. 

I don't like inheriting from a class and then overriding half or more of the inherited API with #shouldNotImplement sends - I think that's a horrible misuse of inheritance.

But it happens in Collection.   This is one of the arguments for multiple inheritance; Collection would not have to have inherited methods that it can't use if it were broken into classes and then you could use multiple inheritance to combine them.   Of course, multiple inheritance introduces its own problems.

Collection doesn't really have that much that has to be overridden.   Typically people don't use #shouldNotImplement for the regular template methods but just for the abstract methods that they call.  So, Array would have it for #add: and #remove: but not for #addAll: and #removeAll:.   I haven't looked at VisualWorks for awhile so I don't know how it does it now, but it used to do it that way.

I agree with you that overriding inherited methods with #shouldNotImplement is a design smell.   But sometimes it is the best alternative.

-Ralph Johnson



--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
In reply to this post by Reinout Heeck-2
Reinout,

Based on Ralph's and your feedback, I'll reconsider my decision about not inheriting from Collection.

Randy


On Thu, May 1, 2014 at 9:16 AM, Reinout Heeck <[hidden email]> wrote:
On 5/1/2014 4:34 PM, Randy Coulman wrote:
Reinout,

Enumerator doesn't inherit from Collection because it isn't a collection.  It implements a small subset of the behavior of a Collection.  It doesn't support adding and removing elements, etc.

Well strictly taken Collection isn't a collection either, it is an abstraction that seems to fit your Enumerator.



I don't like inheriting from a class and then overriding half or more of the inherited API with #shouldNotImplement sends - I think that's a horrible misuse of inheritance.

I understand the sentiment, but that is just what it is.

In practice you are undermining re-use of my code, you are guiding me to either stop using my extension methods (give up polymorphism) or to *copy* them to your hierarchy with all the problems that copying introduce.

As Ralph said it is just the way it is done in VW, and you don't need to override half of the API to make it work.



Cheers,

Reinout
-------


In its current form Enumerator represents a pretty generic enumeration interface; I could see specializing it for different Collection types to provide a richer API when appropriate, or to do key/value iteration as Paul pointed out.

This is a first version and is really an experiment with the concept.  I wanted to play with it for myself and thought I'd share it with the community as well.  I'm happy to consider enhancements and improvements from others ("pull requests" as they say in GitHub-land), and as I use it and find holes, I'll improve it as well.

Randy


On Thu, May 1, 2014 at 4:10 AM, Reinout Heeck <[hidden email]> wrote:
Hi Randy,

I see that that Enumerator does not subclass from Collection.
Is there a particular reason for this? I would expect collection-like classes to automatically inherit my collection extensions....

One could even make a case for multiple Enumerator classes, for example one that inherits from Collection and one that inherits from SequenceableCollection.
The latter would automagically support my extensions that use #first, #indexOf: etc.


Reinout
-------





On 4/29/2014 3:49 PM, Randy Coulman wrote:

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc




--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc




--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Stéphane Ducasse
In reply to this post by Randy Coulman
Hi randy

I checked the web site and I did not see that you can have next.
Because if I remember correctly what is nice with the iterator patten is that you can control the enumeration (not like in do).
So we could have an iterator walking a big structure and stopping at specific spots and continue by explicitly invoking 
next. 

Stef

On 29 Apr 2014, at 15:49, Randy Coulman <[hidden email]> wrote:

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Stéphane Ducasse
In reply to this post by Reinout Heeck-2
+1 

I would expect an external iterator to implement ReadStream protocol (which overlaps for a large part with collection protocol).
The page you link to seems to say so in its very first sentence: "An ExternalIterator is an object that represents a location within a collection."


The Wikipedia article even makes it explicit:
"Note that an iterator performs traversal and also gives access to data elements in a container, but does not perform iteration (i.e., not without some significant liberty taken with that concept or with trivial use of the terminology)." http://en.wikipedia.org/wiki/Iterator

I write this not because there is anything wrong with your code but because I object to overloading pattern names.
One of the motivations to introduce patterns was to create a common vocabulary between programmers, gradually stretching 'External Iterator' to mean any/all objects that traverse and are external to the collection goes against that goal. (Particularly if they don't implement #next ;-)


Reinout
-------

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc


_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] ExternalEnumeration

Randy Coulman
In reply to this post by Stéphane Ducasse
Hi, Stephane,

No, I don't have a next method yet, though it was suggested earlier in this thread.  I'll likely look at providing ReadStream-like semantics at some point.

Randy


On Fri, May 2, 2014 at 10:15 AM, Stéphane Ducasse <[hidden email]> wrote:
Hi randy

I checked the web site and I did not see that you can have next.
Because if I remember correctly what is nice with the iterator patten is that you can control the enumeration (not like in do).
So we could have an iterator walking a big structure and stopping at specific spots and continue by explicitly invoking 
next. 

Stef

On 29 Apr 2014, at 15:49, Randy Coulman <[hidden email]> wrote:

Hello,

I have just released ExternalEnumeration.

It defines Enumerator, an object that takes a target object and a do: selector and provides the full, rich enumeration API that Smalltalk programmers expect.

Enumerator is an implementation of the External Iterator pattern.

If you have a collection-like domain object, or a complex data structure, you can implement a do: method for it, and then wrap it with an Enumerator to give client code access to the full enumeration API.

For example:

subclasses := Enumerator on: Collection selector: #allSubclassesDo:.
(subclasses groupedBy: #superclass) inspect.

ExternalEnumeration is available in the Cincom Public Store Repository. A snapshot of the current version is also on GitHub.

The full release announcement is on my blog, Courageous Software

Randy

--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman
_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc




--
Randy Coulman
Email: [hidden email]
Home: http://randycoulman.com
Twitter: @randycoulman      GitHub: randycoulman

_______________________________________________
vwnc mailing list
[hidden email]
http://lists.cs.uiuc.edu/mailman/listinfo/vwnc