[vwnc] Feedback on New Prerequisite Engine

Some time back (I can’t find it in my email) someone posted a note about the new prerequisite engine.  I’ve been trying to use it, and wanted to provide some feedback.

The Good News:

1. It’s very fast – so fast that it’s done almost before you remove your finger from the mouse.  That’s an amazing improvement.
2. For the most part, it’s accurate.  Combine that with its speed and this is a very useful tool indeed.

The Not-So-Good News:

1. I think you need to be able to remove any prerequisite because sometimes the derivation thinks there’s a dependency when there really isn’t.  For example, we have a lot of GemStone and Opentalk client and server code in our development image.  A dependency may be “computed” between a client code package sending a message to the server code package.  This is not a valid dependency because of the forwarding nature of proxies. 
2. Prerequisites that are determined solely by literal bindings (specifically, #{…}), should be ignored or mark the dependency as optional.  That’s because literal bindings tend to be weak references used to check whether something is loaded before actually using the particular object.  I’m not sure, but it may mean that the derivation of a dependency may need several possible states which “escalate” to the strongest type of reference.  This may complicate message send derivations above.  In fact, should message send derivations, even if they are only found in a single package, be considered “weak” (optional) dependencies?
3. Duplicate removal appears to be incomplete or inappropriately depth-limited.  We have a number of packages that override or extend Opentalk-HTTP or -SOAP, but the computed dependencies show them, plus -Core, -Core-support, and -Prerequisites.  Since -SOAP requires -HTTP which requires the other, only the top-most should be left after duplicate removal.
4. The “reduce to bundles” option could use some secondary switches.  If you turn on the reduction, you get the bundle that is the most “inclusive.”  What I think you really want is the one that has the most “containment.”  We have a top-level development bundle that loads everything by prerequisite, having no contents, yet when we turn on this switch, universally our dependencies compute to this one bundle rather than the actual containing bundle(s).  I’d like to suggest the following:
1. I think bundle dependencies should only resolve to those that actually contain dependent packages and never to those that might reference them through other inclusion.  If you want to take into account inclusion, that should be a separate option.  I think this is independent of but related to duplicate removal.
2. I think you need to be able to select whether a computed prerequisite to a bundle is the nearest (least inclusive) or farthest (most inclusive) bundle that actually contains a prerequisite package.  This would affect only bundles with depth > 1 (that is, bundles of bundles of arbitrary depth).
3. For this last switch, I think you need to be able to set it differently for package or for bundle prerequisite computation.  For packages, I usually want things to reference other packages, but for bundles, I usually want things to reference bundles.  There are cases where I want things to reference peers of the same depth.
4. Because the low-level computation is to packages, I think you need a switch to control how computation works in the presence of parcels that have bundle structure, or at least be able to “mark” certain parcels as being “atomic” – that is, resolution always is to the top-most bundle.  This would support, for example, loading the GemStone client code, which is three parcels [bundles] rather than 20-odd packages.  Because they’re loaded from a parcel, you can’t even find the individual packages to load unless you do reference the bundle.
5. If these aren’t feasible, and even if they are, I think it would be nice to be able to select various computed prerequisites and have a menu item that would replace them with their [common] parent containing bundle or the top-most including bundle.

Great job!

Cheers!

One more item.

If you have a package inside a bundle and you compute it’s prerequisites with “reduce to bundle” enabled, then the bundle(s) containing the package should be eliminated from the set of possible bundles available as a prerequisite.  After all, you don’t want to compute to a circular dependency.  I’m not sure if it does this or not.