ConfigurationOfX project

Hi!

I was wondering why #project defined on the class side of each configuration does not use the singleton pattern?
Executing "ConfigurationOfMetacello project currentVersion" is deadly slow. In #currentVersion there is a cache mechanism (which I do not fully understand). But it does not help getting "ConfigurationOfMetacello project currentVersion" quickly executed.

Any plan to implement the singleton pattern?

Cheers,
Alexandre

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Hi Alex,
I think those problems are solved with Gofer Project Loader, take a look at it...

Cheers,
Esteban, from Venice :)

El 23/09/2010, a las 8:31p.m., Alexandre Bergel escribió:

Alexandre Bergel wrote:
Alexandre,

Metacello is still in the "get it right" phase so I am intentionally
avoiding caching anything. There _are_ caching mechanisms in Metacello,
but they are dynamic caches and are dropped at the end of execution
(#currentVersion is one of the places where dynamic caching is used).

To cache correctly, Metacello would need events that are triggered when
new versions of mcz files are loaded/unloaded and at the moment I don't
believe that these events exist.

I think that tools could/should do caching. They need to worry about
invalidating the caches, but a mistake in caching at the tool level
wouldn't cause Metacello to load the wrong packages...

Dale

Ok, I get the point, even though I would have introduced caches since the beginning. I tried to insert some, but running the tests takes ages... Hard to see if I brake something or not...

Alexandre


On 23 Sep 2010, at 17:30, Dale Henrichs wrote:

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Alexandre Bergel wrote:
> Ok, I get the point, even though I would have introduced caches since the beginning. I tried to insert some, but running the tests takes ages... Hard to see if I brake something or not...
>
> Alexandre
>

Alexandre,

If you are patient, the tests do eventually finish:)

I've found that to reproduce the types of errors that I've run into the
only sure way was to actually load _real_ packages, which means that the
bulk of the tests are loading and unloading multiple packages (including
configuration packages) for each individual test ... I don't like how
slow it is either, but I'd rather have tests that I trust...

#currentVersion is probably the slowest operation in all of Metacello,
but the #currentVersion _must_ be calculated correctly otherwise all is
lost:)

I assume that you are concerned about caching because of the tool that
you and Simone are working on? There are tricks that you can play at the
tool level that might make things run faster (although the initial
calculation must be done to warm the cache) including stashing results
away between tool invocations ... at the tool level, you could stash
away instances of MetacelloMCProject along with the #currentVersion of
each project... saving tons of calculations. Providing the user with an
invalidate cache button would "solve" the problem of cache invalidation
and as I said before, you can then have your cache without affecting the
validity of Metacello's operations.

BTW, are you using the mcz files that Simone and I created at ESUG?

Dale

> #currentVersion is probably the slowest operation in all of Metacello, but the #currentVersion _must_ be calculated correctly otherwise all is lost:)

Indeed

> I assume that you are concerned about caching because of the tool that you and Simone are working on? There are tricks that you can play at the tool level that might make things run faster (although the initial calculation must be done to warm the cache) including stashing results away between tool invocations ... at the tool level, you could stash away instances of MetacelloMCProject along with the #currentVersion of each project... saving tons of calculations. Providing the user with an invalidate cache button would "solve" the problem of cache invalidation and as I said before, you can then have your cache without affecting the validity of Metacello's operations.

Not only. I am working on a memory profiler, and I am looking for optimization opportunities.

> BTW, are you using the mcz files that Simone and I created at ESUG?

Yes, but it seems that it can be faster...

Alexandre
--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Alexandre Bergel wrote:
>> #currentVersion is probably the slowest operation in all of Metacello, but the #currentVersion _must_ be calculated correctly otherwise all is lost:)
>
> Indeed
>
>> I assume that you are concerned about caching because of the tool that you and Simone are working on? There are tricks that you can play at the tool level that might make things run faster (although the initial calculation must be done to warm the cache) including stashing results away between tool invocations ... at the tool level, you could stash away instances of MetacelloMCProject along with the #currentVersion of each project... saving tons of calculations. Providing the user with an invalidate cache button would "solve" the problem of cache invalidation and as I said before, you can then have your cache without affecting the validity of Metacello's operations.
>
> Not only. I am working on a memory profiler, and I am looking for optimization opportunities.

Aha! Now I understand your desire to cache the project instance... in
your analysis are you seeing the repeated calculation of projects having
an excessive cost?

It would be relatively straightforward to create a dynamic cache for the
project instance ...

>
>> BTW, are you using the mcz files that Simone and I created at ESUG?
>
> Yes, but it seems that it can be faster...

No doubt:)

Thanks,

Dale

> Aha! Now I understand your desire to cache the project instance... in your analysis are you seeing the repeated calculation of projects having an excessive cost?
>
> It would be relatively straightforward to create a dynamic cache for the project instance ...

I will privately send you a paper I am currently writing on memory profiling.
Anyone interested in it, let me know. This is for Smalltalks'10.

Cheers,
Alexandre

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Alexandre Bergel wrote: Alexandre,

I'm honored that I Metacello has given you such a good example of a
memory/time hog:)

Also, if you think #currentVersion is slow now, you should have seen
it's performance before I put in the dynamic caching:)

Technically, the #currentVersion of a project can and will change during
a load operation, depending upon the interrelationships of the dependent
projects...In your example (which doesn't involve a load) that isn't an
issue.

Nonetheless, your analysis shows that there's an order of magnitude
speedup lurking out there, so I will take another crack at that sun of a
gun!

Thanks,

Dale