Fwd: Building package cache

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Building package cache

Simon Denier-3

okaaay, wrong reply-to, retweet....


Begin forwarded message:

From: Simon Denier <[hidden email]>
Date: 28 janvier 2010 23:45:50 HNEC
To: Simon Denier <[hidden email]>
Subject: Re: [Moose-dev] Building package cache


On 26 janv. 2010, at 22:03, Simon Denier wrote:


On 26 janv. 2010, at 21:07, Alexandre Bergel wrote:

I do not have a solution, but this is really annoying. In addition to the time taken by loading Moose, it makes 10 minutes just to check what are are dependencies Mondrian has.


Well, temporary solutions sometimes become less than temporary... I fear this is the case.

Maybe we can do something with an incremental, on-demand cache: whenever a class or method is imported, we check whether the cache has been built for it - if not, we built it.


After playing a bit with the idea tonight, now I remember why I choose the global cache solution. So I explain how it works in the current system:

To build the system cache, I go over all classes and all class extensions of PackageInfos and register their most specific package.
Since I go over the whole system, I assume I know all class extensions  in all classes.
Now when I ask the package for a method, there are two cases:
- either I can find it in the cache of class extensions and I return its package
- either I can't find it, so I assume it is a normal method and retrieve the package of its class (from the cache).

Now in an incremental cache system, I can cache a class when I import it, I can even cache a package so that its class extensions are imported, as above. The big problem is then that I can't assume that I know all class extensions within the class, because other packages (which I don't necessarily import) may do some. To do that, I should ask all methods within each imported class for its most specific package. This was more or less the situation we had before the cache, which brought the system to its knees on large projects. Perhaps I will try tomorrow but I don't have high hope. We need something better than that.


--
 Simon




--
 Simon




_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Building package cache

Alexandre Bergel
ok, I see. Have you considered extending the class PackageInfo with  
these caches instead?
Probably what would be tricky, is when to reset these caches. These  
could be done using the system change notification I imagine.

Cheers,
Alexandre


On 29 Jan 2010, at 06:54, Simon Denier wrote:

>
> okaaay, wrong reply-to, retweet....
>
>
> Begin forwarded message:
>
>> From: Simon Denier <[hidden email]>
>> Date: 28 janvier 2010 23:45:50 HNEC
>> To: Simon Denier <[hidden email]>
>> Subject: Re: [Moose-dev] Building package cache
>>
>>
>> On 26 janv. 2010, at 22:03, Simon Denier wrote:
>>
>>>
>>> On 26 janv. 2010, at 21:07, Alexandre Bergel wrote:
>>>
>>>> I do not have a solution, but this is really annoying. In  
>>>> addition to the time taken by loading Moose, it makes 10 minutes  
>>>> just to check what are are dependencies Mondrian has.
>>>>
>>>
>>> Well, temporary solutions sometimes become less than temporary...  
>>> I fear this is the case.
>>>
>>> Maybe we can do something with an incremental, on-demand cache:  
>>> whenever a class or method is imported, we check whether the cache  
>>> has been built for it - if not, we built it.
>>
>>
>>
>> After playing a bit with the idea tonight, now I remember why I  
>> choose the global cache solution. So I explain how it works in the  
>> current system:
>>
>> To build the system cache, I go over all classes and all class  
>> extensions of PackageInfos and register their most specific package.
>> Since I go over the whole system, I assume I know all class  
>> extensions  in all classes.
>> Now when I ask the package for a method, there are two cases:
>> - either I can find it in the cache of class extensions and I  
>> return its package
>> - either I can't find it, so I assume it is a normal method and  
>> retrieve the package of its class (from the cache).
>>
>> Now in an incremental cache system, I can cache a class when I  
>> import it, I can even cache a package so that its class extensions  
>> are imported, as above. The big problem is then that I can't assume  
>> that I know all class extensions within the class, because other  
>> packages (which I don't necessarily import) may do some. To do  
>> that, I should ask all methods within each imported class for its  
>> most specific package. This was more or less the situation we had  
>> before the cache, which brought the system to its knees on large  
>> projects. Perhaps I will try tomorrow but I don't have high hope.  
>> We need something better than that.
>>
>>
>> --
>>  Simon
>>
>>
>>
>
> --
>  Simon
>
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
_,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
Alexandre Bergel  http://www.bergel.eu
^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.





_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: Building package cache

Simon Denier-3

On 31 janv. 2010, at 18:22, Alexandre Bergel wrote:

> ok, I see. Have you considered extending the class PackageInfo with these caches instead?


Nope, because I didn't want to dive into this notorious mess :)
Obviously, the solution is to push the ongoing work on packages.


> Probably what would be tricky, is when to reset these caches. These could be done using the system change notification I imagine.
>
> Cheers,
> Alexandre
>
>
> On 29 Jan 2010, at 06:54, Simon Denier wrote:
>
>>
>> okaaay, wrong reply-to, retweet....
>>
>>
>> Begin forwarded message:
>>
>>> From: Simon Denier <[hidden email]>
>>> Date: 28 janvier 2010 23:45:50 HNEC
>>> To: Simon Denier <[hidden email]>
>>> Subject: Re: [Moose-dev] Building package cache
>>>
>>>
>>> On 26 janv. 2010, at 22:03, Simon Denier wrote:
>>>
>>>>
>>>> On 26 janv. 2010, at 21:07, Alexandre Bergel wrote:
>>>>
>>>>> I do not have a solution, but this is really annoying. In addition to the time taken by loading Moose, it makes 10 minutes just to check what are are dependencies Mondrian has.
>>>>>
>>>>
>>>> Well, temporary solutions sometimes become less than temporary... I fear this is the case.
>>>>
>>>> Maybe we can do something with an incremental, on-demand cache: whenever a class or method is imported, we check whether the cache has been built for it - if not, we built it.
>>>
>>>
>>>
>>> After playing a bit with the idea tonight, now I remember why I choose the global cache solution. So I explain how it works in the current system:
>>>
>>> To build the system cache, I go over all classes and all class extensions of PackageInfos and register their most specific package.
>>> Since I go over the whole system, I assume I know all class extensions  in all classes.
>>> Now when I ask the package for a method, there are two cases:
>>> - either I can find it in the cache of class extensions and I return its package
>>> - either I can't find it, so I assume it is a normal method and retrieve the package of its class (from the cache).
>>>
>>> Now in an incremental cache system, I can cache a class when I import it, I can even cache a package so that its class extensions are imported, as above. The big problem is then that I can't assume that I know all class extensions within the class, because other packages (which I don't necessarily import) may do some. To do that, I should ask all methods within each imported class for its most specific package. This was more or less the situation we had before the cache, which brought the system to its knees on large projects. Perhaps I will try tomorrow but I don't have high hope. We need something better than that.
>>>
>>>
>>> --
>>> Simon
>>>
>>>
>>>
>>
>> --
>> Simon
>>
>>
>>
>> _______________________________________________
>> Moose-dev mailing list
>> [hidden email]
>> https://www.iam.unibe.ch/mailman/listinfo/moose-dev
>
> --
> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:
> Alexandre Bergel  http://www.bergel.eu
> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.
>
>
>
>
>
> _______________________________________________
> Moose-dev mailing list
> [hidden email]
> https://www.iam.unibe.ch/mailman/listinfo/moose-dev

--
 Simon




_______________________________________________
Moose-dev mailing list
[hidden email]
https://www.iam.unibe.ch/mailman/listinfo/moose-dev