Why isn't this a weak dictionary of some ilk?
Locked
Why isn't this a weak dictionary of some ilk?
 
|
WAHalo class>>initialize
Halos := SeasidePlatformSupport weakDictionaryOfSize: 20 SeasidePlatformSupport class>>weakDictionaryOfSize: aNumber ^ IdentityDictionary new: aNumber Why isn't this a weak dictionary of some ilk? ---jski _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Why isn't this a weak dictionary of some ilk?
|
|
Squeak has no weak dictionaries that reliably works. If you want you
can replace this call with a WeakIdentityDictionary, but we found it less problematic like this. Lukas On Wed, Apr 22, 2009 at 5:17 PM, John Chludzinski <[hidden email]> wrote: > WAHalo class>>initialize > Halos := SeasidePlatformSupport weakDictionaryOfSize: 20 > > SeasidePlatformSupport class>>weakDictionaryOfSize: aNumber > ^ IdentityDictionary new: aNumber > > Why isn't this a weak dictionary of some ilk? > > ---jski > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > > -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Why isn't this a weak dictionary of some ilk?
|
|
2009/4/22 Lukas Renggli <[hidden email]>:
> Squeak has no weak dictionaries that reliably works. If you want you > can replace this call with a WeakIdentityDictionary, but we found it > less problematic like this. > Lukas, can you please share a details, what problems you have met. A Weak*** stuff is one of the important parts of the system, and its in our best interests to make it work rock solid. > Lukas > > On Wed, Apr 22, 2009 at 5:17 PM, John Chludzinski > <[hidden email]> wrote: >> WAHalo class>>initialize >> Halos := SeasidePlatformSupport weakDictionaryOfSize: 20 >> >> SeasidePlatformSupport class>>weakDictionaryOfSize: aNumber >> ^ IdentityDictionary new: aNumber >> >> Why isn't this a weak dictionary of some ilk? >> >> ---jski >> _______________________________________________ >> seaside mailing list >> [hidden email] >> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside >> >> > > > > -- > Lukas Renggli > http://www.lukas-renggli.ch > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Why isn't this a weak dictionary of some ilk?
|
|
In reply to this post by Lukas Renggli
Lukas Renggli wrote:
> Squeak has no weak dictionaries that reliably works. Can you say more about this? To what extent do weak dictionaries not work? Cheers, - Andreas > If you want you > can replace this call with a WeakIdentityDictionary, but we found it > less problematic like this. > > Lukas > > On Wed, Apr 22, 2009 at 5:17 PM, John Chludzinski > <[hidden email]> wrote: >> WAHalo class>>initialize >> Halos := SeasidePlatformSupport weakDictionaryOfSize: 20 >> >> SeasidePlatformSupport class>>weakDictionaryOfSize: aNumber >> ^ IdentityDictionary new: aNumber >> >> Why isn't this a weak dictionary of some ilk? >> >> ---jski >> _______________________________________________ >> seaside mailing list >> [hidden email] >> http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside >> >> > > > _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
> Can you say more about this? To what extent do weak dictionaries not work?
Earlier versions of Seaside were using weak data structures excessively to keep track of application state. The use of weak data structures simplified the Seaside implementation considerably, but causes a lot of problems under high load: http://lists.squeakfoundation.org/pipermail/squeak-dev/2004-November/084328.html http://lists.squeakfoundation.org/pipermail/squeak-dev/2005-March/089449.html http://lists.squeakfoundation.org/pipermail/seaside/2006-March/007258.html http://lists.squeakfoundation.org/pipermail/seaside/2006-May/007706.html At some point we removed all weak data structures from Seaside, because it could bring images to a halt when a lots of data was stored and eventually finalized. Squeak performance behaved much more predictable if we wasted memory. > Lukas, can you please share a details, what problems you have met. > A Weak*** stuff is one of the important parts of the system, and its > in our best interests to make it work rock solid. We would really appreciate a clean solution using weak data structures, but at that time we did not find another solution than use an ugly workarounds. Cheers, Lukas -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
2009/4/22 Lukas Renggli <[hidden email]>:
>> Can you say more about this? To what extent do weak dictionaries not work? > > Earlier versions of Seaside were using weak data structures > excessively to keep track of application state. The use of weak data > structures simplified the Seaside implementation considerably, but > causes a lot of problems under high load: > > http://lists.squeakfoundation.org/pipermail/squeak-dev/2004-November/084328.html > http://lists.squeakfoundation.org/pipermail/squeak-dev/2005-March/089449.html > http://lists.squeakfoundation.org/pipermail/seaside/2006-March/007258.html > http://lists.squeakfoundation.org/pipermail/seaside/2006-May/007706.html > Lukas, all these topics is about a weak array finalization process, which is just made for a convenience and on top of the fact, that VM supports weak references. But you are free to implement own finalization schemes(if you need to) instead of using default one. I don't see how this related to WeakIdentityDictionary per se. > At some point we removed all weak data structures from Seaside, > because it could bring images to a halt when a lots of data was stored > and eventually finalized. Squeak performance behaved much more > predictable if we wasted memory. > >> Lukas, can you please share a details, what problems you have met. >> A Weak*** stuff is one of the important parts of the system, and its >> in our best interests to make it work rock solid. > > We would really appreciate a clean solution using weak data > structures, but at that time we did not find another solution than use > an ugly workarounds. > > Cheers, > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
> Lukas, all these topics is about a weak array finalization process,
> which is just made for a convenience and on top of the fact, that VM > supports weak references. I understand that. And I remember that the suggestion was to implement our own finalization code. The reasons why we didn't do so were the following: - It was pretty easy to get rid of the requirement for weak data structures. The wasted memory was not significant enough to make it worth worrying about a special solution. - Introducing custom finalization code is highly non-portable across. At the time of Seaside 2.7 and 2.8, there were no easy possibilities to factor such code out. - Last but not least, I might be wrong on this, we had the impression that this was a Squeak only problem. I've never heard of similar problems with other Smalltalk. The finalization of session state just works out of the box. Maybe the real root of the problem is not even the weak finalization, but the way Squeak handles and rehashes large Dictionaries? Lukas -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
2009/4/22 Lukas Renggli <[hidden email]>:
>> Lukas, all these topics is about a weak array finalization process, >> which is just made for a convenience and on top of the fact, that VM >> supports weak references. > > I understand that. And I remember that the suggestion was to implement > our own finalization code. The reasons why we didn't do so were the > following: > > - It was pretty easy to get rid of the requirement for weak data > structures. The wasted memory was not significant enough to make it > worth worrying about a special solution. > > - Introducing custom finalization code is highly non-portable across. > At the time of Seaside 2.7 and 2.8, there were no easy possibilities > to factor such code out. > > - Last but not least, I might be wrong on this, we had the impression > that this was a Squeak only problem. I've never heard of similar > problems with other Smalltalk. The finalization of session state just > works out of the box. > > Maybe the real root of the problem is not even the weak finalization, > but the way Squeak handles and rehashes large Dictionaries? > To avoid rehashing, one could use a binary tree. But it having own cost: access to specific element takes O(log n) time, while current hash-based implementations is O(1) for best case, and, technically, O(n) for the worst case. If you remember, i implemented a separate set of Dictionaries (inluding Weak ones) for Magma, which were optimized for handling a large data sets at the cost of additional memory per key/value pair. It is also, have a different worst case access time - O(k), where k is number of elements which having same hash value under given key. As for finalization - its obvious, that you need to keep the amount of weak objects, which require finalization, small. Because to determine which object dies, you have to iterate over every item in collection - which is strictly linear. That's why putting too much objects into WeakRegistry makes squeak very slow, because it has to iterate over this collection over and over at _each_ garbage collection cycle, to determine which objects became garbage. Thus, without efficient support from VM side, a finalization scheme is doomed to be slow. > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > -- Best regards, Igor Stasenko AKA sig. _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
2009/4/22 Igor Stasenko <[hidden email]>:
> 2009/4/22 Lukas Renggli <[hidden email]>: >>> Lukas, all these topics is about a weak array finalization process, >>> which is just made for a convenience and on top of the fact, that VM >>> supports weak references. >> >> I understand that. And I remember that the suggestion was to implement >> our own finalization code. The reasons why we didn't do so were the >> following: >> >> - It was pretty easy to get rid of the requirement for weak data >> structures. The wasted memory was not significant enough to make it >> worth worrying about a special solution. >> >> - Introducing custom finalization code is highly non-portable across. >> At the time of Seaside 2.7 and 2.8, there were no easy possibilities >> to factor such code out. >> >> - Last but not least, I might be wrong on this, we had the impression >> that this was a Squeak only problem. I've never heard of similar >> problems with other Smalltalk. The finalization of session state just >> works out of the box. >> >> Maybe the real root of the problem is not even the weak finalization, >> but the way Squeak handles and rehashes large Dictionaries? >> > Yes, rehashing is sloow and in linear dependency from a number of elements. > > To avoid rehashing, one could use a binary tree. But it having own > cost: access to specific element takes O(log n) time, while current > hash-based implementations is O(1) for best case, and, technically, > O(n) for the worst case. > If you remember, i implemented a separate set of Dictionaries > (inluding Weak ones) for Magma, which were optimized for handling a > large data sets at the cost of additional memory per key/value pair. > It is also, have a different worst case access time - O(k), where k is > number of elements which having same hash value under given key. > > As for finalization - its obvious, that you need to keep the amount of > weak objects, which require finalization, small. Because to determine > which object dies, you have to iterate over every item in collection - > which is strictly linear. > That's why putting too much objects into WeakRegistry makes squeak > very slow, because it has to iterate over this collection over and > over at _each_ garbage collection cycle, to determine which objects > became garbage. > Thus, without efficient support from VM side, a finalization scheme is > doomed to be slow. That's why we don't use it. Seriously we are talking about images being blocked for 10 min to 20 min because of weak array finalization. See also: http://bugs.squeak.org/view.php?id=2910 Cheers Philippe _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Why isn't this a weak dictionary of some ilk?
|
|
In reply to this post by Lukas Renggli
Lukas Renggli wrote:
>> Lukas, all these topics is about a weak array finalization process, >> which is just made for a convenience and on top of the fact, that VM >> supports weak references. > > I understand that. And I remember that the suggestion was to implement > our own finalization code. The reasons why we didn't do so were the > following: The reasoning makes sense to me. I have one question though: Do you actually require finalization or is this "just" for caching? If the latter, I think there is a good chance of devising an alternative solution that clears out weak dicts as you go, only touching the slots that you would touch in a normal insertion/removal process. This could solve the caching problems but it wouldn't work if you expect this to work for finalization since even a full GC wouldn't necessarily cause the entries to be cleared if they aren't touched otherwise. Would this be a useful tradeoff for your usage patterns or completely pointless? Cheers, - Andreas > - It was pretty easy to get rid of the requirement for weak data > structures. The wasted memory was not significant enough to make it > worth worrying about a special solution. > > - Introducing custom finalization code is highly non-portable across. > At the time of Seaside 2.7 and 2.8, there were no easy possibilities > to factor such code out. > > - Last but not least, I might be wrong on this, we had the impression > that this was a Squeak only problem. I've never heard of similar > problems with other Smalltalk. The finalization of session state just > works out of the box. > > Maybe the real root of the problem is not even the weak finalization, > but the way Squeak handles and rehashes large Dictionaries? > > Lukas > _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
> The reasoning makes sense to me. I have one question though: Do you actually
> require finalization or is this "just" for caching? Let me explain what WeakDictionary was exactly used for: * Seaside 2.0 -- 2.7 To backtrack (with the back and forward buttons of the web browser) a model object, that object had to be registered in a session global registry. This registry would create a snapshot of all registered objects per request. It did so by adding all registered objects to a WeakIdentityKeyDictionary using the original object as key and the object copy as value. After a configurable number of requests old snapshots would be thrown away. Some of the registered objects that were carried on from request to request but not referenced from the application code would then automatically disappear from these snapshots. No finalization of the objects themselves is required. * Seaside 2.8 -- In todays versions of Seaside backtracked model objects have to be declared. For every request Seaside collects all the backtracked objects and creates a snapshot object without weak data structures. This is memory efficient and works well in most cases, however in some rare situations it can give unexpected results as objects are only backtracked as long as they are declared as backtracked. Cheers, Luka -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
On Thu, Apr 23, 2009 at 12:51 AM, Lukas Renggli <[hidden email]> wrote:
>> The reasoning makes sense to me. I have one question though: Do you actually >> require finalization or is this "just" for caching? > > Let me explain what WeakDictionary was exactly used for: There's also the Halos, of course. These ought to be in a weak collection. Julian _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
Probably we could safely put back that WeakDictionary there? I guess
that was an overreaction to remove it from that place. Halos are not used on high-load servers where the discussed problem could appear. Lukas On Thu, Apr 23, 2009 at 8:39 PM, Julian Fitzell <[hidden email]> wrote: > On Thu, Apr 23, 2009 at 12:51 AM, Lukas Renggli <[hidden email]> wrote: >>> The reasoning makes sense to me. I have one question though: Do you actually >>> require finalization or is this "just" for caching? >> >> Let me explain what WeakDictionary was exactly used for: > > There's also the Halos, of course. These ought to be in a weak collection. > > Julian > _______________________________________________ > seaside mailing list > [hidden email] > http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside > -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
Re: Re: Why isn't this a weak dictionary of some ilk?
|
|
2009/4/23 Lukas Renggli <[hidden email]>:
> Probably we could safely put back that WeakDictionary there? I guess > that was an overreaction to remove it from that place. Halos are not > used on high-load servers where the discussed problem could appear. What, www.seaside.st is not a high-load server? ;-) Cheers Philippe _______________________________________________ seaside mailing list [hidden email] http://lists.squeakfoundation.org/cgi-bin/mailman/listinfo/seaside |
«
Return to Seaside General
|
1 view|%1 views
| Free forum by Nabble | Edit this page |