Announcement real problems - please read and comment.

Hi guys

We found a problem with ben and RPackage.
Here is the scenario and may be you can help us finding the right solutions.

Nautilus renames a class
        the class got renamed
        -> an announcement is raised
                - nautilus gets notified, it asks rpackageOrganizer (which also listens to the same event)
                                -> rpackageOrg breaks because the class if was referting to does not exist anymore.

Problem 1
        - the second announcement was never sent, because the first one broke the second was blocked.
        >> we should make sure that if an announcement leads to an error, other annoucements on the same emit should pass

Problem 2
        If rappckageOrganizer would be reached before Nautilus we would not get any problem (beside the problem 1).

        - we spent one hour playing with alternatives.
        We thought that if RPackage would points to classes a rename would not break the system.
        While this is true for class, the same problem will occur for method rename since compiled method are not shared and a new compiled method
        is compiled.

        - If RPackage was as deep inside the system as superclass/subclass, the invariant would be directly maintained by the system
        and we would not have to rely on announcement. But now we should find a solution.

        >> may be we should be able to say that an announcement has a priority.
        Typically we would like to say that when RPackageOrganizer  register interests it should be served first.
        we could have a systemLevel priority that could be specified on registration.

        what do you think?

Stef
       

On 01.04.2011 21:07, Stéphane Ducasse wrote: This sounds like mess in logic.
You ask package of class that is in the process of being renamed?
Why would you expect this to work?
Use more announcements to represent multiple steps in renaming process,
for example both ClassRenaming and ClassRenamed.
Changing it's package is something you do as an action that should be
_part_ of renaming, while updating it's location is something you would
do _after_ renaming is complete.
Bad, bad, bad, bad, baaaaaad idea.
I know from experience that once you concede that announcement listener
order should matter, you are setting yourself up to fail.
Although you know one listener _HAS_ to respond first, there will always
be someone else who want to respond "firstester".
Not to mention the logic in this case becomes next to impossible to
follow/debug/ maintain, as you've already experienced.

For a proper solution, introduce more Announcements for actions which
should happen in different phases of a process, as explained above.

Cheers,
Henry

PS. I was unable to load RPackage.
The initialize method of PackageOrganizer in RPackage-Core calls
registerInterestToSystemAnnouncement, which is an extensions in
RPackage-SystemIntegration.

PPS. There's a neat method called #on:send:to:,  which would be perfect
for the different actions in RPackage-SystemIntegration.

PPPS. Is the code for Nautilus available anywhere?

Henrik what is your answer to problem 1

Problem 1
        - the second announcement was never sent, because the first one broke the second was blocked.
        >> we should make sure that if an announcement leads to an error, other annoucements on the same emit should pass

Because we can get the system down just because one guy can register a bug.

No
        Just the browser ask the package its classes to display them and the package has not been refreshed because this is done by the announcement
        that is after the one that led to a refresh

> Why would you expect this to work?
> Use more announcements to represent multiple steps in renaming process, for example both ClassRenaming and ClassRenamed.
> Changing it's package is something you do as an action that should be _part_ of renaming, while updating it's location is something you would do _after_ renaming is complete.

Probably we will check that.


In which image?

> The initialize method of PackageOrganizer in RPackage-Core calls registerInterestToSystemAnnouncement, which is an extensions in RPackage-SystemIntegration.

Yes this is normal.
I will try to load it to check. May the configurationOfRPackage has not been updated

> PPS. There's a neat method called #on:send:to:,  which would be perfect for the different actions in RPackage-SystemIntegration.

But why should we rely on a different event system?

> PPPS. Is the code for Nautilus available anywhere?
Yes in Nautilus on squeaksource.

Hello,.


I have similar problems in my domain. And my solution is asynchronous announcer. Each announcement  subscriber should be notified in separate process. It is very easy to implement

Gofer new
        squeaksource:  'PharoTaskForces';
        package: 'ConfigurationOfRPackage';
        load
       
ConfigurationOfRPackage project  load: '1.0'

load RPackage in latest 1.3

Stef


On Apr 1, 2011, at 10:36 PM, Henrik Sperre Johansen wrote:

> PPS. There's a neat method called #on:send:to:,  which would be perfect for the different actions in RPackage-SystemIntegration.

I'm trying to understand how this on:send:to: would help us.
Because
        Nautilus registered to system announcer
Now I do not understand why RPackageOrganizer should not register to systemAnnouncer too.
And even if we use on:send:to: who can garantee that the on:send:to: will notify first the RPackageOrganizer and
not that an announcement will reach the browser before.

What was your idea?

Stef
       

On 1 April 2011 22:36, Henrik Sperre Johansen
<[hidden email]> wrote:
I want to add that there should be single place handling this event.
And if there multiple subscribers, they should not operate with same
data. Otherwise you will have many problems.

Exactly.



--
Best regards,
Igor Stasenko AKA sig.

Stef,

This sounds like a job of a unit (acceptance) test :)  One suggestion was to break the rename into pieces.  I follow the thinking, but fear it will make the problem worse.

Dolphin has a really slick event system, which was perhaps more of a big deal c 1997 than it is now.  One thing that became clear early on is that events are a best effort system, and I ended up adding some messages to, at the expense of performance, provide a hardened version of the system.  It's no big deal: I just trapped errors so that down-stream receivers would get their events.  The events in question caused network activity, so failures were far more common than one would expect renaming a class.

However, I think what you need here is a #rename:to:in: event that provides both old and new class names and the package that owns the affected class (the renamed version goes in the same package, right???).  I **think** that, combined with some pre-rename checks (Is this going to work?  If not, stop now...) will take care of the problem.

HTH.

Bill


________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Stéphane Ducasse [[hidden email]]
Sent: Friday, April 01, 2011 3:07 PM
To: Pharo Development
Subject: [Pharo-project] Announcement real problems - please read and   comment.

Hi guys

We found a problem with ben and RPackage.
Here is the scenario and may be you can help us finding the right solutions.

Nautilus renames a class
        the class got renamed
        -> an announcement is raised
                - nautilus gets notified, it asks rpackageOrganizer (which also listens to the same event)
                                -> rpackageOrg breaks because the class if was referting to does not exist anymore.

Problem 1
        - the second announcement was never sent, because the first one broke the second was blocked.
        >> we should make sure that if an announcement leads to an error, other annoucements on the same emit should pass

Problem 2
        If rappckageOrganizer would be reached before Nautilus we would not get any problem (beside the problem 1).

        - we spent one hour playing with alternatives.
        We thought that if RPackage would points to classes a rename would not break the system.
        While this is true for class, the same problem will occur for method rename since compiled method are not shared and a new compiled method
        is compiled.

        - If RPackage was as deep inside the system as superclass/subclass, the invariant would be directly maintained by the system
        and we would not have to rely on announcement. But now we should find a solution.

        >> may be we should be able to say that an announcement has a priority.
        Typically we would like to say that when RPackageOrganizer  register interests it should be served first.
        we could have a systemLevel priority that could be specified on registration.

        what do you think?

Stef

On 02.04.2011 12:13, Stéphane Ducasse wrote: It has nothing to do with registration ordering, but using the proper API.
If you look at the RPackageOrganizer registrations in
RPackage-SystemIntegration, all they do in the action block is send self
a message.
Instead of:
anAnnouncer
         when: SystemCategoryAddedAnnouncement do: [ :ann | self
systemCategoryAddedActionFrom: ann ];
you write:
announcer on: SystemCategoryAddedAnnouncement send:
#systemCategoryAddedActionFrom: to: self

It also has the added benefit you can make them weak if you want :)

Cheers,
Henry

And ugh, why the Announcement at the end of the Announcement names? :/

On 02.04.2011 09:16, Stéphane Ducasse wrote:
> Henrik what is your answer to problem 1
>
> Problem 1
> - the second announcement was never sent, because the first one broke the second was blocked.
> >>  we should make sure that if an announcement leads to an error, other annoucements on the same emit should pass
>
> Because we can get the system down just because one guy can register a bug.
http://forum.world.st/Another-finalization-concern-error-handling-td2989615.html

It's basically the same.

Cheers,
Henry

On 02.04.2011 13:04, Igor Stasenko wrote: Why? If multiple different systems are interested in the same
announcement, that is definately not the case, you want to keep the
listeners separate for modularity.
> And if there multiple subscribers, they should not operate with same
> data. Otherwise you will have many problems.
Right.
Immutability would come in handy here :)
In other words, don't mess with the contents of an Announcements when
you handle it.

Cheers,
Henry

On Apr 2, 2011, at 5:14 PM, Henrik Sperre Johansen wrote:

??

>
> Cheers,
> Henry
>

I do not get what is the difference. Can you explain how this would solve my problem?


I was discussing with luc and noury yesterday and we thought about the following:
        The system should be layered in two layers

        system (Rename) should emit annoucement to system objects (like RPakcageOrganizer)
       
        UI tools should register not to system but to RPackageOrganizer.

So this is probably the solution but I'm thinking that in the long turn may be pushing packages inside the system wiuthout annoucement
would be better.

> It also has the added benefit you can make them weak if you want :)
>
> Cheers,
> Henry
>
> And ugh, why the Announcement at the end of the Announcement names? :/

Who cares for now we have a system on its knees if one annoucnement breaks so to me this is totally useless.

Stef

Hi,

I think that this dialogue goes in too many directions, so I will try to provide a summary. Maybe this helps.

As I understand, Stef has the following problem:
- there is one announcer with two listeners
- the problem is that if the first listener raises an error, the second listener is not announced anymore
- thus, the base system can be broken simply by creating an extra listener that raises an error

As a solution, he proposed the idea of introducing the order of announcements.

Igor and Henrik had strong arguments against this solution. I agree with them. They suggested to either decompose the announcements more fine grained, or as you are pointing out now to layer them. I think this is the best solution, and like that we can have the basic announcements to be private.

However, the original question still remains valid. I am not quite sure what the solution is, but the idea of Denis of using asynchronous announcements sounds interesting. On the other hand, if I mess up something in the private workings of the kernel, my image will be broken, and that is expected.

More inside
 

On 3 Apr 2011, at 10:40, Stéphane Ducasse wrote:

This does not fix your problem. Henrik simply looked at the code and he saw that it would be better to use the message passing instead of the block.

> I was discussing with luc and noury yesterday and we thought about the following:
> The system should be layered in two layers
>
> system (Rename) should emit annoucement to system objects (like RPakcageOrganizer)
>
> UI tools should register not to system but to RPackageOrganizer.

Exactly!

> So this is probably the solution but I'm thinking that in the long turn may be pushing packages inside the system wiuthout annoucement
> would be better.

That could be the case, but I think the solution we have now is Ok to bootstrap.

Cheers,
Doru



--
www.tudorgirba.com

"Problem solving efficiency grows with the abstractness level of problem understanding."

On Apr 3, 2011, at 12:30 PM, Tudor Girba wrote:

> Hi,
>
> I think that this dialogue goes in too many directions, so I will try to provide a summary. Maybe this helps.
>
> As I understand, Stef has the following problem:
> - there is one announcer with two listeners
> - the problem is that if the first listener raises an error, the second listener is not announced anymore
> - thus, the base system can be broken simply by creating an extra listener that raises an error

Yes this was the problem number 1 of my original mail.
What I do not understand is why a
        ifCurtailed: [ emit and process next announcement does not solve the problem]



Now we also have the problem number 2.

> As a solution, he proposed the idea of introducing the order of announcements.
>
> Igor and Henrik had strong arguments against this solution. I agree with them. They suggested to either decompose the announcements more fine grained, or as you are pointing out now to layer them. I think this is the best solution, and like that we can have the basic announcements to be private.

Yes this is what I think.

ok

Yes I thought about that too.
So I will try to see with ben that nautilus register to RPackageOrganizer instaed of SystemAnnouncer.

On 03.04.2011 10:35, Stéphane Ducasse wrote: Same basic problem, same solutions apply.
I'd rather not repeat the exact same discussion.
Just exchange "finalization" with "announcement delivery" when reading
the thread.

Cheers,
Henry

Igor what was exactly the summary?

Stef

On 4 April 2011 12:55, Stéphane Ducasse <[hidden email]> wrote:
Summary, in short, is following:

we want to iterate over some arbitrary collection and perform an
action per each element, like:

collection do: [:each | each doSomeAction ]

but the problem here is that if one action triggers an error, then
your loop are interrupted and no other actions performed
for the remaining elements of collection.

So, we need such behavior , that guarantees that:
 a) all elements , except errorneous get processed
 b) if an error is unhandled by action , then show a debugger window

in terms of implementation this means that we should run each action
in separate forked process (and Eliot described a clever scheme how to
avoid most overhead of forking).

So then, even if some action will trigger an error, it won't interrupt
the loop and error could be handled in debugger

> Stef
>


--
Best regards,
Igor Stasenko AKA sig.

On Apr 4, 2011, at 1:16 29PM, Igor Stasenko wrote:

Or instead of forking each action, simply:
(changed to make OC copy)
deliver: anAnnouncement
        | cls copy |
        cls := anAnnouncement class.
       
        "using a copy, so subscribers can unsubscribe from announcer "
        "Use OrderedCollection , so unwind protection is easier"
        self protected: [ copy := subscriptions as: OrderedCollection ].
        self deliver: anAnnouncement to: copy

(private category)
deliver: anAnnouncement to: subs
       
        subs do:
                [:each |
                [each deliver: anAnnouncement]
                        ifCurtailed: [self deliver: anAnnouncement to: (subs after: each)]]


This has the added benefits of:
- No thread creation overhead (Not sure if how it's written avoids creating a block for argument to each ifCurtailed call though)
- Reproducibility.
        Especially in the case Stephane gave, if you forking in separate threads, his code would appear to work when stepping in debugger.
        There'd be mucho frustration and grumbling over announcements failing one second, but working the next. :)

Downside is ofc that rest of subscribers won't be notified until you proceed/exit the debugger, but compared to reproducibility, that's a small price to pay imho.

Cheers,
Henry