What the state of unifying Beacon with SystemLogger?

Le 18/4/16 18:28, Denis Kudriashov a écrit :
> Hello.
>
> Year ago Beacon and SystemLogger were announced. There was big
> discussion about them. And there were plans to provide single
> solution. What was done around that?

Nothing. We should have done it at Cambridge and we just discussed.
I did not like some Beacon class names because they are totally confusing.
>
> What exactly should be merged between this libraries? Was Beacon or
> SystemLogger planned to be end solution?
We planned to merge SystemLogger into Beacon to use announcements
> We definitely need legacy logging tool for Pharo. And I am going to
> work on this direction.
> Personally I prefer metaphor and names from Beacon. But both libraries
> implement similar idea.
Remember that you want to have them short especially for the main one.
This is why in SystemLogger
we have Log instead of what it is LogObject if I remember correctly.
>
> Best regards,
> Denis

Hi,

Beacon is being used systematically. We even said we will integrate it in the image for Pharo 5 at the end of last year, but after the agitated start of the year, I did not push it anymore to not increase the level of agitation.

The only class name that encountered some opposition was BoundedBeacon.

We also agreed that the only things we want from SystemLogger were the concrete connections to external storage. This was not done, but the idea was that we should first integrate them and then allow people to extend it with bindings to external storage.

I use Beacon on a regular basis, and I extended it recently with a couple of more Signal types that can capture exceptions and stack information. You now also have a better integration in the inspector to be able to start/pause an in-memory log stream. In the process I also added the possibility to remove Announcements from an AnnouncementSet and I would like to push this in Pharo 6.0. One thing I would work on is to add to Annoucements the possibility of filtering announcements based on instances not just types. This is a longer term plan.

Another thing to do is to ensure that the log signals are not expensive to create. For example, the ExceptionSignal, ContextStackSignal and MethodStackSignal manipulate thisContext right at creation time, while this should happen only when we actually capture it in a BoundedBeacon to manipulate/store it in a specific way. This part should be refined.

@Dennis: it would be great to join efforts. The two frameworks look similar, but there are 2 significant differences:
1. SystemLogger has its own log events propagation mechanism, Beacon uses Announcements.
2. SystemLogger relies on levels of filtering, Beacon uses the mechanisms from Announcements to pick and choose at a fine grain level which log signals should be logged.

For reference:
- description: http://www.humane-assessment.com/blog/beacon/
- repository: http://www.smalltalkhub.com/#!/~Pharo/Beacon

Cheers,
Doru

 
--
www.tudorgirba.com
www.feenk.com

“Live like you mean it."

You might want to read

  https://medium.com/concerning-pharo/lampsort-revisited-visualised-6652055ef858

that discusses object logging and what you can do with it.

My take on this is that if you write your code to produce log events (as Announcements), you can do whatever you want in the end. There is no real need to depend on some framework.

Hi,

I even reviewed this post :).

Of course, you can just use plain Announcements, but I still need to nuance your too strong statement. A framework is worthwhile when there is a recurrent need that it can serve. And when it comes to logging there are several such needs. For example:
- reusing typical events (e.g., logging an exception)
- reusing storage possibilities
- having a timestamp for events

I think it is worthwhile to have support out of the box for handling these.

Cheers,
Doru


--
www.tudorgirba.com
www.feenk.com

"Problem solving efficiency grows with the abstractness level of problem understanding."

What I would like and that I wanted to SystemLogger was that the
compiler for example could issue a log event and that we could build
a tools to browse only such events and that I can click on it and jump
right in the method with shadowing variables for example.
So to get something more than dead strings.


Le 19/4/16 15:55, Tudor Girba a écrit :

On 19 Apr 2016, at 20:44, stepharo <[hidden email]> wrote:

What I would like and that I wanted to SystemLogger was that the compiler for example could issue a log event and that we could build
a tools to browse only such events and that I can click on it and jump right in the method with shadowing variables for example.
So to get something more than dead strings.

And then do

Hi,

I imagine well because I could also do it with SystemLogger.
Now one of the question I faced when I started to think about the UI was:
what is the protocol so that each event can expose domain specific actions to the tools browsing them
(I did not think at that time about yet another pragma)

For the compiler I would like to have
   
    jumptomethod
        <action>       
        MessageBrowser openWith: self method selection: self variables

So what we see is that we could have a pragma to describe action
but I did not like it.

So now I would have

CompilerLog packaged in Compiler
And
JumpToMethod packaged in CompilerExtrasTools (a package that import Tools and extend Compiler)

For database
    -...

And in the compilerLogItem I do not want to have reference to UI tools



PS: I do not like the name Event because I would prefer to have a separate domain name from UI. 

On 19 Apr 2016, at 20:44, stepharo <[hidden email]> wrote:

What I would like and that I wanted to SystemLogger was that the compiler for example could issue a log event and that we could build
a tools to browse only such events and that I can click on it and jump right in the method with shadowing variables for example.
So to get something more than dead strings.

Of course.

This is already the case with Zinc.

Just inspect 

  ZnLogEvent announcer.

And then do

  ZnClient new get: 'http://pharo.org'.

Now go to the first inspector and click on the Announcements tab. You see events generated by the Zinc client, real objects with all kinds of interesting objects inside of them:

Le 19/4/16 15:55, Tudor Girba a écrit :

Hi,

I even reviewed this post :).

Of course, you can just use plain Announcements, but I still need to nuance your too strong statement. A framework is worthwhile when there is a recurrent need that it can serve. And when it comes to logging there are several such needs. For example:
- reusing typical events (e.g., logging an exception)
- reusing storage possibilities
- having a timestamp for events

I think it is worthwhile to have support out of the box for handling these.

Cheers,
Doru

On Apr 19, 2016, at 3:44 PM, Sven Van Caekenberghe <[hidden email]> wrote:

You might want to read

 https://medium.com/concerning-pharo/lampsort-revisited-visualised-6652055ef858

that discusses object logging and what you can do with it.

My take on this is that if you write your code to produce log events (as Announcements), you can do whatever you want in the end. There is no real need to depend on some framework.

On 19 Apr 2016, at 15:34, Tudor Girba <[hidden email]> wrote:

Hi,

Beacon is being used systematically. We even said we will integrate it in the image for Pharo 5 at the end of last year, but after the agitated start of the year, I did not push it anymore to not increase the level of agitation.

The only class name that encountered some opposition was BoundedBeacon.

We also agreed that the only things we want from SystemLogger were the concrete connections to external storage. This was not done, but the idea was that we should first integrate them and then allow people to extend it with bindings to external storage.

I use Beacon on a regular basis, and I extended it recently with a couple of more Signal types that can capture exceptions and stack information. You now also have a better integration in the inspector to be able to start/pause an in-memory log stream. In the process I also added the possibility to remove Announcements from an AnnouncementSet and I would like to push this in Pharo 6.0. One thing I would work on is to add to Annoucements the possibility of filtering announcements based on instances not just types. This is a longer term plan.

Another thing to do is to ensure that the log signals are not expensive to create. For example, the ExceptionSignal, ContextStackSignal and MethodStackSignal manipulate thisContext right at creation time, while this should happen only when we actually capture it in a BoundedBeacon to manipulate/store it in a specific way. This part should be refined.

@Dennis: it would be great to join efforts. The two frameworks look similar, but there are 2 significant differences:
1. SystemLogger has its own log events propagation mechanism, Beacon uses Announcements.
2. SystemLogger relies on levels of filtering, Beacon uses the mechanisms from Announcements to pick and choose at a fine grain level which log signals should be logged.

For reference:
- description: http://www.humane-assessment.com/blog/beacon/
- repository: http://www.smalltalkhub.com/#!/~Pharo/Beacon

Cheers,
Doru

On Apr 19, 2016, at 2:59 PM, stepharo <[hidden email]> wrote:



Le 18/4/16 18:28, Denis Kudriashov a écrit :

Hello.

Year ago Beacon and SystemLogger were announced. There was big discussion about them. And there were plans to provide single solution. What was done around that?

Nothing. We should have done it at Cambridge and we just discussed.
I did not like some Beacon class names because they are totally confusing.

What exactly should be merged between this libraries? Was Beacon or SystemLogger planned to be end solution?

We planned to merge SystemLogger into Beacon to use announcements

We definitely need legacy logging tool for Pharo. And I am going to work on this direction.
Personally I prefer metaphor and names from Beacon. But both libraries implement similar idea.

Remember that you want to have them short especially for the main one. This is why in SystemLogger
we have Log instead of what it is LogObject if I remember correctly.

Best regards,
Denis

--
www.tudorgirba.com
www.feenk.com

“Live like you mean it."

--
www.tudorgirba.com
www.feenk.com

"Problem solving efficiency grows with the abstractness level of problem understanding."

AnnouncementSet is just a normal set, so it has always been possible to remove items, just not recommended.

Hi,

> On Apr 20, 2016, at 9:37 AM, Henrik Johansen <[hidden email]> wrote:
>
>
>> On 19 Apr 2016, at 3:34 , Tudor Girba <[hidden email]> wrote:
>>
>> In the process I also added the possibility to remove Announcements from an AnnouncementSet and I would like to push this in Pharo 6.0. One thing I would work on is to add to Annoucements the possibility of filtering announcements based on instances not just types. This is a longer term plan.
> AnnouncementSet is just a normal set, so it has always been possible to remove items, just not recommended.
> Filtering on instances is in the same category of making actual delivery after subscription more dynamic; both also have the effect that it becomes even harder to reason about what is going to happen as a result of an Announcement without actually running the code.

The fact that AnnouncementSet is a Set is actually debatable because it basically reuses the implementation, not the type. Indeed, Set does allow for removing items, but I was referring to excluding items which is a slightly different use case (like is the case for Exceptions).

> IMHO, if the logic is "I want to handle this announcement some of the time", that is much easier to understand if handled explicitly in announcement handling code, rather than through state changes to the announcement delivery.

I disagree. I have a project which has about 50 different signals, and I am interested in all but one. Listing all 49 would be impractical and not particularly clear. Furthermore, when you have inheritance, it becomes even more complicated. You have the same case with exceptions and there we allow exclusions for good reasons.

> As I think Glamour is a poster child for, current usage patterns can already be hard to follow ;)

I do not understand this statement in the context of this email.

Cheers,
Doru



> Cheers,
> Henry

--
www.tudorgirba.com
www.feenk.com

"Reasonable is what we are accustomed with."

Ah, exclusions are a different matter from removal, that sounds like a good change indeed!

My point was, messing with the subscription to alter what is being delivered outside of registration/de-registration, would lead to hard(er)-to-understand code; that's would not be the case for exclusions like:
announcer when: SuperAnnouncement - SpecificAnnouncementImNotInterestedIn do: [:ann | ... ]

Cheers,
Henry

Denis,

Sorry for not having a better reviewed understanding of both
frameworks, but I have some thoughts from perspective of one user...

On Thu, Apr 21, 2016 at 12:05 AM, Denis Kudriashov <[hidden email]> wrote:
You are considering Log as a noun, but it can also be used as a verb.
defn "13. to enter in a log; keep a record of; e.g. to log a day's events." [1]

Stef said: "Remember that you want to have them short especially for
the main one. This is why in SystemLogger
we have Log instead of what it is LogObject"
from which I understood that Log was the client interface, to be used
like Transcript.  I think this is a good idea.  It would create the
LogEntry

[1] http://www.dictionary.com/browse/log

> And now these backends are represented by subclasses of Logger:
> StdoutStreamLogger, SysLogSender. I would call them StdoutLog, SysLog.
> But with such change we need to call log entry differently. It can be
> LogRecord or LogEntry. I not understand what was the problem with such
> names? (there was some explanation why Log was chosen for this)

Considering a "log" being the final resting point of the logged
objects, that is the external disk text file or database, then within
Pharo we don't have logs, only processes that write to the external
logs, so Logger seems appropriate.  This is except for an in memory
log, which for backend might need both MemoryLogger and
MemoryObjectLog.

Log would create and send theLogEntry to SystemLogger
which forwards theLogEntry to other registered XXXLogger backend.

> name SystemLogger for collection based log.
>
> SystemLogger makes me think that it is singleton which is used by system and
> I should not use it for my own application.

I guess your application would make use SystemLogger via Log at the
front end, and registering backends with SystemLogger.

> I would call it like ObjectLog or NativeLog (following my first sentence
> about Log).

NativeLog doesn't seems right.  ObjectLog might be suitable for an
in-memory backend, like above I mention MemoryObjectLog.

> Also it is intercepted with SysLog name and I guess that's why it was called
> SysLogSender and not SysLogLogger.
>
> name LogDispatcher.

LogDispatcher is okay, but I'm not sure of convention to be associate
the "doer" with the "target" rather than the "item", where
doer=Dispatcher; item=LogEntry; target=final-backend-log;   A little
it sounds like its dispatching a whole log rather than one log entry.
Maybe should be LogEntryDispatcher(??) or... SystemLogger as the
marshalling point to forward to other XXXLoggers.

>
> Here same problem as Log. What means physical logs dispatching? I know it is
> introduced around Log as log entry. But it is not intuitive for me (same as
> Log).

I don't think Log should be a log entry.  It should be the user
interface that creates and sends LogEntries via some XXXLogger.

GTInspector would need good live updating, filtering, clearing,
scrolling to keep cursor position as new entries arrive.

cheers -ben

Hi.

Thank's for answers.

2016-04-20 10:53 GMT+02:00 Norbert Hartl <[hidden email]>:

Agreed. As I said above. If you take to point of view from legacy logging tool Beacon is not a big help. But the ideas in it are good and should be integrated somewhere. If you are talking about legacy tooling it might ok to use the terms logging as well, no? I personally like the work Beacon but in my opinion it is  wrong for this purpose. A beacon sends a uniform signal in order to be discovered. The Beacon code sends signals to a beacon. And I can register at the beacon in which signals I'm interested in. Feels wrong to me.

Your words are reasonable. Interesting what Tudor think about it?

There are few things which I not like in SystemLogger (which not means that everything is good with Beacon):

• name Log to represent records of the underlying "physical log". 

IMHO Log means exactly end logging backend like text file, syslog or external database. Log is the place where we can explore (read) what was happen. 

And now these backends are represented by subclasses of Logger: StdoutStreamLogger, SysLogSender. I would call them StdoutLog, SysLog.

But with such change we need to call log entry differently. It can be LogRecord or LogEntry. I not understand what was the problem with such names? (there was some explanation why Log was chosen for this)

We picked Log because we wanted to have it short.

• name SystemLogger for collection based log. 

SystemLogger makes me think that it is singleton which is used by system and I should not use it for my own application. 

I would call it like ObjectLog or NativeLog (following my first sentence about Log).

It was the same as SystemAnnouncer.
It could be name Logger.

Also it is intercepted with SysLog name and I guess that's why it was called SysLogSender and not SysLogLogger.

• name LogDispatcher. 

Here same problem as Log. What means physical logs dispatching? I know it is introduced around Log as log entry. But it is not intuitive for me (same as Log).

In context of Log as real log (FileLog, SysLog) I would call this dispatcher as Logger with same responsibility to register new events in all registered logs.

LogDispatcher is coherent with Log.
If you change Log to be LogEntry then LogDispatcher could be named LogEntryDispatcher.

• announcements LogAdded and LogRemoved in the core.

Why any Logger should announce every log entry?

Because you can build tools to reflect changes.
You do not want to use changed? We will remove changed from Pharo.

I think we not need special LoggerUI app. We can just inspect SystemLogger instance to look at all recorded objects.

So we do not have the same dream and need. Not all tools should fit in an inspector.

And in perspective of my suggestions it will mean that we just explore particular ObjectLog. And we can extend GTInspector for this.

> You are considering Log as a noun, but it can also be used as a verb.
> defn "13. to enter in a log; keep a record of; e.g. to log a day's events." [1]
>
> Stef said: "Remember that you want to have them short especially for
> the main one. This is why in SystemLogger
> we have Log instead of what it is LogObject"
> from which I understood that Log was the client interface, to be used
> like Transcript.

Yes

Yes this is exactly that

Log -> Dispatcher -> Loggers
Here since we did not want to have two class

     Log for the creation API
     LogEntry/LogObject for representing the entry
     we choose to keep Log even if Log was not name LogEntry

Denis

you see, we (norbert and me) did not take this names randonmly we really
thought about them.

Stef

Hi,


Good summary.

When we discussed about adopting Beacon we also agreed that we will use the code or at least the experience of binding to concrete storage facilities that exist in SystemLogger.


> As an example I can put one of my projects where I use SystemLogger. When sending a push notification I log/emit an object/event of type COMessageSent with sublcasses for apple, google, etc. It is emitted to a central location. In SystemLogger there a three loggers configured. One is in memory and counts the number of events per class. One takes the object, converts it into a short string representation like "apple push sent to XXXX" and sends it via syslog/UDP to a central logging server. Finally the third one takes the object, converts it to json and stores the json in a elasticsearch database. The database allows me to do all sorts of statistical analysis of the data over a long time frame such as number of push messages per month for one year. Which looks like this
>
>  <Bildschirmfoto 2016-04-20 um 10.24.36.png>
>
> So if you take my point of view then Beacon is pretty useless. SystemLogger and Beacon both use a singleton to dispatch log entities.

This is incorrect. Beacon does not require the singleton. We just have one for convenience, but you can use any announcer for the purpose. That is the beauty of it … the fact that we do not have to reinvent anything because it is already provided by Announcements :). This was also a point that was made by Sven and is what I took into account when I designed Beacon.

Please read the sections "Beyond global bindings”, and “Logging any announcements from any announcer” from:
http://www.humane-assessment.com/blog/beacon

The singleton convenience is for replacing the places where Transcript is being currently used.


> In SystemLogger has own implementation of registry for that loggers. Beacon uses Announcements for it. That's basically the difference. I like the idea of using Announcements and not invent yet another distribution mechanism. But with Announcements you can only filter by class and not by instance.
> So it cannot do what system logger can do. Doru writes that he is working on that and it would be great. Although I have doubt this can be done performant I would be lucky to have it.

Indeed, I think this is important. However, I also found that going extreme in creating signal types is quite powerful and enough for practical purposes. The cool thing about having fine-grained signal types is that you can also define easily dedicated inspector presentations.


> On the other hand Beacon implements a handful of classes for some use cases but nothing more. So I see Svens point that you don't need it really because Announcements are there anyway and most of the stuff is application specific so I can just do that without having to integrate a "framework”.

Exactly :). These are convenience, and the cool thing is that people can just choose various levels without having to necessarily buy into the complete framework. We do not get often chances like this :)


>> What exactly should be merged between this libraries? Was Beacon or SystemLogger planned to be end solution?
>>
> I'm not sure anymore. I can see some good things in Beacon that enable to register a logger for part of the execution. The drawback is that if you think about output facilities like log files there is no central place to attach it. And I surely don't want to have log configurations in the middle of my code. Beacon is not well thought in that direction, yet.

I do not understand this part and I might be missing something. Could you go into details about where you see the log configurations problem in Beacon?


> IMHO I'd like to have a SystemLogger with the distribution of objects like it is done in Beacon. Filtering is to be solved. This is hard to do with Announcements and to be honest I don't really like it how it is done in SystemLogger.

I think that if we invest in the filtering part we can consolidate around Announcements.


>> We definitely need legacy logging tool for Pharo. And I am going to work on this direction.
>> Personally I prefer metaphor and names from Beacon. But both libraries implement similar idea.
>>
> Agreed. As I said above. If you take to point of view from legacy logging tool Beacon is not a big help.

I do not understand what you mean by legacy logging. Could you explain?


> But the ideas in it are good and should be integrated somewhere. If you are talking about legacy tooling it might ok to use the terms logging as well, no? I personally like the work Beacon but in my opinion it is  wrong for this purpose. A beacon sends a uniform signal in order to be discovered. The Beacon code sends signals to a beacon. And I can register at the beacon in which signals I'm interested in. Feels wrong to me.

Beacon is a central place which is both discoverable and propagates messages around to whomever is interested in consuming them. From this point of view, I think it works.

Cheers,
Doru


> Norbert
>

--
www.tudorgirba.com
www.feenk.com

"Some battles are better lost than fought."