Some clarifications (was DebugSession>>activePC:)

On 17 Jan 2019, at 03:21, Ben Coman via Pharo-dev <[hidden email]> wrote:

From: Ben Coman <[hidden email]>

Subject: CMake

Date: 17 January 2019 at 03:21:18 CET

To: Pharo Development List <[hidden email]>, Squeak Virtual Machine Development Discussion <[hidden email]>

On Thu, 17 Jan 2019 at 06:17, Eliot Miranda <[hidden email]> wrote:

        - About CMake, you may be right that makefile is better than CMake but a part of the world is using
        CMake and the net result is that we lost our effort and infrastructure just to follow you. Ronie uses CMake. 
        Igor which I consider as a talented developer used CMake because he thought it was the best tool
        he should use.

Yes, and I disagree about the way that they use it, and for good reason.  I have defended my use of Makefiles for a long time, for objective reasons. 

These reasons may be technically correct, but in terms of *community* consider it similar to premature optimization.

For yourself, the priority is a faster build.  Others may prioritize a faster coding workflow using an IDE like Visual Studio.

Indeed, back when I trialed building minheadless on Windows a *primary* consideration was that it looked easy to 

use Visual Studio because minheadless had a CMake build.  That led to me contributing a couple of small fixes for win64,

but without the enticement of CMake I might never have opened that box.   

Consider then the possibility that a portion of our Windows using community remains untapped 

because their skill set is Visual Studio and they don't see an easy path to using it with plain makefiles**.

So it depends on what is better for the *community* to optimize for:

* faster build-time for incumbents (important because thats where the majority of contributions come from)   

* broader community involvement with a workflow accelerating IDE (important because growing the vm community is important, from which additional core devs may arise) 

** I do understand that plain makefiles can be used with VS, but I'm not clear on the setup and 

unsure if all the fancy intellisense tools work.

I have also proposed good ways for using CMake (to derive a platform-specific header file defining available platform-specific features). 

I presume its the additional multi-build-system features that people want CMake for, not just the using it in name only.

I can't remember the trade-off between Automake-configure and CMake-configure.  Make using CMake-configure

would make it easier to co-ordinate parallel CMake and GnuMake systems.  I think I've noticed several large

code-bases providing both (but I'd have to check)

 

But my objection to Igor's process was that he generated sources on each build. 

 

And my objections to Ronie's use of CMake for the minheadless build are that a) it is slow and

I'd like to quantify that.  
@esteban, I remember you converted minheadless from CMake to Gnumake, but I'm not sure if I've got that right.

Can both be run off the current HEAD for minheadless?  Or I could compare HEAD with a previous commit that had CMake. 

b) explicit feature sets are much better than the implicit feature sets that arise when using CMake.

I'd like to understand this better. Could you expand?

cheers -ben

On 16 Jan 2019, at 23:15, Eliot Miranda <[hidden email]> wrote:

Hi Stef,

    thanks for taking the time to respond so thoughtfully.

On Mon, Jan 14, 2019 at 12:20 AM ducasse <[hidden email]> wrote:

Hi Eliot

I would like to make some clarifications. 

Preamble: 
--------------
I was thinking that I should not reply but I think that I have the right to clarify a bit because I do not like several points. 
I reply as Stef "the guy that had the vision of Pharo and that spent 10 years pushing and building Pharo.”
So I think that it makes me credible enough.

You do not have to defend your credentials.  The community is aware of your contributions.

 

My points:
-------------
        - With you this is always the same: I’m too emotional and it ends everything. Stephane is emotional 
        so we cannot communicate with him. You often place yourself as a professional and that I’m emotional. 
        Could you stop this little game because to me it starts to be a bit boring? Or I will just put a filter and 
        trash systematically your emails. 

I do not say "Stephane is emotional so we cannot communicate with him.".  I do think you let your emotions show too much and that it makes communication with you difficult.  But more importantly, a leader is more effective if they can reign in their emotions and not criticize people in public, and so on.  I mention your emotionality not to denigrate you but to hope that communication in the community can improve.  You have as much to gain as anyone, arguably more, from not taking things so personally and being less emotional.  If we analyze the above statement we see that it is coercive: stop criticizing my emotionality or I will stop reading your emails.  One could instead be open: "Why is it that you criticize me as being emotional?", "Can you give me evidence of me being emotional?" etc.  Instead you open with a defensive position, and with a threat.  If, later on, having started filtering out my emails, you found you had to respond again, because we have common interests and those interests cause us to need to interact, you will put yourself in a weak position having to back track on your filtering.

        - Last week you told us that time bombing our process was a bad idea in an answer about settings
        keeping references instead of releasing them. 

No.  I said that my opinion is that time boxing releases is a bad idea. This is not about settings, it is about the content of releases. For me, a release is done when it satisfactorily meets objective release criteria: tests pass, a subset of new features planned for the release are functional, etc.  That releasing something that is incomplete or broken does not help.  I base this on long experience with VisualWorks, Qwaq and Squeak.  

        First this had nothing to do with the problem.
        You see P7 was delayed because we considered that the system was not ready but P7 should be released.   
        May I make the remark that the world is using time-based delivery?

May I make the remark that what others choose to do does not make it right?

 

        - About CMake, you may be right that makefile is better than CMake but a part of the world is using 
        CMake and the net result is that we lost our effort and infrastructure just to follow you. Ronie uses CMake.  
        Igor which I consider as a talented developer used CMake because he thought it was the best tool 
        he should use. 

Yes, and I disagree about the way that they use it, and for good reason.  I have defended my use of Makefiles for a long time, for objective reasons.  I have also proposed good ways for using CMake (to derive a platform-specific header file defining available platform-specific features).  But my objection to Igor's process was that he generated sources on each build.  And my objections to Ronie's use of CMake for the minheadless build are that a) it is slow and b) explicit feature sets are much better than the implicit feature sets that arise when using CMake.

 

        - About infrastructure and process.
        I wanted to check the REplugin this week end because we should use it since the world
        is using Perl reg-expression and I could not find the code of the plugin on github.
        I saw that some plugin code is not even versioned and only available on a strange ftp and it was surprising 
        to me. I’m still surprised that after 3 years (when esteban requested that all plugin code are grouped together
        in a single place, this is still not the case). 

Esteban is free to move the code into VMMaker.oscog.  VMMaker.oscog below to the community.  It is wrong to expect me, a member of that community, to do all the leg work.  The VM is called opensmalltalk-vm for a reason.

  

        - Did it not look strange to you that Esteban left the VM list in the past? 
        Esteban looks like a reasonable and stable person to me.

Would you like to expand on this?  I find your sentence vague.  If you have specifics you would like to address I am open to hearing them.

 

        - You are systematically telling to me that I’m offended when people criticized us. I’m not offended by critics.
        We are not offended by critics. 
        We have internal emails that are much more violent about our own process. 

Wow, that does not sound good.  Surely a discussion over process does not have to be violent.

 

        We are not blind and we reflect on our own process and we invite people to REPORT problems. 

Do you invite them to discuss problems also?  Is the only avenue by which a problem can be raised through a bug tracker?  What is the bug that covers this discussion?

 

        Now we ask people to be credible when they do it because we are busy
                - report situation
                - step by step reproducibility
                - proportionality of the voice versus the problem
                - and not been offensive

        We did not see any official reports about problems that we may have introduced. 

        - You are vocal about “instability". And you report that one method changed. Seriously?
        Are you not exagerating a bit?

My comment comes from the frustration of having prepared a functional system in Pharo 6 only to find that it no longer works in Pharo 7.  Am I wrong to feel frustrated?  Would you not also feel frustrated?  Would you not also want to discuss this?  Would you not also, as the maintainer of a package upon which Pharo depends, want to have that package well-supported and n to be one that is frequently broken?

        Let us talk about instability:
                - When we talk about instability we talk about the fact that we cannot use for real ephemerons 
                because they corrupt the memory and that we get random crash.

Um, I have reached out on numerous occasions for test cases.  I haven't received a test case or complaint on ephemeris for many months, perhaps over a year (I will check).  I am very motivated to fix these problems.  Please bring them to my attention on vm-dev.  They will receive prompt attention.  I love working on this part of the system, ads does Clément.  I am eager to see ephemeris deployed in Squeak, Pharo and Cuis, especially to improve file handling, specifically arranging that files are flushed on close, and that we do not have two copies (one inevitably out of date) of each file, simply to arrange that file handles are closed on finalization.

 

                - We had instability when we have memory leaks that randomly slows down Pharo. 

Indeed, and memory leaks are not necessarily caused by the GC, but by handles to external memory, something that affects Pharo deeply because its graphics model uses external libraries, and hence requires more careful interfacing with external libraries, something that would be improved by having reliable ephemeron support.

I am quite confident that there are no memory leaks in the Spur GC; but I'm happy to be proved wrong and, along with Clément, who is currently doing great work on the GC, work that we hope to show a string publication for very soon, I will be happy to attempt to fix any memory leaks that are the fault of the GC, and not thin the FFI.

 

                - We had instability which delays our release with a bug in FFI that made cairo crash on windows 64 bits.

Yes, there are bugs in the VM. And what's your point?

 

If you want to interact with us (and not only me, because I’m the tip of the iceberg
and many people are frustrated by your attitude), let us start with positive communication and attitude:

        - You express your problems and we see how we (together) can fix them. 

That's what I'm trying to do.

 

        - You can contribute by writing tests and by entering bug entries in the bug tracker.

I write tests in VMMaker and elsewhere.  Being able to load VMMaker into Pharo and run the simulator, generate sources, do in-image compilation, and run its test suites is a significant number of tests. I work principally in Squeak.  I interact with lots of people in the Pharo community.  I am here, interacting now.

 

About Pharo:
————————
We understand that some people do not like it but we do not force anybody use and make business with it. 
Now I want to make something super clear: Pharo will not sacrifice agility and improvements on the altar of 
compatibility with Squeak. 

I am not asking that the Pharo community do that.  I am asking that the VMMaker package continue to work from releases to release.  From Pharo 6 to Pharo 7 that did not happen.

Pharo will change because Pharo is agile and because many things should be improved. 

and because it has a high-quality VM beneath it that has improved performance exponentially in the move from interpreter, stack interpreter, cog v1 and spur, and should continue to do so through Sista.  We are all working hard to make things better.

We pay real attention about backwards compatibility. Much more than people think. 
Because we have many external projects and libraries that we support. 
Now if Pharo is used to build the VM then we will have to find a way to pay attention. 
And we will reinvest in building a process that checks this. 

Good; thank you.

 

Our problem is that integration time cannot take hours and right now validating Pharo is
a bit too long. We will work on it. 

We strongly advocate to invest in tests. Tests are a good mechanisms to support evolution. 
If you have tests we can automatically rewrite deprecated methods and we will use this more often. 

We finally start to have:
        - a better compiler and not an ancient one which reported error as Morph.
        - a better architecture, more modular
        - a real bootstrap (we should improve it and build tools to support it - we are working on it)
        - strong libraries (File, Stream, HTTP) more documented and tested
        - better tools (Iceberg and Calypso are definitively steps in the right direction)
And we will continue to improve. 
We will iterate on all these to make Pharo even better. 
We are writing more tests to support those changes. 

Glad to hear it.  And I'm very much aware of this work and support it wholeheartedly.  And in the VM we are doing similar things.

 

I repeat it: I think that we are doing a pretty good job about the quality that we deliver. 

I agree.

 

Stef

Thank you.

Eliot

 

> On 11 Jan 2019, at 18:54, Eliot Miranda <[hidden email]> wrote:
> 
> Hi Stef,
> 
>> On Jan 10, 2019, at 7:59 AM, ducasse <[hidden email]> wrote:
>> 
>> Eliot I would like also two points to this.
>> 
>> - First we asked thomas to write tests about the debugger model and you see if they would be tests about methods we could understand 
>> that they are used and control what they do. So we should all thank thomas for his energy in this not so easy task.
>> 
>> - Second it would be nice if you could refrain to be systematically negative about what we are doing. I think that our development process
>> is much better than many others :) It is not perfect because this does not exist. 
>> I think that we are doing a great job make Smalltalk cool. And yes it may happen that one untested, undocumented method
>> get lost. I think that we are doing pretty good given the resources we have. 
> 
> Even more serious an issue for the Pharo community than a development process which fails to support the Ned’s of users is a defensive attitude that does not want to discuss serious issues maturely. I bring up the stability and backward-portability issue because it is *important*; it has affected Clément’s ability to deliver Sista and my and feenk’s efforts to support VM development on Pharo.  If your response to my trying to discuss seriously and objectively a problem that needs discussion is always to say “please don’t be negative” I have even less confidence that Pharo can be a realistic platform for my work and the work of others.
> 
> 
>> Stef
>> 
>>> On 10 Jan 2019, at 15:11, Eliot Miranda <[hidden email]> wrote:
>>> 
>>> Hi Thomas,
>>> 
>>>> On Jan 10, 2019, at 2:24 AM, Thomas Dupriez via Pharo-dev <[hidden email]> wrote:
>>>> 
>>>> <mime-attachment>
>>> 
>>> in a stack of contexts the active pc is different for the top context.  For other than the top context, a context’s pc will be pointing after the send that created the context above it, so to find the pc of the send one finds the previous pc.  For the top context its pc is the active pc.
>>> 
>>> Typically the debugger is invoked in two different modes, interruption or exception. When interrupted, a process is stopped at the next suspension point (method entry or backward branch) and the top context in the process is the context to be displayed in the debugger.  When an exception occurs the exception search machinery will find the signaling context, the context that raised the exception, which will be below the search machinery and the debugger invocation above that. The active pc of the signaling context will be the of for the send of digbsl et al.
>>> 
>>> So the distinction is important and the utility method is probably useful.
>>> 
>>> Do you want to remove the method simply because there are no senders in the image?
>>> 
>>> If so, this is indicative of a serious problem with the Pharo development process.  In the summer I ported VMMaker.oscog to Pharo 6.  Now as feenk try and build a VMMaker.oscog image on Pharo 7, the system is broken, in part because of depreciations and in part because useful methods (isOptimisedBlock (isOptimizedBlock?) in the Opal compiler) have been removed.
>>> 
>>> Just because a method is not in the image does not imply it is not in use.  It simply means that it is not in use in the base image.  As the system gets modularised this issue will only increase.  There are lots of collection methods that exist as a library that are not used in the base image and removing them would clearly damage the library for users.  This is the case for lots of so-called system code.  There are users out there, like those of us in the vm team, who rely on such system code, and it is extremely unsettling and frustrating to have that system code change all the time.  If Pharo is to be a useful platform to the vm team it has to be more stable.
>> 
>> 
>> 

-- 

_,,,^..^,,,_

best, Eliot