build.macos32x86/pharo.cog.spur and dependency tracking

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

build.macos32x86/pharo.cog.spur and dependency tracking

Holger Freyther
 
Hey,

I am working on the macos VM right now and I wondered if that is normal/to be expected. When typing make in an already built VM with no modifications I get 655 lines of output and it takes about 7.2s realtime. Is this to be expected? Can this to be improved? No output and 1s?

thank you

        holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Eliot Miranda-2
 
Hi Holger,

> On Aug 25, 2017, at 2:46 AM, Holger Freyther <[hidden email]> wrote:
>
>
> Hey,
>
> I am working on the macos VM right now and I wondered if that is normal/to be expected. When typing make in an already built VM with no modifications I get 655 lines of output and it takes about 7.2s realtime. Is this to be expected?

Yes.  The extra output is due to spawning a sub make for each plugin and/or to check that each support library is built.

> Can this to be improved? No output and 1s?

Maybe it can.  But if that spawn was silent it might be more confusing to solve errors and it might be more difficult to figure out how to spawn a sub make.  But feel free to alter it.

>
> thank you
>
>    holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Holger Freyther
 

> On 25. Aug 2017, at 22:29, Eliot Miranda <[hidden email]> wrote:
>


Hey Eliot!


thank you for the reply.

> Yes.  The extra output is due to spawning a sub make for each plugin and/or to check that each support library is built.

I have not looked at the specific Makefile but how far away is the dependency tracking from something like that.

PluginFoo.so: first.o second.o
        $(LINK)

-include first.o.deps (to track header dependencies)?
-include second.o.deps



>> Can this to be improved? No output and 1s?
>
> Maybe it can.  But if that spawn was silent it might be more confusing to solve errors and it might be more difficult to figure out how to spawn a sub make.  But feel free to alter it.

The turn-a-round time would be more important than the output but I feel that there is a reltionship. I will have a look.


        holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Eliot Miranda-2
 
Hi Holger,

> On Aug 25, 2017, at 7:42 AM, Holger Freyther <[hidden email]> wrote:
>
>
>
>> On 25. Aug 2017, at 22:29, Eliot Miranda <[hidden email]> wrote:
>>
>
>
> Hey Eliot!
>
>
> thank you for the reply.
>
>> Yes.  The extra output is due to spawning a sub make for each plugin and/or to check that each support library is built.
>
> I have not looked at the specific Makefile but how far away is the dependency tracking from something like that.
>
> PluginFoo.so: first.o second.o
>    $(LINK)
>
> -include first.o.deps    (to track header dependencies)?
> -include second.o.deps

A fair distance away :-). Each plugin and its support files get built with a separate make step which allows an external plugin to specify additional support libraries, include paths, etc.

>>> Can this to be improved? No output and 1s?
>>
>> Maybe it can.  But if that spawn was silent it might be more confusing to solve errors and it might be more difficult to figure out how to spawn a sub make.  But feel free to alter it.
>
> The turn-a-round time would be more important than the output but I feel that there is a reltionship. I will have a look.

You could modify the mvm script to just log output and not send it to the console.  That might speed things up.  Then if there are errors the script could output the log files.

>
>
>    holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Holger Freyther
 

> On 25. Aug 2017, at 23:07, Eliot Miranda <[hidden email]> wrote:
>
>
> Hi Holger,


>
> A fair distance away :-). Each plugin and its support files get built with a separate make step which allows an external plugin to specify additional support libraries, include paths, etc.

okay, so a recursive make. But in each target it seems to unconditionally execute the linking step? Why is that necessary? What dependency might not be easily trackable?

holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Eliot Miranda-2
 
Hi Holger,

> On Aug 25, 2017, at 8:10 AM, Holger Freyther <[hidden email]> wrote:
>
>
>
>> On 25. Aug 2017, at 23:07, Eliot Miranda <[hidden email]> wrote:
>>
>>
>> Hi Holger,
>
>
>>
>> A fair distance away :-). Each plugin and its support files get built with a separate make step which allows an external plugin to specify additional support libraries, include paths, etc.
>
> okay, so a recursive make. But in each target it seems to unconditionally execute the linking step? Why is that necessary?

Clearly it isn't.  If it does always link then it's a bug.  But beware; there needs to be dependencies on things like the makefile, plugins.int, etc to get things to be remade when things change.  I remember I had difficulties here when I was first using the makefiles.  So if you try and fix this check that you can move some plugin from plugins.int to plugins.ext and vice verse and the relevant files do get rebuilt appropriately.

> What dependency might not be easily trackable?
>
> holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Holger Freyther
 

> On 25. Aug 2017, at 23:59, Eliot Miranda <[hidden email]> wrote:
>
>
> Hi Holger,


Hey Eliot,

> Clearly it isn't.  If it does always link then it's a bug.  But beware; there needs to be dependencies on things like the makefile, plugins.int, etc to get things to be remade when things change.  I remember I had difficulties here when I was first using the makefiles.  So if you try and fix this check that you can move some plugin from plugins.int to plugins.ext and vice verse and the relevant files do get rebuilt appropriately.


"re-linking" was not the right word for the internal plugin but a new ar archive is created all the time? It seems it is because the rule is forced to re-execute

Makefile.vm:

$(OBJDIR)/%.lib: FORCE
...


I understand why more is re-built than necessary but I do wonder about why was a recursive system chosen? I wanted to propose to write a %.lib.d dependency but it might depend more on than just the .c .d and Makefile.plugin so I would have to duplicate logic into Makefile.vm. Just removing the FORCE (and being aware of the potential issues) cuts the build time on a clean directory in half. Would you have objections to make it a non-recursive build?


have a nice weekend

        holger
Reply | Threaded
Open this post in threaded view
|

Re: build.macos32x86/pharo.cog.spur and dependency tracking

Eliot Miranda-2
 
Hi Holger,


> On Aug 26, 2017, at 8:55 AM, Holger Freyther <[hidden email]> wrote:
>
>
>
>> On 25. Aug 2017, at 23:59, Eliot Miranda <[hidden email]> wrote:
>>
>>
>> Hi Holger,
>
>
> Hey Eliot,
>
>> Clearly it isn't.  If it does always link then it's a bug.  But beware; there needs to be dependencies on things like the makefile, plugins.int, etc to get things to be remade when things change.  I remember I had difficulties here when I was first using the makefiles.  So if you try and fix this check that you can move some plugin from plugins.int to plugins.ext and vice verse and the relevant files do get rebuilt appropriately.
>
>
> "re-linking" was not the right word for the internal plugin but a new ar archive is created all the time? It seems it is because the rule is forced to re-execute
>
> Makefile.vm:
>
> $(OBJDIR)/%.lib: FORCE
> ...

The necessity is that the vm correctly builds when a plugin is moved from plugins.int and plugins.ext and vice verse.  The above is a hack which I used to meet the constraint.  If you can meet the constraint without the hack then change it.

> I understand why more is re-built than necessary but I do wonder about why was a recursive system chosen? I wanted to propose to write a %.lib.d dependency but it might depend more on than just the .c .d and Makefile.plugin so I would have to duplicate logic into Makefile.vm. Just removing the FORCE (and being aware of the potential issues) cuts the build time on a clean directory in half. Would you have objections to make it a non-recursive build?

None.  Again the constraint is that individual plugin directories (in this case in platforms/iOS/plugins/XXX/Makefile.plugin, bug similarly in platforms/win32/plugins et al) can override defaults to pull in additional include files, support libraries, linker (C vs C++) and support builds (in the Pharo vm).

> have a nice weekend
>
>    holger