Why the hell directives for compiling with clang were not taken into account

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Why the hell directives for compiling with clang were not taken into account

Nicolas Cellier
 
Gah!

despite my stuttutututter command:
    CC=clang ./configure CC=clang
the travis build is completely ignoring the directives and keep using gcc

See:
I suspect that it's lack of clean build, but not sure...

Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
Re-exec as x86
clean? ok but this isn't safe!!
Or would the clang compiler be absent?
Or what?

It ain't fun...

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Holger Freyther
 


> On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:
>
> Gah!
>
> despite my stuttutututter command:
>     CC=clang ./configure CC=clang
> the travis build is completely ignoring the directives and keep using gcc
>
> See:
> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>
>
> I suspect that it's lack of clean build, but not sure...
>
> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
>  <>Re-exec as x86
>  <>clean? ok but this isn't safe!!
> Or would the clang compiler be absent?
> Or what?
>
> It ain't fun...

The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.

* One would need to pass CC to configure and make
* But even for configure CC=clang seems to go missing (https://api.travis-ci.org/v3/job/361174043/log.txt): "checking whether gcc -std=gnu99 is Clang... no"


Try executing the script with shell tracing on "sh -x ./build..." and see if CC is being passed a long? And cat config.log to see which compiler was used during the configuration step?


[1] I have worked with ARM, MIPS, PowerPC, x86 CPUs in my career and I know no other project that needs such build scripts to support different architectures and build flags...
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Nicolas Cellier
 


2018-04-02 16:00 GMT+02:00 Holger Freyther <[hidden email]>:



> On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:
>
> Gah!
>
> despite my stuttutututter command:
>     CC=clang ./configure CC=clang
> the travis build is completely ignoring the directives and keep using gcc
>
> See:
> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>
>
> I suspect that it's lack of clean build, but not sure...
>
> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
>  <>Re-exec as x86
>  <>clean? ok but this isn't safe!!
> Or would the clang compiler be absent?
> Or what?
>
> It ain't fun...


Hi Holger,
 
The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.

me neither, it's super easy to clone those mvm & Makefile and then impossible to manage all the copies...
 

* One would need to pass CC to configure and make
* But even for configure CC=clang seems to go missing (https://api.travis-ci.org/v3/job/361174043/log.txt): "checking whether gcc -std=gnu99 is Clang... no"
 

Try executing the script with shell tracing on "sh -x ./build..." and see if CC is being passed a long? And cat config.log to see which compiler was used during the configuration step?

Thanks for the hints, the problem is that it works well on my debian, and that such remote debugging is a hurdle for me...
 

[1] I have worked with ARM, MIPS, PowerPC, x86 CPUs in my career and I know no other project that needs such build scripts to support different architectures and build flags...

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Ben Coman
In reply to this post by Nicolas Cellier
 


On 2 April 2018 at 21:51, Nicolas Cellier <[hidden email]> wrote:
 
Gah!

despite my stuttutututter command:
    CC=clang ./configure CC=clang
the travis build is completely ignoring the directives and keep using gcc

Maybe... "You can test projects against either GCC or Clang, or both. 
To do so, specify the compiler to use using the 'compiler:' key in .travis.yml."

cheers -ben
 

See:
I suspect that it's lack of clean build, but not sure...

Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
Re-exec as x86
clean? ok but this isn't safe!!
Or would the clang compiler be absent?
Or what?

It ain't fun...



Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

K K Subbu
In reply to this post by Nicolas Cellier
 
On Monday 02 April 2018 07:21 PM, Nicolas Cellier wrote:
>
> Gah!
>
> despite my stuttutututter command:
>      CC=clang ./configure CC=clang
> the travis build is completely ignoring the directives and keep using gcc

I found the following clue in build.win32x86/common/Makefile.tools
-----
  10 ifeq ($(COMPILER_TO_USE),)
  11 COMPILER_TO_USE:=gcc
  12 endif
--------------------------

HTH .. Subbu
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Nicolas Cellier
 
Hi Subbu, thanks,
this apply to windows build only, not linux build
it means that if the variable is not set, then use gcc as default.
But what I did in windows build is to explicitely set COMPILER_TO_USE:=clang in the pharo.sista.spur/Makefile
See
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/37d4a3d39933227e305bf2610a0ec685e6f101d2

2018-04-02 17:40 GMT+02:00 K K Subbu <[hidden email]>:

On Monday 02 April 2018 07:21 PM, Nicolas Cellier wrote:

Gah!

despite my stuttutututter command:
     CC=clang ./configure CC=clang
the travis build is completely ignoring the directives and keep using gcc

I found the following clue in build.win32x86/common/Makefile.tools
-----
 10 ifeq ($(COMPILER_TO_USE),)
 11 COMPILER_TO_USE:=gcc
 12 endif
--------------------------

HTH .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Nicolas Cellier
In reply to this post by Ben Coman
 


2018-04-02 16:15 GMT+02:00 Ben Coman <[hidden email]>:
 


On 2 April 2018 at 21:51, Nicolas Cellier <[hidden email]> wrote:
 
Gah!

despite my stuttutututter command:
    CC=clang ./configure CC=clang
the travis build is completely ignoring the directives and keep using gcc

Maybe... "You can test projects against either GCC or Clang, or both. 
To do so, specify the compiler to use using the 'compiler:' key in .travis.yml."

cheers -ben
 

Waouh, you opened my eyes to a new universe of indirection and complexity!


See:
I suspect that it's lack of clean build, but not sure...

Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
Re-exec as x86
clean? ok but this isn't safe!!
Or would the clang compiler be absent?
Or what?

It ain't fun...





Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

K K Subbu
In reply to this post by Nicolas Cellier
 
The extract was from win32 build only but on the Cog branch. My bad :-(.

ifeq can be tricky. Try replacing the ifeq in Makefile.tools with:

COMPILER_TO_USE ?= gcc

and then try

$ make COMPILER_TO_USE=clang

You could also add this target at the end of Makefile.

echo.% :
        @echo $($(*))

and then run

$ make COMPILER_TO_USE=clang echo.COMPILER_TO_USE

to debug your rules.

HTH .. Subbu

On Monday 02 April 2018 09:15 PM, Nicolas Cellier wrote:

> Hi Subbu, thanks,
> this apply to windows build only, not linux build
> it means that if the variable is not set, then use gcc as default.
> But what I did in windows build is to explicitely set
> COMPILER_TO_USE:=clang in the pharo.sista.spur/Makefile
> See
> https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/37d4a3d39933227e305bf2610a0ec685e6f101d2
>
> 2018-04-02 17:40 GMT+02:00 K K Subbu <[hidden email]
> <mailto:[hidden email]>>:
>
>
>     On Monday 02 April 2018 07:21 PM, Nicolas Cellier wrote:
>
>
>         Gah!
>
>         despite my stuttutututter command:
>               CC=clang ./configure CC=clang
>         the travis build is completely ignoring the directives and keep
>         using gcc
>
>
>     I found the following clue in build.win32x86/common/Makefile.tools
>     -----
>       10 ifeq ($(COMPILER_TO_USE),)
>       11 COMPILER_TO_USE:=gcc
>       12 endif
>     --------------------------
>
>     HTH .. Subbu
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Tobias Pape
In reply to this post by Nicolas Cellier
 

> On 02.04.2018, at 15:51, Nicolas Cellier <[hidden email]> wrote:
>
> Gah!
>
> despite my stuttutututter command:
>     CC=clang ./configure CC=clang
> the travis build is completely ignoring the directives and keep using gcc
>
> See:
> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043
>

Have you seen this?
        https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043#L448

OTOH, I did it that way:
        https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/krono/upgrade-gcc#diff-f36a4fd868404ca84d806453aabb6f3bR65



> I suspect that it's lack of clean build, but not sure...
>
> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
> Re-exec as x86
> clean? ok but this isn't safe!!
> Or would the clang compiler be absent?
> Or what?
>
> It ain't fun...
>

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Nicolas Cellier
 
Hi Tobias,
Ah, that's enlightning !

2018-04-02 19:12 GMT+02:00 Tobias Pape <[hidden email]>:


> On 02.04.2018, at 15:51, Nicolas Cellier <[hidden email]> wrote:
>
> Gah!
>
> despite my stuttutututter command:
>     CC=clang ./configure CC=clang
> the travis build is completely ignoring the directives and keep using gcc
>
> See:
> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043
>

Have you seen this?
        https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043#L448

OTOH, I did it that way:
        https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/krono/upgrade-gcc#diff-f36a4fd868404ca84d806453aabb6f3bR65



> I suspect that it's lack of clean build, but not sure...
>
> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
> Re-exec as x86
> clean? ok but this isn't safe!!
> Or would the clang compiler be absent?
> Or what?
>
> It ain't fun...
>


Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Nicolas Cellier
 
So with the help of Ben and Tobias, I can build linux32x86 pharo.sista.spur on travis now
The right incantation is in .travis.yml thru compiler:clang directive, because it later export CC=clang in root build shell.

Failing to do so exports a CC=gcc default, so overriding via configure command args is not enough, it must be repeated for make and everywhere I did not look yet ;)
It's better to follow the "standard" way.

I'm still waiting the rest of the build to finish, then I will emit a PR.
There is an alternative of using another version of gcc by Tobias which seems to work also.

Anyway, all i did was a quick hack.
Because the mvm is duplicated in other directories (assert and debug version...), there will be more trouble if we not fix it everywhere.
Or maybe the configure CC=clang argument is not required?




2018-04-02 19:38 GMT+02:00 Nicolas Cellier <[hidden email]>:
Hi Tobias,
Ah, that's enlightning !

2018-04-02 19:12 GMT+02:00 Tobias Pape <[hidden email]>:


> On 02.04.2018, at 15:51, Nicolas Cellier <[hidden email]> wrote:
>
> Gah!
>
> despite my stuttutututter command:
>     CC=clang ./configure CC=clang
> the travis build is completely ignoring the directives and keep using gcc
>
> See:
> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043
>

Have you seen this?
        https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043#L448

OTOH, I did it that way:
        https://github.com/OpenSmalltalk/opensmalltalk-vm/compare/krono/upgrade-gcc#diff-f36a4fd868404ca84d806453aabb6f3bR65



> I suspect that it's lack of clean build, but not sure...
>
> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
> Re-exec as x86
> clean? ok but this isn't safe!!
> Or would the clang compiler be absent?
> Or what?
>
> It ain't fun...
>



Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Eliot Miranda-2
In reply to this post by Holger Freyther
 
Hi Holger,

> On Apr 2, 2018, at 7:00 AM, Holger Freyther <[hidden email]> wrote:
>
>
>
>
>> On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:
>>
>> Gah!
>>
>> despite my stuttutututter command:
>>    CC=clang ./configure CC=clang
>> the travis build is completely ignoring the directives and keep using gcc
>>
>> See:
>> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>
>>
>> I suspect that it's lack of clean build, but not sure...
>>
>> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
>> <>Re-exec as x86
>> <>clean? ok but this isn't safe!!
>> Or would the clang compiler be absent?
>> Or what?
>>
>> It ain't fun...
>
> The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.

The complexity is due to the arcane nature of the Linux/Unix build system and its use of autoconf and several scripts to generate makefiles.  I have been saying for years that we need to rewrite the Linux build system to use convention makefiles as per Andreas' Windows makefiles and my subsequent Mac OS Makefiles.  I rewrote the Mac OS build system to use makefiles (*) and you'll notice that
- all share a small suite of makefiles in build.macosXXX/common
- all have a single top-level Makefile that defines the vm variant (the vm source directory) and some branding
- all mvm scripts under build.macosXXX are identical and serve only to invoke make with the three variants, production, assert & debug VMs
Indeed were it not for no symbolic link support for git/svn et al on windows the mvm script would be a symbolic link to a single file

Note that using the makefiles approach eliminates the separate build directories for production, assert & debug VMs


(*) rewriting the Mac OS build system to use makefiles was the path of least resistance when adding building of 64-bit Spur VMs because Xcode project files offer no abstraction and no sharing and are much much worse to edit than the Linux mvm scripts


So instead of bashing the awful little bud build system can we rewrite them together follow the windows and Mac OS scheme?  We would likely end up with one set of makefiles for all Linux builds apart from a separate rules file (compilation flags etc) per platform per word size.  And of course there would be a single mvm script and one would never need to run autoconfig and hence get the phase errors that result from some settings between my made when configure gets built, said me getting made when configure is run and yet others being made when make is run.

>
> * One would need to pass CC to configure and make
> * But even for configure CC=clang seems to go missing (https://api.travis-ci.org/v3/job/361174043/log.txt): "checking whether gcc -std=gnu99 is Clang... no"
>
>
> Try executing the script with shell tracing on "sh -x ./build..." and see if CC is being passed a long? And cat config.log to see which compiler was used during the configuration step?
>
>
> [1] I have worked with ARM, MIPS, PowerPC, x86 CPUs in my career and I know no other project that needs such build scripts to support different architectures and build flags...
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Eliot Miranda-2
 
Hi All,


    and from a related thread:

On Apr 2, 2018, at 12:31 PM, Nicolas Cellier <[hidden email]> wrote:
[snip]
Note that the ARM builds are very slow (not finished after 16 minutes), I think that it was the reason to put them at end of Queue.

the use of just makefiles avoids the configure run and so builds are /much/ faster

_,,,^..^,,,_ (phone)

On Apr 2, 2018, at 3:54 PM, Eliot Miranda <[hidden email]> wrote:

Hi Holger,

On Apr 2, 2018, at 7:00 AM, Holger Freyther <[hidden email]> wrote:




On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:

Gah!

despite my stuttutututter command:
  CC=clang ./configure CC=clang
the travis build is completely ignoring the directives and keep using gcc

See:
https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>

I suspect that it's lack of clean build, but not sure...

Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
<>Re-exec as x86
<>clean? ok but this isn't safe!!
Or would the clang compiler be absent?
Or what?

It ain't fun...

The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.

The complexity is due to the arcane nature of the Linux/Unix build system and its use of autoconf and several scripts to generate makefiles.  I have been saying for years that we need to rewrite the Linux build system to use convention makefiles as per Andreas' Windows makefiles and my subsequent Mac OS Makefiles.  I rewrote the Mac OS build system to use makefiles (*) and you'll notice that
- all share a small suite of makefiles in build.macosXXX/common
- all have a single top-level Makefile that defines the vm variant (the vm source directory) and some branding
- all mvm scripts under build.macosXXX are identical and serve only to invoke make with the three variants, production, assert & debug VMs
Indeed were it not for no symbolic link support for git/svn et al on windows the mvm script would be a symbolic link to a single file

Note that using the makefiles approach eliminates the separate build directories for production, assert & debug VMs


(*) rewriting the Mac OS build system to use makefiles was the path of least resistance when adding building of 64-bit Spur VMs because Xcode project files offer no abstraction and no sharing and are much much worse to edit than the Linux mvm scripts


So instead of bashing the awful little bud build system can we rewrite them together follow the windows and Mac OS scheme?  We would likely end up with one set of makefiles for all Linux builds apart from a separate rules file (compilation flags etc) per platform per word size.  And of course there would be a single mvm script and one would never need to run autoconfig and hence get the phase errors that result from some settings between my made when configure gets built, said me getting made when configure is run and yet others being made when make is run.


* One would need to pass CC to configure and make
* But even for configure CC=clang seems to go missing (https://api.travis-ci.org/v3/job/361174043/log.txt): "checking whether gcc -std=gnu99 is Clang... no"


Try executing the script with shell tracing on "sh -x ./build..." and see if CC is being passed a long? And cat config.log to see which compiler was used during the configuration step?


[1] I have worked with ARM, MIPS, PowerPC, x86 CPUs in my career and I know no other project that needs such build scripts to support different architectures and build flags...
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

David T. Lewis
In reply to this post by Eliot Miranda-2
 
On Mon, Apr 02, 2018 at 03:54:15PM -0700, Eliot Miranda wrote:

>  
> Hi Holger,
>
> > On Apr 2, 2018, at 7:00 AM, Holger Freyther <[hidden email]> wrote:
> >
> >> On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:
> >>
> >> Gah!
> >>
> >> despite my stuttutututter command:
> >>    CC=clang ./configure CC=clang
> >> the travis build is completely ignoring the directives and keep using gcc
> >>
> >> See:
> >> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>
> >>
> >> I suspect that it's lack of clean build, but not sure...
> >>
> >> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
> >> <>Re-exec as x86
> >> <>clean? ok but this isn't safe!!
> >> Or would the clang compiler be absent?
> >> Or what?
> >>
> >> It ain't fun...
> >
> > The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.
>
> The complexity is due to the arcane nature of the Linux/Unix build system and its use of autoconf and several scripts to generate makefiles.  I have been saying for years that we need to rewrite the Linux build system to use convention makefiles as per Andreas' Windows makefiles and my subsequent Mac OS Makefiles.  I rewrote the Mac OS build system to use makefiles (*) and you'll notice that
> - all share a small suite of makefiles in build.macosXXX/common
> - all have a single top-level Makefile that defines the vm variant (the vm source directory) and some branding
> - all mvm scripts under build.macosXXX are identical and serve only to invoke make with the three variants, production, assert & debug VMs

Hi Eliot,

To be perfectly honest, I do not know what it meant by "we need to rewrite the Linux
build system to use conventional makefiles". Using makefiles would be a given for any
build system on unix/Linux, the differences (and pain points) are in the configuration
step. Regardless of whether you use autoconf or CMake or something else, we would still
need something to specify the environment in which the makefile is operating. In other
words, we need a build/config.h to specify the things that the makefile does not know
about. I am pretty sure I am misunderstanding something in this discussion, but I am
comfused and I may as well admit it.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

tblanchard
 
Speaking of which, I would love it if CMake were used if for no other reason CMake can generate an Xcode project which would make for much easier debugging and porting to iOS.



On Apr 2, 2018, at 8:11 PM, David T. Lewis <[hidden email]> wrote:

Regardless of whether you use autoconf or CMake or something else, we would still
need something to specify the environment in which the makefile is operating.

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Eliot Miranda-2
In reply to this post by David T. Lewis
 
Hi David,

> On Apr 2, 2018, at 8:11 PM, David T. Lewis <[hidden email]> wrote:
>
>
>> On Mon, Apr 02, 2018 at 03:54:15PM -0700, Eliot Miranda wrote:
>>
>> Hi Holger,
>>
>>>> On Apr 2, 2018, at 7:00 AM, Holger Freyther <[hidden email]> wrote:
>>>>
>>>> On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:
>>>>
>>>> Gah!
>>>>
>>>> despite my stuttutututter command:
>>>>   CC=clang ./configure CC=clang
>>>> the travis build is completely ignoring the directives and keep using gcc
>>>>
>>>> See:
>>>> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>
>>>>
>>>> I suspect that it's lack of clean build, but not sure...
>>>>
>>>> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
>>>> <>Re-exec as x86
>>>> <>clean? ok but this isn't safe!!
>>>> Or would the clang compiler be absent?
>>>> Or what?
>>>>
>>>> It ain't fun...
>>>
>>> The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.
>>
>> The complexity is due to the arcane nature of the Linux/Unix build system and its use of autoconf and several scripts to generate makefiles.  I have been saying for years that we need to rewrite the Linux build system to use convention makefiles as per Andreas' Windows makefiles and my subsequent Mac OS Makefiles.  I rewrote the Mac OS build system to use makefiles (*) and you'll notice that
>> - all share a small suite of makefiles in build.macosXXX/common
>> - all have a single top-level Makefile that defines the vm variant (the vm source directory) and some branding
>> - all mvm scripts under build.macosXXX are identical and serve only to invoke make with the three variants, production, assert & debug VMs
>
> Hi Eliot,
>
> To be perfectly honest, I do not know what it meant by "we need to rewrite the Linux
> build system to use conventional makefiles". Using makefiles would be a given for any
> build system on unix/Linux, the differences (and pain points) are in the configuration
> step. Regardless of whether you use autoconf or CMake or something else, we would still
> need something to specify the environment in which the makefile is operating. In other
> words, we need a build/config.h to specify the things that the makefile does not know
> about. I am pretty sure I am misunderstanding something in this discussion, but I am
> comfused and I may as well admit it.

I'm feeling beaten down.  Perhaps you could read the code?  Compare what's in
    platforms/unix/config
with what's in
    build.macos32x86/common
    build.macos64x64/common
    build.win32x86/common
    build.win64x64/common

And then think about the "necessary configure step".  To what extent does it make sense to run a configure step on a build machine that makes a binary that is downloaded and used on machines that may differ markedly from that build machine?



>
> Dave
>
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

David T. Lewis
 
On Tue, Apr 03, 2018 at 05:30:06AM -0700, Eliot Miranda wrote:

>  
> Hi David,
>
> > On Apr 2, 2018, at 8:11 PM, David T. Lewis <[hidden email]> wrote:
> >
> >
> >> On Mon, Apr 02, 2018 at 03:54:15PM -0700, Eliot Miranda wrote:
> >>
> >> Hi Holger,
> >>
> >>>> On Apr 2, 2018, at 7:00 AM, Holger Freyther <[hidden email]> wrote:
> >>>>
> >>>> On 2. Apr 2018, at 14:51, Nicolas Cellier <[hidden email]> wrote:
> >>>>
> >>>> Gah!
> >>>>
> >>>> despite my stuttutututter command:
> >>>>   CC=clang ./configure CC=clang
> >>>> the travis build is completely ignoring the directives and keep using gcc
> >>>>
> >>>> See:
> >>>> https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043 <https://travis-ci.org/OpenSmalltalk/opensmalltalk-vm/jobs/361174043>
> >>>>
> >>>> I suspect that it's lack of clean build, but not sure...
> >>>>
> >>>> Building OpenSmalltalk VM in /home/travis/build/OpenSmalltalk/opensmalltalk-vm/build.linux32x86/pharo.sista.spur/build.itimerheartbeat...
> >>>> <>Re-exec as x86
> >>>> <>clean? ok but this isn't safe!!
> >>>> Or would the clang compiler be absent?
> >>>> Or what?
> >>>>
> >>>> It ain't fun...
> >>>
> >>> The cost of abstraction... I am not a fan of the "build" scripts (a lot of code cloning for what[1] gain?) and the travis logs don't show us the invocation of the actual commands.
> >>
> >> The complexity is due to the arcane nature of the Linux/Unix build system and its use of autoconf and several scripts to generate makefiles.  I have been saying for years that we need to rewrite the Linux build system to use convention makefiles as per Andreas' Windows makefiles and my subsequent Mac OS Makefiles.  I rewrote the Mac OS build system to use makefiles (*) and you'll notice that
> >> - all share a small suite of makefiles in build.macosXXX/common
> >> - all have a single top-level Makefile that defines the vm variant (the vm source directory) and some branding
> >> - all mvm scripts under build.macosXXX are identical and serve only to invoke make with the three variants, production, assert & debug VMs
> >
> > Hi Eliot,
> >
> > To be perfectly honest, I do not know what it meant by "we need to rewrite the Linux
> > build system to use conventional makefiles". Using makefiles would be a given for any
> > build system on unix/Linux, the differences (and pain points) are in the configuration
> > step. Regardless of whether you use autoconf or CMake or something else, we would still
> > need something to specify the environment in which the makefile is operating. In other
> > words, we need a build/config.h to specify the things that the makefile does not know
> > about. I am pretty sure I am misunderstanding something in this discussion, but I am
> > comfused and I may as well admit it.
>
> I'm feeling beaten down.  Perhaps you could read the code?  Compare what's in
>     platforms/unix/config
> with what's in
>     build.macos32x86/common
>     build.macos64x64/common
>     build.win32x86/common
>     build.win64x64/common
>
> And then think about the "necessary configure step".  To what extent does it make sense to run a configure step on a build machine that makes a binary that is downloaded and used on machines that may differ markedly from that build machine?
>

Hi Eliot,

First an apology - this was not a very good time to be asking a dumb question.

Yes I did look at the code, but probably not closely enough. I will do so.

Dave

Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

K K Subbu
In reply to this post by Eliot Miranda-2
 
On Tuesday 03 April 2018 06:00 PM, Eliot Miranda wrote:
> I'm feeling beaten down.  Perhaps you could read the code?  Compare
> what's in platforms/unix/config with what's in
> build.macos32x86/common build.macos64x64/common build.win32x86/common
> build.win64x64/common
>
> And then think about the "necessary configure step".  To what extent
> does it make sense to run a configure step on a build machine that
> makes a binary that is downloaded and used on machines that may
> differ markedly from that build machine?

The configure script in Autoconf accumulates lots of platform-fu (see
configure:5668) for all the UNIX/Linux variants and quirks.

Mac or Win ports have to deal only with a handful of variants, so
hand-crafted Makefiles are sufficient. But Unix/GNU/Linux ports are so
numerous and diverse that Ian Piumarta (?) brought in autoconf to ease
porting Squeak to all these variants.

If we don't intend to support all the 100+ variants of GNU/UNIX/Linux
out there but only a handful, then manual makefiles are sufficient.

GNU Make is available on GNU/Linux/Mac/Win so we could create a single
top level Makefile for all three platform builds and let the configure
handle other ports. CMake is another possibility but it requires meta
makefiles to be created from scratch :-(.

Regards .. Subbu
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

Eliot Miranda-2
 
Hi Subbu

> On Apr 3, 2018, at 7:34 AM, K K Subbu <[hidden email]> wrote:
>
>> On Tuesday 03 April 2018 06:00 PM, Eliot Miranda wrote:
>> I'm feeling beaten down.  Perhaps you could read the code?  Compare what's in platforms/unix/config with what's in build.macos32x86/common build.macos64x64/common build.win32x86/common
>> build.win64x64/common
>> And then think about the "necessary configure step".  To what extent does it make sense to run a configure step on a build machine that makes a binary that is downloaded and used on machines that may differ markedly from that build machine?
>
> The configure script in Autoconf accumulates lots of platform-fu (see
> configure:5668) for all the UNIX/Linux variants and quirks.

Right.

>
> Mac or Win ports have to deal only with a handful of variants, so
> hand-crafted Makefiles are sufficient. But Unix/GNU/Linux ports are so
> numerous and diverse that Ian Piumarta (?) brought in autoconf to ease
> porting Squeak to all these variants.

I see things a little differently.  I don't want to speak for Ian, but I expect that the reason Ian went the way he did was more to do with the fact that to run Squeak one had to build a vm oneself. Being a Unix expert Ian could support a build of (at first just) an X11 vm across a wide range of Unix variants at a time (the early 90's) when many manufactures were oroducing Unix workstations which were widely used, especially in academia.  That predated a lot of culture we now take god granted, the downloading of prepackaged releases, automated build and test environments, and a much reduced more consumer-oriented laptop market.

> If we don't intend to support all the 100+ variants of GNU/UNIX/Linux out there but only a handful, then manual makefiles are sufficient.

It's not that we do of do not intend, it's that we simply cannot afford to and it is not relevant to our current needs.  We instead need to support a much smaller number of operating systems well, namely Linux and maybe FreeBSD, although I don't see much energy there.  We could keep the old build system for those that want to build their own vm on something exotic (although I'd still argue that they would do as well working with conventional makefiles).  But it is no longer fit for purpose in the continuous integration context we now inhabit.

>
> GNU Make is available on GNU/Linux/Mac/Win so we could create a single top level Makefile for all three platform builds and let the configure
> handle other ports.

Having modified Andreas' makefiles for windows, and having written the makefiles for the Mac build I can say that a common top level makefiles is not a good idea.  It would be even more of a mess of ifdefs.  But certain not following the way nudes and Mac OS schemes is sensible.  The important thing is to get rid of the extremely hard to modify, slow autoconf based system used for Linux builds.

> CMake is another possibility but it requires meta makefiles to be created from scratch :-(.

In a vm text where build slaves build binaries god more general consumption the specificity of CMake and autoconfig is of marginal utility.  The Windows and Mac OS sources still have config.h's; they simply aren't generated on every build.  The information in them doesn't change in practice, except when a major change, such as the addition of s 64-bit build, occurs, and at that point a lot more work than merely updating config.h is needed (although IIRC  the mac's config.h didn't change when adding 64-bit Spur).

The main utility in CMake and autoconf is /not/ capturing platform fu, but in capturing what ancillary packages have been installed to know what additional plugins to build; for example the 3D plugins are only built if OpenGL is installed.  But in our CI infrastructure we make damn sure OpenGL is installed on the build machines so we. An provide VMs comprising a full set of plugins, some of which may not work on the user's machine because they haven't installed the libraries.

Let's have a build system that suits where we are now, where the vm is a utility we want to build reliably and download, not something we get a frisson form building ourselves.  Let's leave that yo those that work on the vm and let the rest of us get timely CI vm builds we can download at our leisure.

> Regards .. Subbu

Cheers!
Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Why the hell directives for compiling with clang were not taken into account

K K Subbu
 
On Tuesday 03 April 2018 08:54 PM, Eliot Miranda wrote:

>> If we don't intend to support all the 100+ variants of
>> GNU/UNIX/Linux out there but only a handful, then manual makefiles
>> are sufficient.
>
> It's not that we do of do not intend, it's that we simply cannot
> afford to and it is not relevant to our current needs.  We instead
> need to support a much smaller number of operating systems well,
> namely Linux and maybe FreeBSD, although I don't see much energy
> there.  We could keep the old build system for those that want to
> build their own vm on something exotic (although I'd still argue that
> they would do as well working with conventional makefiles).  But it
> is no longer fit for purpose in the continuous integration context we
> now inhabit.

Valid point. The CI pipeline requires a different approach.

>>
>> GNU Make is available on GNU/Linux/Mac/Win so we could create a
>> single top level Makefile for all three platform builds and let the
>> configure handle other ports.
>
> Having modified Andreas' makefiles for windows, and having written
> the makefiles for the Mac build I can say that a common top level
> makefiles is not a good idea.  It would be even more of a mess of
> ifdefs.  But certain not following the way nudes and Mac OS schemes
> is sensible.  The important thing is to get rid of the extremely hard
> to modify, slow autoconf based system used for Linux builds.

A top level Makefile does not exclude platform-specific Makefiles, nor
should it attempt a cross-platform build by itself.

The purpose of top level Makefile is general help, handle version tags,
possible products and variants etc. It should delegate the actual build
to platform-specific make. Similar to how we use OSProcess as a facade
over OSUnixProcess etc.

In fact, I am making one for my own use. Once it stabilizes, I will
offer a PR.

The downsides to spreading Makefiles into build dirs is explosion of
directories under version control and duplication of code. It also means
that I cannot mount build on tmpfs/ramfs. Keeping Makefile templates
under platform/win32/.. and copying them to build/ as needed will
simplify version control and avoid duplicate code.

Regards .. Subbu
12