Rant about generated files: should they stay in repository?
Locked
Rant about generated files: should they stay in repository?
 
|
I am against it , because it is going to nowhere. So let me elaborate... In Cog, we have an image which generating an universal sources for VM on all platforms. These generated files then put under source control. Pros: - to build VM you can just checkout and build Cons: - err , wait.. which exactly VM you going to build? can you use same sources for different builds? Yes, I can use same generated sources for everything, but depending on configuration and/or target operating system i can receive completely different results. Because, for recreating VM as an artifact to reproduce same circumstances as they were (when some bug was triggered) a generated sources snapshot is not enough. Simple example: - VM compiled for Macs using -O3 optimization are stable. In contrary to linux, where using -O2 with certain files makes your VM crash few (milli)seconds after startup. Second ( i don't like repeating myself but still ;): - the only way to make sure that VMMaker loads into our latest and greatest image(s) and producing same output(s) is to have someone to continuously try load it, then use it to generate sources, then compile the VM and then run tests for that VM. We have a generous guys in our community, who can do that for us. This is a Mr Hudson, and Mr Jenkins that joined party lately. I'd like to stress that generated files are just an intermediate data, which nobody interested in. Because as input what is relevant should be only : - a VMMaker - a platform-specific sources/libs - a build configuration and as output we always have: - VM what happens in the middle is really unimportant as long as you can guarantee reproducibility of the same output from same inputs. VM functioning should not depend on some random noise which comes from unrelated sources like - generated source files line ending convention - the date stamp when VM was built - anything which is not listed in the inputs list above and if it does, we should eliminate it. Because if it does, then you don't have a stable builds. And having a snapshot of sources which kinda works and kinda stable worth nothing, because at any moment in ongoing development you can meet same issue which will pop up out of nowhere. And it will pop out exactly because you don't care to always do things from scratch , starting from image and VMMaker. Because it is so convenient to just pull the generated sources from repo and you done. And those silly stupids which trying to use VMMaker are not of your concern.. because when they fail to build stable VM (and they will), you can always tell them that they are using wrong sources and because you can't reproduce it (things is always working well for you, in your warm corner :) So, i think that keeping a generated files works as some kind of defensive policy which doesn't allows you to address certain problems for real. And sure thing, because we are lazy :) -- Best regards, Igor Stasenko AKA sig. |
Re: Rant about generated files: should they stay in repository?
|
|
On Fri, 11 Mar 2011, Igor Stasenko wrote: > > I am against it , because it is going to nowhere. > > So let me elaborate... > > In Cog, we have an image which generating an universal sources for VM > on all platforms. > These generated files then put under source control. > > Pros: > - to build VM you can just checkout and build > > Cons: > - err , wait.. which exactly VM you going to build? can you use same > sources for different builds? It depends on the revision, doesn't it? If the sources are generated for every build (which is the case), then you can check out any version you want and you'll get matching platform files and generated sources. > > Yes, I can use same generated sources for everything, but depending on > configuration and/or target operating system i can receive completely > different results. > > Because, for recreating VM as an artifact to reproduce same > circumstances as they were (when some bug was triggered) a generated > sources snapshot is not enough. > > Simple example: > - VM compiled for Macs using -O3 optimization are stable. > In contrary to linux, where using -O2 with certain files makes your > VM crash few (milli)seconds after startup. So, will generating the sources from VMMaker magically fix all bugs in the C compiler, libraries and OS? > > Second ( i don't like repeating myself but still ;): > - the only way to make sure that VMMaker loads into our latest and > greatest image(s) and producing same output(s) is to have someone to > continuously try load it, > then use it to generate sources, then compile the VM and then run > tests for that VM. This is image dependent. Will you add all images to your build server to make sure if VMMaker works in all of them? > We have a generous guys in our community, who can do that for us. This > is a Mr Hudson, and Mr Jenkins that joined party lately. > > I'd like to stress that generated files are just an intermediate data, > which nobody interested in. > > Because as input what is relevant should be only : > - a VMMaker > - a platform-specific sources/libs > - a build configuration > > and as output we always have: > - VM > > what happens in the middle is really unimportant as long as you can > guarantee reproducibility of the same output from same inputs. > > VM functioning should not depend on some random noise which comes from > unrelated sources like > - generated source files line ending convention > - the date stamp when VM was built > - anything which is not listed in the inputs list above > and if it does, we should eliminate it. > > Because if it does, then you don't have a stable builds. And having a > snapshot of sources which kinda works and kinda stable > worth nothing, because at any moment in ongoing development you can > meet same issue which will pop up out of nowhere. > And it will pop out exactly because you don't care to always do things > from scratch , starting from image and VMMaker. > Because it is so convenient to just pull the generated sources from > repo and you done. > And those silly stupids which trying to use VMMaker are not of your > concern.. because when they fail to build stable VM (and they will), > you can always tell them that they are using wrong sources and because > you can't reproduce it (things is always working well for you, in your > warm corner :) > > So, i think that keeping a generated files works as some kind of > defensive policy which doesn't allows you to address certain > problems for real. > And sure thing, because we are lazy :) This is nonsense IMHO. You don't have to use the intermediate sources, you can generate the C sources from VMMaker if you want. And if you find a problem related to "line ending convention" or "the date stamp when VM was built", then you'll have a chance to find the cause of it by comparing your generated sources with the sources in svn. Levente > > > -- > Best regards, > Igor Stasenko AKA sig. > |
|
|
In reply to this post by Igor Stasenko
Igor, I'm with you :) let's get rid of generated sources from repository! best, Esteban El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: > > I am against it , because it is going to nowhere. > > So let me elaborate... > > In Cog, we have an image which generating an universal sources for VM > on all platforms. > These generated files then put under source control. > > Pros: > - to build VM you can just checkout and build > > Cons: > - err , wait.. which exactly VM you going to build? can you use same > sources for different builds? > > Yes, I can use same generated sources for everything, but depending on > configuration and/or target operating system i can receive completely > different results. > > Because, for recreating VM as an artifact to reproduce same > circumstances as they were (when some bug was triggered) a generated > sources snapshot is not enough. > > Simple example: > - VM compiled for Macs using -O3 optimization are stable. > In contrary to linux, where using -O2 with certain files makes your > VM crash few (milli)seconds after startup. > > Second ( i don't like repeating myself but still ;): > - the only way to make sure that VMMaker loads into our latest and > greatest image(s) and producing same output(s) is to have someone to > continuously try load it, > then use it to generate sources, then compile the VM and then run > tests for that VM. > We have a generous guys in our community, who can do that for us. This > is a Mr Hudson, and Mr Jenkins that joined party lately. > > I'd like to stress that generated files are just an intermediate data, > which nobody interested in. > > Because as input what is relevant should be only : > - a VMMaker > - a platform-specific sources/libs > - a build configuration > > and as output we always have: > - VM > > what happens in the middle is really unimportant as long as you can > guarantee reproducibility of the same output from same inputs. > > VM functioning should not depend on some random noise which comes from > unrelated sources like > - generated source files line ending convention > - the date stamp when VM was built > - anything which is not listed in the inputs list above > and if it does, we should eliminate it. > > Because if it does, then you don't have a stable builds. And having a > snapshot of sources which kinda works and kinda stable > worth nothing, because at any moment in ongoing development you can > meet same issue which will pop up out of nowhere. > And it will pop out exactly because you don't care to always do things > from scratch , starting from image and VMMaker. > Because it is so convenient to just pull the generated sources from > repo and you done. > And those silly stupids which trying to use VMMaker are not of your > concern.. because when they fail to build stable VM (and they will), > you can always tell them that they are using wrong sources and because > you can't reproduce it (things is always working well for you, in your > warm corner :) > > So, i think that keeping a generated files works as some kind of > defensive policy which doesn't allows you to address certain > problems for real. > And sure thing, because we are lazy :) > > > -- > Best regards, > Igor Stasenko AKA sig. |
Re: Rant about generated files: should they stay in repository?
|
|
We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html frank On 2011/03/11 11:34, Esteban Lorenzano wrote: > > Igor, I'm with you :) > let's get rid of generated sources from repository! > > best, > Esteban > > El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: > >> >> I am against it , because it is going to nowhere. >> >> So let me elaborate... >> >> In Cog, we have an image which generating an universal sources for VM >> on all platforms. >> These generated files then put under source control. >> >> Pros: >> - to build VM you can just checkout and build >> >> Cons: >> - err , wait.. which exactly VM you going to build? can you use same >> sources for different builds? >> >> Yes, I can use same generated sources for everything, but depending on >> configuration and/or target operating system i can receive completely >> different results. >> >> Because, for recreating VM as an artifact to reproduce same >> circumstances as they were (when some bug was triggered) a generated >> sources snapshot is not enough. >> >> Simple example: >> - VM compiled for Macs using -O3 optimization are stable. >> In contrary to linux, where using -O2 with certain files makes your >> VM crash few (milli)seconds after startup. >> >> Second ( i don't like repeating myself but still ;): >> - the only way to make sure that VMMaker loads into our latest and >> greatest image(s) and producing same output(s) is to have someone to >> continuously try load it, >> then use it to generate sources, then compile the VM and then run >> tests for that VM. >> We have a generous guys in our community, who can do that for us. This >> is a Mr Hudson, and Mr Jenkins that joined party lately. >> >> I'd like to stress that generated files are just an intermediate data, >> which nobody interested in. >> >> Because as input what is relevant should be only : >> - a VMMaker >> - a platform-specific sources/libs >> - a build configuration >> >> and as output we always have: >> - VM >> >> what happens in the middle is really unimportant as long as you can >> guarantee reproducibility of the same output from same inputs. >> >> VM functioning should not depend on some random noise which comes from >> unrelated sources like >> - generated source files line ending convention >> - the date stamp when VM was built >> - anything which is not listed in the inputs list above >> and if it does, we should eliminate it. >> >> Because if it does, then you don't have a stable builds. And having a >> snapshot of sources which kinda works and kinda stable >> worth nothing, because at any moment in ongoing development you can >> meet same issue which will pop up out of nowhere. >> And it will pop out exactly because you don't care to always do things >> from scratch , starting from image and VMMaker. >> Because it is so convenient to just pull the generated sources from >> repo and you done. >> And those silly stupids which trying to use VMMaker are not of your >> concern.. because when they fail to build stable VM (and they will), >> you can always tell them that they are using wrong sources and because >> you can't reproduce it (things is always working well for you, in your >> warm corner :) >> >> So, i think that keeping a generated files works as some kind of >> defensive policy which doesn't allows you to address certain >> problems for real. >> And sure thing, because we are lazy :) >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > > |
|
|
I wasn't aware of that discussion, but I'm still disagree with their conclusions. I'm certain that sources must be versioned. But sources are: VMMaker + configuration + platform files... generated files are not sources and to keep them in source tree is prone to error. That's my feeling about this... anyway, I'm also not sure about re-discuss this :P best, Esteban El 11/03/2011, a las 8:50a.m., Frank Shearar escribió: > We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html > > frank > > On 2011/03/11 11:34, Esteban Lorenzano wrote: >> >> Igor, I'm with you :) >> let's get rid of generated sources from repository! >> >> best, >> Esteban >> >> El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: >> >>> >>> I am against it , because it is going to nowhere. >>> >>> So let me elaborate... >>> >>> In Cog, we have an image which generating an universal sources for VM >>> on all platforms. >>> These generated files then put under source control. >>> >>> Pros: >>> - to build VM you can just checkout and build >>> >>> Cons: >>> - err , wait.. which exactly VM you going to build? can you use same >>> sources for different builds? >>> >>> Yes, I can use same generated sources for everything, but depending on >>> configuration and/or target operating system i can receive completely >>> different results. >>> >>> Because, for recreating VM as an artifact to reproduce same >>> circumstances as they were (when some bug was triggered) a generated >>> sources snapshot is not enough. >>> >>> Simple example: >>> - VM compiled for Macs using -O3 optimization are stable. >>> In contrary to linux, where using -O2 with certain files makes your >>> VM crash few (milli)seconds after startup. >>> >>> Second ( i don't like repeating myself but still ;): >>> - the only way to make sure that VMMaker loads into our latest and >>> greatest image(s) and producing same output(s) is to have someone to >>> continuously try load it, >>> then use it to generate sources, then compile the VM and then run >>> tests for that VM. >>> We have a generous guys in our community, who can do that for us. This >>> is a Mr Hudson, and Mr Jenkins that joined party lately. >>> >>> I'd like to stress that generated files are just an intermediate data, >>> which nobody interested in. >>> >>> Because as input what is relevant should be only : >>> - a VMMaker >>> - a platform-specific sources/libs >>> - a build configuration >>> >>> and as output we always have: >>> - VM >>> >>> what happens in the middle is really unimportant as long as you can >>> guarantee reproducibility of the same output from same inputs. >>> >>> VM functioning should not depend on some random noise which comes from >>> unrelated sources like >>> - generated source files line ending convention >>> - the date stamp when VM was built >>> - anything which is not listed in the inputs list above >>> and if it does, we should eliminate it. >>> >>> Because if it does, then you don't have a stable builds. And having a >>> snapshot of sources which kinda works and kinda stable >>> worth nothing, because at any moment in ongoing development you can >>> meet same issue which will pop up out of nowhere. >>> And it will pop out exactly because you don't care to always do things >>> from scratch , starting from image and VMMaker. >>> Because it is so convenient to just pull the generated sources from >>> repo and you done. >>> And those silly stupids which trying to use VMMaker are not of your >>> concern.. because when they fail to build stable VM (and they will), >>> you can always tell them that they are using wrong sources and because >>> you can't reproduce it (things is always working well for you, in your >>> warm corner :) >>> >>> So, i think that keeping a generated files works as some kind of >>> defensive policy which doesn't allows you to address certain >>> problems for real. >>> And sure thing, because we are lazy :) >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko AKA sig. >> >> >> > |
Re: Rant about generated files: should they stay in repository?
|
|
Hi: Even so you don't like the generated code, you should provide tarballs for certain versions so that people like me who do not know how to generate the code can just grab some version of the VM and compile a debug version easily. I just run into a problem that prevents the standard interpreter from loading one of our RoarVM images. And just going to http://squeakvm.org/unix/ and grabbing the tarball, compiling it with debugging settings and hacking in some instrumentation was enough to give me a first clue what went wrong. If I would have had to learn how to generate the code first, that would have taken a day instead of an hour. Thanks Stefan On 11 Mar 2011, at 13:19, Esteban Lorenzano wrote: > > I wasn't aware of that discussion, but I'm still disagree with their conclusions. > I'm certain that sources must be versioned. But sources are: VMMaker + configuration + platform files... generated files are not sources and to keep them in source tree is prone to error. > That's my feeling about this... anyway, I'm also not sure about re-discuss this :P > > best, > Esteban > > > El 11/03/2011, a las 8:50a.m., Frank Shearar escribió: > >> We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html >> >> frank >> >> On 2011/03/11 11:34, Esteban Lorenzano wrote: >>> >>> Igor, I'm with you :) >>> let's get rid of generated sources from repository! >>> >>> best, >>> Esteban >>> >>> El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: >>> >>>> >>>> I am against it , because it is going to nowhere. >>>> >>>> So let me elaborate... >>>> >>>> In Cog, we have an image which generating an universal sources for VM >>>> on all platforms. >>>> These generated files then put under source control. >>>> >>>> Pros: >>>> - to build VM you can just checkout and build >>>> >>>> Cons: >>>> - err , wait.. which exactly VM you going to build? can you use same >>>> sources for different builds? >>>> >>>> Yes, I can use same generated sources for everything, but depending on >>>> configuration and/or target operating system i can receive completely >>>> different results. >>>> >>>> Because, for recreating VM as an artifact to reproduce same >>>> circumstances as they were (when some bug was triggered) a generated >>>> sources snapshot is not enough. >>>> >>>> Simple example: >>>> - VM compiled for Macs using -O3 optimization are stable. >>>> In contrary to linux, where using -O2 with certain files makes your >>>> VM crash few (milli)seconds after startup. >>>> >>>> Second ( i don't like repeating myself but still ;): >>>> - the only way to make sure that VMMaker loads into our latest and >>>> greatest image(s) and producing same output(s) is to have someone to >>>> continuously try load it, >>>> then use it to generate sources, then compile the VM and then run >>>> tests for that VM. >>>> We have a generous guys in our community, who can do that for us. This >>>> is a Mr Hudson, and Mr Jenkins that joined party lately. >>>> >>>> I'd like to stress that generated files are just an intermediate data, >>>> which nobody interested in. >>>> >>>> Because as input what is relevant should be only : >>>> - a VMMaker >>>> - a platform-specific sources/libs >>>> - a build configuration >>>> >>>> and as output we always have: >>>> - VM >>>> >>>> what happens in the middle is really unimportant as long as you can >>>> guarantee reproducibility of the same output from same inputs. >>>> >>>> VM functioning should not depend on some random noise which comes from >>>> unrelated sources like >>>> - generated source files line ending convention >>>> - the date stamp when VM was built >>>> - anything which is not listed in the inputs list above >>>> and if it does, we should eliminate it. >>>> >>>> Because if it does, then you don't have a stable builds. And having a >>>> snapshot of sources which kinda works and kinda stable >>>> worth nothing, because at any moment in ongoing development you can >>>> meet same issue which will pop up out of nowhere. >>>> And it will pop out exactly because you don't care to always do things >>>> from scratch , starting from image and VMMaker. >>>> Because it is so convenient to just pull the generated sources from >>>> repo and you done. >>>> And those silly stupids which trying to use VMMaker are not of your >>>> concern.. because when they fail to build stable VM (and they will), >>>> you can always tell them that they are using wrong sources and because >>>> you can't reproduce it (things is always working well for you, in your >>>> warm corner :) >>>> >>>> So, i think that keeping a generated files works as some kind of >>>> defensive policy which doesn't allows you to address certain >>>> problems for real. >>>> And sure thing, because we are lazy :) >>>> >>>> >>>> -- >>>> Best regards, >>>> Igor Stasenko AKA sig. >>> >>> >>> >> > -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 |
Re: Rant about generated files: should they stay in repository?
|
|
stefan apparently you did not follow the lecture of Eliot because it takes less time than that :) > Hi: > > Even so you don't like the generated code, you should provide tarballs for certain versions so that people like me who do not know how to generate the code can just grab some version of the VM and compile a debug version easily. > > I just run into a problem that prevents the standard interpreter from loading one of our RoarVM images. > And just going to http://squeakvm.org/unix/ and grabbing the tarball, compiling it with debugging settings and hacking in some instrumentation was enough to give me a first clue what went wrong. > > If I would have had to learn how to generate the code first, that would have taken a day instead of an hour. > > Thanks > Stefan > > On 11 Mar 2011, at 13:19, Esteban Lorenzano wrote: > >> >> I wasn't aware of that discussion, but I'm still disagree with their conclusions. >> I'm certain that sources must be versioned. But sources are: VMMaker + configuration + platform files... generated files are not sources and to keep them in source tree is prone to error. >> That's my feeling about this... anyway, I'm also not sure about re-discuss this :P >> >> best, >> Esteban >> >> >> El 11/03/2011, a las 8:50a.m., Frank Shearar escribió: >> >>> We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html >>> >>> frank >>> >>> On 2011/03/11 11:34, Esteban Lorenzano wrote: >>>> >>>> Igor, I'm with you :) >>>> let's get rid of generated sources from repository! >>>> >>>> best, >>>> Esteban >>>> >>>> El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: >>>> >>>>> >>>>> I am against it , because it is going to nowhere. >>>>> >>>>> So let me elaborate... >>>>> >>>>> In Cog, we have an image which generating an universal sources for VM >>>>> on all platforms. >>>>> These generated files then put under source control. >>>>> >>>>> Pros: >>>>> - to build VM you can just checkout and build >>>>> >>>>> Cons: >>>>> - err , wait.. which exactly VM you going to build? can you use same >>>>> sources for different builds? >>>>> >>>>> Yes, I can use same generated sources for everything, but depending on >>>>> configuration and/or target operating system i can receive completely >>>>> different results. >>>>> >>>>> Because, for recreating VM as an artifact to reproduce same >>>>> circumstances as they were (when some bug was triggered) a generated >>>>> sources snapshot is not enough. >>>>> >>>>> Simple example: >>>>> - VM compiled for Macs using -O3 optimization are stable. >>>>> In contrary to linux, where using -O2 with certain files makes your >>>>> VM crash few (milli)seconds after startup. >>>>> >>>>> Second ( i don't like repeating myself but still ;): >>>>> - the only way to make sure that VMMaker loads into our latest and >>>>> greatest image(s) and producing same output(s) is to have someone to >>>>> continuously try load it, >>>>> then use it to generate sources, then compile the VM and then run >>>>> tests for that VM. >>>>> We have a generous guys in our community, who can do that for us. This >>>>> is a Mr Hudson, and Mr Jenkins that joined party lately. >>>>> >>>>> I'd like to stress that generated files are just an intermediate data, >>>>> which nobody interested in. >>>>> >>>>> Because as input what is relevant should be only : >>>>> - a VMMaker >>>>> - a platform-specific sources/libs >>>>> - a build configuration >>>>> >>>>> and as output we always have: >>>>> - VM >>>>> >>>>> what happens in the middle is really unimportant as long as you can >>>>> guarantee reproducibility of the same output from same inputs. >>>>> >>>>> VM functioning should not depend on some random noise which comes from >>>>> unrelated sources like >>>>> - generated source files line ending convention >>>>> - the date stamp when VM was built >>>>> - anything which is not listed in the inputs list above >>>>> and if it does, we should eliminate it. >>>>> >>>>> Because if it does, then you don't have a stable builds. And having a >>>>> snapshot of sources which kinda works and kinda stable >>>>> worth nothing, because at any moment in ongoing development you can >>>>> meet same issue which will pop up out of nowhere. >>>>> And it will pop out exactly because you don't care to always do things >>>>> from scratch , starting from image and VMMaker. >>>>> Because it is so convenient to just pull the generated sources from >>>>> repo and you done. >>>>> And those silly stupids which trying to use VMMaker are not of your >>>>> concern.. because when they fail to build stable VM (and they will), >>>>> you can always tell them that they are using wrong sources and because >>>>> you can't reproduce it (things is always working well for you, in your >>>>> warm corner :) >>>>> >>>>> So, i think that keeping a generated files works as some kind of >>>>> defensive policy which doesn't allows you to address certain >>>>> problems for real. >>>>> And sure thing, because we are lazy :) >>>>> >>>>> >>>>> -- >>>>> Best regards, >>>>> Igor Stasenko AKA sig. >>>> >>>> >>>> >>> >> > > -- > Stefan Marr > Software Languages Lab > Vrije Universiteit Brussel > Pleinlaan 2 / B-1050 Brussels / Belgium > http://soft.vub.ac.be/~smarr > Phone: +32 2 629 2974 > Fax: +32 2 629 3525 > |
|
|
The problem of having the generated files in or not in the repository clouds the real problem. In reflection what you really need is the ability for anyone to do squeakVMGeneratorBuilderWhatever generate: 'OSXCog" version: 5.8.2.12 And what ever that process is will fetch generate and enable the client to actually have 5.8.2.12 of the OSX Cog VM compile and ready to go... In my opinion that is the real problem here that you need to: (a) determining the VM version you have (this is murky btw). (b) ensuring the toolset need to build the VM is there (c) ensuring any third party tools needed to build are there. (and ensure the versions are sane). (d) running the process and determining that the build of 5.8.2.12 is sane. This harkens back to the days of Envy where you could say build me version 5.8.2.12 of product X and the tool would *always* ensure what came out was 5.8.2.12 Frankly "the client" here doesn't care what happens or how the files are stored or constructed. He *does* care that if he thinks he is building 5.8.2.12 then that is what he will get once the process completes. On Fri, Mar 11, 2011 at 6:25 AM, stephane ducasse <[hidden email]> wrote: > > stefan apparently you did not follow the lecture of Eliot because it takes less time than that :) > >> Hi: >> >> Even so you don't like the generated code, you should provide tarballs for certain versions so that people like me who do not know how to generate the code can just grab some version of the VM and compile a debug version easily. >> >> I just run into a problem that prevents the standard interpreter from loading one of our RoarVM images. >> And just going to http://squeakvm.org/unix/ and grabbing the tarball, compiling it with debugging settings and hacking in some instrumentation was enough to give me a first clue what went wrong. >> >> If I would have had to learn how to generate the code first, that would have taken a day instead of an hour. >> >> Thanks >> Stefan >> >> On 11 Mar 2011, at 13:19, Esteban Lorenzano wrote: >> >>> >>> I wasn't aware of that discussion, but I'm still disagree with their conclusions. >>> I'm certain that sources must be versioned. But sources are: VMMaker + configuration + platform files... generated files are not sources and to keep them in source tree is prone to error. >>> That's my feeling about this... anyway, I'm also not sure about re-discuss this :P >>> >>> best, >>> Esteban >>> >>> >>> El 11/03/2011, a las 8:50a.m., Frank Shearar escribió: >>> >>>> We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html >>>> >>>> frank >>>> >>>> On 2011/03/11 11:34, Esteban Lorenzano wrote: >>>>> >>>>> Igor, I'm with you :) >>>>> let's get rid of generated sources from repository! >>>>> >>>>> best, >>>>> Esteban >>>>> >>>>> El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: >>>>> >>>>>> >>>>>> I am against it , because it is going to nowhere. >>>>>> >>>>>> So let me elaborate... >>>>>> >>>>>> In Cog, we have an image which generating an universal sources for VM >>>>>> on all platforms. >>>>>> These generated files then put under source control. >>>>>> >>>>>> Pros: >>>>>> - to build VM you can just checkout and build >>>>>> >>>>>> Cons: >>>>>> - err , wait.. which exactly VM you going to build? can you use same >>>>>> sources for different builds? >>>>>> >>>>>> Yes, I can use same generated sources for everything, but depending on >>>>>> configuration and/or target operating system i can receive completely >>>>>> different results. >>>>>> >>>>>> Because, for recreating VM as an artifact to reproduce same >>>>>> circumstances as they were (when some bug was triggered) a generated >>>>>> sources snapshot is not enough. >>>>>> >>>>>> Simple example: >>>>>> - VM compiled for Macs using -O3 optimization are stable. >>>>>> In contrary to linux, where using -O2 with certain files makes your >>>>>> VM crash few (milli)seconds after startup. >>>>>> >>>>>> Second ( i don't like repeating myself but still ;): >>>>>> - the only way to make sure that VMMaker loads into our latest and >>>>>> greatest image(s) and producing same output(s) is to have someone to >>>>>> continuously try load it, >>>>>> then use it to generate sources, then compile the VM and then run >>>>>> tests for that VM. >>>>>> We have a generous guys in our community, who can do that for us. This >>>>>> is a Mr Hudson, and Mr Jenkins that joined party lately. >>>>>> >>>>>> I'd like to stress that generated files are just an intermediate data, >>>>>> which nobody interested in. >>>>>> >>>>>> Because as input what is relevant should be only : >>>>>> - a VMMaker >>>>>> - a platform-specific sources/libs >>>>>> - a build configuration >>>>>> >>>>>> and as output we always have: >>>>>> - VM >>>>>> >>>>>> what happens in the middle is really unimportant as long as you can >>>>>> guarantee reproducibility of the same output from same inputs. >>>>>> >>>>>> VM functioning should not depend on some random noise which comes from >>>>>> unrelated sources like >>>>>> - generated source files line ending convention >>>>>> - the date stamp when VM was built >>>>>> - anything which is not listed in the inputs list above >>>>>> and if it does, we should eliminate it. >>>>>> >>>>>> Because if it does, then you don't have a stable builds. And having a >>>>>> snapshot of sources which kinda works and kinda stable >>>>>> worth nothing, because at any moment in ongoing development you can >>>>>> meet same issue which will pop up out of nowhere. >>>>>> And it will pop out exactly because you don't care to always do things >>>>>> from scratch , starting from image and VMMaker. >>>>>> Because it is so convenient to just pull the generated sources from >>>>>> repo and you done. >>>>>> And those silly stupids which trying to use VMMaker are not of your >>>>>> concern.. because when they fail to build stable VM (and they will), >>>>>> you can always tell them that they are using wrong sources and because >>>>>> you can't reproduce it (things is always working well for you, in your >>>>>> warm corner :) >>>>>> >>>>>> So, i think that keeping a generated files works as some kind of >>>>>> defensive policy which doesn't allows you to address certain >>>>>> problems for real. >>>>>> And sure thing, because we are lazy :) >>>>>> >>>>>> >>>>>> -- >>>>>> Best regards, >>>>>> Igor Stasenko AKA sig. >>>>> >>>>> >>>>> >>>> >>> >> >> -- >> Stefan Marr >> Software Languages Lab >> Vrije Universiteit Brussel >> Pleinlaan 2 / B-1050 Brussels / Belgium >> http://soft.vub.ac.be/~smarr >> Phone: +32 2 629 2974 >> Fax: +32 2 629 3525 >> > > -- =========================================================================== John M. McIntosh <[hidden email]> Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com =========================================================================== |
Re: Rant about generated files: should they stay in repository?
|
|
Hi: On 11 Mar 2011, at 18:58, John McIntosh wrote: > > The problem of having the generated files in or not in the repository > clouds the real problem. Right, actually I also don't care whether the generated source is in version control. It would be nice I think, especially if git is used, to generate the source and tag it with the Monticello version, but thats not my main concern. I just hope, that even after the Hudson-ification, there are a tar balls which I can take, change some compilation flags and then just make the VM from source for a specific release. Thus, when a VM is released, there should also be tar ball. I just ignored the actual problem being discussed here... Never mind. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525 |
Re: Rant about generated files: should they stay in repository?
|
|
In reply to this post by johnmci
On Fri, 11 Mar 2011, John McIntosh wrote: > > The problem of having the generated files in or not in the repository > clouds the real problem. > In reflection what you really need is the ability for anyone to do > > squeakVMGeneratorBuilderWhatever generate: 'OSXCog" version: 5.8.2.12 > > And what ever that process is will fetch generate and enable the > client to actually have 5.8.2.12 of the OSX Cog VM compile and ready > to go... > > In my opinion that is the real problem here that you need to: > > (a) determining the VM version you have (this is murky btw). > (b) ensuring the toolset need to build the VM is there > (c) ensuring any third party tools needed to build are there. (and > ensure the versions are sane). > (d) running the process and determining that the build of 5.8.2.12 is sane. > > This harkens back to the days of Envy where you could say build me > version 5.8.2.12 of product X and the tool would *always* ensure what > came out was 5.8.2.12 > > Frankly "the client" here doesn't care what happens or how the files > are stored or constructed. He *does* care that if he thinks he is > building 5.8.2.12 then that is what he will get once the process > completes. do it on Un*x. When it works for n different unix systems, there will be at least another n where it doesn't work. Levente > > > On Fri, Mar 11, 2011 at 6:25 AM, stephane ducasse > <[hidden email]> wrote: >> >> stefan apparently you did not follow the lecture of Eliot because it takes less time than that :) >> >>> Hi: >>> >>> Even so you don't like the generated code, you should provide tarballs for certain versions so that people like me who do not know how to generate the code can just grab some version of the VM and compile a debug version easily. >>> >>> I just run into a problem that prevents the standard interpreter from loading one of our RoarVM images. >>> And just going to http://squeakvm.org/unix/ and grabbing the tarball, compiling it with debugging settings and hacking in some instrumentation was enough to give me a first clue what went wrong. >>> >>> If I would have had to learn how to generate the code first, that would have taken a day instead of an hour. >>> >>> Thanks >>> Stefan >>> >>> On 11 Mar 2011, at 13:19, Esteban Lorenzano wrote: >>> >>>> >>>> I wasn't aware of that discussion, but I'm still disagree with their conclusions. >>>> I'm certain that sources must be versioned. But sources are: VMMaker + configuration + platform files... generated files are not sources and to keep them in source tree is prone to error. >>>> That's my feeling about this... anyway, I'm also not sure about re-discuss this :P >>>> >>>> best, >>>> Esteban >>>> >>>> >>>> El 11/03/2011, a las 8:50a.m., Frank Shearar escribió: >>>> >>>>> We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html >>>>> >>>>> frank >>>>> >>>>> On 2011/03/11 11:34, Esteban Lorenzano wrote: >>>>>> >>>>>> Igor, I'm with you :) >>>>>> let's get rid of generated sources from repository! >>>>>> >>>>>> best, >>>>>> Esteban >>>>>> >>>>>> El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió: >>>>>> >>>>>>> >>>>>>> I am against it , because it is going to nowhere. >>>>>>> >>>>>>> So let me elaborate... >>>>>>> >>>>>>> In Cog, we have an image which generating an universal sources for VM >>>>>>> on all platforms. >>>>>>> These generated files then put under source control. >>>>>>> >>>>>>> Pros: >>>>>>> - to build VM you can just checkout and build >>>>>>> >>>>>>> Cons: >>>>>>> - err , wait.. which exactly VM you going to build? can you use same >>>>>>> sources for different builds? >>>>>>> >>>>>>> Yes, I can use same generated sources for everything, but depending on >>>>>>> configuration and/or target operating system i can receive completely >>>>>>> different results. >>>>>>> >>>>>>> Because, for recreating VM as an artifact to reproduce same >>>>>>> circumstances as they were (when some bug was triggered) a generated >>>>>>> sources snapshot is not enough. >>>>>>> >>>>>>> Simple example: >>>>>>> - VM compiled for Macs using -O3 optimization are stable. >>>>>>> In contrary to linux, where using -O2 with certain files makes your >>>>>>> VM crash few (milli)seconds after startup. >>>>>>> >>>>>>> Second ( i don't like repeating myself but still ;): >>>>>>> - the only way to make sure that VMMaker loads into our latest and >>>>>>> greatest image(s) and producing same output(s) is to have someone to >>>>>>> continuously try load it, >>>>>>> then use it to generate sources, then compile the VM and then run >>>>>>> tests for that VM. >>>>>>> We have a generous guys in our community, who can do that for us. This >>>>>>> is a Mr Hudson, and Mr Jenkins that joined party lately. >>>>>>> >>>>>>> I'd like to stress that generated files are just an intermediate data, >>>>>>> which nobody interested in. >>>>>>> >>>>>>> Because as input what is relevant should be only : >>>>>>> - a VMMaker >>>>>>> - a platform-specific sources/libs >>>>>>> - a build configuration >>>>>>> >>>>>>> and as output we always have: >>>>>>> - VM >>>>>>> >>>>>>> what happens in the middle is really unimportant as long as you can >>>>>>> guarantee reproducibility of the same output from same inputs. >>>>>>> >>>>>>> VM functioning should not depend on some random noise which comes from >>>>>>> unrelated sources like >>>>>>> - generated source files line ending convention >>>>>>> - the date stamp when VM was built >>>>>>> - anything which is not listed in the inputs list above >>>>>>> and if it does, we should eliminate it. >>>>>>> >>>>>>> Because if it does, then you don't have a stable builds. And having a >>>>>>> snapshot of sources which kinda works and kinda stable >>>>>>> worth nothing, because at any moment in ongoing development you can >>>>>>> meet same issue which will pop up out of nowhere. >>>>>>> And it will pop out exactly because you don't care to always do things >>>>>>> from scratch , starting from image and VMMaker. >>>>>>> Because it is so convenient to just pull the generated sources from >>>>>>> repo and you done. >>>>>>> And those silly stupids which trying to use VMMaker are not of your >>>>>>> concern.. because when they fail to build stable VM (and they will), >>>>>>> you can always tell them that they are using wrong sources and because >>>>>>> you can't reproduce it (things is always working well for you, in your >>>>>>> warm corner :) >>>>>>> >>>>>>> So, i think that keeping a generated files works as some kind of >>>>>>> defensive policy which doesn't allows you to address certain >>>>>>> problems for real. >>>>>>> And sure thing, because we are lazy :) >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Best regards, >>>>>>> Igor Stasenko AKA sig. >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> -- >>> Stefan Marr >>> Software Languages Lab >>> Vrije Universiteit Brussel >>> Pleinlaan 2 / B-1050 Brussels / Belgium >>> http://soft.vub.ac.be/~smarr >>> Phone: +32 2 629 2974 >>> Fax: +32 2 629 3525 >>> >> >> > > > > -- > =========================================================================== > John M. McIntosh <[hidden email]> > Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com > =========================================================================== > |
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |