Hudson Status?

Hello all,

I have chunk of free time coming up, and a bunch of professional experience with CI. Seems I end up being the guy who sets it up and drives the effort wherever I go.

Curious where the status for the Hudson effort is. Is the blocker mostly time? I should be able to get it done, if so.

I'm interested in doing both VM-level and image-level CI. If you're currently working on the Hudson effort for Squeak, please let me know so that I don't have to duplicate your efforts.

Thank you!

 
On 4 May 2011 03:39, Casey Ransberger <[hidden email]> wrote:
>
> Hello all,
>
> I have chunk of free time coming up, and a bunch of professional experience with CI. Seems I end up being the guy who sets it up and drives the effort wherever I go.
>
> Curious where the status for the Hudson effort is. Is the blocker mostly time? I should be able to get it done, if so.
>
> I'm interested in doing both VM-level and image-level CI. If you're currently working on the Hudson effort for Squeak, please let me know so that I don't have to duplicate your efforts.
>

Thank you, Casey for willing to help.
Hudson status: up and working. And most of the times it does its job.
Except that we would like a more finer control on it.

Currently Hudson are set up for us by INRIA engineers, and while it
was good from the beginning (we don't need to spend time setting up
things by ourselves), today our demands looks like outgrew what admins
at INRIA could propose :)

We will have a meeting on this week and discuss our problems.
We're certainly want to have more open access to servers by community
members, so they could set up, configure and run their own jobs.
Stef says that he want servers to burn in flames (means busy with
always building something).

But we meet some organizational/political problems we need to overcome first.

I think Stephane could say more about it.


What help i personally might need? Well.. building VMs on many various
platforms and making working cmake configs for them.
So, mainly its about creating and testing new cmake configurations.

Oh.. yeah.. we would like to set up a benchmarking suites for running
benchmarks after each VM build. And then outputting results to some
file(s), so we can detect performance regression from build to build
etc etc.
That would be really nice.
But that means some coding, because first we need to create some kind
of benchmarking suite, and also some layer to output results, which
hudson can understand (expect heavy xml resistance ;)


--
Best regards,
Igor Stasenko AKA sig.

Thanks for your reply, Igor! Inline.

On May 3, 2011, at 7:44 PM, Igor Stasenko <[hidden email]> wrote:

> Thank you, Casey for willing to help.
> Hudson status: up and working. And most of the times it does its job.
> Except that we would like a more finer control on it.

Finer control of what tests get run? Out of curiosity, are you just building the VM with Hudson, or are you also running SUnit tests there? Has anyone done the work to make SUnit output JUnit XML reports (needed for Hudson to do detailed reporting?)

I'm more focused on doing image level CI, personally, because I want to know right away when I break something in the image, and because I'm not doing much VM hacking (yet.)

> Currently Hudson are set up for us by INRIA engineers, and while it
> was good from the beginning (we don't need to spend time setting up
> things by ourselves), today our demands looks like outgrew what admins
> at INRIA could propose :)

How so? What are the actual problems? I solve people/process problems like these professionally, in addition to grinding out test automation. For you people, for the love you've shown, I'll do it for free;) Let me at em!

> We will have a meeting on this week and discuss our problems.

Can I come? Seriously, I live for solving this problem. Please invite me if you can: I promise to be on my very best behavior.

> We're certainly want to have more open access to servers by community
> members, so they could set up, configure and run their own jobs.

That's a great idea, if people in the community are lining up to do so.

> Stef says that he want servers to burn in flames (means busy with
> always building something).

Good thing you qualified that, because in vernacular American English, that means he wishes someone would actually set the boxes on fire:P

I agree though. A CI box should always be hot! As in, put another fan on that thing: it's too hot.

> But we meet some organizational/political problems we need to overcome first.

Would you be willing develop that statement?

> I think Stephane could say more about it.

CC'ing him then. Hi Stephane! I found some free time and I want to help the Pharo and Squeak communities with their CI efforts, because CI is love. Can you help me understand any of the organizational/political challenges Igor speaks of? I'm also interested in technical stumbling blocks that I might be able to help with.

> What help i personally might need? Well.. building VMs on many various
> platforms and making working cmake configs for them.
> So, mainly its about creating and testing new cmake configurations.

Ah, I see. I mostly want to do CI on the image(s.) I'm *really* glad you guys are running tests against the VM, but the VM is currently mostly outside of my area of focus. I want to know right away if my code is going to break the Trunk; this is my main motivator -- er, that and did I mention -- CI is love!

> Oh.. yeah.. we would like to set up a benchmarking suites for running
> benchmarks after each VM build. And then outputting results to some
> file(s), so we can detect performance regression from build to build
> etc etc.

Also really cool.

> That would be really nice.
> But that means some coding, because first we need to create some kind
> of benchmarking suite, and also some layer to output results, which
> hudson can understand (expect heavy xml resistance ;)

If you're using Hudson, you want to output JUnit XML. There isn't a better way to do it without modifying Hudson.

Except... Well, you could write a plugin (yuck) or better yet...

What's the strongest resistance? Would people rather hack a lot of Java code than write a little bit of Smalltalk (or Perl or whatever in the case of the VM) to get XML output? Cause I'd rather do the latter.

If you guys *hate* XML so much (I hate it almost that much) that you'd rather do more work than less in order to avoid dealing with it, before you go, you might look at TAP.

TAP stands for Test Anything Protocol, and originated with Perl's Test::More (to my knowledge anyway -- Randal? Historian hat?)

The reason I'm telling you this is, the reporting output format is ridiculously simple to implement, and IIRC there's a plugin for Hudson that'll let you use TAP output instead of JUnit XML. You don't need Perl to use TAP output because the format is really just a dirt simple convention for reporting on test results.

This is what we did to deal with the fact that we had a bunch of different test runners for different languages with different output formats and wanted consistent reporting when I was working on the problem at WhitePages.com, we integrated TAP output support for the few languages that didn't just have a library that did it.

At WP, we were using a different CI server (CruiseControl.rb) and at that point in time, we had to modify CruiseControl in order to make it understand TAP output. In the case of Hudson, as I stated, I'm pretty sure there's a plugin that does it for you.

Just a thought.

> --
> Best regards,
> Igor Stasenko AKA sig.

>> Thank you, Casey for willing to help.
>> Hudson status: up and working. And most of the times it does its job.
>> Except that we would like a more finer control on it.
>
> Finer control of what tests get run? Out of curiosity, are you just building the VM with Hudson, or are you also running SUnit tests there? Has anyone done the work to make SUnit output JUnit XML reports (needed for Hudson to do detailed reporting?)

yes this is done since months.
Now we want SmallLint rules.
Now if you want help building a testsuite for the vm and displaying results similarly to the pypy
one is what we want.

>> Currently Hudson are set up for us by INRIA engineers, and while it
>> was good from the beginning (we don't need to spend time setting up
>> things by ourselves), today our demands looks like outgrew what admins
>> at INRIA could propose :)
>
> How so? What are the actual problems? I solve people/process problems like these professionally, in addition to grinding out test automation. For you people, for the love you've shown, I'll do it for free;) Let me at em!

This is good to know.
This is just engineer time lacking. I will be back to france and discuss with them. I can ask them if they would let you get access to the server.

>> We will have a meeting on this week and discuss our problems.
>
> Can I come? Seriously, I live for solving this problem. Please invite me if you can: I promise to be on my very best behavior.

INRIA is a minister to army so servers  and we have to comply with the rules. Now if somebody has a server somewhere may be we should migrate.
:)

>>
> Would you be willing develop that statement?

Army security compliance.

>> CC'ing him then. Hi Stephane! I found some free time and I want to help the Pharo and Squeak communities with their CI efforts, because CI is love. Can you help me understand any of the organizational/political challenges Igor speaks of? I'm also interested in technical stumbling blocks that I might be able to help with.

Yes I will.
Normally I hope to meet th guy tomorrow afternoon.

we have a pharo plugin.

> What's the strongest resistance? Would people rather hack a lot of Java code than write a little bit of Smalltalk (or Perl or whatever in the case of the VM) to get XML output? Cause I'd rather do the latter.
>
> If you guys *hate* XML so much (I hate it almost that much) that you'd rather do more work than less in order to avoid dealing with it, before you go, you might look at TAP.

no problem of hate we are pragmatic.
Right now what is missing is a tests and test collection results for the vms. This would be really good to have it.

 
2011/5/4 Igor Stasenko <[hidden email]>:
About bench, I would begin with what's already there, the shootout.
Eliot used it.

Nicolas

>
> --
> Best regards,
> Igor Stasenko AKA sig.
>