Running tests for auto-built Cog VMs
Locked
Running tests for auto-built Cog VMs
 
|
Hello all. Our next plan is to extend the Hudson build jobs with tests which are run after VM has built. I'd like to ask, what kind of tests we should run for VMs. So, here's my questions: - do we need a VM specific tests? Or we just use ordinal tests which are in image(s) by default? - in case if we need to run VM-specific tests, is they are available somewhere or not? do we need to create separate package/repository for holding them? i'd like to brainstorm that a bit. And then we will do that. -- Best regards, Igor Stasenko AKA sig. |
Re: Running tests for auto-built Cog VMs
|
|
On Thu, Mar 17, 2011 at 10:29 AM, Igor Stasenko <[hidden email]> wrote:
SOme of the standard tests in the Squeak 4.2 suite are specifically for VM stuff. The entire suite depends on the VM in some way. It seems straight-forward to include tests for specific pieces of functionality, for example the mirror primitives, cannotInterpret: and objects-as-methods. Adding tests that test new VM functionality is clearly more than a good idea; it should be de rigueur.
- in case if we need to run VM-specific tests, is they are available Check out a 4.2 image and look at the tests. The organization might not be perfect but it's there.
|
Re: Running tests for auto-built Cog VMs
|
|
On 18 March 2011 01:30, Eliot Miranda <[hidden email]> wrote: > > > > On Thu, Mar 17, 2011 at 10:29 AM, Igor Stasenko <[hidden email]> wrote: >> >> Hello all. >> >> Our next plan is to extend the Hudson build jobs with tests which are >> run after VM has built. >> >> I'd like to ask, what kind of tests we should run for VMs. >> >> So, here's my questions: >> >> - do we need a VM specific tests? Or we just use ordinal tests which >> are in image(s) by default? > > SOme of the standard tests in the Squeak 4.2 suite are specifically for VM stuff. The entire suite depends on the VM in some way. It seems straight-forward to include tests for specific pieces of functionality, for example the mirror primitives, cannotInterpret: and objects-as-methods. Adding tests that test new VM functionality is clearly more than a good idea; it should be de rigueur. > >> >> - in case if we need to run VM-specific tests, is they are available >> somewhere or not? do we need to create separate package/repository for >> holding them? > > Check out a 4.2 image and look at the tests. The organization might not be perfect but it's there. > You mean non-VM-specific tests, just those which lying in image(s)? Yes, that's the first thing which comes into mind. And sure thing for start it is ok. For example, Hudson runs tests for PharoCore1.3 image, each time new update there. https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%20Core%201.3/ It is easy to tweak this config and run it using freshly built VM(s), one per each platform * JIT/Stack/MT configs. But i wonder if we may need to be more focused and create some package(s) to group tests around VM or its individual plugins. >> >> i'd like to brainstorm that a bit. And then we will do that. >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > -- Best regards, Igor Stasenko AKA sig. |
Re: Running tests for auto-built Cog VMs
|
|
On Thu, Mar 17, 2011 at 6:21 PM, Igor Stasenko <[hidden email]> wrote:
Have you looked, as I suggested, in a Squeak 4.2 image? You'll find Tests-VM, Tests-Finalization, Tests-ObjectsAsMethods
|
Re: Running tests for auto-built Cog VMs
|
|
On 18 March 2011 02:26, Eliot Miranda <[hidden email]> wrote: > > > > On Thu, Mar 17, 2011 at 6:21 PM, Igor Stasenko <[hidden email]> wrote: >> >> On 18 March 2011 01:30, Eliot Miranda <[hidden email]> wrote: >> > >> > >> > >> > On Thu, Mar 17, 2011 at 10:29 AM, Igor Stasenko <[hidden email]> wrote: >> >> >> >> Hello all. >> >> >> >> Our next plan is to extend the Hudson build jobs with tests which are >> >> run after VM has built. >> >> >> >> I'd like to ask, what kind of tests we should run for VMs. >> >> >> >> So, here's my questions: >> >> >> >> - do we need a VM specific tests? Or we just use ordinal tests which >> >> are in image(s) by default? >> > >> > SOme of the standard tests in the Squeak 4.2 suite are specifically for VM stuff. The entire suite depends on the VM in some way. It seems straight-forward to include tests for specific pieces of functionality, for example the mirror primitives, cannotInterpret: and objects-as-methods. Adding tests that test new VM functionality is clearly more than a good idea; it should be de rigueur. >> > >> >> >> >> - in case if we need to run VM-specific tests, is they are available >> >> somewhere or not? do we need to create separate package/repository for >> >> holding them? >> > >> > Check out a 4.2 image and look at the tests. The organization might not be perfect but it's there. >> > >> >> You mean non-VM-specific tests, just those which lying in image(s)? >> >> Yes, that's the first thing which comes into mind. And sure thing for >> start it is ok. >> >> For example, Hudson runs tests for PharoCore1.3 image, each time new >> update there. >> https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%20Core%201.3/ >> >> It is easy to tweak this config and run it using freshly built VM(s), >> one per each platform * JIT/Stack/MT configs. >> >> But i wonder if we may need to be more focused and create some package(s) >> to group tests around VM or its individual plugins. > > Have you looked, as I suggested, in a Squeak 4.2 image? You'll find Tests-VM, Tests-Finalization, Tests-ObjectsAsMethods kk.. will take a look. There is simply too many tests, so its hard to tell which ones are made specifically to test VM. >> -- Best regards, Igor Stasenko AKA sig. |
Re: Running tests for auto-built Cog VMs
|
|
In reply to this post by Igor Stasenko
It'd be really great if you kept more than Pharo in mind: I'd like to be able to shoot someone an email and say "can we add a Cuis image to our shared CI setup?" and have the answer be "yes, not hard, I need you to do X before we can set that up." If we're running Hudson, I'd sure like to use it to test *more* than just the VM. Squeak has a great big suite of tests and I think we ought to use 'em without making people open the test runner manually. Igor! CI! You rock! On Mar 17, 2011, at 6:21 PM, Igor Stasenko <[hidden email]> wrote: > > On 18 March 2011 01:30, Eliot Miranda <[hidden email]> wrote: >> >> >> >> On Thu, Mar 17, 2011 at 10:29 AM, Igor Stasenko <[hidden email]> wrote: >>> >>> Hello all. >>> >>> Our next plan is to extend the Hudson build jobs with tests which are >>> run after VM has built. >>> >>> I'd like to ask, what kind of tests we should run for VMs. >>> >>> So, here's my questions: >>> >>> - do we need a VM specific tests? Or we just use ordinal tests which >>> are in image(s) by default? >> >> SOme of the standard tests in the Squeak 4.2 suite are specifically for VM stuff. The entire suite depends on the VM in some way. It seems straight-forward to include tests for specific pieces of functionality, for example the mirror primitives, cannotInterpret: and objects-as-methods. Adding tests that test new VM functionality is clearly more than a good idea; it should be de rigueur. >> >>> >>> - in case if we need to run VM-specific tests, is they are available >>> somewhere or not? do we need to create separate package/repository for >>> holding them? >> >> Check out a 4.2 image and look at the tests. The organization might not be perfect but it's there. >> > > You mean non-VM-specific tests, just those which lying in image(s)? > > Yes, that's the first thing which comes into mind. And sure thing for > start it is ok. > > For example, Hudson runs tests for PharoCore1.3 image, each time new > update there. > https://pharo-ic.lille.inria.fr/hudson/view/Pharo/job/Pharo%20Core%201.3/ > > It is easy to tweak this config and run it using freshly built VM(s), > one per each platform * JIT/Stack/MT configs. > > But i wonder if we may need to be more focused and create some package(s) > to group tests around VM or its individual plugins. > > > >>> >>> i'd like to brainstorm that a bit. And then we will do that. >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko AKA sig. >> > > -- > Best regards, > Igor Stasenko AKA sig. |
Re: Running tests for auto-built Cog VMs
|
|
In reply to this post by Eliot Miranda-2
On Fri, Mar 18, 2011 at 2:26 AM, Eliot Miranda <[hidden email]> wrote:
So maybe we should integrate them in Pharo too ?
|
Re: [Pharo-project] [Vm-dev] Running tests for auto-built Cog VMs
|
|
In reply to this post by Eliot Miranda-2
:) In addition to running all the tests of all the subsystems of course. S. >> > In Pharo there is: > > Tests-VM > BecomeTest > IslandVMTweaksTestCase > Tests-Finalization > ObjectFinalizerTest > Tests-ObjectsAsMethods > TestObjectsAsMethods > > .. and they are run every day at least once (in 1.2, 1.3 and before 1.2 was stable even 1.1) since a couple of months. > > e.g. > > https://pharo-ic.lille.inria.fr/hudson/job/Pharo%20Core%201.2/lastCompletedBuild/testReport/Tests.VM/ > > > Marcus |
Re: [Pharo-project] [Vm-dev] Running tests for auto-built Cog VMs
|
|
Yeah , yeah.. Maybe people was not aware that tests are running for couple of months now. But my question if we need something in addition. For instance, threaded FFI. How to test if it there, and if it is, how to test that it works ok? Apparently standard FFI test are not covering new threading capabilities of VM. On 18 March 2011 10:41, stephane ducasse <[hidden email]> wrote: > > :) > In addition to running all the tests of all the subsystems of course. > > S. > >>> >> In Pharo there is: >> >> Tests-VM >> BecomeTest >> IslandVMTweaksTestCase >> Tests-Finalization >> ObjectFinalizerTest >> Tests-ObjectsAsMethods >> TestObjectsAsMethods >> >> .. and they are run every day at least once (in 1.2, 1.3 and before 1.2 was stable even 1.1) since a couple of months. >> >> e.g. >> >> https://pharo-ic.lille.inria.fr/hudson/job/Pharo%20Core%201.2/lastCompletedBuild/testReport/Tests.VM/ >> >> >> Marcus > -- Best regards, Igor Stasenko AKA sig. |
Advertising VM features (was Re: [Pharo-project] [Vm-dev] Running tests for auto-built Cog VMs)
|
|
On 2011/03/18 10:41, Igor Stasenko wrote: > > Yeah , yeah.. > Maybe people was not aware that tests are running for couple of months now. > > But my question if we need something in addition. > For instance, threaded FFI. How to test if it there, and if it is, how > to test that it works ok? > Apparently standard FFI test are not covering new threading capabilities of VM. Is it possible to expose what a VM supports? Actually, mirrors here would be great - you could ask the mirror what plugins it supports, what special features it provides (immutability, tail call optimisation, ...). frank > On 18 March 2011 10:41, stephane ducasse<[hidden email]> wrote: >> >> :) >> In addition to running all the tests of all the subsystems of course. >> >> S. >> >>>> >>> In Pharo there is: >>> >>> Tests-VM >>> BecomeTest >>> IslandVMTweaksTestCase >>> Tests-Finalization >>> ObjectFinalizerTest >>> Tests-ObjectsAsMethods >>> TestObjectsAsMethods >>> >>> .. and they are run every day at least once (in 1.2, 1.3 and before 1.2 was stable even 1.1) since a couple of months. >>> >>> e.g. >>> >>> https://pharo-ic.lille.inria.fr/hudson/job/Pharo%20Core%201.2/lastCompletedBuild/testReport/Tests.VM/ >>> >>> >>> Marcus >> > > > |
Re: Advertising VM features (was Re: [Pharo-project] [Vm-dev] Running tests for auto-built Cog VMs)
|
|
On Fri, Mar 18, 2011 at 4:19 AM, Frank Shearar <[hidden email]> wrote:
Of course. See loadedModules, getSystemAttribute: and vmParameterAt:. Increasingly people are writing useful wrappers for them, typically in SmalltalkImage.
|
Re: [Pharo-project] [Vm-dev] Running tests for auto-built Cog VMs
|
|
In reply to this post by Igor Stasenko
On Fri, Mar 18, 2011 at 3:41 AM, Igor Stasenko <[hidden email]> wrote:
A cross-platform test might be hard to write. So far I've used - running a threaded call to system(1, i.e. unix) with some argument like sleep 60 while forking a loop that counts and printing the count afterward. if the count is small the call blocked
- calling a complex Mac OS X native file dialog with callbacks for the filename filter and the accept and cancel actions - making non-blocking calls on a MySQL db through ODBC - (and my favourite) running an image that forks a reader that loops writing a squeak> prompt, reading from stdin and evaluating the chunk.
Of course the last, while fun to use (one can interact with the image concurrent with using the reader) isn't that easy to adapt to an automated test. One of the nest tests is to create an OS semaphore and then fork a threaded call to wait on it, wait on a delay and then signal the semaphore with another threaded call. Tests can be built up upon this basic approach. i.e. we need to write appropriate automated tests of the threaded FFI.
|
«
Return to Squeak VM
|
1 view|%1 views
| Free forum by Nabble | Edit this page |