[Glass] Does Gemstone transform integration testing into unit testing?

Hello Glass! I am still excited about Gemstone/S and I wanted to start a conversation about testing.

It strikes me that a system based on Gemstone/S can be tested in a more natural and seamless way than a system based on a "database." Have you ever written tests that interact with a database? I have, and it is not as smooth as writing tests for code based on the native data structures of a programming language. You have to establish connections and submit queries in ways that obscure the logic of the code you are trying to test.

Testing experts sometimes use the terms "unit testing" and "integration testing" to refer to different kinds of testing. A "unit test" is a test of a small piece of your system while an integration is something more complicated, approaching the system as a whole. Here is what a poster on stack overflow wrote.

"An integration test is done to demonstrate that different pieces of the system work together. Integration tests cover whole applications, and they require much more effort to put together. They usually require resources like database instances and hardware to be allocated for them." [1]

Notice that for this poster a "unit test" is less likely to require a "database" than an "integration test." But to my mind, Gemstone/S undermines this distinction. In a typical Gemstone system, your application is just Smalltalk objects in a huge and persistent memory. You don't do anything special to persist them, you just create objects. So what does this do to the notion of "integration testing" in such a system? Perhaps with Gemstone, all testing is unit testing, or at least, perhaps it could be?

Lyn Headley

[1] http://stackoverflow.com/questions/5357601/whats-the-difference-between-unit-tests-and-integration-tests

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Hi James,

You make a good point about global system state. Any time a test is creating or modifying such state it can bleed into other parts of the system and cause problems for subsequent tests.

Still, I am struck by how much more likely such "bleeding" is in systems based on traditional databases. At least in a Gemstone system it is easy to create temporary versions of many of the core classes in an application. In a traditional application, where your logic is wedded to database operations, this process is harder to do on a temporary basis. If just creating a business object means adding a row to a table, then I have to make sure to delete that row after my test (or use a special "test database"). In gemstone, however, this is simply creating an object, which is eventually garbage collected.

Lyn

On Sat, Jan 11, 2014 at 9:22 PM, James Foster <[hidden email]> wrote:

Hi Lyn,

Like most Smalltalks, GemStone/S had adopted the SUnit testing framework. 

You are right that testing in GemStone/S is not quite like testing a typical application in a traditional programming language. The usual difficulties with the database (create a connection, login, …) are not present, but it isn’t quite safe to ignore the database characteristics.

Since GemStone/S is also a database you get the persistence built-in and that usually helps but sometimes adds complexity. One of the goals of unit tests is that they can be run in any order without any side-effects. If you run every test in a transaction, and abort the transaction (without any intermediate commits), then all is well. The complexity comes if you are testing code that has side effects and commits. At that point your tests are no longer completely isolated.

I recently wrote a test that showed how ExternalSession handled a compile error (unrecognized symbol) in the remote session. The test ran fine in isolation and in the test suite, but failed when run with other tests. After a few hours of investigation I found that an earlier test had defined (and committed) a global with the exact name that I was using for what was to be an unrecognized symbol (so there was no compile error!). 

Also, it still makes some sense to have tests with different breadth of scope—whether you consider the labels to be unit/integration or something else. The concepts of modularity and encapsulation are still important, and tests that apply to one module are useful to allow those working on that module to reduce their dependency on other modules in order to do some testing. Of course, you still need to test that the modules work with each other, but when that fails you can often create a narrower test that can be isolated to a specific module.

So, integration testing is easier (everything is easier with GemStone!), but still required and something distinct from unit testing.

James

On Jan 11, 2014, at 9:00 PM, Lyn Headley <[hidden email]> wrote:

Hello Glass! I am still excited about Gemstone/S and I wanted to start a conversation about testing.

It strikes me that a system based on Gemstone/S can be tested in a more natural and seamless way than a system based on a "database." Have you ever written tests that interact with a database? I have, and it is not as smooth as writing tests for code based on the native data structures of a programming language. You have to establish connections and submit queries in ways that obscure the logic of the code you are trying to test.

Testing experts sometimes use the terms "unit testing" and "integration testing" to refer to different kinds of testing. A "unit test" is a test of a small piece of your system while an integration is something more complicated, approaching the system as a whole. Here is what a poster on stack overflow wrote.

"An integration test is done to demonstrate that different pieces of the system work together. Integration tests cover whole applications, and they require much more effort to put together. They usually require resources like database instances and hardware to be allocated for them." [1]

Notice that for this poster a "unit test" is less likely to require a "database" than an "integration test." But to my mind, Gemstone/S undermines this distinction. In a typical Gemstone system, your application is just Smalltalk objects in a huge and persistent memory. You don't do anything special to persist them, you just create objects. So what does this do to the notion of "integration testing" in such a system? Perhaps with Gemstone, all testing is unit testing, or at least, perhaps it could be?

Lyn Headley

[1] http://stackoverflow.com/questions/5357601/whats-the-difference-between-unit-tests-and-integration-tests

_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass