[ANN] Tuio - Tangible User Interaction by video tracking physical objects.

Hi All

I have just uploaded the first version of Tuio for Squeak.

Tuio allows Squeak to use the reacTIVision computer vision system to track physical objects in real time. The package includes a Tuio client and demonstration application program. To make practical use of Tuio for Squeak you will need a webcam or other camera, and should download the free Reactivision application for your operating system.  To learn more, see http://mcs.open.ac.uk/sh2/squeakmusic.html).

Tuio for Squeak makes it easy to build tangible user interfaces using fiducial markers on physical objects.

Tuio for Squeak allows the rapid development of table-based tangible user interfaces (TUI) and multi-touch interactive surfaces.

Prerequisite Package -  OSC 5.1 or later
Please let me know if you find any problems.
Thanks

Simon Holland

Very cool.

* Are there non-table uses (such as registering facial features, as
opposed to opentouch fingers-on-a-table stuff)? I couldn't find demo
pics/movies of any.

* Do I understand correctly that:

   + Tuio for Squeak and the pre-requisite OSC are MIT license.

   + Any application will require that the user separately have the
reacTIVision framework installed, which is GPL (not LGPL). Being
separately installable, it should be sufficiently "arms length" to use
with non GPL licenses.

   + No other parts to worry about?

-Howard

Simon Holland wrote:

Hope I didn't mislead anyone by keeping the announcement brief.

What the video camera actually tracks is  small paper markers  (called fiducial markers) that you can print off and attach to things. The camera can sense the orientation of each marker, and each marker also has a number that can be parsed from it. This may sound a bit meh but can be used to quite spectacular effect, especially using a light table.  You put the markers on the bottom of objects  and move, remove and twist  them around on a transparent table underlaid with semi-transparent paper. The camera then goes underneath the table, along with a data projector so that you can back project the UI.
Try it in a semi-dark room.

The reactivision engine is a separate, cross-platform open source program, mostly GPL but cobbled together from subcomponents with various open-source type licences. But that component is distinct and stand alone from anything to do with squeak. The engine squirts out OSC at a rate of knots and Tuio for squeak, which is MIT licence, picks this up & parses it to make morphs dance. OSC for Squeak is also MIT licence.

Thats it. Then you just need a video camera or webcam.

More can be found at
http://mcs.open.ac.uk/sh2/squeakmusic.html
and here
http://mtg.upf.es/reactable/?software

Look at the great video here ** NB proprietary NON-open source music software NOT available**
http://mtg.upf.es/reactable/?media
of  NON-AVAILABLE music software using the original barcelona version of Tuio and the Reactivision engine. Gives you an idea of what can be done with a Tuio & a light table.


Cheers
Simon Holland

<quote author="Howard Stearns-3">
Very cool.

* Are there non-table uses (such as registering facial features, as
opposed to opentouch fingers-on-a-table stuff)? I couldn't find demo
pics/movies of any.

* Do I understand correctly that:

   + Tuio for Squeak and the pre-requisite OSC are MIT license.

   + Any application will require that the user separately have the
reacTIVision framework installed, which is GPL (not LGPL). Being
separately installable, it should be sufficiently "arms length" to use
with non GPL licenses.

   + No other parts to worry about?

-Howard

"Simon Holland" <[hidden email]> wrote:

> Hi All
>
> I have just uploaded the first version of Tuio for Squeak.
>
> Tuio allows Squeak to use the reacTIVision computer vision system to track
> physical objects in real time. The package includes a Tuio client and
> demonstration application program. To make practical use of Tuio for
Squeak
> you will need a webcam or other camera, and should download the free
> Reactivision application for your operating system.  To learn more, see
> http://mcs.open.ac.uk/sh2/squeakmusic.html).
>
> Tuio for Squeak makes it easy to build tangible user interfaces using
> fiducial markers on physical objects.
>
> Tuio for Squeak allows the rapid development of table-based tangible user
> interfaces (TUI) and multi-touch interactive surfaces.

Fantastic stuff!

There's also the NUI group [1] and David Wallin [2] working on multitouch
UIs. David Wallin, in particular, works on touchlib [3], a TUIO-based
multitouch library. touchlib will (AFAIK) be adding fiducial support as well
[4].

Simon, with your TUIO client - does this mean that one could use other
libraries (say, touchlib) to send data to Tuio for Squeak?

Since touchlib's licence is the New BSD Licence, this might address Howard
Stearns' licence concerns.

frank

[1] http://www.nuigroup.com/
[2] http://www.whitenoiseaudio.com/blog/
[3] http://www.whitenoiseaudio.com/touchlib/
[4]
http://www.whitenoiseaudio.com/blog/2007/06/fiducial-detection-first-hack.html

Frank,

I have implemented touchlib support for visual smalltalk, and it is easy
 to port the interface to Squeak (the wraper use normal API calls
 and 4 callbacks for event catching).
We are using touchlib interface for custom (FTIR) multitouch based
 devices and also use direct inetrfaces to OpenCV for high perfomance
 video and image processing (photograph based products).
The sources (for visual smalltalk) can be downloaded from our swiki
 at http://www.aleReimondo.com.ar/Multitouch
 and http://www.aleReimondo.com.ar/OpenCV
 both are open projects and anyone can participate.
If there is interest in touchlib interface to be ported to Squeak I can
 guide someone to write port or help to doit.

cheers,
Ale.


----- Original Message -----
From: "Frank Shearar" <[hidden email]>
To: "The general-purpose Squeak developers list"
<[hidden email]>
Sent: Friday, October 05, 2007 5:04 AM
Subject: Re: [ANN] Tuio - Tangible User Interaction by video
trackingphysical objects.

Thanks Frank. That’s very useful.
Since touchlib now uses the TUIO protocol, I would be very surprised if Tuio for Squeak did not work with touchlib right away (possibly after adjusting a port number), and I would expect the basic demo app to run too. However, I haven’t tested it yet, as the install for Touchlib on OS X seems a bit daunting – it was Windows-only originally. I will investigate but  would be interested to hear from anyone with touchlib already running on any platform (it wouldn’t be that hard to do a quick collaborative test over the net).

I’m assuming that  touchlib uses the  2D Tuio protocol, rather than the 2.5D or 3D, though it would be no big deal to extend Tuio for Squeak to deal with those – I’ll investigate just in case touchlib uses any little used parts of OSC and Tuio.

Interoperability with touchlib would indeed be valuable as  this would make it easy to write Tangible user interaction applications in Squeak based on blob recognition and tracking, and with  FITR technologies as well as with fiducials.

Cheers
Simon

Howard Stearns-3 wrote

Very cool.

* Are there non-table uses (such as registering facial features, as
opposed to opentouch fingers-on-a-table stuff)? I couldn't find demo
pics/movies of any.

* Do I understand correctly that:

   + Tuio for Squeak and the pre-requisite OSC are MIT license.

   + Any application will require that the user separately have the
reacTIVision framework installed, which is GPL (not LGPL). Being
separately installable, it should be sufficiently "arms length" to use
with non GPL licenses.

   + No other parts to worry about?

-Howard

Simon Holland wrote:
> Hi All
>
> I have just uploaded the first version of Tuio for Squeak.
>
> Tuio allows Squeak to use the reacTIVision computer vision system to track
> physical objects in real time. The package includes a Tuio client and
> demonstration application program. To make practical use of Tuio for Squeak
> you will need a webcam or other camera, and should download the free
> Reactivision application for your operating system.  To learn more, see
> http://mcs.open.ac.uk/sh2/squeakmusic.html).
>
> Tuio for Squeak makes it easy to build tangible user interfaces using
> fiducial markers on physical objects.
>
> Tuio for Squeak allows the rapid development of table-based tangible user
> interfaces (TUI) and multi-touch interactive surfaces.
>
> Prerequisite Package -  OSC 5.1 or later
> Please let me know if you find any problems.
> Thanks
>
> Simon Holland
>

Oops - sorry - I previously posted this to the wrong place in the thread

Thanks Frank. That’s very useful.
Since touchlib now uses the TUIO protocol, I would be very surprised if Tuio for Squeak did not work with touchlib right away (possibly after adjusting a port number), and I would expect the basic demo app to run too. However, I haven’t tested it yet, as the install for Touchlib on OS X seems a bit daunting – it was Windows-only originally. I will investigate but  would be interested to hear from anyone with touchlib already running on any platform (it wouldn’t be that hard to do a quick collaborative test over the net).

I’m assuming that  touchlib uses the  2D Tuio protocol, rather than the 2.5D or 3D, though it would be no big deal to extend Tuio for Squeak to deal with those – I’ll investigate just in case touchlib uses any little used parts of OSC and Tuio.

Interoperability with touchlib would indeed be valuable as  this would make it easy to write Tangible user interaction applications in Squeak based on blob recognition and tracking, and with  FITR technologies as well as with fiducials.

Cheers
Simon

Frank Shearar-2 wrote

"Simon Holland" <s.holland@open.ac.uk> wrote:

> Hi All
>
> I have just uploaded the first version of Tuio for Squeak.
>
> Tuio allows Squeak to use the reacTIVision computer vision system to track
> physical objects in real time. The package includes a Tuio client and
> demonstration application program. To make practical use of Tuio for
Squeak
> you will need a webcam or other camera, and should download the free
> Reactivision application for your operating system.  To learn more, see
> http://mcs.open.ac.uk/sh2/squeakmusic.html).
>
> Tuio for Squeak makes it easy to build tangible user interfaces using
> fiducial markers on physical objects.
>
> Tuio for Squeak allows the rapid development of table-based tangible user
> interfaces (TUI) and multi-touch interactive surfaces.

Fantastic stuff!

There's also the NUI group [1] and David Wallin [2] working on multitouch
UIs. David Wallin, in particular, works on touchlib [3], a TUIO-based
multitouch library. touchlib will (AFAIK) be adding fiducial support as well
[4].

Simon, with your TUIO client - does this mean that one could use other
libraries (say, touchlib) to send data to Tuio for Squeak?

Since touchlib's licence is the New BSD Licence, this might address Howard
Stearns' licence concerns.

frank

[1] http://www.nuigroup.com/
[2] http://www.whitenoiseaudio.com/blog/
[3] http://www.whitenoiseaudio.com/touchlib/
[4]
http://www.whitenoiseaudio.com/blog/2007/06/fiducial-detection-first-hack.html

I am building a multi-touch screen using FTIR [1].
I installed Touchlib on Windows and it communicates with Squeak through OSC (Open Sound Control) and it works fine.
Since you can download ready-to-use executables for Windows [2] it was very easy to set it up. I haven't tried the OS X version, yet.
Yes, Touchlib uses the 2D Tuio protocol.

[1] http://cs.nyu.edu/~jhan/ftirsense/
[2] http://nuigroup.com/touchlib/downloads/

Regards, Grit.

<quote author="Simon Holland">

Thanks Frank. That’s very useful.
Since touchlib now uses the TUIO protocol, I would be very surprised if Tuio for Squeak did not work with touchlib right away (possibly after adjusting a port number), and I would expect the basic demo app to run too. However, I haven’t tested it yet, as the install for Touchlib on OS X seems a bit daunting – it was Windows-only originally. I will investigate but  would be interested to hear from anyone with touchlib already running on any platform (it wouldn’t be that hard to do a quick collaborative test over the net).

I’m assuming that  touchlib uses the  2D Tuio protocol, rather than the 2.5D or 3D, though it would be no big deal to extend Tuio for Squeak to deal with those – I’ll investigate just in case touchlib uses any little used parts of OSC and Tuio.