Smalltalk
› Squeak
› Croquet
›
Croquet - User
cross application/language communication
Locked
12
12
cross application/language communication
 
|
Florent THIERY wrote :
"- can croquet launch applications with command line arguments such as an IP adress? This would allow direct launching of http://en.wikipedia.org/wiki/NINJAM for example" I believe the question was could croquet make something like a unix command shell which, in turn, would call to a remote application. Interesting idea. Though I think interface is something else than running (and selling) other peoples programs, I excitedly used such command line interface to run my own. I was very excited to search mamasan archive of Apple programmers to find how to do a unix command line call wrapped inside a graphic interface. My gut gleefully disagreed with my defense of interface against subscription. There's a lonely genius in Silicon Valley : http://netjam.org/ I wrote him the url you gave. Florent THIERY wrote : "while you come nearer to a certain artist, you hear his music; it would be like walking in a marketplace" there was a gps project that had folks walk around a real 3D musesum space and the sound they were listening to would change. I forgot the reference. There is a metaphor story about open software to explain it : "The Cathedral and the Bazarre". I fooled with binding things between languages long ago. In my industry experience, you find legacy programming languages defended by old men who invested their lives in it. The young wish to invest their own lives. Young languages might be more fruitful and yet stifled by jealous old men. So achieving bindings between languages might instead stifle polemics of old and young to allow language development. |
Re: cross application/language communication
|
|
On Apr 2, 2007, at 6:29 PM, [hidden email] wrote: > Florent THIERY wrote : > "while you come nearer to a certain artist, you hear his music; it > would be like walking in a marketplace" not ready for prime time, midi only, not updated to the newest croquet release, but maybe of interest in this discussion: http://www.iam.unibe.ch/~scg/Archive/Papers/Thal06aJamTomorrow.pdf http://map.squeak.org/package/be9e93d1-744f-4f9f-bafa-5dc4298dfca2 Please tell me if you are so interested to be included as a developer, I would certainly love to see some update and progress here. (...what's in name... ;-) ) Markus |
Re: cross application/language communication
|
|
The existing KAT demo in hedgehog includes a trivial example of this kind of
thing. The space with an electronic drum kit on its portal features a number of flat panels. Visitors can hit these panels (click on them with the mouse) to make them play their short sound. Visitors can move panels in real-time or move their avatars to adjust volume and spatialization. [Playback is polyphonic, so one or more visitors can each click on multiple sounds to play simultaneously. However, the standard Croquet Window (panel) interface allows only one user to click on something at a time. This could be changed... Another limitation is that the FIRST time you visit and a sound is played, your system needs to cache the sound. (This is done automatically through a separate channel, in the same way that high-res textures are cached.) This can currently take longer than the playback time. After that, you have it. These are trivial limitations. A more serious one is that the latency between striking a "drum head" and hearing the playback can be long enough to interfere with the experience.] Although deliberately childishly simple, this already includes much of the key features described in the Jam Tomorrow paper. (Great paper, by the way!) For example, Jam Tomorrow is already restricted to using pre-distributed sampled sounds. The Add-and-generate process from the paper is simpler in the electronic drum kit example: just drag in a pre-recorded sound from your desktop. Croquet itself distributes the sound, rather than an application-specific thing. This all raises some subtle tradeoffs that I think are very interesting. On the one hand, you want a complete "whole product" application, not just an exercise, and one way to do that is to work really hard to cover the issues and technologies that those in the application domain have determined are important. On the other hand, reproducing everything from the old media can be both difficult for the implementor and overwhelming for the user. The electronic drum kit is deliberately made to be approachable by anyone, starting with my five year old son. In the domain of traditional music, the corners cut by the electronic drum kit are unacceptable. But in a new medium judged on its own terms? This tradeoff of "whole product in the old domain" vs. "use the new medium as-is" applies for both technical issues (e.g., support for MIDI, which Jam Tomorrow took for granted as being necessary, but which neither approach ends up delivering) and for functional issues (e.g., how the users coordinate time for live or for recorded playback). [My instinct is that one has to be far more creative than I am in order to overcome functional issues in a new medium, but technical issues are trivial to ignore.] It might also be interesting to see how this tradeoff can inform design decisions going forward. I would like Croquet to have a general recording of inputs (like http://croquet-bento.blogspot.com/2006/07/living-dead-robotsanimatronic.html) and playback on a timeline. As I imagine it, users would use multiple selection to choose which objects should have their event history included in a merged timeline. The timeline can be directly edited to effect playback, but it also provides a handle for editing the events themselves (e.g., to edit the externally provided sound that was input at a given point). This is a different model than what might come out of an approach that seeks to reproduce what music professionals do. But on the other hand, the same technology and interface would apply for machinima, video, factory automation, accounting... Markus Gaelli wrote: > > On Apr 2, 2007, at 6:29 PM, [hidden email] wrote: > >> Florent THIERY wrote : >> "while you come nearer to a certain artist, you hear his music; it >> would be like walking in a marketplace" > > not ready for prime time, midi only, not updated to the newest croquet > release, but maybe of interest in this discussion: > > http://www.iam.unibe.ch/~scg/Archive/Papers/Thal06aJamTomorrow.pdf > http://map.squeak.org/package/be9e93d1-744f-4f9f-bafa-5dc4298dfca2 > > Please tell me if you are so interested to be included as a developer, I > would certainly love to see some update and progress here. (...what's in > name... ;-) ) > > Markus -- Howard Stearns University of Wisconsin - Madison Division of Information Technology mailto:[hidden email] jabber:[hidden email] office:+1-608-262-3724 mobile:+1-608-658-2419 |
Re: cross application/language communication
|
|
Hi
About the musical features: it's simply awesome ! And, considering croquet, the most natural way is using MIDI, as MIDI data is so much lighter than raw sound... I play some instruments, in which there is a MIDI percussion (Roland Handsonic 10), i'd be glad to try using this ! The only thing one has to do for WAN interaction, is define a synchronized metronome among participants, and add a 4 beats delay for everybody. You don't play simultaneously, but with a round retard. That's ninjam's concept... We can do the same analogy with 3D avatar animation VS video stream. If we managed to track a user's body movement in realtime using a camera/webcam (as many as the avatar's degrees of liberty allow it), we could animate avatars without transferring the video itself, resulting in much lighter data flow.... That's why i'm pretty sure that Video_To_3D automatic reconstruction/modelization will give online 3D a huge boost (apart from games). Check out this awesome logitech video: http://www.youtube.com/watch?v=r7Gn2TyEyHw |
Re: cross application/language communication
|
|
> We can do the same analogy with 3D avatar animation VS video stream.
> If we managed to track a user's body movement in realtime using a > camera/webcam (as many as the avatar's degrees of liberty allow it), > we could animate avatars without transferring the video itself, > resulting in much lighter data flow.... That's why i'm pretty sure > that Video_To_3D automatic reconstruction/modelization will give > online 3D a huge boost (apart from games). Check out this awesome > logitech video: http://www.youtube.com/watch?v=r7Gn2TyEyHw I saw someone using this about a year ago and it seemed to me that this might be the future of facial animation for avatars in 3D spaces like Croquet. Cameras are being built into many of todays laptops and with the online video explosion I would expect most PCs to be bundled with one. Anyone who's read the novel Infinite Jest knows that the video phone is doomed to failure. ;-) Peter Moore University Technology Development Center University of Minnesota [hidden email] |
Re: cross application/language communication
|
|
On Apr 3, 2007, at 17:57 , Peter Moore wrote: >> We can do the same analogy with 3D avatar animation VS video stream. >> If we managed to track a user's body movement in realtime using a >> camera/webcam (as many as the avatar's degrees of liberty allow it), >> we could animate avatars without transferring the video itself, >> resulting in much lighter data flow.... That's why i'm pretty sure >> that Video_To_3D automatic reconstruction/modelization will give >> online 3D a huge boost (apart from games). Check out this awesome >> logitech video: http://www.youtube.com/watch?v=r7Gn2TyEyHw > > I saw someone using this about a year ago and it seemed to me that > this might be the future of facial animation for avatars in 3D > spaces like Croquet. Cameras are being built into many of todays > laptops and with the online video explosion I would expect most PCs > to be bundled with one. Anyone who's read the novel Infinite Jest > knows that the video phone is doomed to failure. ;-) Yeah, where's Juanita from Snow Crash when you need her ... - Bert - |
Re: cross application/language communication
|
|
In reply to this post by Peter Moore-5
...
http://www-evasion.imag.fr/Membres/Lionel.Reveret/animated-speakers/index.html Any plans yet for such work into croquet? |
Re: cross application/language communication
|
|
In reply to this post by Bert Freudenberg
Snow Crash? Never heard of it. ;-)
It looks like Logitech licensed the facial recognition tech from something called Neven Vision has been acquired by Google (suprise) for something called Picasa, a free photo-organizing app. Does anyone know of any good open-source projects for facial recognition/tracking? Peter Moore University Technology Development Center University of Minnesota On Apr 3, 2007, at 11:09 AM, Bert Freudenberg wrote:
|
Re: cross application/language communication
|
|
In reply to this post by Florent THIERY-2
I imagine that the newly formed Croquet Consortium will allow us to explore adding these kinds of technologies to Croquet. Peter Moore University Technology Development Center University of Minnesota On Apr 3, 2007, at 11:33 AM, Florent THIERY wrote:
|
Re: cross application/language communication
|
|
In reply to this post by Peter Moore-5
> Does anyone know of any good open-source projects for facial
> recognition/tracking? There are tons of things on sourceforge... I'll post some links later. Still, i found something that (finally) looks useable: http://mxrtoolkit.sourceforge.net/ As an application of MXRToolkit, 3DLive is the augmented reality system capturing a real person from many viewpoints, then re-generating and superimposing the 3D image of that person onto the real fiducial markers in real time. This system has 2 parts. The first part, the 3DLive Capturing, is responsible for receiving captured images of the real object from many cameras, and subtracting the background scene to get the foreground images from different viewpoints. The second part, 3DLive Rendering, calculates the virtual viewpoint, generates and renders the 3D image from that viewpoint onto the markers. |
Re: cross application/language communication
|
|
In reply to this post by Peter Moore-5
Huh? Anybody can add anything, you do not need the Consortium's
consent for that. - Bert - On Apr 3, 2007, at 18:39 , Peter Moore wrote: > I imagine that the newly formed Croquet Consortium will allow us to > explore adding these kinds of technologies to Croquet. > > Peter Moore > University Technology Development Center > University of Minnesota > [hidden email] > > On Apr 3, 2007, at 11:33 AM, Florent THIERY wrote: > >> ... >> >> http://www-evasion.imag.fr/Membres/Lionel.Reveret/animated- >> speakers/index.html >> >> Any plans yet for such work into croquet? > |
Re: cross application/language communication
|
|
Sorry Bert, I think you misunderstood my meaning of "allow". What I
meant is that the Consortium might set certain priorities and establish funding for development of interesting technology. I am certainly not speaking for the Consortium when I say this of course. On Apr 3, 2007, at 12:15 PM, Bert Freudenberg wrote: > Huh? Anybody can add anything, you do not need the Consortium's > consent for that. > > - Bert - > > On Apr 3, 2007, at 18:39 , Peter Moore wrote: > >> I imagine that the newly formed Croquet Consortium will allow us >> to explore adding these kinds of technologies to Croquet. >> >> Peter Moore >> University Technology Development Center >> University of Minnesota >> [hidden email] >> >> On Apr 3, 2007, at 11:33 AM, Florent THIERY wrote: >> >>> ... >>> >>> http://www-evasion.imag.fr/Membres/Lionel.Reveret/animated- >>> speakers/index.html >>> >>> Any plans yet for such work into croquet? >> > > > > |
Re: cross application/language communication
|
|
What I meant is that the Consortium might set certain priorities and establish funding for development of interesting technology. Regarding this particular subject (funding), are there possibilities of sponsored doctorate thesis, or jobs, in the opencroquet context? Florent |
Re: cross application/language communication
|
|
In reply to this post by Howard Stearns
Hi, Howard,
If one only provides record and playback, much can still be accomplished. If the waveform recorded is maintained as a simple file type, whose format, and organization are Open, then music rendering utilities could generate new waveforms to provide unique sounds. For instance, combining a ring-modulator (which creates the effect of a bell ringing for example) with a tonal generator (to produce off set tones) and an ADSR (attack, decay, sustain and release) modulation envelope, one could generate gongs or bells with the resonance and key characteristics of a piano. A sequence of these could be saved as a wave file, then combined at runtime to add that component to an orchestral composition. The editor could be made independent, with a play button so the composition, both the single instrument being created, and the total composition could be played as new information is added. If the editor also permitted clock editing against the wave files, the relative timing could even be adjusted, and bits played back against one another for shorter intervals to speed the editing process, kind of like the window of a text editor looks at only a part of a page. Some adventuresome soul could add effects like scroll bars or maybe something like an amenusis to help create phonically correct compositions (like a spell checker for music). Big project, but possible to divide into bits to make it play. Don't know if there is any equivalent or if there would be patent issues with this form. Someone would need to do the research on how this could be accomplished without violating some rule. If not previously patented, or copyrighted, I permit use of this idea under GPL as my idea at this time. regards, Les H On Tue, 2007-04-03 at 10:21 -0500, Howard Stearns wrote: > The existing KAT demo in hedgehog includes a trivial example of this kind of > thing. The space with an electronic drum kit on its portal features a number of > flat panels. Visitors can hit these panels (click on them with the mouse) to > make them play their short sound. Visitors can move panels in real-time or move > their avatars to adjust volume and spatialization. > > [Playback is polyphonic, so one or more visitors can each click on multiple > sounds to play simultaneously. However, the standard Croquet Window (panel) > interface allows only one user to click on something at a time. This could be > changed... Another limitation is that the FIRST time you visit and a sound is > played, your system needs to cache the sound. (This is done automatically > through a separate channel, in the same way that high-res textures are cached.) > This can currently take longer than the playback time. After that, you have it. > These are trivial limitations. A more serious one is that the latency between > striking a "drum head" and hearing the playback can be long enough to interfere > with the experience.] > > Although deliberately childishly simple, this already includes much of the key > features described in the Jam Tomorrow paper. (Great paper, by the way!) For > example, Jam Tomorrow is already restricted to using pre-distributed sampled > sounds. The Add-and-generate process from the paper is simpler in the > electronic drum kit example: just drag in a pre-recorded sound from your > desktop. Croquet itself distributes the sound, rather than an > application-specific thing. > > This all raises some subtle tradeoffs that I think are very interesting. > > On the one hand, you want a complete "whole product" application, not just an > exercise, and one way to do that is to work really hard to cover the issues and > technologies that those in the application domain have determined are important. > > On the other hand, reproducing everything from the old media can be both > difficult for the implementor and overwhelming for the user. The electronic drum > kit is deliberately made to be approachable by anyone, starting with my five > year old son. In the domain of traditional music, the corners cut by the > electronic drum kit are unacceptable. But in a new medium judged on its own terms? > > This tradeoff of "whole product in the old domain" vs. "use the new medium > as-is" applies for both technical issues (e.g., support for MIDI, which Jam > Tomorrow took for granted as being necessary, but which neither approach ends up > delivering) and for functional issues (e.g., how the users coordinate time for > live or for recorded playback). [My instinct is that one has to be far more > creative than I am in order to overcome functional issues in a new medium, but > technical issues are trivial to ignore.] > > It might also be interesting to see how this tradeoff can inform design > decisions going forward. I would like Croquet to have a general recording of > inputs (like > http://croquet-bento.blogspot.com/2006/07/living-dead-robotsanimatronic.html) > and playback on a timeline. As I imagine it, users would use multiple selection > to choose which objects should have their event history included in a merged > timeline. The timeline can be directly edited to effect playback, but it also > provides a handle for editing the events themselves (e.g., to edit the > externally provided sound that was input at a given point). This is a different > model than what might come out of an approach that seeks to reproduce what music > professionals do. But on the other hand, the same technology and interface would > apply for machinima, video, factory automation, accounting... > > > Markus Gaelli wrote: > > > > On Apr 2, 2007, at 6:29 PM, [hidden email] wrote: > > > >> Florent THIERY wrote : > >> "while you come nearer to a certain artist, you hear his music; it > >> would be like walking in a marketplace" > > > > not ready for prime time, midi only, not updated to the newest croquet > > release, but maybe of interest in this discussion: > > > > http://www.iam.unibe.ch/~scg/Archive/Papers/Thal06aJamTomorrow.pdf > > http://map.squeak.org/package/be9e93d1-744f-4f9f-bafa-5dc4298dfca2 > > > > Please tell me if you are so interested to be included as a developer, I > > would certainly love to see some update and progress here. (...what's in > > name... ;-) ) > > > > Markus > |
|
|
In reply to this post by Florent THIERY-2
On Tue, 2007-04-03 at 20:39 +0200, Florent THIERY wrote:
> What I meant is that the Consortium might set certain > priorities and establish funding for development of > interesting technology. > > Regarding this particular subject (funding), are there possibilities > of sponsored doctorate thesis, or jobs, in the opencroquet context? Under most open-source models where you are in "gift-debt" and "stand on the shoulders of giants" to begin with and any improvements due back to the initial developer, you do amass "street-cred" by your contributions. The more you appear significantly in the contrib file, the more notice you receive and the opportunities theoretically then come to you. Or, opportunities are made... using your contributions as a significant part of your resume or application to some University system for enrollment or even a grant for development in a specific area. In all cases possibilities are always there. They just don't always arrive on our time-schedule. To me that point is where I exercise a little Faith in the Universe to provide. I can't say in all cases have I gotten what I wanted, but in all cases I got what I needed every time, even if it was a rough time. I personally had to be aware that possibilities would come my way and seize the moment responsibly when they did. That is admittedly the tough part. While I'm not eating rib-eyes for breakfast, the providence of the Universe has never failed to provide, as long as I'm on track. Stay focused on your Goal, as well as your unique purposes. Like in that Kevin Costner movie, "Build it and they will come." <sermon /off> :) Ric -- ================================================ My father, Victor Moore (Vic) used to say: "There are two Great Sins in the world... ..the Sin of Ignorance, and the Sin of Stupidity. Only the former may be overcome." R.I.P. Dad. Linux user# 44256 Sign up at: http://counter.li.org/ http://www.sourceforge.net/projects/oar http://www.wayward4now.net ================================================ |
Re: cross application/language communication
|
|
In reply to this post by Peter Moore-5
This one is apparently a defunct project
http://expression.sourceforge.net/news.html This one seems to have stalled in 2005 http://sourceforge.net/projects/expression This is ongoing research into tracking. http://www.umiacs.umd.edu/~knkim/SoftwareDevelopment-KimUMD.htm The issue with facial tracking (indeed any form of real time analysis of 3d motion) is how to achieve the 3d data. One method relies on movement of either camera or subject. If the amount of motion is known or can be calculated, that constitutes the offset for triangulation of points, yeilding the 3d data. Another is the use of stereo camera(s) to present two views with a known offset, which then allows the triangulation computation to find 3d points. Yet another utilizes frequency separation, that is different color filters to break the scene down by wavelength, and utilizes calculation of the phase error to compute distance in wavelength. This has not seen wide use except in radar imaging. Light is still too physically limited for phase resolution, although some work is being done in this area. Finally the number of computations (a mainimum of a sine function, two multiplies, and a divide to achieve the distance to each point chosen, a data reduction algorithm to chose points, and a reconstitution of data to achieve the resultant base figure or armature on which to post the synthetic face, along with default mechanisms for any obscured points or non captured points) is really time consuming. Generally these algorithms are run on some form of parallel system where computation is in the gigaflops to achieve sufficient through put. Not to say that someone couldn't implement this in some form of 3d transform and radically reduce the computational overhead, but at this time, the math is a limiting factor. As an example of the rendering check out blender and run some of the rendering, along with rotational algorithms. It is close to realtime on my current system, but the facial tracking would triple the computation or more to achieve reasonable results for real time tracking. A good simulation might get by with tracking a few points, such as the eyebrows, eyes, corners of the mouth and the nose, then using some library to create a few end points and some morphing to achieve intermediate results. This might be possible with reduced delays. This could give reasonable movement between end points, and result in psuedo realistic animation. Regards, Les H |
Re: cross application/language communication
|
|
A good simulation might get by with tracking a few points, such as the
eyebrows, eyes, corners of the mouth and the nose, then using some I guess that's exactly what logitech does in the demo video... Only eyes, eyebrows, general head orientation and mouth. |
3D Video Avatar technology -- Silhouette by IVN, Inc.
|
|
As I have been the Chief Operations Officer of IVN for some time, I am very familiar with the Silhouette technology, and even contributed to it to some extent. Silhouette, a patented invention by L.A. patent lawyer and IVN-founder William Benman,... is still rough around the edges, including literally. Speaking from personal experience, I can testify that it does work though. Simply put, you point a generic webcam at the place where you normally sit. Background can be totally arbitrary, as long as there is no or very little movement in it. No blue or green background needed as with classical chromakeying. After hitting a key to set this background, you come into view and the software extracts -makes a 'silhouette' of- your (moving) video image in real time from each frame. Your moving Silhouetted video image is then transposed unto a (otherwise transparent) polygon inside a (multi-user) 3D virtual world (such as CyberTown, which William Benman bought from Blaxxun USA quite a few years ago) and your voice's audio VOIP-stream is 3D spatialized with its source roughly at the location of your head. Instead of a user's regular avatar there now is now a real-time moving video avatar. For the rest, one uses such a Silhouette video-av in just the same way as one uses regular avs. Some notes: - Heavily protected by patents made by the inventor who is himself a very astute patent lawyer. So, don't even think about it. Some of the relevant patents are referenced online -- see USPTO Patent # 5784546 and USPTO Patent # 5966130. Google on http://www.google.com/search?hl=en&q=Integrated+Virtual+Networks+patent&btnG=Search - So far, upper body only. So it -naturally- only displays (extracted and transposed) video of the parts of your body which your webcam sees. (Part of my involvement was providing a seamlessly aligning "rest-of-the-body" solution... which was not quite implemented yet the last time I used Silhouette myself.) - The Silhouetted real-time video sprite itself is 2D -- flat video. When, inside of a 3D virtual world, you go and look around the back of a Silhouette-based avatar, you see the same video as on the front, but mirrored. (Here too, one of the original goals of my involvement was to look into and possibly develop ways of not only providing the rest of the body below the shoulders, but also in full 3D. One obvious way which was explored was mapping the real-time Silhouetted video stream unto a one-mesh-fits-all polygon mesh of a generic head & shoulders. That works relatively well for men, as long as they don't have long or big hair. For women with long or big hair, it does not work, unless you provide them with a set of template 3D meshes to drape their Silhouetted video on which roughly suit the 3D space occupied by their heirdo of that particular moment. That still results -for those cases- in an often unsatisfactory hit-or-miss solution. The variety of 3D volumes occupied by female heirdos out there is just too big a space to do justice with a small set of 3D mesh templates. If there is interest in this from the Croquet community, I can offer to be the interface to IVN and William Benman to negoatiate a friendly licensing agreement.) Philippe Van Nedervelde E-SPACES SkypeID: philippevannedervelde http://www.e-spaces.com/lbw/portfolio/portfolio_time.htm |
Re: cross application/language communication
|
|
In reply to this post by Howard Stearns
On Apr 3, 2007, at 5:21 PM, Howard Stearns wrote: > The existing KAT demo in hedgehog includes a trivial example of > this kind of thing. The space with an electronic drum kit on its > portal features a number of flat panels. Visitors can hit these > panels (click on them with the mouse) to make them play their short > sound. Visitors can move panels in real-time or move their avatars > to adjust volume and spatialization. > > [Playback is polyphonic, so one or more visitors can each click on > multiple sounds to play simultaneously. However, the standard > Croquet Window (panel) interface allows only one user to click on > something at a time. This could be changed... Another limitation is > that the FIRST time you visit and a sound is played, your system > needs to cache the sound. (This is done automatically through a > separate channel, in the same way that high-res textures are > cached.) This can currently take longer than the playback time. > After that, you have it. These are trivial limitations. A more > serious one is that the latency between striking a "drum head" and > hearing the playback can be long enough to interfere with the > experience.] > > Although deliberately childishly simple, this already includes much > of the key features described in the Jam Tomorrow paper. (Great > paper, by the way!) For example, Jam Tomorrow is already restricted > to using pre-distributed sampled sounds. The Add-and-generate > process from the paper is simpler in the electronic drum kit > example: just drag in a pre-recorded sound from your desktop. > Croquet itself distributes the sound, rather than an application- > specific thing. Thanks a lot for beating the drum and for the flowers, most of them really belong to my bright student of that time Florian and of course to my former supervisors Oscar and Stef who allowed us to do such funky stuff. I am very happy to hear that you included this electronic drum kit into the KAT demo. Got to try this out. > > This all raises some subtle tradeoffs that I think are very > interesting. > > On the one hand, you want a complete "whole product" application, > not just an exercise, and one way to do that is to work really hard > to cover the issues and technologies that those in the application > domain have determined are important. > > On the other hand, reproducing everything from the old media can be > both difficult for the implementor and overwhelming for the user. > The electronic drum kit is deliberately made to be approachable by > anyone, starting with my five year old son. In the domain of > traditional music, the corners cut by the electronic drum kit are > unacceptable. But in a new medium judged on its own terms? > > This tradeoff of "whole product in the old domain" vs. "use the new > medium as-is" applies for both technical issues (e.g., support for > MIDI, which Jam Tomorrow took for granted as being necessary, but > which neither approach ends up delivering) Right, I took my mouth a bit full in the last mail. Though I think we could do midi, we "only" generated sounds with the Squeak Midi player and _then_ played the generated wavs which was kind of slow, if I remember correctly. A possibly interesting approach might be to exploit the csound sound processor (e.g. via osc [1]) to generate the same sound-wavs, be it via midi or not, on different machines. If you get over the strange syntax of csound and build some decent csound instruments yourself, much is possible. Maybe "jacking" a generated realtime csound into some OpenAL stream? Might be fun. Some links which might help to link all of this: - csound / osc: http://csounds.com/manual/html/OSClisten.html - squeak / osc: http://map.squeak.org/package/61f807be-83a3-4944-bfa1-686ddac7153c - jack: http://jackaudio.org/ - Iannix, a cool composition software, connectable to csound via osc: http://bradfuller.com/blog/2007/01/10/iannis-xenakis-lives-on/ - Csound-block building in Squeak: http://www.zogotounga.net/comp/squeak/csblocks.htm Ok, it's getting late here, one last thing: I must admit that I like "Jam tomorrow" better than "Electronic drum kit"... as a name! So it would be a great honour if whatever software jams today would be called "jam tomorrow" -- if possible yesterday... - keep the flow, Markus [1] I am wondering why not using OSC on a much grander scheme for OpenCroquet..., udp based, fast, tested, more and more common, at least in the (artistic) electronic music jamming world. > and for functional issues (e.g., how the users coordinate time for > live or for recorded playback). [My instinct is that one has to be > far more creative than I am in order to overcome functional issues > in a new medium, but technical issues are trivial to ignore.] > > It might also be interesting to see how this tradeoff can inform > design decisions going forward. I would like Croquet to have a > general recording of inputs (like http://croquet-bento.blogspot.com/ > 2006/07/living-dead-robotsanimatronic.html) and playback on a > timeline. As I imagine it, users would use multiple selection to > choose which objects should have their event history included in a > merged timeline. The timeline can be directly edited to effect > playback, but it also provides a handle for editing the events > themselves (e.g., to edit the externally provided sound that was > input at a given point). This is a different model than what might > come out of an approach that seeks to reproduce what music > professionals do. But on the other hand, the same technology and > interface would apply for machinima, video, factory automation, > accounting... > > > Markus Gaelli wrote: >> On Apr 2, 2007, at 6:29 PM, [hidden email] wrote: >>> Florent THIERY wrote : >>> "while you come nearer to a certain artist, you hear his music; it >>> would be like walking in a marketplace" >> not ready for prime time, midi only, not updated to the newest >> croquet release, but maybe of interest in this discussion: >> http://www.iam.unibe.ch/~scg/Archive/Papers/Thal06aJamTomorrow.pdf >> http://map.squeak.org/package/be9e93d1-744f-4f9f-bafa-5dc4298dfca2 >> Please tell me if you are so interested to be included as a >> developer, I would certainly love to see some update and progress >> here. (...what's in name... ;-) ) >> Markus > > -- > Howard Stearns > University of Wisconsin - Madison > Division of Information Technology > mailto:[hidden email] > jabber:[hidden email] > office:+1-608-262-3724 > mobile:+1-608-658-2419 |
Re: 3D Video Avatar technology -- Silhouette by IVN, Inc.
|
|
In reply to this post by Philippe Van Nedervelde
On Tue, 2007-04-03 at 21:59 +0100, Philippe Van Nedervelde wrote:
> Check out the "Silhouette" technology by Integrated Virtual networks > (IVN) -- http://www.ivn3.com/technology0.php > > As I have been the Chief Operations Officer of IVN for some time, I am > very familiar with the Silhouette technology, and even contributed to > it to some extent. Silhouette, a patented invention by L.A. patent > lawyer and IVN-founder William Benman,... is still rough around the > edges, including literally. Speaking from personal experience, I can > testify that it does work though. > > Simply put, you point a generic webcam at the place where you normally > sit. Background can be totally arbitrary, as long as there is no or > very little movement in it. No blue or green background needed as with > classical chromakeying. After hitting a key to set this background, > you come into view and the software extracts -makes a 'silhouette' of- > your (moving) video image in real time from each frame. Your moving > Silhouetted video image is then transposed unto a (otherwise > transparent) polygon inside a (multi-user) 3D virtual world (such as > CyberTown, which William Benman bought from Blaxxun USA quite a few > years ago) and your voice's audio VOIP-stream is 3D spatialized with > its source roughly at the location of your head. Instead of a user's > regular avatar there now is now a real-time moving video avatar. For > the rest, one uses such a Silhouette video-av in just the same way as > one uses regular avs. > > Some notes: > > - Heavily protected by patents made by the inventor who is himself a > very astute patent lawyer. So, don't even think about it. Some of the > relevant patents are referenced online -- see USPTO Patent # 5784546 > and USPTO Patent # 5966130. Google on > http://www.google.com/search?hl=en&q=Integrated+Virtual+Networks > +patent&btnG=Search > > - So far, upper body only. So it -naturally- only displays (extracted > and transposed) video of the parts of your body which your webcam > sees. (Part of my involvement was providing a seamlessly aligning > "rest-of-the-body" solution... which was not quite implemented yet the > last time I used Silhouette myself.) > > - The Silhouetted real-time video sprite itself is 2D -- flat video. > When, inside of a 3D virtual world, you go and look around the back of > a Silhouette-based avatar, you see the same video as on the front, but > mirrored. (Here too, one of the original goals of my involvement was > to look into and possibly develop ways of not only providing the rest > of the body below the shoulders, but also in full 3D. One obvious way > which was explored was mapping the real-time Silhouetted video stream > unto a one-mesh-fits-all polygon mesh of a generic head & shoulders. > That works relatively well for men, as long as they don't have long or > big hair. For women with long or big hair, it does not work, unless > you provide them with a set of template 3D meshes to drape their > Silhouetted video on which roughly suit the 3D space occupied by their > heirdo of that particular moment. That still results -for those cases- > in an often unsatisfactory hit-or-miss solution. The variety of 3D > volumes occupied by female heirdos out there is just too big a space > to do justice with a small set of 3D mesh templates. > > If there is interest in this from the Croquet community, I can offer > to be the interface to IVN and William Benman to negoatiate a friendly > licensing agreement.) > I read the two patent listings that came up on Google, and as that runs it is a trivial extension of prior art. The means described have been available from a number of image manipulation packages for many years. I cannot argue patent applicability, as I am not an attorney, but simply a programmer with over 3 decades of experience. However what I have proposed is much more complex, and may or may not be easily produced. It is much closer to puppet animation technology, and matches with work done in Blender and several other 3D packages only adding the facial tracking capability. Regards, Les H |
«
Return to Croquet - User
|
1 view|%1 views
| Free forum by Nabble | Edit this page |