Smalltalk › Squeak › Squeak - Dev

Quicktime player

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

4 messages Options

Chris Cunnington

Quicktime player

Seems I was labouring under a misapprehension. The MPEG plugin basically does what one would expect. I thought the Quicktime plugin would be as self sufficient. Then I could abandon the FFI. Nope. The QT plugin does ... little. It has six primitives. It's been around since Squeak 3.8, and if it does anything substantial, I can't see what it is.

I thought Quicktime.framework (Mac HD>>System>>Library>>Frameworks) could be dispensed with. It's so indispensable that Sophie won't open without it. It won't blink if the Quicktime plugin is missing.

Soooooo, for this to work at all Quicktime.framework it is. I thought there was a QT player somewhere in Squeak's history. There isn't.

The subclassing of the movie class to the audio class is to emulate what's happening in Objective-C. A subclass of NSSound or such. I guess you're using semaphores to represent NSTimeIntervals.

I still can't get a clear grok on what gworld is. I suppose the "g" stands for OpenGL or what. It's some kind of skin you can place to move the projection screen to a place where it's not expected: like a Form.

And these buttons are being supplied by QTKit somewhere. And the whole thing will play on Windows with things like QTMLClient.dll. Not gonna fly on Linux, though. As you said.

Chris

johnmci

Re: Quicktime player

On 2010-09-08, at 5:06 PM, Chris Cunnington wrote:

Seems I was labouring under a misapprehension. The MPEG plugin basically does what one would expect. I thought the Quicktime plugin would be as self sufficient. Then I could abandon the FFI. Nope. The QT plugin does ... little. It has six primitives. It's been around since Squeak 3.8, and if it does anything substantial, I can't see what it is.

I thought Quicktime.framework (Mac HD>>System>>Library>>Frameworks) could be dispensed with. It's so indispensable that Sophie won't open without it. It won't blink if the Quicktime plugin is missing.

Soooooo, for this to work at all Quicktime.framework it is. I thought there was a QT player somewhere in Squeak's history. There isn't.

The subclassing of the movie class to the audio class is to emulate what's happening in Objective-C. A subclass of NSSound or such. I guess you're using semaphores to represent NSTimeIntervals.

I still can't get a clear grok on what gworld is. I suppose the "g" stands for OpenGL or what. It's some kind of skin you can place to move the projection screen to a place where it's not expected: like a Form.

And these buttons are being supplied by QTKit somewhere. And the whole thing will play on Windows with things like QTMLClient.dll. Not gonna fly on Linux, though. As you said.

A gWorld is 1990's Apple technology, it's a structure that defines a graphical world (aka Graphics World, gWorld).

One of it's attributes is a pointer to a fixed memory address and it contains the definition of what the bits mean.

Thus you setup a RGBA 32bit 320x480 at 32bits of depth. You will find a Form Object is conceptually very similar.

Then we tell Quicktime via FFI to render the movie frame to that gWorld (aka off screen memory buffer), versus to the video display **

Now there are two problems

(a) How do you know when a frame is finished rendering?

(b) How do we get the bytes from the gWorld into a Squeak Form?

So there two approaches. The first slow way is without the Plugin. There is no plugin for Window or for Linux because it was too hard to compile the quicktime clone junkware in linux and

no-one knows how to build plugins in Windows so we abandon that as unworkable. Thus:

(a) You basically run a timer loop in squeak based on the frame rate frequency and grab the data from the gWorld every 29.97 seconds as an example.

You hope it seems reasonable and there is no frame tearing.

(b) On each poll, I believe we make a memcpy call to move the bytes from gWorld to the ByteArray instance pointed to by the Squeak Form. Then we ensure the Form is painted to the Squeak Display.

This is expensive since you are moving bytes from a QuickTime buffer, into a Squeak Oops, then immediately ask the VM to copy the Bytes from the Form to the Display oops then to

send that to the supporting platform API to send off to be rendered to the Video device(s).

Mm technically in 5.8 it would mean going from quicktime vram to gworld buffer, to Squeak ByteArray oops, to Squeak Display, to Open/GL vram ***

As you see that is quite a bit of needless byte movement.

Now the faster way is to use the Quicktime plugin which is on the mac and technically could be compiled for Windows if anyone cares.

(a) We use a FrameCompletedSemaphore, a squeak external semaphore you setup to be signalled by C code. Then we tell the Quicktime logic to tap the plugin routine which signals the Squeak Semaphore when a frame has been render. Thus we can halt the Squeak drawing logic on the external semaphore, it will wake up when the frame is ready to be processed.

primitiveClearFrameCompletedSemaphore

primitiveSetFrameCompletedSemaphore

(b) Use the cleverly done SurfacePlugin Logic.

http://wiki.squeak.org/squeak/5893

A Form can point to either a ByteArray instance, or it can have a Surface plugin surface index number. That means the storage for the Form is outside of the Squeak oops space and being rendered by something else. When you draw that form why the VM knows about the Surface plugin logic and then fetches the bytes from the SurfacePlugin (aka quicktimeplugin) that is managing the data, in this case the data still is stored in the gworld object. The data pointer is supplied to supporting platform API to send off to be rendered to the Video device(s). Which reduces lots of byte copying..

in 5.8 it would mean going from quicktime vram to gworld buffer, to Open/GL vram ***

{A bonus point in here which you would have to be checked is that you could allocate the gworld in VRAM so no DMA is done!).

That is handled by:

primitiveSetGWorldPtrOntoSurface

primitiveSetGWorldPtrOntoExistingSurface

primitiveDestroySurface

Of course the plugin has to know about the quicktime instance so it can setup the logic to hear about frame render completion.

primitiveDestroyHandle

***

{we optimize and don't do Display to VRAM, rather we allow the video hardware to DMA the bytes out of Display as part of the glflush(), this avoids the memory copy using the CPU}

** The fastest way is to go from Quicktime to Open/GL viewport overlaying the squeak window. But then you loose any ability for squeak to alter the bits since it never sees them.

===========================================================================

John M. McIntosh <[hidden email]> Twitter: squeaker68882

Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com

===========================================================================

smime.p7s (5K) Download Attachment

Bert Freudenberg

Re: Quicktime player

On 09.09.2010, at 02:57, John M McIntosh wrote:

The fastest way is to go from Quicktime to Open/GL viewport overlaying the squeak window. But then you loose any ability for squeak to alter the bits since it never sees them.

The best way would be to give the bits to Squeak only if it asks for them. The UI could then enable fast overlay rendering while the video is unobscured, and switch to slow bitblt rendering when needed. It might be tricky to make the transition be seamless, but it is the only way to have our cake and eat it too.

Well, unless we switch to a layered rendering model where the VM does the compositing. Now that would be neat :) Though we'd still need access to the composited frame buffer for things like color picking and Etoys color tests.

- Bert -

K K Subbu

Re: Quicktime player

On Thursday 09 Sep 2010 2:50:43 pm Bert Freudenberg wrote:

> The best way would be to give the bits to Squeak only if it asks for them.
> The UI could then enable fast overlay rendering while the video is
> unobscured, and switch to slow bitblt rendering when needed. It might be
> tricky to make the transition be seamless, but it is the only way to have
> our cake and eat it too.
>
> Well, unless we switch to a layered rendering model where the VM does the
> compositing. Now that would be neat :) Though we'd still need access to
> the composited frame buffer for things like color picking and Etoys color
> tests.

Why not have both modes and switch between them? Overlay is sufficient for ops
like moving/resizing/rotating etc which involve only frames. You could always
do a mode switch (say pause/step) to go into bitblt mode for picking or
checking colors. Or have an explicit frame grab to obtain a form for these
ops?

Subbu