Cobalt Firefox Browser Plugin

Thanks Joshua.

Sorry if I confused anyone with my earlier post. I should have thought about it a bit more before posting.

Steve

On Sat, Jan 24, 2009 at 2:30 PM, Joshua Gargus <[hidden email]> wrote:

On Jan 24, 2009, at 1:11 PM, Rich White wrote:

I see your points Joshua ...

Just to be clear, I'm not arguing against a browser plugin in general (even if it wouldn't be the first thing that I would personally focus on), but instead to avoid "mission creep" where the goal of "browser plugin" becomes "secure browser plugin".  I think that the latter is overkill for now, and that we should leverage the working code that we already have.

(my opinion might change if, for example, Google were to announce that cross-platform hardware-accelerated OpenGL is a priority for NaCl... in this case, the easiest way to achieve the goal of "browser plugin" would also give us "secure browser plugin")

I curiously wonder if Cobalt-Croquet in
the browser would attract web developers to the projects (wishful
thinking?)  looking to extend their projects & adding 3D VW
capabilities (Moodle for example).

"Extend their projects" is the crux of the issue.  I think that successfully recruiting web developers would require (Javascript?) APIs to allow them to integrate the 3D spaces with the enclosing page.  If you have a Croquet browser plugin without that type of API, then you still have something useful, but you'll probably only get a few more of the same type of developers who would experiment with Croquet anyway.  But what do I know, I'm not a web developer :-)

Cheers,
Josh

=

On Sat, Jan 24, 2009 at 1:51 PM, Joshua Gargus <[hidden email]> wrote:

A few points on NaCl...

NaCl does not appear to provide an OpenGL solution.  The Quake demo uses a
software renderer.  It may very well be that Google (or someone) will create
a trusted, cross-platform NaCl interface to OpenGL, but (AFAICT) nobody has
demonstrated or announced one.

The security model appears to be fundamentally at odds with JIT compilers,
so Croquet-in-NaCl wouldn't be able to take advantage of Eliot Miranda's
forthcoming Cog VM.  For a given use-case, the lower performance may be
justified by the benefits of running in a browser (i.e. YMMV)

Finally, I question the notion that "NaCl can handle sandboxing and
cross-platform issues for us".  NaCl does not look like the shortest/easiest
way to get to a cross-platform browser plugin (it may be the shortest path
to a "secure" plugin, which wasn't the initial goal... if we're changing the
goal we should be explicit about why we're doing it).  NaCl doesn't allow
direct access to the files or the network, both of which Croquet currently
depends on.  It is probably possible to address file-storage via Google
Gears and network access by using the browser's HTTP connections, but this
would require significant work.

A secure NaCl browser plugin might become necessary some day, if there is
evidence that there are significant numbers of potential users who would
otherwise use Croquet, except that the security characteristics are
unacceptable.  However, we're not yet at the point where Croquet is broadly
usefu.  I argue that the community's energy would be better spent on making
Croquet more useful.  Examples include a robust Cobalt client, high-quality
VOIP, and a easily-installable router/server that tech-savvy non-hackers can
run (the last is my personal bias, there are probably good reasons for
focusing on heavier technologies like LDAP and Shibboleth).

Cheers,
Josh



On Jan 24, 2009, at 10:26 AM, Steve Wart wrote:

Does it make sense to continue using a custom plugin if NaCl can handle
sandboxing and cross-platform issues for us?

Steve

On 24 Jan 2009, at 10:02, "David P. Reed" <[hidden email]> wrote:

OK,there is a reasonable approach to make this work.  If you look at the
new opensource OpenGL solution for Linux in VMs called VMGL (q.v.. i.e. with
Google), you will find it is made up of two parts.

Part 1 runs in the client VM as its GL library, and is basically an
implementation of a cross-network GL rendering protocol called WireGL.
WireGL is capable of rendering/compressing OpenGL 1.5 commands across a
wire connection where the connection for a typical video game needs about 1
Gb/sec of capacity to achieve full performance.  (Pat Hanrahan's group did
the WireGL concept as I recall).

Part 2 is a stub that takes WireGL in and puts out (optimized) OpenGL
calls to an actual host processor window.

If you put part 1 in the "background" process, and part 2 in the browser,
you could make this approach work for Squeak.

Please note that part 1 is only fully implemented as a library under the
Linux kernel, afaik.  And the communications protocol is any socket stream
(in the VM case, it is done by a virtio (q.v.) paravirtualized network
driver).  But any streaming path would work (e.g. shared memory or pipes).




John M McIntosh wrote:

Ok, let me comment.

(a) the plugin uses the 1990's netscape browser api.
(b) A fair amount of the code actually comes from Bert's work on the
unix plugin (thank you)

The issue is we are given a handle to the drawing surface which exists
in the browser memory space.

Years ago we would launch the squeak vm as a thread (using threads or
co-operative multi-processing) and live within the browser memory space,
over time
this proved to be a hassle since the VM wasn't written to support
multiple images & display surfaces, and bad behaviour on part of the plugin
would kill the browser.

I changed this a few years back when Viewpoints provided some funding to
update the plugin, this work moved towards the unix model of process forking
off
the squeak VM using execv() and having the plugin pass UI events to the
squeak process via a pipe, along with the squeak process passing commands
back to the browser plugin via another pipe.

Now the hard part happens. For windows and unix you have this handle to
the drawing area and the squeak process can just use that to perform any
drawing needed.
However on the macintosh using the quartz based drawing model this isn't
possible since the surface exists in memory only accessible to the browser.

To solve this I used shared memory to hold the drawing surface.
Fortunately (hah) there are three or four ways to do this, and the BSD and
Linux folks have
different opinions, plus the logic is spread across 4 or 6 man pages and
code examples? Unlikely.

MultiCPU machines make this more difficult since the drawing can outrun
the ability for the browser to do the actual draw so you need to use a
semaphore to control that. Oh and either partner in the dance can go away so
you need to manage that situation.

So Squeak draws to the display surface, and sends an update screen
request via the pipe to the browser. The browser wakes up and copies the
bits from the shared display surface to the real drawing surface with the
complication that it's only allowed to do this at certain times when the
browser has setup the drawing surface for the plugin to draw into, yet more
synchronization problems.

Anyway the issue is any OpenGL drawning would occur in the Squeak
Process which is a separate process from the Browser process.
How would you affect what the user sees on the screen is an issue for
the developer to solve at this point.

I'd guess I'd google for "opengl drawing from background process in os
x"

Full Screen?

In order to do full screen, we cheat. We actually bring Squeak to the
foreground, open a full screen window and draw to it, versus sending
requests to the browser. UI events are of course captured by the Squeak
foreground window, in the background the browser still lurks, but is
passive.


On 23-Jan-09, at 7:18 AM, Steve Wart wrote:

Google's Native Client (NaCl) project looks like it might provide an
interesting way around this

http://code.google.com/p/nativeclient/

On Fri, Jan 23, 2009 at 1:04 AM, Bert Freudenberg
<[hidden email]> wrote:

On 23.01.2009, at 16:28, Andreas Raab wrote:

There is a known problem with the plugin on the Mac that it doesn't
support OpenGL. I don't remember what the reason for this was though ...


John did serious magic in the plugin, IIRC the VM is rendering to an
offscreen window and that is copied into the browser. That might very well
not work with OpenGL.

- Bert -

--

===========================================================================
John M. McIntosh <[hidden email]>
Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com

===========================================================================