[General] [Ann] Lively kernel 0.8 is out!

Dan, as you know, I'm eagerly looking forward to seeing a system that
brings together the strengths of our respective projects. We would
both like to see lightweight, interactively created active content
become as prevalent and easily shared as html text is today. As
section 2 <http://google-caja-discuss.googlegroups.com/web/caja-spec.pdf>
argues, I believe security problems have been the fatal flaw which
killed most previous attempts@active content. Caja gives us a basis
for addressing these security problems. Lively gives authors a medium
for creating active content worth sharing.

However, your recent message has me concerned that Lively is deviating
ever farther from programming patterns that can be secured. Before
making these changes, what thought was given to their possible
compatibility with object-capability security?


On Feb 1, 2008 12:59 PM, Dan Ingalls <[hidden email]> wrote:
> 1.  The entire Morphic architecture has been converted to wrap host
> SVG objects instead of extending them.  This was done in such a way
> as to change almost nothing@the level of most applications.  The
> major benefit afforded by this change is compatibility, since many
> JavaScript implementations do not support the ability to extend host
> objects.

Good. The Caja effort has made the same decision regarding DOM and
other conventional browser host objects.


> 2.  We have adopted a class system derived from prototype.js ver 1.60
> and extended with built-in support for serialization and copy
> constructors.  These conventions make it much more natural to use
> classes and class inheritance in JavaScript (the .subclass() method
> and $super optional parameter).

The Caja group has been puzzling over the extent to which Caja can
support the programming patterns that Prototype encourages. Our
conclusions so far are that Class.addMethods can't be rescued, and
that Object.extend is problematic@best. OTOH, Figures 7 and 8 of
<http://google-caja-discuss.googlegroups.com/web/caja-spec.pdf> show
the recommended Cajita inheritance pattern which seems to have all the
virtues of Smalltalk's, including even a proper "super", while
remaining friendly to object-capability principles. See also Figures
14 and 15 for an inheritance pattern that's more familiar to
JavaScript programmers, still Smalltalk-like, and adequately friendly
to object-capability principles.


> 6.  We have the beginnings of a reflective model for execution state.
> If one sets Config.debugExtras = true, most methods are wrapped with
> a function that keeps track of the stack with method names *and* even
> the arguments.  This can be tested by, eg, typing
>         Function.showStack()
> in any textMorph, selecting it and evaluating it with alt-d.  This
> should result in the stack being printed to the console with method
> names and argument values.

Are you thereby providing ambient access to the ability to reflect on
one's caller? What security properties do you imagine might survive
the availability of such an operation?

As you heard me say@OOPSLA, Squeak-E failed because Smalltalk code
was not facing a hostile environment, so the Smalltalk community never
became willing to sacrifice anything else they value for the sake of
security. I have no argument with that. If that's the right tradeoff
for Lively, fine. But please make the choice with your eyes open.

--
Text by me above is hereby placed in the public domain

    Cheers,
    --MarkM

[Folks -
I know this is a bit long, but it articulates some seeds of this community,
and, for that reason, needs your criticism as well]


Hi, Mark -

Thanks for your candid response.  This is the beginning...

>Dan, as you know, I'm eagerly looking forward to seeing a system that
>brings together the strengths of our respective projects. We would
>both like to see lightweight, interactively created active content
>become as prevalent and easily shared as html text is today. As
>section 2 <http://google-caja-discuss.googlegroups.com/web/caja-spec.pdf>
>argues, I believe security problems have been the fatal flaw which
>killed most previous attempts@active content. Caja gives us a basis
>for addressing these security problems. Lively gives authors a medium
>for creating active content worth sharing.

I'm looking forward to this convergence too.  Don't give up...

>However, your recent message has me concerned that Lively is deviating
>ever farther from programming patterns that can be secured. Before
>making these changes, what thought was given to their possible
>compatibility with object-capability security?

Zero...

... and therefore we owe you (and the higher value of secure architecture that you represent) an apology for this slight.  Yet we do not feel@all apologetic for the following logic of how we are trying to bootstrap this project...

Our first goal for this project is survival, and it is not a slam dunk by any means.

We are following our intuition regarding a number of sub-goals where we are trying to maximize the impact (/utility/novelty/coolness...) divided by the effort required in each case.  Here are my favorites among those "sweet spots"...

1.  A real (and lively) development environment for JavaScript

2.  A cool demo of "simple things are simple"

3.  A educational playground like EToys for JavaScript

4.  A useful PIM like Hypercard, and able to handle content like our engine demo

5.  Some way to encapsulate arbitrary (not just LK) web content in LK so it becomes
        a mega-malleable mashup framework

6.  A compelling demo of a collaborative web page

7.  A really easy to use application builder along the lines of Fabrik, but with a parts-list
        spreadsheet alternative to wires

My feeling is that if any of these really works, then it/they can carry the project for a while as it matures and gathers interest.


Of course one of these could be,

N.  Demonstrate a Caja-based version with true modularity and security properties,

and another could be,

M.  Rewrite the whole Morphic framework along the lines the Viewpoints folks are exploring with Lessphic(tm) and their simplified outline-based graphic kernel.

The only reason these are not on our list, is that, when it comes down to priorities, we'll first try to get bootstrapped in a manner we understand and can pull off ourselves


Now let me tell the story of the first four Lively Kernels

#1:  Canvas model with  random JavaScript, transcribed from various other (Java) apps

#2:  Morphic architecture on top of Java2D graphics engine

#3:  Morphic on SVG extended by JavaScript

#4:  Morphic on SVG with wrapper API

Wow, there have been four generations already?  That's right, and here is how it has unfolded:  In every case we did an experiment that was a little proof of concept.  We started with #1 around October of 2006.  It was just a year ago that I wrote the first Morphic kernel that gave us #2.  It worked, and so we continued and made it real.  Then in May of last year Krzysztof wrote just enough to demonstrate the Morphic in SVG, and we realized we could run LK with no installation.  There too, we followed the experiment with a full rewrite and moved forward to full SVG.  Then last October we faced the non-portability of our SVG interface, and Krzysztof did another experiment to demonstrate the Morphic kernel running with wrappers (and thus able to run in, eg, Opera or Batik/Squiggle).  Here again, we moved forward with the result knowing that Opera would eventually work, and Batik would allow us to make a plugin for, notably, Internet Explorer.

So, every experiment worked?  No, not really.  There was another one in there between #1 and #2 where we tried integrating everything with Dojo.  Here we did the kernel experiment in a week as usual, but we discovered that Dojo was not lightweight, and was (at least@the time) full of circumlocutions to deal with all the old HTML models and the incompatibilities between browsers, and this simply scared us.


The point is...
LK is relatively small, and you can experiment with new versions in a week or so.  We could help by restructuring a few files to make this inner kernel more apparent.  Basically what is needed is a single world along the lines of the "Simple example morphs" world, and just the classes needed to support it.

And to return@last to your concern, Mark, we should then be able more easily to scrutinize that smaller corpus and actually rewrite it a couple of times as an experiment in Caja.  And this is just the approach we anticipate in the case of the Viewpoints rewrite of Morphic.  We want all these things to happen, we just need to follow our "survival agenda" for a while, and hopefully more folks will join up and help to carry out those other experiments.

        - Dan

Hi, Mark -

>However, your recent message has me concerned that Lively is deviating
>ever farther from programming patterns that can be secured. Before
>making these changes, what thought was given to their possible
>compatibility with object-capability security?

I realize that my previous somewhat flip answer of "Zero" might have you a bit
concerned.  Also, I let my response go in other directions that may have
obscured our very real interest in Caja and serious security for JavaScript.

So far we have made our best progress by small experiments followed by
significant rewrites.  Things are still small enough (and we hope to keep
it that way) that global rewrites are still tractable.

I think the next step would be to have a meeting with you folks in which
(in fact probably before which) we scrutinize the Lively Kernel code and
sketch a few rewrites that would put us on the high road as soon as possible.

While some aspects of 0.8 may bother you, it captures the functionality
we're after pretty well, so we can probably carry out some of these
experiments without too much interference with other ongoing changes.

While I'd like to get together off line, I think this is a project of great
potential importance, so we should try to document as much as possible
as we go, and include our respective mail list communities in the process.

I think that it would be enormously instructive and validating
to show that a relatively complete and self-supporting system can
be rewritten as a secure architecture without making things more
complex and, hopefully even improving its clarity.

        - Dan, for the LK team

Hi,

I think that there may be some confusion as to the intent of LK...  
perhaps mine. Despite Dan's hallowed past, LK does not intend to  
create a new runtime/VM. LK's stated intent is to program the browser  
using the browser's environment sans plug-ins (if possible). "Zero" as  
in zero install. So that's the browser's JavaScript with a few 3rd  
party libraries such as Prototype.

Please correct me if I'm wrong but the most successful implementation  
of Caja is on Google's servers to provide a sandboxed runtime for  
widget/gadget execution. The Apache implementation seems to follow  
this server-side pre-processing approach.

Comparisons to the Squeak's runtime are not relevant since LK does not  
propose to create a runtime.

LK has every chance to fundamentally change browser programming. With  
Sun almost betting Java's future on JavaFX, I think that LK just might  
be the technology to establish Sun as the preeminent browser  
programming vendor and finally compete with Flash on the desktop. And  
once the Opera desktop engine is fully ported to its mobile browser  
next year that will mean that LK becomes the de-facto standard for all  
browser programming.

So unlike in the Smalltalk and Java worlds where you build almost  
everything from scratch, LK has to rely on other technologies for its  
success since@it's core LK is just a JavaScript library with an SVG  
UI toolkit (and some cool networking stuff in the future).


I'd appreciate any corrections to my understanding of LK and Caja  
since as a mash-up vendor we are exploring Caja's potential and are  
already developing for LK.


thanks for making this an open forum.


--Zaid

ALT Mobile

http://altmobile.com/Home.html (web site)
http://web.mac.com/altmobile (official blog)



On Feb 1, 2008,@5:38 PM, Mark Miller wrote:

On Feb 6, 2008 5:44 PM, ALT Mobile DEV <[hidden email]> wrote:
> I think that there may be some confusion as to the intent of LK...
> perhaps mine. Despite Dan's hallowed past, LK does not intend to
> create a new runtime/VM. LK's stated intent is to program the browser
> using the browser's environment sans plug-ins (if possible). "Zero" as
> in zero install. So that's the browser's JavaScript with a few 3rd
> party libraries such as Prototype.

Hi Zaid,

I believe we have the same understanding, but there may be a minor
terminology issue. Certainly, LK seeks zero install, and using the
browser environment as is sans plugins, just as you say. Caja also
seeks these attributes.

However, it may be more slippery than it appears whether either LK,
Caja, or indeed any comprehensive JS library constitutes a new
runtime/VM. All create a new programmable level of abstraction on top
that provided by the browser. To the extent that this new level of
abstraction seems self contained, so that one can use it without a
detailed understanding of how it is mapped onto the browser's
platform, it does, in effect, a new platform/runtime/VM. But whether
or not LK or Caja should be described in this way, both LK and Caja
would benefit if LK were written in the Caja-compliant subset of
JavaScript.


> Please correct me if I'm wrong but the most successful implementation
> of Caja is on Google's servers to provide a sandboxed runtime for
> widget/gadget execution. The Apache implementation seems to follow
> this server-side pre-processing approach.

You are completely correct today, and you are mostly correct for the
foreseeable future: The full Caja translator is written in Java, not
JavaScript, and is too large to consider downloading to the browser.
As a result, Caja itself cannot support the LK style of self-contained
live programming environment within the browser *for Caja code*.

To address these very kinds of concerns, we have defined a small
subset of Caja called Cajita. Cajita is no less useful, expressive,
convenient, or secure a programming language than Caja. However, it
leaves out so much of JavaScript that one would only recommend it for
new code. Caja is a much larger subset of JavaScript in order to be
friendlier to old JavaScript code. Cajita is sufficiently small that
we expect to port the Cajita->JavaScript portion of the translator
into Cajita, in which case we can support live programming (and eval)
of Cajita code safely, purely within the browser.


> So unlike in the Smalltalk and Java worlds where you build almost
> everything from scratch, LK has to rely on other technologies for its
> success since@it's core LK is just a JavaScript library with an SVG
> UI toolkit (and some cool networking stuff in the future).

Of these other technologies, how much of their code could be within
the scope of LK's browser system, and subject to live reprogramming?
Within the constraints of the existing browser platform, I think the
answer must be essentially none. The code that appears in LK's
browsers will almost all be new code anyway. Were the code in LK's
browsers in Cajita, and were we to realize our fantasy of a
browser-side Cajita translator, then the LK browsers could eval it
live, as LK currently does for JS code. Since Cajita (like Caja) is a
subset of JavaScript, this can be explored today, without waiting for
Cajita to be ready.


> I'd appreciate any corrections to my understanding of LK and Caja
> since as a mash-up vendor we are exploring Caja's potential and are
> already developing for LK.

Glad to hear it; both are wonderful news!


> thanks for making this an open forum.

You are quite welcome. Openness is always my first choice.

--
Text by me above is hereby placed in the public domain

    Cheers,
    --MarkM

Hi Mark,

thank you for your insight. It's always a pleasure to learn  
alternative view points. My comments are embedded:


--Zaid

ALT Mobile

http://altmobile.com/Home.html (web site)
http://web.mac.com/altmobile (official blog)



On Feb 7, 2008,@2:13 AM, Mark Miller wrote:

[ZAID] I apologize... I disagree with this premise because LK is by  
definition not a sealed environment where apps are "certified" like in  
the mobile world. LK requires Prototype and even if it didn't, a  
developer can use all of JavaScript@anytime to manipulate most of  
LK. Unless LK implemented some type of defensive programming in each  
of its APIs-- and somehow was able to stop direct manipulation of the  
SVG and JavaScript objects--  isn't this just making LK into a  
security R&D prototype with no possible commercial viability?

I honestly believe that a more pressing security and feature issue for  
the LK team is centered around networking and cross-domain support. I  
just announced on Tuesday our Dynamic Mashup Server as part of our  
support for the Comet (HTTP Streaming/Push) implementation done by Sun  
in Grizzly. What I didn't mention in the press release is that I built/
tested the functionality against LK clients. What that means is that  
any LK client can be updated/synchronized from a Dynamic Mashup Server  
en-mass. So with the new LK serialization capability we can have  
scalable shared sub-worlds across the internet.


But as you know, the paramount issue in this is the same origin server  
issue and trusting the server. Cross-domain support has been solved on  
the client in Flash, Opera Widgets, and will be addressed in FireFox 3  
(and HTML5).  It's good that Stephen Uhler  is associated with the LK  
team since he did an interesting proxy implementation in Brazil.


So what will LK do? go the Flash or Java Applet route? create a web  
server plug-in? use Apache's solution? implement that proxy back door  
server that was in the LK .7 code? Or just skip shared worlds because  
they introduce a whole range of network security issues far more than  
just browser programming.

If the LK team doesn't get this write, then only trivial network apps  
will be written for LK because confidential or personal information  
(aka social apps) will not be shared because they do not trust the LK  
server. If the LK team implements dynamic, state-full inspection and  
verification during sub-world sharing using Caja-like technology, then  
that's where LK becomes more than just a single user or small-group  
JavaScript and SVG programming environment.


So, I would like you to consider that maybe getting the network  
security model done correctly is more important than ensuring that LK  
itself and personal morphlets are secure. So my vote is to secure on  
sharing sub-worlds to deny XSS and script injection, etc. but not the  
execution within the browser.


[ZAID] I think we agree if by "support live programming (and eval)"  
only refers to remote sub-worlds/morphlets and whatever is the unit of  
transmission/sharing.


[ZAID] sure, but to what aim? A limited availability security  
prototype? LK can accomplish far more and can solve the security issue  
on sharing sub-worlds/morphlets because when LK goes P2P (peer-to-
peer) via an LK server you're really introducing unknown live code.  
The normal LK code is coming from a server to which you contacted  
since that's the normal web browser-to-server client-server stuff.

P2P LK is the only new aspect that needs to be secured IMO. But the LK  
team will decide according to its SunLabs mandate.


I would also like you to consider implementing Caja in Single-Site  
Browsers (SSB) such as Mozilla Prism and Todd Ditchendorf's WebKit  
implementation of the SSB concept. First, they make wickedly fast LK  
clients. We call them the ultimate mash-up client because they are  
built to support browsing a single site or mash-up and can't interact  
with other sites. So in this context, Caja moves the browser into a  
really secure platform. . Fast and secure.


[ZAID] thanks again for the opportunity to correspond with you and I  
think that my opposition is to when and where we enforce security:  
constantly in the browser or on the server only when sharing among  
strangers.

> --
> Text by me above is hereby placed in the public domain
>
>    Cheers,
>    --MarkM