Context status 2015-01-16

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Context status 2015-01-16

ccrraaiigg

Hoi all--

     Context[1] is the umbrella project for Naiad (a distributed module
system for all Smalltalks[2]), Spoon (a minimal object memory that
provides the starting point for Naiad), and Lightning (a
remote-messaging framework which performs live serialization, used by
Naiad for moving methods and other objects between systems). I intend
for it to be a future release of Squeak, and a launcher and module
system for all the other Smalltalks. I'm writing Context apps for cloud
computing, web services, and distributed computation.

     Commits b7676ba2cc and later of the Context git repo[3] have:

-    Support for installable object memories as git submodule repos.

-    Submodule repos for memories for each of the known Smalltalk
     dialects, with Naiad support pre-loaded. I'm currently working on
     the submodules for Squeak[4] and Pharo[5].

-    A web-browser-based console for launching and managing object
     memories.

-    A WebDAV-based virtual filesystem that enables Smalltalk to appear
     as a network-attached storage device, and mappings of the system
     to that filesystem that make Smalltalk accessible from external
     text editors (e.g., for editing code, managing processes and
     object memories).

-    Remote code and process browsers.

     Live discussion at [6]. Mailing list at [7]. The newsgroup is
gmane.comp.lang.smalltalk.squeak.context.


     Thanks for checking it out!

-C

[1] http://thiscontext.com
[2] http://thiscontext.com/a-detailed-naiad-description
[3] https://github.com/ccrraaiigg/context
[4] https://github.com/ccrraaiigg/3EAD9A45-F65F-445F-89C1-4CA0A9D5C2F8
[5] https://github.com/ccrraaiigg/CFE10A14-D883-4ACE-990A-0DDA86AA362B
[6] http://squeak.slack.com
[7] mailto:[hidden email]

--
Craig Latta
netjam.org
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

Re: Context status 2015-01-16

Chris Muller-3
That sounds awesome Craig, congratulations.  If you ever do a video of
a reasonably-sized imprinting experiment onto your core-classes image
from a modern Squeak image, I will be very interested to see it!

On Fri, Jan 16, 2015 at 2:01 PM, Craig Latta <[hidden email]> wrote:

>
> Hoi all--
>
>      Context[1] is the umbrella project for Naiad (a distributed module
> system for all Smalltalks[2]), Spoon (a minimal object memory that
> provides the starting point for Naiad), and Lightning (a
> remote-messaging framework which performs live serialization, used by
> Naiad for moving methods and other objects between systems). I intend
> for it to be a future release of Squeak, and a launcher and module
> system for all the other Smalltalks. I'm writing Context apps for cloud
> computing, web services, and distributed computation.
>
>      Commits b7676ba2cc and later of the Context git repo[3] have:
>
> -    Support for installable object memories as git submodule repos.
>
> -    Submodule repos for memories for each of the known Smalltalk
>      dialects, with Naiad support pre-loaded. I'm currently working on
>      the submodules for Squeak[4] and Pharo[5].
>
> -    A web-browser-based console for launching and managing object
>      memories.
>
> -    A WebDAV-based virtual filesystem that enables Smalltalk to appear
>      as a network-attached storage device, and mappings of the system
>      to that filesystem that make Smalltalk accessible from external
>      text editors (e.g., for editing code, managing processes and
>      object memories).
>
> -    Remote code and process browsers.
>
>      Live discussion at [6]. Mailing list at [7]. The newsgroup is
> gmane.comp.lang.smalltalk.squeak.context.
>
>
>      Thanks for checking it out!
>
> -C
>
> [1] http://thiscontext.com
> [2] http://thiscontext.com/a-detailed-naiad-description
> [3] https://github.com/ccrraaiigg/context
> [4] https://github.com/ccrraaiigg/3EAD9A45-F65F-445F-89C1-4CA0A9D5C2F8
> [5] https://github.com/ccrraaiigg/CFE10A14-D883-4ACE-990A-0DDA86AA362B
> [6] http://squeak.slack.com
> [7] mailto:[hidden email]
>
> --
> Craig Latta
> netjam.org
> +31 6 2757 7177 (SMS ok)
> + 1 415 287 3547 (no SMS)
>
>

Reply | Threaded
Open this post in threaded view
|

re: Context status 2015-01-16

ccrraaiigg

> If you ever do a video of a reasonably-sized imprinting experiment
> onto your core-classes image from a modern Squeak image, I will be
> very interested to see it!

     Sure, right now you can browse the kernel memory from Squeak 4.5.
Every time you accept a method, you're compiling it in Squeak 4.5 and
imprinting it onto the kernel. But you're referring to imprinting driven
by method execution? (Each method that is run is imprinted elsewhere, as
a side-effect.) What code would you like to see imprinted?

     In any case, I do plan to make videos showing all the top-level
features.


     thanks,

-C

--
Craig Latta
netjam.org
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

Re: Context status 2015-01-16

David T. Lewis
In reply to this post by ccrraaiigg
On Fri, Jan 16, 2015 at 09:01:48PM +0100, Craig Latta wrote:

>
> Hoi all--
>
>      Context[1] is the umbrella project for Naiad (a distributed module
> system for all Smalltalks[2]), Spoon (a minimal object memory that
> provides the starting point for Naiad), and Lightning (a
> remote-messaging framework which performs live serialization, used by
> Naiad for moving methods and other objects between systems). I intend
> for it to be a future release of Squeak, and a launcher and module
> system for all the other Smalltalks. I'm writing Context apps for cloud
> computing, web services, and distributed computation.

Hi Craig,

I'm not fully up to speed on a lot of this, but I have to ask - Is it
possible to get a Spoon memory with Context running under SqueakJS? I
mean not just in theory, but can this actually be done in practice, such
that it can be demonstrated on the little-known but otherwise amazing
run.squeak.org?

I'm not sure that I have fully wrapped my head around the possibilities
here, but the idea of incrementally growing an image from a tiny Spur
memory using SqueakJS in a modern web browser seems like it might have
some potential.

If it is possible, then what needs to be done to make it happen? More
funding? More beer?

Dave
 

Reply | Threaded
Open this post in threaded view
|

re: Context status 2015-01-16

ccrraaiigg

Hi Dave--

> Is it possible to get a Spoon memory with Context running under
> SqueakJS? I mean not just in theory, but can this actually be done in
> practice, such that it can be demonstrated on the little-known but
> otherwise amazing run.squeak.org?

     Yes indeed. I'll do it next after Pharo.

> I'm not sure that I have fully wrapped my head around the
> possibilities here, but the idea of incrementally growing an image
> from a tiny Spur memory using SqueakJS in a modern web browser seems
> like it might have some potential.

     Definitely! There are all sorts of interesting web apps you could
write with that.

> ...what needs to be done to make it happen? More funding? More beer?

     More funding is always good! That would increase the time I can
spend on it, and make it happen sooner. Is there funding available?


-C

--
Craig Latta
netjam.org
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

re: Context status 2015-01-16

David T. Lewis
On Sat, Jan 17, 2015 at 11:16:36AM +0100, Craig Latta wrote:
>
> Hi Dave--
>
> > Is it possible to get a Spoon memory with Context running under
> > SqueakJS? I mean not just in theory, but can this actually be done in
> > practice, such that it can be demonstrated on the little-known but
> > otherwise amazing run.squeak.org?
>
>      Yes indeed. I'll do it next after Pharo.

Cool! I am looking forward to this.

>
> > I'm not sure that I have fully wrapped my head around the
> > possibilities here, but the idea of incrementally growing an image
> > from a tiny Spur memory using SqueakJS in a modern web browser seems
> > like it might have some potential.
>
>      Definitely! There are all sorts of interesting web apps you could
> write with that.
>
> > ...what needs to be done to make it happen? More funding? More beer?
>
>      More funding is always good! That would increase the time I can
> spend on it, and make it happen sooner. Is there funding available?
>

I am no so sure about funding, but I will be happy to buy you a beer :-)

Dave


Reply | Threaded
Open this post in threaded view
|

re: Context status 2015-01-16

Chris Muller-3
In reply to this post by ccrraaiigg
     Sure, right now you can browse the kernel memory from Squeak 4.5.
Every time you accept a method, you're compiling it in Squeak 4.5 and
imprinting it onto the kernel. But you're referring to imprinting driven
by method execution?

Yes exactly!
  
(Each method that is run is imprinted elsewhere, as
a side-effect.) What code would you like to see imprinted?

Test suites.  To me, the holy-grail of deployment scaling is I first I develop and configure in a big luxurious Cadillac image loaded with tools until I'm ready to deploy.  When I'm ready to deplooy I fire up a copy of your 1MB core image as the "target" with my luxury image as the 'source' and then I simply bring over a top-level test-suite method into the target and run it.

As the methods are executed and found to be missing in the 1MB target, they are brought over from the source.

By the time the tests are done, /all/ and /only/ the methods which were needed to run the tests were brought over by Spoon / Naiad.

Now I want to save that target image (maybe 5MB now) and deploy it.  I will want to run many multiple copies of of that 5MB image but since my tests probably don't have 100% coverage it would be too risky to run in production unless each of those 5MB images could have backup Spoon access to a single running copy of my mother Caddi' image, in case one more method is found to be needed...

So I see Spoon as a way for learning about systems like you, but also as a solution for scaling -- from the smallest single core for limited hardware <---> to the largest multi-core, applications because one can run MORE images concurrently due to their smaller size.



     In any case, I do plan to make videos showing all the top-level
features.


     thanks,

-C

--
Craig Latta
netjam.org
<a href="tel:%2B31%20%20%206%202757%207177" value="+31627577177">+31 6 2757 7177 (SMS ok)
<a href="tel:%2B%201%20415%20%20287%203547" value="+14152873547">+ 1 415 287 3547 (no SMS)





Reply | Threaded
Open this post in threaded view
|

re: Context status 2015-01-16

ccrraaiigg

     Okay, I'll add both of the execution-driven imprinters to the
repository. They're called "active" and "passive" imprinting.

     Active imprinting is directed by the system that initially has the
desired code. An ActiveImprintingServer has clients in the systems which
will receive the code. Every time the server system runs a method in a
certain process, it imprints that method onto each of the clients. One
use case for this is giving the code of a demo to an audience as you run it.

     Passive imprinting is directed by the system that wants the code.
The target system makes a remote-messaging connection to a system which
has the code, and runs an expression which will use the code. Every time
a method is missing from the target system (in any process), the target
system requests the missing method from the provider system, installs
it, and retries running that method.

     I have imprinted the exception-handling system, the compiler, and
the class builder with both approaches.


     thanks,

-C

--
Craig Latta
netjam.org
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

re: Context status 2015-01-16

Nicolas Cellier


2015-01-19 15:41 GMT+01:00 Craig Latta <[hidden email]>:

     Okay, I'll add both of the execution-driven imprinters to the
repository. They're called "active" and "passive" imprinting.

     Active imprinting is directed by the system that initially has the
desired code. An ActiveImprintingServer has clients in the systems which
will receive the code. Every time the server system runs a method in a
certain process, it imprints that method onto each of the clients. One
use case for this is giving the code of a demo to an audience as you run it.

     Passive imprinting is directed by the system that wants the code.
The target system makes a remote-messaging connection to a system which
has the code, and runs an expression which will use the code. Every time
a method is missing from the target system (in any process), the target
system requests the missing method from the provider system, installs
it, and retries running that method.

     I have imprinted the exception-handling system, the compiler, and
the class builder with both approaches.



In this scheme, something is striking me.
Some images share some code (classes, compiledMethods)
But what about code mutations/updates?

Without such mutations, is it still Smalltalk?

Without active imprinting, such mutations might not be obvious to propagate (for example a subclass now overrides a message of super)

And since you import class builder and compiler in the target, on what purpose? Is the target going to change a class locally? What if it then imports incompatible methods from provider? Or is the goal to just replicate some mutations from the provider?

Maybe the scheme is more interesting for deployment of static code, but I'm curious to know if ever live updates would still be possible...

 
     thanks,

-C

--
Craig Latta
netjam.org
<a href="tel:%2B31%20%20%206%202757%207177" value="+31627577177">+31 6 2757 7177 (SMS ok)
<a href="tel:%2B%201%20415%20%20287%203547" value="+14152873547">+ 1 415 287 3547 (no SMS)





Reply | Threaded
Open this post in threaded view
|

re: execution-driven imprinting (was "Context status 2015-01-16")

ccrraaiigg

Hi Nicolas--

> In this scheme, something is striking me. Some images share some code
> (classes, compiledMethods). But what about code mutations/updates?

     That's doable. A target system can record what it has received, and
from where. Through a continuous or periodic remote-messaging connection
to a providing system (not necessarily the original one), or by
subscribing to update broadcasts from a repository that a providing
system updates, the target system can stay up to date.

> Without such mutations, is it still Smalltalk?

     I think it would be, but we don't have to give them up so it's only
a philosophical point.

> With active imprinting, such mutations might not be obvious to
> propagate (for example a subclass now overrides a message of super)

     I think a target system could arrange to receive notifications
about absolutely every change which might effect the code it has, and
discard the ones it deems irrelevant.

> ...you import class builder and compiler in the target; for what
> purpose? Is the target going to change a class locally?

     No, those were just examples of rather complex and interconnected
frameworks which are tricky to transfer via source code recompilation.
They made good demonstrations of the execution-driven imprinting
technique. One big feature is that the person doing the transfer doesn't
need to understand anything about the code being transferred, and the
technique works without modification no matter what the code is.

> What if it then imports incompatible methods from the provider?

     The imprinting mechanism doesn't transfer incompatible methods. It
supports a negotiation between the systems involved, though, to make the
targets compatible before transferring a method, if the person operating
the system approves.

> Maybe the scheme is more interesting for deployment of static code,
> but I'm curious to know if live updates would ever still be
> possible...

     Yes indeed! I wouldn't be as interested in it if they weren't. This
is meant to bring a fully-functional livecoding system, like we're all
used to having, to distributed situations.


     thanks,

-C

--
Craig Latta
netjam.org
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Reply | Threaded
Open this post in threaded view
|

Re: [Pharo-dev] [squeak-dev] re: Context status 2015-01-16

Chris Muller-3
In reply to this post by Nicolas Cellier
On Tue, Jan 20, 2015 at 7:39 PM, Nicolas Cellier
<[hidden email]> wrote:

>
>
> 2015-01-19 15:41 GMT+01:00 Craig Latta <[hidden email]>:
>>
>>
>>      Okay, I'll add both of the execution-driven imprinters to the
>> repository. They're called "active" and "passive" imprinting.
>>
>>      Active imprinting is directed by the system that initially has the
>> desired code. An ActiveImprintingServer has clients in the systems which
>> will receive the code. Every time the server system runs a method in a
>> certain process, it imprints that method onto each of the clients. One
>> use case for this is giving the code of a demo to an audience as you run
>> it.
>>
>>      Passive imprinting is directed by the system that wants the code.
>> The target system makes a remote-messaging connection to a system which
>> has the code, and runs an expression which will use the code. Every time
>> a method is missing from the target system (in any process), the target
>> system requests the missing method from the provider system, installs
>> it, and retries running that method.
>>
>>      I have imprinted the exception-handling system, the compiler, and
>> the class builder with both approaches.
>>
>>
>
> In this scheme, something is striking me.
> Some images share some code (classes, compiledMethods)
> But what about code mutations/updates?
>
> Without such mutations, is it still Smalltalk?
>
> Without active imprinting, such mutations might not be obvious to propagate
> (for example a subclass now overrides a message of super)
>
> And since you import class builder and compiler in the target, on what
> purpose? Is the target going to change a class locally? What if it then
> imports incompatible methods from provider? Or is the goal to just replicate
> some mutations from the provider?
>
> Maybe the scheme is more interesting for deployment of static code, but I'm
> curious to know if ever live updates would still be possible...

The particular scenario I was dreaming of would never have changes
made to the imprinted images.  At worst, the source Caddilac image
might be have a patch applied in production, in which case it would
need some way to invalidate the updated methods in all clients...