Redline: Talking Runtime basics ...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Redline: Talking Runtime basics ...

James Ladd
Let's do that Stef
Jamesladd67 is my Skype

Thank you so much 

Sent from my Commodore 64

On 1 Jan 2017, at 1:21 am, Stephane Ducasse <[hidden email]> wrote:

James,

Nice to hear that you are still pushing. I would love to be able to developed in Pharo and deployed on red. 
Noury is working on the same scenario for PharoJS. 

May be we should organise a skype with Guille because we can use his infrastructure (hespel) to create out of Pharo automatically 
a kernel for Red. 

Guille has an infrastructure Hespel (check his phd) to 
- bootstrap different kernels such as Candlelight, microsqueak.. 
- tailor application (you give a seed expression and it collects all the mandatory methods and classes) probably like spoon

He bootstrapped different small kernels during his phd. 
In tornado the tailoring version of Hespel we can create imae that are as small as 12k :)

We put a list of the class in a techreport. https://hal.inria.fr/hal-00996908

In Pharo 70 we will use in production a bootstrap kernel: basically Pharo core with some better handled mechanisms (such as the startup)
We should remove the traits from class/metaclasses to gain some space. 
Now our goal for this version is not to have the smallest core but a full process based on a bootstrap image. 
We shuffled/repackaged many classes around and we will continue to cut spurious dependencies. 

Stef




On Sat, Dec 31, 2016 at 12:50 PM, James Ladd <[hidden email]> wrote:
I'll check it out.
The version I remember was black n white.


Sent from my Commodore 64

> On 31 Dec 2016, at 9:04 pm, Stefan Marr <[hidden email]> wrote:
>
> Hi James:
>
>> On 30 Dec 2016, at 22:50, James Ladd <[hidden email]> wrote:
>>
>> I even tried to find the old Potatoe Smalltalk which was a port of an old Smalltalk-80 image as it had a working ide and yet the smallest count of classes.
>
> Did you find it?
> https://news.squeak.org/2008/07/07/potato-version-of-jsqueak-from-hpi/
>
> Best regards
> Stefan
>
>
> --
> Stefan Marr
> Johannes Kepler Universität Linz
> http://stefan-marr.de/research/
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Redline: Talking Runtime basics ...

James Ladd
In reply to this post by Stephane Ducasse-3
Stef,

That is very useful.
I'd certainly like to see code written w Pharo running on Redline :)

- James 

Sent from my Commodore 64

On 1 Jan 2017, at 1:21 am, Stephane Ducasse <[hidden email]> wrote:

James,

Nice to hear that you are still pushing. I would love to be able to developed in Pharo and deployed on red. 
Noury is working on the same scenario for PharoJS. 

May be we should organise a skype with Guille because we can use his infrastructure (hespel) to create out of Pharo automatically 
a kernel for Red. 

Guille has an infrastructure Hespel (check his phd) to 
- bootstrap different kernels such as Candlelight, microsqueak.. 
- tailor application (you give a seed expression and it collects all the mandatory methods and classes) probably like spoon

He bootstrapped different small kernels during his phd. 
In tornado the tailoring version of Hespel we can create imae that are as small as 12k :)

We put a list of the class in a techreport. https://hal.inria.fr/hal-00996908

In Pharo 70 we will use in production a bootstrap kernel: basically Pharo core with some better handled mechanisms (such as the startup)
We should remove the traits from class/metaclasses to gain some space. 
Now our goal for this version is not to have the smallest core but a full process based on a bootstrap image. 
We shuffled/repackaged many classes around and we will continue to cut spurious dependencies. 

Stef




On Sat, Dec 31, 2016 at 12:50 PM, James Ladd <[hidden email]> wrote:
I'll check it out.
The version I remember was black n white.


Sent from my Commodore 64

> On 31 Dec 2016, at 9:04 pm, Stefan Marr <[hidden email]> wrote:
>
> Hi James:
>
>> On 30 Dec 2016, at 22:50, James Ladd <[hidden email]> wrote:
>>
>> I even tried to find the old Potatoe Smalltalk which was a port of an old Smalltalk-80 image as it had a working ide and yet the smallest count of classes.
>
> Did you find it?
> https://news.squeak.org/2008/07/07/potato-version-of-jsqueak-from-hpi/
>
> Best regards
> Stefan
>
>
> --
> Stefan Marr
> Johannes Kepler Universität Linz
> http://stefan-marr.de/research/
>
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Redline: Talking Runtime basics ...

Ben Coman
In reply to this post by James Ladd
On Sat, Dec 31, 2016 at 5:53 AM, James Ladd <[hidden email]> wrote:
> This is very helpful thank you.
>
> Maybe I can rephrase things to help capture what I'm trying to do.
>
> In the Smalltalk-80 blue book there is a set of Kernel-Object/Kernel-Classes defined and I'm trying to find a full running implementation of these classes in a Smalltalk environment OR recreate


Just my wandering mind.... an interesting experiment might be running
SqueakJS [1]
on top JVM8's Nashorn javascript engine [2][3], and then gradually
converting parts to it to native Java code as profiling dictates.

cheers -ben


[1] http://freudenbergs.de/bert/publications/Freudenberg-2014-SqueakJS.pdf
[2] https://ariya.io/2014/03/nashorn-the-new-rhino-on-the-block
[3] https://www.infoq.com/articles/nashorn

---------------
On Sat, Dec 31, 2016 at 5:14 PM, [hidden email]
<[hidden email]> wrote:
> Isn't there a typo in the first expression?

I've looked but don't see it.  btw the first '$' is the shell prompt,
the second '$' is a variable holding the vm location, since my
download of bootstrap.image wasn't located under pharolauncher.

But I do notice the second expression should have been...
$  $PHARO bootstrap.image eval "Object class allSubclasses size"

Reply | Threaded
Open this post in threaded view
|

Re: Redline: Talking Runtime basics ...

James Ladd
Ben,

That certainly is an approach.

One key goal for Redline is for people using Java to be able to use their debugging tools and step from Java code into Smalltalk, and if that Smalltalk code calls other Java code then step into that code.

Redline generates 100% compatible bytecode and class files (in memory) making this possible

Redline also supports the generation of inline bytecode so no primitives just the equivalent Jvm bytecode

- James

Sent from my Commodore 64

> On 2 Jan 2017, at 3:46 am, Ben Coman <[hidden email]> wrote:
>
>> On Sat, Dec 31, 2016 at 5:53 AM, James Ladd <[hidden email]> wrote:
>> This is very helpful thank you.
>>
>> Maybe I can rephrase things to help capture what I'm trying to do.
>>
>> In the Smalltalk-80 blue book there is a set of Kernel-Object/Kernel-Classes defined and I'm trying to find a full running implementation of these classes in a Smalltalk environment OR recreate
>
>
> Just my wandering mind.... an interesting experiment might be running
> SqueakJS [1]
> on top JVM8's Nashorn javascript engine [2][3], and then gradually
> converting parts to it to native Java code as profiling dictates.
>
> cheers -ben
>
>
> [1] http://freudenbergs.de/bert/publications/Freudenberg-2014-SqueakJS.pdf
> [2] https://ariya.io/2014/03/nashorn-the-new-rhino-on-the-block
> [3] https://www.infoq.com/articles/nashorn
>
> ---------------
> On Sat, Dec 31, 2016 at 5:14 PM, [hidden email]
> <[hidden email]> wrote:
>> Isn't there a typo in the first expression?
>
> I've looked but don't see it.  btw the first '$' is the shell prompt,
> the second '$' is a variable holding the vm location, since my
> download of bootstrap.image wasn't located under pharolauncher.
>
> But I do notice the second expression should have been...
> $  $PHARO bootstrap.image eval "Object class allSubclasses size"
>

Reply | Threaded
Open this post in threaded view
|

Re: Redline: Talking Runtime basics ...

Guillermo Polito
In reply to this post by philippe.back@highoctane.be
Hi Phil,

On Sat, Dec 31, 2016 at 10:14 AM, [hidden email] <[hidden email]> wrote:
Isn't there a typo in the first exptession?

Also the list contains a lot of RB things. What purpose do they have in a bootstrap image?

Yes, that is because the bootstrap image contains the compiler. And the compiler requires (amongst others) the AST, implemented initially as part of the refactoring browser, and so it has the RB prefix.

However, the bootstrap does not contain the refactoring browser.
 

Same for RelationSlot and X11 something.

RelationSlot belongs to the class builder package, which is needed to create classes.
Of course RelationSlot is part of an 'Examples', and as such it could be packaged separately, but this was not done yet.

Regarding the X11 thingy, this is probably X11Encoding that you're talking about. This one belongs to the Multilingual-encondings package, and if we are picky you'll see this package is required because it contains mainly Unicode, which is required by the Kernel so far to do a lot of stuff. See for example:

Character>>asLowercase
"If the receiver is uppercase, answer its matching lowercase Character."
^ self characterSet toLowercase: self

Character>>characterSet
^ EncodedCharSet charsetAt: self leadingChar

...

What lead to EncodedCharSet (and the default charset that is Unicode).


Is there a scope and purpose statement for the bootstrap somewhere?

Not writen that I know of. Maybe we should do it. But from many discussions here in the team I'd define the pharo bootstrap as a pharo environment that is 
 - minimal (in the sense that it keeps only stuff required to run)
 - clean (in the sense that there is the less dead code and dead dependencies as possible)
 - and is able to grow (in the sense that you can install new code on it). Think that Pharo is not just the bootstrap. People will not accept a Pharo without a development environment, without a debugger or a workspace.

These three points are a long term goal I woud say, and particularly the last point is in big conflict with the first one. You can have an image that executes code but has no compiler for example.
So making a trade-off but always going towards that goal, what we decided with Christophe one year and a half ago to:

  - select a minimal set of packages that would belong to the bootstrap image
       * To run, you need the Kernel.
       * To be able to install new code you need some kind of I/O and some way to transform the input as classes/methods. Thus we chose to put also basic command line handlers, files, and the compiler. We discussed about putting a flavour of Fuel (say Tanker) instead of the compiler, but Tanker was not stable as it was only experimental and nobody used it for real; the compiler was much more stable as we use it everyday; and besides, Tanker was depending on the class builder, so it only allowed us to remove the compiler but not the class builder.
       * To be clean, we started looking at the dependencies of those packages.
  
  - We finished in a selection of about 53 packages. Remember that Pharo itself has nowadays around 464 packages...
      RPackageOrganizer default packages size "464"
      So this was as good as a ~10% of the original image. Good enough to have a first version.

  - And we started working on dependency analyses on them. The following report made by Christophe lists all the bootstrap packages and their dependencies, and reports when a new dependency is introduced.


      See what the dependency graph looks of the bootstrap


      and extrapolate to the entire image :).
      And that, after we worked a LOT to cut off/separate/untangle/decouple those 53 packages from the rest, rewrite parts of the system (a big example of it is the SessionManager).


Now, this is of course not the end. There are still lots of things to do.

  - the kernel can be still cleaned up even more
  - we could implement some sort of binary serializer specialized for classes and methods to shrink even more the bootstrap
  - the rest of the image has still to be cleaned. Nowadays on top of the bootstrap Monticello and Metacello are bootstrapped using manual a approach and afterwards, the rest of the image is loaded by using a Baseline


    That baseline can be enhanced, splitted in smaller baselines. This require non-bootstrap packages to be revisited, have checked their dependencies and so on... 

So what you see in that list is the result of several decisions and a lot of work. And the focus was to release something and not die in the process. Hope that answers the question (I will copy past this and put it into some wiki btw)

Guille



Phil

Le 31 déc. 2016 05:47, "Ben Coman" <[hidden email]> a écrit :
So for curiosity...

$  $PHARO bootstrap.image eval "Object allSubclasses size"
1677

$  $PHARO bootstrap.image eval "Object allSubclasses size"
837

$  $PHARO bootstrap.image eval "Object class printHierarchy" >
/tmp/60334-bootstrap-hierarchy.txt
(see attached)

cheers -ben


On Sat, Dec 31, 2016 at 4:42 AM, Pavel Krivanek
<[hidden email]> wrote:
> It is better to use smaller bootstrapped image without Monticello but it is
> still quite big.
>
> https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-01-00-Bootstrap/lastSuccessfulBuild/artifact/bootstrap.zip
>
> -- Pavel
>
> 2016-12-30 18:13 GMT+01:00 Ben Coman <[hidden email]>:
>>
>> On Fri, Dec 30, 2016 at 7:51 AM, James Ladd <[hidden email]> wrote:
>> > Hi Pharo People,
>> >
>> > I have continued work on Redline Smalltalk and I'm wanting to discuss
>> > what
>> > the absolute minimum
>> > set of Classes and method should be included in the Redline Runtime.
>> >
>> > Would anyone here like to participate in that discussion?
>> >
>> > - James.
>> > Redline Smalltalk <http://redline.st>
>>
>> Nice to hear you are continuing.
>> I'm not very knowledgable on this, but I'll show you how to pull some
>> data from the work on producing a minimal image.
>>
>> 1. From PharoLauncher > Templates > Pharo 6.0(beta)
>>     download/create an image of build "60334-minimal".
>> 2. Right-click on the image and choose [Copy pathname]
>> 3. In a shell, change to that directory, and execute the following
>>     $ ../../VMs/spur/pharo 60334-minimal.image eval "Object allSubclasses
>> size"
>>         ==> 2801
>>     $ ../../VMs/spur/pharo 60334-minimal.image eval "Object class
>> allSubclasses size"
>>         ==> 1399.
>>     $ ../../VMs/spur/pharo 60334-minimal.image eval "Object class
>> printHierarchy" > /tmp/60334-minimal-class-hierarchy.txt
>>
>> I've attached the output of that last one.
>>
>> 4. For comparison, in a standard 60334 image,
>>     Object allSubclasses size "==>11923".
>>     Object class allSubclasses size "==>5959".
>>
>> Now in Pharo 6, the minimal image starts with a standard image and
>> strips these things out...
>>
>> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/ws/output.txt
>>
>> In Pharo 7, there will be a new build system that it will start with a
>> minimal image and build it up to a normal image.  So this may provide
>> a better way to understand the order that things need to be
>> implemented.
>>
>> cheers -ben
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Redline: Talking Runtime basics ...

stepharong
Thanks Guillermo.

@pharoers 
I think that one of the key point to understand what we did is that we decided not to go deep first to produce a minimal minimal image.
Because we have one: Guille produced candlelight and it is working on Pharo and it can do beep and nothing else because it does not have a 
compiler and no UI :)
But it is working and small, we debuged it writing on files. 
Guilllermo also produced images that are 11k :) yes 11k but doing only 2+3 or any addition not producing large integers. 

So we decided to deliver a working bootstrap with a process to maintain it and improve it over the year. 
This is the key catch. Making sure that we are able to remove / repackage it and continue to clean it while
building the entire pharo on it. 

Stef


 

On Tue, 03 Jan 2017 16:38:30 +0100, Guillermo Polito <[hidden email]> wrote:

Hi Phil,

On Sat, Dec 31, 2016 at 10:14 AM, [hidden email] <[hidden email]> wrote:
Isn't there a typo in the first exptession?

Also the list contains a lot of RB things. What purpose do they have in a bootstrap image?

Yes, that is because the bootstrap image contains the compiler. And the compiler requires (amongst others) the AST, implemented initially as part of the refactoring browser, and so it has the RB prefix.

However, the bootstrap does not contain the refactoring browser.
 

Same for RelationSlot and X11 something.

RelationSlot belongs to the class builder package, which is needed to create classes.
Of course RelationSlot is part of an 'Examples', and as such it could be packaged separately, but this was not done yet.

Regarding the X11 thingy, this is probably X11Encoding that you're talking about. This one belongs to the Multilingual-encondings package, and if we are picky you'll see this package is required because it contains mainly Unicode, which is required by the Kernel so far to do a lot of stuff. See for example:

Character>>asLowercase
"If the receiver is uppercase, answer its matching lowercase Character."
^ self characterSet toLowercase: self

Character>>characterSet
^ EncodedCharSet charsetAt: self leadingChar

...

What lead to EncodedCharSet (and the default charset that is Unicode).


Is there a scope and purpose statement for the bootstrap somewhere?

Not writen that I know of. Maybe we should do it. But from many discussions here in the team I'd define the pharo bootstrap as a pharo environment that is 
 - minimal (in the sense that it keeps only stuff required to run)
 - clean (in the sense that there is the less dead code and dead dependencies as possible)
 - and is able to grow (in the sense that you can install new code on it). Think that Pharo is not just the bootstrap. People will not accept a Pharo without a development environment, without a debugger or a workspace.

These three points are a long term goal I woud say, and particularly the last point is in big conflict with the first one. You can have an image that executes code but has no compiler for example.
So making a trade-off but always going towards that goal, what we decided with Christophe one year and a half ago to:

  - select a minimal set of packages that would belong to the bootstrap image
       * To run, you need the Kernel.
       * To be able to install new code you need some kind of I/O and some way to transform the input as classes/methods. Thus we chose to put also basic command line handlers, files, and the compiler. We discussed about putting a flavour of Fuel (say Tanker) instead of the compiler, but Tanker was not stable as it was only experimental and nobody used it for real; the compiler was much more stable as we use it everyday; and besides, Tanker was depending on the class builder, so it only allowed us to remove the compiler but not the class builder.
       * To be clean, we started looking at the dependencies of those packages.
  
  - We finished in a selection of about 53 packages. Remember that Pharo itself has nowadays around 464 packages...
      RPackageOrganizer default packages size "464"
      So this was as good as a ~10% of the original image. Good enough to have a first version.

  - And we started working on dependency analyses on them. The following report made by Christophe lists all the bootstrap packages and their dependencies, and reports when a new dependency is introduced.


      See what the dependency graph looks of the bootstrap


      and extrapolate to the entire image :).
      And that, after we worked a LOT to cut off/separate/untangle/decouple those 53 packages from the rest, rewrite parts of the system (a big example of it is the SessionManager).


Now, this is of course not the end. There are still lots of things to do.

  - the kernel can be still cleaned up even more
  - we could implement some sort of binary serializer specialized for classes and methods to shrink even more the bootstrap
  - the rest of the image has still to be cleaned. Nowadays on top of the bootstrap Monticello and Metacello are bootstrapped using manual a approach and afterwards, the rest of the image is loaded by using a Baseline


    That baseline can be enhanced, splitted in smaller baselines. This require non-bootstrap packages to be revisited, have checked their dependencies and so on... 

So what you see in that list is the result of several decisions and a lot of work. And the focus was to release something and not die in the process. Hope that answers the question (I will copy past this and put it into some wiki btw)

Guille



Phil

Le 31 déc. 2016 05:47, "Ben Coman" <[hidden email]> a écrit :
So for curiosity...

$  $PHARO bootstrap.image eval "Object allSubclasses size"
1677

$  $PHARO bootstrap.image eval "Object allSubclasses size"
837

$  $PHARO bootstrap.image eval "Object class printHierarchy" >
/tmp/60334-bootstrap-hierarchy.txt
(see attached)

cheers -ben


On Sat, Dec 31, 2016 at 4:42 AM, Pavel Krivanek
<[hidden email]> wrote:
> It is better to use smaller bootstrapped image without Monticello but it is
> still quite big.
>
> https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-01-00-Bootstrap/lastSuccessfulBuild/artifact/bootstrap.zip
>
> -- Pavel
>
> 2016-12-30 18:13 GMT+01:00 Ben Coman <[hidden email]>:
>>
>> On Fri, Dec 30, 2016 at 7:51 AM, James Ladd <[hidden email]> wrote:
>> > Hi Pharo People,
>> >
>> > I have continued work on Redline Smalltalk and I'm wanting to discuss
>> > what
>> > the absolute minimum
>> > set of Classes and method should be included in the Redline Runtime.
>> >
>> > Would anyone here like to participate in that discussion?
>> >
>> > - James.
>> > Redline Smalltalk <http://redline.st>
>>
>> Nice to hear you are continuing.
>> I'm not very knowledgable on this, but I'll show you how to pull some
>> data from the work on producing a minimal image.
>>
>> 1. From PharoLauncher > Templates > Pharo 6.0(beta)
>>     download/create an image of build "60334-minimal".
>> 2. Right-click on the image and choose [Copy pathname]
>> 3. In a shell, change to that directory, and execute the following
>>     $ ../../VMs/spur/pharo 60334-minimal.image eval "Object allSubclasses
>> size"
>>         ==> 2801
>>     $ ../../VMs/spur/pharo 60334-minimal.image eval "Object class
>> allSubclasses size"
>>         ==> 1399.
>>     $ ../../VMs/spur/pharo 60334-minimal.image eval "Object class
>> printHierarchy" > /tmp/60334-minimal-class-hierarchy.txt
>>
>> I've attached the output of that last one.
>>
>> 4. For comparison, in a standard 60334 image,
>>     Object allSubclasses size "==>11923".
>>     Object class allSubclasses size "==>5959".
>>
>> Now in Pharo 6, the minimal image starts with a standard image and
>> strips these things out...
>>
>> https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/ws/output.txt
>>
>> In Pharo 7, there will be a new build system that it will start with a
>> minimal image and build it up to a normal image.  So this may provide
>> a better way to understand the order that things need to be
>> implemented.
>>
>> cheers -ben
>
>




--
Using Opera's mail client: http://www.opera.com/mail/
12