A taste of bootstrap

From the Pharo/Squeak point of view this image is very special because it doesn't contain any object inherited from 70's. Pharo lost its umbilical cord.

On Fri, Jul 22, 2016 at 9:12 PM, Pavel Krivanek
<[hidden email]> wrote:
> Hi,
>
> as you maybe know, we are working on Pharo image bootstrap - the process
> that can generate an image from source codes and initialize it correctly.
> Because of practical reasons we do not bootstrap the standard image at once
> but we are trying to bootstrap a small headless kernel image and then load
> the rest of the system into it.
>
> The good news is that we are successful in our effor. We are already able to
> produce well usable images as you can test here:

Great to hear of your continuing progress.

>
> https://goo.gl/fn1VbP
>
> From the Pharo/Squeak point of view this image is very special because it
> doesn't contain any object inherited from 70's. Pharo lost its umbilical
> cord.

Does this mean you are starting with a zero byte file and adding nil,
true, false, etc...?
Or what is the size of the image you start with?

cheers -ben

>
> Notice that the initial display width is too narrow and and we still need a
> lot of work on the building process, but In the next weeks and months it
> will change a lot the Pharo development - especially as soon as it will be
> combined with Git support.
>
> Cheers,
> -- Pavel
>
>

Hi Pavel,

Congrats for your hard work!  You know me well, so don't take my next question as negative.  

You have been working in minimal images since years. Guille has also done quite some work, and many other people as well. 

My question is.... have we decreased the effort to keep it working as Pharo evolves? Previously, there were always problems with the dependencies: a single commit to Pharo would add or break a dependency and so we cannot bootstrap anymore. In other words...there was still a lot of human action needed to check and fix dependencies. Is this still the case?

Cheers,

  

On Fri, Jul 22, 2016 at 10:37 AM, Pavel Krivanek <[hidden email]> wrote:

2016-07-22 15:22 GMT+02:00 Ben Coman <[hidden email]>:

On Fri, Jul 22, 2016 at 9:12 PM, Pavel Krivanek
<[hidden email]> wrote:
> Hi,
>
> as you maybe know, we are working on Pharo image bootstrap - the process
> that can generate an image from source codes and initialize it correctly.
> Because of practical reasons we do not bootstrap the standard image at once
> but we are trying to bootstrap a small headless kernel image and then load
> the rest of the system into it.
>
> The good news is that we are successful in our effor. We are already able to
> produce well usable images as you can test here:

Great to hear of your continuing progress.

>
> https://goo.gl/fn1VbP
>
> From the Pharo/Squeak point of view this image is very special because it
> doesn't contain any object inherited from 70's. Pharo lost its umbilical
> cord.

Does this mean you are starting with a zero byte file and adding nil,
true, false, etc...?
Or what is the size of the image you start with?

No, we are not generating image file directly. We use special VM simulator and then save its object memory. 

Our bootstrapped image has about 5MB now (in Spur format).

 

cheers -ben

>
> Notice that the initial display width is too narrow and and we still need a
> lot of work on the building process, but In the next weeks and months it
> will change a lot the Pharo development - especially as soon as it will be
> combined with Git support.
>
> Cheers,
> -- Pavel
>
>

--

Mariano
http://marianopeck.wordpress.com

On Fri, Jul 22, 2016 at 4:12 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi Pavel,

Congrats for your hard work!  You know me well, so don't take my next question as negative.  

You have been working in minimal images since years. Guille has also done quite some work, and many other people as well. 

My question is.... have we decreased the effort to keep it working as Pharo evolves? Previously, there were always problems with the dependencies: a single commit to Pharo would add or break a dependency and so we cannot bootstrap anymore. In other words...there was still a lot of human action needed to check and fix dependencies. Is this still the case?

Complex questions requires complex answers :). It's not a yes, neither a no.

- We have automatized the dependency analyses

- We added Lint rules that ensure that no new dependencies are added to the kernel because of an integration

- We will be adding soon enough more tools to automatically validate and ensure the health of the bootstrap, and to detect possible problems.

Buuut,

- Lint rules can (and sometimes are) ignored

- Not everybody is aware of how to maintain dependencies or even they do not know they should care about it

So the human factor is still of big importance (and risk), and educating people is harder than machines ^^

Cheers,

  

On Fri, Jul 22, 2016 at 10:37 AM, Pavel Krivanek <[hidden email]> wrote:

2016-07-22 15:22 GMT+02:00 Ben Coman <[hidden email]>:

On Fri, Jul 22, 2016 at 9:12 PM, Pavel Krivanek
<[hidden email]> wrote:
> Hi,
>
> as you maybe know, we are working on Pharo image bootstrap - the process
> that can generate an image from source codes and initialize it correctly.
> Because of practical reasons we do not bootstrap the standard image at once
> but we are trying to bootstrap a small headless kernel image and then load
> the rest of the system into it.
>
> The good news is that we are successful in our effor. We are already able to
> produce well usable images as you can test here:

Great to hear of your continuing progress.

>
> https://goo.gl/fn1VbP
>
> From the Pharo/Squeak point of view this image is very special because it
> doesn't contain any object inherited from 70's. Pharo lost its umbilical
> cord.

Does this mean you are starting with a zero byte file and adding nil,
true, false, etc...?
Or what is the size of the image you start with?

No, we are not generating image file directly. We use special VM simulator and then save its object memory. 

Our bootstrapped image has about 5MB now (in Spur format).

 

cheers -ben

>
> Notice that the initial display width is too narrow and and we still need a
> lot of work on the building process, but In the next weeks and months it
> will change a lot the Pharo development - especially as soon as it will be
> combined with Git support.
>
> Cheers,
> -- Pavel
>
>

--

Mariano
http://marianopeck.wordpress.com

2016-07-22 16:24 GMT+02:00 Guillermo Polito <[hidden email]>:

On Fri, Jul 22, 2016 at 4:12 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi Pavel,

Congrats for your hard work!  You know me well, so don't take my next question as negative.  

You have been working in minimal images since years. Guille has also done quite some work, and many other people as well. 

My question is.... have we decreased the effort to keep it working as Pharo evolves? Previously, there were always problems with the dependencies: a single commit to Pharo would add or break a dependency and so we cannot bootstrap anymore. In other words...there was still a lot of human action needed to check and fix dependencies. Is this still the case?

Complex questions requires complex answers :). It's not a yes, neither a no.

- We have automatized the dependency analyses

- We added Lint rules that ensure that no new dependencies are added to the kernel because of an integration

- We will be adding soon enough more tools to automatically validate and ensure the health of the bootstrap, and to detect possible problems.

Buuut,

- Lint rules can (and sometimes are) ignored

- Not everybody is aware of how to maintain dependencies or even they do not know they should care about it

So the human factor is still of big importance (and risk), and educating people is harder than machines ^^

I must say that current need of human work required for proper reloading of all Pharo packages is too high. It can be better only as soon as the bootstrapping will be part of our development/release process.

On the other hand, dependency problems are now quite rare and are very rare in the kernel packages. 

-- Pavel

 

Cheers,

  

On Fri, Jul 22, 2016 at 10:37 AM, Pavel Krivanek <[hidden email]> wrote:

2016-07-22 15:22 GMT+02:00 Ben Coman <[hidden email]>:

On Fri, Jul 22, 2016 at 9:12 PM, Pavel Krivanek
<[hidden email]> wrote:
> Hi,
>
> as you maybe know, we are working on Pharo image bootstrap - the process
> that can generate an image from source codes and initialize it correctly.
> Because of practical reasons we do not bootstrap the standard image at once
> but we are trying to bootstrap a small headless kernel image and then load
> the rest of the system into it.
>
> The good news is that we are successful in our effor. We are already able to
> produce well usable images as you can test here:

Great to hear of your continuing progress.

>
> https://goo.gl/fn1VbP
>
> From the Pharo/Squeak point of view this image is very special because it
> doesn't contain any object inherited from 70's. Pharo lost its umbilical
> cord.

Does this mean you are starting with a zero byte file and adding nil,
true, false, etc...?
Or what is the size of the image you start with?

No, we are not generating image file directly. We use special VM simulator and then save its object memory. 

Our bootstrapped image has about 5MB now (in Spur format).

 

cheers -ben

>
> Notice that the initial display width is too narrow and and we still need a
> lot of work on the building process, but In the next weeks and months it
> will change a lot the Pharo development - especially as soon as it will be
> combined with Git support.
>
> Cheers,
> -- Pavel
>
>

--

Mariano
http://marianopeck.wordpress.com

On Fri, Jul 22, 2016 at 4:12 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi Pavel,

Congrats for your hard work!  You know me well, so don't take my next question as negative.  

You have been working in minimal images since years. Guille has also done quite some work, and many other people as well. 

My question is.... have we decreased the effort to keep it working as Pharo evolves? Previously, there were always problems with the dependencies: a single commit to Pharo would add or break a dependency and so we cannot bootstrap anymore. In other words...there was still a lot of human action needed to check and fix dependencies. Is this still the case?

Complex questions requires complex answers :). It's not a yes, neither a no.

- We have automatized the dependency analyses

- We added Lint rules that ensure that no new dependencies are added to the kernel because of an integration

- We will be adding soon enough more tools to automatically validate and ensure the health of the bootstrap, and to detect possible problems.

Buuut,

- Lint rules can (and sometimes are) ignored

- Not everybody is aware of how to maintain dependencies or even they do not know they should care about it

So the human factor is still of big importance (and risk), and educating people is harder than machines ^^

Cheers,

  

On Fri, Jul 22, 2016 at 10:37 AM, Pavel Krivanek <[hidden email]> wrote:

2016-07-22 15:22 GMT+02:00 Ben Coman <[hidden email]>:

On Fri, Jul 22, 2016 at 9:12 PM, Pavel Krivanek
<[hidden email]> wrote:
> Hi,
>
> as you maybe know, we are working on Pharo image bootstrap - the process
> that can generate an image from source codes and initialize it correctly.
> Because of practical reasons we do not bootstrap the standard image at once
> but we are trying to bootstrap a small headless kernel image and then load
> the rest of the system into it.
>
> The good news is that we are successful in our effor. We are already able to
> produce well usable images as you can test here:

Great to hear of your continuing progress.

>
> https://goo.gl/fn1VbP
>
> From the Pharo/Squeak point of view this image is very special because it
> doesn't contain any object inherited from 70's. Pharo lost its umbilical
> cord.

Does this mean you are starting with a zero byte file and adding nil,
true, false, etc...?
Or what is the size of the image you start with?

No, we are not generating image file directly. We use special VM simulator and then save its object memory. 

Our bootstrapped image has about 5MB now (in Spur format).

 

cheers -ben

>
> Notice that the initial display width is too narrow and and we still need a
> lot of work on the building process, but In the next weeks and months it
> will change a lot the Pharo development - especially as soon as it will be
> combined with Git support.
>
> Cheers,
> -- Pavel
>
>

--

Mariano
http://marianopeck.wordpress.com

On Fri, Jul 22, 2016 at 11:38 AM, Pavel Krivanek <[hidden email]> wrote:

2016-07-22 16:24 GMT+02:00 Guillermo Polito <[hidden email]>:

On Fri, Jul 22, 2016 at 4:12 PM, Mariano Martinez Peck <[hidden email]> wrote:

Hi Pavel,

Congrats for your hard work!  You know me well, so don't take my next question as negative.  

You have been working in minimal images since years. Guille has also done quite some work, and many other people as well. 

My question is.... have we decreased the effort to keep it working as Pharo evolves? Previously, there were always problems with the dependencies: a single commit to Pharo would add or break a dependency and so we cannot bootstrap anymore. In other words...there was still a lot of human action needed to check and fix dependencies. Is this still the case?

Complex questions requires complex answers :). It's not a yes, neither a no.

- We have automatized the dependency analyses

- We added Lint rules that ensure that no new dependencies are added to the kernel because of an integration

- We will be adding soon enough more tools to automatically validate and ensure the health of the bootstrap, and to detect possible problems.

Buuut,

- Lint rules can (and sometimes are) ignored

- Not everybody is aware of how to maintain dependencies or even they do not know they should care about it

So the human factor is still of big importance (and risk), and educating people is harder than machines ^^

I must say that current need of human work required for proper reloading of all Pharo packages is too high. It can be better only as soon as the bootstrapping will be part of our development/release process.

Exactly!!! Totally agree. And..what are the plans of getting the bootstrapping as part of the development and release process? I am sure it is on the roadmap. 

 

On the other hand, dependency problems are now quite rare and are very rare in the kernel packages. 

That's very good to hear!

 

-- Pavel

 

Cheers,

  

On Fri, Jul 22, 2016 at 10:37 AM, Pavel Krivanek <[hidden email]> wrote:

2016-07-22 15:22 GMT+02:00 Ben Coman <[hidden email]>:

On Fri, Jul 22, 2016 at 9:12 PM, Pavel Krivanek
<[hidden email]> wrote:
> Hi,
>
> as you maybe know, we are working on Pharo image bootstrap - the process
> that can generate an image from source codes and initialize it correctly.
> Because of practical reasons we do not bootstrap the standard image at once
> but we are trying to bootstrap a small headless kernel image and then load
> the rest of the system into it.
>
> The good news is that we are successful in our effor. We are already able to
> produce well usable images as you can test here:

Great to hear of your continuing progress.

>
> https://goo.gl/fn1VbP
>
> From the Pharo/Squeak point of view this image is very special because it
> doesn't contain any object inherited from 70's. Pharo lost its umbilical
> cord.

Does this mean you are starting with a zero byte file and adding nil,
true, false, etc...?
Or what is the size of the image you start with?

No, we are not generating image file directly. We use special VM simulator and then save its object memory. 

Our bootstrapped image has about 5MB now (in Spur format).

 

cheers -ben

>
> Notice that the initial display width is too narrow and and we still need a
> lot of work on the building process, but In the next weeks and months it
> will change a lot the Pharo development - especially as soon as it will be
> combined with Git support.
>
> Cheers,
> -- Pavel
>
>

--

Mariano
http://marianopeck.wordpress.com

--

Mariano
http://marianopeck.wordpress.com