Smalltalk › Pharo › Pharo Smalltalk Developers

Image and bootstrap dependencies

‹ Previous Topic Next Topic ›
 
classic Classic list List threaded Threaded
♦
♦
 24 messages Options
12
demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Image and bootstrap dependencies

demarey
Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.
Ben Coman
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Ben Coman
On Fri, Sep 4, 2015 at 9:13 PM, Christophe Demarey
<[hidden email]> wrote:

> Hi,
>
> I created a job to automatically generate a report of the Pharo image
> dependencies as well as current bootstrap dependencies.
> You can find them here:
>
> https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/dependency-report.html
> https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report.html
>
>
> There are a lot (too many) dependencies but you can focus on some of them
> with the search field.
> Dependencies inside orange boxes are dependencies that are there but that we
> ignore (for example, an example method adds an extra-dependency). It should
> be fixed later but is not a problem for now. Ignored dependencies are
> declared in the package manifest in #ignoredDependencies.
> Dependencies inside red boxes are dependencies we want to remove for the
> bootstrap. For now, they are listed in the bootstrap dependency report.
>
> People that are willing to help to have a more modular Pharo could first
> take a look at the bootstrap dependency report and check dependencies that
> we should cut.
> Once we know, we can open issues on fogbugz in the 'bootstrap category' and
> update manifests accordingly.
> To investigate more deeply dependencies, you can use the DependencyAnalyzer
> (in the Pharo Catalog) to actually check dependencies (inheritance,
> reference, extension, traits, message send).

I tried the Dependency Analyzer.  First a minor point, it wasn't
obvious how to invoke it.  I expected to see "Dependency Analyzer" in
the tools but its not there.  I think I correctly guessed it was
"Package Dependencies Browser", but it would be better to match the
catalog name.

The sample dependency I looked at from bootstrap-dependency-report.html
was Collections-Streams --> Tool-Transcript
which seems to be due to only single method "Generator class>>examplePrimes".
So should this be coloured orange by marking it in #ignoredDependencies ?

Also, even though Collections-Streams --> Transcript is not
highlighted red nor orange, this dependency also seems due only to
"Generator class>>examplePrimes"

cheers -ben

demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
Hi Ben,

Le 4 sept. 2015 à 17:04, Ben Coman a écrit :

> On Fri, Sep 4, 2015 at 9:13 PM, Christophe Demarey
> <[hidden email]> wrote:
>> Hi,
>>
>> I created a job to automatically generate a report of the Pharo image
>> dependencies as well as current bootstrap dependencies.
>> You can find them here:
>>
>> https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/dependency-report.html
>> https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report.html
>>
>>
>> There are a lot (too many) dependencies but you can focus on some of them
>> with the search field.
>> Dependencies inside orange boxes are dependencies that are there but that we
>> ignore (for example, an example method adds an extra-dependency). It should
>> be fixed later but is not a problem for now. Ignored dependencies are
>> declared in the package manifest in #ignoredDependencies.
>> Dependencies inside red boxes are dependencies we want to remove for the
>> bootstrap. For now, they are listed in the bootstrap dependency report.
>>
>> People that are willing to help to have a more modular Pharo could first
>> take a look at the bootstrap dependency report and check dependencies that
>> we should cut.
>> Once we know, we can open issues on fogbugz in the 'bootstrap category' and
>> update manifests accordingly.
>> To investigate more deeply dependencies, you can use the DependencyAnalyzer
>> (in the Pharo Catalog) to actually check dependencies (inheritance,
>> reference, extension, traits, message send).
>
> I tried the Dependency Analyzer.  First a minor point, it wasn't
> obvious how to invoke it.  I expected to see "Dependency Analyzer" in
> the tools but its not there.  I think I correctly guessed it was
> "Package Dependencies Browser", but it would be better to match the
> catalog name.
ok, I will fix that.
I will also add the analyzer in the system. In a second step, I will introduce rules to check dependencies.

> The sample dependency I looked at from bootstrap-dependency-report.html
> was Collections-Streams --> Tool-Transcript
> which seems to be due to only single method "Generator class>>examplePrimes".
> So should this be coloured orange by marking it in #ignoredDependencies ?

The clean solution is to have a package Collections-Streams-Example with this method.
It depends how we want to be strict with that. I can live with no additional package and the dependency added in #ignoredDependencies.
There are so many wrong dependencies in the system that I will prefer at a first step to put dependencies introduced by example methods or settings by example as ignored dependencies because they will not be called.
We could fix that in a second step if we want and let us focus on stronger dependencies.

Thanks for helping.

PS: Even for people not having time to clean the system, when you contribute code to the image, pleas check that you do not introduce a dependency from a low-level package to a higher level package. It is not obvious to catch but for example, check that you do not introduce a dependency to UI packages when the package could work without UI.

smime.p7s (5K) Download Attachment
Ben Coman
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Ben Coman
On Fri, Sep 4, 2015 at 11:18 PM, Christophe Demarey
<[hidden email]> wrote:

> Hi Ben,
>
> Le 4 sept. 2015 à 17:04, Ben Coman a écrit :
>>
>> I tried the Dependency Analyzer.  First a minor point, it wasn't
>> obvious how to invoke it.  I expected to see "Dependency Analyzer" in
>> the tools but its not there.  I think I correctly guessed it was
>> "Package Dependencies Browser", but it would be better to match the
>> catalog name.
>
> ok, I will fix that.
> I will also add the analyzer in the system.

Sorry I'm (easily) confused - when you say "also" makes me think you
are referring to "Package Dependencies Analysis".  I only meant that
if "Package Dependencies Browser" is the entry point, it would be
better named "Dependency Analyzer".  I did not have an opinion on
whether "Package Dependencies Analysis" should be added to the menu
(but maybe thats not what you meant.)

> In a second step, I will introduce rules to check dependencies.

btw, I'm not sure how much gloss you'd like to put on the Package
Dependencies Browser, but just some immediate thoughts:
* It feels awkward having the <Add package> button at the top and the
<Add all> button at the bottom. It would be nicer if the first was
also at the bottom on the same line (i.e. half the width). Same with
the <remove...> buttons.
* The opposite of the green-plus icon would be a red-minus icon rather
than the trash-can icon.

In Package Dependencies Analysis, taking for example  "(AST-Core -->
Dependent packages : 15 | Dependencies : 134 (+ 14 extension
dependencies)"
I see method Traits > TBehavior >> #parseTreeFor:
whose extension protocol is *ast-core.
This to me doesn't seem a dependency.

>
>> The sample dependency I looked at from bootstrap-dependency-report.html
>> was Collections-Streams --> Tool-Transcript
>> which seems to be due to only single method "Generator class>>examplePrimes".
>> So should this be coloured orange by marking it in #ignoredDependencies ?
>
> The clean solution is to have a package Collections-Streams-Example with this method.
> It depends how we want to be strict with that. I can live with no additional package and the dependency added in #ignoredDependencies.
> There are so many wrong dependencies in the system that I will prefer at a first step to put dependencies introduced by example methods or settings by example as ignored dependencies because they will not be called.

Are examples the only (or majority) kind of ignoredDependencies ?  If
there is a significant number of other kinds, it might be useful to
split the examples out

> We could fix that in a second step if we want and let us focus on stronger dependencies.
>
> Thanks for helping.
>
> PS: Even for people not having time to clean the system, when you contribute code to the image, pleas check that you do not introduce a dependency from a low-level package to a higher level package. It is not obvious to catch but for example, check that you do not introduce a dependency to UI packages when the package could work without UI.

Even with my best intentions, I think the long term reality is that
the workflow delay of installing Dependency Analyzer to do this will
inhibit me doing this.  When working on issues I always start with a
fresh/latest build.  As a tool to facilitate one of Pharo's goals,
perhaps it could be integrated into the Image and be prominently
displayed on the top level World menu to encourage its use.

Now I guess it will be hard to identify an introduced dependency just
by looking at a one shot report after a modification. It would help if
this could be compared to a prior snapshot stored in the image
produced during the CI builds.  Perhaps even it could provide live
advice through integration with Quality Assistant.

cheers -ben

Ben Coman
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Ben Coman
On Sat, Sep 5, 2015 at 12:23 AM, Ben Coman <[hidden email]> wrote:
> In Package Dependencies Analysis, taking for example  "(AST-Core -->
> Dependent packages : 15 | Dependencies : 134 (+ 14 extension
> dependencies)"

If I select Announcements-Core, then click <Show extension
dependencies> to unselect it, I get "MNU
DAPotentialDependenciesNode>>containsOnlyExtensionDependency"

cheers -ben

stepharo
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

stepharo
In reply to this post by Ben Coman

> btw, I'm not sure how much gloss you'd like to put on the Package
> Dependencies Browser, but just some immediate thoughts:
> * It feels awkward having the <Add package> button at the top and the
> <Add all> button at the bottom. It would be nicer if the first was
> also at the bottom on the same line (i.e. half the width). Same with
> the <remove...> buttons.

Yes the add button is awkwark. Christophe is often not using it via the
tool menu.

> * The opposite of the green-plus icon would be a red-minus icon rather
> than the trash-can icon.
>>> The sample dependency I looked at from bootstrap-dependency-report.html
>>> was Collections-Streams --> Tool-Transcript
>>> which seems to be due to only single method "Generator class>>examplePrimes".
>>> So should this be coloured orange by marking it in #ignoredDependencies ?
>> The clean solution is to have a package Collections-Streams-Example with this method.
>> It depends how we want to be strict with that. I can live with no additional package and the dependency added in #ignoredDependencies.
>> There are so many wrong dependencies in the system that I will prefer at a first step to put dependencies introduced by example methods or settings by example as ignored dependencies because they will not be called.
> Are examples the only (or majority) kind of ignoredDependencies ?  If
> there is a significant number of other kinds, it might be useful to
> split the examples out
>
> Even with my best intentions, I think the long term reality is that
> the workflow delay of installing Dependency Analyzer to do this will
> inhibit me doing this.  When working on issues I always start with a
> fresh/latest build.  As a tool to facilitate one of Pharo's goals,
> perhaps it could be integrated into the Image and be prominently
> displayed on the top level World menu to encourage its use.

agree we should add the tool in the system so that we can use it as fast
as possible.
>
> Now I guess it will be hard to identify an introduced dependency just
> by looking at a one shot report after a modification. It would help if
> this could be compared to a prior snapshot stored in the image
> produced during the CI builds.  Perhaps even it could provide live
> advice through integration with Quality Assistant.
No you can really get a good understanding of the dependency.
We should continue to address them. Often the system is not thought with
layer and modularity in mind (for example
all the inform: and cursorWhile: everywhere. I think that we should
remove as much as possible as cursorWhile:. for example.
Stef

>
> cheers -ben
>
>


Nicolai Hess
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Nicolai Hess
In reply to this post by demarey
Hi Christophe,

the tool and this depedency report is great.

But how are the dependencies categories "Ignored dependency" , "Dependency to remove" created,
is this a manual step?

For examlpe, I don't understand why
Graphics-Display Objects Graphics-Primitives
are "Dependency to remove" for package Athens-Balloon.
Aren't these low level packages?



regards
Nicolai




2015-09-04 15:13 GMT+02:00 Christophe Demarey <[hidden email]>:
Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.
demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
In reply to this post by Ben Coman

Le 4 sept. 2015 à 18:23, Ben Coman a écrit :

> On Fri, Sep 4, 2015 at 11:18 PM, Christophe Demarey
> <[hidden email]> wrote:
>> Hi Ben,
>>
>> Le 4 sept. 2015 à 17:04, Ben Coman a écrit :
>>>
>>> I tried the Dependency Analyzer.  First a minor point, it wasn't
>>> obvious how to invoke it.  I expected to see "Dependency Analyzer" in
>>> the tools but its not there.  I think I correctly guessed it was
>>> "Package Dependencies Browser", but it would be better to match the
>>> catalog name.
>>
>> ok, I will fix that.
>> I will also add the analyzer in the system.
>
> Sorry I'm (easily) confused - when you say "also" makes me think you
> are referring to "Package Dependencies Analysis".  I only meant that
> if "Package Dependencies Browser" is the entry point, it would be
> better named "Dependency Analyzer".
You were clear. Maybe I was not. I meant I will open an issue to add the dependency anlayzer packages as part of the default Pharo image.

> I did not have an opinion on
> whether "Package Dependencies Analysis" should be added to the menu
> (but maybe thats not what you meant.)
>
>> In a second step, I will introduce rules to check dependencies.
>
> btw, I'm not sure how much gloss you'd like to put on the Package
> Dependencies Browser, but just some immediate thoughts:
> * It feels awkward having the <Add package> button at the top and the
> <Add all> button at the bottom. It would be nicer if the first was
> also at the bottom on the same line (i.e. half the width). Same with
> the <remove...> buttons.
> * The opposite of the green-plus icon would be a red-minus icon rather
> than the trash-can icon.
I never used this UI because I find easier to just right-click on a selected package in Nautilus and select 'browse dependencies ...'.
I will take a look at this UI to enhance it a bit.

> In Package Dependencies Analysis, taking for example  "(AST-Core -->
> Dependent packages : 15 | Dependencies : 134 (+ 14 extension
> dependencies)"
> I see method Traits > TBehavior >> #parseTreeFor:
> whose extension protocol is *ast-core.
> This to me doesn't seem a dependency.

It is seen as a dependency because if you want to load AST-Core, containing an extension method on TBehavior class packaged in Traits, and Traits is not in the system, it will fail.
We could imagine optional extension methods, i.e. only loaded in the system if the class extended is present but it is not like that now.

>>
>>> The sample dependency I looked at from bootstrap-dependency-report.html
>>> was Collections-Streams --> Tool-Transcript
>>> which seems to be due to only single method "Generator class>>examplePrimes".
>>> So should this be coloured orange by marking it in #ignoredDependencies ?
>>
>> The clean solution is to have a package Collections-Streams-Example with this method.
>> It depends how we want to be strict with that. I can live with no additional package and the dependency added in #ignoredDependencies.
>> There are so many wrong dependencies in the system that I will prefer at a first step to put dependencies introduced by example methods or settings by example as ignored dependencies because they will not be called.
>
> Are examples the only (or majority) kind of ignoredDependencies ?  If
> there is a significant number of other kinds, it might be useful to
> split the examples out
At this time, not a lot.
It is mostly when you know that a code will not be executed.

>> We could fix that in a second step if we want and let us focus on stronger dependencies.
>>
>> Thanks for helping.
>>
>> PS: Even for people not having time to clean the system, when you contribute code to the image, pleas check that you do not introduce a dependency from a low-level package to a higher level package. It is not obvious to catch but for example, check that you do not introduce a dependency to UI packages when the package could work without UI.
>
> Even with my best intentions, I think the long term reality is that
> the workflow delay of installing Dependency Analyzer to do this will
> inhibit me doing this.  When working on issues I always start with a
> fresh/latest build.  As a tool to facilitate one of Pharo's goals,
> perhaps it could be integrated into the Image and be prominently
> displayed on the top level World menu to encourage its use.
Yes, that will be done very soon.

> Now I guess it will be hard to identify an introduced dependency just
> by looking at a one shot report after a modification. It would help if
> this could be compared to a prior snapshot stored in the image
> produced during the CI builds.

it will be the first step to check regressions from the dependency point of view.
It can be done with a CI job if dependency analysis is published for each image.

>  Perhaps even it could provide live
> advice through integration with Quality Assistant.

It is what I want at the end. When you update a method or a class, run a dependency check against the current version and the version about to be saved.
To reach this point, each package needs to have its dependencies specified. It will come ...

Thanks for the feedback

smime.p7s (5K) Download Attachment
demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
In reply to this post by Nicolai Hess
Hi Nicolai,

Le 4 sept. 2015 à 21:12, Nicolai Hess a écrit :

Hi Christophe,

the tool

The tool was originally created by Baptiste Quidé and I enhanced it with message send analysis.

and this depedency report is great.

Thanks.

But how are the dependencies categories "Ignored dependency" ,

Yes, it is a manual step. You find them in the package manifest class, in #ignoredDependencies.

"Dependency to remove" created,
is this a manual step?

This one also. There is a special manifest ManifestBootstrap holding this dependenciesToRemove.

For examlpe, I don't understand why
Graphics-Display Objects Graphics-Primitives
are "Dependency to remove" for package Athens-Balloon.
Aren't these low level packages?

Yes, it is not clear.
Dependencies to remove are packages we do NOT want in the bootstrap. They appear in red everywhere. It is not the dependency we want to remove from the bootstrap but the package itself.
I will update that.

smime.p7s (5K) Download Attachment
stepharo
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

stepharo
In reply to this post by demarey


Le 4/9/15 21:54, Christophe Demarey a écrit :
>> Are examples the only (or majority) kind of ignoredDependencies ?  If
>> >there is a significant number of other kinds, it might be useful to
>> >split the examples out
> At this time, not a lot.
> It is mostly when you know that a code will not be executed.

but in general we would prefer to avoid to tag them as ignored and
addresse the dependencies.
So ignored is just here to make sure that we can bootstrap fast and then
we will have to revise the ignored.
>


demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
In reply to this post by demarey
Hi,

We now have a best overview of bootstrap dependencies.
With Max, we reused Dijkstra algo in Moose to help finding dependencies leading to Morphic and co.
It is now possible to have a visualization that is usable (there is also a button to open it in the report):  https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report-graph.html

Next steps are to cut dependencies in red in the report table.

Christophe.

ps: by the way the dependency analyzer (with fixes asked) will be introduced soon in the image.


Le 4 sept. 2015 à 15:13, Christophe Demarey a écrit :

Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.


smime.p7s (5K) Download Attachment
ccrraaiigg
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

ccrraaiigg

Hoi Christophe--

> We now have a best overview of bootstrap dependencies.

     That's a pretty grim picture. :)  Can you post a movie of you
picking through that ball of connections? (E.g., with the search feature
as you described, or any with any other UI affordance that you have with
this system.)


     thanks!

-C

--
Craig Latta
netjam.org
+31   6 2757 7177 (SMS ok)
+ 1 415  287 3547 (no SMS)


Ben Coman
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Ben Coman
In reply to this post by demarey
The interactive graph is cool.  Some suggestions:
* Mark nodes of dependencies you want to prune in red.  
* Allow nodes to be pinned
* Node size to reflect size of package. A small number of dependencies to a fat node might be best to tackle first.
* Highlighting & Fisheye
* Mark in bootstrap-dependency-report-graph-data.csv which dependencies you want to eliminate, so that others might have a play with visualisation.

More exotic...
* Serve the bootstrap-dependency-report-graph.html and bootstrap-dependency-report-graph-data.csv out from Zinc, so I can point my browser at my local Pharo Image, and when a dependency is deleted, the graph updates automagically
* When I double-click a node in the graph, it selects that item in the dependency analyzer in Image.
* I guess the same might be done with two local Images - Moose as the observer and Pharo as the subject.

cheers -ben

On Wed, Sep 9, 2015 at 5:33 PM, Christophe Demarey <[hidden email]> wrote:
Hi,

We now have a best overview of bootstrap dependencies.
With Max, we reused Dijkstra algo in Moose to help finding dependencies leading to Morphic and co.
It is now possible to have a visualization that is usable (there is also a button to open it in the report):  https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report-graph.html

Next steps are to cut dependencies in red in the report table.

Christophe.

ps: by the way the dependency analyzer (with fixes asked) will be introduced soon in the image.


Le 4 sept. 2015 à 15:13, Christophe Demarey a écrit :

Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.
Tudor Girba-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Tudor Girba-2
In reply to this post by demarey
Hi,

Very interesting.

Could you detail which information you use for building the edges?

Cheers,
Doru



On Wed, Sep 9, 2015 at 11:33 AM, Christophe Demarey <[hidden email]> wrote:
Hi,

We now have a best overview of bootstrap dependencies.
With Max, we reused Dijkstra algo in Moose to help finding dependencies leading to Morphic and co.
It is now possible to have a visualization that is usable (there is also a button to open it in the report):  https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report-graph.html

Next steps are to cut dependencies in red in the report table.

Christophe.

ps: by the way the dependency analyzer (with fixes asked) will be introduced soon in the image.


Le 4 sept. 2015 à 15:13, Christophe Demarey a écrit :

Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.




--

"Every thing has its own flow"
Ben Coman
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Ben Coman
If your browser shows the web page source you will see the data being pulled from...
cheers -ben


On Thu, Sep 10, 2015 at 5:32 AM, Tudor Girba <[hidden email]> wrote:
Hi,

Very interesting.

Could you detail which information you use for building the edges?

Cheers,
Doru



On Wed, Sep 9, 2015 at 11:33 AM, Christophe Demarey <[hidden email]> wrote:
Hi,

We now have a best overview of bootstrap dependencies.
With Max, we reused Dijkstra algo in Moose to help finding dependencies leading to Morphic and co.
It is now possible to have a visualization that is usable (there is also a button to open it in the report):  https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report-graph.html

Next steps are to cut dependencies in red in the report table.

Christophe.

ps: by the way the dependency analyzer (with fixes asked) will be introduced soon in the image.


Le 4 sept. 2015 à 15:13, Christophe Demarey a écrit :

Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.




--

"Every thing has its own flow"
demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
In reply to this post by Tudor Girba-2
The data comes from the dependency analyzer analysis (re-compute each time to avoid wrong information) + the information given in Manifests : manuallyResolvedDependencies (potential dependencies needing a human decision) and ignoredDependencies.

Le 9 sept. 2015 à 23:32, Tudor Girba a écrit :

Hi,

Very interesting.

Could you detail which information you use for building the edges?

Cheers,
Doru



On Wed, Sep 9, 2015 at 11:33 AM, Christophe Demarey <[hidden email]> wrote:
Hi,

We now have a best overview of bootstrap dependencies.
With Max, we reused Dijkstra algo in Moose to help finding dependencies leading to Morphic and co.
It is now possible to have a visualization that is usable (there is also a button to open it in the report):  https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report-graph.html

Next steps are to cut dependencies in red in the report table.

Christophe.

ps: by the way the dependency analyzer (with fixes asked) will be introduced soon in the image.

<Capture d’écran 2015-09-09 à 11.33.02.png>

Le 4 sept. 2015 à 15:13, Christophe Demarey a écrit :

Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.




--

"Every thing has its own flow"


smime.p7s (5K) Download Attachment
demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
In reply to this post by Ben Coman
Hi Ben,

Le 9 sept. 2015 à 16:03, Ben Coman a écrit :

The interactive graph is cool.  Some suggestions:
* Mark nodes of dependencies you want to prune in red.  

yes, I will add it ASAP

* Allow nodes to be pinned

very good idea.
TBH, I justa wanted to have a first visualization usable very fast. The current result can be improved a lot.

* Node size to reflect size of package. A small number of dependencies to a fat node might be best to tackle first.

+1


+1 for hightlihgting

* Mark in bootstrap-dependency-report-graph-data.csv which dependencies you want to eliminate, so that others might have a play with visualisation.

ok


More exotic...
* Serve the bootstrap-dependency-report-graph.html and bootstrap-dependency-report-graph-data.csv out from Zinc, so I can point my browser at my local Pharo Image, and when a dependency is deleted, the graph updates automagically
* When I double-click a node in the graph, it selects that item in the dependency analyzer in Image.
* I guess the same might be done with two local Images - Moose as the observer and Pharo as the subject.

also requires a bit more time.
I would also add: get the same visualization in Pharo with Roassal :)

Thanks for the feedback,
Ben


smime.p7s (5K) Download Attachment
Ben Coman
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Ben Coman
On Thu, Sep 10, 2015 at 3:54 PM, Christophe Demarey
<[hidden email]> wrote:

> Hi Ben,
>
> Le 9 sept. 2015 à 16:03, Ben Coman a écrit :
>
> The interactive graph is cool.  Some suggestions:
> * Mark nodes of dependencies you want to prune in red.
>
>
> yes, I will add it ASAP
>
> * Allow nodes to be pinned
>    * http://bl.ocks.org/GerHobbelt/3637711
>
>
> very good idea.
> TBH, I justa wanted to have a first visualization usable very fast. The
> current result can be improved a lot.

no problem. release early, release often is a good practice.
cheers -ben

demarey
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

demarey
In reply to this post by ccrraaiigg
Hi Craig,

Le 9 sept. 2015 à 11:53, Craig Latta a écrit :


Hoi Christophe--

We now have a best overview of bootstrap dependencies.

    That's a pretty grim picture. :)  Can you post a movie of you
picking through that ball of connections? (E.g., with the search feature
as you described, or any with any other UI affordance that you have with
this system.)

Maybe a misunderstanding, the search feature is for the able only and not the graph itself.
Yes, it is still a bit difficult to get the information because there are a lot of links between nodes.

Christophe.


smime.p7s (5K) Download Attachment
Tudor Girba-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Image and bootstrap dependencies

Tudor Girba-2
In reply to this post by demarey
Hi,

Ok, I understand that part, but my question was: what code relationships does the analyzer take into account?

Cheers,
Doru 

On Thu, Sep 10, 2015 at 9:43 AM, Christophe Demarey <[hidden email]> wrote:
The data comes from the dependency analyzer analysis (re-compute each time to avoid wrong information) + the information given in Manifests : manuallyResolvedDependencies (potential dependencies needing a human decision) and ignoredDependencies.

Le 9 sept. 2015 à 23:32, Tudor Girba a écrit :

Hi,

Very interesting.

Could you detail which information you use for building the edges?

Cheers,
Doru



On Wed, Sep 9, 2015 at 11:33 AM, Christophe Demarey <[hidden email]> wrote:
Hi,

We now have a best overview of bootstrap dependencies.
With Max, we reused Dijkstra algo in Moose to help finding dependencies leading to Morphic and co.
It is now possible to have a visualization that is usable (there is also a button to open it in the report):  https://ci.inria.fr/pharo/job/Pharo-5.0-DependencyAnalysis/Dependencies_HTML_Report/bootstrap-dependency-report-graph.html

Next steps are to cut dependencies in red in the report table.

Christophe.

ps: by the way the dependency analyzer (with fixes asked) will be introduced soon in the image.

<Capture d’écran 2015-09-09 à 11.33.02.png>

Le 4 sept. 2015 à 15:13, Christophe Demarey a écrit :

Hi,

I created a job to automatically generate a report of the Pharo image dependencies as well as current bootstrap dependencies.
You can find them here:
There are a lot (too many) dependencies but you can focus on some of them with the search field.
Dependencies inside orange boxes are dependencies that are there but that we ignore (for example, an example method adds an extra-dependency). It should be fixed later but is not a problem for now. Ignored dependencies are declared in the package manifest in #ignoredDependencies.
Dependencies inside red boxes are dependencies we want to remove for the bootstrap. For now, they are listed in the bootstrap dependency report.

People that are willing to help to have a more modular Pharo could first take a look at the bootstrap dependency report and check dependencies that we should cut.
Once we know, we can open issues on fogbugz in the 'bootstrap category' and update manifests accordingly.
To investigate more deeply dependencies, you can use the DependencyAnalyzer (in the Pharo Catalog) to actually check dependencies (inheritance, reference, extension, traits, message send).

Regards,
Christophe.




--

"Every thing has its own flow"




--

"Every thing has its own flow"
12
Free forum by Nabble Edit this page