Discussing the roadmap

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Discussing the roadmap

Stephane Ducasse-3
We would like to share this list with you and get your feedback and inputs.
It will be presented and discussed again at ESUG.

Stef on the behalf of the engineering team of the Pharo consortium.

# Pharo 7 (and 8) potential roadmap

This document contains a list of actions that should be done during
Pharo 7 and 8.
It is not a definitive list. It is the result of a discussion within
the engineer team and RModers.
It first lists actions that should be performed at the image level
then lists the actions for the VM.


## Image
As a general principle, we will try to remove something when we add a
new features or version of a component.

### New tools
- Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
For the moment it supports Git. This tool will become central to the
development of projects in Pharo. The key and first enhancements will
be:
  - cherry picking
  - multiple directories support, subtrees
  - better new development process support

- Calypso: Calypso is a new tool suite for editing and navigating
code. It is modular and can easily be extended.
  - integration, set as the default browser

- Hermes (binary loader): Hermes is a code loader that does not
require the compiler to be loaded. It will be used to speed up the
bootstrap.

- Beacon: Beacon is an announcement-based logger. It should be
introduced but in addition the left over of previous logging should be
cleaned and removed for example many of the transcript show: should be
removed.

- Inspector refreshing: the inspector should refresh its values by default.

- Cargo: Cargo is a new package manager. It supports the expression of
dependencies between packages. We are currently validating that it can
support conditional loading (platform) and building new tooling to
express and validate data.

- Check dependencies when committing. Pharo comes with a dependency
analyser tools. Such tools should be used before commiting to warn the
user when a new dependency is introduced in a package.

### Cleaning bloat
- Removing of Nautilus: Once Calypso will be integrated and exhibit
similar fetaures than Nautilus. Nautilus should be removed.
- Remove old text editor: there is only one or two widgets still using
the old text editor and the rubric text editor.
- Remove TxText: TxText was an attempt to get a new generation text
editor. Now it has been rewritten with a new design in Bloc so we
should remove it since Bloc editor is better and actively maintained.
- Remove Komitter: Iceberg already supports cherrypicking on commit
therefore Komitter can be safely removed from Pharo.
- Remove system categorizer: The old system categorizer is not used
anymore and should be remove.

- Old compiler removal: The old compiler should now get unloaded from Pharo.
  - The old compiler should be moved to an external package and make
sure that it can be reloaded.
  - In addition the encoders should be separated. (@@ more here@@)

- Old inspector cleanup: we should remove the eyeInspector framework
but we should introduce a minimal fallback inspector. This inspector
should work without Spec or any frameworks.

- Clean behavior protocol. The number of methods in the core Behavior,
ClassDescription and Class requires some analysis and cleaning.

- Kernel modularization: the effort on modularising the packages
should be continued.

### Language infrastructure
The following points are more related to the infrastructure of
manipulating and loading class definitions. There are the basis for
the future module system and cleaning some often hidden parts of the
system.

- New class definition: The class definition is not scaling anymore
due to the large number of options (traits, slot, tage, package,
immediate, ephemeron). A fluid-based message API should be designed.

- Support for Undefined classes: When loading old code or code whose
superclass has not yet being loaded, the system has inconsistent
behavior. Depending on the tools, it may not load the code, raise a
warning or decide that the superclass is Object without any other
notice and losing the name of the original superclass. We are working
on a new mechanism to support a unique way to handle such case. To be
presented at IWST/ESUG

- Class definition parser: Class definition is parser in different
place of the system and in addition the output is the direct creation
of a class object instead of an object representing the class
definition that can be further manipulated. We are working on class
definition parser. It produces a separate AST. It will help the
building of tools.

- Better update infrastructure. Pablo Tesone has been working on a
better update mechanism, better modular class builder.

- Ring2: Ring is a metamodel to manipulate code that is not actively
running in the current system. It is useful to perform analysis (such
as browsing, navigating off-line or remote definitions) before
actually loading the code. Ring got redesigned by Pavel Krivanek to
support the bootstrap of Pharo 60. The results is Ring2.

- Opal is the Pharo Compiler. It should be enhanced to support handle
better the warning and a general enhancement pass is needed.

### System enhancements

- Commandline enhancements. RMOD is currently improving the
command-line infrastructure and making sure that the system can work
in read-only mode.

- Cleaning streams: the idea is to make sure that the system does not
use the old streams. The idea is to start using the fileStreams and
make sure that the Stream package can be substituated in the future by
other streams with the same API. Therefore hardcoded class names
should be substituted with factory (readSream writeStream). From that
perspective we do not think that it is wise to introduce Xtreams. We
should analyse the API of the current implementation.

- New theme from the beginning. It is really important that each
version of Pharo is identified open the default opening. Pharo should
come up with two default themes: one light and one dark.

- Better themes/palettes support
- Better and nicer Tabs. Tabs design should be revisited.

- OSWindow world rendering: the effort to remove the logic of the
windowing from the VM should be continued. OSWindow should be used
instead.

- Freetype: The current freetype plugin is the source of many bugs and
problem. Thibault Raffaillac used FFI to do direct call to openGL
(@@to verify@@).


- Refactoring 2: Refactoring 2 is an improved version of the
refactorings developed by Gustavos Santos and they should used to
replace the existing one.


## VM

- 64-bits by default: Mac and Linux 64 bits VMs have been working for
over a year and since June 2017 a Windows 64 bits VM has been working.
The plan is to stabilize each part of Pharo that is not working in 64
bits as well as in 32 bits and make the 64 bits Pharo images and VM
the default download for Pharo.

- Headless VM: Ronie Salgado has built a headless VM on his VM branch,
we need to merge and check everything works fine. With the headless
VM, Pharo can be run in true headless mode (not with a hidden window)
and it is required for future VM features (embeddable VM, ...).

- Embeddable VM: The VM should be able to be embedded in external
applications, the application accessing objects through well-defined
APIs.

- Idle VM: The goal is to avoid the active waiting loop consuming
several per cent of cpu time when the Pharo image is not doing
anything.

- Android VM: integration of the Android VM build and tests in the
integration setup.

- Threaded FFI: all FFI calls should be non-blocking (reviving
prototype of Eliot)

- ZeroConf for ARM: The ARM VM should be available from the zeroConf servers

- Better support for large heaps (GC tuning API, incremental GC).
Pharo 64 bit is now able to manage large heap. However better
performance can be proposed by offering better settings for the
different GC zone.

- Integrate various fixes to support better high resolution display.
In Retina mac display Pharo looks blurry. The fix to support Retina
should be integrated in the VM.

### Sista-related
Sista is an optimizing JIT for Pharo. It is the result of multiple
years of developement by Clement Bera and Eliot Miranda. An optimizing
JIT is manipulating code (folding constant, rewriting it) before
compiling it to assembly code.

- New Block Closure implementation by default: Allows one to implement
in the Opal compiler the copying and clean blocks optimisations,
reduce massively the complexity of some parts of the JIT (debugging
API to map machine code pc to bytecode pc for example) and is required
for the Sista support. Some work remains in debugging/IDE support.

- New Bytecode set in production: Eases bytecode decoding (simplifying
the code base of the bytecode to source code decompiler, the debugger,
the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
and is required for the Sista support. Some work remains in debugging
support.

- Read-only objects: They work in Pharo 6, but the in-image support
needs to be improved (fall-back code of primitives, etc.) and some
polishing needs to be done (primitive error code not always correct,
etc.). Used for the support of different project, including GBS.

- Literals immutability (based on read-only objects). In particular an
analysis of the image usage of literal (string mutation) is needed.

- Sista preview: A first version of Sista will be integrated, yielding
1.5x performance boost on most applications, but will be optional and
will require specific constraints (not toying too much with literal
mutability, partial IDE support, etc.).


### Already done

- [DONE] Remove Shoreline reporter.
- [DONE] Kernel should not depend on Traits: This will speed up the
bootstrap and support the modular introduction of alternate traits
implementation currently designed by Pablo Tesone and tested in the
new generation of Moose.

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Pavel Krivanek-3
TxText removal is already done too.

-- Pavel

2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
We would like to share this list with you and get your feedback and inputs.
It will be presented and discussed again at ESUG.

Stef on the behalf of the engineering team of the Pharo consortium.

# Pharo 7 (and 8) potential roadmap

This document contains a list of actions that should be done during
Pharo 7 and 8.
It is not a definitive list. It is the result of a discussion within
the engineer team and RModers.
It first lists actions that should be performed at the image level
then lists the actions for the VM.


## Image
As a general principle, we will try to remove something when we add a
new features or version of a component.

### New tools
- Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
For the moment it supports Git. This tool will become central to the
development of projects in Pharo. The key and first enhancements will
be:
  - cherry picking
  - multiple directories support, subtrees
  - better new development process support

- Calypso: Calypso is a new tool suite for editing and navigating
code. It is modular and can easily be extended.
  - integration, set as the default browser

- Hermes (binary loader): Hermes is a code loader that does not
require the compiler to be loaded. It will be used to speed up the
bootstrap.

- Beacon: Beacon is an announcement-based logger. It should be
introduced but in addition the left over of previous logging should be
cleaned and removed for example many of the transcript show: should be
removed.

- Inspector refreshing: the inspector should refresh its values by default.

- Cargo: Cargo is a new package manager. It supports the expression of
dependencies between packages. We are currently validating that it can
support conditional loading (platform) and building new tooling to
express and validate data.

- Check dependencies when committing. Pharo comes with a dependency
analyser tools. Such tools should be used before commiting to warn the
user when a new dependency is introduced in a package.

### Cleaning bloat
- Removing of Nautilus: Once Calypso will be integrated and exhibit
similar fetaures than Nautilus. Nautilus should be removed.
- Remove old text editor: there is only one or two widgets still using
the old text editor and the rubric text editor.
- Remove TxText: TxText was an attempt to get a new generation text
editor. Now it has been rewritten with a new design in Bloc so we
should remove it since Bloc editor is better and actively maintained.
- Remove Komitter: Iceberg already supports cherrypicking on commit
therefore Komitter can be safely removed from Pharo.
- Remove system categorizer: The old system categorizer is not used
anymore and should be remove.

- Old compiler removal: The old compiler should now get unloaded from Pharo.
  - The old compiler should be moved to an external package and make
sure that it can be reloaded.
  - In addition the encoders should be separated. (@@ more here@@)

- Old inspector cleanup: we should remove the eyeInspector framework
but we should introduce a minimal fallback inspector. This inspector
should work without Spec or any frameworks.

- Clean behavior protocol. The number of methods in the core Behavior,
ClassDescription and Class requires some analysis and cleaning.

- Kernel modularization: the effort on modularising the packages
should be continued.

### Language infrastructure
The following points are more related to the infrastructure of
manipulating and loading class definitions. There are the basis for
the future module system and cleaning some often hidden parts of the
system.

- New class definition: The class definition is not scaling anymore
due to the large number of options (traits, slot, tage, package,
immediate, ephemeron). A fluid-based message API should be designed.

- Support for Undefined classes: When loading old code or code whose
superclass has not yet being loaded, the system has inconsistent
behavior. Depending on the tools, it may not load the code, raise a
warning or decide that the superclass is Object without any other
notice and losing the name of the original superclass. We are working
on a new mechanism to support a unique way to handle such case. To be
presented at IWST/ESUG

- Class definition parser: Class definition is parser in different
place of the system and in addition the output is the direct creation
of a class object instead of an object representing the class
definition that can be further manipulated. We are working on class
definition parser. It produces a separate AST. It will help the
building of tools.

- Better update infrastructure. Pablo Tesone has been working on a
better update mechanism, better modular class builder.

- Ring2: Ring is a metamodel to manipulate code that is not actively
running in the current system. It is useful to perform analysis (such
as browsing, navigating off-line or remote definitions) before
actually loading the code. Ring got redesigned by Pavel Krivanek to
support the bootstrap of Pharo 60. The results is Ring2.

- Opal is the Pharo Compiler. It should be enhanced to support handle
better the warning and a general enhancement pass is needed.

### System enhancements

- Commandline enhancements. RMOD is currently improving the
command-line infrastructure and making sure that the system can work
in read-only mode.

- Cleaning streams: the idea is to make sure that the system does not
use the old streams. The idea is to start using the fileStreams and
make sure that the Stream package can be substituated in the future by
other streams with the same API. Therefore hardcoded class names
should be substituted with factory (readSream writeStream). From that
perspective we do not think that it is wise to introduce Xtreams. We
should analyse the API of the current implementation.

- New theme from the beginning. It is really important that each
version of Pharo is identified open the default opening. Pharo should
come up with two default themes: one light and one dark.

- Better themes/palettes support
- Better and nicer Tabs. Tabs design should be revisited.

- OSWindow world rendering: the effort to remove the logic of the
windowing from the VM should be continued. OSWindow should be used
instead.

- Freetype: The current freetype plugin is the source of many bugs and
problem. Thibault Raffaillac used FFI to do direct call to openGL
(@@to verify@@).


- Refactoring 2: Refactoring 2 is an improved version of the
refactorings developed by Gustavos Santos and they should used to
replace the existing one.


## VM

- 64-bits by default: Mac and Linux 64 bits VMs have been working for
over a year and since June 2017 a Windows 64 bits VM has been working.
The plan is to stabilize each part of Pharo that is not working in 64
bits as well as in 32 bits and make the 64 bits Pharo images and VM
the default download for Pharo.

- Headless VM: Ronie Salgado has built a headless VM on his VM branch,
we need to merge and check everything works fine. With the headless
VM, Pharo can be run in true headless mode (not with a hidden window)
and it is required for future VM features (embeddable VM, ...).

- Embeddable VM: The VM should be able to be embedded in external
applications, the application accessing objects through well-defined
APIs.

- Idle VM: The goal is to avoid the active waiting loop consuming
several per cent of cpu time when the Pharo image is not doing
anything.

- Android VM: integration of the Android VM build and tests in the
integration setup.

- Threaded FFI: all FFI calls should be non-blocking (reviving
prototype of Eliot)

- ZeroConf for ARM: The ARM VM should be available from the zeroConf servers

- Better support for large heaps (GC tuning API, incremental GC).
Pharo 64 bit is now able to manage large heap. However better
performance can be proposed by offering better settings for the
different GC zone.

- Integrate various fixes to support better high resolution display.
In Retina mac display Pharo looks blurry. The fix to support Retina
should be integrated in the VM.

### Sista-related
Sista is an optimizing JIT for Pharo. It is the result of multiple
years of developement by Clement Bera and Eliot Miranda. An optimizing
JIT is manipulating code (folding constant, rewriting it) before
compiling it to assembly code.

- New Block Closure implementation by default: Allows one to implement
in the Opal compiler the copying and clean blocks optimisations,
reduce massively the complexity of some parts of the JIT (debugging
API to map machine code pc to bytecode pc for example) and is required
for the Sista support. Some work remains in debugging/IDE support.

- New Bytecode set in production: Eases bytecode decoding (simplifying
the code base of the bytecode to source code decompiler, the debugger,
the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
and is required for the Sista support. Some work remains in debugging
support.

- Read-only objects: They work in Pharo 6, but the in-image support
needs to be improved (fall-back code of primitives, etc.) and some
polishing needs to be done (primitive error code not always correct,
etc.). Used for the support of different project, including GBS.

- Literals immutability (based on read-only objects). In particular an
analysis of the image usage of literal (string mutation) is needed.

- Sista preview: A first version of Sista will be integrated, yielding
1.5x performance boost on most applications, but will be optional and
will require specific constraints (not toying too much with literal
mutability, partial IDE support, etc.).


### Already done

- [DONE] Remove Shoreline reporter.
- [DONE] Kernel should not depend on Traits: This will speed up the
bootstrap and support the modular introduction of alternate traits
implementation currently designed by Pablo Tesone and tested in the
new generation of Moose.


Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Tim Mackinnon
That's an exciting list - that keeps up the momentum of what you guys started (and in fact called out at ESUG 2017 in Lugano - so interesting this is a 10 year milestone). 

I’m pleased to see the Retina item listed as well as building on refactoring and command line support (I’ll bet others see lots of their favourites too).

Only 2 minor items stick out as missing:

1) Continuing to improve keyboard shortcut support (its a lot better, but not quite completed - I really miss some shortcuts, particularly the ability to meta-tab between windows - ALT-Tab only works in some windows, and widen selection in the editor - e.g. CMD-W in IntelliJ to increasing select a word, a statement, a function etc)

2) Keeping the website documentation more up to date (again the pharo.org website is very slick, and a great showcase, however often there are late breaking changes which new users won’t know about unless they trace through the news groups or subscribe to some blogs). If we could also focus on keeping it simple but up to date - that might help.

I also have 1 small query - the references to the text editor, is this making the editor that was demo’d at PharoDays 2107 integrated? As that one was very cool? I don’t know how extensible it is (I’m hoping so - and that it might help making it easier to improve code editing tools such that refactorings can more intelligently understand where your cursor is with regards to the AST node underneath it, or we can add more intellisense options that match some of the things we see in IntelliJ).

Tim

On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:

TxText removal is already done too.

-- Pavel

2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
We would like to share this list with you and get your feedback and inputs.
It will be presented and discussed again at ESUG.

Stef on the behalf of the engineering team of the Pharo consortium.

# Pharo 7 (and 8) potential roadmap

This document contains a list of actions that should be done during
Pharo 7 and 8.
It is not a definitive list. It is the result of a discussion within
the engineer team and RModers.
It first lists actions that should be performed at the image level
then lists the actions for the VM.


## Image
As a general principle, we will try to remove something when we add a
new features or version of a component.

### New tools
- Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
For the moment it supports Git. This tool will become central to the
development of projects in Pharo. The key and first enhancements will
be:
  - cherry picking
  - multiple directories support, subtrees
  - better new development process support

- Calypso: Calypso is a new tool suite for editing and navigating
code. It is modular and can easily be extended.
  - integration, set as the default browser

- Hermes (binary loader): Hermes is a code loader that does not
require the compiler to be loaded. It will be used to speed up the
bootstrap.

- Beacon: Beacon is an announcement-based logger. It should be
introduced but in addition the left over of previous logging should be
cleaned and removed for example many of the transcript show: should be
removed.

- Inspector refreshing: the inspector should refresh its values by default.

- Cargo: Cargo is a new package manager. It supports the expression of
dependencies between packages. We are currently validating that it can
support conditional loading (platform) and building new tooling to
express and validate data.

- Check dependencies when committing. Pharo comes with a dependency
analyser tools. Such tools should be used before commiting to warn the
user when a new dependency is introduced in a package.

### Cleaning bloat
- Removing of Nautilus: Once Calypso will be integrated and exhibit
similar fetaures than Nautilus. Nautilus should be removed.
- Remove old text editor: there is only one or two widgets still using
the old text editor and the rubric text editor.
- Remove TxText: TxText was an attempt to get a new generation text
editor. Now it has been rewritten with a new design in Bloc so we
should remove it since Bloc editor is better and actively maintained.
- Remove Komitter: Iceberg already supports cherrypicking on commit
therefore Komitter can be safely removed from Pharo.
- Remove system categorizer: The old system categorizer is not used
anymore and should be remove.

- Old compiler removal: The old compiler should now get unloaded from Pharo.
  - The old compiler should be moved to an external package and make
sure that it can be reloaded.
  - In addition the encoders should be separated. (@@ more here@@)

- Old inspector cleanup: we should remove the eyeInspector framework
but we should introduce a minimal fallback inspector. This inspector
should work without Spec or any frameworks.

- Clean behavior protocol. The number of methods in the core Behavior,
ClassDescription and Class requires some analysis and cleaning.

- Kernel modularization: the effort on modularising the packages
should be continued.

### Language infrastructure
The following points are more related to the infrastructure of
manipulating and loading class definitions. There are the basis for
the future module system and cleaning some often hidden parts of the
system.

- New class definition: The class definition is not scaling anymore
due to the large number of options (traits, slot, tage, package,
immediate, ephemeron). A fluid-based message API should be designed.

- Support for Undefined classes: When loading old code or code whose
superclass has not yet being loaded, the system has inconsistent
behavior. Depending on the tools, it may not load the code, raise a
warning or decide that the superclass is Object without any other
notice and losing the name of the original superclass. We are working
on a new mechanism to support a unique way to handle such case. To be
presented at IWST/ESUG

- Class definition parser: Class definition is parser in different
place of the system and in addition the output is the direct creation
of a class object instead of an object representing the class
definition that can be further manipulated. We are working on class
definition parser. It produces a separate AST. It will help the
building of tools.

- Better update infrastructure. Pablo Tesone has been working on a
better update mechanism, better modular class builder.

- Ring2: Ring is a metamodel to manipulate code that is not actively
running in the current system. It is useful to perform analysis (such
as browsing, navigating off-line or remote definitions) before
actually loading the code. Ring got redesigned by Pavel Krivanek to
support the bootstrap of Pharo 60. The results is Ring2.

- Opal is the Pharo Compiler. It should be enhanced to support handle
better the warning and a general enhancement pass is needed.

### System enhancements

- Commandline enhancements. RMOD is currently improving the
command-line infrastructure and making sure that the system can work
in read-only mode.

- Cleaning streams: the idea is to make sure that the system does not
use the old streams. The idea is to start using the fileStreams and
make sure that the Stream package can be substituated in the future by
other streams with the same API. Therefore hardcoded class names
should be substituted with factory (readSream writeStream). From that
perspective we do not think that it is wise to introduce Xtreams. We
should analyse the API of the current implementation.

- New theme from the beginning. It is really important that each
version of Pharo is identified open the default opening. Pharo should
come up with two default themes: one light and one dark.

- Better themes/palettes support
- Better and nicer Tabs. Tabs design should be revisited.

- OSWindow world rendering: the effort to remove the logic of the
windowing from the VM should be continued. OSWindow should be used
instead.

- Freetype: The current freetype plugin is the source of many bugs and
problem. Thibault Raffaillac used FFI to do direct call to openGL
(@@to verify@@).


- Refactoring 2: Refactoring 2 is an improved version of the
refactorings developed by Gustavos Santos and they should used to
replace the existing one.


## VM

- 64-bits by default: Mac and Linux 64 bits VMs have been working for
over a year and since June 2017 a Windows 64 bits VM has been working.
The plan is to stabilize each part of Pharo that is not working in 64
bits as well as in 32 bits and make the 64 bits Pharo images and VM
the default download for Pharo.

- Headless VM: Ronie Salgado has built a headless VM on his VM branch,
we need to merge and check everything works fine. With the headless
VM, Pharo can be run in true headless mode (not with a hidden window)
and it is required for future VM features (embeddable VM, ...).

- Embeddable VM: The VM should be able to be embedded in external
applications, the application accessing objects through well-defined
APIs.

- Idle VM: The goal is to avoid the active waiting loop consuming
several per cent of cpu time when the Pharo image is not doing
anything.

- Android VM: integration of the Android VM build and tests in the
integration setup.

- Threaded FFI: all FFI calls should be non-blocking (reviving
prototype of Eliot)

- ZeroConf for ARM: The ARM VM should be available from the zeroConf servers

- Better support for large heaps (GC tuning API, incremental GC).
Pharo 64 bit is now able to manage large heap. However better
performance can be proposed by offering better settings for the
different GC zone.

- Integrate various fixes to support better high resolution display.
In Retina mac display Pharo looks blurry. The fix to support Retina
should be integrated in the VM.

### Sista-related
Sista is an optimizing JIT for Pharo. It is the result of multiple
years of developement by Clement Bera and Eliot Miranda. An optimizing
JIT is manipulating code (folding constant, rewriting it) before
compiling it to assembly code.

- New Block Closure implementation by default: Allows one to implement
in the Opal compiler the copying and clean blocks optimisations,
reduce massively the complexity of some parts of the JIT (debugging
API to map machine code pc to bytecode pc for example) and is required
for the Sista support. Some work remains in debugging/IDE support.

- New Bytecode set in production: Eases bytecode decoding (simplifying
the code base of the bytecode to source code decompiler, the debugger,
the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
and is required for the Sista support. Some work remains in debugging
support.

- Read-only objects: They work in Pharo 6, but the in-image support
needs to be improved (fall-back code of primitives, etc.) and some
polishing needs to be done (primitive error code not always correct,
etc.). Used for the support of different project, including GBS.

- Literals immutability (based on read-only objects). In particular an
analysis of the image usage of literal (string mutation) is needed.

- Sista preview: A first version of Sista will be integrated, yielding
1.5x performance boost on most applications, but will be optional and
will require specific constraints (not toying too much with literal
mutability, partial IDE support, etc.).


### Already done

- [DONE] Remove Shoreline reporter.
- [DONE] Kernel should not depend on Traits: This will speed up the
bootstrap and support the modular introduction of alternate traits
implementation currently designed by Pablo Tesone and tested in the
new generation of Moose.



Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Eliot Miranda-2
In reply to this post by Stephane Ducasse-3
Hi Stef,

> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
>
> We would like to share this list with you and get your feedback and inputs.
> It will be presented and discussed again at ESUG.
>
> Stef on the behalf of the engineering team of the Pharo consortium.
>
> # Pharo 7 (and 8) potential roadmap
>
> This document contains a list of actions that should be done during
> Pharo 7 and 8.
> It is not a definitive list. It is the result of a discussion within
> the engineer team and RModers.
> It first lists actions that should be performed at the image level
> then lists the actions for the VM.
>
>
> ## Image
> As a general principle, we will try to remove something when we add a
> new features or version of a component.
>
> ### New tools
> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
> For the moment it supports Git. This tool will become central to the
> development of projects in Pharo. The key and first enhancements will
> be:
>  - cherry picking
>  - multiple directories support, subtrees
>  - better new development process support
>
> - Calypso: Calypso is a new tool suite for editing and navigating
> code. It is modular and can easily be extended.
>  - integration, set as the default browser
>
> - Hermes (binary loader): Hermes is a code loader that does not
> require the compiler to be loaded. It will be used to speed up the
> bootstrap.
>
> - Beacon: Beacon is an announcement-based logger. It should be
> introduced but in addition the left over of previous logging should be
> cleaned and removed for example many of the transcript show: should be
> removed.
>
> - Inspector refreshing: the inspector should refresh its values by default.
>
> - Cargo: Cargo is a new package manager. It supports the expression of
> dependencies between packages. We are currently validating that it can
> support conditional loading (platform) and building new tooling to
> express and validate data.
>
> - Check dependencies when committing. Pharo comes with a dependency
> analyser tools. Such tools should be used before commiting to warn the
> user when a new dependency is introduced in a package.
>
> ### Cleaning bloat
> - Removing of Nautilus: Once Calypso will be integrated and exhibit
> similar fetaures than Nautilus. Nautilus should be removed.
> - Remove old text editor: there is only one or two widgets still using
> the old text editor and the rubric text editor.
> - Remove TxText: TxText was an attempt to get a new generation text
> editor. Now it has been rewritten with a new design in Bloc so we
> should remove it since Bloc editor is better and actively maintained.
> - Remove Komitter: Iceberg already supports cherrypicking on commit
> therefore Komitter can be safely removed from Pharo.
> - Remove system categorizer: The old system categorizer is not used
> anymore and should be remove.
>
> - Old compiler removal: The old compiler should now get unloaded from Pharo.
>  - The old compiler should be moved to an external package and make
> sure that it can be reloaded.
>  - In addition the encoders should be separated. (@@ more here@@)
>
> - Old inspector cleanup: we should remove the eyeInspector framework
> but we should introduce a minimal fallback inspector. This inspector
> should work without Spec or any frameworks.
>
> - Clean behavior protocol. The number of methods in the core Behavior,
> ClassDescription and Class requires some analysis and cleaning.
>
> - Kernel modularization: the effort on modularising the packages
> should be continued.
>
> ### Language infrastructure
> The following points are more related to the infrastructure of
> manipulating and loading class definitions. There are the basis for
> the future module system and cleaning some often hidden parts of the
> system.
>
> - New class definition: The class definition is not scaling anymore
> due to the large number of options (traits, slot, tage, package,
> immediate, ephemeron). A fluid-based message API should be designed.
>
> - Support for Undefined classes: When loading old code or code whose
> superclass has not yet being loaded, the system has inconsistent
> behavior. Depending on the tools, it may not load the code, raise a
> warning or decide that the superclass is Object without any other
> notice and losing the name of the original superclass. We are working
> on a new mechanism to support a unique way to handle such case. To be
> presented at IWST/ESUG
>
> - Class definition parser: Class definition is parser in different
> place of the system and in addition the output is the direct creation
> of a class object instead of an object representing the class
> definition that can be further manipulated. We are working on class
> definition parser. It produces a separate AST. It will help the
> building of tools.
>
> - Better update infrastructure. Pablo Tesone has been working on a
> better update mechanism, better modular class builder.
>
> - Ring2: Ring is a metamodel to manipulate code that is not actively
> running in the current system. It is useful to perform analysis (such
> as browsing, navigating off-line or remote definitions) before
> actually loading the code. Ring got redesigned by Pavel Krivanek to
> support the bootstrap of Pharo 60. The results is Ring2.
>
> - Opal is the Pharo Compiler. It should be enhanced to support handle
> better the warning and a general enhancement pass is needed.
>
> ### System enhancements
>
> - Commandline enhancements. RMOD is currently improving the
> command-line infrastructure and making sure that the system can work
> in read-only mode.
>
> - Cleaning streams: the idea is to make sure that the system does not
> use the old streams. The idea is to start using the fileStreams and
> make sure that the Stream package can be substituated in the future by
> other streams with the same API. Therefore hardcoded class names
> should be substituted with factory (readSream writeStream). From that
> perspective we do not think that it is wise to introduce Xtreams. We
> should analyse the API of the current implementation.
>
> - New theme from the beginning. It is really important that each
> version of Pharo is identified open the default opening. Pharo should
> come up with two default themes: one light and one dark.
>
> - Better themes/palettes support
> - Better and nicer Tabs. Tabs design should be revisited.
>
> - OSWindow world rendering: the effort to remove the logic of the
> windowing from the VM should be continued. OSWindow should be used
> instead.
>
> - Freetype: The current freetype plugin is the source of many bugs and
> problem. Thibault Raffaillac used FFI to do direct call to openGL
> (@@to verify@@).
>
>
> - Refactoring 2: Refactoring 2 is an improved version of the
> refactorings developed by Gustavos Santos and they should used to
> replace the existing one.
>
>
> ## VM
>
> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
> over a year and since June 2017 a Windows 64 bits VM has been working.
> The plan is to stabilize each part of Pharo that is not working in 64
> bits as well as in 32 bits and make the 64 bits Pharo images and VM
> the default download for Pharo.
>
> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
> we need to merge and check everything works fine. With the headless
> VM, Pharo can be run in true headless mode (not with a hidden window)
> and it is required for future VM features (embeddable VM, ...).
>
> - Embeddable VM: The VM should be able to be embedded in external
> applications, the application accessing objects through well-defined
> APIs.
>
> - Idle VM: The goal is to avoid the active waiting loop consuming
> several per cent of cpu time when the Pharo image is not doing
> anything.
>
> - Android VM: integration of the Android VM build and tests in the
> integration setup.
>
> - Threaded FFI: all FFI calls should be non-blocking (reviving
> prototype of Eliot)
>
> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
>
> - Better support for large heaps (GC tuning API, incremental GC).
> Pharo 64 bit is now able to manage large heap. However better
> performance can be proposed by offering better settings for the
> different GC zone.

The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.

Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.

The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.

[Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]


> - Integrate various fixes to support better high resolution display.
> In Retina mac display Pharo looks blurry. The fix to support Retina
> should be integrated in the VM.
>
> ### Sista-related
> Sista is an optimizing JIT for Pharo. It is the result of multiple
> years of developement by Clement Bera and Eliot Miranda. An optimizing
> JIT is manipulating code (folding constant, rewriting it) before
> compiling it to assembly code.

Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.

> - New Block Closure implementation by default: Allows one to implement
> in the Opal compiler the copying and clean blocks optimisations,
> reduce massively the complexity of some parts of the JIT (debugging
> API to map machine code pc to bytecode pc for example) and is required
> for the Sista support. Some work remains in debugging/IDE support.
>
> - New Bytecode set in production: Eases bytecode decoding (simplifying
> the code base of the bytecode to source code decompiler, the debugger,
> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
> and is required for the Sista support. Some work remains in debugging
> support.
>
> - Read-only objects: They work in Pharo 6, but the in-image support
> needs to be improved (fall-back code of primitives, etc.) and some
> polishing needs to be done (primitive error code not always correct,
> etc.). Used for the support of different project, including GBS.

> - Literals immutability (based on read-only objects). In particular an
> analysis of the image usage of literal (string mutation) is needed.

> - Sista preview: A first version of Sista will be integrated, yielding
> 1.5x performance boost on most applications, but will be optional and
> will require specific constraints (not toying too much with literal
> mutability, partial IDE support, etc.).
>
>
> ### Already done
>
> - [DONE] Remove Shoreline reporter.
> - [DONE] Kernel should not depend on Traits: This will speed up the
> bootstrap and support the modular introduction of alternate traits
> implementation currently designed by Pablo Tesone and tested in the
> new generation of Moose.

Eliot
_,,,^..^,,,_ (phone)
Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Sven Van Caekenberghe-2

> On 6 Jul 2017, at 18:41, Eliot Miranda <[hidden email]> wrote:
>
> Hi Stef,
>
>> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
>>
>> We would like to share this list with you and get your feedback and inputs.
>> It will be presented and discussed again at ESUG.
>>
>> Stef on the behalf of the engineering team of the Pharo consortium.
>>
>> # Pharo 7 (and 8) potential roadmap
>>
>> This document contains a list of actions that should be done during
>> Pharo 7 and 8.
>> It is not a definitive list. It is the result of a discussion within
>> the engineer team and RModers.
>> It first lists actions that should be performed at the image level
>> then lists the actions for the VM.
>>
>>
>> ## Image
>> As a general principle, we will try to remove something when we add a
>> new features or version of a component.
>>
>> ### New tools
>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>> For the moment it supports Git. This tool will become central to the
>> development of projects in Pharo. The key and first enhancements will
>> be:
>> - cherry picking
>> - multiple directories support, subtrees
>> - better new development process support
>>
>> - Calypso: Calypso is a new tool suite for editing and navigating
>> code. It is modular and can easily be extended.
>> - integration, set as the default browser
>>
>> - Hermes (binary loader): Hermes is a code loader that does not
>> require the compiler to be loaded. It will be used to speed up the
>> bootstrap.
>>
>> - Beacon: Beacon is an announcement-based logger. It should be
>> introduced but in addition the left over of previous logging should be
>> cleaned and removed for example many of the transcript show: should be
>> removed.
>>
>> - Inspector refreshing: the inspector should refresh its values by default.
>>
>> - Cargo: Cargo is a new package manager. It supports the expression of
>> dependencies between packages. We are currently validating that it can
>> support conditional loading (platform) and building new tooling to
>> express and validate data.
>>
>> - Check dependencies when committing. Pharo comes with a dependency
>> analyser tools. Such tools should be used before commiting to warn the
>> user when a new dependency is introduced in a package.
>>
>> ### Cleaning bloat
>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>> similar fetaures than Nautilus. Nautilus should be removed.
>> - Remove old text editor: there is only one or two widgets still using
>> the old text editor and the rubric text editor.
>> - Remove TxText: TxText was an attempt to get a new generation text
>> editor. Now it has been rewritten with a new design in Bloc so we
>> should remove it since Bloc editor is better and actively maintained.
>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>> therefore Komitter can be safely removed from Pharo.
>> - Remove system categorizer: The old system categorizer is not used
>> anymore and should be remove.
>>
>> - Old compiler removal: The old compiler should now get unloaded from Pharo.
>> - The old compiler should be moved to an external package and make
>> sure that it can be reloaded.
>> - In addition the encoders should be separated. (@@ more here@@)
>>
>> - Old inspector cleanup: we should remove the eyeInspector framework
>> but we should introduce a minimal fallback inspector. This inspector
>> should work without Spec or any frameworks.
>>
>> - Clean behavior protocol. The number of methods in the core Behavior,
>> ClassDescription and Class requires some analysis and cleaning.
>>
>> - Kernel modularization: the effort on modularising the packages
>> should be continued.
>>
>> ### Language infrastructure
>> The following points are more related to the infrastructure of
>> manipulating and loading class definitions. There are the basis for
>> the future module system and cleaning some often hidden parts of the
>> system.
>>
>> - New class definition: The class definition is not scaling anymore
>> due to the large number of options (traits, slot, tage, package,
>> immediate, ephemeron). A fluid-based message API should be designed.
>>
>> - Support for Undefined classes: When loading old code or code whose
>> superclass has not yet being loaded, the system has inconsistent
>> behavior. Depending on the tools, it may not load the code, raise a
>> warning or decide that the superclass is Object without any other
>> notice and losing the name of the original superclass. We are working
>> on a new mechanism to support a unique way to handle such case. To be
>> presented at IWST/ESUG
>>
>> - Class definition parser: Class definition is parser in different
>> place of the system and in addition the output is the direct creation
>> of a class object instead of an object representing the class
>> definition that can be further manipulated. We are working on class
>> definition parser. It produces a separate AST. It will help the
>> building of tools.
>>
>> - Better update infrastructure. Pablo Tesone has been working on a
>> better update mechanism, better modular class builder.
>>
>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>> running in the current system. It is useful to perform analysis (such
>> as browsing, navigating off-line or remote definitions) before
>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>> support the bootstrap of Pharo 60. The results is Ring2.
>>
>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>> better the warning and a general enhancement pass is needed.
>>
>> ### System enhancements
>>
>> - Commandline enhancements. RMOD is currently improving the
>> command-line infrastructure and making sure that the system can work
>> in read-only mode.
>>
>> - Cleaning streams: the idea is to make sure that the system does not
>> use the old streams. The idea is to start using the fileStreams and
>> make sure that the Stream package can be substituated in the future by
>> other streams with the same API. Therefore hardcoded class names
>> should be substituted with factory (readSream writeStream). From that
>> perspective we do not think that it is wise to introduce Xtreams. We
>> should analyse the API of the current implementation.
>>
>> - New theme from the beginning. It is really important that each
>> version of Pharo is identified open the default opening. Pharo should
>> come up with two default themes: one light and one dark.
>>
>> - Better themes/palettes support
>> - Better and nicer Tabs. Tabs design should be revisited.
>>
>> - OSWindow world rendering: the effort to remove the logic of the
>> windowing from the VM should be continued. OSWindow should be used
>> instead.
>>
>> - Freetype: The current freetype plugin is the source of many bugs and
>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>> (@@to verify@@).
>>
>>
>> - Refactoring 2: Refactoring 2 is an improved version of the
>> refactorings developed by Gustavos Santos and they should used to
>> replace the existing one.
>>
>>
>> ## VM
>>
>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>> over a year and since June 2017 a Windows 64 bits VM has been working.
>> The plan is to stabilize each part of Pharo that is not working in 64
>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>> the default download for Pharo.
>>
>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>> we need to merge and check everything works fine. With the headless
>> VM, Pharo can be run in true headless mode (not with a hidden window)
>> and it is required for future VM features (embeddable VM, ...).
>>
>> - Embeddable VM: The VM should be able to be embedded in external
>> applications, the application accessing objects through well-defined
>> APIs.
>>
>> - Idle VM: The goal is to avoid the active waiting loop consuming
>> several per cent of cpu time when the Pharo image is not doing
>> anything.
>>
>> - Android VM: integration of the Android VM build and tests in the
>> integration setup.
>>
>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>> prototype of Eliot)
>>
>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
>>
>> - Better support for large heaps (GC tuning API, incremental GC).
>> Pharo 64 bit is now able to manage large heap. However better
>> performance can be proposed by offering better settings for the
>> different GC zone.
>
> The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.
>
> Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.
>
> The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.
>
> [Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]

Actually, being able to efficiently keep a couple of GB worth of objects in memory, even better 10s of GB seems like an important short term goal. I for one don't really care for games or music. I do care about server software and there we need to use the memory that is available so cheaply today. Luckily we now have 64-bit support.

>> - Integrate various fixes to support better high resolution display.
>> In Retina mac display Pharo looks blurry. The fix to support Retina
>> should be integrated in the VM.
>>
>> ### Sista-related
>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>> JIT is manipulating code (folding constant, rewriting it) before
>> compiling it to assembly code.
>
> Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.
>
>> - New Block Closure implementation by default: Allows one to implement
>> in the Opal compiler the copying and clean blocks optimisations,
>> reduce massively the complexity of some parts of the JIT (debugging
>> API to map machine code pc to bytecode pc for example) and is required
>> for the Sista support. Some work remains in debugging/IDE support.
>>
>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>> the code base of the bytecode to source code decompiler, the debugger,
>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>> and is required for the Sista support. Some work remains in debugging
>> support.
>>
>> - Read-only objects: They work in Pharo 6, but the in-image support
>> needs to be improved (fall-back code of primitives, etc.) and some
>> polishing needs to be done (primitive error code not always correct,
>> etc.). Used for the support of different project, including GBS.
>
>> - Literals immutability (based on read-only objects). In particular an
>> analysis of the image usage of literal (string mutation) is needed.
>
>> - Sista preview: A first version of Sista will be integrated, yielding
>> 1.5x performance boost on most applications, but will be optional and
>> will require specific constraints (not toying too much with literal
>> mutability, partial IDE support, etc.).
>>
>>
>> ### Already done
>>
>> - [DONE] Remove Shoreline reporter.
>> - [DONE] Kernel should not depend on Traits: This will speed up the
>> bootstrap and support the modular introduction of alternate traits
>> implementation currently designed by Pablo Tesone and tested in the
>> new generation of Moose.
>
> Eliot
> _,,,^..^,,,_ (phone)


Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Stephane Ducasse-3
In reply to this post by Eliot Miranda-2
Tx eliot
Clement edited the text so I thought that this is correct and normally
yes I know these details. Just that I wrote all the comments
for all the points and after 1 hour it gets boring :)

On Thu, Jul 6, 2017 at 6:41 PM, Eliot Miranda <[hidden email]> wrote:

> Hi Stef,
>
>> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
>>
>> We would like to share this list with you and get your feedback and inputs.
>> It will be presented and discussed again at ESUG.
>>
>> Stef on the behalf of the engineering team of the Pharo consortium.
>>
>> # Pharo 7 (and 8) potential roadmap
>>
>> This document contains a list of actions that should be done during
>> Pharo 7 and 8.
>> It is not a definitive list. It is the result of a discussion within
>> the engineer team and RModers.
>> It first lists actions that should be performed at the image level
>> then lists the actions for the VM.
>>
>>
>> ## Image
>> As a general principle, we will try to remove something when we add a
>> new features or version of a component.
>>
>> ### New tools
>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>> For the moment it supports Git. This tool will become central to the
>> development of projects in Pharo. The key and first enhancements will
>> be:
>>  - cherry picking
>>  - multiple directories support, subtrees
>>  - better new development process support
>>
>> - Calypso: Calypso is a new tool suite for editing and navigating
>> code. It is modular and can easily be extended.
>>  - integration, set as the default browser
>>
>> - Hermes (binary loader): Hermes is a code loader that does not
>> require the compiler to be loaded. It will be used to speed up the
>> bootstrap.
>>
>> - Beacon: Beacon is an announcement-based logger. It should be
>> introduced but in addition the left over of previous logging should be
>> cleaned and removed for example many of the transcript show: should be
>> removed.
>>
>> - Inspector refreshing: the inspector should refresh its values by default.
>>
>> - Cargo: Cargo is a new package manager. It supports the expression of
>> dependencies between packages. We are currently validating that it can
>> support conditional loading (platform) and building new tooling to
>> express and validate data.
>>
>> - Check dependencies when committing. Pharo comes with a dependency
>> analyser tools. Such tools should be used before commiting to warn the
>> user when a new dependency is introduced in a package.
>>
>> ### Cleaning bloat
>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>> similar fetaures than Nautilus. Nautilus should be removed.
>> - Remove old text editor: there is only one or two widgets still using
>> the old text editor and the rubric text editor.
>> - Remove TxText: TxText was an attempt to get a new generation text
>> editor. Now it has been rewritten with a new design in Bloc so we
>> should remove it since Bloc editor is better and actively maintained.
>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>> therefore Komitter can be safely removed from Pharo.
>> - Remove system categorizer: The old system categorizer is not used
>> anymore and should be remove.
>>
>> - Old compiler removal: The old compiler should now get unloaded from Pharo.
>>  - The old compiler should be moved to an external package and make
>> sure that it can be reloaded.
>>  - In addition the encoders should be separated. (@@ more here@@)
>>
>> - Old inspector cleanup: we should remove the eyeInspector framework
>> but we should introduce a minimal fallback inspector. This inspector
>> should work without Spec or any frameworks.
>>
>> - Clean behavior protocol. The number of methods in the core Behavior,
>> ClassDescription and Class requires some analysis and cleaning.
>>
>> - Kernel modularization: the effort on modularising the packages
>> should be continued.
>>
>> ### Language infrastructure
>> The following points are more related to the infrastructure of
>> manipulating and loading class definitions. There are the basis for
>> the future module system and cleaning some often hidden parts of the
>> system.
>>
>> - New class definition: The class definition is not scaling anymore
>> due to the large number of options (traits, slot, tage, package,
>> immediate, ephemeron). A fluid-based message API should be designed.
>>
>> - Support for Undefined classes: When loading old code or code whose
>> superclass has not yet being loaded, the system has inconsistent
>> behavior. Depending on the tools, it may not load the code, raise a
>> warning or decide that the superclass is Object without any other
>> notice and losing the name of the original superclass. We are working
>> on a new mechanism to support a unique way to handle such case. To be
>> presented at IWST/ESUG
>>
>> - Class definition parser: Class definition is parser in different
>> place of the system and in addition the output is the direct creation
>> of a class object instead of an object representing the class
>> definition that can be further manipulated. We are working on class
>> definition parser. It produces a separate AST. It will help the
>> building of tools.
>>
>> - Better update infrastructure. Pablo Tesone has been working on a
>> better update mechanism, better modular class builder.
>>
>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>> running in the current system. It is useful to perform analysis (such
>> as browsing, navigating off-line or remote definitions) before
>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>> support the bootstrap of Pharo 60. The results is Ring2.
>>
>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>> better the warning and a general enhancement pass is needed.
>>
>> ### System enhancements
>>
>> - Commandline enhancements. RMOD is currently improving the
>> command-line infrastructure and making sure that the system can work
>> in read-only mode.
>>
>> - Cleaning streams: the idea is to make sure that the system does not
>> use the old streams. The idea is to start using the fileStreams and
>> make sure that the Stream package can be substituated in the future by
>> other streams with the same API. Therefore hardcoded class names
>> should be substituted with factory (readSream writeStream). From that
>> perspective we do not think that it is wise to introduce Xtreams. We
>> should analyse the API of the current implementation.
>>
>> - New theme from the beginning. It is really important that each
>> version of Pharo is identified open the default opening. Pharo should
>> come up with two default themes: one light and one dark.
>>
>> - Better themes/palettes support
>> - Better and nicer Tabs. Tabs design should be revisited.
>>
>> - OSWindow world rendering: the effort to remove the logic of the
>> windowing from the VM should be continued. OSWindow should be used
>> instead.
>>
>> - Freetype: The current freetype plugin is the source of many bugs and
>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>> (@@to verify@@).
>>
>>
>> - Refactoring 2: Refactoring 2 is an improved version of the
>> refactorings developed by Gustavos Santos and they should used to
>> replace the existing one.
>>
>>
>> ## VM
>>
>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>> over a year and since June 2017 a Windows 64 bits VM has been working.
>> The plan is to stabilize each part of Pharo that is not working in 64
>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>> the default download for Pharo.
>>
>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>> we need to merge and check everything works fine. With the headless
>> VM, Pharo can be run in true headless mode (not with a hidden window)
>> and it is required for future VM features (embeddable VM, ...).
>>
>> - Embeddable VM: The VM should be able to be embedded in external
>> applications, the application accessing objects through well-defined
>> APIs.
>>
>> - Idle VM: The goal is to avoid the active waiting loop consuming
>> several per cent of cpu time when the Pharo image is not doing
>> anything.
>>
>> - Android VM: integration of the Android VM build and tests in the
>> integration setup.
>>
>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>> prototype of Eliot)
>>
>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
>>
>> - Better support for large heaps (GC tuning API, incremental GC).
>> Pharo 64 bit is now able to manage large heap. However better
>> performance can be proposed by offering better settings for the
>> different GC zone.
>
> The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.
>
> Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.
>
> The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.
>
> [Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]
>
>
>> - Integrate various fixes to support better high resolution display.
>> In Retina mac display Pharo looks blurry. The fix to support Retina
>> should be integrated in the VM.
>>
>> ### Sista-related
>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>> JIT is manipulating code (folding constant, rewriting it) before
>> compiling it to assembly code.
>
> Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.
>
>> - New Block Closure implementation by default: Allows one to implement
>> in the Opal compiler the copying and clean blocks optimisations,
>> reduce massively the complexity of some parts of the JIT (debugging
>> API to map machine code pc to bytecode pc for example) and is required
>> for the Sista support. Some work remains in debugging/IDE support.
>>
>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>> the code base of the bytecode to source code decompiler, the debugger,
>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>> and is required for the Sista support. Some work remains in debugging
>> support.
>>
>> - Read-only objects: They work in Pharo 6, but the in-image support
>> needs to be improved (fall-back code of primitives, etc.) and some
>> polishing needs to be done (primitive error code not always correct,
>> etc.). Used for the support of different project, including GBS.
>
>> - Literals immutability (based on read-only objects). In particular an
>> analysis of the image usage of literal (string mutation) is needed.
>
>> - Sista preview: A first version of Sista will be integrated, yielding
>> 1.5x performance boost on most applications, but will be optional and
>> will require specific constraints (not toying too much with literal
>> mutability, partial IDE support, etc.).
>>
>>
>> ### Already done
>>
>> - [DONE] Remove Shoreline reporter.
>> - [DONE] Kernel should not depend on Traits: This will speed up the
>> bootstrap and support the modular introduction of alternate traits
>> implementation currently designed by Pablo Tesone and tested in the
>> new generation of Moose.
>
> Eliot
> _,,,^..^,,,_ (phone)

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Stephane Ducasse-3
In reply to this post by Tim Mackinnon
> Only 2 minor items stick out as missing:
>
> 1) Continuing to improve keyboard shortcut support (its a lot better, but
> not quite completed - I really miss some shortcuts, particularly the ability
> to meta-tab between windows - ALT-Tab only works in some windows, and widen
> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
> word, a statement, a function etc)

There are three issues there:
- moving from the vm the keyboard generation (this will be done via
OSWindow and the VM can focus its main job: execution).
- then there is the way we declare/manage shortcuts: this is an
ongoing efforts where everybody can join. We should remove the
hardcoded shortcut and turn them into KeyBindings
- Most of the tools will be throw away when bloc will be integrated.
Now it does not mean that we should not do it.

Now an important point: as a user you should report but you can also
contribute.
This is super difficult for me to fix something that I have no clue
how to reproduce/never encounter.
You see the mental model of a real open-source movement is to share
but also to produce together and the point is
what is the reward model: why should I spend time on pain I do not
feel. Of course we are already doing a lot
of actions that is not focus on our immediate needs and we should
improve the situation but I wanted to share
with you this perspective.


>
> 2) Keeping the website documentation more up to date (again the pharo.org
> website is very slick, and a great showcase, however often there are late
> breaking changes which new users won’t know about unless they trace through
> the news groups or subscribe to some blogs). If we could also focus on
> keeping it simple but up to date - that might help.

Where is it not up to date?
We are always reacting to any broken piece there.
You see producing Pharo by Example 5 was A LOT of work and I thank
again Nicolai Hesse for all his huge efforts.


> I also have 1 small query - the references to the text editor, is this
> making the editor that was demo’d at PharoDays 2107 integrated? As that one
> was very cool?

Yes calypso will replace our old friend Nautilus and we will certainly
need to put a lot of love into calypso to get
it to the right level. Now the basis is sound.
About the node we will revisit smart suggestions and indeed sometimes
there are problems to catch the correct AST nodes.

> I don’t know how extensible it is (I’m hoping so - and that
> it might help making it easier to improve code editing tools such that
> refactorings can more intelligently understand where your cursor is with
> regards to the AST node underneath it, or we can add more intellisense
> options that match some of the things we see in IntelliJ).
>
> Tim
>
> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>
>
> TxText removal is already done too.
>
> -- Pavel
>
> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>
>> We would like to share this list with you and get your feedback and
>> inputs.
>> It will be presented and discussed again at ESUG.
>>
>> Stef on the behalf of the engineering team of the Pharo consortium.
>>
>> # Pharo 7 (and 8) potential roadmap
>>
>> This document contains a list of actions that should be done during
>> Pharo 7 and 8.
>> It is not a definitive list. It is the result of a discussion within
>> the engineer team and RModers.
>> It first lists actions that should be performed at the image level
>> then lists the actions for the VM.
>>
>>
>> ## Image
>> As a general principle, we will try to remove something when we add a
>> new features or version of a component.
>>
>> ### New tools
>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>> For the moment it supports Git. This tool will become central to the
>> development of projects in Pharo. The key and first enhancements will
>> be:
>>   - cherry picking
>>   - multiple directories support, subtrees
>>   - better new development process support
>>
>> - Calypso: Calypso is a new tool suite for editing and navigating
>> code. It is modular and can easily be extended.
>>   - integration, set as the default browser
>>
>> - Hermes (binary loader): Hermes is a code loader that does not
>> require the compiler to be loaded. It will be used to speed up the
>> bootstrap.
>>
>> - Beacon: Beacon is an announcement-based logger. It should be
>> introduced but in addition the left over of previous logging should be
>> cleaned and removed for example many of the transcript show: should be
>> removed.
>>
>> - Inspector refreshing: the inspector should refresh its values by
>> default.
>>
>> - Cargo: Cargo is a new package manager. It supports the expression of
>> dependencies between packages. We are currently validating that it can
>> support conditional loading (platform) and building new tooling to
>> express and validate data.
>>
>> - Check dependencies when committing. Pharo comes with a dependency
>> analyser tools. Such tools should be used before commiting to warn the
>> user when a new dependency is introduced in a package.
>>
>> ### Cleaning bloat
>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>> similar fetaures than Nautilus. Nautilus should be removed.
>> - Remove old text editor: there is only one or two widgets still using
>> the old text editor and the rubric text editor.
>> - Remove TxText: TxText was an attempt to get a new generation text
>> editor. Now it has been rewritten with a new design in Bloc so we
>> should remove it since Bloc editor is better and actively maintained.
>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>> therefore Komitter can be safely removed from Pharo.
>> - Remove system categorizer: The old system categorizer is not used
>> anymore and should be remove.
>>
>> - Old compiler removal: The old compiler should now get unloaded from
>> Pharo.
>>   - The old compiler should be moved to an external package and make
>> sure that it can be reloaded.
>>   - In addition the encoders should be separated. (@@ more here@@)
>>
>> - Old inspector cleanup: we should remove the eyeInspector framework
>> but we should introduce a minimal fallback inspector. This inspector
>> should work without Spec or any frameworks.
>>
>> - Clean behavior protocol. The number of methods in the core Behavior,
>> ClassDescription and Class requires some analysis and cleaning.
>>
>> - Kernel modularization: the effort on modularising the packages
>> should be continued.
>>
>> ### Language infrastructure
>> The following points are more related to the infrastructure of
>> manipulating and loading class definitions. There are the basis for
>> the future module system and cleaning some often hidden parts of the
>> system.
>>
>> - New class definition: The class definition is not scaling anymore
>> due to the large number of options (traits, slot, tage, package,
>> immediate, ephemeron). A fluid-based message API should be designed.
>>
>> - Support for Undefined classes: When loading old code or code whose
>> superclass has not yet being loaded, the system has inconsistent
>> behavior. Depending on the tools, it may not load the code, raise a
>> warning or decide that the superclass is Object without any other
>> notice and losing the name of the original superclass. We are working
>> on a new mechanism to support a unique way to handle such case. To be
>> presented at IWST/ESUG
>>
>> - Class definition parser: Class definition is parser in different
>> place of the system and in addition the output is the direct creation
>> of a class object instead of an object representing the class
>> definition that can be further manipulated. We are working on class
>> definition parser. It produces a separate AST. It will help the
>> building of tools.
>>
>> - Better update infrastructure. Pablo Tesone has been working on a
>> better update mechanism, better modular class builder.
>>
>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>> running in the current system. It is useful to perform analysis (such
>> as browsing, navigating off-line or remote definitions) before
>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>> support the bootstrap of Pharo 60. The results is Ring2.
>>
>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>> better the warning and a general enhancement pass is needed.
>>
>> ### System enhancements
>>
>> - Commandline enhancements. RMOD is currently improving the
>> command-line infrastructure and making sure that the system can work
>> in read-only mode.
>>
>> - Cleaning streams: the idea is to make sure that the system does not
>> use the old streams. The idea is to start using the fileStreams and
>> make sure that the Stream package can be substituated in the future by
>> other streams with the same API. Therefore hardcoded class names
>> should be substituted with factory (readSream writeStream). From that
>> perspective we do not think that it is wise to introduce Xtreams. We
>> should analyse the API of the current implementation.
>>
>> - New theme from the beginning. It is really important that each
>> version of Pharo is identified open the default opening. Pharo should
>> come up with two default themes: one light and one dark.
>>
>> - Better themes/palettes support
>> - Better and nicer Tabs. Tabs design should be revisited.
>>
>> - OSWindow world rendering: the effort to remove the logic of the
>> windowing from the VM should be continued. OSWindow should be used
>> instead.
>>
>> - Freetype: The current freetype plugin is the source of many bugs and
>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>> (@@to verify@@).
>>
>>
>> - Refactoring 2: Refactoring 2 is an improved version of the
>> refactorings developed by Gustavos Santos and they should used to
>> replace the existing one.
>>
>>
>> ## VM
>>
>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>> over a year and since June 2017 a Windows 64 bits VM has been working.
>> The plan is to stabilize each part of Pharo that is not working in 64
>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>> the default download for Pharo.
>>
>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>> we need to merge and check everything works fine. With the headless
>> VM, Pharo can be run in true headless mode (not with a hidden window)
>> and it is required for future VM features (embeddable VM, ...).
>>
>> - Embeddable VM: The VM should be able to be embedded in external
>> applications, the application accessing objects through well-defined
>> APIs.
>>
>> - Idle VM: The goal is to avoid the active waiting loop consuming
>> several per cent of cpu time when the Pharo image is not doing
>> anything.
>>
>> - Android VM: integration of the Android VM build and tests in the
>> integration setup.
>>
>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>> prototype of Eliot)
>>
>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>> servers
>>
>> - Better support for large heaps (GC tuning API, incremental GC).
>> Pharo 64 bit is now able to manage large heap. However better
>> performance can be proposed by offering better settings for the
>> different GC zone.
>>
>> - Integrate various fixes to support better high resolution display.
>> In Retina mac display Pharo looks blurry. The fix to support Retina
>> should be integrated in the VM.
>>
>> ### Sista-related
>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>> JIT is manipulating code (folding constant, rewriting it) before
>> compiling it to assembly code.
>>
>> - New Block Closure implementation by default: Allows one to implement
>> in the Opal compiler the copying and clean blocks optimisations,
>> reduce massively the complexity of some parts of the JIT (debugging
>> API to map machine code pc to bytecode pc for example) and is required
>> for the Sista support. Some work remains in debugging/IDE support.
>>
>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>> the code base of the bytecode to source code decompiler, the debugger,
>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>> and is required for the Sista support. Some work remains in debugging
>> support.
>>
>> - Read-only objects: They work in Pharo 6, but the in-image support
>> needs to be improved (fall-back code of primitives, etc.) and some
>> polishing needs to be done (primitive error code not always correct,
>> etc.). Used for the support of different project, including GBS.
>>
>> - Literals immutability (based on read-only objects). In particular an
>> analysis of the image usage of literal (string mutation) is needed.
>>
>> - Sista preview: A first version of Sista will be integrated, yielding
>> 1.5x performance boost on most applications, but will be optional and
>> will require specific constraints (not toying too much with literal
>> mutability, partial IDE support, etc.).
>>
>>
>> ### Already done
>>
>> - [DONE] Remove Shoreline reporter.
>> - [DONE] Kernel should not depend on Traits: This will speed up the
>> bootstrap and support the modular introduction of alternate traits
>> implementation currently designed by Pablo Tesone and tested in the
>> new generation of Moose.
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Denis Kudriashov
In reply to this post by Sven Van Caekenberghe-2

2017-07-06 19:10 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
Actually, being able to efficiently keep a couple of GB worth of objects in memory, even better 10s of GB seems like an important short term goal. I for one don't really care for games or music. I do care about server software and there we need to use the memory that is available so cheaply today. Luckily we now have 64-bit support.

And non blocking FFI is super important for this purpose
Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Stephane Ducasse-3
In reply to this post by Eliot Miranda-2
I updated the roadmap.

Btw eliot I think that you should sync with clement and esteban.
I'm not the guy taking decision here. We should agree on the key
elements of the roadmap
as well as requirements expressed by people that put money on the table.
Once all the agenda/requirements/analyse is done then the consortium
can take a decision for the
benefit of all. Now about large images, some companies really need that.
How much time do you think that an incremental GC implementation would take?

Stef

On Thu, Jul 6, 2017 at 6:41 PM, Eliot Miranda <[hidden email]> wrote:

> Hi Stef,
>
>> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
>>
>> We would like to share this list with you and get your feedback and inputs.
>> It will be presented and discussed again at ESUG.
>>
>> Stef on the behalf of the engineering team of the Pharo consortium.
>>
>> # Pharo 7 (and 8) potential roadmap
>>
>> This document contains a list of actions that should be done during
>> Pharo 7 and 8.
>> It is not a definitive list. It is the result of a discussion within
>> the engineer team and RModers.
>> It first lists actions that should be performed at the image level
>> then lists the actions for the VM.
>>
>>
>> ## Image
>> As a general principle, we will try to remove something when we add a
>> new features or version of a component.
>>
>> ### New tools
>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>> For the moment it supports Git. This tool will become central to the
>> development of projects in Pharo. The key and first enhancements will
>> be:
>>  - cherry picking
>>  - multiple directories support, subtrees
>>  - better new development process support
>>
>> - Calypso: Calypso is a new tool suite for editing and navigating
>> code. It is modular and can easily be extended.
>>  - integration, set as the default browser
>>
>> - Hermes (binary loader): Hermes is a code loader that does not
>> require the compiler to be loaded. It will be used to speed up the
>> bootstrap.
>>
>> - Beacon: Beacon is an announcement-based logger. It should be
>> introduced but in addition the left over of previous logging should be
>> cleaned and removed for example many of the transcript show: should be
>> removed.
>>
>> - Inspector refreshing: the inspector should refresh its values by default.
>>
>> - Cargo: Cargo is a new package manager. It supports the expression of
>> dependencies between packages. We are currently validating that it can
>> support conditional loading (platform) and building new tooling to
>> express and validate data.
>>
>> - Check dependencies when committing. Pharo comes with a dependency
>> analyser tools. Such tools should be used before commiting to warn the
>> user when a new dependency is introduced in a package.
>>
>> ### Cleaning bloat
>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>> similar fetaures than Nautilus. Nautilus should be removed.
>> - Remove old text editor: there is only one or two widgets still using
>> the old text editor and the rubric text editor.
>> - Remove TxText: TxText was an attempt to get a new generation text
>> editor. Now it has been rewritten with a new design in Bloc so we
>> should remove it since Bloc editor is better and actively maintained.
>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>> therefore Komitter can be safely removed from Pharo.
>> - Remove system categorizer: The old system categorizer is not used
>> anymore and should be remove.
>>
>> - Old compiler removal: The old compiler should now get unloaded from Pharo.
>>  - The old compiler should be moved to an external package and make
>> sure that it can be reloaded.
>>  - In addition the encoders should be separated. (@@ more here@@)
>>
>> - Old inspector cleanup: we should remove the eyeInspector framework
>> but we should introduce a minimal fallback inspector. This inspector
>> should work without Spec or any frameworks.
>>
>> - Clean behavior protocol. The number of methods in the core Behavior,
>> ClassDescription and Class requires some analysis and cleaning.
>>
>> - Kernel modularization: the effort on modularising the packages
>> should be continued.
>>
>> ### Language infrastructure
>> The following points are more related to the infrastructure of
>> manipulating and loading class definitions. There are the basis for
>> the future module system and cleaning some often hidden parts of the
>> system.
>>
>> - New class definition: The class definition is not scaling anymore
>> due to the large number of options (traits, slot, tage, package,
>> immediate, ephemeron). A fluid-based message API should be designed.
>>
>> - Support for Undefined classes: When loading old code or code whose
>> superclass has not yet being loaded, the system has inconsistent
>> behavior. Depending on the tools, it may not load the code, raise a
>> warning or decide that the superclass is Object without any other
>> notice and losing the name of the original superclass. We are working
>> on a new mechanism to support a unique way to handle such case. To be
>> presented at IWST/ESUG
>>
>> - Class definition parser: Class definition is parser in different
>> place of the system and in addition the output is the direct creation
>> of a class object instead of an object representing the class
>> definition that can be further manipulated. We are working on class
>> definition parser. It produces a separate AST. It will help the
>> building of tools.
>>
>> - Better update infrastructure. Pablo Tesone has been working on a
>> better update mechanism, better modular class builder.
>>
>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>> running in the current system. It is useful to perform analysis (such
>> as browsing, navigating off-line or remote definitions) before
>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>> support the bootstrap of Pharo 60. The results is Ring2.
>>
>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>> better the warning and a general enhancement pass is needed.
>>
>> ### System enhancements
>>
>> - Commandline enhancements. RMOD is currently improving the
>> command-line infrastructure and making sure that the system can work
>> in read-only mode.
>>
>> - Cleaning streams: the idea is to make sure that the system does not
>> use the old streams. The idea is to start using the fileStreams and
>> make sure that the Stream package can be substituated in the future by
>> other streams with the same API. Therefore hardcoded class names
>> should be substituted with factory (readSream writeStream). From that
>> perspective we do not think that it is wise to introduce Xtreams. We
>> should analyse the API of the current implementation.
>>
>> - New theme from the beginning. It is really important that each
>> version of Pharo is identified open the default opening. Pharo should
>> come up with two default themes: one light and one dark.
>>
>> - Better themes/palettes support
>> - Better and nicer Tabs. Tabs design should be revisited.
>>
>> - OSWindow world rendering: the effort to remove the logic of the
>> windowing from the VM should be continued. OSWindow should be used
>> instead.
>>
>> - Freetype: The current freetype plugin is the source of many bugs and
>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>> (@@to verify@@).
>>
>>
>> - Refactoring 2: Refactoring 2 is an improved version of the
>> refactorings developed by Gustavos Santos and they should used to
>> replace the existing one.
>>
>>
>> ## VM
>>
>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>> over a year and since June 2017 a Windows 64 bits VM has been working.
>> The plan is to stabilize each part of Pharo that is not working in 64
>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>> the default download for Pharo.
>>
>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>> we need to merge and check everything works fine. With the headless
>> VM, Pharo can be run in true headless mode (not with a hidden window)
>> and it is required for future VM features (embeddable VM, ...).
>>
>> - Embeddable VM: The VM should be able to be embedded in external
>> applications, the application accessing objects through well-defined
>> APIs.
>>
>> - Idle VM: The goal is to avoid the active waiting loop consuming
>> several per cent of cpu time when the Pharo image is not doing
>> anything.
>>
>> - Android VM: integration of the Android VM build and tests in the
>> integration setup.
>>
>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>> prototype of Eliot)
>>
>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
>>
>> - Better support for large heaps (GC tuning API, incremental GC).
>> Pharo 64 bit is now able to manage large heap. However better
>> performance can be proposed by offering better settings for the
>> different GC zone.
>
> The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.
>
> Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.
>
> The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.
>
> [Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]
>
>
>> - Integrate various fixes to support better high resolution display.
>> In Retina mac display Pharo looks blurry. The fix to support Retina
>> should be integrated in the VM.
>>
>> ### Sista-related
>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>> JIT is manipulating code (folding constant, rewriting it) before
>> compiling it to assembly code.
>
> Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.
>
>> - New Block Closure implementation by default: Allows one to implement
>> in the Opal compiler the copying and clean blocks optimisations,
>> reduce massively the complexity of some parts of the JIT (debugging
>> API to map machine code pc to bytecode pc for example) and is required
>> for the Sista support. Some work remains in debugging/IDE support.
>>
>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>> the code base of the bytecode to source code decompiler, the debugger,
>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>> and is required for the Sista support. Some work remains in debugging
>> support.
>>
>> - Read-only objects: They work in Pharo 6, but the in-image support
>> needs to be improved (fall-back code of primitives, etc.) and some
>> polishing needs to be done (primitive error code not always correct,
>> etc.). Used for the support of different project, including GBS.
>
>> - Literals immutability (based on read-only objects). In particular an
>> analysis of the image usage of literal (string mutation) is needed.
>
>> - Sista preview: A first version of Sista will be integrated, yielding
>> 1.5x performance boost on most applications, but will be optional and
>> will require specific constraints (not toying too much with literal
>> mutability, partial IDE support, etc.).
>>
>>
>> ### Already done
>>
>> - [DONE] Remove Shoreline reporter.
>> - [DONE] Kernel should not depend on Traits: This will speed up the
>> bootstrap and support the modular introduction of alternate traits
>> implementation currently designed by Pablo Tesone and tested in the
>> new generation of Moose.
>
> Eliot
> _,,,^..^,,,_ (phone)

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Tim Mackinnon
In reply to this post by Stephane Ducasse-3
I didn’t mean to touch a nerve - and this was why I wrote “minor” points - but you did ask for feedback…

Point noted on giving user feedback - I’d actually like to fix things, but currently its just too hard to submit fixes other than pull requests for documentation that is sitting in git hub (and I have submitted those). I know you are working on a simpler way to contribute (and getting the GitHub process smoothed out is in progress so its not fair to comment on that yet as its early days - and much appreciated.

The Alt-Tab issue in the tracker since 2015 - but I will try and report others and possibly Calypso might have some better keyboard navigation tricks that help.

For website documentation - I’ll take it back, as I went back again just now and everything I could think of was there (a lot in the news section which I hadn’t noticed). I do think the download page is a bit confusing but I know there is work being done on that.

Tim


> On 6 Jul 2017, at 19:11, Stephane Ducasse <[hidden email]> wrote:
>
>> Only 2 minor items stick out as missing:
>>
>> 1) Continuing to improve keyboard shortcut support (its a lot better, but
>> not quite completed - I really miss some shortcuts, particularly the ability
>> to meta-tab between windows - ALT-Tab only works in some windows, and widen
>> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
>> word, a statement, a function etc)
>
> There are three issues there:
> - moving from the vm the keyboard generation (this will be done via
> OSWindow and the VM can focus its main job: execution).
> - then there is the way we declare/manage shortcuts: this is an
> ongoing efforts where everybody can join. We should remove the
> hardcoded shortcut and turn them into KeyBindings
> - Most of the tools will be throw away when bloc will be integrated.
> Now it does not mean that we should not do it.
>
> Now an important point: as a user you should report but you can also
> contribute.
> This is super difficult for me to fix something that I have no clue
> how to reproduce/never encounter.
> You see the mental model of a real open-source movement is to share
> but also to produce together and the point is
> what is the reward model: why should I spend time on pain I do not
> feel. Of course we are already doing a lot
> of actions that is not focus on our immediate needs and we should
> improve the situation but I wanted to share
> with you this perspective.
>
>
>>
>> 2) Keeping the website documentation more up to date (again the pharo.org
>> website is very slick, and a great showcase, however often there are late
>> breaking changes which new users won’t know about unless they trace through
>> the news groups or subscribe to some blogs). If we could also focus on
>> keeping it simple but up to date - that might help.
>
> Where is it not up to date?
> We are always reacting to any broken piece there.
> You see producing Pharo by Example 5 was A LOT of work and I thank
> again Nicolai Hesse for all his huge efforts.
>
>
>> I also have 1 small query - the references to the text editor, is this
>> making the editor that was demo’d at PharoDays 2107 integrated? As that one
>> was very cool?
>
> Yes calypso will replace our old friend Nautilus and we will certainly
> need to put a lot of love into calypso to get
> it to the right level. Now the basis is sound.
> About the node we will revisit smart suggestions and indeed sometimes
> there are problems to catch the correct AST nodes.
>
>> I don’t know how extensible it is (I’m hoping so - and that
>> it might help making it easier to improve code editing tools such that
>> refactorings can more intelligently understand where your cursor is with
>> regards to the AST node underneath it, or we can add more intellisense
>> options that match some of the things we see in IntelliJ).
>>
>> Tim
>>
>> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>>
>>
>> TxText removal is already done too.
>>
>> -- Pavel
>>
>> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>>
>>> We would like to share this list with you and get your feedback and
>>> inputs.
>>> It will be presented and discussed again at ESUG.
>>>
>>> Stef on the behalf of the engineering team of the Pharo consortium.
>>>
>>> # Pharo 7 (and 8) potential roadmap
>>>
>>> This document contains a list of actions that should be done during
>>> Pharo 7 and 8.
>>> It is not a definitive list. It is the result of a discussion within
>>> the engineer team and RModers.
>>> It first lists actions that should be performed at the image level
>>> then lists the actions for the VM.
>>>
>>>
>>> ## Image
>>> As a general principle, we will try to remove something when we add a
>>> new features or version of a component.
>>>
>>> ### New tools
>>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>>> For the moment it supports Git. This tool will become central to the
>>> development of projects in Pharo. The key and first enhancements will
>>> be:
>>>  - cherry picking
>>>  - multiple directories support, subtrees
>>>  - better new development process support
>>>
>>> - Calypso: Calypso is a new tool suite for editing and navigating
>>> code. It is modular and can easily be extended.
>>>  - integration, set as the default browser
>>>
>>> - Hermes (binary loader): Hermes is a code loader that does not
>>> require the compiler to be loaded. It will be used to speed up the
>>> bootstrap.
>>>
>>> - Beacon: Beacon is an announcement-based logger. It should be
>>> introduced but in addition the left over of previous logging should be
>>> cleaned and removed for example many of the transcript show: should be
>>> removed.
>>>
>>> - Inspector refreshing: the inspector should refresh its values by
>>> default.
>>>
>>> - Cargo: Cargo is a new package manager. It supports the expression of
>>> dependencies between packages. We are currently validating that it can
>>> support conditional loading (platform) and building new tooling to
>>> express and validate data.
>>>
>>> - Check dependencies when committing. Pharo comes with a dependency
>>> analyser tools. Such tools should be used before commiting to warn the
>>> user when a new dependency is introduced in a package.
>>>
>>> ### Cleaning bloat
>>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>>> similar fetaures than Nautilus. Nautilus should be removed.
>>> - Remove old text editor: there is only one or two widgets still using
>>> the old text editor and the rubric text editor.
>>> - Remove TxText: TxText was an attempt to get a new generation text
>>> editor. Now it has been rewritten with a new design in Bloc so we
>>> should remove it since Bloc editor is better and actively maintained.
>>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>>> therefore Komitter can be safely removed from Pharo.
>>> - Remove system categorizer: The old system categorizer is not used
>>> anymore and should be remove.
>>>
>>> - Old compiler removal: The old compiler should now get unloaded from
>>> Pharo.
>>>  - The old compiler should be moved to an external package and make
>>> sure that it can be reloaded.
>>>  - In addition the encoders should be separated. (@@ more here@@)
>>>
>>> - Old inspector cleanup: we should remove the eyeInspector framework
>>> but we should introduce a minimal fallback inspector. This inspector
>>> should work without Spec or any frameworks.
>>>
>>> - Clean behavior protocol. The number of methods in the core Behavior,
>>> ClassDescription and Class requires some analysis and cleaning.
>>>
>>> - Kernel modularization: the effort on modularising the packages
>>> should be continued.
>>>
>>> ### Language infrastructure
>>> The following points are more related to the infrastructure of
>>> manipulating and loading class definitions. There are the basis for
>>> the future module system and cleaning some often hidden parts of the
>>> system.
>>>
>>> - New class definition: The class definition is not scaling anymore
>>> due to the large number of options (traits, slot, tage, package,
>>> immediate, ephemeron). A fluid-based message API should be designed.
>>>
>>> - Support for Undefined classes: When loading old code or code whose
>>> superclass has not yet being loaded, the system has inconsistent
>>> behavior. Depending on the tools, it may not load the code, raise a
>>> warning or decide that the superclass is Object without any other
>>> notice and losing the name of the original superclass. We are working
>>> on a new mechanism to support a unique way to handle such case. To be
>>> presented at IWST/ESUG
>>>
>>> - Class definition parser: Class definition is parser in different
>>> place of the system and in addition the output is the direct creation
>>> of a class object instead of an object representing the class
>>> definition that can be further manipulated. We are working on class
>>> definition parser. It produces a separate AST. It will help the
>>> building of tools.
>>>
>>> - Better update infrastructure. Pablo Tesone has been working on a
>>> better update mechanism, better modular class builder.
>>>
>>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>>> running in the current system. It is useful to perform analysis (such
>>> as browsing, navigating off-line or remote definitions) before
>>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>>> support the bootstrap of Pharo 60. The results is Ring2.
>>>
>>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>>> better the warning and a general enhancement pass is needed.
>>>
>>> ### System enhancements
>>>
>>> - Commandline enhancements. RMOD is currently improving the
>>> command-line infrastructure and making sure that the system can work
>>> in read-only mode.
>>>
>>> - Cleaning streams: the idea is to make sure that the system does not
>>> use the old streams. The idea is to start using the fileStreams and
>>> make sure that the Stream package can be substituated in the future by
>>> other streams with the same API. Therefore hardcoded class names
>>> should be substituted with factory (readSream writeStream). From that
>>> perspective we do not think that it is wise to introduce Xtreams. We
>>> should analyse the API of the current implementation.
>>>
>>> - New theme from the beginning. It is really important that each
>>> version of Pharo is identified open the default opening. Pharo should
>>> come up with two default themes: one light and one dark.
>>>
>>> - Better themes/palettes support
>>> - Better and nicer Tabs. Tabs design should be revisited.
>>>
>>> - OSWindow world rendering: the effort to remove the logic of the
>>> windowing from the VM should be continued. OSWindow should be used
>>> instead.
>>>
>>> - Freetype: The current freetype plugin is the source of many bugs and
>>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>>> (@@to verify@@).
>>>
>>>
>>> - Refactoring 2: Refactoring 2 is an improved version of the
>>> refactorings developed by Gustavos Santos and they should used to
>>> replace the existing one.
>>>
>>>
>>> ## VM
>>>
>>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>>> over a year and since June 2017 a Windows 64 bits VM has been working.
>>> The plan is to stabilize each part of Pharo that is not working in 64
>>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>>> the default download for Pharo.
>>>
>>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>>> we need to merge and check everything works fine. With the headless
>>> VM, Pharo can be run in true headless mode (not with a hidden window)
>>> and it is required for future VM features (embeddable VM, ...).
>>>
>>> - Embeddable VM: The VM should be able to be embedded in external
>>> applications, the application accessing objects through well-defined
>>> APIs.
>>>
>>> - Idle VM: The goal is to avoid the active waiting loop consuming
>>> several per cent of cpu time when the Pharo image is not doing
>>> anything.
>>>
>>> - Android VM: integration of the Android VM build and tests in the
>>> integration setup.
>>>
>>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>>> prototype of Eliot)
>>>
>>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>>> servers
>>>
>>> - Better support for large heaps (GC tuning API, incremental GC).
>>> Pharo 64 bit is now able to manage large heap. However better
>>> performance can be proposed by offering better settings for the
>>> different GC zone.
>>>
>>> - Integrate various fixes to support better high resolution display.
>>> In Retina mac display Pharo looks blurry. The fix to support Retina
>>> should be integrated in the VM.
>>>
>>> ### Sista-related
>>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>>> JIT is manipulating code (folding constant, rewriting it) before
>>> compiling it to assembly code.
>>>
>>> - New Block Closure implementation by default: Allows one to implement
>>> in the Opal compiler the copying and clean blocks optimisations,
>>> reduce massively the complexity of some parts of the JIT (debugging
>>> API to map machine code pc to bytecode pc for example) and is required
>>> for the Sista support. Some work remains in debugging/IDE support.
>>>
>>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>>> the code base of the bytecode to source code decompiler, the debugger,
>>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>>> and is required for the Sista support. Some work remains in debugging
>>> support.
>>>
>>> - Read-only objects: They work in Pharo 6, but the in-image support
>>> needs to be improved (fall-back code of primitives, etc.) and some
>>> polishing needs to be done (primitive error code not always correct,
>>> etc.). Used for the support of different project, including GBS.
>>>
>>> - Literals immutability (based on read-only objects). In particular an
>>> analysis of the image usage of literal (string mutation) is needed.
>>>
>>> - Sista preview: A first version of Sista will be integrated, yielding
>>> 1.5x performance boost on most applications, but will be optional and
>>> will require specific constraints (not toying too much with literal
>>> mutability, partial IDE support, etc.).
>>>
>>>
>>> ### Already done
>>>
>>> - [DONE] Remove Shoreline reporter.
>>> - [DONE] Kernel should not depend on Traits: This will speed up the
>>> bootstrap and support the modular introduction of alternate traits
>>> implementation currently designed by Pablo Tesone and tested in the
>>> new generation of Moose.
>>>
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Eliot Miranda-2
In reply to this post by Denis Kudriashov


On Thu, Jul 6, 2017 at 12:44 PM, Denis Kudriashov <[hidden email]> wrote:

2017-07-06 19:10 GMT+02:00 Sven Van Caekenberghe <[hidden email]>:
Actually, being able to efficiently keep a couple of GB worth of objects in memory, even better 10s of GB seems like an important short term goal. I for one don't really care for games or music. I do care about server software and there we need to use the memory that is available so cheaply today. Luckily we now have 64-bit support.

And non blocking FFI is super important for this purpose

+1.  The non-blocking FFI is a major enabler.  It is a strong priority alongside Scorc/Sista (alongside because Clément and I are approaching releasing it; and good engineers ship). 

_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Eliot Miranda-2
In reply to this post by Stephane Ducasse-3
Hi Stef,

On Thu, Jul 6, 2017 at 1:20 PM, Stephane Ducasse <[hidden email]> wrote:
I updated the roadmap.

Btw eliot I think that you should sync with clement and esteban.

trust me, we do.
 
I'm not the guy taking decision here. We should agree on the key
elements of the roadmap
as well as requirements expressed by people that put money on the table.
Once all the agenda/requirements/analyse is done then the consortium
can take a decision for the
benefit of all. Now about large images, some companies really need that.

Indeed.  See my reply to Sven.
 
How much time do you think that an incremental GC implementation would take?

Three months.
 

Stef

On Thu, Jul 6, 2017 at 6:41 PM, Eliot Miranda <[hidden email]> wrote:
> Hi Stef,
>
>> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
>>
>> We would like to share this list with you and get your feedback and inputs.
>> It will be presented and discussed again at ESUG.
>>
>> Stef on the behalf of the engineering team of the Pharo consortium.
>>
>> # Pharo 7 (and 8) potential roadmap
>>
>> This document contains a list of actions that should be done during
>> Pharo 7 and 8.
>> It is not a definitive list. It is the result of a discussion within
>> the engineer team and RModers.
>> It first lists actions that should be performed at the image level
>> then lists the actions for the VM.
>>
>>
>> ## Image
>> As a general principle, we will try to remove something when we add a
>> new features or version of a component.
>>
>> ### New tools
>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>> For the moment it supports Git. This tool will become central to the
>> development of projects in Pharo. The key and first enhancements will
>> be:
>>  - cherry picking
>>  - multiple directories support, subtrees
>>  - better new development process support
>>
>> - Calypso: Calypso is a new tool suite for editing and navigating
>> code. It is modular and can easily be extended.
>>  - integration, set as the default browser
>>
>> - Hermes (binary loader): Hermes is a code loader that does not
>> require the compiler to be loaded. It will be used to speed up the
>> bootstrap.
>>
>> - Beacon: Beacon is an announcement-based logger. It should be
>> introduced but in addition the left over of previous logging should be
>> cleaned and removed for example many of the transcript show: should be
>> removed.
>>
>> - Inspector refreshing: the inspector should refresh its values by default.
>>
>> - Cargo: Cargo is a new package manager. It supports the expression of
>> dependencies between packages. We are currently validating that it can
>> support conditional loading (platform) and building new tooling to
>> express and validate data.
>>
>> - Check dependencies when committing. Pharo comes with a dependency
>> analyser tools. Such tools should be used before commiting to warn the
>> user when a new dependency is introduced in a package.
>>
>> ### Cleaning bloat
>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>> similar fetaures than Nautilus. Nautilus should be removed.
>> - Remove old text editor: there is only one or two widgets still using
>> the old text editor and the rubric text editor.
>> - Remove TxText: TxText was an attempt to get a new generation text
>> editor. Now it has been rewritten with a new design in Bloc so we
>> should remove it since Bloc editor is better and actively maintained.
>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>> therefore Komitter can be safely removed from Pharo.
>> - Remove system categorizer: The old system categorizer is not used
>> anymore and should be remove.
>>
>> - Old compiler removal: The old compiler should now get unloaded from Pharo.
>>  - The old compiler should be moved to an external package and make
>> sure that it can be reloaded.
>>  - In addition the encoders should be separated. (@@ more here@@)
>>
>> - Old inspector cleanup: we should remove the eyeInspector framework
>> but we should introduce a minimal fallback inspector. This inspector
>> should work without Spec or any frameworks.
>>
>> - Clean behavior protocol. The number of methods in the core Behavior,
>> ClassDescription and Class requires some analysis and cleaning.
>>
>> - Kernel modularization: the effort on modularising the packages
>> should be continued.
>>
>> ### Language infrastructure
>> The following points are more related to the infrastructure of
>> manipulating and loading class definitions. There are the basis for
>> the future module system and cleaning some often hidden parts of the
>> system.
>>
>> - New class definition: The class definition is not scaling anymore
>> due to the large number of options (traits, slot, tage, package,
>> immediate, ephemeron). A fluid-based message API should be designed.
>>
>> - Support for Undefined classes: When loading old code or code whose
>> superclass has not yet being loaded, the system has inconsistent
>> behavior. Depending on the tools, it may not load the code, raise a
>> warning or decide that the superclass is Object without any other
>> notice and losing the name of the original superclass. We are working
>> on a new mechanism to support a unique way to handle such case. To be
>> presented at IWST/ESUG
>>
>> - Class definition parser: Class definition is parser in different
>> place of the system and in addition the output is the direct creation
>> of a class object instead of an object representing the class
>> definition that can be further manipulated. We are working on class
>> definition parser. It produces a separate AST. It will help the
>> building of tools.
>>
>> - Better update infrastructure. Pablo Tesone has been working on a
>> better update mechanism, better modular class builder.
>>
>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>> running in the current system. It is useful to perform analysis (such
>> as browsing, navigating off-line or remote definitions) before
>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>> support the bootstrap of Pharo 60. The results is Ring2.
>>
>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>> better the warning and a general enhancement pass is needed.
>>
>> ### System enhancements
>>
>> - Commandline enhancements. RMOD is currently improving the
>> command-line infrastructure and making sure that the system can work
>> in read-only mode.
>>
>> - Cleaning streams: the idea is to make sure that the system does not
>> use the old streams. The idea is to start using the fileStreams and
>> make sure that the Stream package can be substituated in the future by
>> other streams with the same API. Therefore hardcoded class names
>> should be substituted with factory (readSream writeStream). From that
>> perspective we do not think that it is wise to introduce Xtreams. We
>> should analyse the API of the current implementation.
>>
>> - New theme from the beginning. It is really important that each
>> version of Pharo is identified open the default opening. Pharo should
>> come up with two default themes: one light and one dark.
>>
>> - Better themes/palettes support
>> - Better and nicer Tabs. Tabs design should be revisited.
>>
>> - OSWindow world rendering: the effort to remove the logic of the
>> windowing from the VM should be continued. OSWindow should be used
>> instead.
>>
>> - Freetype: The current freetype plugin is the source of many bugs and
>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>> (@@to verify@@).
>>
>>
>> - Refactoring 2: Refactoring 2 is an improved version of the
>> refactorings developed by Gustavos Santos and they should used to
>> replace the existing one.
>>
>>
>> ## VM
>>
>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>> over a year and since June 2017 a Windows 64 bits VM has been working.
>> The plan is to stabilize each part of Pharo that is not working in 64
>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>> the default download for Pharo.
>>
>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>> we need to merge and check everything works fine. With the headless
>> VM, Pharo can be run in true headless mode (not with a hidden window)
>> and it is required for future VM features (embeddable VM, ...).
>>
>> - Embeddable VM: The VM should be able to be embedded in external
>> applications, the application accessing objects through well-defined
>> APIs.
>>
>> - Idle VM: The goal is to avoid the active waiting loop consuming
>> several per cent of cpu time when the Pharo image is not doing
>> anything.
>>
>> - Android VM: integration of the Android VM build and tests in the
>> integration setup.
>>
>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>> prototype of Eliot)
>>
>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
>>
>> - Better support for large heaps (GC tuning API, incremental GC).
>> Pharo 64 bit is now able to manage large heap. However better
>> performance can be proposed by offering better settings for the
>> different GC zone.
>
> The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.
>
> Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.
>
> The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.
>
> [Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]
>
>
>> - Integrate various fixes to support better high resolution display.
>> In Retina mac display Pharo looks blurry. The fix to support Retina
>> should be integrated in the VM.
>>
>> ### Sista-related
>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>> JIT is manipulating code (folding constant, rewriting it) before
>> compiling it to assembly code.
>
> Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.
>
>> - New Block Closure implementation by default: Allows one to implement
>> in the Opal compiler the copying and clean blocks optimisations,
>> reduce massively the complexity of some parts of the JIT (debugging
>> API to map machine code pc to bytecode pc for example) and is required
>> for the Sista support. Some work remains in debugging/IDE support.
>>
>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>> the code base of the bytecode to source code decompiler, the debugger,
>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>> and is required for the Sista support. Some work remains in debugging
>> support.
>>
>> - Read-only objects: They work in Pharo 6, but the in-image support
>> needs to be improved (fall-back code of primitives, etc.) and some
>> polishing needs to be done (primitive error code not always correct,
>> etc.). Used for the support of different project, including GBS.
>
>> - Literals immutability (based on read-only objects). In particular an
>> analysis of the image usage of literal (string mutation) is needed.
>
>> - Sista preview: A first version of Sista will be integrated, yielding
>> 1.5x performance boost on most applications, but will be optional and
>> will require specific constraints (not toying too much with literal
>> mutability, partial IDE support, etc.).
>>
>>
>> ### Already done
>>
>> - [DONE] Remove Shoreline reporter.
>> - [DONE] Kernel should not depend on Traits: This will speed up the
>> bootstrap and support the modular introduction of alternate traits
>> implementation currently designed by Pablo Tesone and tested in the
>> new generation of Moose.
>
> Eliot
> _,,,^..^,,,_ (phone)




--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Eliot Miranda-2
In reply to this post by Sven Van Caekenberghe-2
Hi Sven,

On Thu, Jul 6, 2017 at 10:10 AM, Sven Van Caekenberghe <[hidden email]> wrote:

> On 6 Jul 2017, at 18:41, Eliot Miranda <[hidden email]> wrote:
>
> Hi Stef,
>
>> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
>>
>> We would like to share this list with you and get your feedback and inputs.
>> It will be presented and discussed again at ESUG.
>>
>> Stef on the behalf of the engineering team of the Pharo consortium.
>>
>> # Pharo 7 (and 8) potential roadmap
>>
>> This document contains a list of actions that should be done during
>> Pharo 7 and 8.
>> It is not a definitive list. It is the result of a discussion within
>> the engineer team and RModers.
>> It first lists actions that should be performed at the image level
>> then lists the actions for the VM.
>>
>>
>> ## Image
>> As a general principle, we will try to remove something when we add a
>> new features or version of a component.
>>
>> ### New tools
>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>> For the moment it supports Git. This tool will become central to the
>> development of projects in Pharo. The key and first enhancements will
>> be:
>> - cherry picking
>> - multiple directories support, subtrees
>> - better new development process support
>>
>> - Calypso: Calypso is a new tool suite for editing and navigating
>> code. It is modular and can easily be extended.
>> - integration, set as the default browser
>>
>> - Hermes (binary loader): Hermes is a code loader that does not
>> require the compiler to be loaded. It will be used to speed up the
>> bootstrap.
>>
>> - Beacon: Beacon is an announcement-based logger. It should be
>> introduced but in addition the left over of previous logging should be
>> cleaned and removed for example many of the transcript show: should be
>> removed.
>>
>> - Inspector refreshing: the inspector should refresh its values by default.
>>
>> - Cargo: Cargo is a new package manager. It supports the expression of
>> dependencies between packages. We are currently validating that it can
>> support conditional loading (platform) and building new tooling to
>> express and validate data.
>>
>> - Check dependencies when committing. Pharo comes with a dependency
>> analyser tools. Such tools should be used before commiting to warn the
>> user when a new dependency is introduced in a package.
>>
>> ### Cleaning bloat
>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>> similar fetaures than Nautilus. Nautilus should be removed.
>> - Remove old text editor: there is only one or two widgets still using
>> the old text editor and the rubric text editor.
>> - Remove TxText: TxText was an attempt to get a new generation text
>> editor. Now it has been rewritten with a new design in Bloc so we
>> should remove it since Bloc editor is better and actively maintained.
>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>> therefore Komitter can be safely removed from Pharo.
>> - Remove system categorizer: The old system categorizer is not used
>> anymore and should be remove.
>>
>> - Old compiler removal: The old compiler should now get unloaded from Pharo.
>> - The old compiler should be moved to an external package and make
>> sure that it can be reloaded.
>> - In addition the encoders should be separated. (@@ more here@@)
>>
>> - Old inspector cleanup: we should remove the eyeInspector framework
>> but we should introduce a minimal fallback inspector. This inspector
>> should work without Spec or any frameworks.
>>
>> - Clean behavior protocol. The number of methods in the core Behavior,
>> ClassDescription and Class requires some analysis and cleaning.
>>
>> - Kernel modularization: the effort on modularising the packages
>> should be continued.
>>
>> ### Language infrastructure
>> The following points are more related to the infrastructure of
>> manipulating and loading class definitions. There are the basis for
>> the future module system and cleaning some often hidden parts of the
>> system.
>>
>> - New class definition: The class definition is not scaling anymore
>> due to the large number of options (traits, slot, tage, package,
>> immediate, ephemeron). A fluid-based message API should be designed.
>>
>> - Support for Undefined classes: When loading old code or code whose
>> superclass has not yet being loaded, the system has inconsistent
>> behavior. Depending on the tools, it may not load the code, raise a
>> warning or decide that the superclass is Object without any other
>> notice and losing the name of the original superclass. We are working
>> on a new mechanism to support a unique way to handle such case. To be
>> presented at IWST/ESUG
>>
>> - Class definition parser: Class definition is parser in different
>> place of the system and in addition the output is the direct creation
>> of a class object instead of an object representing the class
>> definition that can be further manipulated. We are working on class
>> definition parser. It produces a separate AST. It will help the
>> building of tools.
>>
>> - Better update infrastructure. Pablo Tesone has been working on a
>> better update mechanism, better modular class builder.
>>
>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>> running in the current system. It is useful to perform analysis (such
>> as browsing, navigating off-line or remote definitions) before
>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>> support the bootstrap of Pharo 60. The results is Ring2.
>>
>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>> better the warning and a general enhancement pass is needed.
>>
>> ### System enhancements
>>
>> - Commandline enhancements. RMOD is currently improving the
>> command-line infrastructure and making sure that the system can work
>> in read-only mode.
>>
>> - Cleaning streams: the idea is to make sure that the system does not
>> use the old streams. The idea is to start using the fileStreams and
>> make sure that the Stream package can be substituated in the future by
>> other streams with the same API. Therefore hardcoded class names
>> should be substituted with factory (readSream writeStream). From that
>> perspective we do not think that it is wise to introduce Xtreams. We
>> should analyse the API of the current implementation.
>>
>> - New theme from the beginning. It is really important that each
>> version of Pharo is identified open the default opening. Pharo should
>> come up with two default themes: one light and one dark.
>>
>> - Better themes/palettes support
>> - Better and nicer Tabs. Tabs design should be revisited.
>>
>> - OSWindow world rendering: the effort to remove the logic of the
>> windowing from the VM should be continued. OSWindow should be used
>> instead.
>>
>> - Freetype: The current freetype plugin is the source of many bugs and
>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>> (@@to verify@@).
>>
>>
>> - Refactoring 2: Refactoring 2 is an improved version of the
>> refactorings developed by Gustavos Santos and they should used to
>> replace the existing one.
>>
>>
>> ## VM
>>
>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>> over a year and since June 2017 a Windows 64 bits VM has been working.
>> The plan is to stabilize each part of Pharo that is not working in 64
>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>> the default download for Pharo.
>>
>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>> we need to merge and check everything works fine. With the headless
>> VM, Pharo can be run in true headless mode (not with a hidden window)
>> and it is required for future VM features (embeddable VM, ...).
>>
>> - Embeddable VM: The VM should be able to be embedded in external
>> applications, the application accessing objects through well-defined
>> APIs.
>>
>> - Idle VM: The goal is to avoid the active waiting loop consuming
>> several per cent of cpu time when the Pharo image is not doing
>> anything.
>>
>> - Android VM: integration of the Android VM build and tests in the
>> integration setup.
>>
>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>> prototype of Eliot)
>>
>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
>>
>> - Better support for large heaps (GC tuning API, incremental GC).
>> Pharo 64 bit is now able to manage large heap. However better
>> performance can be proposed by offering better settings for the
>> different GC zone.
>
> The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.
>
> Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.
>
> The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.
>
> [Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]

Actually, being able to efficiently keep a couple of GB worth of objects in memory, even better 10s of GB seems like an important short term goal. I for one don't really care for games or music. I do care about server software and there we need to use the memory that is available so cheaply today. Luckily we now have 64-bit support.

Indeed, and the reason I designed Spur to be both 32-bit and 64-bit was because my employer Cadence wanted to scale to much larger models (the application manipulates large models of SystemOnAChip (SOC) components expressed in XML).  But their application is batch and they're happy just with the ability to go somewhat above the approximately 3 Gb limit of the 32-bit system.

But what I was getting at in my comment was that, while 64-bit server applications are indeed very important, I have not been contacted by anyone yet complaining about scaling problems with the 64-bit system.  This could be for a number of reasons, including that there are not many large applications out there.  We know that pause times in the full GC are an issue for animation ad music software.  I didn't articulate it, but I also think that an incremental GC provides somewhat of an answer to scaling for 64-bit servers.

Let's say that currently the mark-compact collector collects at about 1Gb per second (that's about what I'm seeing on my MacMini for a 600Mb heap, collected and compacted in about 530ms on a 2.3GHz Core i7, = 1.1Gb/s).  For the moment let's assume the GC scales linearly.  Then a 10Gb server application will occasionally suffer 10 second pauses, which is unacceptable for any kind of high-availability server.  The total execution time overhead of GC may be in the 1%-10% range, but the pause times may be unacceptable (note that they're /not/ unacceptable for my employer's batch usage).

So were the GC to be extended with an incremental collector we may be able to bring pause times back down to the tens of milliseconds rage or below.  Clearly how the server uses memory, and how the GC scales are crucially important.  Right now I have no data on either; no large 64-bit server applications to analyze and consequently not much idea of how the current GC or an incremental GC will scale.  So for the moment I'm going to prioritize doing an incremental GC, and making it measurable and tunable. Hopefully the incremental GC will benefit both large server applications as well as music and animation.

But nothing will substitute for experience with real applications.  So far there are very few large 64-bit applications running on Cog for me to analyze.  Even Cadence's customers are too protective of their data for me to analyze real applications.

>> - Integrate various fixes to support better high resolution display.
>> In Retina mac display Pharo looks blurry. The fix to support Retina
>> should be integrated in the VM.
>>
>> ### Sista-related
>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>> JIT is manipulating code (folding constant, rewriting it) before
>> compiling it to assembly code.
>
> Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.
>
>> - New Block Closure implementation by default: Allows one to implement
>> in the Opal compiler the copying and clean blocks optimisations,
>> reduce massively the complexity of some parts of the JIT (debugging
>> API to map machine code pc to bytecode pc for example) and is required
>> for the Sista support. Some work remains in debugging/IDE support.
>>
>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>> the code base of the bytecode to source code decompiler, the debugger,
>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>> and is required for the Sista support. Some work remains in debugging
>> support.
>>
>> - Read-only objects: They work in Pharo 6, but the in-image support
>> needs to be improved (fall-back code of primitives, etc.) and some
>> polishing needs to be done (primitive error code not always correct,
>> etc.). Used for the support of different project, including GBS.
>
>> - Literals immutability (based on read-only objects). In particular an
>> analysis of the image usage of literal (string mutation) is needed.
>
>> - Sista preview: A first version of Sista will be integrated, yielding
>> 1.5x performance boost on most applications, but will be optional and
>> will require specific constraints (not toying too much with literal
>> mutability, partial IDE support, etc.).
>>
>>
>> ### Already done
>>
>> - [DONE] Remove Shoreline reporter.
>> - [DONE] Kernel should not depend on Traits: This will speed up the
>> bootstrap and support the modular introduction of alternate traits
>> implementation currently designed by Pablo Tesone and tested in the
>> new generation of Moose.
>
> Eliot
> _,,,^..^,,,_ (phone)





--
_,,,^..^,,,_
best, Eliot
Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Sven Van Caekenberghe-2

> On 6 Jul 2017, at 23:56, Eliot Miranda <[hidden email]> wrote:
>
> Hi Sven,
>
> On Thu, Jul 6, 2017 at 10:10 AM, Sven Van Caekenberghe <[hidden email]> wrote:
>
> > On 6 Jul 2017, at 18:41, Eliot Miranda <[hidden email]> wrote:
> >
> > Hi Stef,
> >
> >> On Jul 6, 2017, at 1:55 AM, Stephane Ducasse <[hidden email]> wrote:
> >>
> >> We would like to share this list with you and get your feedback and inputs.
> >> It will be presented and discussed again at ESUG.
> >>
> >> Stef on the behalf of the engineering team of the Pharo consortium.
> >>
> >> # Pharo 7 (and 8) potential roadmap
> >>
> >> This document contains a list of actions that should be done during
> >> Pharo 7 and 8.
> >> It is not a definitive list. It is the result of a discussion within
> >> the engineer team and RModers.
> >> It first lists actions that should be performed at the image level
> >> then lists the actions for the VM.
> >>
> >>
> >> ## Image
> >> As a general principle, we will try to remove something when we add a
> >> new features or version of a component.
> >>
> >> ### New tools
> >> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
> >> For the moment it supports Git. This tool will become central to the
> >> development of projects in Pharo. The key and first enhancements will
> >> be:
> >> - cherry picking
> >> - multiple directories support, subtrees
> >> - better new development process support
> >>
> >> - Calypso: Calypso is a new tool suite for editing and navigating
> >> code. It is modular and can easily be extended.
> >> - integration, set as the default browser
> >>
> >> - Hermes (binary loader): Hermes is a code loader that does not
> >> require the compiler to be loaded. It will be used to speed up the
> >> bootstrap.
> >>
> >> - Beacon: Beacon is an announcement-based logger. It should be
> >> introduced but in addition the left over of previous logging should be
> >> cleaned and removed for example many of the transcript show: should be
> >> removed.
> >>
> >> - Inspector refreshing: the inspector should refresh its values by default.
> >>
> >> - Cargo: Cargo is a new package manager. It supports the expression of
> >> dependencies between packages. We are currently validating that it can
> >> support conditional loading (platform) and building new tooling to
> >> express and validate data.
> >>
> >> - Check dependencies when committing. Pharo comes with a dependency
> >> analyser tools. Such tools should be used before commiting to warn the
> >> user when a new dependency is introduced in a package.
> >>
> >> ### Cleaning bloat
> >> - Removing of Nautilus: Once Calypso will be integrated and exhibit
> >> similar fetaures than Nautilus. Nautilus should be removed.
> >> - Remove old text editor: there is only one or two widgets still using
> >> the old text editor and the rubric text editor.
> >> - Remove TxText: TxText was an attempt to get a new generation text
> >> editor. Now it has been rewritten with a new design in Bloc so we
> >> should remove it since Bloc editor is better and actively maintained.
> >> - Remove Komitter: Iceberg already supports cherrypicking on commit
> >> therefore Komitter can be safely removed from Pharo.
> >> - Remove system categorizer: The old system categorizer is not used
> >> anymore and should be remove.
> >>
> >> - Old compiler removal: The old compiler should now get unloaded from Pharo.
> >> - The old compiler should be moved to an external package and make
> >> sure that it can be reloaded.
> >> - In addition the encoders should be separated. (@@ more here@@)
> >>
> >> - Old inspector cleanup: we should remove the eyeInspector framework
> >> but we should introduce a minimal fallback inspector. This inspector
> >> should work without Spec or any frameworks.
> >>
> >> - Clean behavior protocol. The number of methods in the core Behavior,
> >> ClassDescription and Class requires some analysis and cleaning.
> >>
> >> - Kernel modularization: the effort on modularising the packages
> >> should be continued.
> >>
> >> ### Language infrastructure
> >> The following points are more related to the infrastructure of
> >> manipulating and loading class definitions. There are the basis for
> >> the future module system and cleaning some often hidden parts of the
> >> system.
> >>
> >> - New class definition: The class definition is not scaling anymore
> >> due to the large number of options (traits, slot, tage, package,
> >> immediate, ephemeron). A fluid-based message API should be designed.
> >>
> >> - Support for Undefined classes: When loading old code or code whose
> >> superclass has not yet being loaded, the system has inconsistent
> >> behavior. Depending on the tools, it may not load the code, raise a
> >> warning or decide that the superclass is Object without any other
> >> notice and losing the name of the original superclass. We are working
> >> on a new mechanism to support a unique way to handle such case. To be
> >> presented at IWST/ESUG
> >>
> >> - Class definition parser: Class definition is parser in different
> >> place of the system and in addition the output is the direct creation
> >> of a class object instead of an object representing the class
> >> definition that can be further manipulated. We are working on class
> >> definition parser. It produces a separate AST. It will help the
> >> building of tools.
> >>
> >> - Better update infrastructure. Pablo Tesone has been working on a
> >> better update mechanism, better modular class builder.
> >>
> >> - Ring2: Ring is a metamodel to manipulate code that is not actively
> >> running in the current system. It is useful to perform analysis (such
> >> as browsing, navigating off-line or remote definitions) before
> >> actually loading the code. Ring got redesigned by Pavel Krivanek to
> >> support the bootstrap of Pharo 60. The results is Ring2.
> >>
> >> - Opal is the Pharo Compiler. It should be enhanced to support handle
> >> better the warning and a general enhancement pass is needed.
> >>
> >> ### System enhancements
> >>
> >> - Commandline enhancements. RMOD is currently improving the
> >> command-line infrastructure and making sure that the system can work
> >> in read-only mode.
> >>
> >> - Cleaning streams: the idea is to make sure that the system does not
> >> use the old streams. The idea is to start using the fileStreams and
> >> make sure that the Stream package can be substituated in the future by
> >> other streams with the same API. Therefore hardcoded class names
> >> should be substituted with factory (readSream writeStream). From that
> >> perspective we do not think that it is wise to introduce Xtreams. We
> >> should analyse the API of the current implementation.
> >>
> >> - New theme from the beginning. It is really important that each
> >> version of Pharo is identified open the default opening. Pharo should
> >> come up with two default themes: one light and one dark.
> >>
> >> - Better themes/palettes support
> >> - Better and nicer Tabs. Tabs design should be revisited.
> >>
> >> - OSWindow world rendering: the effort to remove the logic of the
> >> windowing from the VM should be continued. OSWindow should be used
> >> instead.
> >>
> >> - Freetype: The current freetype plugin is the source of many bugs and
> >> problem. Thibault Raffaillac used FFI to do direct call to openGL
> >> (@@to verify@@).
> >>
> >>
> >> - Refactoring 2: Refactoring 2 is an improved version of the
> >> refactorings developed by Gustavos Santos and they should used to
> >> replace the existing one.
> >>
> >>
> >> ## VM
> >>
> >> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
> >> over a year and since June 2017 a Windows 64 bits VM has been working.
> >> The plan is to stabilize each part of Pharo that is not working in 64
> >> bits as well as in 32 bits and make the 64 bits Pharo images and VM
> >> the default download for Pharo.
> >>
> >> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
> >> we need to merge and check everything works fine. With the headless
> >> VM, Pharo can be run in true headless mode (not with a hidden window)
> >> and it is required for future VM features (embeddable VM, ...).
> >>
> >> - Embeddable VM: The VM should be able to be embedded in external
> >> applications, the application accessing objects through well-defined
> >> APIs.
> >>
> >> - Idle VM: The goal is to avoid the active waiting loop consuming
> >> several per cent of cpu time when the Pharo image is not doing
> >> anything.
> >>
> >> - Android VM: integration of the Android VM build and tests in the
> >> integration setup.
> >>
> >> - Threaded FFI: all FFI calls should be non-blocking (reviving
> >> prototype of Eliot)
> >>
> >> - ZeroConf for ARM: The ARM VM should be available from the zeroConf servers
> >>
> >> - Better support for large heaps (GC tuning API, incremental GC).
> >> Pharo 64 bit is now able to manage large heap. However better
> >> performance can be proposed by offering better settings for the
> >> different GC zone.
> >
> > The most important thing here is the incremental GC.  Spur has a generation scavenger that collects garbage in newly created objects (new space), and a mark-compact collector that collects and compacts garbage in old space.
> >
> > Right now on my 2.3GHz MacMini doing normal development, the generation scavenger causes pauses of 1ms or less, and the mark-compact collector than causes pauses of around 200ms.  Both account for about 0.75% of entire execution time (1.5% total), so the scavenger is invoked frequently and the pauses that it creates are not noticeable.  But the occasional pauses created by the mark-compact collector /are/ noticeable, especially in games and music.
> >
> > The idea for the incremental collector is to implement a mark-sweep or mark-sweep-compact collector for old space that works incrementally, doing a little bit of work on each invocation, probably immediately after a scavenge.  It is intended to avoid the long pauses caused by the non-incremental mark-compact collector and make the system more suitable for games, music, etc.
> >
> > [Work on scaling for large 64-bit heaps is somewhere in the future, but right now we don't have any applications like this generating demand for a solution.]
>
> Actually, being able to efficiently keep a couple of GB worth of objects in memory, even better 10s of GB seems like an important short term goal. I for one don't really care for games or music. I do care about server software and there we need to use the memory that is available so cheaply today. Luckily we now have 64-bit support.
>
> Indeed, and the reason I designed Spur to be both 32-bit and 64-bit was because my employer Cadence wanted to scale to much larger models (the application manipulates large models of SystemOnAChip (SOC) components expressed in XML).  But their application is batch and they're happy just with the ability to go somewhat above the approximately 3 Gb limit of the 32-bit system.
>
> But what I was getting at in my comment was that, while 64-bit server applications are indeed very important, I have not been contacted by anyone yet complaining about scaling problems with the 64-bit system.  This could be for a number of reasons, including that there are not many large applications out there.  We know that pause times in the full GC are an issue for animation ad music software.  I didn't articulate it, but I also think that an incremental GC provides somewhat of an answer to scaling for 64-bit servers.
>
> Let's say that currently the mark-compact collector collects at about 1Gb per second (that's about what I'm seeing on my MacMini for a 600Mb heap, collected and compacted in about 530ms on a 2.3GHz Core i7, = 1.1Gb/s).  For the moment let's assume the GC scales linearly.  Then a 10Gb server application will occasionally suffer 10 second pauses, which is unacceptable for any kind of high-availability server.  The total execution time overhead of GC may be in the 1%-10% range, but the pause times may be unacceptable (note that they're /not/ unacceptable for my employer's batch usage).
>
> So were the GC to be extended with an incremental collector we may be able to bring pause times back down to the tens of milliseconds rage or below.  Clearly how the server uses memory, and how the GC scales are crucially important.  Right now I have no data on either; no large 64-bit server applications to analyze and consequently not much idea of how the current GC or an incremental GC will scale.  So for the moment I'm going to prioritize doing an incremental GC, and making it measurable and tunable. Hopefully the incremental GC will benefit both large server applications as well as music and animation.
>
> But nothing will substitute for experience with real applications.  So far there are very few large 64-bit applications running on Cog for me to analyze.  Even Cadence's customers are too protective of their data for me to analyze real applications.

Thanks for the reply, Eliot.

Maybe is was (up until we got 64-bit for real) a chicken-egg problem: it was not realistic to build big applications due to the limits of 32-bit images. I am sure this will change in the future.

Sven

> >> - Integrate various fixes to support better high resolution display.
> >> In Retina mac display Pharo looks blurry. The fix to support Retina
> >> should be integrated in the VM.
> >>
> >> ### Sista-related
> >> Sista is an optimizing JIT for Pharo. It is the result of multiple
> >> years of developement by Clement Bera and Eliot Miranda. An optimizing
> >> JIT is manipulating code (folding constant, rewriting it) before
> >> compiling it to assembly code.
> >
> > Actually this is a speculative inliner (also known as an adaptive optimizer).  It works by creating special optimized versions of methods that inline smaller methods, thereby allowing optimising away unnecessary tests, sends and blocks, etc.  it is driven by "hot spot" detectors in the first-level JIT code and so inlines code that is better my used a lot.  It is quite similar to the optimisers in Java HotSpot, JavaScript V8, etc.  its implementation and chitecture is more like Graal, because the inliner, optimiser and deoptimiser are all in Smalltalk up in the image.  This in-image optimizer is called Scorch.  The overall architecture is called Sista, for Speculative Inlining Smalltalk Architecture.
> >
> >> - New Block Closure implementation by default: Allows one to implement
> >> in the Opal compiler the copying and clean blocks optimisations,
> >> reduce massively the complexity of some parts of the JIT (debugging
> >> API to map machine code pc to bytecode pc for example) and is required
> >> for the Sista support. Some work remains in debugging/IDE support.
> >>
> >> - New Bytecode set in production: Eases bytecode decoding (simplifying
> >> the code base of the bytecode to source code decompiler, the debugger,
> >> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
> >> and is required for the Sista support. Some work remains in debugging
> >> support.
> >>
> >> - Read-only objects: They work in Pharo 6, but the in-image support
> >> needs to be improved (fall-back code of primitives, etc.) and some
> >> polishing needs to be done (primitive error code not always correct,
> >> etc.). Used for the support of different project, including GBS.
> >
> >> - Literals immutability (based on read-only objects). In particular an
> >> analysis of the image usage of literal (string mutation) is needed.
> >
> >> - Sista preview: A first version of Sista will be integrated, yielding
> >> 1.5x performance boost on most applications, but will be optional and
> >> will require specific constraints (not toying too much with literal
> >> mutability, partial IDE support, etc.).
> >>
> >>
> >> ### Already done
> >>
> >> - [DONE] Remove Shoreline reporter.
> >> - [DONE] Kernel should not depend on Traits: This will speed up the
> >> bootstrap and support the modular introduction of alternate traits
> >> implementation currently designed by Pablo Tesone and tested in the
> >> new generation of Moose.
> >
> > Eliot
> > _,,,^..^,,,_ (phone)
>
>
>
>
>
> --
> _,,,^..^,,,_
> best, Eliot


Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

K K Subbu
In reply to this post by Eliot Miranda-2
On Friday 07 July 2017 03:26 AM, Eliot Miranda wrote:
> Let's say that currently the mark-compact collector collects at about
> 1Gb per second (that's about what I'm seeing on my MacMini for a 600Mb
> heap, collected and compacted in about 530ms on a 2.3GHz Core i7, =
> 1.1Gb/s).  For the moment let's assume the GC scales linearly.  Then a
> 10Gb server application will occasionally suffer 10 second pauses, which
> is unacceptable for any kind of high-availability server.  The total
> execution time overhead of GC may be in the 1%-10% range, but the pause
> times may be unacceptable (note that they're /not/ unacceptable for my
> employer's batch usage).

Good point! Object memory is a shared resource and like any shared
resource will eventually become a bottleneck as it grows.

We could also tackle the problem of large apps with a different
approach. The overheads of VM process may become negligible enough to
make message passing across multiple co-operating processes more
efficient than within a large monolithic process. E.g. if a computation
of 1GB of data generates 10GB of garbage, then the extra 10GB need not
be explicitly garbage collected if the task is done in a child (worker)
process and terminated after the computed result is passed back to its
parent.

Regards .. Subbu

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Stephane Ducasse-3
In reply to this post by Tim Mackinnon
On Thu, Jul 6, 2017 at 11:27 PM, Tim Mackinnon <[hidden email]> wrote:
> I didn’t mean to touch a nerve - and this was why I wrote “minor” points - but you did ask for feedback…

you did not :)
Just that if we just count on fully book engineers working on super
important features we will not make it.
This is why once in a while I sit and fix a boring pop up or related.
I hate the stupid pop up for pakcgae selection and one day I will have
to retry to kill it.

>
> Point noted on giving user feedback - I’d actually like to fix things, but currently its just too hard to submit fixes other than pull requests for documentation that is sitting in git hub (and I have submitted those). I know you are working on a simpler way to contribute (and getting the GitHub process smoothed out is in progress so its not fair to comment on that yet as its early days - and much appreciated.
>
> The Alt-Tab issue in the tracker since 2015 - but I will try and report others and possibly Calypso might have some better keyboard navigation tricks that help.

do you have the bug entry?

>
> For website documentation - I’ll take it back, as I went back again just now and everything I could think of was there (a lot in the news section which I hadn’t noticed). I do think the download page is a bit confusing but I know there is work being done on that.
>
> Tim
>
>
>> On 6 Jul 2017, at 19:11, Stephane Ducasse <[hidden email]> wrote:
>>
>>> Only 2 minor items stick out as missing:
>>>
>>> 1) Continuing to improve keyboard shortcut support (its a lot better, but
>>> not quite completed - I really miss some shortcuts, particularly the ability
>>> to meta-tab between windows - ALT-Tab only works in some windows, and widen
>>> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
>>> word, a statement, a function etc)
>>
>> There are three issues there:
>> - moving from the vm the keyboard generation (this will be done via
>> OSWindow and the VM can focus its main job: execution).
>> - then there is the way we declare/manage shortcuts: this is an
>> ongoing efforts where everybody can join. We should remove the
>> hardcoded shortcut and turn them into KeyBindings
>> - Most of the tools will be throw away when bloc will be integrated.
>> Now it does not mean that we should not do it.
>>
>> Now an important point: as a user you should report but you can also
>> contribute.
>> This is super difficult for me to fix something that I have no clue
>> how to reproduce/never encounter.
>> You see the mental model of a real open-source movement is to share
>> but also to produce together and the point is
>> what is the reward model: why should I spend time on pain I do not
>> feel. Of course we are already doing a lot
>> of actions that is not focus on our immediate needs and we should
>> improve the situation but I wanted to share
>> with you this perspective.
>>
>>
>>>
>>> 2) Keeping the website documentation more up to date (again the pharo.org
>>> website is very slick, and a great showcase, however often there are late
>>> breaking changes which new users won’t know about unless they trace through
>>> the news groups or subscribe to some blogs). If we could also focus on
>>> keeping it simple but up to date - that might help.
>>
>> Where is it not up to date?
>> We are always reacting to any broken piece there.
>> You see producing Pharo by Example 5 was A LOT of work and I thank
>> again Nicolai Hesse for all his huge efforts.
>>
>>
>>> I also have 1 small query - the references to the text editor, is this
>>> making the editor that was demo’d at PharoDays 2107 integrated? As that one
>>> was very cool?
>>
>> Yes calypso will replace our old friend Nautilus and we will certainly
>> need to put a lot of love into calypso to get
>> it to the right level. Now the basis is sound.
>> About the node we will revisit smart suggestions and indeed sometimes
>> there are problems to catch the correct AST nodes.
>>
>>> I don’t know how extensible it is (I’m hoping so - and that
>>> it might help making it easier to improve code editing tools such that
>>> refactorings can more intelligently understand where your cursor is with
>>> regards to the AST node underneath it, or we can add more intellisense
>>> options that match some of the things we see in IntelliJ).
>>>
>>> Tim
>>>
>>> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>>>
>>>
>>> TxText removal is already done too.
>>>
>>> -- Pavel
>>>
>>> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>>>
>>>> We would like to share this list with you and get your feedback and
>>>> inputs.
>>>> It will be presented and discussed again at ESUG.
>>>>
>>>> Stef on the behalf of the engineering team of the Pharo consortium.
>>>>
>>>> # Pharo 7 (and 8) potential roadmap
>>>>
>>>> This document contains a list of actions that should be done during
>>>> Pharo 7 and 8.
>>>> It is not a definitive list. It is the result of a discussion within
>>>> the engineer team and RModers.
>>>> It first lists actions that should be performed at the image level
>>>> then lists the actions for the VM.
>>>>
>>>>
>>>> ## Image
>>>> As a general principle, we will try to remove something when we add a
>>>> new features or version of a component.
>>>>
>>>> ### New tools
>>>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>>>> For the moment it supports Git. This tool will become central to the
>>>> development of projects in Pharo. The key and first enhancements will
>>>> be:
>>>>  - cherry picking
>>>>  - multiple directories support, subtrees
>>>>  - better new development process support
>>>>
>>>> - Calypso: Calypso is a new tool suite for editing and navigating
>>>> code. It is modular and can easily be extended.
>>>>  - integration, set as the default browser
>>>>
>>>> - Hermes (binary loader): Hermes is a code loader that does not
>>>> require the compiler to be loaded. It will be used to speed up the
>>>> bootstrap.
>>>>
>>>> - Beacon: Beacon is an announcement-based logger. It should be
>>>> introduced but in addition the left over of previous logging should be
>>>> cleaned and removed for example many of the transcript show: should be
>>>> removed.
>>>>
>>>> - Inspector refreshing: the inspector should refresh its values by
>>>> default.
>>>>
>>>> - Cargo: Cargo is a new package manager. It supports the expression of
>>>> dependencies between packages. We are currently validating that it can
>>>> support conditional loading (platform) and building new tooling to
>>>> express and validate data.
>>>>
>>>> - Check dependencies when committing. Pharo comes with a dependency
>>>> analyser tools. Such tools should be used before commiting to warn the
>>>> user when a new dependency is introduced in a package.
>>>>
>>>> ### Cleaning bloat
>>>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>>>> similar fetaures than Nautilus. Nautilus should be removed.
>>>> - Remove old text editor: there is only one or two widgets still using
>>>> the old text editor and the rubric text editor.
>>>> - Remove TxText: TxText was an attempt to get a new generation text
>>>> editor. Now it has been rewritten with a new design in Bloc so we
>>>> should remove it since Bloc editor is better and actively maintained.
>>>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>>>> therefore Komitter can be safely removed from Pharo.
>>>> - Remove system categorizer: The old system categorizer is not used
>>>> anymore and should be remove.
>>>>
>>>> - Old compiler removal: The old compiler should now get unloaded from
>>>> Pharo.
>>>>  - The old compiler should be moved to an external package and make
>>>> sure that it can be reloaded.
>>>>  - In addition the encoders should be separated. (@@ more here@@)
>>>>
>>>> - Old inspector cleanup: we should remove the eyeInspector framework
>>>> but we should introduce a minimal fallback inspector. This inspector
>>>> should work without Spec or any frameworks.
>>>>
>>>> - Clean behavior protocol. The number of methods in the core Behavior,
>>>> ClassDescription and Class requires some analysis and cleaning.
>>>>
>>>> - Kernel modularization: the effort on modularising the packages
>>>> should be continued.
>>>>
>>>> ### Language infrastructure
>>>> The following points are more related to the infrastructure of
>>>> manipulating and loading class definitions. There are the basis for
>>>> the future module system and cleaning some often hidden parts of the
>>>> system.
>>>>
>>>> - New class definition: The class definition is not scaling anymore
>>>> due to the large number of options (traits, slot, tage, package,
>>>> immediate, ephemeron). A fluid-based message API should be designed.
>>>>
>>>> - Support for Undefined classes: When loading old code or code whose
>>>> superclass has not yet being loaded, the system has inconsistent
>>>> behavior. Depending on the tools, it may not load the code, raise a
>>>> warning or decide that the superclass is Object without any other
>>>> notice and losing the name of the original superclass. We are working
>>>> on a new mechanism to support a unique way to handle such case. To be
>>>> presented at IWST/ESUG
>>>>
>>>> - Class definition parser: Class definition is parser in different
>>>> place of the system and in addition the output is the direct creation
>>>> of a class object instead of an object representing the class
>>>> definition that can be further manipulated. We are working on class
>>>> definition parser. It produces a separate AST. It will help the
>>>> building of tools.
>>>>
>>>> - Better update infrastructure. Pablo Tesone has been working on a
>>>> better update mechanism, better modular class builder.
>>>>
>>>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>>>> running in the current system. It is useful to perform analysis (such
>>>> as browsing, navigating off-line or remote definitions) before
>>>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>>>> support the bootstrap of Pharo 60. The results is Ring2.
>>>>
>>>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>>>> better the warning and a general enhancement pass is needed.
>>>>
>>>> ### System enhancements
>>>>
>>>> - Commandline enhancements. RMOD is currently improving the
>>>> command-line infrastructure and making sure that the system can work
>>>> in read-only mode.
>>>>
>>>> - Cleaning streams: the idea is to make sure that the system does not
>>>> use the old streams. The idea is to start using the fileStreams and
>>>> make sure that the Stream package can be substituated in the future by
>>>> other streams with the same API. Therefore hardcoded class names
>>>> should be substituted with factory (readSream writeStream). From that
>>>> perspective we do not think that it is wise to introduce Xtreams. We
>>>> should analyse the API of the current implementation.
>>>>
>>>> - New theme from the beginning. It is really important that each
>>>> version of Pharo is identified open the default opening. Pharo should
>>>> come up with two default themes: one light and one dark.
>>>>
>>>> - Better themes/palettes support
>>>> - Better and nicer Tabs. Tabs design should be revisited.
>>>>
>>>> - OSWindow world rendering: the effort to remove the logic of the
>>>> windowing from the VM should be continued. OSWindow should be used
>>>> instead.
>>>>
>>>> - Freetype: The current freetype plugin is the source of many bugs and
>>>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>>>> (@@to verify@@).
>>>>
>>>>
>>>> - Refactoring 2: Refactoring 2 is an improved version of the
>>>> refactorings developed by Gustavos Santos and they should used to
>>>> replace the existing one.
>>>>
>>>>
>>>> ## VM
>>>>
>>>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>>>> over a year and since June 2017 a Windows 64 bits VM has been working.
>>>> The plan is to stabilize each part of Pharo that is not working in 64
>>>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>>>> the default download for Pharo.
>>>>
>>>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>>>> we need to merge and check everything works fine. With the headless
>>>> VM, Pharo can be run in true headless mode (not with a hidden window)
>>>> and it is required for future VM features (embeddable VM, ...).
>>>>
>>>> - Embeddable VM: The VM should be able to be embedded in external
>>>> applications, the application accessing objects through well-defined
>>>> APIs.
>>>>
>>>> - Idle VM: The goal is to avoid the active waiting loop consuming
>>>> several per cent of cpu time when the Pharo image is not doing
>>>> anything.
>>>>
>>>> - Android VM: integration of the Android VM build and tests in the
>>>> integration setup.
>>>>
>>>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>>>> prototype of Eliot)
>>>>
>>>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>>>> servers
>>>>
>>>> - Better support for large heaps (GC tuning API, incremental GC).
>>>> Pharo 64 bit is now able to manage large heap. However better
>>>> performance can be proposed by offering better settings for the
>>>> different GC zone.
>>>>
>>>> - Integrate various fixes to support better high resolution display.
>>>> In Retina mac display Pharo looks blurry. The fix to support Retina
>>>> should be integrated in the VM.
>>>>
>>>> ### Sista-related
>>>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>>>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>>>> JIT is manipulating code (folding constant, rewriting it) before
>>>> compiling it to assembly code.
>>>>
>>>> - New Block Closure implementation by default: Allows one to implement
>>>> in the Opal compiler the copying and clean blocks optimisations,
>>>> reduce massively the complexity of some parts of the JIT (debugging
>>>> API to map machine code pc to bytecode pc for example) and is required
>>>> for the Sista support. Some work remains in debugging/IDE support.
>>>>
>>>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>>>> the code base of the bytecode to source code decompiler, the debugger,
>>>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>>>> and is required for the Sista support. Some work remains in debugging
>>>> support.
>>>>
>>>> - Read-only objects: They work in Pharo 6, but the in-image support
>>>> needs to be improved (fall-back code of primitives, etc.) and some
>>>> polishing needs to be done (primitive error code not always correct,
>>>> etc.). Used for the support of different project, including GBS.
>>>>
>>>> - Literals immutability (based on read-only objects). In particular an
>>>> analysis of the image usage of literal (string mutation) is needed.
>>>>
>>>> - Sista preview: A first version of Sista will be integrated, yielding
>>>> 1.5x performance boost on most applications, but will be optional and
>>>> will require specific constraints (not toying too much with literal
>>>> mutability, partial IDE support, etc.).
>>>>
>>>>
>>>> ### Already done
>>>>
>>>> - [DONE] Remove Shoreline reporter.
>>>> - [DONE] Kernel should not depend on Traits: This will speed up the
>>>> bootstrap and support the modular introduction of alternate traits
>>>> implementation currently designed by Pablo Tesone and tested in the
>>>> new generation of Moose.
>>>>
>>>
>>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Tim Mackinnon
Actually that's a good point - should the roadmap encompass what the community can offer too (i hadn't appreciated the distinction )?

We collectively pay for some things to be achieved, but also we expect that we can contribute some of the lower hanging fruit.

Of course some things are at the whim of what itches to scratch, but others we may be able to rally around?

Tim

Sent from my iPhone

> On 7 Jul 2017, at 06:11, Stephane Ducasse <[hidden email]> wrote:
>
>> On Thu, Jul 6, 2017 at 11:27 PM, Tim Mackinnon <[hidden email]> wrote:
>> I didn’t mean to touch a nerve - and this was why I wrote “minor” points - but you did ask for feedback…
>
> you did not :)
> Just that if we just count on fully book engineers working on super
> important features we will not make it.
> This is why once in a while I sit and fix a boring pop up or related.
> I hate the stupid pop up for pakcgae selection and one day I will have
> to retry to kill it.
>
>>
>> Point noted on giving user feedback - I’d actually like to fix things, but currently its just too hard to submit fixes other than pull requests for documentation that is sitting in git hub (and I have submitted those). I know you are working on a simpler way to contribute (and getting the GitHub process smoothed out is in progress so its not fair to comment on that yet as its early days - and much appreciated.
>>
>> The Alt-Tab issue in the tracker since 2015 - but I will try and report others and possibly Calypso might have some better keyboard navigation tricks that help.
>
> do you have the bug entry?
>
>>
>> For website documentation - I’ll take it back, as I went back again just now and everything I could think of was there (a lot in the news section which I hadn’t noticed). I do think the download page is a bit confusing but I know there is work being done on that.
>>
>> Tim
>>
>>
>>>> On 6 Jul 2017, at 19:11, Stephane Ducasse <[hidden email]> wrote:
>>>>
>>>> Only 2 minor items stick out as missing:
>>>>
>>>> 1) Continuing to improve keyboard shortcut support (its a lot better, but
>>>> not quite completed - I really miss some shortcuts, particularly the ability
>>>> to meta-tab between windows - ALT-Tab only works in some windows, and widen
>>>> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
>>>> word, a statement, a function etc)
>>>
>>> There are three issues there:
>>> - moving from the vm the keyboard generation (this will be done via
>>> OSWindow and the VM can focus its main job: execution).
>>> - then there is the way we declare/manage shortcuts: this is an
>>> ongoing efforts where everybody can join. We should remove the
>>> hardcoded shortcut and turn them into KeyBindings
>>> - Most of the tools will be throw away when bloc will be integrated.
>>> Now it does not mean that we should not do it.
>>>
>>> Now an important point: as a user you should report but you can also
>>> contribute.
>>> This is super difficult for me to fix something that I have no clue
>>> how to reproduce/never encounter.
>>> You see the mental model of a real open-source movement is to share
>>> but also to produce together and the point is
>>> what is the reward model: why should I spend time on pain I do not
>>> feel. Of course we are already doing a lot
>>> of actions that is not focus on our immediate needs and we should
>>> improve the situation but I wanted to share
>>> with you this perspective.
>>>
>>>
>>>>
>>>> 2) Keeping the website documentation more up to date (again the pharo.org
>>>> website is very slick, and a great showcase, however often there are late
>>>> breaking changes which new users won’t know about unless they trace through
>>>> the news groups or subscribe to some blogs). If we could also focus on
>>>> keeping it simple but up to date - that might help.
>>>
>>> Where is it not up to date?
>>> We are always reacting to any broken piece there.
>>> You see producing Pharo by Example 5 was A LOT of work and I thank
>>> again Nicolai Hesse for all his huge efforts.
>>>
>>>
>>>> I also have 1 small query - the references to the text editor, is this
>>>> making the editor that was demo’d at PharoDays 2107 integrated? As that one
>>>> was very cool?
>>>
>>> Yes calypso will replace our old friend Nautilus and we will certainly
>>> need to put a lot of love into calypso to get
>>> it to the right level. Now the basis is sound.
>>> About the node we will revisit smart suggestions and indeed sometimes
>>> there are problems to catch the correct AST nodes.
>>>
>>>> I don’t know how extensible it is (I’m hoping so - and that
>>>> it might help making it easier to improve code editing tools such that
>>>> refactorings can more intelligently understand where your cursor is with
>>>> regards to the AST node underneath it, or we can add more intellisense
>>>> options that match some of the things we see in IntelliJ).
>>>>
>>>> Tim
>>>>
>>>> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>>>>
>>>>
>>>> TxText removal is already done too.
>>>>
>>>> -- Pavel
>>>>
>>>> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>>>>
>>>>> We would like to share this list with you and get your feedback and
>>>>> inputs.
>>>>> It will be presented and discussed again at ESUG.
>>>>>
>>>>> Stef on the behalf of the engineering team of the Pharo consortium.
>>>>>
>>>>> # Pharo 7 (and 8) potential roadmap
>>>>>
>>>>> This document contains a list of actions that should be done during
>>>>> Pharo 7 and 8.
>>>>> It is not a definitive list. It is the result of a discussion within
>>>>> the engineer team and RModers.
>>>>> It first lists actions that should be performed at the image level
>>>>> then lists the actions for the VM.
>>>>>
>>>>>
>>>>> ## Image
>>>>> As a general principle, we will try to remove something when we add a
>>>>> new features or version of a component.
>>>>>
>>>>> ### New tools
>>>>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>>>>> For the moment it supports Git. This tool will become central to the
>>>>> development of projects in Pharo. The key and first enhancements will
>>>>> be:
>>>>> - cherry picking
>>>>> - multiple directories support, subtrees
>>>>> - better new development process support
>>>>>
>>>>> - Calypso: Calypso is a new tool suite for editing and navigating
>>>>> code. It is modular and can easily be extended.
>>>>> - integration, set as the default browser
>>>>>
>>>>> - Hermes (binary loader): Hermes is a code loader that does not
>>>>> require the compiler to be loaded. It will be used to speed up the
>>>>> bootstrap.
>>>>>
>>>>> - Beacon: Beacon is an announcement-based logger. It should be
>>>>> introduced but in addition the left over of previous logging should be
>>>>> cleaned and removed for example many of the transcript show: should be
>>>>> removed.
>>>>>
>>>>> - Inspector refreshing: the inspector should refresh its values by
>>>>> default.
>>>>>
>>>>> - Cargo: Cargo is a new package manager. It supports the expression of
>>>>> dependencies between packages. We are currently validating that it can
>>>>> support conditional loading (platform) and building new tooling to
>>>>> express and validate data.
>>>>>
>>>>> - Check dependencies when committing. Pharo comes with a dependency
>>>>> analyser tools. Such tools should be used before commiting to warn the
>>>>> user when a new dependency is introduced in a package.
>>>>>
>>>>> ### Cleaning bloat
>>>>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>>>>> similar fetaures than Nautilus. Nautilus should be removed.
>>>>> - Remove old text editor: there is only one or two widgets still using
>>>>> the old text editor and the rubric text editor.
>>>>> - Remove TxText: TxText was an attempt to get a new generation text
>>>>> editor. Now it has been rewritten with a new design in Bloc so we
>>>>> should remove it since Bloc editor is better and actively maintained.
>>>>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>>>>> therefore Komitter can be safely removed from Pharo.
>>>>> - Remove system categorizer: The old system categorizer is not used
>>>>> anymore and should be remove.
>>>>>
>>>>> - Old compiler removal: The old compiler should now get unloaded from
>>>>> Pharo.
>>>>> - The old compiler should be moved to an external package and make
>>>>> sure that it can be reloaded.
>>>>> - In addition the encoders should be separated. (@@ more here@@)
>>>>>
>>>>> - Old inspector cleanup: we should remove the eyeInspector framework
>>>>> but we should introduce a minimal fallback inspector. This inspector
>>>>> should work without Spec or any frameworks.
>>>>>
>>>>> - Clean behavior protocol. The number of methods in the core Behavior,
>>>>> ClassDescription and Class requires some analysis and cleaning.
>>>>>
>>>>> - Kernel modularization: the effort on modularising the packages
>>>>> should be continued.
>>>>>
>>>>> ### Language infrastructure
>>>>> The following points are more related to the infrastructure of
>>>>> manipulating and loading class definitions. There are the basis for
>>>>> the future module system and cleaning some often hidden parts of the
>>>>> system.
>>>>>
>>>>> - New class definition: The class definition is not scaling anymore
>>>>> due to the large number of options (traits, slot, tage, package,
>>>>> immediate, ephemeron). A fluid-based message API should be designed.
>>>>>
>>>>> - Support for Undefined classes: When loading old code or code whose
>>>>> superclass has not yet being loaded, the system has inconsistent
>>>>> behavior. Depending on the tools, it may not load the code, raise a
>>>>> warning or decide that the superclass is Object without any other
>>>>> notice and losing the name of the original superclass. We are working
>>>>> on a new mechanism to support a unique way to handle such case. To be
>>>>> presented at IWST/ESUG
>>>>>
>>>>> - Class definition parser: Class definition is parser in different
>>>>> place of the system and in addition the output is the direct creation
>>>>> of a class object instead of an object representing the class
>>>>> definition that can be further manipulated. We are working on class
>>>>> definition parser. It produces a separate AST. It will help the
>>>>> building of tools.
>>>>>
>>>>> - Better update infrastructure. Pablo Tesone has been working on a
>>>>> better update mechanism, better modular class builder.
>>>>>
>>>>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>>>>> running in the current system. It is useful to perform analysis (such
>>>>> as browsing, navigating off-line or remote definitions) before
>>>>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>>>>> support the bootstrap of Pharo 60. The results is Ring2.
>>>>>
>>>>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>>>>> better the warning and a general enhancement pass is needed.
>>>>>
>>>>> ### System enhancements
>>>>>
>>>>> - Commandline enhancements. RMOD is currently improving the
>>>>> command-line infrastructure and making sure that the system can work
>>>>> in read-only mode.
>>>>>
>>>>> - Cleaning streams: the idea is to make sure that the system does not
>>>>> use the old streams. The idea is to start using the fileStreams and
>>>>> make sure that the Stream package can be substituated in the future by
>>>>> other streams with the same API. Therefore hardcoded class names
>>>>> should be substituted with factory (readSream writeStream). From that
>>>>> perspective we do not think that it is wise to introduce Xtreams. We
>>>>> should analyse the API of the current implementation.
>>>>>
>>>>> - New theme from the beginning. It is really important that each
>>>>> version of Pharo is identified open the default opening. Pharo should
>>>>> come up with two default themes: one light and one dark.
>>>>>
>>>>> - Better themes/palettes support
>>>>> - Better and nicer Tabs. Tabs design should be revisited.
>>>>>
>>>>> - OSWindow world rendering: the effort to remove the logic of the
>>>>> windowing from the VM should be continued. OSWindow should be used
>>>>> instead.
>>>>>
>>>>> - Freetype: The current freetype plugin is the source of many bugs and
>>>>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>>>>> (@@to verify@@).
>>>>>
>>>>>
>>>>> - Refactoring 2: Refactoring 2 is an improved version of the
>>>>> refactorings developed by Gustavos Santos and they should used to
>>>>> replace the existing one.
>>>>>
>>>>>
>>>>> ## VM
>>>>>
>>>>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>>>>> over a year and since June 2017 a Windows 64 bits VM has been working.
>>>>> The plan is to stabilize each part of Pharo that is not working in 64
>>>>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>>>>> the default download for Pharo.
>>>>>
>>>>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>>>>> we need to merge and check everything works fine. With the headless
>>>>> VM, Pharo can be run in true headless mode (not with a hidden window)
>>>>> and it is required for future VM features (embeddable VM, ...).
>>>>>
>>>>> - Embeddable VM: The VM should be able to be embedded in external
>>>>> applications, the application accessing objects through well-defined
>>>>> APIs.
>>>>>
>>>>> - Idle VM: The goal is to avoid the active waiting loop consuming
>>>>> several per cent of cpu time when the Pharo image is not doing
>>>>> anything.
>>>>>
>>>>> - Android VM: integration of the Android VM build and tests in the
>>>>> integration setup.
>>>>>
>>>>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>>>>> prototype of Eliot)
>>>>>
>>>>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>>>>> servers
>>>>>
>>>>> - Better support for large heaps (GC tuning API, incremental GC).
>>>>> Pharo 64 bit is now able to manage large heap. However better
>>>>> performance can be proposed by offering better settings for the
>>>>> different GC zone.
>>>>>
>>>>> - Integrate various fixes to support better high resolution display.
>>>>> In Retina mac display Pharo looks blurry. The fix to support Retina
>>>>> should be integrated in the VM.
>>>>>
>>>>> ### Sista-related
>>>>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>>>>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>>>>> JIT is manipulating code (folding constant, rewriting it) before
>>>>> compiling it to assembly code.
>>>>>
>>>>> - New Block Closure implementation by default: Allows one to implement
>>>>> in the Opal compiler the copying and clean blocks optimisations,
>>>>> reduce massively the complexity of some parts of the JIT (debugging
>>>>> API to map machine code pc to bytecode pc for example) and is required
>>>>> for the Sista support. Some work remains in debugging/IDE support.
>>>>>
>>>>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>>>>> the code base of the bytecode to source code decompiler, the debugger,
>>>>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>>>>> and is required for the Sista support. Some work remains in debugging
>>>>> support.
>>>>>
>>>>> - Read-only objects: They work in Pharo 6, but the in-image support
>>>>> needs to be improved (fall-back code of primitives, etc.) and some
>>>>> polishing needs to be done (primitive error code not always correct,
>>>>> etc.). Used for the support of different project, including GBS.
>>>>>
>>>>> - Literals immutability (based on read-only objects). In particular an
>>>>> analysis of the image usage of literal (string mutation) is needed.
>>>>>
>>>>> - Sista preview: A first version of Sista will be integrated, yielding
>>>>> 1.5x performance boost on most applications, but will be optional and
>>>>> will require specific constraints (not toying too much with literal
>>>>> mutability, partial IDE support, etc.).
>>>>>
>>>>>
>>>>> ### Already done
>>>>>
>>>>> - [DONE] Remove Shoreline reporter.
>>>>> - [DONE] Kernel should not depend on Traits: This will speed up the
>>>>> bootstrap and support the modular introduction of alternate traits
>>>>> implementation currently designed by Pablo Tesone and tested in the
>>>>> new generation of Moose.
>>>>>
>>>>
>>>>
>>>
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Stephane Ducasse-3
> Actually that's a good point - should the roadmap encompass what the community can offer too (i hadn't appreciated the distinction )?

Yes it does.
Esteban (and christophe 60% paid by rmod) and guille (60% but also
writing papers) cannot do all that alone?


> We collectively pay for some things to be achieved, but also we expect that we can contribute some of the lower hanging fruit.

Yes and even more.
We at RMOD are not paid to produce Pharo. Pharo only represents a
small part of our job evaluation.
Now if everybody takes 1 hour per week to improve Pharo it will be huge.


> Of course some things are at the whim of what itches to scratch, but others we may be able to rally around?
>
> Tim
>
> Sent from my iPhone
>
>> On 7 Jul 2017, at 06:11, Stephane Ducasse <[hidden email]> wrote:
>>
>>> On Thu, Jul 6, 2017 at 11:27 PM, Tim Mackinnon <[hidden email]> wrote:
>>> I didn’t mean to touch a nerve - and this was why I wrote “minor” points - but you did ask for feedback…
>>
>> you did not :)
>> Just that if we just count on fully book engineers working on super
>> important features we will not make it.
>> This is why once in a while I sit and fix a boring pop up or related.
>> I hate the stupid pop up for pakcgae selection and one day I will have
>> to retry to kill it.
>>
>>>
>>> Point noted on giving user feedback - I’d actually like to fix things, but currently its just too hard to submit fixes other than pull requests for documentation that is sitting in git hub (and I have submitted those). I know you are working on a simpler way to contribute (and getting the GitHub process smoothed out is in progress so its not fair to comment on that yet as its early days - and much appreciated.
>>>
>>> The Alt-Tab issue in the tracker since 2015 - but I will try and report others and possibly Calypso might have some better keyboard navigation tricks that help.
>>
>> do you have the bug entry?
>>
>>>
>>> For website documentation - I’ll take it back, as I went back again just now and everything I could think of was there (a lot in the news section which I hadn’t noticed). I do think the download page is a bit confusing but I know there is work being done on that.
>>>
>>> Tim
>>>
>>>
>>>>> On 6 Jul 2017, at 19:11, Stephane Ducasse <[hidden email]> wrote:
>>>>>
>>>>> Only 2 minor items stick out as missing:
>>>>>
>>>>> 1) Continuing to improve keyboard shortcut support (its a lot better, but
>>>>> not quite completed - I really miss some shortcuts, particularly the ability
>>>>> to meta-tab between windows - ALT-Tab only works in some windows, and widen
>>>>> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
>>>>> word, a statement, a function etc)
>>>>
>>>> There are three issues there:
>>>> - moving from the vm the keyboard generation (this will be done via
>>>> OSWindow and the VM can focus its main job: execution).
>>>> - then there is the way we declare/manage shortcuts: this is an
>>>> ongoing efforts where everybody can join. We should remove the
>>>> hardcoded shortcut and turn them into KeyBindings
>>>> - Most of the tools will be throw away when bloc will be integrated.
>>>> Now it does not mean that we should not do it.
>>>>
>>>> Now an important point: as a user you should report but you can also
>>>> contribute.
>>>> This is super difficult for me to fix something that I have no clue
>>>> how to reproduce/never encounter.
>>>> You see the mental model of a real open-source movement is to share
>>>> but also to produce together and the point is
>>>> what is the reward model: why should I spend time on pain I do not
>>>> feel. Of course we are already doing a lot
>>>> of actions that is not focus on our immediate needs and we should
>>>> improve the situation but I wanted to share
>>>> with you this perspective.
>>>>
>>>>
>>>>>
>>>>> 2) Keeping the website documentation more up to date (again the pharo.org
>>>>> website is very slick, and a great showcase, however often there are late
>>>>> breaking changes which new users won’t know about unless they trace through
>>>>> the news groups or subscribe to some blogs). If we could also focus on
>>>>> keeping it simple but up to date - that might help.
>>>>
>>>> Where is it not up to date?
>>>> We are always reacting to any broken piece there.
>>>> You see producing Pharo by Example 5 was A LOT of work and I thank
>>>> again Nicolai Hesse for all his huge efforts.
>>>>
>>>>
>>>>> I also have 1 small query - the references to the text editor, is this
>>>>> making the editor that was demo’d at PharoDays 2107 integrated? As that one
>>>>> was very cool?
>>>>
>>>> Yes calypso will replace our old friend Nautilus and we will certainly
>>>> need to put a lot of love into calypso to get
>>>> it to the right level. Now the basis is sound.
>>>> About the node we will revisit smart suggestions and indeed sometimes
>>>> there are problems to catch the correct AST nodes.
>>>>
>>>>> I don’t know how extensible it is (I’m hoping so - and that
>>>>> it might help making it easier to improve code editing tools such that
>>>>> refactorings can more intelligently understand where your cursor is with
>>>>> regards to the AST node underneath it, or we can add more intellisense
>>>>> options that match some of the things we see in IntelliJ).
>>>>>
>>>>> Tim
>>>>>
>>>>> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> TxText removal is already done too.
>>>>>
>>>>> -- Pavel
>>>>>
>>>>> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>>>>>
>>>>>> We would like to share this list with you and get your feedback and
>>>>>> inputs.
>>>>>> It will be presented and discussed again at ESUG.
>>>>>>
>>>>>> Stef on the behalf of the engineering team of the Pharo consortium.
>>>>>>
>>>>>> # Pharo 7 (and 8) potential roadmap
>>>>>>
>>>>>> This document contains a list of actions that should be done during
>>>>>> Pharo 7 and 8.
>>>>>> It is not a definitive list. It is the result of a discussion within
>>>>>> the engineer team and RModers.
>>>>>> It first lists actions that should be performed at the image level
>>>>>> then lists the actions for the VM.
>>>>>>
>>>>>>
>>>>>> ## Image
>>>>>> As a general principle, we will try to remove something when we add a
>>>>>> new features or version of a component.
>>>>>>
>>>>>> ### New tools
>>>>>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>>>>>> For the moment it supports Git. This tool will become central to the
>>>>>> development of projects in Pharo. The key and first enhancements will
>>>>>> be:
>>>>>> - cherry picking
>>>>>> - multiple directories support, subtrees
>>>>>> - better new development process support
>>>>>>
>>>>>> - Calypso: Calypso is a new tool suite for editing and navigating
>>>>>> code. It is modular and can easily be extended.
>>>>>> - integration, set as the default browser
>>>>>>
>>>>>> - Hermes (binary loader): Hermes is a code loader that does not
>>>>>> require the compiler to be loaded. It will be used to speed up the
>>>>>> bootstrap.
>>>>>>
>>>>>> - Beacon: Beacon is an announcement-based logger. It should be
>>>>>> introduced but in addition the left over of previous logging should be
>>>>>> cleaned and removed for example many of the transcript show: should be
>>>>>> removed.
>>>>>>
>>>>>> - Inspector refreshing: the inspector should refresh its values by
>>>>>> default.
>>>>>>
>>>>>> - Cargo: Cargo is a new package manager. It supports the expression of
>>>>>> dependencies between packages. We are currently validating that it can
>>>>>> support conditional loading (platform) and building new tooling to
>>>>>> express and validate data.
>>>>>>
>>>>>> - Check dependencies when committing. Pharo comes with a dependency
>>>>>> analyser tools. Such tools should be used before commiting to warn the
>>>>>> user when a new dependency is introduced in a package.
>>>>>>
>>>>>> ### Cleaning bloat
>>>>>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>>>>>> similar fetaures than Nautilus. Nautilus should be removed.
>>>>>> - Remove old text editor: there is only one or two widgets still using
>>>>>> the old text editor and the rubric text editor.
>>>>>> - Remove TxText: TxText was an attempt to get a new generation text
>>>>>> editor. Now it has been rewritten with a new design in Bloc so we
>>>>>> should remove it since Bloc editor is better and actively maintained.
>>>>>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>>>>>> therefore Komitter can be safely removed from Pharo.
>>>>>> - Remove system categorizer: The old system categorizer is not used
>>>>>> anymore and should be remove.
>>>>>>
>>>>>> - Old compiler removal: The old compiler should now get unloaded from
>>>>>> Pharo.
>>>>>> - The old compiler should be moved to an external package and make
>>>>>> sure that it can be reloaded.
>>>>>> - In addition the encoders should be separated. (@@ more here@@)
>>>>>>
>>>>>> - Old inspector cleanup: we should remove the eyeInspector framework
>>>>>> but we should introduce a minimal fallback inspector. This inspector
>>>>>> should work without Spec or any frameworks.
>>>>>>
>>>>>> - Clean behavior protocol. The number of methods in the core Behavior,
>>>>>> ClassDescription and Class requires some analysis and cleaning.
>>>>>>
>>>>>> - Kernel modularization: the effort on modularising the packages
>>>>>> should be continued.
>>>>>>
>>>>>> ### Language infrastructure
>>>>>> The following points are more related to the infrastructure of
>>>>>> manipulating and loading class definitions. There are the basis for
>>>>>> the future module system and cleaning some often hidden parts of the
>>>>>> system.
>>>>>>
>>>>>> - New class definition: The class definition is not scaling anymore
>>>>>> due to the large number of options (traits, slot, tage, package,
>>>>>> immediate, ephemeron). A fluid-based message API should be designed.
>>>>>>
>>>>>> - Support for Undefined classes: When loading old code or code whose
>>>>>> superclass has not yet being loaded, the system has inconsistent
>>>>>> behavior. Depending on the tools, it may not load the code, raise a
>>>>>> warning or decide that the superclass is Object without any other
>>>>>> notice and losing the name of the original superclass. We are working
>>>>>> on a new mechanism to support a unique way to handle such case. To be
>>>>>> presented at IWST/ESUG
>>>>>>
>>>>>> - Class definition parser: Class definition is parser in different
>>>>>> place of the system and in addition the output is the direct creation
>>>>>> of a class object instead of an object representing the class
>>>>>> definition that can be further manipulated. We are working on class
>>>>>> definition parser. It produces a separate AST. It will help the
>>>>>> building of tools.
>>>>>>
>>>>>> - Better update infrastructure. Pablo Tesone has been working on a
>>>>>> better update mechanism, better modular class builder.
>>>>>>
>>>>>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>>>>>> running in the current system. It is useful to perform analysis (such
>>>>>> as browsing, navigating off-line or remote definitions) before
>>>>>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>>>>>> support the bootstrap of Pharo 60. The results is Ring2.
>>>>>>
>>>>>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>>>>>> better the warning and a general enhancement pass is needed.
>>>>>>
>>>>>> ### System enhancements
>>>>>>
>>>>>> - Commandline enhancements. RMOD is currently improving the
>>>>>> command-line infrastructure and making sure that the system can work
>>>>>> in read-only mode.
>>>>>>
>>>>>> - Cleaning streams: the idea is to make sure that the system does not
>>>>>> use the old streams. The idea is to start using the fileStreams and
>>>>>> make sure that the Stream package can be substituated in the future by
>>>>>> other streams with the same API. Therefore hardcoded class names
>>>>>> should be substituted with factory (readSream writeStream). From that
>>>>>> perspective we do not think that it is wise to introduce Xtreams. We
>>>>>> should analyse the API of the current implementation.
>>>>>>
>>>>>> - New theme from the beginning. It is really important that each
>>>>>> version of Pharo is identified open the default opening. Pharo should
>>>>>> come up with two default themes: one light and one dark.
>>>>>>
>>>>>> - Better themes/palettes support
>>>>>> - Better and nicer Tabs. Tabs design should be revisited.
>>>>>>
>>>>>> - OSWindow world rendering: the effort to remove the logic of the
>>>>>> windowing from the VM should be continued. OSWindow should be used
>>>>>> instead.
>>>>>>
>>>>>> - Freetype: The current freetype plugin is the source of many bugs and
>>>>>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>>>>>> (@@to verify@@).
>>>>>>
>>>>>>
>>>>>> - Refactoring 2: Refactoring 2 is an improved version of the
>>>>>> refactorings developed by Gustavos Santos and they should used to
>>>>>> replace the existing one.
>>>>>>
>>>>>>
>>>>>> ## VM
>>>>>>
>>>>>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>>>>>> over a year and since June 2017 a Windows 64 bits VM has been working.
>>>>>> The plan is to stabilize each part of Pharo that is not working in 64
>>>>>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>>>>>> the default download for Pharo.
>>>>>>
>>>>>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>>>>>> we need to merge and check everything works fine. With the headless
>>>>>> VM, Pharo can be run in true headless mode (not with a hidden window)
>>>>>> and it is required for future VM features (embeddable VM, ...).
>>>>>>
>>>>>> - Embeddable VM: The VM should be able to be embedded in external
>>>>>> applications, the application accessing objects through well-defined
>>>>>> APIs.
>>>>>>
>>>>>> - Idle VM: The goal is to avoid the active waiting loop consuming
>>>>>> several per cent of cpu time when the Pharo image is not doing
>>>>>> anything.
>>>>>>
>>>>>> - Android VM: integration of the Android VM build and tests in the
>>>>>> integration setup.
>>>>>>
>>>>>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>>>>>> prototype of Eliot)
>>>>>>
>>>>>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>>>>>> servers
>>>>>>
>>>>>> - Better support for large heaps (GC tuning API, incremental GC).
>>>>>> Pharo 64 bit is now able to manage large heap. However better
>>>>>> performance can be proposed by offering better settings for the
>>>>>> different GC zone.
>>>>>>
>>>>>> - Integrate various fixes to support better high resolution display.
>>>>>> In Retina mac display Pharo looks blurry. The fix to support Retina
>>>>>> should be integrated in the VM.
>>>>>>
>>>>>> ### Sista-related
>>>>>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>>>>>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>>>>>> JIT is manipulating code (folding constant, rewriting it) before
>>>>>> compiling it to assembly code.
>>>>>>
>>>>>> - New Block Closure implementation by default: Allows one to implement
>>>>>> in the Opal compiler the copying and clean blocks optimisations,
>>>>>> reduce massively the complexity of some parts of the JIT (debugging
>>>>>> API to map machine code pc to bytecode pc for example) and is required
>>>>>> for the Sista support. Some work remains in debugging/IDE support.
>>>>>>
>>>>>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>>>>>> the code base of the bytecode to source code decompiler, the debugger,
>>>>>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>>>>>> and is required for the Sista support. Some work remains in debugging
>>>>>> support.
>>>>>>
>>>>>> - Read-only objects: They work in Pharo 6, but the in-image support
>>>>>> needs to be improved (fall-back code of primitives, etc.) and some
>>>>>> polishing needs to be done (primitive error code not always correct,
>>>>>> etc.). Used for the support of different project, including GBS.
>>>>>>
>>>>>> - Literals immutability (based on read-only objects). In particular an
>>>>>> analysis of the image usage of literal (string mutation) is needed.
>>>>>>
>>>>>> - Sista preview: A first version of Sista will be integrated, yielding
>>>>>> 1.5x performance boost on most applications, but will be optional and
>>>>>> will require specific constraints (not toying too much with literal
>>>>>> mutability, partial IDE support, etc.).
>>>>>>
>>>>>>
>>>>>> ### Already done
>>>>>>
>>>>>> - [DONE] Remove Shoreline reporter.
>>>>>> - [DONE] Kernel should not depend on Traits: This will speed up the
>>>>>> bootstrap and support the modular introduction of alternate traits
>>>>>> implementation currently designed by Pablo Tesone and tested in the
>>>>>> new generation of Moose.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Clément Béra
The incremental GC is already mentioned in "Better support for large heaps (GC tuning API, incremental GC)". Now we have a second paragraph about the incremental GC, that's redundant and the form of the paragraph is not consistent with the rest of the roadmap where each feature has 1 or 2 lines of explanations, not 10 lines.

Same thing, the Sista-related introduction is 10 lines instead of 2-3, which is not consistent with the form of the rest of the roadmap. 

Inconsistent form makes the document look unprofessional. We need to make it consistent, we can't let it like that.

On Sat, Jul 8, 2017 at 8:22 AM, Stephane Ducasse <[hidden email]> wrote:
> Actually that's a good point - should the roadmap encompass what the community can offer too (i hadn't appreciated the distinction )?

Yes it does.
Esteban (and christophe 60% paid by rmod) and guille (60% but also
writing papers) cannot do all that alone?


> We collectively pay for some things to be achieved, but also we expect that we can contribute some of the lower hanging fruit.

Yes and even more.
We at RMOD are not paid to produce Pharo. Pharo only represents a
small part of our job evaluation.
Now if everybody takes 1 hour per week to improve Pharo it will be huge.


> Of course some things are at the whim of what itches to scratch, but others we may be able to rally around?
>
> Tim
>
> Sent from my iPhone
>
>> On 7 Jul 2017, at 06:11, Stephane Ducasse <[hidden email]> wrote:
>>
>>> On Thu, Jul 6, 2017 at 11:27 PM, Tim Mackinnon <[hidden email]> wrote:
>>> I didn’t mean to touch a nerve - and this was why I wrote “minor” points - but you did ask for feedback…
>>
>> you did not :)
>> Just that if we just count on fully book engineers working on super
>> important features we will not make it.
>> This is why once in a while I sit and fix a boring pop up or related.
>> I hate the stupid pop up for pakcgae selection and one day I will have
>> to retry to kill it.
>>
>>>
>>> Point noted on giving user feedback - I’d actually like to fix things, but currently its just too hard to submit fixes other than pull requests for documentation that is sitting in git hub (and I have submitted those). I know you are working on a simpler way to contribute (and getting the GitHub process smoothed out is in progress so its not fair to comment on that yet as its early days - and much appreciated.
>>>
>>> The Alt-Tab issue in the tracker since 2015 - but I will try and report others and possibly Calypso might have some better keyboard navigation tricks that help.
>>
>> do you have the bug entry?
>>
>>>
>>> For website documentation - I’ll take it back, as I went back again just now and everything I could think of was there (a lot in the news section which I hadn’t noticed). I do think the download page is a bit confusing but I know there is work being done on that.
>>>
>>> Tim
>>>
>>>
>>>>> On 6 Jul 2017, at 19:11, Stephane Ducasse <[hidden email]> wrote:
>>>>>
>>>>> Only 2 minor items stick out as missing:
>>>>>
>>>>> 1) Continuing to improve keyboard shortcut support (its a lot better, but
>>>>> not quite completed - I really miss some shortcuts, particularly the ability
>>>>> to meta-tab between windows - ALT-Tab only works in some windows, and widen
>>>>> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
>>>>> word, a statement, a function etc)
>>>>
>>>> There are three issues there:
>>>> - moving from the vm the keyboard generation (this will be done via
>>>> OSWindow and the VM can focus its main job: execution).
>>>> - then there is the way we declare/manage shortcuts: this is an
>>>> ongoing efforts where everybody can join. We should remove the
>>>> hardcoded shortcut and turn them into KeyBindings
>>>> - Most of the tools will be throw away when bloc will be integrated.
>>>> Now it does not mean that we should not do it.
>>>>
>>>> Now an important point: as a user you should report but you can also
>>>> contribute.
>>>> This is super difficult for me to fix something that I have no clue
>>>> how to reproduce/never encounter.
>>>> You see the mental model of a real open-source movement is to share
>>>> but also to produce together and the point is
>>>> what is the reward model: why should I spend time on pain I do not
>>>> feel. Of course we are already doing a lot
>>>> of actions that is not focus on our immediate needs and we should
>>>> improve the situation but I wanted to share
>>>> with you this perspective.
>>>>
>>>>
>>>>>
>>>>> 2) Keeping the website documentation more up to date (again the pharo.org
>>>>> website is very slick, and a great showcase, however often there are late
>>>>> breaking changes which new users won’t know about unless they trace through
>>>>> the news groups or subscribe to some blogs). If we could also focus on
>>>>> keeping it simple but up to date - that might help.
>>>>
>>>> Where is it not up to date?
>>>> We are always reacting to any broken piece there.
>>>> You see producing Pharo by Example 5 was A LOT of work and I thank
>>>> again Nicolai Hesse for all his huge efforts.
>>>>
>>>>
>>>>> I also have 1 small query - the references to the text editor, is this
>>>>> making the editor that was demo’d at PharoDays 2107 integrated? As that one
>>>>> was very cool?
>>>>
>>>> Yes calypso will replace our old friend Nautilus and we will certainly
>>>> need to put a lot of love into calypso to get
>>>> it to the right level. Now the basis is sound.
>>>> About the node we will revisit smart suggestions and indeed sometimes
>>>> there are problems to catch the correct AST nodes.
>>>>
>>>>> I don’t know how extensible it is (I’m hoping so - and that
>>>>> it might help making it easier to improve code editing tools such that
>>>>> refactorings can more intelligently understand where your cursor is with
>>>>> regards to the AST node underneath it, or we can add more intellisense
>>>>> options that match some of the things we see in IntelliJ).
>>>>>
>>>>> Tim
>>>>>
>>>>> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> TxText removal is already done too.
>>>>>
>>>>> -- Pavel
>>>>>
>>>>> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>>>>>
>>>>>> We would like to share this list with you and get your feedback and
>>>>>> inputs.
>>>>>> It will be presented and discussed again at ESUG.
>>>>>>
>>>>>> Stef on the behalf of the engineering team of the Pharo consortium.
>>>>>>
>>>>>> # Pharo 7 (and 8) potential roadmap
>>>>>>
>>>>>> This document contains a list of actions that should be done during
>>>>>> Pharo 7 and 8.
>>>>>> It is not a definitive list. It is the result of a discussion within
>>>>>> the engineer team and RModers.
>>>>>> It first lists actions that should be performed at the image level
>>>>>> then lists the actions for the VM.
>>>>>>
>>>>>>
>>>>>> ## Image
>>>>>> As a general principle, we will try to remove something when we add a
>>>>>> new features or version of a component.
>>>>>>
>>>>>> ### New tools
>>>>>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>>>>>> For the moment it supports Git. This tool will become central to the
>>>>>> development of projects in Pharo. The key and first enhancements will
>>>>>> be:
>>>>>> - cherry picking
>>>>>> - multiple directories support, subtrees
>>>>>> - better new development process support
>>>>>>
>>>>>> - Calypso: Calypso is a new tool suite for editing and navigating
>>>>>> code. It is modular and can easily be extended.
>>>>>> - integration, set as the default browser
>>>>>>
>>>>>> - Hermes (binary loader): Hermes is a code loader that does not
>>>>>> require the compiler to be loaded. It will be used to speed up the
>>>>>> bootstrap.
>>>>>>
>>>>>> - Beacon: Beacon is an announcement-based logger. It should be
>>>>>> introduced but in addition the left over of previous logging should be
>>>>>> cleaned and removed for example many of the transcript show: should be
>>>>>> removed.
>>>>>>
>>>>>> - Inspector refreshing: the inspector should refresh its values by
>>>>>> default.
>>>>>>
>>>>>> - Cargo: Cargo is a new package manager. It supports the expression of
>>>>>> dependencies between packages. We are currently validating that it can
>>>>>> support conditional loading (platform) and building new tooling to
>>>>>> express and validate data.
>>>>>>
>>>>>> - Check dependencies when committing. Pharo comes with a dependency
>>>>>> analyser tools. Such tools should be used before commiting to warn the
>>>>>> user when a new dependency is introduced in a package.
>>>>>>
>>>>>> ### Cleaning bloat
>>>>>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>>>>>> similar fetaures than Nautilus. Nautilus should be removed.
>>>>>> - Remove old text editor: there is only one or two widgets still using
>>>>>> the old text editor and the rubric text editor.
>>>>>> - Remove TxText: TxText was an attempt to get a new generation text
>>>>>> editor. Now it has been rewritten with a new design in Bloc so we
>>>>>> should remove it since Bloc editor is better and actively maintained.
>>>>>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>>>>>> therefore Komitter can be safely removed from Pharo.
>>>>>> - Remove system categorizer: The old system categorizer is not used
>>>>>> anymore and should be remove.
>>>>>>
>>>>>> - Old compiler removal: The old compiler should now get unloaded from
>>>>>> Pharo.
>>>>>> - The old compiler should be moved to an external package and make
>>>>>> sure that it can be reloaded.
>>>>>> - In addition the encoders should be separated. (@@ more here@@)
>>>>>>
>>>>>> - Old inspector cleanup: we should remove the eyeInspector framework
>>>>>> but we should introduce a minimal fallback inspector. This inspector
>>>>>> should work without Spec or any frameworks.
>>>>>>
>>>>>> - Clean behavior protocol. The number of methods in the core Behavior,
>>>>>> ClassDescription and Class requires some analysis and cleaning.
>>>>>>
>>>>>> - Kernel modularization: the effort on modularising the packages
>>>>>> should be continued.
>>>>>>
>>>>>> ### Language infrastructure
>>>>>> The following points are more related to the infrastructure of
>>>>>> manipulating and loading class definitions. There are the basis for
>>>>>> the future module system and cleaning some often hidden parts of the
>>>>>> system.
>>>>>>
>>>>>> - New class definition: The class definition is not scaling anymore
>>>>>> due to the large number of options (traits, slot, tage, package,
>>>>>> immediate, ephemeron). A fluid-based message API should be designed.
>>>>>>
>>>>>> - Support for Undefined classes: When loading old code or code whose
>>>>>> superclass has not yet being loaded, the system has inconsistent
>>>>>> behavior. Depending on the tools, it may not load the code, raise a
>>>>>> warning or decide that the superclass is Object without any other
>>>>>> notice and losing the name of the original superclass. We are working
>>>>>> on a new mechanism to support a unique way to handle such case. To be
>>>>>> presented at IWST/ESUG
>>>>>>
>>>>>> - Class definition parser: Class definition is parser in different
>>>>>> place of the system and in addition the output is the direct creation
>>>>>> of a class object instead of an object representing the class
>>>>>> definition that can be further manipulated. We are working on class
>>>>>> definition parser. It produces a separate AST. It will help the
>>>>>> building of tools.
>>>>>>
>>>>>> - Better update infrastructure. Pablo Tesone has been working on a
>>>>>> better update mechanism, better modular class builder.
>>>>>>
>>>>>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>>>>>> running in the current system. It is useful to perform analysis (such
>>>>>> as browsing, navigating off-line or remote definitions) before
>>>>>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>>>>>> support the bootstrap of Pharo 60. The results is Ring2.
>>>>>>
>>>>>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>>>>>> better the warning and a general enhancement pass is needed.
>>>>>>
>>>>>> ### System enhancements
>>>>>>
>>>>>> - Commandline enhancements. RMOD is currently improving the
>>>>>> command-line infrastructure and making sure that the system can work
>>>>>> in read-only mode.
>>>>>>
>>>>>> - Cleaning streams: the idea is to make sure that the system does not
>>>>>> use the old streams. The idea is to start using the fileStreams and
>>>>>> make sure that the Stream package can be substituated in the future by
>>>>>> other streams with the same API. Therefore hardcoded class names
>>>>>> should be substituted with factory (readSream writeStream). From that
>>>>>> perspective we do not think that it is wise to introduce Xtreams. We
>>>>>> should analyse the API of the current implementation.
>>>>>>
>>>>>> - New theme from the beginning. It is really important that each
>>>>>> version of Pharo is identified open the default opening. Pharo should
>>>>>> come up with two default themes: one light and one dark.
>>>>>>
>>>>>> - Better themes/palettes support
>>>>>> - Better and nicer Tabs. Tabs design should be revisited.
>>>>>>
>>>>>> - OSWindow world rendering: the effort to remove the logic of the
>>>>>> windowing from the VM should be continued. OSWindow should be used
>>>>>> instead.
>>>>>>
>>>>>> - Freetype: The current freetype plugin is the source of many bugs and
>>>>>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>>>>>> (@@to verify@@).
>>>>>>
>>>>>>
>>>>>> - Refactoring 2: Refactoring 2 is an improved version of the
>>>>>> refactorings developed by Gustavos Santos and they should used to
>>>>>> replace the existing one.
>>>>>>
>>>>>>
>>>>>> ## VM
>>>>>>
>>>>>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>>>>>> over a year and since June 2017 a Windows 64 bits VM has been working.
>>>>>> The plan is to stabilize each part of Pharo that is not working in 64
>>>>>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>>>>>> the default download for Pharo.
>>>>>>
>>>>>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>>>>>> we need to merge and check everything works fine. With the headless
>>>>>> VM, Pharo can be run in true headless mode (not with a hidden window)
>>>>>> and it is required for future VM features (embeddable VM, ...).
>>>>>>
>>>>>> - Embeddable VM: The VM should be able to be embedded in external
>>>>>> applications, the application accessing objects through well-defined
>>>>>> APIs.
>>>>>>
>>>>>> - Idle VM: The goal is to avoid the active waiting loop consuming
>>>>>> several per cent of cpu time when the Pharo image is not doing
>>>>>> anything.
>>>>>>
>>>>>> - Android VM: integration of the Android VM build and tests in the
>>>>>> integration setup.
>>>>>>
>>>>>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>>>>>> prototype of Eliot)
>>>>>>
>>>>>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>>>>>> servers
>>>>>>
>>>>>> - Better support for large heaps (GC tuning API, incremental GC).
>>>>>> Pharo 64 bit is now able to manage large heap. However better
>>>>>> performance can be proposed by offering better settings for the
>>>>>> different GC zone.
>>>>>>
>>>>>> - Integrate various fixes to support better high resolution display.
>>>>>> In Retina mac display Pharo looks blurry. The fix to support Retina
>>>>>> should be integrated in the VM.
>>>>>>
>>>>>> ### Sista-related
>>>>>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>>>>>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>>>>>> JIT is manipulating code (folding constant, rewriting it) before
>>>>>> compiling it to assembly code.
>>>>>>
>>>>>> - New Block Closure implementation by default: Allows one to implement
>>>>>> in the Opal compiler the copying and clean blocks optimisations,
>>>>>> reduce massively the complexity of some parts of the JIT (debugging
>>>>>> API to map machine code pc to bytecode pc for example) and is required
>>>>>> for the Sista support. Some work remains in debugging/IDE support.
>>>>>>
>>>>>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>>>>>> the code base of the bytecode to source code decompiler, the debugger,
>>>>>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>>>>>> and is required for the Sista support. Some work remains in debugging
>>>>>> support.
>>>>>>
>>>>>> - Read-only objects: They work in Pharo 6, but the in-image support
>>>>>> needs to be improved (fall-back code of primitives, etc.) and some
>>>>>> polishing needs to be done (primitive error code not always correct,
>>>>>> etc.). Used for the support of different project, including GBS.
>>>>>>
>>>>>> - Literals immutability (based on read-only objects). In particular an
>>>>>> analysis of the image usage of literal (string mutation) is needed.
>>>>>>
>>>>>> - Sista preview: A first version of Sista will be integrated, yielding
>>>>>> 1.5x performance boost on most applications, but will be optional and
>>>>>> will require specific constraints (not toying too much with literal
>>>>>> mutability, partial IDE support, etc.).
>>>>>>
>>>>>>
>>>>>> ### Already done
>>>>>>
>>>>>> - [DONE] Remove Shoreline reporter.
>>>>>> - [DONE] Kernel should not depend on Traits: This will speed up the
>>>>>> bootstrap and support the modular introduction of alternate traits
>>>>>> implementation currently designed by Pablo Tesone and tested in the
>>>>>> new generation of Moose.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Discussing the roadmap

Tim Mackinnon
In reply to this post by Stephane Ducasse-3
So should/could we take the roadmap and put two main headings:

Consortium
Community

And under those, headings we put the things you have suggested and try to order them a bit?

It might be nice at PharoDays next year to be able to point to that list and show show what the consortium made possible (which you certainly did well this year), but also what the wider community achieved over and above all the excellent libraries that keep appearing.

Tim

Sent from my iPhone

On 8 Jul 2017, at 07:22, Stephane Ducasse <[hidden email]> wrote:

>> Actually that's a good point - should the roadmap encompass what the community can offer too (i hadn't appreciated the distinction )?
>
> Yes it does.
> Esteban (and christophe 60% paid by rmod) and guille (60% but also
> writing papers) cannot do all that alone?
>
>
>> We collectively pay for some things to be achieved, but also we expect that we can contribute some of the lower hanging fruit.
>
> Yes and even more.
> We at RMOD are not paid to produce Pharo. Pharo only represents a
> small part of our job evaluation.
> Now if everybody takes 1 hour per week to improve Pharo it will be huge.
>
>
>> Of course some things are at the whim of what itches to scratch, but others we may be able to rally around?
>>
>> Tim
>>
>> Sent from my iPhone
>>
>>>> On 7 Jul 2017, at 06:11, Stephane Ducasse <[hidden email]> wrote:
>>>>
>>>> On Thu, Jul 6, 2017 at 11:27 PM, Tim Mackinnon <[hidden email]> wrote:
>>>> I didn’t mean to touch a nerve - and this was why I wrote “minor” points - but you did ask for feedback…
>>>
>>> you did not :)
>>> Just that if we just count on fully book engineers working on super
>>> important features we will not make it.
>>> This is why once in a while I sit and fix a boring pop up or related.
>>> I hate the stupid pop up for pakcgae selection and one day I will have
>>> to retry to kill it.
>>>
>>>>
>>>> Point noted on giving user feedback - I’d actually like to fix things, but currently its just too hard to submit fixes other than pull requests for documentation that is sitting in git hub (and I have submitted those). I know you are working on a simpler way to contribute (and getting the GitHub process smoothed out is in progress so its not fair to comment on that yet as its early days - and much appreciated.
>>>>
>>>> The Alt-Tab issue in the tracker since 2015 - but I will try and report others and possibly Calypso might have some better keyboard navigation tricks that help.
>>>
>>> do you have the bug entry?
>>>
>>>>
>>>> For website documentation - I’ll take it back, as I went back again just now and everything I could think of was there (a lot in the news section which I hadn’t noticed). I do think the download page is a bit confusing but I know there is work being done on that.
>>>>
>>>> Tim
>>>>
>>>>
>>>>>> On 6 Jul 2017, at 19:11, Stephane Ducasse <[hidden email]> wrote:
>>>>>>
>>>>>> Only 2 minor items stick out as missing:
>>>>>>
>>>>>> 1) Continuing to improve keyboard shortcut support (its a lot better, but
>>>>>> not quite completed - I really miss some shortcuts, particularly the ability
>>>>>> to meta-tab between windows - ALT-Tab only works in some windows, and widen
>>>>>> selection in the editor - e.g. CMD-W in IntelliJ to increasing select a
>>>>>> word, a statement, a function etc)
>>>>>
>>>>> There are three issues there:
>>>>> - moving from the vm the keyboard generation (this will be done via
>>>>> OSWindow and the VM can focus its main job: execution).
>>>>> - then there is the way we declare/manage shortcuts: this is an
>>>>> ongoing efforts where everybody can join. We should remove the
>>>>> hardcoded shortcut and turn them into KeyBindings
>>>>> - Most of the tools will be throw away when bloc will be integrated.
>>>>> Now it does not mean that we should not do it.
>>>>>
>>>>> Now an important point: as a user you should report but you can also
>>>>> contribute.
>>>>> This is super difficult for me to fix something that I have no clue
>>>>> how to reproduce/never encounter.
>>>>> You see the mental model of a real open-source movement is to share
>>>>> but also to produce together and the point is
>>>>> what is the reward model: why should I spend time on pain I do not
>>>>> feel. Of course we are already doing a lot
>>>>> of actions that is not focus on our immediate needs and we should
>>>>> improve the situation but I wanted to share
>>>>> with you this perspective.
>>>>>
>>>>>
>>>>>>
>>>>>> 2) Keeping the website documentation more up to date (again the pharo.org
>>>>>> website is very slick, and a great showcase, however often there are late
>>>>>> breaking changes which new users won’t know about unless they trace through
>>>>>> the news groups or subscribe to some blogs). If we could also focus on
>>>>>> keeping it simple but up to date - that might help.
>>>>>
>>>>> Where is it not up to date?
>>>>> We are always reacting to any broken piece there.
>>>>> You see producing Pharo by Example 5 was A LOT of work and I thank
>>>>> again Nicolai Hesse for all his huge efforts.
>>>>>
>>>>>
>>>>>> I also have 1 small query - the references to the text editor, is this
>>>>>> making the editor that was demo’d at PharoDays 2107 integrated? As that one
>>>>>> was very cool?
>>>>>
>>>>> Yes calypso will replace our old friend Nautilus and we will certainly
>>>>> need to put a lot of love into calypso to get
>>>>> it to the right level. Now the basis is sound.
>>>>> About the node we will revisit smart suggestions and indeed sometimes
>>>>> there are problems to catch the correct AST nodes.
>>>>>
>>>>>> I don’t know how extensible it is (I’m hoping so - and that
>>>>>> it might help making it easier to improve code editing tools such that
>>>>>> refactorings can more intelligently understand where your cursor is with
>>>>>> regards to the AST node underneath it, or we can add more intellisense
>>>>>> options that match some of the things we see in IntelliJ).
>>>>>>
>>>>>> Tim
>>>>>>
>>>>>> On 6 Jul 2017, at 13:00, Pavel Krivanek <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> TxText removal is already done too.
>>>>>>
>>>>>> -- Pavel
>>>>>>
>>>>>> 2017-07-06 10:55 GMT+02:00 Stephane Ducasse <[hidden email]>:
>>>>>>>
>>>>>>> We would like to share this list with you and get your feedback and
>>>>>>> inputs.
>>>>>>> It will be presented and discussed again at ESUG.
>>>>>>>
>>>>>>> Stef on the behalf of the engineering team of the Pharo consortium.
>>>>>>>
>>>>>>> # Pharo 7 (and 8) potential roadmap
>>>>>>>
>>>>>>> This document contains a list of actions that should be done during
>>>>>>> Pharo 7 and 8.
>>>>>>> It is not a definitive list. It is the result of a discussion within
>>>>>>> the engineer team and RModers.
>>>>>>> It first lists actions that should be performed at the image level
>>>>>>> then lists the actions for the VM.
>>>>>>>
>>>>>>>
>>>>>>> ## Image
>>>>>>> As a general principle, we will try to remove something when we add a
>>>>>>> new features or version of a component.
>>>>>>>
>>>>>>> ### New tools
>>>>>>> - Iceberg: Iceberg is the new tool suite to handle new VCS in Pharo.
>>>>>>> For the moment it supports Git. This tool will become central to the
>>>>>>> development of projects in Pharo. The key and first enhancements will
>>>>>>> be:
>>>>>>> - cherry picking
>>>>>>> - multiple directories support, subtrees
>>>>>>> - better new development process support
>>>>>>>
>>>>>>> - Calypso: Calypso is a new tool suite for editing and navigating
>>>>>>> code. It is modular and can easily be extended.
>>>>>>> - integration, set as the default browser
>>>>>>>
>>>>>>> - Hermes (binary loader): Hermes is a code loader that does not
>>>>>>> require the compiler to be loaded. It will be used to speed up the
>>>>>>> bootstrap.
>>>>>>>
>>>>>>> - Beacon: Beacon is an announcement-based logger. It should be
>>>>>>> introduced but in addition the left over of previous logging should be
>>>>>>> cleaned and removed for example many of the transcript show: should be
>>>>>>> removed.
>>>>>>>
>>>>>>> - Inspector refreshing: the inspector should refresh its values by
>>>>>>> default.
>>>>>>>
>>>>>>> - Cargo: Cargo is a new package manager. It supports the expression of
>>>>>>> dependencies between packages. We are currently validating that it can
>>>>>>> support conditional loading (platform) and building new tooling to
>>>>>>> express and validate data.
>>>>>>>
>>>>>>> - Check dependencies when committing. Pharo comes with a dependency
>>>>>>> analyser tools. Such tools should be used before commiting to warn the
>>>>>>> user when a new dependency is introduced in a package.
>>>>>>>
>>>>>>> ### Cleaning bloat
>>>>>>> - Removing of Nautilus: Once Calypso will be integrated and exhibit
>>>>>>> similar fetaures than Nautilus. Nautilus should be removed.
>>>>>>> - Remove old text editor: there is only one or two widgets still using
>>>>>>> the old text editor and the rubric text editor.
>>>>>>> - Remove TxText: TxText was an attempt to get a new generation text
>>>>>>> editor. Now it has been rewritten with a new design in Bloc so we
>>>>>>> should remove it since Bloc editor is better and actively maintained.
>>>>>>> - Remove Komitter: Iceberg already supports cherrypicking on commit
>>>>>>> therefore Komitter can be safely removed from Pharo.
>>>>>>> - Remove system categorizer: The old system categorizer is not used
>>>>>>> anymore and should be remove.
>>>>>>>
>>>>>>> - Old compiler removal: The old compiler should now get unloaded from
>>>>>>> Pharo.
>>>>>>> - The old compiler should be moved to an external package and make
>>>>>>> sure that it can be reloaded.
>>>>>>> - In addition the encoders should be separated. (@@ more here@@)
>>>>>>>
>>>>>>> - Old inspector cleanup: we should remove the eyeInspector framework
>>>>>>> but we should introduce a minimal fallback inspector. This inspector
>>>>>>> should work without Spec or any frameworks.
>>>>>>>
>>>>>>> - Clean behavior protocol. The number of methods in the core Behavior,
>>>>>>> ClassDescription and Class requires some analysis and cleaning.
>>>>>>>
>>>>>>> - Kernel modularization: the effort on modularising the packages
>>>>>>> should be continued.
>>>>>>>
>>>>>>> ### Language infrastructure
>>>>>>> The following points are more related to the infrastructure of
>>>>>>> manipulating and loading class definitions. There are the basis for
>>>>>>> the future module system and cleaning some often hidden parts of the
>>>>>>> system.
>>>>>>>
>>>>>>> - New class definition: The class definition is not scaling anymore
>>>>>>> due to the large number of options (traits, slot, tage, package,
>>>>>>> immediate, ephemeron). A fluid-based message API should be designed.
>>>>>>>
>>>>>>> - Support for Undefined classes: When loading old code or code whose
>>>>>>> superclass has not yet being loaded, the system has inconsistent
>>>>>>> behavior. Depending on the tools, it may not load the code, raise a
>>>>>>> warning or decide that the superclass is Object without any other
>>>>>>> notice and losing the name of the original superclass. We are working
>>>>>>> on a new mechanism to support a unique way to handle such case. To be
>>>>>>> presented at IWST/ESUG
>>>>>>>
>>>>>>> - Class definition parser: Class definition is parser in different
>>>>>>> place of the system and in addition the output is the direct creation
>>>>>>> of a class object instead of an object representing the class
>>>>>>> definition that can be further manipulated. We are working on class
>>>>>>> definition parser. It produces a separate AST. It will help the
>>>>>>> building of tools.
>>>>>>>
>>>>>>> - Better update infrastructure. Pablo Tesone has been working on a
>>>>>>> better update mechanism, better modular class builder.
>>>>>>>
>>>>>>> - Ring2: Ring is a metamodel to manipulate code that is not actively
>>>>>>> running in the current system. It is useful to perform analysis (such
>>>>>>> as browsing, navigating off-line or remote definitions) before
>>>>>>> actually loading the code. Ring got redesigned by Pavel Krivanek to
>>>>>>> support the bootstrap of Pharo 60. The results is Ring2.
>>>>>>>
>>>>>>> - Opal is the Pharo Compiler. It should be enhanced to support handle
>>>>>>> better the warning and a general enhancement pass is needed.
>>>>>>>
>>>>>>> ### System enhancements
>>>>>>>
>>>>>>> - Commandline enhancements. RMOD is currently improving the
>>>>>>> command-line infrastructure and making sure that the system can work
>>>>>>> in read-only mode.
>>>>>>>
>>>>>>> - Cleaning streams: the idea is to make sure that the system does not
>>>>>>> use the old streams. The idea is to start using the fileStreams and
>>>>>>> make sure that the Stream package can be substituated in the future by
>>>>>>> other streams with the same API. Therefore hardcoded class names
>>>>>>> should be substituted with factory (readSream writeStream). From that
>>>>>>> perspective we do not think that it is wise to introduce Xtreams. We
>>>>>>> should analyse the API of the current implementation.
>>>>>>>
>>>>>>> - New theme from the beginning. It is really important that each
>>>>>>> version of Pharo is identified open the default opening. Pharo should
>>>>>>> come up with two default themes: one light and one dark.
>>>>>>>
>>>>>>> - Better themes/palettes support
>>>>>>> - Better and nicer Tabs. Tabs design should be revisited.
>>>>>>>
>>>>>>> - OSWindow world rendering: the effort to remove the logic of the
>>>>>>> windowing from the VM should be continued. OSWindow should be used
>>>>>>> instead.
>>>>>>>
>>>>>>> - Freetype: The current freetype plugin is the source of many bugs and
>>>>>>> problem. Thibault Raffaillac used FFI to do direct call to openGL
>>>>>>> (@@to verify@@).
>>>>>>>
>>>>>>>
>>>>>>> - Refactoring 2: Refactoring 2 is an improved version of the
>>>>>>> refactorings developed by Gustavos Santos and they should used to
>>>>>>> replace the existing one.
>>>>>>>
>>>>>>>
>>>>>>> ## VM
>>>>>>>
>>>>>>> - 64-bits by default: Mac and Linux 64 bits VMs have been working for
>>>>>>> over a year and since June 2017 a Windows 64 bits VM has been working.
>>>>>>> The plan is to stabilize each part of Pharo that is not working in 64
>>>>>>> bits as well as in 32 bits and make the 64 bits Pharo images and VM
>>>>>>> the default download for Pharo.
>>>>>>>
>>>>>>> - Headless VM: Ronie Salgado has built a headless VM on his VM branch,
>>>>>>> we need to merge and check everything works fine. With the headless
>>>>>>> VM, Pharo can be run in true headless mode (not with a hidden window)
>>>>>>> and it is required for future VM features (embeddable VM, ...).
>>>>>>>
>>>>>>> - Embeddable VM: The VM should be able to be embedded in external
>>>>>>> applications, the application accessing objects through well-defined
>>>>>>> APIs.
>>>>>>>
>>>>>>> - Idle VM: The goal is to avoid the active waiting loop consuming
>>>>>>> several per cent of cpu time when the Pharo image is not doing
>>>>>>> anything.
>>>>>>>
>>>>>>> - Android VM: integration of the Android VM build and tests in the
>>>>>>> integration setup.
>>>>>>>
>>>>>>> - Threaded FFI: all FFI calls should be non-blocking (reviving
>>>>>>> prototype of Eliot)
>>>>>>>
>>>>>>> - ZeroConf for ARM: The ARM VM should be available from the zeroConf
>>>>>>> servers
>>>>>>>
>>>>>>> - Better support for large heaps (GC tuning API, incremental GC).
>>>>>>> Pharo 64 bit is now able to manage large heap. However better
>>>>>>> performance can be proposed by offering better settings for the
>>>>>>> different GC zone.
>>>>>>>
>>>>>>> - Integrate various fixes to support better high resolution display.
>>>>>>> In Retina mac display Pharo looks blurry. The fix to support Retina
>>>>>>> should be integrated in the VM.
>>>>>>>
>>>>>>> ### Sista-related
>>>>>>> Sista is an optimizing JIT for Pharo. It is the result of multiple
>>>>>>> years of developement by Clement Bera and Eliot Miranda. An optimizing
>>>>>>> JIT is manipulating code (folding constant, rewriting it) before
>>>>>>> compiling it to assembly code.
>>>>>>>
>>>>>>> - New Block Closure implementation by default: Allows one to implement
>>>>>>> in the Opal compiler the copying and clean blocks optimisations,
>>>>>>> reduce massively the complexity of some parts of the JIT (debugging
>>>>>>> API to map machine code pc to bytecode pc for example) and is required
>>>>>>> for the Sista support. Some work remains in debugging/IDE support.
>>>>>>>
>>>>>>> - New Bytecode set in production: Eases bytecode decoding (simplifying
>>>>>>> the code base of the bytecode to source code decompiler, the debugger,
>>>>>>> the JIT, etc.), lifts compiled code limitations (size of jumps, etc.)
>>>>>>> and is required for the Sista support. Some work remains in debugging
>>>>>>> support.
>>>>>>>
>>>>>>> - Read-only objects: They work in Pharo 6, but the in-image support
>>>>>>> needs to be improved (fall-back code of primitives, etc.) and some
>>>>>>> polishing needs to be done (primitive error code not always correct,
>>>>>>> etc.). Used for the support of different project, including GBS.
>>>>>>>
>>>>>>> - Literals immutability (based on read-only objects). In particular an
>>>>>>> analysis of the image usage of literal (string mutation) is needed.
>>>>>>>
>>>>>>> - Sista preview: A first version of Sista will be integrated, yielding
>>>>>>> 1.5x performance boost on most applications, but will be optional and
>>>>>>> will require specific constraints (not toying too much with literal
>>>>>>> mutability, partial IDE support, etc.).
>>>>>>>
>>>>>>>
>>>>>>> ### Already done
>>>>>>>
>>>>>>> - [DONE] Remove Shoreline reporter.
>>>>>>> - [DONE] Kernel should not depend on Traits: This will speed up the
>>>>>>> bootstrap and support the modular introduction of alternate traits
>>>>>>> implementation currently designed by Pablo Tesone and tested in the
>>>>>>> new generation of Moose.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>


12