Refactoring File Package

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Refactoring File Package

Guillermo Polito
Have you ever browsed the implementation of FileStream and checked subclasses and friends?

Well, I made a cleaner implementation at the side with

- a simple File object that is a sequential File as we all know
- a binary File stream on top of it that is composable with Zn encoders and other decorators
- a new interface to access Stdio streams

Stdio stdin.
Stdio stdout.
...

All that with tests and comments.


Now, the system still has tons of references to the following classes

FileStream -> 127
StandardFileStream -> 24
MultiByteFileStream -> 9

Most of them should be easily replaceable. Others may require wrapping the binary file stream provided by a file with a Zn encoder or similar.

So, if you have some minutes in your week and can replace 3, 4, 5 of them, we will be a little step closer to simplify file handling and dependencies :)

Guille
Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Damien Cassou-2

Guillermo Polito <[hidden email]> writes:

> Well, I made a cleaner implementation at the side with
>
> - a simple File object that is a sequential File as we all know
> - a binary File stream on top of it that is composable with Zn encoders and
> other decorators
> - a new interface to access Stdio streams

that's really good news Guillermo. Is it ok to make File reading depend
on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?


--
Damien Cassou
http://damiencassou.seasidehosting.st

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill

Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Sven Van Caekenberghe-2

> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>
>
> Guillermo Polito <[hidden email]> writes:
>
>> Well, I made a cleaner implementation at the side with
>>
>> - a simple File object that is a sequential File as we all know
>> - a binary File stream on top of it that is composable with Zn encoders and
>> other decorators
>> - a new interface to access Stdio streams
>
> that's really good news Guillermo.

Yes it is (need time to look at this in detail)

> Is it ok to make File reading depend
> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?

It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.

> --
> Damien Cassou
> http://damiencassou.seasidehosting.st
>
> "Success is the ability to go from one failure to another without
> losing enthusiasm." --Winston Churchill
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

stepharo
Sven

we were talking about splitting your package into two parts :)
Would you be ok to get the basic encoders in a separate package?
We were also thinking that NullEncoder could be called AsciiEncoder?

Stef

>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>>
>>
>> Guillermo Polito <[hidden email]> writes:
>>
>>> Well, I made a cleaner implementation at the side with
>>>
>>> - a simple File object that is a sequential File as we all know
>>> - a binary File stream on top of it that is composable with Zn encoders and
>>> other decorators
>>> - a new interface to access Stdio streams
>> that's really good news Guillermo.
> Yes it is (need time to look at this in detail)
>
>> Is it ok to make File reading depend
>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
>
>> --
>> Damien Cassou
>> http://damiencassou.seasidehosting.st
>>
>> "Success is the ability to go from one failure to another without
>> losing enthusiasm." --Winston Churchill
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

stepharo
In reply to this post by Guillermo Polito
Yes this is really exciting.
People we will need your help.

PS: We are about to unload StringHolder and PseudoClasses :)
5 years that we dreamed about it.


> Have you ever browsed the implementation of FileStream and checked
> subclasses and friends?
>
> Well, I made a cleaner implementation at the side with
>
> - a simple File object that is a sequential File as we all know
> - a binary File stream on top of it that is composable with Zn
> encoders and other decorators
> - a new interface to access Stdio streams
>
> Stdio stdin.
> Stdio stdout.
> ...
>
> All that with tests and comments.
>
> https://pharo.fogbugz.com/f/cases/15486/Refactor-File-Package
>
> Now, the system still has tons of references to the following classes
>
> FileStream -> 127
> StandardFileStream -> 24
> MultiByteFileStream -> 9
>
> Most of them should be easily replaceable. Others may require wrapping
> the binary file stream provided by a file with a Zn encoder or similar.
>
> So, if you have some minutes in your week and can replace 3, 4, 5 of
> them, we will be a little step closer to simplify file handling and
> dependencies :)
>
> Guille


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Sven Van Caekenberghe-2
In reply to this post by stepharo

> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>
> Sven
>
> we were talking about splitting your package into two parts :)
> Would you be ok to get the basic encoders in a separate package?

Zinc-Character-Encoding is already a separate package, it depends on nothing.
Zinc-Resource-Meta is next up (containing URL and Mime-Type).
Both are completely independent of any HTTP stuff.
All this is by design.

You probably mean that you want a separate config ? Right now they are just a groups.

> We were also thinking that NullEncoder could be called AsciiEncoder?

Maybe, maybe not, let me think about that a bit.

> Stef
>
>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>>>
>>>
>>> Guillermo Polito <[hidden email]> writes:
>>>
>>>> Well, I made a cleaner implementation at the side with
>>>>
>>>> - a simple File object that is a sequential File as we all know
>>>> - a binary File stream on top of it that is composable with Zn encoders and
>>>> other decorators
>>>> - a new interface to access Stdio streams
>>> that's really good news Guillermo.
>> Yes it is (need time to look at this in detail)
>>
>>> Is it ok to make File reading depend
>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
>> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
>>
>>> --
>>> Damien Cassou
>>> http://damiencassou.seasidehosting.st
>>>
>>> "Success is the ability to go from one failure to another without
>>> losing enthusiasm." --Winston Churchill
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Guillermo Polito
The only encoder that makes a bit of noise to me is the ZnByteEncoder that contains a lot of mapping tables for mostly unused encodings (plus methods with metadata to recreate them)...

El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe <[hidden email]> escribió:

> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>
> Sven
>
> we were talking about splitting your package into two parts :)
> Would you be ok to get the basic encoders in a separate package?

Zinc-Character-Encoding is already a separate package, it depends on nothing.
Zinc-Resource-Meta is next up (containing URL and Mime-Type).
Both are completely independent of any HTTP stuff.
All this is by design.

You probably mean that you want a separate config ? Right now they are just a groups.

> We were also thinking that NullEncoder could be called AsciiEncoder?

Maybe, maybe not, let me think about that a bit.

> Stef
>
>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>>>
>>>
>>> Guillermo Polito <[hidden email]> writes:
>>>
>>>> Well, I made a cleaner implementation at the side with
>>>>
>>>> - a simple File object that is a sequential File as we all know
>>>> - a binary File stream on top of it that is composable with Zn encoders and
>>>> other decorators
>>>> - a new interface to access Stdio streams
>>> that's really good news Guillermo.
>> Yes it is (need time to look at this in detail)
>>
>>> Is it ok to make File reading depend
>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
>> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
>>
>>> --
>>> Damien Cassou
>>> http://damiencassou.seasidehosting.st
>>>
>>> "Success is the ability to go from one failure to another without
>>> losing enthusiasm." --Winston Churchill
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Sven Van Caekenberghe-2

> On 05 Jun 2015, at 18:43, Guillermo Polito <[hidden email]> wrote:
>
> The only encoder that makes a bit of noise to me is the ZnByteEncoder that contains a lot of mapping tables for mostly unused encodings

67 encoding specifications, each a 128 array. The method constant is shared when used.
In the beginning there were only a couple, one day I added many more, some people need them.
For me, the cost is reasonable.

> (plus methods with metadata to recreate them)...

That is just two method (which is pretty cool, I love spec based programming).

> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe <[hidden email]> escribió:
>
> > On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
> >
> > Sven
> >
> > we were talking about splitting your package into two parts :)
> > Would you be ok to get the basic encoders in a separate package?
>
> Zinc-Character-Encoding is already a separate package, it depends on nothing.
> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
> Both are completely independent of any HTTP stuff.
> All this is by design.
>
> You probably mean that you want a separate config ? Right now they are just a groups.
>
> > We were also thinking that NullEncoder could be called AsciiEncoder?
>
> Maybe, maybe not, let me think about that a bit.
>
> > Stef
> >
> >>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
> >>>
> >>>
> >>> Guillermo Polito <[hidden email]> writes:
> >>>
> >>>> Well, I made a cleaner implementation at the side with
> >>>>
> >>>> - a simple File object that is a sequential File as we all know
> >>>> - a binary File stream on top of it that is composable with Zn encoders and
> >>>> other decorators
> >>>> - a new interface to access Stdio streams
> >>> that's really good news Guillermo.
> >> Yes it is (need time to look at this in detail)
> >>
> >>> Is it ok to make File reading depend
> >>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
> >> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
> >>
> >>> --
> >>> Damien Cassou
> >>> http://damiencassou.seasidehosting.st
> >>>
> >>> "Success is the ability to go from one failure to another without
> >>> losing enthusiasm." --Winston Churchill
> >>>
> >>
> >>
> >
> >
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

stepharo
Yes sven but with guille we are working on a really small kernel. So we
can duplicate just the classes we need but I would prefer not
but we can do it. The size is important for us because it takes time to
bootstrap.

Stef

Le 5/6/15 19:06, Sven Van Caekenberghe a écrit :

>> On 05 Jun 2015, at 18:43, Guillermo Polito <[hidden email]> wrote:
>>
>> The only encoder that makes a bit of noise to me is the ZnByteEncoder that contains a lot of mapping tables for mostly unused encodings
> 67 encoding specifications, each a 128 array. The method constant is shared when used.
> In the beginning there were only a couple, one day I added many more, some people need them.
> For me, the cost is reasonable.
>
>> (plus methods with metadata to recreate them)...
> That is just two method (which is pretty cool, I love spec based programming).
>
>> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe <[hidden email]> escribió:
>>
>>> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>>>
>>> Sven
>>>
>>> we were talking about splitting your package into two parts :)
>>> Would you be ok to get the basic encoders in a separate package?
>> Zinc-Character-Encoding is already a separate package, it depends on nothing.
>> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
>> Both are completely independent of any HTTP stuff.
>> All this is by design.
>>
>> You probably mean that you want a separate config ? Right now they are just a groups.
>>
>>> We were also thinking that NullEncoder could be called AsciiEncoder?
>> Maybe, maybe not, let me think about that a bit.
>>
>>> Stef
>>>
>>>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> Guillermo Polito <[hidden email]> writes:
>>>>>
>>>>>> Well, I made a cleaner implementation at the side with
>>>>>>
>>>>>> - a simple File object that is a sequential File as we all know
>>>>>> - a binary File stream on top of it that is composable with Zn encoders and
>>>>>> other decorators
>>>>>> - a new interface to access Stdio streams
>>>>> that's really good news Guillermo.
>>>> Yes it is (need time to look at this in detail)
>>>>
>>>>> Is it ok to make File reading depend
>>>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
>>>> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
>>>>
>>>>> --
>>>>> Damien Cassou
>>>>> http://damiencassou.seasidehosting.st
>>>>>
>>>>> "Success is the ability to go from one failure to another without
>>>>> losing enthusiasm." --Winston Churchill
>>>>>
>>>>
>>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Guillermo Polito
Actually we just want to have a kind of split in:

- essential
- non-essential

Then the bootstrap will include essential packages at the beginning and non-essential will be just loaded on top.

The rationale is: the smaller the kernel, the fastest the bootstrap, and with it the feedback loop we have.

That's why I was working on the File abstractions, and splitting some other kernel package.

El vie., 5 de jun. de 2015 a la(s) 7:35 p. m., stepharo <[hidden email]> escribió:
Yes sven but with guille we are working on a really small kernel. So we
can duplicate just the classes we need but I would prefer not
but we can do it. The size is important for us because it takes time to
bootstrap.

Stef

Le 5/6/15 19:06, Sven Van Caekenberghe a écrit :
>> On 05 Jun 2015, at 18:43, Guillermo Polito <[hidden email]> wrote:
>>
>> The only encoder that makes a bit of noise to me is the ZnByteEncoder that contains a lot of mapping tables for mostly unused encodings
> 67 encoding specifications, each a 128 array. The method constant is shared when used.
> In the beginning there were only a couple, one day I added many more, some people need them.
> For me, the cost is reasonable.
>
>> (plus methods with metadata to recreate them)...
> That is just two method (which is pretty cool, I love spec based programming).
>
>> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe <[hidden email]> escribió:
>>
>>> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>>>
>>> Sven
>>>
>>> we were talking about splitting your package into two parts :)
>>> Would you be ok to get the basic encoders in a separate package?
>> Zinc-Character-Encoding is already a separate package, it depends on nothing.
>> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
>> Both are completely independent of any HTTP stuff.
>> All this is by design.
>>
>> You probably mean that you want a separate config ? Right now they are just a groups.
>>
>>> We were also thinking that NullEncoder could be called AsciiEncoder?
>> Maybe, maybe not, let me think about that a bit.
>>
>>> Stef
>>>
>>>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> Guillermo Polito <[hidden email]> writes:
>>>>>
>>>>>> Well, I made a cleaner implementation at the side with
>>>>>>
>>>>>> - a simple File object that is a sequential File as we all know
>>>>>> - a binary File stream on top of it that is composable with Zn encoders and
>>>>>> other decorators
>>>>>> - a new interface to access Stdio streams
>>>>> that's really good news Guillermo.
>>>> Yes it is (need time to look at this in detail)
>>>>
>>>>> Is it ok to make File reading depend
>>>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
>>>> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
>>>>
>>>>> --
>>>>> Damien Cassou
>>>>> http://damiencassou.seasidehosting.st
>>>>>
>>>>> "Success is the ability to go from one failure to another without
>>>>> losing enthusiasm." --Winston Churchill
>>>>>
>>>>
>>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Ben Coman
On Sat, Jun 6, 2015 at 5:26 PM, Guillermo Polito
<[hidden email]> wrote:
> Actually we just want to have a kind of split in:
>
> - essential
> - non-essential

I guess you have scripts outside the image to shrink the image and
then bootstrap it.  However these are not necessarily visible to
everyone and I am wondering once the bootstrap goal is achieved you
will handle the entropy of ongoing development by many others
accidentally introducing dependencies that breaks the bootstrap?

Also what I have seen is classes being moved out of their current
packaging (sorry I forget the details, but I think it was moved out of
System) into a new top-level package, which I think loses something in
structure and might pollute the top-level packages.

So considering the recent discussion of package tags, I wonder if
somehow we could have a "Bootstrap" tag and realtime critics that
complain if Bootstrap-tagged-class references any
non-Bootstrap-tagged-class.  There might even be several levels of
Bootstrap tags.  This would improve ongoing visibility of the
bootstrap structure and help the rest of us to not shoot it in the
foot.

I wonder also if tags might also be extended to method protocols, so
you can have a method with the "accessors" tag and also the
"bootstrap" tag, so that even a Bootstrap-tagged-class can be further
minimised by the methods it needs during bootstrap.  Otherwise I guess
to minimise the methods in a Bootstrap-tagged-class, later extension
by another package could not add methods to the "accessors" protocol.

P.S. I know I do a lot of wondering, and of course divergent thought
is easier than concrete answers.
cheers -ben


>
> Then the bootstrap will include essential packages at the beginning and
> non-essential will be just loaded on top.
>
> The rationale is: the smaller the kernel, the fastest the bootstrap, and
> with it the feedback loop we have.
>
> That's why I was working on the File abstractions, and splitting some other
> kernel package.
>
> El vie., 5 de jun. de 2015 a la(s) 7:35 p. m., stepharo <[hidden email]>
> escribió:
>>
>> Yes sven but with guille we are working on a really small kernel. So we
>> can duplicate just the classes we need but I would prefer not
>> but we can do it. The size is important for us because it takes time to
>> bootstrap.
>>
>> Stef
>>
>> Le 5/6/15 19:06, Sven Van Caekenberghe a écrit :
>> >> On 05 Jun 2015, at 18:43, Guillermo Polito <[hidden email]>
>> >> wrote:
>> >>
>> >> The only encoder that makes a bit of noise to me is the ZnByteEncoder
>> >> that contains a lot of mapping tables for mostly unused encodings
>> > 67 encoding specifications, each a 128 array. The method constant is
>> > shared when used.
>> > In the beginning there were only a couple, one day I added many more,
>> > some people need them.
>> > For me, the cost is reasonable.
>> >
>> >> (plus methods with metadata to recreate them)...
>> > That is just two method (which is pretty cool, I love spec based
>> > programming).
>> >
>> >> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe
>> >> <[hidden email]> escribió:
>> >>
>> >>> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>> >>>
>> >>> Sven
>> >>>
>> >>> we were talking about splitting your package into two parts :)
>> >>> Would you be ok to get the basic encoders in a separate package?
>> >> Zinc-Character-Encoding is already a separate package, it depends on
>> >> nothing.
>> >> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
>> >> Both are completely independent of any HTTP stuff.
>> >> All this is by design.
>> >>
>> >> You probably mean that you want a separate config ? Right now they are
>> >> just a groups.
>> >>
>> >>> We were also thinking that NullEncoder could be called AsciiEncoder?
>> >> Maybe, maybe not, let me think about that a bit.
>> >>
>> >>> Stef
>> >>>
>> >>>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]>
>> >>>>> wrote:
>> >>>>>
>> >>>>>
>> >>>>> Guillermo Polito <[hidden email]> writes:
>> >>>>>
>> >>>>>> Well, I made a cleaner implementation at the side with
>> >>>>>>
>> >>>>>> - a simple File object that is a sequential File as we all know
>> >>>>>> - a binary File stream on top of it that is composable with Zn
>> >>>>>> encoders and
>> >>>>>> other decorators
>> >>>>>> - a new interface to access Stdio streams
>> >>>>> that's really good news Guillermo.
>> >>>> Yes it is (need time to look at this in detail)
>> >>>>
>> >>>>> Is it ok to make File reading depend
>> >>>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping
>> >>>>> harder?
>> >>>> It does not depend on the HTTP part, but on the Encoding part below
>> >>>> it, so that should be OK.
>> >>>>
>> >>>>> --
>> >>>>> Damien Cassou
>> >>>>> http://damiencassou.seasidehosting.st
>> >>>>>
>> >>>>> "Success is the ability to go from one failure to another without
>> >>>>> losing enthusiasm." --Winston Churchill
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>> >
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Guillermo Polito
Hi Ben!

I answer between lines. If the discussion continues maybe we should move to another thread also ^^.

El sáb., 6 de jun. de 2015 a la(s) 12:39 p. m., Ben Coman <[hidden email]> escribió:
On Sat, Jun 6, 2015 at 5:26 PM, Guillermo Polito
<[hidden email]> wrote:
> Actually we just want to have a kind of split in:
>
> - essential
> - non-essential

I guess you have scripts outside the image to shrink the image and
then bootstrap it.  However these are not necessarily visible to
everyone and I am wondering once the bootstrap goal is achieved you
will handle the entropy of ongoing development by many others
accidentally introducing dependencies that breaks the bootstrap?

Well no. I try to not do that. Currently what I have working is:

- a metacello configuration describing the packages+versions that will be part of the built kernel

- a simple script that will create the initial objects of an image before class creation (e.g., create Smalltalk, initialize the symbol table, etc). I want this guy to be the smallest as possible to avoid what you describe above

- a script to initialize the image after class creation. e.g., create the first process, and execute some class side initialize.

Then again, the ad-hoc bootstrap scripts try to be minimal to avoid being impacted. Of course they could be impacted by changes of APIs, but I think it is still controllable in the size of the scripts :).
And then, that is why I try to push as many "modularity" concerns to Pharo itself, to keep these scripts small.
 

Also what I have seen is classes being moved out of their current
packaging (sorry I forget the details, but I think it was moved out of
System) into a new top-level package, which I think loses something in
structure and might pollute the top-level packages.

Yes, mea culpa there. Some times when extracting classes from one package I didn't put the new package under the System-* or whatever top level package.

However, I believe also that the agreement on what are the top-level packages is kind of implicit and obscure to me. I remember some discussions in the list with the Alt browser from Thierry using the idea of top level packages, and others talking about the navigation of the system. I check the latest Pharo Image and I found:
 - AST (should be system?)
 - Announcements (should be system?)
 - Collections (should be system?)
 - Files (should be system?)
 - Networking (should be system?)
...

And then if we check what is inside the System top-level package today I'd say it is a sack full of random things, with no evident criteria. So maybe we should discuss and agree on how to name packages and top-level packages!
 
So considering the recent discussion of package tags, I wonder if
somehow we could have a "Bootstrap" tag and realtime critics that
complain if Bootstrap-tagged-class references any
non-Bootstrap-tagged-class.  There might even be several levels of
Bootstrap tags.  This would improve ongoing visibility of the
bootstrap structure and help the rest of us to not shoot it in the
foot. 
v
I wonder also if tags might also be extended to method protocols, so
you can have a method with the "accessors" tag and also the
"bootstrap" tag, so that even a Bootstrap-tagged-class can be further
minimised by the methods it needs during bootstrap.  Otherwise I guess
to minimise the methods in a Bootstrap-tagged-class, later extension
by another package could not add methods to the "accessors" protocol.
 
That would be amazing if I could add meta-data to packages to only install specific classes. Because we could keep a coarser granularity of packages while keeping the bootstrap small.

However, the problem is that you cannot predict what classes/methods the bootstrap will use. So far, I'm using the class builder to build classes in the bootstrap. In such way, I ensure that all classes that I create will honor the same invariants as a normal class and I don't have to duplicate the creation code :).

That means that we can change the internals of the class builder as long as we don't change the API + preconditions and the bootstrap will continue working.

Guille
Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Richard Sargent
Administrator
In reply to this post by Ben Coman
Ben Coman wrote
On Sat, Jun 6, 2015 at 5:26 PM, Guillermo Polito
<[hidden email]> wrote:
> Actually we just want to have a kind of split in:
>
> - essential
> - non-essential

I guess you have scripts outside the image to shrink the image and
then bootstrap it.  However these are not necessarily visible to
everyone and I am wondering once the bootstrap goal is achieved you
will handle the entropy of ongoing development by many others
accidentally introducing dependencies that breaks the bootstrap?

Also what I have seen is classes being moved out of their current
packaging (sorry I forget the details, but I think it was moved out of
System) into a new top-level package, which I think loses something in
structure and might pollute the top-level packages.

So considering the recent discussion of package tags, I wonder if
somehow we could have a "Bootstrap" tag and realtime critics that
complain if Bootstrap-tagged-class references any
non-Bootstrap-tagged-class.  There might even be several levels of
Bootstrap tags.  This would improve ongoing visibility of the
bootstrap structure and help the rest of us to not shoot it in the
foot.
One of the things I really like about ENVY (as found in VA Smalltalk) is that packages are formally modelled and are distinct from method categories. (Another is that a method can have multiple categories.) And by formally modelling packages, one can easily include proper dependency relationships and more easily detect references to behaviours outside a packages prerequisites chain.

I realize that is a huge departure from current Squeak and Pharo practice. But, I hope it's one that will get consideration. (Given all the other first class modelling you've been adding to Pharo, I think this would be consistent with the current philosophy!)


I wonder also if tags might also be extended to method protocols, so
you can have a method with the "accessors" tag and also the
"bootstrap" tag, so that even a Bootstrap-tagged-class can be further
minimised by the methods it needs during bootstrap.  Otherwise I guess
to minimise the methods in a Bootstrap-tagged-class, later extension
by another package could not add methods to the "accessors" protocol.

P.S. I know I do a lot of wondering, and of course divergent thought
is easier than concrete answers.
cheers -ben


>
> Then the bootstrap will include essential packages at the beginning and
> non-essential will be just loaded on top.
>
> The rationale is: the smaller the kernel, the fastest the bootstrap, and
> with it the feedback loop we have.
>
> That's why I was working on the File abstractions, and splitting some other
> kernel package.
>
> El vie., 5 de jun. de 2015 a la(s) 7:35 p. m., stepharo <[hidden email]>
> escribió:
>>
>> Yes sven but with guille we are working on a really small kernel. So we
>> can duplicate just the classes we need but I would prefer not
>> but we can do it. The size is important for us because it takes time to
>> bootstrap.
>>
>> Stef
>>
>> Le 5/6/15 19:06, Sven Van Caekenberghe a écrit :
>> >> On 05 Jun 2015, at 18:43, Guillermo Polito <[hidden email]>
>> >> wrote:
>> >>
>> >> The only encoder that makes a bit of noise to me is the ZnByteEncoder
>> >> that contains a lot of mapping tables for mostly unused encodings
>> > 67 encoding specifications, each a 128 array. The method constant is
>> > shared when used.
>> > In the beginning there were only a couple, one day I added many more,
>> > some people need them.
>> > For me, the cost is reasonable.
>> >
>> >> (plus methods with metadata to recreate them)...
>> > That is just two method (which is pretty cool, I love spec based
>> > programming).
>> >
>> >> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe
>> >> <[hidden email]> escribió:
>> >>
>> >>> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>> >>>
>> >>> Sven
>> >>>
>> >>> we were talking about splitting your package into two parts :)
>> >>> Would you be ok to get the basic encoders in a separate package?
>> >> Zinc-Character-Encoding is already a separate package, it depends on
>> >> nothing.
>> >> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
>> >> Both are completely independent of any HTTP stuff.
>> >> All this is by design.
>> >>
>> >> You probably mean that you want a separate config ? Right now they are
>> >> just a groups.
>> >>
>> >>> We were also thinking that NullEncoder could be called AsciiEncoder?
>> >> Maybe, maybe not, let me think about that a bit.
>> >>
>> >>> Stef
>> >>>
>> >>>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]>
>> >>>>> wrote:
>> >>>>>
>> >>>>>
>> >>>>> Guillermo Polito <[hidden email]> writes:
>> >>>>>
>> >>>>>> Well, I made a cleaner implementation at the side with
>> >>>>>>
>> >>>>>> - a simple File object that is a sequential File as we all know
>> >>>>>> - a binary File stream on top of it that is composable with Zn
>> >>>>>> encoders and
>> >>>>>> other decorators
>> >>>>>> - a new interface to access Stdio streams
>> >>>>> that's really good news Guillermo.
>> >>>> Yes it is (need time to look at this in detail)
>> >>>>
>> >>>>> Is it ok to make File reading depend
>> >>>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping
>> >>>>> harder?
>> >>>> It does not depend on the HTTP part, but on the Encoding part below
>> >>>> it, so that should be OK.
>> >>>>
>> >>>>> --
>> >>>>> Damien Cassou
>> >>>>> http://damiencassou.seasidehosting.st
>> >>>>>
>> >>>>> "Success is the ability to go from one failure to another without
>> >>>>> losing enthusiasm." --Winston Churchill
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>> >
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Sergio Fedi
> One of the things I really like about ENVY (as found in VA Smalltalk) is

> that packages are formally modelled and are distinct from method categories.
> (Another is that a method can have multiple categories.) And by formally
> modelling packages, one can easily include proper dependency relationships
> and more easily detect references to behaviours outside a packages
> prerequisites chain.
>
> I realize that is a huge departure from current Squeak and Pharo practice.
> But, I hope it's one that will get consideration. (Given all the other first
> class modelling you've been adding to Pharo, I think this would be
> consistent with the current philosophy!)

+1

Multiple categories for methods and proper modelling of Packages seems like a very strong feature to have.

Multiple tags per class is also a big plus.

Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

Sven Van Caekenberghe-2
In reply to this post by stepharo
Here is a first step in making the Zinc character encoding package lighter:

===
Name: Zinc-Character-Encoding-Core-SvenVanCaekenberghe.38
Author: SvenVanCaekenberghe
Time: 9 June 2015, 5:21:51.278641 pm
UUID: 8b4b6a04-d2a2-4fec-97fe-6faf498bc54b
Ancestors: Zinc-Character-Encoding-Core-SvenVanCaekenberghe.37

Experimental change: introduction of ZnSimplifiedByteEncoder as a low memory variant of the full featured ZnByteEncoder, supporting only latin1 (iso-8859-1) and ascii.

Move ASCII handling from ZnNullEncoder to ZnSimplifiedByteEncoder making it strict by default.

Needs

  ZnSimplifiedByteEncoder initialize.
  ZnByteEncoder initialize.
===

Of course, now we have to decide how to really split the package, which parts to keep in the low level version, and which to move to the extended version. The tests might need splitting too.

Sven

> On 05 Jun 2015, at 19:35, stepharo <[hidden email]> wrote:
>
> Yes sven but with guille we are working on a really small kernel. So we can duplicate just the classes we need but I would prefer not
> but we can do it. The size is important for us because it takes time to bootstrap.
>
> Stef
>
> Le 5/6/15 19:06, Sven Van Caekenberghe a écrit :
>>> On 05 Jun 2015, at 18:43, Guillermo Polito <[hidden email]> wrote:
>>>
>>> The only encoder that makes a bit of noise to me is the ZnByteEncoder that contains a lot of mapping tables for mostly unused encodings
>> 67 encoding specifications, each a 128 array. The method constant is shared when used.
>> In the beginning there were only a couple, one day I added many more, some people need them.
>> For me, the cost is reasonable.
>>
>>> (plus methods with metadata to recreate them)...
>> That is just two method (which is pretty cool, I love spec based programming).
>>
>>> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe <[hidden email]> escribió:
>>>
>>>> On 05 Jun 2015, at 18:20, stepharo <[hidden email]> wrote:
>>>>
>>>> Sven
>>>>
>>>> we were talking about splitting your package into two parts :)
>>>> Would you be ok to get the basic encoders in a separate package?
>>> Zinc-Character-Encoding is already a separate package, it depends on nothing.
>>> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
>>> Both are completely independent of any HTTP stuff.
>>> All this is by design.
>>>
>>> You probably mean that you want a separate config ? Right now they are just a groups.
>>>
>>>> We were also thinking that NullEncoder could be called AsciiEncoder?
>>> Maybe, maybe not, let me think about that a bit.
>>>
>>>> Stef
>>>>
>>>>>> On 05 Jun 2015, at 18:09, Damien Cassou <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> Guillermo Polito <[hidden email]> writes:
>>>>>>
>>>>>>> Well, I made a cleaner implementation at the side with
>>>>>>>
>>>>>>> - a simple File object that is a sequential File as we all know
>>>>>>> - a binary File stream on top of it that is composable with Zn encoders and
>>>>>>> other decorators
>>>>>>> - a new interface to access Stdio streams
>>>>>> that's really good news Guillermo.
>>>>> Yes it is (need time to look at this in detail)
>>>>>
>>>>>> Is it ok to make File reading depend
>>>>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping harder?
>>>>> It does not depend on the HTTP part, but on the Encoding part below it, so that should be OK.
>>>>>
>>>>>> --
>>>>>> Damien Cassou
>>>>>> http://damiencassou.seasidehosting.st
>>>>>>
>>>>>> "Success is the ability to go from one failure to another without
>>>>>> losing enthusiasm." --Winston Churchill
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

stepharo
In reply to this post by Richard Sargent

>> I guess you have scripts outside the image to shrink the image and
>> then bootstrap it.  However these are not necessarily visible to
>> everyone and I am wondering once the bootstrap goal is achieved you
>> will handle the entropy of ongoing development by many others
>> accidentally introducing dependencies that breaks the bootstrap?
>>
>> Also what I have seen is classes being moved out of their current
>> packaging (sorry I forget the details, but I think it was moved out of
>> System) into a new top-level package, which I think loses something in
>> structure and might pollute the top-level packages.
>>
>> So considering the recent discussion of package tags, I wonder if
>> somehow we could have a "Bootstrap" tag and realtime critics that
>> complain if Bootstrap-tagged-class references any
>> non-Bootstrap-tagged-class.  There might even be several levels of
>> Bootstrap tags.  This would improve ongoing visibility of the
>> bootstrap structure and help the rest of us to not shoot it in the
>> foot.
> One of the things I really like about ENVY (as found in VA Smalltalk) is
> that packages are formally modelled and are distinct from method categories.
> (Another is that a method can have multiple categories.) And by formally
> modelling packages, one can easily include proper dependency relationships
> and more easily detect references to behaviours outside a packages
> prerequisites chain.

we have the same with the dependency browser. Christophe is adding some
logic to
get the messages sent but not implemented in a package and its dependents.
We are also adding dependencies to the packaege themselves.
>
> I realize that is a huge departure from current Squeak and Pharo practice.
> But, I hope it's one that will get consideration. (Given all the other first
> class modelling you've been adding to Pharo, I think this would be
> consistent with the current philosophy!)
So we are slowly adding metadata to package manifest.
Now packages are not added the Smalltalk namespace and I do not know if
this would be good to have.

>
>> I wonder also if tags might also be extended to method protocols, so
>> you can have a method with the "accessors" tag and also the
>> "bootstrap" tag, so that even a Bootstrap-tagged-class can be further
>> minimised by the methods it needs during bootstrap.  Otherwise I guess
>> to minimise the methods in a Bootstrap-tagged-class, later extension
>> by another package could not add methods to the "accessors" protocol.
>>
>> P.S. I know I do a lot of wondering, and of course divergent thought
>> is easier than concrete answers.
>> cheers -ben
>>
>>
>>> Then the bootstrap will include essential packages at the beginning and
>>> non-essential will be just loaded on top.
>>>
>>> The rationale is: the smaller the kernel, the fastest the bootstrap, and
>>> with it the feedback loop we have.
>>>
>>> That's why I was working on the File abstractions, and splitting some
>>> other
>>> kernel package.
>>>
>>> El vie., 5 de jun. de 2015 a la(s) 7:35 p. m., stepharo &lt;
>> stepharo@
>> &gt;
>>> escribió:
>>>> Yes sven but with guille we are working on a really small kernel. So we
>>>> can duplicate just the classes we need but I would prefer not
>>>> but we can do it. The size is important for us because it takes time to
>>>> bootstrap.
>>>>
>>>> Stef
>>>>
>>>> Le 5/6/15 19:06, Sven Van Caekenberghe a écrit :
>>>>>> On 05 Jun 2015, at 18:43, Guillermo Polito &lt;
>> guillermopolito@
>> &gt;
>>>>>> wrote:
>>>>>>
>>>>>> The only encoder that makes a bit of noise to me is the ZnByteEncoder
>>>>>> that contains a lot of mapping tables for mostly unused encodings
>>>>> 67 encoding specifications, each a 128 array. The method constant is
>>>>> shared when used.
>>>>> In the beginning there were only a couple, one day I added many more,
>>>>> some people need them.
>>>>> For me, the cost is reasonable.
>>>>>
>>>>>> (plus methods with metadata to recreate them)...
>>>>> That is just two method (which is pretty cool, I love spec based
>>>>> programming).
>>>>>
>>>>>> El vie., 5 de jun. de 2015 a la(s) 6:30 p. m., Sven Van Caekenberghe
>>>>>> &lt;
>> sven@
>> &gt; escribió:
>>>>>>> On 05 Jun 2015, at 18:20, stepharo &lt;
>> stepharo@
>> &gt; wrote:
>>>>>>> Sven
>>>>>>>
>>>>>>> we were talking about splitting your package into two parts :)
>>>>>>> Would you be ok to get the basic encoders in a separate package?
>>>>>> Zinc-Character-Encoding is already a separate package, it depends on
>>>>>> nothing.
>>>>>> Zinc-Resource-Meta is next up (containing URL and Mime-Type).
>>>>>> Both are completely independent of any HTTP stuff.
>>>>>> All this is by design.
>>>>>>
>>>>>> You probably mean that you want a separate config ? Right now they
>>>> are
>>>>>> just a groups.
>>>>>>
>>>>>>> We were also thinking that NullEncoder could be called AsciiEncoder?
>>>>>> Maybe, maybe not, let me think about that a bit.
>>>>>>
>>>>>>> Stef
>>>>>>>
>>>>>>>>> On 05 Jun 2015, at 18:09, Damien Cassou &lt;
>> damien.cassou@
>> &gt;
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Guillermo Polito &lt;
>> guillermopolito@
>> &gt; writes:
>>>>>>>>>> Well, I made a cleaner implementation at the side with
>>>>>>>>>>
>>>>>>>>>> - a simple File object that is a sequential File as we all know
>>>>>>>>>> - a binary File stream on top of it that is composable with Zn
>>>>>>>>>> encoders and
>>>>>>>>>> other decorators
>>>>>>>>>> - a new interface to access Stdio streams
>>>>>>>>> that's really good news Guillermo.
>>>>>>>> Yes it is (need time to look at this in detail)
>>>>>>>>
>>>>>>>>> Is it ok to make File reading depend
>>>>>>>>> on Zinc? This sounds strange. Wouldn't that make bootstrapping
>>>>>>>>> harder?
>>>>>>>> It does not depend on the HTTP part, but on the Encoding part below
>>>>>>>> it, so that should be OK.
>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Damien Cassou
>>>>>>>>> http://damiencassou.seasidehosting.st
>>>>>>>>>
>>>>>>>>> "Success is the ability to go from one failure to another without
>>>>>>>>> losing enthusiasm." --Winston Churchill
>>>>>>>>>
>>>>>
>>>>
>
>
>
>
> --
> View this message in context: http://forum.world.st/Refactoring-File-Package-tp4830625p4830987.html
> Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Refactoring File Package

stepharo
In reply to this post by Sergio Fedi


Le 8/6/15 20:00, Sergio Fedi a écrit :

> > One of the things I really like about ENVY (as found in VA Smalltalk) is
> > that packages are formally modelled and are distinct from method
> categories.
> > (Another is that a method can have multiple categories.) And by formally
> > modelling packages, one can easily include proper dependency
> relationships
> > and more easily detect references to behaviours outside a packages
> > prerequisites chain.
> >
> > I realize that is a huge departure from current Squeak and Pharo
> practice.
> > But, I hope it's one that will get consideration. (Given all the
> other first
> > class modelling you've been adding to Pharo, I think this would be
> > consistent with the current philosophy!)
>
> +1
>
> Multiple categories for methods and proper modelling of Packages seems
> like a very strong feature to have.

We have a far more important features list:
     Spur
     FFI
     threaded FFI
     Better cleaner widgets
     Database support
     Bootstrap
     new graphics kernel
     new tools.

:)
>
> Multiple tags per class is also a big plus.
>