New source code subsystem

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

New source code subsystem

Klaus D. Witzel
List,

I put together what I think is relevant for a new source code subsystem  
and, if I'm mistaken or have overlooked one of your recent suggestions  
and/or requirements, please do not hesitate and say so.

What follows is not intended as a replacement for MC or SM, rather the new  
subsystem should be as independent as possible. If you load something with  
MC, the new subsystem can treat it like any other contribution (more about  
contributions later).

Trying to keep preliminaries as short as possible, we have as  
organizational elements (for the sake of discussion): system categories,  
classes, message categories and methods. The unit of source code "entity"  
we consider to be a unit of change belongs to one of these elements. We  
can add, replace, rename and delete. We have not only methods, but also  
DoIt's (definitions, organizational, etc) to take into account. And let's  
say that the browser is our tool, just to keep it simple here.

-------------------------------

At the time an image is distributed, it is supposed to be in sync (not  
necessarily *full* sync) with its respective source code subsystem. So  
when you open a browser, all entities are colored black on white, and my  
story begins.

When you want to *observe* a certain area of the source code subsytem, you  
put the respective entity (item from here on) onto your watchlist (cmd+ if  
you like) from where the item gets the color blue. After putting a class  
onto your watchlist you get all its message categories and methods blue;  
un-watch those (sub)items which are of no interest to you; etc for other  
organizational levels.

When you do a *change* to a blue colored item, its new source is  
contributed to the source code subsystem (from here on scs for short) and  
the item is colored green in your browser. If later somebody else  
contributes a change to the same blue item, it gets the color yellow in  
both browsers.

And if an entity you have in your image cannot be *matched* with the scs,  
it is colored red.

As is tradition, if you don't save your image before the next crash, all  
this coloring information is lost, and you are back to black and white.  
The scs does NOT know the state of your image.

-------------------------------

A contribution to the scs either has a predecessor or else it is new (to  
the scs). Let's say three developers make contributions to the same blue  
item, then we arrive at three branches (and the three browsers color the  
respective item yellow). It is up to you to accept one of these branches,  
but you are no forced to do so. An accepted branch is colored green (it  
will turn yellow when the next contribution relative to it arrives).

-------------------------------

By chasing the predecessor it is always possible to roll-back an item (or  
a group of items), this is independent of the coloring I used for  
illustration. The same "backwards" information is readily available for  
analyzing the complete history of an item, including all unaccepted  
branches.

-------------------------------

Doing this into the other direction (roll-forward) depends on which  
branches where accepted. At any point in time, accepted branches represent  
a "cut" in the scs.

-------------------------------

If you work alone on a project, you'll never see any yellow colored item,  
unless you revert a change (because the dismissed becomes an unaccepted  
branch). For people who are used to make small incremental changes, a  
preference tells whether or not every change is treated as a branch or  
else is to be consolidated until you do a "big save" to the scs. The  
latter requires, I think, a local buffering scs on your harddisk (or on  
your local area server).

-------------------------------

The amount of conversation between browser and scs is small, the browser  
only has to ask for colors (one byte answer per item) for those items it  
shows *and* which are on your watch list. But if you want to see more  
(like, for example, all branches of an item), more data will have to  
travel.

-------------------------------

It's now time for me to sit back and relax :-)

/Klaus


Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Nicolas Cellier-3
Klaus, i would say this is another source version control system.

This bottom up approach is interesting but will not work. Squeakers will continue to use their preferred tools, not your specific browser.

Let's be more pragmatic and have a small incremental bottom up approach.

I see two solutions:
- store source code and history in image... I do not believe we should go in this direction even in compressed forms. We are rather making efforts to make images shorter... But you can have other opinion of course.
- continue storing source in external repositories (files, URL, NET  servers...)

In the later case, we could have many files instead of two (sources changes) and we should be able to have large files: conclusion: let's just change source code pointer encoding.

Using SmallIntegers (30 bits?) we can address 1GByte of space. More if bits are not encoded as file offset but as Oop on a server...

Using two SmallInteger, we have at least 60 bits, 1 ExaByte of data! Let's go for it (count how many compiledMethod are in your image... and you'll see this is not a drama).

Maybe we can have an encoding supporting 1 single SmallInteger for certain patterns (for example it can be sufficient for code stored in 'sources' file, 'sources' file could also be organised with an indirection table instead of using byte offsets...Given average source code length, that can save a factor between 10 and 100 i presume).

We should be able to extend source code being in several places in the future... (several files or URL or ...). So let use change encoding by splitting bits : 20 bits for sourceRegistry offset, 40 bits for file offset, or 30bits/30bits, or both (pattern starting with bits 0001 will be 20/40, all other 30/30 for example).

Then we have to create the SourceRegistry with objects able to handle source code stored in files, in compressed files, on the Net, with all necessary caching to reduce number of connections and design of the source code servers...

However, i would keep the principle of copying changes in the 'changes' file to recover easily in case of image crash, and commiting to another source would still be the responsibility of user, as is now.

Small details: make sure the FileSystem primitives can deal with huge offsets (and also underlying OS of course).

You see, nothing really new, nothing really difficult.
With just a little bit more reflexion we can go in this direction and that should be enough for a few years of development and future enhancements...

Nicolas

Klaus D. Witzel:

> List,
>
> I put together what I think is relevant for a new source code subsystem  
> and, if I'm mistaken or have overlooked one of your recent suggestions  
> and/or requirements, please do not hesitate and say so.
>
> What follows is not intended as a replacement for MC or SM, rather the new  
> subsystem should be as independent as possible. If you load something with  
> MC, the new subsystem can treat it like any other contribution (more about  
> contributions later).
>
> Trying to keep preliminaries as short as possible, we have as  
> organizational elements (for the sake of discussion): system categories,  
> classes, message categories and methods. The unit of source code "entity"  
> we consider to be a unit of change belongs to one of these elements. We  
> can add, replace, rename and delete. We have not only methods, but also  
> DoIt's (definitions, organizational, etc) to take into account. And let's  
> say that the browser is our tool, just to keep it simple here.
>
> -------------------------------
>
> At the time an image is distributed, it is supposed to be in sync (not  
> necessarily *full* sync) with its respective source code subsystem. So  
> when you open a browser, all entities are colored black on white, and my  
> story begins.
>
> When you want to *observe* a certain area of the source code subsytem, you  
> put the respective entity (item from here on) onto your watchlist (cmd+ if  
> you like) from where the item gets the color blue. After putting a class  
> onto your watchlist you get all its message categories and methods blue;  
> un-watch those (sub)items which are of no interest to you; etc for other  
> organizational levels.
>
> When you do a *change* to a blue colored item, its new source is  
> contributed to the source code subsystem (from here on scs for short) and  
> the item is colored green in your browser. If later somebody else  
> contributes a change to the same blue item, it gets the color yellow in  
> both browsers.
>
> And if an entity you have in your image cannot be *matched* with the scs,  
> it is colored red.
>
> As is tradition, if you don't save your image before the next crash, all  
> this coloring information is lost, and you are back to black and white.  
> The scs does NOT know the state of your image.
>
> -------------------------------
>
> A contribution to the scs either has a predecessor or else it is new (to  
> the scs). Let's say three developers make contributions to the same blue  
> item, then we arrive at three branches (and the three browsers color the  
> respective item yellow). It is up to you to accept one of these branches,  
> but you are no forced to do so. An accepted branch is colored green (it  
> will turn yellow when the next contribution relative to it arrives).
>
> -------------------------------
>
> By chasing the predecessor it is always possible to roll-back an item (or  
> a group of items), this is independent of the coloring I used for  
> illustration. The same "backwards" information is readily available for  
> analyzing the complete history of an item, including all unaccepted  
> branches.
>
> -------------------------------
>
> Doing this into the other direction (roll-forward) depends on which  
> branches where accepted. At any point in time, accepted branches represent  
> a "cut" in the scs.
>
> -------------------------------
>
> If you work alone on a project, you'll never see any yellow colored item,  
> unless you revert a change (because the dismissed becomes an unaccepted  
> branch). For people who are used to make small incremental changes, a  
> preference tells whether or not every change is treated as a branch or  
> else is to be consolidated until you do a "big save" to the scs. The  
> latter requires, I think, a local buffering scs on your harddisk (or on  
> your local area server).
>
> -------------------------------
>
> The amount of conversation between browser and scs is small, the browser  
> only has to ask for colors (one byte answer per item) for those items it  
> shows *and* which are on your watch list. But if you want to see more  
> (like, for example, all branches of an item), more data will have to  
> travel.
>
> -------------------------------
>
> It's now time for me to sit back and relax :-)
>
> /Klaus
>
>
>

________________________________________________________________________
iFRANCE, exprimez-vous !
http://web.ifrance.com


Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Klaus D. Witzel
Nicolas

thank you for taking the time for a response.

On Wed, 26 Jul 2006 20:55:44 +0200, you wrote:

> Klaus, i would say this is another source version control system.

I did not have version *control* in mind but yes, scs could be used for  
such ancient concepts (used them for > 30 years, quite an experience).

No, I was lead by the aim for more collaborative features (hmm, since  
there are none in Squeak, should I say for *at least* one or two  
collaborative features).

> This bottom up approach is interesting but will not work. Squeakers will  
> continue to use their preferred tools, not your specific browser.

No specific browser, just colors in the list views of the existing  
browsers. One or two additional items on the context menus, that's all  
about it :)

In my imagination you're running 2-3 Squeak images at the same time (on  
your 2-4GB memory / 2-4 GHz quadruple core notebook, what else would you  
like to to with your hardware?).

One image in which you do all the "abstract" work, incl. docs (hopefully  
;-) the second image for the hard stuff (compile&debug the new project) in  
the 3rd image you do all the testing; and (perhaps) in the 4th you hunt  
the nasty bugs left over from the previous project(s).

These images all work with the same source code subsystem, integrity  
guaranteed. And if you do a mistake in one of the images, the others will  
not be infected. Now how about this?

> Let's be more pragmatic and have a small incremental bottom up approach.
>
> I see two solutions:
> - store source code and history in image... I do not believe we should  
> go in this direction even in compressed forms. We are rather making  
> efforts to make images shorter... But you can have other opinion of  
> course.
> - continue storing source in external repositories (files, URL, NET  
> servers...)
>
> In the later case, we could have many files instead of two (sources  
> changes) and we should be able to have large files: conclusion: let's  
> just change source code pointer encoding.

The source code pointer is only bytes. If a quick solution is needed, why  
not add just one more byte and take a breath until the then 256 fold limit  
is reached?

Nicolas, as said above I really appreciate you take care for a response  
BUT aren't you comparing apples (new source code subsystem) with oranges  
(one or two byte lengthening of a pointer).

> Using SmallIntegers (30 bits?) we can address 1GByte of space. More if  
> bits are not encoded as file offset but as Oop on a server...
>
> Using two SmallInteger, we have at least 60 bits, 1 ExaByte of data!  
> Let's go for it (count how many compiledMethod are in your image... and  
> you'll see this is not a drama).
>
> Maybe we can have an encoding supporting 1 single SmallInteger for  
> certain patterns (for example it can be sufficient for code stored in  
> 'sources' file, 'sources' file could also be organised with an  
> indirection table instead of using byte offsets...Given average source  
> code length, that can save a factor between 10 and 100 i presume).
>
> We should be able to extend source code being in several places in the  
> future... (several files or URL or ...). So let use change encoding by  
> splitting bits : 20 bits for sourceRegistry offset, 40 bits for file  
> offset, or 30bits/30bits, or both (pattern starting with bits 0001 will  
> be 20/40, all other 30/30 for example).
>
> Then we have to create the SourceRegistry with objects able to handle  
> source code stored in files, in compressed files, on the Net, with all  
> necessary caching to reduce number of connections and design of the  
> source code servers...

This sounds easy but relies on the current Changes mechanism which is  
rotten in the image. Isn't it time to overhaul the framework instead of  
attaching new pneus with new rims? Buying new rims and pneus from Ferrari  
(they're just around the corner here :) doesn't buy you a Ferrari ;-)

> However, i would keep the principle of copying changes in the 'changes'  
> file to recover easily in case of image crash, and commiting to another  
> source would still be the responsibility of user, as is now.

"to another repository"?

> Small details: make sure the FileSystem primitives can deal with huge  
> offsets (and also underlying OS of course).

Sure, must be verified in advance.

> You see, nothing really new, nothing really difficult.

Absolutely; adding one more byte to the length of the current source  
pointer is a matter of minutes (followed by roughly the time it'd take for  
#condenseChanges and friends).

But will that byte add you productivity, flexibility, reliability, choice,  
collaborative facilities?

> With just a little bit more reflexion we can go in this direction and  
> that should be enough

What a different *one* byte makes :)

> for a few years of development and future enhancements...

No. Comparing just the capacity (in bytes) is about quantity, and quality  
would be ignored (once again, Marcus, you're right with your critique).

The existing source code subsystem is for a one man show, you and your  
.changes file (regardless where it's stored).

No collaborative work backed by a repository possible, except you do it  
all manually: on a COMPUTER, of course!

/Klaus

> Nicolas


Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Marcus Denker

On 27.07.2006, at 09:39, Klaus D. Witzel wrote:

> Nicolas
>
> thank you for taking the time for a response.
>
> On Wed, 26 Jul 2006 20:55:44 +0200, you wrote:
>
>> Klaus, i would say this is another source version control system.
>
> I did not have version *control* in mind but yes, scs could be used  
> for such ancient concepts (used them for > 30 years, quite an  
> experience).
>
> No, I was lead by the aim for more collaborative features (hmm,  
> since there are none in Squeak, should I say for *at least* one or  
> two collaborative features).
>
What I would suggest: Do not plan to save the world, at least not at  
first. If you now set out to build "the solution" we won't have for a  
looong
time, most likely never.

What we need first is

  - source offset in a property, not at the end of the bytecode. Use  
an Integer , so we get indefinite size of .source and .changens --
    One Problem Solved.
- Fix dan's source compression. This can work on the .changes, too,  
giving smaller releases or full development deployment
   for embedded systems (dan it for the weather station) or web servers.

With that, we have solved those problems that started the  
discussion... quite a step in the right direction.

After that, I personally would continue by cleaning up the sources/
changes handling and fileOut mechanism, thus then enabeling
experiments to make something really better. But this "real better"  
thing is the last step, not the first.

1) fix the real problem
2) clean up / refactor / generalise
3) now invent the future...

     Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

stéphane ducasse-2
In reply to this post by Klaus D. Witzel
> List,
>
> I put together what I think is relevant for a new source code  
> subsystem and, if I'm mistaken or have overlooked one of your  
> recent suggestions and/or requirements, please do not hesitate and  
> say so.
>
> What follows is not intended as a replacement for MC or SM, rather  
> the new subsystem should be as independent as possible. If you load  
> something with MC, the new subsystem can treat it like any other  
> contribution (more about contributions later).
>
> Trying to keep preliminaries as short as possible, we have as  
> organizational elements (for the sake of discussion): system  
> categories, classes, message categories and methods. The unit of  
> source code "entity" we consider to be a unit of change belongs to  
> one of these elements. We can add, replace, rename and delete. We  
> have not only methods, but also DoIt's (definitions,  
> organizational, etc) to take into account. And let's say that the  
> browser is our tool, just to keep it simple here.

I would really like to have a distinction between Doit and  
definition. Because this way we could build much clever tools. In VW  
classDef are not doit and you can eliminate doits when you want to do  
a replay all.

Do I understand correctly by saying that this would be a component  
that could be put between the sources/changes and MC for example.

> -------------------------------
>
> At the time an image is distributed, it is supposed to be in sync  
> (not necessarily *full* sync) with its respective source code  
> subsystem. So when you open a browser, all entities are colored  
> black on white, and my story begins.
>
> When you want to *observe* a certain area of the source code  
> subsytem, you put the respective entity (item from here on) onto  
> your watchlist (cmd+ if you like) from where the item gets the  
> color blue. After putting a class onto your watchlist you get all  
> its message categories and methods blue; un-watch those (sub)items  
> which are of no interest to you; etc for other organizational levels.
>
> When you do a *change* to a blue colored item, its new source is  
> contributed to the source code subsystem (from here on scs for  
> short) and the item is colored green in your browser. If later  
> somebody else contributes a change to the same blue item, it gets  
> the color yellow in both browsers.
>
> And if an entity you have in your image cannot be *matched* with  
> the scs, it is colored red.
>
> As is tradition, if you don't save your image before the next  
> crash, all this coloring information is lost, and you are back to  
> black and white. The scs does NOT know the state of your image.
>
> -------------------------------
>
> A contribution to the scs either has a predecessor or else it is  
> new (to the scs). Let's say three developers make contributions to  
> the same blue item, then we arrive at three branches (and the three  
> browsers color the respective item yellow). It is up to you to  
> accept one of these branches, but you are no forced to do so. An  
> accepted branch is colored green (it will turn yellow when the next  
> contribution relative to it arrives).
>
> -------------------------------
>
> By chasing the predecessor it is always possible to roll-back an  
> item (or a group of items), this is independent of the coloring I  
> used for illustration. The same "backwards" information is readily  
> available for analyzing the complete history of an item, including  
> all unaccepted branches.
>
> -------------------------------
>
> Doing this into the other direction (roll-forward) depends on which  
> branches where accepted. At any point in time, accepted branches  
> represent a "cut" in the scs.
>
> -------------------------------
>
> If you work alone on a project, you'll never see any yellow colored  
> item, unless you revert a change (because the dismissed becomes an  
> unaccepted branch). For people who are used to make small  
> incremental changes, a preference tells whether or not every change  
> is treated as a branch or else is to be consolidated until you do a  
> "big save" to the scs. The latter requires, I think, a local  
> buffering scs on your harddisk (or on your local area server).
>
> -------------------------------
>
> The amount of conversation between browser and scs is small, the  
> browser only has to ask for colors (one byte answer per item) for  
> those items it shows *and* which are on your watch list. But if you  
> want to see more (like, for example, all branches of an item), more  
> data will have to travel.
>
> -------------------------------
>
> It's now time for me to sit back and relax :-)
>
> /Klaus
>
>


Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

stéphane ducasse-2
In reply to this post by Nicolas Cellier-3
it would be nice to really see something like that happening.

Stef


> Klaus, i would say this is another source version control system.
>
> This bottom up approach is interesting but will not work. Squeakers  
> will continue to use their preferred tools, not your specific browser.
>
> Let's be more pragmatic and have a small incremental bottom up  
> approach.
>
> I see two solutions:
> - store source code and history in image... I do not believe we  
> should go in this direction even in compressed forms. We are rather  
> making efforts to make images shorter... But you can have other  
> opinion of course.
> - continue storing source in external repositories (files, URL,  
> NET  servers...)
>
> In the later case, we could have many files instead of two (sources  
> changes) and we should be able to have large files: conclusion:  
> let's just change source code pointer encoding.
>
> Using SmallIntegers (30 bits?) we can address 1GByte of space. More  
> if bits are not encoded as file offset but as Oop on a server...
>
> Using two SmallInteger, we have at least 60 bits, 1 ExaByte of  
> data! Let's go for it (count how many compiledMethod are in your  
> image... and you'll see this is not a drama).
>
> Maybe we can have an encoding supporting 1 single SmallInteger for  
> certain patterns (for example it can be sufficient for code stored  
> in 'sources' file, 'sources' file could also be organised with an  
> indirection table instead of using byte offsets...Given average  
> source code length, that can save a factor between 10 and 100 i  
> presume).
>
> We should be able to extend source code being in several places in  
> the future... (several files or URL or ...). So let use change  
> encoding by splitting bits : 20 bits for sourceRegistry offset, 40  
> bits for file offset, or 30bits/30bits, or both (pattern starting  
> with bits 0001 will be 20/40, all other 30/30 for example).
>
> Then we have to create the SourceRegistry with objects able to  
> handle source code stored in files, in compressed files, on the  
> Net, with all necessary caching to reduce number of connections and  
> design of the source code servers...
>
> However, i would keep the principle of copying changes in the  
> 'changes' file to recover easily in case of image crash, and  
> commiting to another source would still be the responsibility of  
> user, as is now.
>
> Small details: make sure the FileSystem primitives can deal with  
> huge offsets (and also underlying OS of course).
>
> You see, nothing really new, nothing really difficult.
> With just a little bit more reflexion we can go in this direction  
> and that should be enough for a few years of development and future  
> enhancements...
>
> Nicolas
>
> Klaus D. Witzel:
>> List,
>>
>> I put together what I think is relevant for a new source code  
>> subsystem
>> and, if I'm mistaken or have overlooked one of your recent  
>> suggestions
>> and/or requirements, please do not hesitate and say so.
>>
>> What follows is not intended as a replacement for MC or SM, rather  
>> the new
>> subsystem should be as independent as possible. If you load  
>> something with
>> MC, the new subsystem can treat it like any other contribution  
>> (more about
>> contributions later).
>>
>> Trying to keep preliminaries as short as possible, we have as
>> organizational elements (for the sake of discussion): system  
>> categories,
>> classes, message categories and methods. The unit of source code  
>> "entity"
>> we consider to be a unit of change belongs to one of these  
>> elements. We
>> can add, replace, rename and delete. We have not only methods, but  
>> also
>> DoIt's (definitions, organizational, etc) to take into account.  
>> And let's
>> say that the browser is our tool, just to keep it simple here.
>>
>> -------------------------------
>>
>> At the time an image is distributed, it is supposed to be in sync  
>> (not
>> necessarily *full* sync) with its respective source code  
>> subsystem. So
>> when you open a browser, all entities are colored black on white,  
>> and my
>> story begins.
>>
>> When you want to *observe* a certain area of the source code  
>> subsytem, you
>> put the respective entity (item from here on) onto your watchlist  
>> (cmd+ if
>> you like) from where the item gets the color blue. After putting a  
>> class
>> onto your watchlist you get all its message categories and methods  
>> blue;
>> un-watch those (sub)items which are of no interest to you; etc for  
>> other
>> organizational levels.
>>
>> When you do a *change* to a blue colored item, its new source is
>> contributed to the source code subsystem (from here on scs for  
>> short) and
>> the item is colored green in your browser. If later somebody else
>> contributes a change to the same blue item, it gets the color  
>> yellow in
>> both browsers.
>>
>> And if an entity you have in your image cannot be *matched* with  
>> the scs,
>> it is colored red.
>>
>> As is tradition, if you don't save your image before the next  
>> crash, all
>> this coloring information is lost, and you are back to black and  
>> white.
>> The scs does NOT know the state of your image.
>>
>> -------------------------------
>>
>> A contribution to the scs either has a predecessor or else it is  
>> new (to
>> the scs). Let's say three developers make contributions to the  
>> same blue
>> item, then we arrive at three branches (and the three browsers  
>> color the
>> respective item yellow). It is up to you to accept one of these  
>> branches,
>> but you are no forced to do so. An accepted branch is colored  
>> green (it
>> will turn yellow when the next contribution relative to it arrives).
>>
>> -------------------------------
>>
>> By chasing the predecessor it is always possible to roll-back an  
>> item (or
>> a group of items), this is independent of the coloring I used for
>> illustration. The same "backwards" information is readily  
>> available for
>> analyzing the complete history of an item, including all unaccepted
>> branches.
>>
>> -------------------------------
>>
>> Doing this into the other direction (roll-forward) depends on which
>> branches where accepted. At any point in time, accepted branches  
>> represent
>> a "cut" in the scs.
>>
>> -------------------------------
>>
>> If you work alone on a project, you'll never see any yellow  
>> colored item,
>> unless you revert a change (because the dismissed becomes an  
>> unaccepted
>> branch). For people who are used to make small incremental changes, a
>> preference tells whether or not every change is treated as a  
>> branch or
>> else is to be consolidated until you do a "big save" to the scs. The
>> latter requires, I think, a local buffering scs on your harddisk  
>> (or on
>> your local area server).
>>
>> -------------------------------
>>
>> The amount of conversation between browser and scs is small, the  
>> browser
>> only has to ask for colors (one byte answer per item) for those  
>> items it
>> shows *and* which are on your watch list. But if you want to see more
>> (like, for example, all branches of an item), more data will have to
>> travel.
>>
>> -------------------------------
>>
>> It's now time for me to sit back and relax :-)
>>
>> /Klaus
>>
>>
>>
>
>
> ______________________________________________________________________
> __
> iFRANCE, exprimez-vous !
> http://web.ifrance.com
>


Reply | Threaded
Open this post in threaded view
|

Old source code subsystem [was: New source code subsystem]

Klaus D. Witzel
In reply to this post by Marcus Denker
On Thu, 27 Jul 2006 09:57:24 +0200, Marcus Denker wrote:

> On 27.07.2006, at 09:39, Klaus D. Witzel wrote:
>> No, I was lead by the aim for more collaborative features (hmm,
>> since there are none in Squeak, should I say for *at least* one or
>> two collaborative features).
>>
>
> What I would suggest: Do not plan to save the world, at least not at
> first. If you now set out to build "the solution" we won't have for a
> looong
> time, most likely never.
>
> What we need first is
>
>   - source offset in a property, not at the end of the bytecode. Use
> an Integer , so we get indefinite size of .source and .changens --
>     One Problem Solved.

Wait a minute, a quick question before that can be solved

(nil systemNavigation
        allMethodsSelect: [:cm | (cm fileIndex = 0
        or: [(SourceFiles at: cm fileIndex) == nil])
        or: [cm filePosition = 0]])asArray

This has size 204 in #7048, do the elements have anything to do with  
potential changes to SourceFiles, fileIndex or filePosition?

/Klaus


Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Michael Rueger-6
In reply to this post by Marcus Denker
Marcus Denker wrote:

>  - source offset in a property, not at the end of the bytecode. Use an
> Integer , so we get indefinite size of .source and .changens --
>    One Problem Solved.

in a smalltalk system far away a full object was used for pointing to
the source. That object would normally be a source and file index,
allowing for unlimited number of source files. That was used in the
AppMan package that kept the package specific source in package source
files. So installing a package did not add any entries to the changes file.
But, that source object could also be an object answering the compressed
source, a repository access or whatever smart way you could come up with.

Michael


Reply | Threaded
Open this post in threaded view
|

Re: Old source code subsystem [was: New source code subsystem]

Marcus Denker
In reply to this post by Klaus D. Witzel

On 27.07.2006, at 10:47, Klaus D. Witzel wrote:

> On Thu, 27 Jul 2006 09:57:24 +0200, Marcus Denker wrote:
>> On 27.07.2006, at 09:39, Klaus D. Witzel wrote:
>>> No, I was lead by the aim for more collaborative features (hmm,
>>> since there are none in Squeak, should I say for *at least* one or
>>> two collaborative features).
>>>
>>
>> What I would suggest: Do not plan to save the world, at least not at
>> first. If you now set out to build "the solution" we won't have for a
>> looong
>> time, most likely never.
>>
>> What we need first is
>>
>>   - source offset in a property, not at the end of the bytecode. Use
>> an Integer , so we get indefinite size of .source and .changens --
>>     One Problem Solved.
>
> Wait a minute, a quick question before that can be solved
>
> (nil systemNavigation
> allMethodsSelect: [:cm | (cm fileIndex = 0
> or: [(SourceFiles at: cm fileIndex) == nil])
> or: [cm filePosition = 0]])asArray
>
> This has size 204 in #7048, do the elements have anything to do  
> with potential changes to SourceFiles, fileIndex or filePosition?
>
These are generated methods without source. The source you see when  
looking at them is the result of a decompilation.
(see e.g Preferences class>>#compileAccessMethodForPreference: ).  
Used for Preferences, Services and in some
old stuff, PlayWithMe1 should be removed...).

CompiledMethod>>#getSourceFor: selector in: class
        "Retrieve or reconstruct the source code for this method."
        | source flagByte sourceSelector |
        flagByte _ self last.
        (flagByte = 0
                or: [flagByte = 251 "some source-less methods have flag = 251, rest  
= 0"
                        and: [((1 to: 3) collect: [:i | self at: self size - i]) = #(0 0  
0)]])
                ifTrue: ["No source pointer -- decompile without temp names"
                ^ self decompileString].
        ......

        Marcus


smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Old source code subsystem [was: New source code subsystem]

Klaus D. Witzel
In reply to this post by Klaus D. Witzel
The 204 methods without sourcePointer in #7048 are from

        BrowserProvider
        WorldMenuProvider
        PlayWithMe1
        PlayWithMe1 class
        Preferences class

Marcus, can that be fixed (fileIn from some source/package) in the next to  
come image.

If anybody has up-to-date sources for the above which are not in SqSource  
or on SqMap would you please make them available, thank you.

/Klaus

On Thu, 27 Jul 2006 10:47:29 +0200, Klaus D. Witzel  
<[hidden email]> wrote:

> On Thu, 27 Jul 2006 09:57:24 +0200, Marcus Denker wrote:
>> On 27.07.2006, at 09:39, Klaus D. Witzel wrote:
>>> No, I was lead by the aim for more collaborative features (hmm,
>>> since there are none in Squeak, should I say for *at least* one or
>>> two collaborative features).
>>>
>>
>> What I would suggest: Do not plan to save the world, at least not at
>> first. If you now set out to build "the solution" we won't have for a
>> looong
>> time, most likely never.
>>
>> What we need first is
>>
>>   - source offset in a property, not at the end of the bytecode. Use
>> an Integer , so we get indefinite size of .source and .changens --
>>     One Problem Solved.
>
> Wait a minute, a quick question before that can be solved
>
> (nil systemNavigation
> allMethodsSelect: [:cm | (cm fileIndex = 0
> or: [(SourceFiles at: cm fileIndex) == nil])
> or: [cm filePosition = 0]])asArray
>
> This has size 204 in #7048, do the elements have anything to do with  
> potential changes to SourceFiles, fileIndex or filePosition?
>
> /Klaus
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Old source code subsystem [was: New source code subsystem]

Marcus Denker

On 27.07.2006, at 11:36, Klaus D. Witzel wrote:

> The 204 methods without sourcePointer in #7048 are from
>
> BrowserProvider
> WorldMenuProvider
> PlayWithMe1
> PlayWithMe1 class
> Preferences class
>
> Marcus, can that be fixed (fileIn from some source/package) in the  
> next to come image
No, these methods are generated, there was never code for them. We  
need to be able to
support code-less methods... even if it is very bad style, IMHO.

     Marcus




smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Old source code subsystem [was: New source code subsystem]

Klaus D. Witzel
Hi Marcus,

on Thu, 27 Jul 2006 11:40:37 +0200, you <[hidden email]> wrote:

> On 27.07.2006, at 11:36, Klaus D. Witzel wrote:
>
>> The 204 methods without sourcePointer in #7048 are from
>>
>> BrowserProvider
>> WorldMenuProvider
>> PlayWithMe1
>> PlayWithMe1 class
>> Preferences class
>>
>> Marcus, can that be fixed (fileIn from some source/package) in the
>> next to come image
>
> No, these methods are generated, there was never code for them.

Filed the classes out, no problem seen (doesn't say much, as we still have  
no consensus on aCompiledMethod>>#= for compiling the fileOut and doing a  
= for comparision with the in-image methods ;-)

> We
> need to be able to
> support code-less methods... even if it is very bad style, IMHO.

Yes, must support them.

But how about Decompiler [which is involved during fileOut ( = source code  
move) of sourceless methods]? Would you trust Decompiler in the light of  
its bugs?

/Klaus

>      Marcus
>



Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Pavel Krivanek
In reply to this post by Marcus Denker
I have to draw the attention to the next one perspective. If we want
to have future versions of Squeak based on some small UI-less minimal
image, we will have got quite small sources file, big changes and we
will lose history information (we will have versions of individual
packages).
The question is if we want to have compression in the kernel image (to
support the compressed sources).

-- Pavel

On 7/27/06, Marcus Denker <[hidden email]> wrote:

>
> On 27.07.2006, at 09:39, Klaus D. Witzel wrote:
>
> > Nicolas
> >
> > thank you for taking the time for a response.
> >
> > On Wed, 26 Jul 2006 20:55:44 +0200, you wrote:
> >
> >> Klaus, i would say this is another source version control system.
> >
> > I did not have version *control* in mind but yes, scs could be used
> > for such ancient concepts (used them for > 30 years, quite an
> > experience).
> >
> > No, I was lead by the aim for more collaborative features (hmm,
> > since there are none in Squeak, should I say for *at least* one or
> > two collaborative features).
> >
>
> What I would suggest: Do not plan to save the world, at least not at
> first. If you now set out to build "the solution" we won't have for a
> looong
> time, most likely never.
>
> What we need first is
>
>   - source offset in a property, not at the end of the bytecode. Use
> an Integer , so we get indefinite size of .source and .changens --
>     One Problem Solved.
> - Fix dan's source compression. This can work on the .changes, too,
> giving smaller releases or full development deployment
>    for embedded systems (dan it for the weather station) or web servers.
>
> With that, we have solved those problems that started the
> discussion... quite a step in the right direction.
>
> After that, I personally would continue by cleaning up the sources/
> changes handling and fileOut mechanism, thus then enabeling
> experiments to make something really better. But this "real better"
> thing is the last step, not the first.
>
> 1) fix the real problem
> 2) clean up / refactor / generalise
> 3) now invent the future...
>
>      Marcus
>
>
>
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Edgar J. De Cleene
Pavel Krivanek puso en su mail :

> I have to draw the attention to the next one perspective. If we want
> to have future versions of Squeak based on some small UI-less minimal
> image, we will have got quite small sources file, big changes and we
> will lose history information (we will have versions of individual
> packages).
> The question is if we want to have compression in the kernel image (to
> support the compressed sources).
Yes, you need it at some point.

Could have a minimal support for gz only , not for all kinds of compression.

I do test for .sources and .gz and is almost the same size as Dan trick and
your could load .sources.gz or .changes.gz and keep in RWBinaryOrTextStream
on startUp for later use.

Again , wonderful job, still trying  to understand all .

To masters, we need a special "kernel folder" in ftp site , as now we have
one for Spoon

Edgar



       
       
               
__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas


Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Klaus D. Witzel
In reply to this post by Pavel Krivanek
On Thu, 27 Jul 2006 13:15:11 +0200, Pavel Krivanek wrote:

> I have to draw the attention to the next one perspective. If we want
> to have future versions of Squeak based on some small UI-less minimal
> image, we will have got quite small sources file, big changes and we
> will lose history information (we will have versions of individual
> packages).
> The question is if we want to have compression in the kernel image (to
> support the compressed sources).

+1 a GUI-less (or small-GUI) image will most likely access compressed  
contents, for various reasons.

/Klaus


Reply | Threaded
Open this post in threaded view
|

Re: Old source code subsystem

Klaus D. Witzel
In reply to this post by Marcus Denker
On Thu, 27 Jul 2006 09:57:24 +0200, Marcus Denker wrote:
...
> What we need first is
>
>   - source offset in a property, not at the end of the bytecode.

Done. Would some good soul please explain why the file index bits are at  
the left of the current sourcePointer (need no explanation for the bytes  
at the end of the bytecode, just about the intention for the encoding).

Especially, does the current sourcePointer encoding support other than the  
TWO existing SourceFiles? If not, that would make migration easier.

> Use
> an Integer , so we get indefinite size of .source and .changens --

It now has (SmallInteger maxVal bitShift: -1) for position and one bit for  
file index. This gives 512MB capacity (16 fold) for the moment.

The transformation is {already} factored in StandardSourceFileArray, so  
any change (for example to an oop) will be a piece of cake.

>     One Problem Solved.

After all the tests pass :)

/Klaus


Reply | Threaded
Open this post in threaded view
|

Towards a small kernel image

Bryce Kampjes
In reply to this post by Pavel Krivanek
Pavel Krivanek writes:
 > I have to draw the attention to the next one perspective. If we want
 > to have future versions of Squeak based on some small UI-less minimal
 > image, we will have got quite small sources file, big changes and we
 > will lose history information (we will have versions of individual
 > packages).
 > The question is if we want to have compression in the kernel image (to
 > support the compressed sources).

Are there any plans to use your small image as the official base
image? Using a small base image then importing externally supported
packages should make image maintenence much easier as there will be
less code. Is this a good time to burn bridges to move forward to
a more modular future?

Sure there's a risk that we might drop code, but code that isn't
maintained will rot inside the image as well as outside. If people
know that packages will be gone unless they do something, there is a
chance that they will step forward knowing the stakes even though they
might not have volunteered if the code was in the image rotting slowly
through lack of attention.

Bryce

Reply | Threaded
Open this post in threaded view
|

Re: Towards a small kernel image

Pavel Krivanek
> Are there any plans to use your small image as the official base
> image? Using a small base image then importing externally supported
> packages should make image maintenence much easier as there will be
> less code. Is this a good time to burn bridges to move forward to
> a more modular future?

Good question. Version 3.9 is in the final stage and I think that
Stef, Marcus, Cees, Craig, Bert, Tim and Yoshiki should set the plan
for 3.10 - goals and priorities.

Stef declared that he wants to burn bridges in this version.

We want to have Squeak free and modular and there are three possible
starting points.  The free version One, Craig's Spoon and the image of
mine.

I wanted to show that my way is the right one. We have some GUI-less
image with SUnit tests, we are able to run Seaside applications and we
are able to load the rest of current Squeak sources and run Morphic.

Unfortunately there's almost nobody who can maintain the next release.
And if we will create the basic kernel imege, there will be no human
resources to unify it as the base for the current Squeak forks.

-- Pavel

Reply | Threaded
Open this post in threaded view
|

Doits vs. methods (was: Re: New source code subsystem)

Wolfgang Eder
In reply to this post by stéphane ducasse-2
stéphane ducasse wrote:
[lots of stuff snipped]

>> Trying to keep preliminaries as short as possible, we have as
>> organizational elements (for the sake of discussion): system
>> categories, classes, message categories and methods. The unit of
>> source code "entity" we consider to be a unit of change belongs to one
>> of these elements. We can add, replace, rename and delete. We have not
>> only methods, but also DoIt's (definitions, organizational, etc) to
>> take into account. And let's say that the browser is our tool, just to
>> keep it simple here.
>
> I would really like to have a distinction between Doit and definition.
> Because this way we could build much clever tools. In VW classDef are
> not doit and you can eliminate doits when you want to do a replay all.
>
> Do I understand correctly by saying that this would be a component that
> could be put between the sources/changes and MC for example.

I would like to make a point that Doits should not be
a separate concept:
I am aware that it is common practice to have Doit
methods in workspaces, for example.
For a long time now, I and my colleages have stored all
the Doits as methods in a Script class made for this purpose.

The rationale is that Doits are part of the code,
and should be treated the same by the CM system (Envy in our case).

This works out very well. I especially like the fact that
refactorings reliably catch the Doits as well, since they are
normal methods.

I cannot see any disatvantages from this approach, and it keeps
things really simple, imho.

Thanks,
Wolfgang

Reply | Threaded
Open this post in threaded view
|

Re: New source code subsystem

Klaus D. Witzel
In reply to this post by stéphane ducasse-2
On Thu, 27 Jul 2006 10:22:09 +0200, stéphane ducasse  
<[hidden email]> wrote:

> Klaus wrote:
>> I put together what I think is relevant for a new source code subsystem  
>> and, if I'm mistaken or have overlooked one of your recent suggestions  
>> and/or requirements, please do not hesitate and say so.
>>
>> What follows is not intended as a replacement for MC or SM, rather the  
>> new subsystem should be as independent as possible. If you load  
>> something with MC, the new subsystem can treat it like any other  
>> contribution (more about contributions later).
>
> I would really like to have a distinction between Doit and definition.  
> Because this way we could build much clever tools. In VW classDef are  
> not doit and you can eliminate doits when you want to do a replay all.

Sure. In my huble opinion there must be a backpointer for each and every  
class,trait definition like it is for method definitions and class  
comments. Alternately we can provide
- http://www.sticky-notes.net/
together with all Squeak distributions (putting the [|] logo on them as  
default) such that the developer can write down the previous definition in  
case they are going to be changed ;-)

> Do I understand correctly by saying that this would be a component that  
> could be put between the sources/changes and MC for example.

I think it'll work on the sources/changes interface (and unify both  
files). If MC & friends use that interface, fine. I have as yet not  
completed analysis of what interface MC and SM (the prominent examples)  
use from the "base" image. I'd appreciate someone shares the respective  
requirement def's (of MC and SM), that'd save me quite some time.

/Klaus


12