[squeak-dev] Perl is to CPAN as Squeak is to (what)?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
91 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Igor Stasenko
2008/6/30 Avi Bryant <[hidden email]>:

> On Mon, Jun 30, 2008 at 7:57 AM, Colin Putney <[hidden email]> wrote:
>
>> The other big problem is that tens of thousands of tiny files is a horribly
>> inefficient way to store source code. Yes, disk is cheap. But disk IO is
>> not. I discovered this early in the development of MC2, when I implemented a
>> type of repository that stored each method in a separate file. Loading
>> OmniBrowser from that repository involved opening, reading, and closing over
>> 600 files, and was very slow. I don't remember the exact timing, but I think
>> it was like 5 to 10 minutes, and in any case it was far too slow. Avi wrote
>> a repository that stored every thing in a single indexed file, and now load
>> time is dominated by compilation.
>
> It's worth pointing out that file-based version control has advanced
> significantly since we did this work - CVS and SVN are now far from
> the state of the art.  I haven't used git much, for example, but it
> seems to be a well layered system, and it may be that we can build an
> alternative front end to its database which is image-based rather than
> working directory based.  For example, imagine comparing an image
> directly to this index file rather than to a directory full of files
> on disk:
>
> http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#the-index
>
> And look at this description of the workflow:
>
> http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#the-workflow
>
> I personally believe that we're better off with Smalltalk-specific
> version control, but if someone *is* looking at integration with more
> mainstream tools, I would strongly suggest they start with git rather
> than SVN.
>

This is quite interesting.
Really, if we could just avoid using file-based working directory and
make image to be a kind of 'working directory' this would simplify
things  a lot - no need to make file outs and synchronization is
always made between current methods in image and git repository.
Interesting, how much coding needed to connect image with git
repository in that way.

> Avi
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Avi Bryant-2
On Mon, Jun 30, 2008 at 11:10 AM, Igor Stasenko <[hidden email]> wrote:

> This is quite interesting.
> Really, if we could just avoid using file-based working directory and
> make image to be a kind of 'working directory' this would simplify
> things  a lot - no need to make file outs and synchronization is
> always made between current methods in image and git repository.
> Interesting, how much coding needed to connect image with git
> repository in that way.

Well, I don't know how much coding it would be, but from a *very*
cursory look I would say what you need to do is model the index file
format in Smalltalk, and reproduce the update-index and checkout-index
commands.  Once you had that, you could reuse all the normal
merge/commit/etc commands from git, because those all operate on the
index and on the repository database.

Avi

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Avi Bryant-2
On Mon, Jun 30, 2008 at 11:18 AM, Avi Bryant <[hidden email]> wrote:

> Well, I don't know how much coding it would be, but from a *very*
> cursory look I would say what you need to do is model the index file
> format in Smalltalk, and reproduce the update-index and checkout-index
> commands.  Once you had that, you could reuse all the normal
> merge/commit/etc commands from git, because those all operate on the
> index and on the repository database.

Here's (I think) the equivalent Java code to what you'd need...

http://repo.or.cz/w/egit.git?a=blob;f=org.spearce.jgit/src/org/spearce/jgit/lib/GitIndex.java;h=5be404e63041471d64a45e268d0ee04a7378d3c6;hb=HEAD

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Stephen Pair
In reply to this post by Avi Bryant-2


On Mon, Jun 30, 2008 at 1:53 PM, Avi Bryant <[hidden email]> wrote:
On Mon, Jun 30, 2008 at 7:57 AM, Colin Putney <[hidden email]> wrote:

> The other big problem is that tens of thousands of tiny files is a horribly
> inefficient way to store source code. Yes, disk is cheap. But disk IO is
> not. I discovered this early in the development of MC2, when I implemented a
> type of repository that stored each method in a separate file. Loading
> OmniBrowser from that repository involved opening, reading, and closing over
> 600 files, and was very slow. I don't remember the exact timing, but I think
> it was like 5 to 10 minutes, and in any case it was far too slow. Avi wrote
> a repository that stored every thing in a single indexed file, and now load
> time is dominated by compilation.

It's worth pointing out that file-based version control has advanced
significantly since we did this work - CVS and SVN are now far from
the state of the art.  I haven't used git much, for example, but it
seems to be a well layered system, and it may be that we can build an
alternative front end to its database which is image-based rather than
working directory based.  For example, imagine comparing an image
directly to this index file rather than to a directory full of files
on disk:

http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#the-index

And look at this description of the workflow:

http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#the-workflow

I personally believe that we're better off with Smalltalk-specific
version control, but if someone *is* looking at integration with more
mainstream tools, I would strongly suggest they start with git rather
than SVN.

Avi

I've been using git quite a bit lately and really like its approach (though I've heard good things about Hg too).  I liked it so much that I did a Smalltalk version of it a while back.  The idea was to do basically what git does, but make it general purpose such that it could be used for managing versions of any object for which you could define the serialization.  I started out making it do version control for contacts and later hooked it up to store Monticello packages (and built a new merge tool in the process).  I've had it mostly done for several months but have resisted mentioning it, because once you mention something, you're on the hook to get it out.  I'll see if I can clean it up enough in the next few days and publish it.  I had hoped to record a screencast to show both how to use it and how the code is currently organized...probably won't do that, but I'll try to get the code out soon.

- Stephen



Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion

keith1y
Having tried Hg and looked at git, I have decided that I much prefer
bzr, since it offers mor flexibility and a lightweight checkout.

Keith


Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Bootstrapping (Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?))

Yoshiki Ohshima-2
In reply to this post by Ralph Johnson
At Sun, 29 Jun 2008 07:31:26 -0500,
Ralph Johnson wrote:

>
> Pavel Krivanek
> http://www.comtalk.eu/Squeak/98
>
> On Sun, Jun 29, 2008 at 4:09 AM, Yoshiki Ohshima <[hidden email]> wrote:
>
> >  - Write a compiler in another language.  That can generates the bits
> >    that are same as CompiledMethods.  For a class definition, it
> >    creates (yes) the network of pointers.
> >
> >  - The compiler sticks the class definitions, method dictionaries,
> >    subclass structure, and compiled methods into a big "int*" array.
> >    The goal is to make that something just run, so for example, the
> >    stuff managed by ClassDescription (instanceVariable names and
> >    organization) don't have to be compiled.  Stuff like the source
> >    pointer is not needed at this stage.
>
> The compiler doesn't have to be in another language.  What is
> important is that the compiler builds a Squeak image from a text file.
>  The compiler can be written in Smalltalk as long as it uses only the
> class definitions in the test file, not in the compiler's image.

  This is more or less settled but...

  If the compiler is written in Smalltalk, we would have a binary
image to store the compiler; and some people wouldn't like it.
Nonetheless, it can be in Slang and we can just show the translated C
code as if it is the "egg".

-- Yoshiki

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Bootstrapping

Yoshiki Ohshima-2
In reply to this post by Bert Freudenberg
At Sun, 29 Jun 2008 20:27:33 +0200,
Bert Freudenberg wrote:

>
>
> Am 29.06.2008 um 20:08 schrieb Yoshiki Ohshima:
>
> > At Sun, 29 Jun 2008 18:48:01 +0200,
> > Bert Freudenberg wrote:
> >>
> >> Am 29.06.2008 um 16:51 schrieb Randal L. Schwartz:
> >>
> >>>>>>>> "Bert" == Bert Freudenberg <[hidden email]> writes:
> >>>
> >>> Bert> No it would not. The main issue for them is that you have to
> >>> start  from what
> >>> Bert> they perceive as "binary blob" which is monkey-patched into
> >>> newer versions.
> >>>
> >>> The C compiler would fit the same definition, by that reasoning.
> >>
> >>
> >> No. You need a C compiler, true, but it builds the next C compiler
> >> from text sources only, it does not clone itself.
> >
> >  Come to think of it, we don't have to write the bootstrapper in
> > another language.  They already accept Squeak VM, so we can write the
> > Smalltalk-to-CompiledMethod compiler in Slang (even better; write it
> > in OMeta and generate Slang), and generate a modified VM that has
> > different main().  That main() reads the "source" files, compile them
> > (by using the memory management in ObjectMemory), and write the result
> > to file.  (Yes, you need a C compiler^^;)
> >
> >  Isn't it sound a bit more doable?
>
>
> A bit. But I suspect getting classes and methods assembled into an  
> image is not even half the work. We'll have to create objects, too,  
> that were manually assembled (I'm thinking of the PaintBox prototype  
> for example). Recreating a full Etoys image would still be a major  
> effort, since many parts would have to be rewritten to actually be  
> bootstrappable.

  Certainly, the PaintBox is one of the worst examples!  The basic
part may be able to "take off", but yes, there are stuff build around.

  It sounds like I'm not going to do it (of course)...

-- Yoshiki

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Bootstrapping (Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?))

Yoshiki Ohshima-2
In reply to this post by Igor Stasenko
> What Linux people proposing looks controversial to me.

  The attitude toward Etoys/Squeak vary widely among *them*, also.
Fortunately, the guys who are viciously bashing Etoys on the OLPC
devel list aren't the guys who make decisions there.  Hopefully a bit
by bit, we can make some progress.  That thread is great for these
decision makers because it is yet another example of how not to
discuss stuff^^;

-- Yoshiki

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Colin Putney
In reply to this post by Igor Stasenko

On 30-Jun-08, at 11:10 AM, Igor Stasenko wrote:

> This is quite interesting.
> Really, if we could just avoid using file-based working directory and
> make image to be a kind of 'working directory' this would simplify
> things  a lot - no need to make file outs and synchronization is
> always made between current methods in image and git repository.
> Interesting, how much coding needed to connect image with git
> repository in that way.

One option would be to use FUSE (or something similar) to make the  
image appear to be a filesystem. That wouldn't be git-specific, and  
would even appease all the folks that want to edit their code with  
emacs.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion

keith1y
Colin Putney wrote:

>
> On 30-Jun-08, at 11:10 AM, Igor Stasenko wrote:
>
>> This is quite interesting.
>> Really, if we could just avoid using file-based working directory and
>> make image to be a kind of 'working directory' this would simplify
>> things  a lot - no need to make file outs and synchronization is
>> always made between current methods in image and git repository.
>> Interesting, how much coding needed to connect image with git
>> repository in that way.
>
> One option would be to use FUSE (or something similar) to make the
> image appear to be a filesystem. That wouldn't be git-specific, and
> would even appease all the folks that want to edit their code with emacs.
>
> Colin
>
>
It has been done

Keith

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion

Igor Stasenko
2008/7/1 Keith Hodges <[hidden email]>:

> Colin Putney wrote:
>>
>> On 30-Jun-08, at 11:10 AM, Igor Stasenko wrote:
>>
>>> This is quite interesting.
>>> Really, if we could just avoid using file-based working directory and
>>> make image to be a kind of 'working directory' this would simplify
>>> things  a lot - no need to make file outs and synchronization is
>>> always made between current methods in image and git repository.
>>> Interesting, how much coding needed to connect image with git
>>> repository in that way.
>>
>> One option would be to use FUSE (or something similar) to make the image
>> appear to be a filesystem. That wouldn't be git-specific, and would even
>> appease all the folks that want to edit their code with emacs.
>>
>> Colin
>>
>>
> It has been done
>

Haha, really, run a file system adaptor in image. And then run git, or
any other file based version control tool on it.
The complication is , that most of such tools using subdirectories to
maintain own specific info (like CVS in CVS, and .svn with SVN).
So, you will need to make it able to store these files somewhere in
image , not only methods which are subject of version control.

> Keith
>
>



--
Best regards,
Igor Stasenko AKA sig.

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Michael Haupt-3
In reply to this post by Colin Putney
Hi Colin,

On Tue, Jul 1, 2008 at 4:07 AM, Colin Putney <[hidden email]> wrote:
> One option would be to use FUSE (or something similar) to make the image
> appear to be a filesystem. That wouldn't be git-specific, and would even
> appease all the folks that want to edit their code with emacs.

SqueakFS exists, and is able to make an image available to the outside
(file system) world. We're in the process of setting up a development
repository (based on SVN, mind you). In the meantime, it's available
from www.squeaksource.com/SqueakFS.html in the current version.

Best,

Michael

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion

Michael Haupt-3
In reply to this post by Igor Stasenko
Hi Igor,

On Tue, Jul 1, 2008 at 5:34 AM, Igor Stasenko <[hidden email]> wrote:
> Haha, really, run a file system adaptor in image. And then run git, or
> any other file based version control tool on it.

that sounds like heresy to die-hard Smalltalk image paracletes,
agreed. ;-) The approach taken for SqueakSVN, where the image talks,
via a plugin, to the version management libraries directly, is clearly
more appreciable.

> The complication is , that most of such tools using subdirectories to
> maintain own specific info (like CVS in CVS, and .svn with SVN).
> So, you will need to make it able to store these files somewhere in
> image , not only methods which are subject of version control.

Right, that would be an interesting task for the file system to manage.

Best,

Michael

>
>> Keith
>>
>>
>
>
>
> --
> Best regards,
> Igor Stasenko AKA sig.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Brent Pinkney-2
In reply to this post by Michael Haupt-3
Hi,

> SqueakFS exists, and is able to make an image available to the outside
> (file system) world. We're in the process of setting up a development
> repository (based on SVN, mind you). In the meantime, it's available
> from www.squeaksource.com/SqueakFS.html in the current version.

Do be so kind as to update the Project Description field on SqueakSource to describe this project.
It is a tad terse as it stands.

Brent

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Colin Putney
In reply to this post by Michael Haupt-3
On 30-Jun-08, at 10:51 PM, Michael Haupt wrote:

Hi Colin,

On Tue, Jul 1, 2008 at 4:07 AM, Colin Putney <[hidden email]> wrote:
One option would be to use FUSE (or something similar) to make the image
appear to be a filesystem. That wouldn't be git-specific, and would even
appease all the folks that want to edit their code with emacs.

SqueakFS exists, and is able to make an image available to the outside
(file system) world. We're in the process of setting up a development
repository (based on SVN, mind you). In the meantime, it's available
from www.squeaksource.com/SqueakFS.html in the current version.

Ah good. I must have run across it at some point and forgotten about it.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Lukas Renggli
> SqueakFS exists, and is able to make an image available to the outside
> (file system) world. We're in the process of setting up a development
> repository (based on SVN, mind you). In the meantime, it's available
> from www.squeaksource.com/SqueakFS.html in the current
> version.
>
> Ah good. I must have run across it at some point and forgotten about it.

There is also the FTP server that I wrote together with Ian at ESUG 2005.

http://www.cincomsmalltalk.com/userblogs/avi/blogView?showComments=true&printTitle=FTPlets&entry=3303394043

I used it for a while to develop from within an external editor, but
in the end the clumsy Squeak environment is still less annoying than
an external editor.

Lukas

--
Lukas Renggli
http://www.lukas-renggli.ch

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Colin Putney

On 30-Jun-08, at 11:37 PM, Lukas Renggli wrote:

>> SqueakFS exists, and is able to make an image available to the  
>> outside
>> (file system) world. We're in the process of setting up a development
>> repository (based on SVN, mind you). In the meantime, it's available
>> from www.squeaksource.com/SqueakFS.html in the current
>> version.
>>
>> Ah good. I must have run across it at some point and forgotten  
>> about it.
>
> There is also the FTP server that I wrote together with Ian at ESUG  
> 2005.

Yes, it's excellent. I use it for editing Javascript and CSS that live  
inside the image as part of OB-Web.

Colin

Reply | Threaded
Open this post in threaded view
|

Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Michael Haupt-3
In reply to this post by Brent Pinkney-2
Hi Brent,

On Tue, Jul 1, 2008 at 8:00 AM, Brent Pinkney <[hidden email]> wrote:
> Do be so kind as to update the Project Description field on SqueakSource to describe this project.
> It is a tad terse as it stands.

heh, indeed. :-) The Wiki page has loads of information instead.

I've put it on my to-do list; the description will be updated. Thanks
for pointing this out.

Best,

Michael

Reply | Threaded
Open this post in threaded view
|

SqueakSVN was: Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Robert Krahn
In reply to this post by Andreas.Raab
Hi,

it was mentioned already: We developed SqueakSVN, a project-centric  
code management tool, as a student project at HPI. It completely  
integrates SVN, so the workflow is different than the one from DVS.  
Everything can be controlled from within Squeak.

We are currently cleaning up the code and after we are done, SqueakSVN  
will be released.

Robert


On Jun 28, 2008, at 9:45 PM, Andreas Raab wrote:

> Can you say something more about that? A couple of weeks ago I saw a  
> demo at HPI in Potsdam where students used SVN down to the method  
> level, and it seemed to me that this approach might very well work  
> because the SVN granularity is the same as the in-image granularity.  
> It may also be interesting that this wasn't even trying to deal with  
> source files of any sort - it retained the nature of the image and  
> simply hooked it up directly with SVN. From my perspective this  
> looked like an extraordinarily interesting approach that I am  
> certain to try out as soon as it is available.
>
> Cheers,
>  - Andreas
>


Reply | Threaded
Open this post in threaded view
|

Re: SqueakSVN was: Re: [squeak-dev] Subversion (was: Re: Perl is to CPAN as Squeak is to (what)?)

Avi Bryant-2
On Tue, Jul 1, 2008 at 11:10 AM, Robert Krahn
<[hidden email]> wrote:
> Hi,
>
> it was mentioned already: We developed SqueakSVN, a project-centric code
> management tool, as a student project at HPI. It completely integrates SVN,
> so the workflow is different than the one from DVS. Everything can be
> controlled from within Squeak.

Just curious - how is the integration done?  Via a network protocol,
or using OSProcess, or some other way?

Avi

12345