[ANN] Git Browser and Squot

> Hello together,
>
> Over the past two years you may have seen occasional mentions of
> ???Squot??? and more likely of the ???Git Browser,??? which can be installed
> from the menus of a plain Squeak image since 5.2. In case you ever
> wondered: Squot and the Git Browser belong together. Let me finally
> introduce you to them today, since this is the outcome of my master
> thesis from back in 2017.
>
> ==What is Squot?==
>
> Squot is a version control system (VCS) for objects. Not only
> packages, but arbitrary objects (such as Morphs, so you do not have to
> put Smalltalk instructions how to recreate it in the VCS, but you can
> rather track the object itself).
>
> It uses an external VCS as its backend to store versions. As of now,
> only one implementation for Git exists (except for some test stub
> repositories for the unit tests). It should be possible to add further
> backend VCSs without changing the core of Squot.
>
> Squot is an abbreviation of ???Squeak object tracker,??? loosely based on
> Git???s description in its manual page: ???git - the stupid content
> tracker???.
>
> ==What is the Git Browser?==
>
> The Git Browser is a graphical user interface to operate on Squot
> working copies with a Git repository as a backend. It allows you to
> manage working copies, track packages (multiple per working copy,
> which are all versioned together), browse branches, create new
> commits, search for commits, synchronize with remote repositories, and
> perform checkouts and merges. It can be used without a Git outside of
> Squeak (it does not call external processes or shared libraries).
>
> The tool is primarily meant to maintain Squeak projects, which are
> published on GitHub or similar platforms, but can be used with any Git
> repository.
>
> ==Is it ready for use?==
>
> To work on Smalltalk projects: definitely yes. It has been used for at
> least three years in student courses at HPI (Potsdam). I am also using
> the Git Browser to work on Squot itself.
>
> To really track other objects than packages: possibly. It has some
> bugs left, so I would not recommend to rely on the tracking of objects
> other than packages for production use. For this reason, there is also
> no GUI yet to add other objects to a working copy in the Git Browser.
>
> There are occasional quirks when pushing to GitHub (???Connection
> closed???), so you might need to resort to the Git CLI sometimes.
> Currently you must also merge non-Smalltalk files outside of Squeak.
> To view the commit history as a graph, you can use external tools,
> such as gitk, SourceTree, or whatever your preferred Git GUI is.
>
> ==How can I use it?==
>
> 1) Install the Git Browser by choosing Tools > Git Browser in your
> Squeak image (since 5.2). This will also load several of dependencies
> via Metacello.
>
> 2) Once you have the Git Browser open, it will prompt you whether you
> want to create your first project now. If you would like to start a
> fresh and empty working copy on an empty Git repository, select ???yes???
> and provide a path for the Git repository. If you would rather like to
> clone an existing Git repository, select ???no.???
>
> 3) The Git browser itself has five panes:
>
>     * On the top left, there is a list of projects. These are your
> working copies. Each working copy links something in your Squeak image
> to a Git repository outside of that image. Via the context menu of
> this list, you can add, change, or delete working copies and
> repositories. If this list is empty, you will want to start with the
> context menu here.
>
>     * Below that, there is a list of branches in the currently
> selected project (if any). Via the context menu, you can add, rename,
> and remove branches, push to remote repositories, and switch branches.
> One branch is highlighted with a green dot; this is the branch that is
> currently checked-out. If you create new commits, they will go on this
> branch.
>
>     * On the top right, there is the list of commits on the current
> branch (if any). You can compare commits with each other or a commit
> with the working copy, search for commits using the search bar below
> the list (or via list filtering to only search the headlines), create
> a new branch at a commit, reset the current branch to an older commit,
> or check out or merge specific objects from a commit.
>
>     * In the bottom-left pane, the full log message of the selected
> commit will be displayed. In addition to that, it will also show the
> commit metadata, such as the author, committer, and the abbreviated
> SHA-1 hash of the commit and its parents.
>
>     * Finally on the bottom right, there is a list of objects in the
> selected commit. These are usually the packages known in a particular
> commit. Via the context menu you can checkout or merge an object with
> the working copy (read checkout like ???Load??? in Monticello and merge
> like ???Merge???), browse either the object as it was at the time of the
> commit, or what is currently loaded. You can also access the current
> scripts (preamble, postscript etc.) of packages here.
>
> 4) Between the panes, there is a strip of buttons. They operate on the
> current branch or the selected commit. At the far right, you can
> configure how things go to Git: that is your name and e-mail address
> for the commit metadata, the Smalltalk file format you want to produce
> (FileTree/Cypress or Tonel), and whether Squot should suppress writing
> its own configuration files (if you want to submit a pull request to a
> repository that is not developed with Squot, for example). Via the
> Feedback button, you can create an issue for Squot on GitHub directly
> from the image. You can also file these here:
> https://github.com/hpi-swa/Squot/issues
>
> 5) To add another package to a project, or remove a package from the
> set of tracked packages, open the menu of a project and choose ???Change
> tracked packages???. Below  that menu item, you can also change between
> writing FileTree/Cypress or Tonel files (???Set package storage
> format??????).
>
> 6) When you switch branches via the context menu of the branch list,
> note that it will stow away all your unsaved changes and the other
> branch is checked out clean. Your unsaved changes will be restored
> when you switch back to the original branch. This behavior is
> different from the Git CLI, and was conceived in an alternative Git
> frontend called ???Gitless??? (which addresses some of Git???s awkwardness).
>
> 7) When you want to commit, you will find that there is no Git staging
> area as far as the Git Browser and accompanying tools are concerned.
> Instead, you can choose which packages, classes, and methods you want
> to include or exclude in the commit dialog, like in Monticello in
> recent years.
>
> 8) When you checkout or merge something, you always first get a list
> of changes, from which you can choose what to load and what to skip.
>
> 9) To update the Git Browser when a new version was published, open
> the window dropdown menu (third button from the right in the title
> bar) and choose ???self-update??????. It pulls the latest version from the
> master branch on GitHub. Ongoing development happens on the ???develop???
> branch, not on master. You can also pull that version by choosing ???set
> self-update branch?????? in the same menu, and then invoke the update.
>
> ==How does it work internally? Why is it not a Monticello extension?==
>
> Squot knows of three presentations of an object: 1) the ???live??? object
> that is ???the real thing,??? which you can normally handle in Squeak, 2)
> one or more  ???shadows??? of the object, which is a captured
> representation of one version of the ???live??? object (a snapshot); it is
> used to reason about different versions of the same object without
> loading them, 3) a ???stored??? object, for example serialized into a
> file. Objects are tracked in graphs of objects. You add an object
> graph by adding a ???root object??? to the working copy, from which all
> the other relevant objects can be reached. For example, you could add
> a complex Morph, which would include all of its submorphs. The tracked
> objects can to a certain degree control how they are captured; for
> example, the ???root Morph??? will direct Squot to ignore its #owner
> instance variable (because otherwise you would always track the whole
> World). To be able to identify an object in different versions of a
> graph, each object gets a unique name assigned when it is first
> captured. Objects may opt out of getting a name if they are true value
> objects for which identity does not matter when a shadow object is
> rematerialized to a live object during a load/checkout. Identity does
> not matter when it makes no difference whether two objects refer to
> the same instance of the value, or a different, equal instance (like
> Number, Point), or if there is only one instance by definition (like
> UndefinedObject, Boolean, Symbol).
>
> Packages, which are the only kind of object you can currently track in
> Monticello (besides configurations of packages), are just a special
> case of all of this. Read the rest of this paragraph as an example of
> what I wrote in the previous paragraph. The live package is
> represented by its PackageInfo. It subsumes the loaded behaviors and
> methods that are attributed to the package. A shadow package basically
> wraps an MCSnapshot of the package (Squot reuses MCSnapshots and
> MCDefinitions under the hood to represent shadows of packages,
> behaviors, methods). A stored package is a tree of files in
> FileTree/Cypress or Tonel format, for example. In theory, it could
> also be any other file representation, such as a plain file-out, or a
> SmartRefStream that contains the MCSnapshot. A PackageInfo is the
> ???root object??? of a package object graph in the working copy.
> PackageInfo will tell Squot that it does not want a unique name
> assigned, because a PackageInfo is already identified by its
> #packageName. To create the shadow package, PackageInfo also overrides
> the capturing process for Squot to find the classes, and methods
> (which are not directly referenced) and turn them into an MCSnapshot
> with Monticello. When a package is loaded in Squot, it will also use
> Monticello to actually install the changes.
>
> As a general way to capture any kind of object, Squot has shadow
> classes to capture objects as collections of slots that refer to other
> objects. Instance variables and indexed variables are two basic kinds
> of slots in this model. Specializations in this model are also
> possible: for example, Sets and Dictionaries use special kinds of
> slots for their elements or associations, so Squot does not capture
> their internals (the hashtable array and tally) instead. It takes into
> account that associations can be shared among multiple Dictionaries
> (like Bindings).
>
> Squot does not reuse the existing repository and versioning
> infrastructure of Monticello. I did this to be free of the constraints
> of the contemporary Monticello, and to not break it while developing
> Squot and its companion packages. Some concepts are also different: in
> Monticello you have packages and versions of packages, in Squot you
> have versions, each of which can contain one or more packages (or
> something else). That is, a Squot version could be viewed as a version
> of a configuration map. This maps better to the concepts of popular
> file-based version control systems like Git, Mercurial etc. Moreover,
> a Squot repository is not just a store for versions; it also knows
> branches (the managers of history).
>
> The Git Browser belongs to a package called Squit. Squit connects
> Squot and Git, like its name does. It provides Squot repositories and
> versions, that are in fact adapters for Git repositories and Git
> commits. To maintain the actual Git objects, Squit uses a package
> called FileSystem-Git, which allows developers to browse files in Git
> commits like regular file trees using the FileSystem API. (This was
> the first effort to support a Git implementation in Pharo, until Pharo
> abandoned it in favor of the libgit2 binding. If I remember the author
> timestamps correctly, FileSystem-Git was primarily developed by Max
> Leske, Camillo Bruni, Damien Cassou. The core Git classes are in turn
> based on the ones written by Tony Garnock-Jones, which can still be
> found on squeaksource. I converted FileSystem-Git back from Pharo to
> Squeak and made it work for its purpose (it is not a feature-complete
> Git, but you can read and write local and remote repositories
> correctly). Much of the work has already been done by the others, but
> some things needed to be fixed or added to make it usable. To that
> end, I revived the Squeak implementation of FileSystem and extended it
> where necessary. As of now it is not fully compatible with the API in
> Pharo (class names and some method names are different), but that is
> still not impossible to change.)
>
> More details can be found in our technical report [2].
>
> ==Closing words==
>
> I hope you will have fun using Git with Squeak.
>
> Thanks to all those who have contributed to this project (and its
> dependencies/ingredients) so far!
> https://github.com/hpi-swa/Squot/graphs/contributors
>
> Kind regards,
> Jakob
>
> [1] https://www.hpi.uni-potsdam.de/swa/publications/
>
> [2] https://www.hpi.uni-potsdam.de/swa/publications/media/ReschkeTaeumelPapeNiephausHirschfeld_TowardsVersionControlInObjectBasedSystems_HPI121.pdf
>
>

>
> Thank you very much also from my side, Jakob!
>
>
> Did you already copy this documentation somewhere into the swiki? If not, I would propose doing so.
>
>
> Best,
>
> Christoph
>
> ________________________________
> Von: Squeak-dev <[hidden email]> im Auftrag von David T. Lewis <[hidden email]>
> Gesendet: Dienstag, 2. Juni 2020 03:22:40
> An: The general-purpose Squeak developers list
> Betreff: Re: [squeak-dev] [ANN] Git Browser and Squot
>
> Jakob,
>
> Thank you for this comprehensive overview, it is *really* helpful.
> There is lot here to read and absorb, and some of it will take
> time for me to absorb.
>
> Thanks a lot for taking the time to explain, I really appreciate it.
>
> Dave
>
>
> On Sun, May 31, 2020 at 09:30:53PM +0200, Jakob Reschke wrote:
> > Hello together,
> >
> > Over the past two years you may have seen occasional mentions of
> > ???Squot??? and more likely of the ???Git Browser,??? which can be installed
> > from the menus of a plain Squeak image since 5.2. In case you ever
> > wondered: Squot and the Git Browser belong together. Let me finally
> > introduce you to them today, since this is the outcome of my master
> > thesis from back in 2017.
> >
> > ==What is Squot?==
> >
> > Squot is a version control system (VCS) for objects. Not only
> > packages, but arbitrary objects (such as Morphs, so you do not have to
> > put Smalltalk instructions how to recreate it in the VCS, but you can
> > rather track the object itself).
> >
> > It uses an external VCS as its backend to store versions. As of now,
> > only one implementation for Git exists (except for some test stub
> > repositories for the unit tests). It should be possible to add further
> > backend VCSs without changing the core of Squot.
> >
> > Squot is an abbreviation of ???Squeak object tracker,??? loosely based on
> > Git???s description in its manual page: ???git - the stupid content
> > tracker???.
> >
> > ==What is the Git Browser?==
> >
> > The Git Browser is a graphical user interface to operate on Squot
> > working copies with a Git repository as a backend. It allows you to
> > manage working copies, track packages (multiple per working copy,
> > which are all versioned together), browse branches, create new
> > commits, search for commits, synchronize with remote repositories, and
> > perform checkouts and merges. It can be used without a Git outside of
> > Squeak (it does not call external processes or shared libraries).
> >
> > The tool is primarily meant to maintain Squeak projects, which are
> > published on GitHub or similar platforms, but can be used with any Git
> > repository.
> >
> > ==Is it ready for use?==
> >
> > To work on Smalltalk projects: definitely yes. It has been used for at
> > least three years in student courses at HPI (Potsdam). I am also using
> > the Git Browser to work on Squot itself.
> >
> > To really track other objects than packages: possibly. It has some
> > bugs left, so I would not recommend to rely on the tracking of objects
> > other than packages for production use. For this reason, there is also
> > no GUI yet to add other objects to a working copy in the Git Browser.
> >
> > There are occasional quirks when pushing to GitHub (???Connection
> > closed???), so you might need to resort to the Git CLI sometimes.
> > Currently you must also merge non-Smalltalk files outside of Squeak.
> > To view the commit history as a graph, you can use external tools,
> > such as gitk, SourceTree, or whatever your preferred Git GUI is.
> >
> > ==How can I use it?==
> >
> > 1) Install the Git Browser by choosing Tools > Git Browser in your
> > Squeak image (since 5.2). This will also load several of dependencies
> > via Metacello.
> >
> > 2) Once you have the Git Browser open, it will prompt you whether you
> > want to create your first project now. If you would like to start a
> > fresh and empty working copy on an empty Git repository, select ???yes???
> > and provide a path for the Git repository. If you would rather like to
> > clone an existing Git repository, select ???no.???
> >
> > 3) The Git browser itself has five panes:
> >
> >     * On the top left, there is a list of projects. These are your
> > working copies. Each working copy links something in your Squeak image
> > to a Git repository outside of that image. Via the context menu of
> > this list, you can add, change, or delete working copies and
> > repositories. If this list is empty, you will want to start with the
> > context menu here.
> >
> >     * Below that, there is a list of branches in the currently
> > selected project (if any). Via the context menu, you can add, rename,
> > and remove branches, push to remote repositories, and switch branches.
> > One branch is highlighted with a green dot; this is the branch that is
> > currently checked-out. If you create new commits, they will go on this
> > branch.
> >
> >     * On the top right, there is the list of commits on the current
> > branch (if any). You can compare commits with each other or a commit
> > with the working copy, search for commits using the search bar below
> > the list (or via list filtering to only search the headlines), create
> > a new branch at a commit, reset the current branch to an older commit,
> > or check out or merge specific objects from a commit.
> >
> >     * In the bottom-left pane, the full log message of the selected
> > commit will be displayed. In addition to that, it will also show the
> > commit metadata, such as the author, committer, and the abbreviated
> > SHA-1 hash of the commit and its parents.
> >
> >     * Finally on the bottom right, there is a list of objects in the
> > selected commit. These are usually the packages known in a particular
> > commit. Via the context menu you can checkout or merge an object with
> > the working copy (read checkout like ???Load??? in Monticello and merge
> > like ???Merge???), browse either the object as it was at the time of the
> > commit, or what is currently loaded. You can also access the current
> > scripts (preamble, postscript etc.) of packages here.
> >
> > 4) Between the panes, there is a strip of buttons. They operate on the
> > current branch or the selected commit. At the far right, you can
> > configure how things go to Git: that is your name and e-mail address
> > for the commit metadata, the Smalltalk file format you want to produce
> > (FileTree/Cypress or Tonel), and whether Squot should suppress writing
> > its own configuration files (if you want to submit a pull request to a
> > repository that is not developed with Squot, for example). Via the
> > Feedback button, you can create an issue for Squot on GitHub directly
> > from the image. You can also file these here:
> > https://github.com/hpi-swa/Squot/issues
> >
> > 5) To add another package to a project, or remove a package from the
> > set of tracked packages, open the menu of a project and choose ???Change
> > tracked packages???. Below  that menu item, you can also change between
> > writing FileTree/Cypress or Tonel files (???Set package storage
> > format??????).
> >
> > 6) When you switch branches via the context menu of the branch list,
> > note that it will stow away all your unsaved changes and the other
> > branch is checked out clean. Your unsaved changes will be restored
> > when you switch back to the original branch. This behavior is
> > different from the Git CLI, and was conceived in an alternative Git
> > frontend called ???Gitless??? (which addresses some of Git???s awkwardness).
> >
> > 7) When you want to commit, you will find that there is no Git staging
> > area as far as the Git Browser and accompanying tools are concerned.
> > Instead, you can choose which packages, classes, and methods you want
> > to include or exclude in the commit dialog, like in Monticello in
> > recent years.
> >
> > 8) When you checkout or merge something, you always first get a list
> > of changes, from which you can choose what to load and what to skip.
> >
> > 9) To update the Git Browser when a new version was published, open
> > the window dropdown menu (third button from the right in the title
> > bar) and choose ???self-update??????. It pulls the latest version from the
> > master branch on GitHub. Ongoing development happens on the ???develop???
> > branch, not on master. You can also pull that version by choosing ???set
> > self-update branch?????? in the same menu, and then invoke the update.
> >
> > ==How does it work internally? Why is it not a Monticello extension?==
> >
> > Squot knows of three presentations of an object: 1) the ???live??? object
> > that is ???the real thing,??? which you can normally handle in Squeak, 2)
> > one or more  ???shadows??? of the object, which is a captured
> > representation of one version of the ???live??? object (a snapshot); it is
> > used to reason about different versions of the same object without
> > loading them, 3) a ???stored??? object, for example serialized into a
> > file. Objects are tracked in graphs of objects. You add an object
> > graph by adding a ???root object??? to the working copy, from which all
> > the other relevant objects can be reached. For example, you could add
> > a complex Morph, which would include all of its submorphs. The tracked
> > objects can to a certain degree control how they are captured; for
> > example, the ???root Morph??? will direct Squot to ignore its #owner
> > instance variable (because otherwise you would always track the whole
> > World). To be able to identify an object in different versions of a
> > graph, each object gets a unique name assigned when it is first
> > captured. Objects may opt out of getting a name if they are true value
> > objects for which identity does not matter when a shadow object is
> > rematerialized to a live object during a load/checkout. Identity does
> > not matter when it makes no difference whether two objects refer to
> > the same instance of the value, or a different, equal instance (like
> > Number, Point), or if there is only one instance by definition (like
> > UndefinedObject, Boolean, Symbol).
> >
> > Packages, which are the only kind of object you can currently track in
> > Monticello (besides configurations of packages), are just a special
> > case of all of this. Read the rest of this paragraph as an example of
> > what I wrote in the previous paragraph. The live package is
> > represented by its PackageInfo. It subsumes the loaded behaviors and
> > methods that are attributed to the package. A shadow package basically
> > wraps an MCSnapshot of the package (Squot reuses MCSnapshots and
> > MCDefinitions under the hood to represent shadows of packages,
> > behaviors, methods). A stored package is a tree of files in
> > FileTree/Cypress or Tonel format, for example. In theory, it could
> > also be any other file representation, such as a plain file-out, or a
> > SmartRefStream that contains the MCSnapshot. A PackageInfo is the
> > ???root object??? of a package object graph in the working copy.
> > PackageInfo will tell Squot that it does not want a unique name
> > assigned, because a PackageInfo is already identified by its
> > #packageName. To create the shadow package, PackageInfo also overrides
> > the capturing process for Squot to find the classes, and methods
> > (which are not directly referenced) and turn them into an MCSnapshot
> > with Monticello. When a package is loaded in Squot, it will also use
> > Monticello to actually install the changes.
> >
> > As a general way to capture any kind of object, Squot has shadow
> > classes to capture objects as collections of slots that refer to other
> > objects. Instance variables and indexed variables are two basic kinds
> > of slots in this model. Specializations in this model are also
> > possible: for example, Sets and Dictionaries use special kinds of
> > slots for their elements or associations, so Squot does not capture
> > their internals (the hashtable array and tally) instead. It takes into
> > account that associations can be shared among multiple Dictionaries
> > (like Bindings).
> >
> > Squot does not reuse the existing repository and versioning
> > infrastructure of Monticello. I did this to be free of the constraints
> > of the contemporary Monticello, and to not break it while developing
> > Squot and its companion packages. Some concepts are also different: in
> > Monticello you have packages and versions of packages, in Squot you
> > have versions, each of which can contain one or more packages (or
> > something else). That is, a Squot version could be viewed as a version
> > of a configuration map. This maps better to the concepts of popular
> > file-based version control systems like Git, Mercurial etc. Moreover,
> > a Squot repository is not just a store for versions; it also knows
> > branches (the managers of history).
> >
> > The Git Browser belongs to a package called Squit. Squit connects
> > Squot and Git, like its name does. It provides Squot repositories and
> > versions, that are in fact adapters for Git repositories and Git
> > commits. To maintain the actual Git objects, Squit uses a package
> > called FileSystem-Git, which allows developers to browse files in Git
> > commits like regular file trees using the FileSystem API. (This was
> > the first effort to support a Git implementation in Pharo, until Pharo
> > abandoned it in favor of the libgit2 binding. If I remember the author
> > timestamps correctly, FileSystem-Git was primarily developed by Max
> > Leske, Camillo Bruni, Damien Cassou. The core Git classes are in turn
> > based on the ones written by Tony Garnock-Jones, which can still be
> > found on squeaksource. I converted FileSystem-Git back from Pharo to
> > Squeak and made it work for its purpose (it is not a feature-complete
> > Git, but you can read and write local and remote repositories
> > correctly). Much of the work has already been done by the others, but
> > some things needed to be fixed or added to make it usable. To that
> > end, I revived the Squeak implementation of FileSystem and extended it
> > where necessary. As of now it is not fully compatible with the API in
> > Pharo (class names and some method names are different), but that is
> > still not impossible to change.)
> >
> > More details can be found in our technical report [2].
> >
> > ==Closing words==
> >
> > I hope you will have fun using Git with Squeak.
> >
> > Thanks to all those who have contributed to this project (and its
> > dependencies/ingredients) so far!
> > https://github.com/hpi-swa/Squot/graphs/contributors
> >
> > Kind regards,
> > Jakob
> >
> > [1] https://www.hpi.uni-potsdam.de/swa/publications/
> >
> > [2] https://www.hpi.uni-potsdam.de/swa/publications/media/ReschkeTaeumelPapeNiephausHirschfeld_TowardsVersionControlInObjectBasedSystems_HPI121.pdf
> >
> >
>
>